Published: 31 December 2015
As we prepare for our New Year’s Eve celebrations and we reflect on the highs and lows of 2015, we will ultimately have to look towards the future and start planning for what lies ahead in 2016. Some persons will make New Year’s resolutions, some will have detailed documented goals and others will just “wing it”. For us here at exportbi, we decided to highlight the five key business intelligence trends for 2016 and as we look forward to the New Year with great anticipation we hope that companies will take note of these trends and start making plans to maximise the ROI for their BI investment.
1. The rise of data lakes, reservoirs, oceans et al. and the death of the data warehouse?
It is no secret that data warehousing has not delivered on all the promises made decades ago and most companies are growing impatient with the high costs and small returns. Gartner suggested that seven or eight of every ten BI project fails, resulting in a 70-80 percent failure rate. With reasons spanning from complexity, lack of buy-in to data quality issues, they cannot go unnoticed and as more companies take on large scale BI DW implementations the chorus for change is deafening. There is growing need to not just store mountains of data but to quickly and seamlessly analyse and make decisions not on data stored 24 hours ago, but on real time living breathing data.
A major pain point for business users of Enterprise Data Warehouses (EDW) is the time it takes to get data they need to do analysis and reporting and the frustration after waiting days or weeks and then it is not accurate or simply no longer relevant for decision making. Some of these users have moved on to create their own pseudo data warehouses in an attempt to compensate for the shortcomings of the EDW. Data lakes are one of the clear answers to these problems and allows users to bring together unstructured and structured data without the long wait times and SLAs. While the size of a data lake can be overwhelming for the average user, data scientists and SQL savvy users are taking advantage of this all you can eat data buffet and everyone is happy and satisfied. But are they really? The data lake brings forth challenges of data governance and management, this issue can be compounded for companies that initially struggled to implement their BI project due to a lack of strategy and no clear roadmap.
We don’t see the data lake killing off the data warehouse as they can co-exist and serve two separate functions for business users. In some cases, the data lake operates as a staging area for the data warehouse but in an industry where catch phrases are the order of the day, coupled with the bitter taste left from data warehousing, the reality is that companies will tout that they are replacing their data warehouse with a data lake. We would argue however that a data lake is merely an evolution of the data warehouse and is needed to meet the growing complexity of data and the business.
2. Predictive analytics is a must have and no longer a nice to have
For companies that are still stuck reporting and analysing what happened yesterday and then seeking to find out why it happened, you will be left behind in 2016. On the analytics value escalator that shows the progression of analytics, we get the most value when we employ predictive (What will happen?) and prescriptive analytics (How can we make it happen?). The level of difficulty increases as we begin to optimise and transform information into competitive ammunition that can edge out the competition and drive the strategic objectives of an organisation.
If the data lake is the response to shortfalls of the enterprise data warehouse then data science focuses on the high value, high difficulty spectrum that business intelligence failed to deliver. Now more than ever will the Data Scientist be crucial to an organisation realising their big data potential. Leveraging mathematical skill-sets with statistics, machine learning and knowledge of the business and its data will ensure the success of any predictive or prescriptive analytics undertaking.
3. Real time data visualisation and analytics
We cannot stress enough that in a dynamic global marketplace, it is just not enough to know what happened yesterday, we need to conduct analysis on real time data as well as provide visualisation and storytelling. Visual analytics will be a core focus for organisations and will serve as the universal language for data. It will not be limited to just the experts and data visualisation professionals as non-experts will have the ability to create, manipulate and interpret their own visualisations. This will allow users to not only feel empowered but derive quicker insights, and collaborate with substantial gains.
4. Cloud for everything
It may seem like just the hip thing to say so that you don’t feel left out but by now you should realise that “cloud” is the future. Whether you decide to keep your BI environment on-premises, in the cloud or adopt a hybrid model, there will be some components of your architecture that will be better suited for the cloud. With advantages such as scalability, accessibility, ease of use and deployment speed it is no wonder why organisations are flocking to the cloud and boasting cost savings and improved efficiencies.
5. Empowered users who want more than self-service BI
When the business users complained bitterly that IT was not delivering on their BI promises, self-service BI became the answer to the loud screams and tantrums thrown by users who did not want to log a ticket and wait in a queue to get a report to conduct analysis on critical data. While there were benefits and a boost in the efficiency of business analysts, the question begs to be asked, what really changed? If you are still waiting weeks and months to incorporate new data sets into your EDW and you are struggling with the credibility of the data as well as a lack of flexibility with the traditional data warehousing approaches then chances are you have outgrown the initial euphoria of self-service BI and you now need a new fix (hopefully permanent this time). Empowered users want control and we agree that they should have it, with some degree of auditing and control of course. The data lake may be the response for most of these users but as mentioned before data governance and management must be a core fundamental strategy.
After the parties end, our stomachs filled, our hearts warmed and our spirits rejuvenated we will return to our organisations with a renewed energy and outlook for the year that lies ahead. If you have not already identified these key Business Intelligence trends as part of your roadmap then you should revisit and see how these trends will impact your organisation and make adjustments where necessary. We hope you will realise all your big data dreams in 2016!
About the author: Raquel Seville [@quelzseville] is a Business Intelligence Professional, SAP Mentor, BI Evangelist, Founder: exportBI | Co-Founder: eatoutjamaica. To find out more, please visit her about me page.