International gurus view of BI 20169 min read

Kennie Nybo PontoppidanKennie Nybo Pontoppidan
Senior Manager, Technology Applications at IMS Health

About the author: Kennie Nybo Pontoppidan is a senior manager in IMS Health and the former CEO of Effektor, a meta-driven DW/BI product dedicated to the Microsoft BI stack. He has worked in the dangerous field between developers, dbas, customers and project managers for many years and done his part of mistakes as a developer before that in his 15+ years in the it industry. He enjoys working with databases and really, really enjoys his daily dose of sql. Kennie has no humor.

As Rehfeld is a Microsoft Gold partner in BI and Data platform, I work primarily with technologies from or closely related to the Microsoft BI technology stack. I am the product manager of Effektor, a self-service data warehouse and BI platform based on this technology stack, and a principal consultant working with data warehouse requirements, capacity planning and data science projects. My answers are therefore biased towards technology and uses of technology for BI/Analytics projects, mainly within or around the Microsoft BI technology stack.

What was the most important advancement within BI/Analytics in 2015?
It seems to me that the massive work that Microsoft has put into their self-service BI suite PowerBI is finally paying off. The latest Gartner analysis places Microsoft as a leader in the BI and Analytics Magic Quadrant, finally beating other products like Qlik and Tableau. As part of Microsofts open source strategy, they have opened up for the public to add new visualization components. This is in my opinion a genius move from Microsoft. Web based data visualization techniques and frameworks develop so quickly these years that any visualization software is doomed to lack behind due to the time it takes to include and test new technology to a product.

Read more about PowerBi here: https://powerbi.microsoft.com/en-us/

Read more about (and contribute to) the open source PowerBi plugins here: https://github.com/Microsoft/PowerBI-visuals

Within the area of BI and analytics, Microsoft did two very interesting acquisitions in the spring of 2015: Datazen and RevolutionR. Datazen was on its way to become a replacement for Reporting Services, the reporting component in SQL Server, which has been in the product for more than 15 years. The biggest strength of Datazen is its strong focus on support for all mobile platforms (phones and tablets). RevolutionR is an enterprise-ready version of R, the world’s most popular programming language for statistical computing, data science and predictive analytics. The language has replaced statistical software packages like SPSS or SAS in university courses world-wide. The version from RevolutionR is faster and more scalable than the available open source version, and can therefor used in large data warehouses and/or Hadoop systems. We should expect to see both products integrated into the upcoming release of SQL Server 2016.

Read more about Datazen in SQL Server 2016 here: https://blogs.technet.microsoft.com/dataplatforminsider/2015/10/29/microsoft-business-intelligence-our-reporting-roadmap/

Read more about RevolutionR (branded under the new name SQL Server R Services) in SQL Server 2016 here: https://msdn.microsoft.com/en-us/library/mt604845.aspx

Finally, I have seen a rise in the interest and market of data warehouse automation software and frameworks. Demos and use-cases for the developer-centric product like Mist for the almost de-facto standard Biml (Busienss Intelligence Markup Language) are now shown at all the SQL Server conferences world-wide, where I attend as a speaker. I spend a good deal of my time talking about data warehouse requirements with clients, and the topic of automation comes up more and more. Instead of outsourcing development of data warehouse and ETL work, companies now want to do smart in-sourcing (something we with our product Effektor also calls self-service data warehousing), using automation tools like Mist, Effektor, TimeExtender and WhereScape.

Read more about Biml her: https://en.wikipedia.org/wiki/Business_Intelligence_Markup_Language

Read more about TimeExtender here: http://www.timextender.com/

Read more about TimeExtender here:https://www.wherescape.com/

Read more about Effektor here: http://www.effektor.bi/Kennie4

 

In your opinion, what will be the most important advancement within BI/Analytics in 2016?
With self-service BI comes great power to the business users. They are (finally) able to get BI at the speed they need, without waiting for ages on the IT department and/or BI consultants. But with great power comes great responsibility, and this means that companies and organisations using self-service BI tools must deal with data governance. They need to consider who can see data, whether data is correct and up to date, and also correlated correct. I hope to see more organisations using self-service BI as a frontrunner for the corporate data warehouse and BI solution in the sense that self-service BI can be used to rapidly prototype BI needs. Some of these needs will be one-off analysis, and some of them deserves to be incorporated in the corporate data warehouse, where “single version of the truth” resides.

Read more about data governance challenges here: http://www.gartner.com/smarterwithgartner/managing-the-data-chaos-of-self-service-analytics/

It is probably not a trade secret that the demand of BI/analytics skilled people is much higher than what is available in the market, at least this is what we experience in Denmark. And with new needs within data mining, predictive analytics and machine learning, I don’t expect this situation to change must in the years to come. What I find interesting is the growing curriculums on these topics offered as MOOCs (Massive Open Online Courses) by leading universities world-wide. Some of these courses attend students in the number of hundred-thousands. I hope that we will see more IT and business professionals get inspired by MOOC courses within data science and take up a career within our profession. We need them. Now.

Read more about Coursera here: https://www.coursera.org/

Read more about edX here: https://www.edx.org/

Stove-pipe analytics applications (click-click-click, now you have BI on your CRM/ERP/…) has been around for some years now. These products deliver data warehousing, OLAP and reporting on the data from the given application, but are not always the right choice, if an organisation wishes to enrich the auto generated data warehouse with other data domains and/or data sources. I expect the vendor(s) of these products who are able to open up their architecture to integration into bigger data warehouse solutions to win on the longer term. But we might not see this in 2016.

Read more about one example of stove-pipe solutions here (there are many others): http://www.bi4dynamics.com/

People have been talking about big data for a while now. Do you see big data projects becoming mainstream in 2016? Why/why not?
I still see many, many companies and organisations with a very low maturity level of strategic data/BI usage. If a company only do BI and/or analytics on their core ERP data, they are maybe better off exploring other data domains within the enterprise before they jump right into the wild west of Big Data projects.

Read more about Big data and BI maturity here: https://hbr.org/2013/12/you-may-not-need-big-data-after-all

Having said this, I do see more interest in machine learning/data mining/data science related aspects of more traditional data warehouse and BI projects. So, in a way I do see Big Data projects becoming more popular in the coming years. A technology platform which I expect to help us prototyping more within this area is the cloud based machine learning portal and platform AzureML from Microsoft. Using AzureML a (possibly aspiring) data scientist can set up data experiments without having to invest time and money in establishing a Big Data infrastructure. And AzureML comes with a very nice UX as well. But we must forget that data science is difficult and requires skills within computer science, mathematics and statistics. And no UI can replace that, only human intellectual skills.

Read more about AzureML here: https://studio.azureml.net/ 

Internet-Of-Things (IOT) has been a buzzword for some time now. But I expect the availability of low cost sensors and mobile services to gather and integrate data will make IOT more real than ever in 2016. When companies and organisations start to integrate IOT data sources with their data warehouse data, we will see interesting new business models arise, and we will get interesting new insight in our business domains. I colleague of mine has IOTed his house with Philips HUE light bulbs combined with a lot of wireless sensors, including a sensored doorbell and mashed this up into a dashboard for his house. He can also control lights and other things from hos mobile.

Get sensors from wirelesstag here: http://store.wirelesstag.net/collections/all

And a new doorbell here: www.ring.com
See a recipe here: https://ifttt.com/recipes

And do dashboarding here: http://numerousapp.com

What new buzzword within BI/Analytics will we see in 2016?
I can see three things happen in 2016 (or maybe a few years from now).

First, Tableau Software is acquired by one of the big players. Microsoft already bought Datazen recently and their product PowerBI is very successful. IBM clings to its Cognos Suite. So I would bet my money on Oracle or Informatica. Oracle because their BI suite OBIEE in my opinion really, really needs to be updated to this century. Informatica because they only have an ETL tool, and could get a much more full product suite with great visualizations. Tableau stock has dropped 50% in February, so maybe now is the time for an acquisition.

Read more about the 50% drop in Tableau stock price overnight here: http://uk.businessinsider.com/tableau-software-stock-plummets-50-2016-2?r=US&IR=T

Then, the pendulum might swing back from report generators to coding dashboards and reports using state-of-the-art web programming frameworks. For many companies and organisations, a few super-tight dashboards with stunning details reports might do the trick for the corporate BI. OLAP with Excel is still a preferred tool for ad-hoc analysis, and self-service BI could be the tool for exploring new areas of data usage. Why code using web programming frameworks? Well, first of all the newest and coolest frameworks can be utilized. Secondly, you save a lot on licenses, especially for tools like Tableau.

See examples (there a many, many of them out there) of javascript frameworks for data vizualisations her: http://d3js.org/

or here: http://www.highcharts.com/

or here: http://nvd3.org/

Finally, I would hope to see process mining get more popular. Process mining is a type of machine learning, where algorithms can infer possible business processes from data as well as replaying data on process models to measure conformance. I have been working with process data warehousing for the last couple of years (think of this as reporting and KPI’s once you know the business processes), and would love to see process mining being used as part of process data warehouse projects.

Read more about process mining here: http://www.processmining.org/

Best Regards
Kennie

Twitter: @KennieNP

LinkedIn: http://www.linkedin.com/in/kennie

Blog: http://pontop.dk

About Arne Rosness