The success of any Business Intelligence (BI) Program aiming to leverage vast amounts of data depends, amongst other considerations, on the quality and source of the data, and how you go about turning that data into reliable information for your business.
In our experience there are a few key decisions an organisation can make about data at the outset of a BI Program that will contribute to its success.
Find your source of truth. When organisations start their BI investment they believe they know where the source data is, its attributes and its condition. Months into a program there are many revelations. Often there are many sources of what appears to be the same data and the organisation needs to pause to identify which data set is the reliable source of truth. We find effective BI Programs begin with an organisation coming to a common understanding of the source of truth for their BI data sets. More often than not, there is only one reliable source of truth for a specific area of interest.
Help people understand the data. We often think that everyone sees data sets the same way – data often means different things to different people. We’ve seen time and time again that establishing a common understanding of the data (a data dictionary) early in the process removes a lot of error, confusion and rework later in the program.
A data dictionary will contain agreed-upon information about the data: what it is, what it means, relationships to other data sets, origin of the data, who owns it, how the data is used across the business, data formats etc. However, arriving at this kind of agreement will likely involve much discussion and debate amongst stakeholders, some looking at the data from a business value perspective and others from a more technical background. It may be daunting, yet it is such a worthwhile step as it will reduce confusion and give everyone a touchstone to come back to time and again during the BI Program. It can give the team less room for error, a sharper focus and avoid the need for rework later in the program.
Cleanse your data: Organisations often think that data sources are reliable and clean. Yet, a closer look often shows that the source data needs a lot of work to clean it up before it can be relied upon. Understanding data at a granular level early will save a lot of time and money, as will setting up formal data governance at the start of a BI Program.
Classify your data. Similarly, and quite critically, data security should be thought through early in the process. It’s important to understand the security profile for all classes of data that will reside in the BI tool and how that will impact access. The dynamics of how the data warehouse is used and who can access which records opens up a complicated challenge about whether data security standards might be compromised. Unless specific roles and controls are implemented, most BI tools allow the same access to all users so all data classes are exposed. Setting up artificial walls within the BI tool is complex, expensive and diminishes tool performance. Always engage data owners and data security experts early in the BI Program.
Decide how to move your data. Whether you select ETL (extract, transform, load) or ELT (extract, load, transform) will depend on factors such as your target BI platform, your Program team’s ability to perform custom transformations, your database management system and the kind of initiatives you are focussed on.
These aren’t the only considerations, but if you have concerns about any of the above, then you may want to take a little more time to assess where you are before the investment in BI begins.