Trust Your Data So You Can Trust Your BI

While discussions about the high value of ‘business intelligence‘ (BI) are all the rage these days, reliance on that ‘intelligence’ is only justified if the data upon which it’s based is accurate.

 
BI applications typically draw data from several (dozens? hundreds? thousands?) sources, and all that information is rarely perfectly homogenized as a single database when it lands in its raw state in the corporate data lake. Sorting, cleaning, and organizing it into useful datasets takes time and effort. It’s only when that effort produces accurate, relevant, and reliable ‘intelligence’ that business leaders should embrace the truth of the conclusions that flow from it.

Organizations that are considering transitioning from Microsoft’s SSRS (SQL Server Reporting Services) report generator to Microsoft’s Power BI suite of services will be accessing a myriad of new data sources that will test the accuracy and veracity of their on-prem information. To gain the most value from the upgrade, they should evaluate their databases and infrastructure to ensure that their internal information is and remains accurate and relevant as it migrates from one program to the other.

Don’t Do This

Too many enterprises have suffered enormous losses by attempting to integrate different data management programs without assessing their compatibility or aligning their processes. As the leader of your enterprise, you can’t afford to make those kinds of mistakes with your organization’s assets. However, as is true in the rest of life, ‘you don’t have to live it to learn from it,’ so take heed of the lessons offered by these disasters.
 

NASA

On September 23, 1999, NASA’s Mars Climate Orbiter was destroyed as it entered the Red Planet’s orbit. The two teams responsible for the project’s development and deployment – one in Colorado and one in California – had both done exemplary work, so why did the mission go so wrong?

It was a tragic non-communication: one team used Imperial measurements in their calculations, while the other used metric measurements. Without discussing the disparity – not even knowing that it existed – the descent team programmed the Orbiter to descend to a 35-mile orbit, not the intended 87-93 mile orbit, which the other team had planned. The craft wasn’t designed to withstand the atmospheric pressure at that lower orbit level and disintegrated into nothingness on the way down.

Lesson learned: Coordinate all data to consistent standards before integrating it into BI systems.
 

Amazon

Well-trained computing technologists are highly sought after in today’s tech-heavy economy, and recruiting for them takes skill, initiative, and, in many cases, highly sophisticated software programs. In 2014, the megacorporation Amazon was recruiting for highly skilled workers and had developed a new Artificial Intelligence (AI)-powered program to screen incoming candidates. The company was using the project to address the HR question and also to develop further expertise in the automation sector.

The screening program scanned and scored resumes with up to five stars that represented the best match of candidates for available positions. A problem with it arose a year later when it appeared that many highly qualified candidates were not identified as such by the technology simply because they were women.

It turns out the technologists had trained their computer models on a decade’s worth of candidate resumes, the vast majority of which were submitted by men. The industry’s male dominance is a well-known fact, with up to 80% of all technical roles filled by men in the top four mega technology companies. Amazon doesn’t disclose the gender breakdown of its workforce, but this situation suggests that, at that time, it wasn’t weighted for equality.

Lesson learned: Build BI systems on data that is accurate, relevant, and timely. The heavy male presence in tech during that earlier decade is an accurate and truthful reflection of the times; it just isn’t appropriate anymore in a society working at reducing those unnecessary disparities.
 

Target

Some retailers go overboard when it comes to scoping out the realities of their consumer base. In 2002, retail giant Target decided it wanted to know if its customers were pregnant so it could skew its offers to accommodate the newly growing family.

The company hired a statistician to develop a ‘pregnancy score’ based on user data retrieved from the organization’s consumer-facing database. The analysis suggested that people who purchased a distinct combination of 25 specific products were likely to be pregnant. The retailer used that ‘premise’ (because it wasn’t factually accurate all the time) to redirect its maternity advertising toward those customers.

In addition to being simply ‘creepy,’ the marketing and couponing blitz was also highly offensive; it apparently flagged some women to the fact that they were pregnant before they knew it themselves. One mother-to-be was a teenager whose father learned his daughter was pregnant because she was being bombarded with pregnancy and maternity coupons.

Lesson learned: Use your data with caution. Just because you CAN use it a certain way doesn’t mean you SHOULD use it that way.

Datavail Prevents Data Errors

For whatever reason, data can be wrong, go wrong, and cause wrong. Not understanding where your data’s glitches lie before migrating it to the Power BI platform puts your enterprise at risk of creating data-based disasters like these.

The switch from SSRS to Power BI doesn’t have to be stressful or fraught with concern about a future faux pas. The data professionals at Datavail can help you reorganize your data sets to avoid inadvertently misdirecting your long-range BI strategy.

To learn more regarding migrating from SSRS to Power BI including benefits, challenges, and more, download our white paper, Look Ahead: Migrating from SSRS to Power BI.