Will a Data Warehouse Solve My Data Problems?

A two-part series to help you identify and solve your data needs

Have you seen the funny YouTube video, It’s Not About the Nail? It starts with a close-up of a woman talking about this relentless pressure she feels and how she’s afraid it’s never going to stop. Then the camera angle changes to reveal she has a big nail coming out of her forehead. Her boyfriend patiently listens and finally states the obvious solution. Remove the nail.

Too many businesses approach data problems the same way as the couple in the video. They have a data problem (the nail in their head), but they go about identifying symptoms (headaches, pressure) instead of just recognizing the problem and solving it. So they go down a rabbit hole chasing solutions that don’t address their obvious problems.

This week a customer called me in the throws of building a data warehouse. He was feeling the pain of the implementation process and wanted to know if that was normal. Like all of us, he just wants the data he wants at his fingertips to make faster and smarter decisions for his business. And, he wants the data to be accurate!

In my 20-plus years of experience, I have never seen anyone fly through a data warehouse project like it was as easy as making a PB&J. But I wondered why it’s so difficult. To find the answer, I posed the question to some of the smartest people I know. My team at defi SOLUTIONS. (Thank you Brandon Burns, Jose Salinas and Rob Dufalo for your input!)
So here’s what I learned. Data warehouses are difficult to build because they’re often the wrong solution. It just seems like the thing to do. Say you need some quick, accurate, operational metrics and your CIO says the solution is to build a data warehouse, because that’s what he/she did at another company. Or your colleagues at a conference tell you about their data warehouse projects, so you assume you should be building one too. This scenario can happen with any technology need.

The pain of data warehouses could also be the result of how it’s designed. A data warehouse can be over engineered out of fear or vague expectations. Businesses have to have a clear definition of success before they build a data warehouse.
Speaking of expectations, this article isn’t going to tell you how to know if you need a data warehouse or how to successfully implement one. That would be jumping the gun. I want to focus on the critical first step to creating any technology solution – step back and assess what you’re really trying to solve. You’ll save yourself a lot of pain if you do.

Now let’s dig in.

The importance of data is not a new concept. Neither is pulling data together and placing it at our fingertips. It’s old school. Really old. Ancient, in fact. Over 2,000 years ago, the Library of Alexandria was said to contain all the important data of the time under one roof. The very first data warehouse!

Data gives us the ability to learn from the past and the present so we can have a better future. A great example is IBM’s supercomputer named Watson. It gets smarter and smarter as it learns from the past.

I think we can all agree data is important. And it’s true that building or buying a data warehouse may even help you improve your use of data. However, before embarking on such a large project, first assess your pain points. Here are six examples of typical pain points. Please don’t limit yourself to this list. Make it your own.

1. Operational metrics
• Are you missing operational metrics? Why aren’t you getting them? They’re not available? No one has time to get them? They don’t exist anywhere? For instance, the other day I asked for a piece of data. My team said we didn’t have it. I might infer that I need a data warehouse to get it. But in fact, we just had never spent the time to calculate the numbers.
• Are you getting all the metrics you need, but you’re constantly dealing with inaccuracy?

2. Scalability
• Do you have tons of data and maybe you can find it all, but it’s just not in a place or in a way that will scale as you grow? For instance, I used to keep defi sales data in five spreadsheets. I knew exactly where to go to get it, but I couldn’t easily pull it together. There was no common format and if I sent it to anyone else, they wouldn’t be able to make heads or tails of it.

3. Timing of getting data
• Are you on old legacy systems that don’t allow for near real-time data access? Are you always trailing behind? Maybe you have the data and it seems accurate and built in a way that scales, but it’s only available at a certain time during the day, week or month?

4. One system of record
• Do you struggle with getting data across departments to match? Does every team have its own version of the truth?

5. Reporting vs. analytics
• Do you have data housed in a format that’s conducive to reporting, but makes data exploration a challenge? Do you have an environment that can efficiently mine, explore, and quickly present/visualize (if needed) data elements, which may not be currently used for reporting, etc?

6. Self-service business intelligence
• Don’t want everyone on your team to have to put in requests to get analysis and data? Need the data exposed so various users can visualize and construct their own report? Our team faced a similar challenge, but our data warehouse wasn’t built for it.

Seriously, write down your pain points. Don’t assume a data warehouse is the answer or the answer right now. What are you trying to solve or fix? Ask your team to compile a list of the true problems you’re facing and prioritize them. You want it all and you want it all now, but you should figure out what is most important and why.

defi just went through a similar analysis to build out our configurable reporting platform. For us #2 and #6 were key factors. We had to spend a lot of time really honing in on what we were trying to solve. We didn’t start with the assumption that the configurable reporting platform would need a data warehouse. We started with pain points we needed to solve.

Once you have your needs identified, hold on to them and tune in next time as we dig into potential solutions (including data warehouses) and go through an exercise of scoring each pain point against the solution. We’ll also layout a smart implementation approach to minimize your pain, assuming your analysis says the data warehouse is the right path for you.

Stephanie Alsbrooks is the CEO, founder and chief data evangelist of defi SOLUTIONS, the most configurable loan origination platform on the market. Email Stephanie at salsbrooks@defisolutions.com.