Will a Data Warehouse Solve My Data Problems?

A two-part series to help you identify and solve your data needs

Have you seen the funny YouTube video, It’s Not About the Nail? It starts with a close-up of a woman talking about this relentless pressure she feels and how she’s afraid it’s never going to stop. Then the camera angle changes to reveal she has a big nail coming out of her forehead. Her boyfriend patiently listens and finally states the obvious solution. Remove the nail.

Too many businesses approach data problems the same way as the couple in the video. They have a data problem (the nail in their head), but they go about identifying symptoms (headaches, pressure) instead of just recognizing the problem and solving it. So they go down a rabbit hole chasing solutions that don’t address their obvious problems.

This week a customer called me in the throws of building a data warehouse. He was feeling the pain of the implementation process and wanted to know if that was normal. Like all of us, he just wants the data he wants at his fingertips to make faster and smarter decisions for his business. And, he wants the data to be accurate!

In my 20-plus years of experience, I have never seen anyone fly through a data warehouse project like it was as easy as making a PB&J. But I wondered why it’s so difficult. To find the answer, I posed the question to some of the smartest people I know. My team at defi SOLUTIONS. (Thank you Brandon Burns, Jose Salinas and Rob Dufalo for your input!)
So here’s what I learned. Data warehouses are difficult to build because they’re often the wrong solution. It just seems like the thing to do. Say you need some quick, accurate, operational metrics and your CIO says the solution is to build a data warehouse, because that’s what he/she did at another company. Or your colleagues at a conference tell you about their data warehouse projects, so you assume you should be building one too. This scenario can happen with any technology need.

The pain of data warehouses could also be the result of how it’s designed. A data warehouse can be over engineered out of fear or vague expectations. Businesses have to have a clear definition of success before they build a data warehouse.
Speaking of expectations, this article isn’t going to tell you how to know if you need a data warehouse or how to successfully implement one. That would be jumping the gun. I want to focus on the critical first step to creating any technology solution – step back and assess what you’re really trying to solve. You’ll save yourself a lot of pain if you do.

Now let’s dig in.

The importance of data is not a new concept. Neither is pulling data together and placing it at our fingertips. It’s old school. Really old. Ancient, in fact. Over 2,000 years ago, the Library of Alexandria was said to contain all the important data of the time under one roof. The very first data warehouse!

Data gives us the ability to learn from the past and the present so we can have a better future. A great example is IBM’s supercomputer named Watson. It gets smarter and smarter as it learns from the past.

I think we can all agree data is important. And it’s true that building or buying a data warehouse may even help you improve your use of data. However, before embarking on such a large project, first assess your pain points. Here are six examples of typical pain points. Please don’t limit yourself to this list. Make it your own.

1. Operational metrics
• Are you missing operational metrics? Why aren’t you getting them? They’re not available? No one has time to get them? They don’t exist anywhere? For instance, the other day I asked for a piece of data. My team said we didn’t have it. I might infer that I need a data warehouse to get it. But in fact, we just had never spent the time to calculate the numbers.
• Are you getting all the metrics you need, but you’re constantly dealing with inaccuracy?

2. Scalability
• Do you have tons of data and maybe you can find it all, but it’s just not in a place or in a way that will scale as you grow? For instance, I used to keep defi sales data in five spreadsheets. I knew exactly where to go to get it, but I couldn’t easily pull it together. There was no common format and if I sent it to anyone else, they wouldn’t be able to make heads or tails of it.

3. Timing of getting data
• Are you on old legacy systems that don’t allow for near real-time data access? Are you always trailing behind? Maybe you have the data and it seems accurate and built in a way that scales, but it’s only available at a certain time during the day, week or month?

4. One system of record
• Do you struggle with getting data across departments to match? Does every team have its own version of the truth?

5. Reporting vs. analytics
• Do you have data housed in a format that’s conducive to reporting, but makes data exploration a challenge? Do you have an environment that can efficiently mine, explore, and quickly present/visualize (if needed) data elements, which may not be currently used for reporting, etc?

6. Self-service business intelligence
• Don’t want everyone on your team to have to put in requests to get analysis and data? Need the data exposed so various users can visualize and construct their own report? Our team faced a similar challenge, but our data warehouse wasn’t built for it.

Seriously, write down your pain points. Don’t assume a data warehouse is the answer or the answer right now. What are you trying to solve or fix? Ask your team to compile a list of the true problems you’re facing and prioritize them. You want it all and you want it all now, but you should figure out what is most important and why.

defi just went through a similar analysis to build out our configurable reporting platform. For us #2 and #6 were key factors. We had to spend a lot of time really honing in on what we were trying to solve. We didn’t start with the assumption that the configurable reporting platform would need a data warehouse. We started with pain points we needed to solve.

Once you have your needs identified, hold on to them and tune in next time as we dig into potential solutions (including data warehouses) and go through an exercise of scoring each pain point against the solution. We’ll also layout a smart implementation approach to minimize your pain, assuming your analysis says the data warehouse is the right path for you.

Stephanie Alsbrooks is the CEO, founder and chief data evangelist of defi SOLUTIONS, the most configurable loan origination platform on the market. Email Stephanie at salsbrooks@defisolutions.com.

Will a Data Warehouse Solve my Data Problems?

Part two – The answer revealed

Have you ever completed a word search puzzle by first attempting to find all of the words, and then comparing your list of found words with the one that was provided? Of course you haven’t.

The same simple approach applies to solving data problems. You have to truly know what the issues are before you go about fixing them. (Hint: Building a data warehouse is not always the right solution.) Do your homework. Understand your problems before looking for the answers. This is what defi CEO Stephanie Alsbrooks challenged you with in part one of this article in the January/February 2017 Non-Prime Times issue.

We often hear about how a technology solution is doing wonders to fix someone else’s problems. However, their problems may not be your problems. Even if they are, you should ask yourself, “What’s the ROI? Is the existing cost of my problem greater than the implementation, maintenance, and operational overhead costs of the solution I’m considering?” If not, then you may be solving for the wrong problem or attempting to solve the problem with the wrong solution.

Hopefully, you took Stephanie’s advice and did your homework. What should you do next? Well, it involves prioritizing the problems you identified, evaluating potential solutions, and determining which solutions, if any, to implement.

I recommend you start by changing the way you see your problems. When you’re looking for technology solutions to fix business problems, it’s really helpful to think of your problems as the absence of a capability.

Let’s say one of the problems you’ve identified is fragmented, siloed and disparate data being utilized by different groups across your organization. Marketing’s reporting tool may have Metric X coming from Database A, while Customer Care’s reporting tool may be pulling the same metric from Database B. This can cause a clear problem when various business units are operating from business metrics with potentially different values. These data discrepancies are typically caused by differences in load or transformation logic between the two databases or reporting tools. To fix the problem, you may, for example, simply force Marketing to use the same reporting tool as Customer Care. Fixed, right? Sure, unless Finance begins using that same metric in a third and different way next week. But what if they do?

In this example, you ultimately want to address your lack of a single source of truth, a practice of defining a single, trusted, universally agreed upon version of all key data points that are driving business insights. By viewing the problem as a potential capability absence or gap, you focus on defining single sources of truth for each key data point across your enterprise, not why Marketing’s metric X is different from Customer Service’s metric X.

Now, let’s go back to the task of prioritizing your problems, or what we’re now calling capabilities. We’ll use something called a Capability Maturity Matrix. Sounds complicated, but it’s simply a way to visualize capability gaps relative to one another. A Capability Maturity Matrix will help you:
1) Visualize the current state of your capabilities, relative to one another.
2) Visualize the desired state of your capabilities, relative to one another.
3) Visualize the largest gaps between current and desired state.


In the matrix shown above (Graph 2), the biggest pain point is the single source of truth capability. Why? Because the capability doesn’t exist today and it has the farthest to go to reach the desired “Tomorrow” state. In this example, scalability is also a capability that doesn’t exist today.

Data Warehouse Solve my Data Problems 2

However, since there might be a relatively shallow volume trajectory, there’s not an immediate need for this capability to be fully mature.
Once your capabilities have been prioritized, solutions come into play. Solutions can be simple, complex, or everything in between. This is where a knowledgeable and trusted technology vendor might be of help. And with any technology solution, the decision to implement almost always boils down to Impact and Benefit versus Initiative and Effort — Cost versus Benefit or ROI.

How does a solution like a data warehouse address the Capability Maturity Matrix we examined? Does having a data warehouse address a single capability, several or none? Of the capabilities the solution addresses, what will the matrix look like after we’ve implemented it? Evaluating where the solution falls on the scale of Impact and Benefit versus Initiative and Effort becomes easier when you consider potential solutions this way. (Graph 3)

Data Warehouse Solve my Data Problems 3Whether you’re bolstering your operational reports in Excel or building out an enterprise data warehouse, understanding how to view your data problems, the root capabilities behind your problems and how the proposed solution is expected to mature that capability is paramount to success. You need to accurately quantify the benefit of maturing a given capability as well as the costs of implementing and maintaining the solution that furthers that capability.

A data warehouse might be the right solution for you. It might not be.

But what’s important is that you know how to measure and evaluate problems, capabilities, and solutions relative to one another, and ultimately quantify implementation success, instead of searching for a problem to the solution you already have.

In other words, you need to check the word list before you begin the word search.

Brandon Burns is the director of data and analysis for defi ANALYTICS, the new flexible analytics and reporting platform from defi SOLUTIONS. Founded by Stephanie Alsbrooks, the defi platform now includes defi ANALYTICS, defi SERVICING, defi DIGITAL, and, of course, the popular defi LOS. Email Stephanie at salsbrooks@defisolutions.com or Brandon at bburns@defisolutions.com.