Big Data and the Problem of Finding Your Missing Keys

If you're spending all this energy looking through big data for business insights, be sure you have a good idea of what you're looking for, or you won't actually find it.

I’m not one for metaphors, but here’s a metaphor: Big data is like losing your keys.

When you lose your keys, where’s the first place you look? And if your keys aren’t there, you look somewhere else, right? Eventually, you keep looking and you just might find them.

At what point does searching through all this data become a futile exercise, like spending an hour or two looking for your keys in the couch on the off chance they might have fallen just beyond your reach?

In a lot of ways, big data is kind of like that, except your goal is to find new keys, to mine data in the hopes you might find something—something—that you may not have had without all that prior research.

(Perhaps if you get lucky you might find a key that opens a door that you never would’ve opened before.)

But at what point does searching through all this data become a futile exercise, like spending an hour or two looking for your keys in the couch on the off chance they might have fallen just beyond your reach? (Especially when, as it turns out, your keys have been sitting on the counter the whole time?)

Some thoughts on the matter:

Keep your files clean: What’s the point of having big data if you don’t have that data well-organized? Because, ultimately, a needle will be easier to find if the haystack is smaller and more manageable. And your keys will be harder to find if you don’t keep your living space reasonably clean. Make the job easier on yourself; don’t make your data so cumbersome to look through that you can’t figure out an easy way to utilize the data. If your data is messy, audit your stuff ahead of time, ensuring that the data is in a consistent format and easy to read.

Find a goal, and see if the data matches: Need a good example of a neat-and-tidy approach to using big data? Look no further than Yahoo’s recent decision to end its telecommuting policy. In researching her decision, Yahoo CEO Marissa Mayer checked analytics from VPN logs and realized that remote employees weren’t checking in very much—a realization that led to her decision. Now, in cases like these, you may want to cross-check your data with a little gut feeling. But if you’re considering a tough decision like this, this could help sell your business idea a little more. Without setting the right goal, you’re just looking through haystacks.

Stay focused on the goal: If you’re spending all your time figuring out the next place you’re going to search for your keys, you’re never going to get to looking, are you? It’s times like these when having someone in charge of the search—say, a chief data officer, as my colleague Katie Bascuas suggested a while back—might help prevent some of the headaches and infighting that can come with a complicated task … such as figuring out where your keys are.

Don’t miss obvious opportunities: Recently, the gaming company Electronic Arts (EA) has taken significant amounts of heat for the downtime issues it has faced with its just-released SimCity—issues that were so bad Amazon took the unusual step of stopping the sale of the product days after its release. The issues, rooted in the product’s usage of digital rights management (DRM) software, which requires that servers always be available, could have provided a good opportunity for EA to research the potential audience for the product, figure out if the DRM solution was feasible based on the size of the fan base, and adjust its offering accordingly. With the right data, EA could have had a better chance of accurately predicting how many servers it needed, or (more importantly) whether the system it was implementing was feasible.

Ultimately, if you’re not using data in a way that helps your association do its work, what’s the point? Radius Intelligence’s Darian Shirazi recently put it like this: “[Companies] don’t know what they’re looking for, because they think big data will solve the problem.”

Or to put it another way, you’re on a fruitless search for your keys when what you really need to do is call the locksmith and get a new pair.

So maybe I do like metaphors. Sue me.


Ernie Smith

By Ernie Smith

Ernie Smith is a former senior editor for Associations Now. MORE

Got an article tip for us? Contact us and let us know!