Almost everyone is on the data bandwagon these days. Even those who might still be skeptical about the whole “big data” thing will likely admit that using data in decision making is critical, and we all have stories of organizations, leaders, and managers making bad decisions by relying only on anecdotal evidence, ego-based analysis, or my favorite: “because we have always done it that way.”

But data-based decision making is not easy. Elizabeth Engel and Peter Houstle wrote an outstanding white paper earlier this year that provides a great map of the different aspects you will need to keep in mind if you want to get serious about data (free download). One of the points they make (that I see organizations miss), is that using data is actually part of the scientific method, and the scientific method requires experiments. The data never give you the answer. The initial data sets will help you craft some hypotheses, and then you run experiments based on that, and the data from those experiments may help you draw a conclusion.

Organizations frequently don’t want to put in that kind of effort. They want to gather some data that gives them the answer and tells them what to do. From an actual organization:

We surveyed the attendees at our conference, and they overwhelmingly prefer email as their mode of communication, and only 5% said they prefer social media, therefore we are not going to invest in social media marketing for our future events.

Okay, first off, you’re surveying people who came to a conference that was marketed primarily via email. Of COURSE these are the ones that prefer email as a communication method. Second, even if those data were broader (lets say you surveyed from among all potential attendees), you would still want to run experiments to see of the survey data could lead to some actionable conclusions. Could running both an email campaign and a social media campaign generate new registrants? Perhaps among particular populations? The data suggest that your experiments should definitely include email, but they don’t exclude other things. Only when you experiment will you learn enough to make smarter decisions.

But not enough organizations like experiments. They fail too often. They don’t give us the right answers often enough. They take up a lot of time and energy and resources. These objections confuse me. It’s kind of like saying, “Hey, I don’t have the time to make smart decisions, but what I lack in quality I can make up for in volume.” Good luck with that.