Surveys, questionnaires, and polls generate data, but survey data and hard data aren’t the same thing. I often see them treated in the same light in the context of answering business questions or delivering actionable insight, and with equal zeal and qualification. But there are definite differences.

Understanding the difference between data collected from surveys vs. data generated from transactions or operations is crucial. It will help us find the relevant answers to our questions and also save us a lot of time and money in the process.

There’s a science and methodology to developing effective surveys. Design and data collection best practices provide guidance for making sure that we have the best chance to improve the number of responses and generate data sets that are clean, complete, and accurate. And yes, statistics matter. You can’t make a strong argument when your sample size is in single digits and your error margin is +/- 50%.

Perception vs. hard data

Putting aside all of these technical questions, one critical question still remains: What is the perception vs. actual data? If I want to know what my audience or users are thinking or how they feel about something, surveys provide a great starting point, especially in the absence of any other data sources. However, if I want to answer questions that require objective and accurate insight into measurable data points, I need to remove the element of subjectivity and perception.

This distinction is important because I’m not arguing that insight into people’s opinion or perception isn’t important or needed. On the contrary, in many cases perception is reality, and it may be more important than the actual solution or product itself. But relying on perceived value alone for questions that need hard data sets will most likely result in the wrong conclusion.

When survey results fall short

An example that I often use in my lectures may help to further articulate the difference.

Let’s say we want to lower the winter thermostat setting of our classrooms in order to reduce our utility expenses. However, in order to make better-informed decisions, we would need to find out whether it’s actually warm or cold in our classrooms. We want to use data to support our argument, not argument to support data. Although the premise of this question is simple, how we go about answering it may get tricky, especially as it relates to data.

If we want to know how our students feel about the temperature in the classroom, surveys may provide the right insight into their sentiment. The data may correlate with dimensions of our student profile (such as age), or may entirely contradict our initial assumptions.

However, if we’re interested in finding out the average temperature, the student sentiment becomes irrelevant and we simply can’t answer that question with surveys. Instead we would need sensors, for example, that can capture the temperature at different intervals.

When we combine the sentiment analysis with transactional or operational facts, coupled with external (supplemental) data such as weather, occupancy, and so on, we can paint a complete picture and will be better equipped to make better-informed decisions.

Bottom line

Now, I realize that business questions are never this simple and straightforward. However, I believe that many business decisions we face require similar thinking and a similar approach to data.

Our answers can be only as good as our assumptions. Identifying the right questions to ask and leveraging the right sources of data to answer them are key ingredients to driving growth and profitability.