Understand qualitative vs. quantitative research

If you want to be a bad product manager, rely solely on quantitative research. Business is about numbers, after all, and there’s a reason you had to learn statistics in school. If you can’t prove something to a level of statistical significance, it must not be reliable. You would never make a decision about a product that’s used by millions of people by just getting input from a few dozen. What people say is not nearly as important as how many people say it.

If you want to be a good product manager, utilize both quantitative and qualitative research for decision making. Numbers are good, though on their own they can not tell the whole story.

Quantitative research is especially useful in product development as a way of confirming findings through qualitative research. For example, you may conduct customer visits or ethnographic research and uncover some unmet needs. You may want to conduct a survey of a wider group of customers to confirm your findings and get more input on specific aspects, such as how important this need is and the cost to the customer of not being able to currently solve this problem.

Many inexperienced product managers have little if any background in any sort of market research techniques. They may rely on the standard techniques — usually surveys and focus groups — whenever research needs to be conducted. Product managers may not even realize that there are other methods available or may not understand why you would use one method over another.

To generalize, qualitative research is usually better for exploring, understanding, and uncovering, while quantitative research is generally better for confirming and clarifying.

If you are trying to discover unmet needs that consumers can not articulate, there is no way that a survey asking respondents to rank items from 1-10 will offer any insight. However, that method would make sense if you have a good idea about priorities for features and you want to get confirmation from a large sample set.

Similarly, if you want to determine whether a market exists for a new product idea and whether consumers would be willing to pay for the product, a single focus group with 6 participants will not provide the information you need. Instead, that would be a useful approach to take if you are trying to understand underlying issues, preferences, and factors that could be helpful in defining a potential solution.

Product managers do not necessarily need to be experts in conducting qualitative and quantitative research, though some experience is certainly useful. More important, however, is that product managers understand the fundamentals of both types of research, their applications and their limitations.

There are many great resources on qualitative research methods; here are two which are recommended:

• Observing the User Experience: A Practitioner’s Guide to User Research by Mike Kuniavsky. This is a fantastic guide to a huge variety of different user research techniques, especially useful for web, software, and technology products. It is sufficiently detailed to be used as a learning guide, while written succinctly enough to be useful for quick reference. (In the interest of fairness, I will note that this book is published by a division of Morgan Kaufmann, which is an imprint of my former employer, Elsevier. However, I have no specific organizational or financial connection to this book. I would gladly recommend it even if it was the product of another publisher or if I was not formerly employed by Elsevier.)
• Qualitative Market Research: A Comprehensive Guide by Hy Mariampolski. This provides an excellent overview of different methods along with guidance for those actually conducting the research.

Good product managers will use these and other resources to understand more about the different types of market and user research so that they can utilize the right techniques for the right situation.