Skip to main content

Building Better Products Using Quantitative & Qualitative Data

Disclosure: This was an internal post I wrote and shared with my colleagues back in 2015. While cleaning out my Confluence space, I came across it and thought that I'd share it with everyone.

There is a much more recent and polished post by my good friend Glenn Block on Mind the Product that you can read here: https://www.mindtheproduct.com/2018/01/need-quantitative-qualitative-data/

What Data Is/Not

  • Data cannot compensate for bad design or replace good design, but it can inform designers.
  • Data cannot compensate for listening to actual users, but it can inform the kind of user research we conduct.
  • Data cannot tell us what to build, but it can inform what we should investigate.

Example

If you can visualize a funnel report where visitors are leaving a buying process during various stages of the journey, quantitative data can give you information about where you may have a problem.

In the above example, the quantitative data (QT) informs you that 36 people are falling out of the cycle during the Payment Info page/stage. 

Quantitative and Qualitative Data

QT is useful for identifying that there is a problem, but it doesn't tell you why the problem exists or specifically what the problem is.
Qualitative data (QL) is useful for telling you why there is a problem.
So it can be helpful to think:
  • QT = What
  • QL = Why

A Process for Using Both QT and QL

  1. Identify the Biggest Problem you need to solve. You can use QT to inform you of what this may be. Look at the number of users affected, the largest opportunity loss, the highest count of critical failure points, etc. An example of a good QT tool is a funnel report.
  2. Once you've identified the problem area, use QL to understand what is causing the problem, such as observational research.
  3. Based on the QL, propose a solution hypothesis. For example: "We see 36% of the people falling out of the funnel. We observed that a lot people leave the site when they get to the promo code box and search online for promo codes, and then end up finding the same product offered on other sites and go purchase it there. So one proposed solution is to remove the promo code box."  Of course, you have to understand the reason why things were done in the first place before making cavalier proposals.
  4. Learn & Iterate. A/B test the solution. Repeat and tweak until satisfied.
  5. Return to step 1.

Another Process for Using Both QT and QL

  1. Review QL data/observations to formulate a problem hypothesis. i.e. "5 customers don't like this feature."
  2. Use QT to study the relevant metrics related to the hypothesis.
  3. Follow steps 3-5 above.
Why invert 1 and 2? Because you might want to use QT to support or refute QL findings. Sometimes a squeaky wheel is not representative of the real problem.

Comments

Post a Comment

Popular posts from this blog

Social CRM: Putting Customers First

Last night I attended Social CRM: Putting Customers First , hosted by the San Francisco chapter of the Social Media Club .  It was a great event, and I'll briefly recap some of the discussion and my observations. The panelists were first asked why Social CRM (SCRM) was relevant.  Their answers were as follows: Vendors need to know who their customers are out there on the social web. Vendors need to be able to identify customer pain points from conversations on the social web.  Vendors need to know what to do with these conversations on the social web.  These discussions are the center of business today; their importance cannot be understated. The social web grows too fast for vendor solutions to keep up.  Therefore, vendor solutions must embrace the social web, and learn how to crowdsource.  There is no way that you can scale communities and community managers that fast. Customers don't care about your organization structure, they just want answers a...
1 comment
Read more

A Change In Perspective on Mobile Content Creation

Over the past few years, I've championed the idea that smartphones are by-and-large consumption devices; not creation devices. Someone asked me recently: Do you send email from your smartphone? Isn't that creation? I suppose that it is. But I typically send bite-sized emails, SMS, and Tweets; not long-form content. But when you also share photos and videos and links and likes and plus-1s, smartphones are clearly creation devices. This reflection challenged my previous perspective from thinking of content exclusively as robust documents to include rapid-fire info-blurbs. Where now is the dividing line between smartphone content creation expectations and those of laptops and desktops? This question prompted me to try the previously unthinkable; to write a blog post on my mobile phone. This is clearly long-form, but still distinct from the rich authoring experience I enjoy on my full-fledged computers. Perhaps then the discussion ought not be about content creation, but...
Post a Comment
Read more