Skip links

Product Playbook

Problem Research

The goal of this phase is to identify new product opportunities by purely focusing on user problems: What do your users want and need?

Gather as much data as possible from users and existing/competing solutions to understand the problem space. Those results are necessary groundwork before moving into the solution space and vital to decide what to build and whatnot.

If you want to launch a new product you should find answers to the following questions:

  • Who are our users?
  • Is there a real problem that is worth solving?
  • What do they want and need?
  • How do people solve the problem today?

If you want to improve your existing product you should find answers to the following questions:

  • What problems do users face with our current product?
  • How are users using our product?
  • Why are we currently losing users? Why are users not using feature X?
  • What else could our users need/want besides our current offering? What are they missing?
  • Which personas could benefit from our solution and which personas actually use our product?
  • What is our competition doing? What trends are upcoming in our market?

Goals & Hypotheses

To be able to decide on the right research technique(s), first, you need to specify what your research goals are, i.e. the questions/hypotheses you want to get answers for.

Some examples:

  • How do users currently solve XYZ? Are they able to do it with our current product?
  • Which type of people is struggling with XYZ?
  • What would [target group] be willing to pay for XYZ?
  • Could our users also like to receive service XYZ from us?
  • How well does our current product resolve our user’s pains and needs?
  • We believe in 1 year X% of [target group] will buy [X] online.
  • We believe feature [X] could give [target group] more reasons to sign-up/subscribe.
  • We believe the new regulation X will lead to more people need Y.
  • We will know this to be true if…

Prioritize Hypotheses

Make sure you focus on the hypotheses that are most important for your business aka your current goals. You might want to establish a Discovery Backlog in your organization, similarly to your JIRA backlog for delivery.

Assumption Mapping is a great method for prioritizing discovery topics. In this team exercise desirability, viability, and feasibility hypotheses are made explicit and prioritized in terms of importance and evidence. If you use it for your Problem Research you will prioritize mainly desirability hypotheses.

Learn more: https://www.strategyzer.com/blog/how-assumptions-mapping-can-focus-your-teams-on-running-experiments-that-matter

Don’t forget to align the priorities with your stakeholders and make them transparent.

Collect & analyze data

The following are the most common methods to gather data. We explore each in more detail later.

  1. User Surveys: Collect quantitative data from many people about a certain topic.
  2. User Interviews: Quickly and easily collect user data and feedback through 1:1 sessions.
  3. User observations / Focus groups: Collect data through observing behaviours of people.
  4. Competitor Analysis
  5. Product Analytics
  6. Market Studies

Quantitative vs qualitative data

Depending on the questions you want to answer, a quantitative or qualitative test is better suited. Often a combination of quantitative and qualitative methods provides you with the best insights for your product.

quan-qual

While quantitative methods such as surveys are good in finding answers for the WHAT and HOW of users’ behavior and motivations, 1:1 interviews are better suited for a deeper understanding of WHY users are doing certain kinds of actions and behaviors.

Make sense of the collected data

Once the data is available, you need to interpret it. A few common techniques to do so are:

  1. Jobs to be Done (JTBD)
  2. User Personas
  3. Analytics Dashboards


By looking at your data you should be able to either identify a new product opportunity or falsify your research hypothesis (and consequently avoid building something nobody wants).

Identify Opportunities

After collecting your data and analyzing/interpreting it, it is time for you to:

  • Define promising product opportunities to validate further.
  • Recognize hypotheses that were falsified; this is a good thing, if you had built a solution for it right away you would have wasted time and money.


Don’t forget to create a brief research report and align your results with all your stakeholders. Discuss your interpretation of data with other team members.

Here is a great article on how to create great research reports:

https://medium.com/decoding-research/writing-a-user-research-report-tips-and-template-slides-f271cdfea043


Now let’s take a more detailed look at some of the aforementioned research methods.

User Surveys

Surveys are a great and fast way to test hypotheses.

Steps to take

  1. Define your target group (for a new product make an educated guess about the user persona)
  2. Define the goals of your survey and the hypotheses you want to test
  3. Create your questionnaire
  4. Let your questionnaire test and feedback by 1-2 of your own people
  5. Find distribution channels (in case you don’t have an existing user base)
  6. If needed, add participation incentivization (e.g. voucher) to increase the participation rate
  7. Distribute survey
  8. Summarize results (find patterns)

How can you find participants for your survey?

If you already have a customer base, you have a great user base at your disposal.

If you are just starting out, you need to distribute your survey inappropriate channels where you expect your target group to be present:

  • Be your own recruiter: find out where your personas are located and publish your survey there, e.g. post in Facebook groups and other relevant forums, run Facebook ads, reach out to people on LinkedIn, etc.
  • There are also (mostly paid) services you can use to get access to the right users (e.g. Testing Time, Google Survey, Appinio).

To ensure you only get feedback from the right users, you should add “filter questions” at the beginning of your survey to exclude non-fitting participants. For example, if you want only smokers to participate, you can add a binary question “Are you smoking?” and stop the survey for non-smokers right away.

Collect 60-100 responses to get reliable results.

User Interviews

User interviews are great to collect user insights that you cannot reveal through quantitative methods and they can be applied at any stage of your product. Interviews let you explore the WHY.

Experience and best practices show that after talking to 5-7 users you’ll start to see a pattern, and you can almost predict what the next participant will say. Thus around 8 interviews are sufficient to get the answers you are looking for.

Note that in this stage you explore general user behaviors and pains independent of your solution. If you want to interview users to get feedback on your existing products, these so-called usability tests are covered in the Solution Validation stage.

Steps to take

  1. Define your target group (for a new product make an educated guess about the user persona)
  2. Define exactly what you want to learn and the hypotheses you want to test
  3. Create your discussion guide and prepare for the interviews
  4. Let your discussion guide test and feedback by 1-2 of your own people
  5. Recruit users and ensure they match your target profile
  6. If needed, add participation incentivization (e.g. voucher) to increase willingness to participate
  7. Conduct interviews
  8. Summarize results (and find patterns)

How can you find people to interview?

If you already have a customer base, you have a great user base at your disposal, just reach out to your most active users or those who fit best to what you want to find out.

If you are just starting out, you need to search for people in appropriate channels where you expect your target group to be present:

  • Be your own recruiter: find out where your personas are located and contact them. Sometimes that can be friends, family, or colleagues (as long as you make sure they are not biased). Or look for people in co-working spaces or public places such as cafes or on the street. You can also post in Facebook groups and other relevant forums, run Facebook ads, reach out to people on LinkedIn, etc.
  • There are also (mostly paid) services you can use to get access to the right users (e.g. Testing Time, User Interviews).
  • Don’t forget to ask those who already agreed to the interview if they know others who could fit.

How to create a Discussion Guide

Creating a really good discussion guide takes practice and it is best to stick to principles that are proven to work. Here is what you should do in an interview:

1. Set the context

Explain what are you trying to learn? Example: How do people deal with pet health problems?

  • Who are you? Your name
  • Why are you here? Conduct user research
  • Who are they? Can you tell me a little about who you are and your experience in this area?

2. Focus questions

Explain the key topics you want to cover in your interview. Example: tell me the last time you had a health problem with your pet:

  • Did your pet have a health problem recently?
  • When was the last time, tell me about that…

3. Probing questions

These are questions you might not ask if they were covered with the first set of broad questions. They are important things to ask, related to your focus, for example:

  • Why did you decide to go to the vet?
  • What were your steps going to the vet?
  • Where were you? How did you go?
  • Who was with you?
  • What happened there?
  • etc…

4. Closing

Say thank you, ask for referrals and if you can do follow-ups:

  • Thanks, this was really helpful and interesting for us. Do you know anybody we should talk to that had a pet health problem recently? Can we contact you again when we bring the product to market? Thank you.

Best practices and tips for great interviews

    • Prepare in advance
    • Be a good listener: Your job is being there to listen and not to talk
    • Ask open questions and be quiet (silence is really important)
    • Get people to tell you stories ( Eg.: Tell me about the time where this really went bad or well?). Keep in mind that their way of telling the story is not always the reality. For example: When the interviewee answers “I always do X….”, don’t stop here and go to the next question. Ask instead: Can you describe the last time you did X? People have the tendency to label things as always, never, etc. Your goal as an interviewer is to understand what she/he really means by that and stop speculation about the future.
    • Avoid guiding people by saying things as “Great”, “Yes”, “Me too” or making sounds and faces that will give them approval. Instead, use the mirror technique and just repeat what they just said. Example: What did you eat for breakfast today? I ate a Chocolate Croissant. Instead of showing satisfaction with your face or say “Humm, delicious”, you just repeat – Ah, so you had a Chocolate Croissant for breakfast, something else?
    • Be aware that people try to please you (even after testing a bad product, they have the tendency to say the product was good). It is important to be aware of such biases and use the proper techniques to avoid them.
    • People have imperfect memories: When they tell you something as a fact, keep in mind that these are not necessarily facts. You need to triangulate from multiple sources.
    • People will speculate: They will say “It would be great if I have… “. It is your job to unpack those speculations and understand what they are speculating about.
    • People will predict their actions incorrectly: For example if you ask them “Do you think tomorrow you will eat a healthy breakfast?”, they will answer “Sure. Tomorrow I will eat a healthy breakfast”. But if you ask them “What did you eat for breakfast today”, they can answer for example “I ate a cold pizza”.
    • Thus, during user interviews always ask about past/present experiences and never about the future. Avoid sentences like “Would you use that product?” There is a big chance they will say “yes” because they are trying to please you or predict their behavior incorrectly.
    • People like to share projections and aspirations (their wishes for the future).
    • Be aware of leading questions, which prompt the respondent to answer in a particular way. For example: Do you have any problems with your boss? This question prompts the person to question their employment relationship. In a subtle way, it raises the prospect that there are problems. Compare this with “Tell me about your relationship with your boss.” The second question does not seek any judgment and there is less implication that there might be something wrong with the relationship.
    • Ask WHY several times to identify the root problem:
      ask-why

Competitor Analysis

The goal is to get a deep understanding of your competition and any other existing solution and to identify potential limitations or improvements. Some aspects to consider:

  • What features does our competition offer?
  • How does competition solve XYZ?
  • How does competition onboard the user?
  • How do competitor products compare regarding relevant buying decision dimensions?
  • Main Competitors UX Flow (here you can go screen by screen of the competition, you can do screenshots, or/and create a sitemap)
  • Create a chart with all the UX/UI problems you found out and opportunities (the things that you could do better)
  • Users Feedback: try to find out what users are telling about the competition (eg.; Google Play, Apple Store, Facebook, Blog, etc) and map all the reviews (positives and negatives)
  • Social Media Presence – how many followers, and where the competitor is present (eg., Instagram, Facebook, Linkedin, Youtube, etc)
  • Moodboard, colors, inspirations, photos style, logos, etc from the competition

Even if what you plan to build already exists it can be worth considering entering the market if existing solutions are “not good enough” and you believe you can build a 10x better product and solve the user problem better/faster/cheaper/more convenient.

Product Analytics / Desk Research

In case you already have existing product(s), you have the luxury to dive into data from your existing user base. As a product manager, you should create dashboards to have real-time access to your most important metrics and look at them on a very frequent basis.

This data can then help you reveal discovery topics. Often you need to continue with further research to understand the meaning of the data, e.g. if feature X is not often used you might want to interview users why they don’t use it.

Common data sources used:

  • Web or App Analytics
  • A/B Testing or Multivariate Testing
  • Regular Surveys or Questionnaires (e.g. NPS)
  • Eye-tracking and HeatMaps

Learn more about Product Metrics in the product delivery part of the playbook.

Jobs to be Done (JTBD)

Jobs To Be Done (JTBD) is a framework for looking at your product, the problems, and its possible solutions in terms of the jobs customers are trying to get done. In other words, the JTBD is the reason why your customers hire your product or service.

JTBD is about framing the problem not the solution.

JTBD theory provides a way of understanding the foundational question of innovation success: What causes a customer to purchase and use a particular product or service?

The “Job” metaphor in JTBD is meant to evoke the idea of an employer hiring an employee to do work. When applied to innovation: consumers “hire” products to do the work of helping them make progress; the product does the work, while the consumer enjoys the benefits (progress). If the product can deliver those benefits, the consumer will continue to employ it. If it cannot, the consumer fires it, and something else is considered.

JTBD should be conducted in a workshop session with your team. Make sure you collected enough data (at best quantitative and qualitative) before running this workshop.

Learn more about JTBD here: https://firstround.com/review/build-products-that-solve-real-problems-with-this-lightweight-jtbd-framework/


Get started by using this Miro template: https://miro.com/templates/jobs-to-be-done-template/

User Personas

For each product, you should have user personas defined. Clearly defining personas gives you a deeper understanding of your customers which in return helps your team make better product and marketing decisions.

Users can be external (individuals, companies) or internal (e.g. your sales team).

By combining the JTBD results with the findings from qualitative and quantitative researches you can define your target user personas depending on their background, pains and needs.

Learn more here.

Miro Template: https://miro.com/templates/personas/

Further Resources

Tools

Other