Conducting Product Discovery

Jim Morris is the founder of the Product Discovery Group, through which he coaches product teams on idea development, user research, and testing with rapid prototypes. In this guide he explains how to use product discovery to test for value, viability, feasibility, and usability continuously, to minimize waste of precious development resources.

This is a preview of a single guide - to access the full library or request a call

Questions covered in this guide

What is product discovery?

Discovery is the process of finding a solution that is valuable, usable, feasible, and viable:
  • Valuable – will they use it? Figure out whether the idea you have is valuable to somebody. only test with people who’re in the market now, or were in it recently.
  • Usable – can they use it? Test whether someone can figure out what you’ve given them. Is everything self-evident, does it fit modern standards, is it easy to use, or is it confusing to someone? Adhere to what is already out there so that users can move through your system without thinking.
  • Feasible – can we build it? See if the idea is technically feasible. But, push the limits of what engineers are comfortable with, especially if it’s valuable to users. Ask what’s just now possible.
  • Viable – should we build it? See if the idea is viable to the business. Consider factors like business model constraints, ethical responsibility, finances, geographic reach, and government clearance. Sometimes, as a business, you just decide not to go into certain areas.

Why is it valuable to invest in product discovery?

To save engineer hours – you shouldn’t commit 10 engineers to three months of work with nothing more than a vote, or the opinion of the highest-paid person in the room. Instead of making that big bet, you can test assumptions and risk upfront. Using Discovery, you can iterate concepts in design prototypes without spending time on security, privacy, and scalability features that occur when iterating in engineering. We can be more efficient and use just part of one engineer’s time to gather evidence and learn from a qualitative discovery effort. 

To address risk – building a product in a vacuum exposes the team to the risk that users will not use it. Whenever there is risk (money spent, time spent, missing the sweet spot of the market, etc), product discovery can be used to understand and reduce the risk.

To increase innovation – teams under deadline pressure will take a safer route and explore fewer options. With discovery, teams test multiple solutions during the design phase to quickly see which ones are valuable. Creating multiple solutions unlocks the creativity of the whole team and increases the probability of finding a winner.

How is the product discovery mindset different from the traditional delivery mindset?

Traditionally, executives try to see into the future (and it rarely works) – in the delivery mindset executives are making the decisions about what the solution should be.

Discovery is a bottoms-up mindset, it’s up to the team to find the solution – instead of telling the team what to build, you tell your intelligent team to go after an outcome that’s tied to a customer problem or opportunity. A discovery mindset harnesses more of the company’s intellectual capacity.

When does it make sense to pull up and make a big product discovery investment?

When you’re launching your second (or subsequent) product – the second product is a big use case for product discovery, mainly because the first product often seemed fairly obvious to the founders, but it’s harder to know what additional products people will actually use.

If you have renewal problems – with annual contracts, it’s usually the 13th month, during that first renewal, when you lose customers; discovery tells you sooner whether there’s going to be a 13th month.

To revitalize a product – if momentum is flagging and you want to go back into growth mode.

If you have teams who are new to discovery concepts – a big push could make sense if you’re trying to teach the members of a product team.

Who owns product discovery, and who’s involved?

 Product management leads – if there’s a product management discipline in a company. If there isn’t anyone in product management, it’s definitely whoever is responsible for greenlighting engineering or starting a project.

Usually, a 3-5 person team is involved – different tests might fall to different team members
  • Product manager – in charge of deciding what to do with scarce engineering resources, often responsible for business viability 
  • Lead Engineer – often responsible for technical feasibility
  • Designer – often responsible for usability
  • Customer subject matter expert – this can be a CS, tech support, or salespeople who know the customers well.
  • Data subject matter expert  – often a data analyst who can pull numbers at a moment’s notice, and who can educate the team on how to use and when to use numbers.

What are the steps in the product discovery process?

Step 1: Find and Prioritize Problems

Develop context – the team needs to explore how your product or service is currently differentiated. What’s the definition of an activated user? And what behavior change are you seeking to create?

Prioritize problems/opportunities (don’t prioritize solutions) – map the user journey and then identify the pain points. Problems and opportunities are much easier to prioritize. It’s easy to get down to the top 10 in most situations. As for the solutions, there are literally thousands.

Identify the target – what’s the target customer is a question you take and wrap around the problem you have. Most teams have a hard time identifying their target customer. For example, if you sell to companies that have both expert users and general users, which is your focus for this particular product? Making an interface for both of these people is quite difficult.  

Define your goals – I prefer OKRs (objectives and key results) for goal-setting because it’s the simplest form of metric framework that exists (vs. frameworks like MBOs and SMART goals).

Step 2: Design

Do design sprint activities (sketching) – this is where you get into design sprint mode where you’re looking at competitors and doing sketching. This is an activity that everyone does (not just product or design) because you’re trying to take advantage of the intellectual capital in your company.

Don’t group brainstorm – have team members sketch and generate ideas independently, then share. When you have people group brainstorm together out loud, loud voices are loud and quiet voices are quiet. Good ideas are not related to loud or quiet.

Narrow down to 2 or 3  solutions – the sketching is input and then the team will discuss and vote. Get down to two or three solutions, but never one. If you show your users the different solutions, and make the difference between them bigger, you’ll get better feedback. And you’ll send a message to your team that customers are part of the idea selection process.

Step 3: Build a prototype

It needs to be “real enough” – it shouldn’t be fully functional, but it needs to look and feel like a real app or web page. It’s important that the content is realistic–if you put a bunch of fields into Excel and run a graph, the content has to be realistic for people to believe it. In other words, you can’t just throw in weird numbers because people will become distracted by them.

Be experimental – build the prototype to get reactions from users. Don’t try and build a full user experience yet. To get a clear understanding of user intent, you will need to isolate concepts, sometimes putting just one idea on a screen. And provide multiple calls to action to let the user “Choose Their Own Adventure”.

Use tools that make modifications easy, like:
  • Figma – when we do user interviewing, we can actually customize and change the prototype during/ after interviews. 
  • Invision
  • Adobe XD
  • Marvel
  • Axure 
  • Framer
  • A report driven off Excel

Step 4: Test with customers

Recruit customers – these days, testing is all over Zoom. You might use your own customers, or recruit through a site like userinterviews.com.

Conduct 30-minute moderated user interviews – I prefer moderated user interviews, which is where you’re present and asking questions. With unmoderated tests, it’s harder to test value as the user needs to be directed through the various screens. Now that everyone has Zoom installed, they usually dial in with very few IT problems and a 30-minute appointment is enough.

Use a prototype to facilitate the conversation – use the prototype as a talking point and as a medium for conversation. When you are evaluating solutions, you don’t need to know about the whole user, you should be focused on the behaviors that apply to the situation you’re solving for. You’re testing to see whether the user will want to use the technology so make sure to give them control of the prototype and avoid narrating the solution. This allows the user to find their own way at their own speed. Encourage them to think out loud as much as possible. 

Offer an incentive – after we’re done with the interview, users receive an incentive which is usually somewhere between a $25-75 Amazon gift card. 

Step 5: Validate

Have hypotheses going in – this keeps you honest so that you can’t rewrite the results when you’re done. You’ll have a primary hypothesis about the overall value of the concept and then you’ll have secondary hypotheses about particular screens or interactions (i.e. we want to see positive responses from X people when they see Y).

Get a demand signal – will the user enter a credit card, download the app right now or give you their email address? At what price point would they switch to your solution? Find a way to get a form of commitment from the user. You don’t need to actually collect the credit card number or payment but seeing a user’s intention (or lack thereof) builds confidence in qualitative research.

Ask a satisfaction-focused question to gauge actual interest in the overall concept – in discovery we need to separate people who are sort of interested in our concept from those who are really interested. There are three ways to do this that have varying degrees of accuracy.
  • Disappointment question – most useful. Ask how disappointed people would be if they couldn’t use the product. Those who like the concept might answer “Somewhat Disappointed” but those who love it will answer “Very Disappointed”. Look for somewhere north of 40% of people to say they would be “Very Disappointed.”
  • NPS – (net promoter score). Most common. “How likely would you be to recommend this to a friend on a scale of 0-10?” Users who answer 9 and 10 are known as “Promoters” who like your concept enough to tell others about it which is very powerful. The limitation is that not every product lends itself to being recommended. A few examples are products that apply to specific health conditions or products that users don’t want to tell others that they are using.
  • Likert scale – most vague. This one is the hardest to analyze. It’s the question of how valuable is the product to you? The options are: extremely, somewhat valuable, neutral, not that valuable, or totally not valuable. Surveyors tend to group the top two and bottom two answers to get a sense of sentiment. The negative sentiment responses are useful. But the positive sentiment answers are not as strong an indicator of future adoption as the other satisfaction questions listed above.

How should you set up your customer interviews, and what questions should you ask?

Recruit users who are in market – look for people who are currently looking to solve the relevant problem, or who have been in the market very recently.

Tap into a need the user has now or had previously (don’t say “imagine you’re trying to…”) – we don’t want people to imagine and make things up. Instead, focus on concrete current or recent experiences. These actual experiences will ground the user’s feedback in reality.

Keep the setup of the interview short and focused on a behavior – start the interview with a realistic situation in a starting point that would be common for encountering your solution. You’re hoping that person can solve their problem or be compelled to take advantage of the opportunity you’re presenting while thinking out loud as they go. I often encourage them to think out loud. If the conversation lags, remind them of the goal and ask “what would you do next?”

Never ask shallow questions – these don’t actually get at value or natural behavior with the product. The answers won’t help you.
  • What do you see?”
  • “Please read this and tell me what you think” 
  • “Would you use this?”

What techniques and frameworks should the team use when prioritizing ideas?

Measure reach (it’s the most important) – measure how many people will use this and will be affected by this. Are we affecting 70% of users or only 10%? If it’s only 10%, then why? Reach is easier to measure objectively vs. something like impact–with impact it’s easy to put your finger on the scale.

Factor in previous commitments – CEOs and sales make high-integrity commitments to customers and prospects. You want to make sure the company makes as few of them as possible to be successful, but if your company has made a commitment, that needs to be factored in.

Deliver delighters – everyone loves a little bling. If I don’t have an idea that is a little exciting to talk about each quarter, I know as a product person I’m not supporting my sales team.

What should you not do discovery on?

Performance features – when you have nothing to give to engineers, you need a list of performance features ready to be worked on.

Security, Privacy, Reliability enhancements – these are more “when” than “if.” Discovery should be reserved for “if” scenarios where there’s a risk of building the wrong thing. That said, there are always gradations on how secure, privacy-compliant and reliable a given needs to be. These trade-offs should be made hand-in-hand with the business not in an engineering silo.

How many customers should you test with? 

Look for 5-6 to start – there are a lot of studies that tell you how much you can learn from five customers, so plan for six and get at least five rather than four or three (more likely than not, at least one of them won’t show up). 

Iterate more with 3 at a time – after the first 5, start to iterate. Test with three more users and then three more (repeat as necessary).

Show it to at least 15 total – if you have a good idea, it should never see the light of day without going in front of 15 customers. This is not a hard and fast rule though; it just helps you develop the ability to discern high quality from low-quality user feedback. 

Iterate on negative feedback – don’t wait until the last user to remove a concept or tweak it. Once you see a trend especially with negative feedback, make a change so that you can test new and improved concepts right away. Positive feedback typically needs more rounds of users to feel confidence.

How can you continue to run discovery in parallel with development when you start building?

Engineers need to block time for discovery – you need to block the time because engineers can’t have 10 business days worth of dev work in a two-week spirit and still do discovery. The scrum master needs to block eight to twelve hours for the lead engineer (or whoever is participating in discovery) so the person is not double booked. 

Designers need to be dedicated to the team – designers need to be dedicated. An agency model or a Center of Excellence model is not the model for the discovery. To make great user experiences, designers need to know the context of the solution they are working on (just like the Product Manager and other Discovery team members): what is the problem being solved? What are the current metrics? Who are the users? And so on. A part-time designer will only be able to give a superficial visual and usability treatment to the concept and won’t be able to create an impactful experience. 

Product needs help with delivery – you need a Delivery Manager (this could be the engineering lead or scrum master). They’re only bringing the stickiest problems to the product manager, so that this core discovery team is spending half their time doing discovery. Otherwise, product managers end up spending 50% of their time with engineers and 50% of their time with stakeholders.

How do design thinking concepts like Design Sprints and User Experience Mapping fit into product discovery?

Use a design sprint when risk is high, or there’s disagreement – when the risk is high and people don’t agree, a dedicated sprint with an outside facilitator is a good idea. For big strategic questions, sprints help jumpstart discovery.

You always need a user experience map (it lasts), it’s foundational – an experience map should be one of the first things you build for your product–a user encounters a trigger, starts here, does these activities, and ends at a specific goal. Some of these assets are reusable for six to 12 months; you keep coming back to them and refining them.

What are the most important pieces to get right?

You only want to scale what’s working – discovery helps you make sure that new development will work before you put it out on the platform. 

Trust the dream, but verify – you can’t start a company and get funding without the dream, but funding and press don’t count as proof of value for your users.

Test value before usability – always test value first. It’s a lot of work to make something that accurately tests usability (don’t invest in that work before you’ve proven value).

What are the common pitfalls?

Don’t do a “big bang”– too often discovery is this “big bang” where they’ll do a bunch of discovery to create 18 months of dev work–discovery needs to be a constant thing that runs alongside development.

Don’t increase scope without testing – you need to do discovery frequently enough so that you can always test a new request quickly like next week. That way, there’s never an excuse to increase dev scope without testing. 

Be careful with ideas from people who weren’t on the team – collaborative discovery creates shared understanding. There’s no substitute for being in the room, it’s tough but important. New ideas from the outside lack the context and framing that the Discovery team has built up over time.

Don’t outsource your testing to the user research group – user research teams are good for generative research, to find problems that are themselves high potential areas for discovery. But the discovery team needs to do their own user interviews to validate solutions.

To browse the full library of guides

Already a member? Sign in to view the full library.

GET FREE GUIDES IN OUR NEWSLETTER

Scroll to Top

Request Access

 

Want free guides?

We feature guides every week in our newsletter