What’s the difference between output-based roadmapping and outcome-based roadmapping?
A focus on shipping features (outputs) vs. KPIs (outcomes) – in output-based roadmapping, your definition of success is that you shipped features on time. In outcome-based roadmapping, you optimize around success metrics to know if what you’re building is successful in moving the needle for the customer and the business.
What are some examples of key outcomes?
- I use Twitter to stay informed – I don’t use Twitter to write tweets, I use it to keep up with the latest thought leadership and happenings in my space.
- I use Peleton to lose weight – I don’t get on a Peloton because I want to sweat or because I’m just looking to work out or to cycle. I get on the PelotonPeleton because I’m trying to lose weight, which is the key outcome.
Business metrics might take too long to move – for most products, business metrics like revenue, bookings, retention, etc. tend to be trailing indicators of success. Sometimes a key outcome might take a while to deliver; in those cases, it’s important to identify leading indicators. For example, a Peloton user’s key metric might be losing weight, but usage is an earlier indicator to predict whether that’s going to happen.
- ROI Generated – B2B customers often care about return on investment (ROI), calculated by measuring the benefits of a product compared to the costs.
- Topline revenue increases – track the metric that measures how your product helps improve your customers’ topline revenue.
- Cost savings – track metrics that indicate how your product is helping customers save direct and indirect costs.
- Ease of use – track the amount of time it takes to roll out your product in the organization. White-glove implementation can be helpful, and some products require it, but it also takes time away from employees doing something else that they could be doing to generate value for their business.
What are the steps to create an outcome-based roadmap?
- Choose which persona you want to serve – make sure it’s really clear who the personas or the segments of the market are that you’re trying to serve because the key outcomes may vary.
- Choose what outcome you want to drive for them – define what outcomes are valuable to the chosen persona.
- Get alignment across teams – you don’t want to get into a situation where sales thinks they’re trying to sell the one group of people and deliver one set of outcomes and then marketing is targeting a different group of people with a different set of outcomes that product is building for a third persona and different sets of outcomes.
Step 2: Decide how much capacity to allocate to each outcome – how much of our product development capacity do we want to dedicate to each of these different outcomes that we’re trying to drive? It’s a really important top-down allocation decision that should be made before you even try to craft a roadmap.
Step 3: Decide what solutions will move the needle most for each objective – which solutions that we’re considering bringing to market will move the needle most on each of those outcomes? You’ll need rationalization for how those things are prioritized. This rationalization comes from quantitative or qualitative customer discovery, competitive intel, or gut instincts.
How do you balance new functionality, enhancements, and tech debt work on your roadmap?
- Innovation – progress towards the vision.
- Iteration – tweaking of the existing product.
- Operation – the cost of modern SaaS platforms, like performance, security, uptime, bugs, internal tools, tech debt, etc.
- In the beginning – it might be that 80% of your capacity is innovation.
- When you’re seeking traction – you might see 60% of your capacity go towards iteration towards finding product-market fit after launch.
- When you’re trying to scale – you have to increase the operation allocation to build more internal tools for customer support and think about the performance and scalability of your product.
You can also do a percentage allocation across OKRs or by product line – you might say that you want to split your time 50/50 between two personas, or you might have three OKR’s and give each one third of your time. You might also have it sorted out by market segment or product line. You might give an existing product 20% of your capacity and a new product 80% of the team’s time.
How do you avoid being unreasonably ambitious in your planning?
Lean on UX artifacts to make better estimates – I tend to lean on the user experience artifacts such as flow diagrams and wireframes to make sure that everyone is on the same page about what we’re trying to build and deliver and why that’s the sort of experience we think is the right one. When you try to estimate with words alone, everyone might have a different interpretation of what the scope of a feature is.
Don’t fill 100% of your capacity -when you’re estimating build timelines, you can leave a buffer and just say, “I’m going to just try to estimate what 80% of the team’s time could be spent on”, knowing that feedback will come in and the estimates will be wrong. This avoids the domino effect where one delay throws the entire plan off.
Communicate a level of uncertainty – when you’re sharing these timelines or roadmaps with your stakeholders, communicate that some of the timelines are estimates and may need to be shifted, especially the further away from today you get
How do you recommend visualizing and presenting roadmaps? What tools do you like to use?
I hate the Gantt chart – it looks like a project plan, and therefore people assume it’s an accurate timeline. It communicates false precision, and I’ve been burnt by that.
I like the way a now-next-later format conveys priority – the most valuable thing that the roadmap does is spark a discussion about priorities and who we should be serving and what changes we need to make to the product to better serve them. A now-next-later prioritization achieves that.
I like swim lanes on roadmaps – swim lanes help better contextualize the categories of work, especially when lanes are outcome-based. That way, you spell out explicitly, “these are metrics that we’re trying to move as a product organization”. Swim lanes are compatible with a now-next-later roadmap.
When you’re in customer discovery and solution brainstorming, it’s better not to “fake it” – rather than communicating your own ideas for the right solution, it’s better to just be honest and say “hey, I’m not sure what this looks like, but we’re going to spend a month or two months doing a bunch of discovery interviews.” One of the things I love to do is put dotted lines around the box to indicate that this work is penciled in (and time-boxed) as a critical part of the product development process
There are some roadmap tools out there, but they’re not required – I’ve always ended up just coming back to spreadsheets and slides because they offer so much flexibility to tailor the content to your company culture/audience. Some tools, e.g. ProductPlan and Productboard do a really good job connecting the personas and needs to the roadmap.
Who owns the roadmap and who should be involved in the roadmapping process?
Product management owns the roadmap, in the context of a vision and strategy – it’s important to make sure that the roadmap is contextualized within the longer-term vision or big picture strategy that explains what the company is trying to accomplish, where we’re going with the product, and why that’s exciting for customers and the business.
Involve leadership to give context and get feedback on your roadmap: including the exec team, sales, marketing, and customer success. That way stakeholders know that they should talk to the product if they disagree with the prioritization, especially when something they think is a huge deal isn’t being addressed sooner.
Involve design and engineering on the discovery side: product design and engineering need to be involved early to understand the context of the customer needs, brainstorm solutions together, and to estimate how long this body of work might take to deliver. Even in an agile world, you can’t get away from timelines and commitments, but the “now, soon, later” time horizons should suffice to kickstart the relative prioritization discussion.
When (if ever) should you make firm commitments to the timing on a roadmap? How should you manage those commitments?
Commitments are sometimes necessary, but they create a lack of flexibility – there is an important trade-off to be made when commitments do need to be made. In general, avoid firm deadlines when you can, but it’s reasonable to make commitments within shorter time frames (e.g. the next 90 days). Over that type of near term, It’s common to see another artifact known as a delivery plan that explains sprint-by-sprint expected releases in the near term.
In some contexts, timing matters a lot, and a waterfall approach might be appropriate – I’ve worked in some industries like financial services where deadlines matter a lot (e.g. have to do some product work to be in compliance with a new rule by a certain date). In that situation, a waterfall approach might make more sense. You give up flexibility, but that might be necessary to hit a key deadline.
How often should you run the roadmapping process?
For most companies, quarterly is a good cadence – the simple rule of thumb is to run a quarterly roadmapping process, especially as the organization gets larger. You might refresh the roadmap more often, looking 3-4 releases ahead.
Don’t start with a blank slate each time – once you have the first roadmap, all you need to decide is, “do I need to change anything from the last thing I presented? If so, let me rationalize why some of these things might have changed.”
Look at your metrics more frequently than you run the roadmapping process – to transition to a more outcome-driven organization, you need to be looking at the metrics that you’re trying to move the needle on and go report back on them regularly. Then use those results to adjust your prioritization/roadmap.
How do you translate outcomes on your roadmap into development plans? Who’s responsible for deciding what to build to achieve the agreed-upon outcomes?
Product, engineering, and design should brainstorm together – in the more modern product teams, I see engineering, design, and product together, thinking and brainstorming and deeply understanding the customer problem or the metric of success and why it’s important and why now’s the right time to solve it. I’m starting to hear of two-day design sprints where they get together for two full days and just think of ideas. Product should ultimately be accountable for what goes out the door.
Avoid too many people and consensus – you should keep stakeholders in sales, customer success, etc. informed, and you can even ask them to participate in some brainstorming sessions to make it a collaborative effort, but the more people you involve, the more likely it is you might try to get to the census. That often ends up with a bad outcome for customers–a Frankenstein user experience where you’re taking pieces of everyone’s ideas and combining them in a way that’s confusing for the user.
Invest proportionate to confidence – the reality is that you might have a hypothesis initially and depending on your business model and your product operations culture, you might run some experiments. Set up an alpha > beta > general availability model for rolling things out. That way, you can try things with a small population of users or customers and see how it goes before you invest more and release it to more customers/users.
How should product teams manage unexpected changes that arise after the roadmap is set?
First ask: should we accept this change? – align internally about whether the new thing is important enough to derail the plan, or get put into the backlog for consideration with the next major roadmap update.
- Leaving too little time for iteration – did you allocate 100% capacity to building new features? Instead, build in a buffer for unexpected changes in scope as you learn more, and to iterate on customer feedback.
- Leaving too little time for production support – consider allocating a fixed amount of capacity (e.g. 10% of the team’s time) for production support. That way, when production goes down, or there’s a bug, or there’s an issue that you could never plan for, resources are allocated. This work can be handled in more of a Kanban (first in, first out) style instead of Scrum. And if no work comes in, great, take on some stretch goals for the sprint.
How do you measure your success at delivering the outcomes?
- This isn’t simple, it can take 2 years – there is an assumption that it’s a simple process, and you roll it out and immediately it takes hold within a few weeks. I’ve seen it take two to three years for most organizations to get OKR’s right. Have patience and commitment to the model.
- When you have different OKRs for every team, it gets confusing – when you have engineering OKR’s, and marketing OKR’s, and CS OKRs, etc., at some point, there are so many goals, how does an individual employee even know where to be prioritizing their time? This model also creates silos and competing priorities within the org.
- Avoid the over-proliferation of KPIs – if you have 40 KPIs and every KPI takes 20 hours to calculate and double-check, you now have a full-time data analyst. In general, keep your product KPIs to single digits.
- Measure the right things – figure out what the leading indicators of success are.
- Measure them often enough – after every release, look at the metrics to see if the work moved the needle. Build time into your product development process to do this work.
- Make sure you have the ability to measure – make sure your product is instrumented to be able to track those metrics.
What are the most important pieces to get right?
Don’t forget about the customer and their metrics of success – don’t forget about the customer or the user. It’s really easy to get caught up in a world where your entire roadmap is optimized to keep your internal stakeholders happy, but that’s not your job. Your job is to keep the customers happy. Don’t forget about what the customers’ metrics of success would be. Create value for customers, then capture it with your business model.
Don’t forget that a roadmap should be contextualized with your bigger direction – your roadmap should be contextualized within a multi-year vision. Make roadmapping decisions with a sense of direction for where the product is going in different dimensions in which you are trying to add value for customers.
Think hard about how you communicate your roadmap – consider the swim lanes you choose, and what they communicate. I see a lot of teams default to a tech stack framing, e.g. dividing up: web, mobile, and backend. But, that obscures what feature is going to launch to what set of users at any given time because of all the dependencies across the teams. It’s not easy for a non-technical person to understand.
What are the common pitfalls?
Don’t try to be too scientific with scorecards – there’s a belief that there is some scientific method with the right weighted scorecard that’s going to spit out the answer of what their priorities/roadmap should be. In reality, there’s an art and a science to product management. I’m fine with scorecards, but you have to recognize that even the way you score initiatives is subjective. Also recognize that scoring frameworks like RICE spit out quick wins (high impact, low effort) and may preclude you from building big, innovative changes into your product.
Access more roadmapping templates from Prodify here: Build What Matters Resources