INSPIRED - How to Create Tech Products Customers Love (Silicon Valley Product Group)

By: Marty Cagan
Source:https://www.amazon.com/INSPIRED-Create-Tech-Products-Customers-ebook/dp/B077NRB36N/

It doesn't matter how good your engineering team is if they are not given something worthwhile to build.

Generally have three stages of companies:

2021-10-20 Book Club Discussion

Started discussion on Top Reasons for Loss of Velocity

I believe this speaks more toward the lack of product strategy right - for all of cloud?

I would like to add to this list of top reasons for loss of velocity: From my perspective- another reason for the loss of velocity is the engagement with security. They are another partner here to the triad. So we do not have the ability to experiment as much because we are trying to build consensus across 4 pillars. If we have to embrace experimentation, we have to embrace risk.

Rally to objective process sounds similar to the what worked at interactive design and development agencies I worked at.

Pivoting to The Good Things we are doing to build a strong product culture…

The Root Causes of Failed Product Efforts

FIGURE 6.1 Root Causes of Failed Product Efforts failed product

The first truth is that at least half of our ideas are just not going to work.

The second inconvenient truth is that even with the ideas that do prove to have potential, it typically takes several iterations to get the implementation of this idea to the point where it delivers the necessary business value.

We call that time to money.

The biggest flaw of the old waterfall process has always been, and remains, that all the risk is at the end, which means that customer validation happens way too late.

Beyond Lean and Agile

Three overarching principles

Risks are tackled up front, rather than at the end.

Products are defined and designed collaboratively, rather than sequentially.

In strong teams, product, design, and engineering work side by side

Finally, it's all about solving problems, not implementing features.

Conventional product roadmaps are all about output. Strong teams know it's not only about implementing a solution. They must ensure that solution solves the underlying problem.

It's about business results.

Key Concepts

Holistic Product

Continuous Discovery and Delivery

FIGURE 8.1 Continuous Discovery and Delivery failed product

We are always working in parallel to both discover the necessary product to be built—which is primarily what the product manager and designer work on every day—while the engineers work to deliver production‐quality product.

Product Discovery

The purpose of product discovery is to quickly separate the good ideas from the bad. The output of product discovery is a validated product backlog.

Will the user buy this (or choose to use it)? Can the user figure out how to use this? Can our engineers build this? Can our stakeholders support this?

Prototypes

Product discovery involves running a series of very quick experiments, and to do these experiments quickly and inexpensively,

Product Delivery

The purpose of product delivery is to build and deliver these production‐quality technology products, something you can sell and run a business on.

Products and Product/Market Fit

Just because we've invested the time and effort to create a robust product does not mean anyone will want to buy it. So, in the product world, we strive to achieve product/market fit.

Product Vision

This refers to the longer‐term objective of this product, normally 2–10 years out. It is how we as a product organization intend to deliver on the company's mission.

So, we use prototypes to conduct rapid experiments in product discovery, and then in delivery, we build and release products in hopes of achieving product/market fit, which is a key step on the way to delivering on the company's product vision.

The MVP should be a prototype, not a product.

PART II The Right People

Principles of Strong Product Teams

A product team is a group of people who bring together different specialized skills and responsibilities and feel real ownership for a product or at least a substantial piece of a larger product.

Team of Missionaries

John Doerr, the famous Silicon Valley venture capitalist: “We need teams of missionaries, not teams of mercenaries.”

Mercenaries build whatever they're told to build. Missionaries are true believers in the vision and are committed to solving problems for their customers.

They are empowered to figure out the best way to meet those objectives, and they are accountable for the results. This literally means product, design, and engineering working out solutions together.

Team Scope

It's nearly impossible to have a team of missionaries when they're pulled together for a project that lasts only a few months and is then disbanded.

Try to minimize dependencies between teams.

Why It Works

First, collaboration is built on relationships, and product teams—especially co‐located teams—are designed to nurture these relationships.

Second, to innovate you need expertise, and the durable nature of product teams lets people go deep enough to gain that expertise.

Third, instead of just building what others determine might be valuable, in the product team model the full team understands—needs to understand—the business objectives and context. And most important, the full team feels ownership and responsibility for the outcome.

If your company is not yet set up around dedicated product teams, this is probably the most important thing for you to fix. Everything else depends on this.

The Product Manager

When a product succeeds, it's because everyone on the team did what they needed to do. But when a product fails, it's the product manager's fault.

You need to become an acknowledged expert on the customer: their issues, pains, desires, how they think

A big part of knowing your customer is understanding what they're doing with your product. Most product managers start their day with half an hour or so in the analytics tools, understanding what's been happening in the past 24 hours. They're looking at sales analytics and usage analytics. They're looking at the results of A/B tests.

The third critical contribution—and the one that is often considered the most difficult by many product managers—is a deep understanding of your business and how it works, and the role your product plays in your business.

This means knowing who your various stakeholders are and especially learning the constraints they operate under.

Succeeding in the job of product means convincing each key stakeholder that you understand their constraints and that you are committed to only delivering solutions that you believe are consistent with those constraints.

The fourth critical contribution is deep knowledge of the market and industry in which you're competing. This includes not only your competitors but also key trends in technology, customer behaviors and expectations, following the relevant industry analysts, and understanding the role of social media for your market and customers.

The successful product manager must be the very best versions of smart, creative, and persistent.

Start by becoming an expert in your users and customers. Share very openly what you learn, both the good and the bad. Become your team's and your company's go‐to person for understanding anything about your customer—quantitative and qualitative.

Work to establish a strong relationship with your key stakeholders and business partners. Convince them of two things: (1) You understand the constraints they operate under. (2) You will only bring to them solutions that you believe will work within those constraints.

Become an undisputed expert on your product and your industry. Again, share your knowledge openly and generously.

Finally, work very hard to build and nurture the strong collaborative relationship with your product team.

In product companies, it is critical that the product manager also be the product owner. If you split these roles into two people, some very common and predictable problems result—most commonly, the loss of your team's ability to innovate and consistently create new value

There are two specific academic courses that every product manager should take:

You will need to understand how for‐profit companies work and the main business key performance indicators (KPIs) that are important to your business—

A good general marketing course will often cover these topics

The Product Designer

Product managers who need to learn how to work effectively with designers.

Modern product designers are responsible for the following:

Product Discovery

Modern product designers continuously collaborate with product managers and engineers—from discovery to delivery.

Rather than sitting with fellow designers, the modern product designer sits side by side with his or her product manager, a full partner with the product manager on product discovery.

Rather than being measured on the output of their design work, the product designer is measured on the success of the product.

Holistic User Experience Design

User experience (UX) is much bigger than user interface (UI).

For modern products, this usually includes multiple different interfaces, as well as other customer touch points (e‐mail, marketing campaigns, sales process, customer support, and so forth).

With some products, UX also includes offline services, such as riding in a car summoned through Uber or staying in a house sourced through Airbnb.

The list of touch points could be very long, considering questions as: How will customers first learn about the product? How will we onboard a first‐time user and (perhaps gradually) reveal new functionality? How might users interact at different times during their day? What other things are competing for the user's attention? How might things be different for a one‐month‐old customer versus a one‐year‐old customer? How will we motivate a user to a higher level of commitment to the product? How will we create moments of gratification? How will a user share his experience with others? How will customers receive an offline service? What is the perceived responsiveness of the product?

Prototyping

Good product designers use prototypes as their primary canvas for communicating ideas, both internally and externally.

User Testing

Good product designers are constantly testing their ideas with real users and customers. They don't just test when a prototype or idea is ready; they build testing into their weekly cadence, so they're able to constantly validate and refine ideas as well as collect new insights

User testing is broader than usability testing. Product designers and their product teams utilize the opportunity to assess the value of their ideas.

Interaction and Visual Design

Interaction design generally includes the underlying conceptual models

Task flows, and control layouts to manipulate those concepts.

Visual design includes composition, typography, and how the visual brand is expressed.

The Absence of Product Design

Incredibly common and serious problems:

Apple is one of the most valuable and design‐conscious companies on the planet; yet few tech companies understand the importance of design talent.

We need design—not just as a service to make our product beautiful—but to discover the right product.

Do whatever you need to do to have your designer sit next to you.

Include your designer from the very inception of every idea.

Include your designer in as many customer and user interactions as possible. Learn about the users and customers together.

Fight your temptation to provide your designer with your own design ideas. Give your designer as much room as possible

Encourage your designer to iterate early and often.

Not get all nitpicky about design details with the very early iterations.

The Engineers

Learn a programming language.

Purpose of developing this programming literacy is not so you start telling your engineers how to do their job but, rather, to significantly improve your ability to engage with and collaborate

You must constantly demonstrate to your team that you're open minded, you know how to listen, and you want and need their help in coming up with the right product.

While it's good if you have a strong technology understanding, it's not good if you use that knowledge to try to do their jobs for them.

Give your engineers as much latitude as possible in coming up with the best solution.

It is your job to make sure they feel like missionaries and not mercenaries. You do this by involving them deeply in the customer pain you are trying to solve and in the business problems you face.

Product Marketing Managers

Modern product marketing managers represent the market to the product team—the positioning, the messaging, and a winning go‐to‐market plan. They are deeply engaged with the sales channel and know their capabilities, limitations, and current competitive issues.

If your company has a sales organization, and you don't have a product marketing partner, then this responsibility likely falls on you as product manager.

This can easily become a full‐time job. And given the cost of the sales organization, it's really not an option to ignore them.

Make sure you have a product marketing manager to work with, and it's absolutely worth your time to make sure you understand the market—

The Supporting Roles

User Researchers

We are continuously doing two kinds of rapid learning and experimentation. One kind of learning is qualitative, and the other is quantitative.

User researchers are trained in this range of qualitative techniques

Data Analysts

Data analysts help teams collect the right sort of analytics, manage data privacy constraints, analyze the data, plan live‐data tests, and understand and interpret the results.

Test Automation Engineers

Most companies have a blended approach in which the engineers write some of the automated tests (e.g., the unit level tests), and the test automation engineers write the higher‐level automated tests.

The level of test automation necessary to release with confidence is significant

The Role of Leadership

The primary job of leadership in any technology organization is to recruit, to develop, and to retain strong talent. However, in a product company, the role goes beyond people development and into what we call holistic view of product.

The three distinct but critical elements to the holistic view of product are described next.

Leaders of Product Management

Either the leaders of the product management organization (VP product, directors of product), or a principal product manager.

This person should regularly review the work of the various product managers and product teams, identifying and helping to resolve conflicts.

Leaders of Product Design

People responsible for the holistic user experience.

Leaders of Technology Organization

Technology organization leader (often titled CTO or VP engineering).

Responsible for the holistic view of the system implementation. They should be reviewing the architecture and systems design of all the software—

Holistic View Leadership Roles

The systems always seem to grow in complexity and size much faster than anyone can document, and with software, the definitive answer always lives in the source code itself (at least the current answer—not usually the rationale or the history).

These three holistic‐view leaders—the head of product, the head of design, and the head of technology—are obviously very valuable individually, but in combination you can see their real power. This is why my personal preference is to have these three people sit very close to one another,

The Head of Product Role

VP product is your most senior product role in your company or business unit.

Strong in four key competencies:

  1. team development,
  2. product vision,
  3. execution, and
  4. product culture.

Team Development

Develop a strong team of product managers and designers.

Hire someone who has proven ability to develop others. They should have a track record of identifying and recruiting potential talent, and then working actively and continuously with those people to address their weaknesses and exploit their strengths.

Product Vision and Strategy

Two very different types of product leaders needed for two very different situations: Where there is a CEO or a founder who is the clear product visionary Where there is no clear product visionary—usually in situations where the founder has moved on

VP product needs to complement the CEO. If you have a strong, visionary CEO, there may be some very strong VP product candidates that won't want the position because they know that, in this company, their job is primarily to execute the vision of the CEO.

Execution

You need a product leader who knows how to get things done and has absolutely proved her ability to do so.

Product Culture

A strong product culture means that the team understands the importance of continuous and rapid testing and learning.

Experience

At a minimum, you are looking for someone with the combination of a strong technology background with an understanding of the economics and dynamics of your business and your market.

Chemistry

Your product leader must be able to work well on a personal level with the other key execs, especially the CEO and CTO.

The Head of Technology Role

This is the organization responsible for architecture, engineering, quality, site operations, site security, release management, and usually delivery management. This group is responsible for building and running the company's products and services.

Titles vary but often include VP engineering, or chief technology officer (CTO).

The CIO role is very different from the CTO role. In fact, if your technology organization reports to the CIO, that is a warning flag for many of the pathologies discussed in Chapter 6,

Removing technology as a barrier, as well as broadening the art of the possible for business and product leaders, is the overarching objective.

Build an excellent organization with a strong management team committed to developing the skills of your employees.

Represent technology in the overall strategic direction and leadership of the company, working with other company executives to help inform direction, M&A activity, and build/buy/partner decisions.

Make sure this organization can rapidly, reliably, and repeatedly deliver quality product to market.

Make sure the company has an architecture capable of delivering the functionality, scalability, reliability, security and performance it needs to compete and thrive.

Make sure that members of the senior engineering staff are participating actively and contributing significantly throughout product discovery.

The CTO will serve as the company spokesperson for the engineering organization, demonstrating leadership in the community with developers, partners, and customers.

The Delivery Manager Role

In growth‐stage and enterprise companies, many product managers complain that they have to spend far too much of their time doing project management activities.

Delivery managers are a special type of project manager whose mission is all about removing obstacles

These delivery managers are typically also the Scrum Masters

Principles of Structuring Product Teams

One of the most difficult issues facing every product organization at scale is just how to split up your product across your many product teams.

There are some critical core principles, and the key is to understand those principles and then weigh the options for your particular circumstances.

Alignment with investment strategy

Phase out products that no longer carry their own weight, and we can often reduce the investments in our cash‐cow products so that we can invest more in future sources of revenue and growth.

Some people like the three horizons model, while others take more of a portfolio management approach.

Minimize Dependencies

While we can never entirely eliminate dependencies, we can work to reduce and minimize them.

Ownership and Autonomy

A team should feel empowered, yet accountable for some significant part of the product offering.

Maximize Leverage

We often find common needs and the increased importance of shared services.

Realize, however, that creating shared services also creates dependencies and can impinge on autonomy.

Product Vision and Strategy

Many larger and older organizations no longer have a relevant vision and strategy, but this is key.

Team Size

The minimum size for a product team is usually two engineers and a product manager,

For user‐facing technology, then a product designer

It's really difficult for one product manager and product designer to keep more than about 10–12 engineers busy with good stuff

Alignment with Architecture

Start with the product vision, come up with an architectural approach to deliver on that vision, and then design the teams around that architecture.

While we'd love for every team to be a full stack team that can work on any layer of the architecture, in practice that's often not an option.

Be sure to staff these common services teams with strong and highly technical product managers (often called platform product managers).

Alignment with User or Customer

If, for example, your company provides a two‐sided marketplace with buyers on one side and sellers on the other, there are real advantages to having some teams focus on buyers and others focus on sellers.

Alignment with Business

While there are advantages to aligning with business units, this usually comes after the other factors in priority.

Structure Is a Moving Target

The organization's needs should and will change over time.

PART III The Right Product

One of the key themes of this book is focusing on outcome and not output.

The Problems with Product Roadmaps

The first inconvenient truth is that at least half of our product ideas are just not going to work.

The second inconvenient truth is that, even with the ideas that do prove to be valuable, usable, feasible, and viable, it typically takes several iterations to get the execution of this idea to the point where it delivers the expected business value

This is often referred to as time to money.

Weak teams just plod through the roadmap they've been assigned,

Strong product teams understand these truths and embrace them rather than deny them. They are very good at quickly tackling the risks (no matter where that idea originated) and are fast at iterating to an effective solution.

Anytime you put a list of ideas on a document entitled “roadmap,” no matter how many disclaimers you put on it, people across the company will interpret the items as a commitment.

Sometimes, we do need to commit to a delivery on a date. We try to minimize those cases, but there are always some. But we need to make what is called a high‐integrity commitment.

The Alternative to Roadmaps

In the empowered product team model this book is predicated on, the teams are themselves equipped to figure out the best ways to solve the particular business problems assigned to them.

There are two main components that provide this business context:

Prioritizing business results, rather than product ideas.

The way we manage commitments is with a little bit of give and take.

We ask the executives and our other stakeholders to give us a little time in product discovery to investigate the necessary solution.

Once we have come up with a solution that works for our business, we now can make an informed and high‐integrity commitment

Product Vision and Product Strategy

The Product Vision

It's mainly a persuasive piece that might be in the form of a storyboard, a narrative such as a white paper, or a special type of prototype referred to as a visiontype.

Its primary purpose is to communicate this vision and inspire the teams

The Product Strategy

The product strategy is our sequence of products or releases we plan to deliver on the path to realizing the product vision.

For most types of businesses, I encourage teams to construct product strategy around a series of product/market fits.

For business‐focused companies, you might have each product/market fit focus on a different vertical market (e.g., financial services, manufacturing, automotive).

For consumer‐focused companies, we often structure each product/market fit around a different customer or user persona. For example, an education‐related service might have a strategy that targets high school students first, college students next,

There's no single approach to product strategy that is ideal for everyone,

The difference between vision and strategy is analogous to the difference between good leadership and good management. Leadership inspires and sets the direction, and management helps get us there.

Prioritizing Markets

There is no one right way to do this, but there are three critical inputs to your decision:

The first is market sizing, usually referred to as total addressable market (TAM). All things considered equal, we like big markets rather than small markets.

The second factor concerns distribution, usually referred to as go to market (GTM). Different markets may require different sales channels and go‐to‐market strategies.

The third factor is a (very rough) estimation of how long it will take, referred to as time to market (TTM).

Principles of Product Vision

10 key principles for coming up with an effective product vision.

  1. Start with why.
  2. Fall in love with the problem, not with the solution.
  3. Don't be afraid to think big with vision.
  4. Don't be afraid to disrupt yourselves because, if you don't, someone else will.
  5. The product vision needs to inspire. Remember that we need product teams of missionaries, not mercenaries.
  6. Determine and embrace relevant and meaningful trends.
  7. Skate to where the puck is heading, not to where it was.
  8. Be stubborn on vision but flexible on the details.
  9. So many teams give up on their product vision far too soon. This is usually called a vision pivot, but mostly it's a sign of a weak product organization.
  10. Realize that any product vision is a leap of faith. If you could truly validate a vision, then your vision probably isn't ambitious enough.

Evangelize continuously and relentlessly. There is no such thing as over‐communicating when it comes to explaining and selling the vision.

Principles of Product Strategy

Good strategies have these five principles in common:

  1. Focus on one target market or persona at a time. Don't try to please everyone in a single release.
  2. Product strategy needs to be aligned with business strategy. The vision is meant to inspire the organization, but the organization ultimately is there to come up with solutions that deliver on the business strategy.
  3. Product strategy needs to be aligned with sales and go‐to‐market strategy.
  4. Obsess over customers, not over competitors.
  5. Communicate the strategy across the organization.

Product Principles

Where the product vision describes the future you want to create, and the product strategy describes your path to achieving that vision, the product principles speak to the nature of the products you want to create.

As an example, early on at eBay we found we needed a product principle that spoke to the relationship between buyers and sellers. Most of the revenue came from sellers, so we had a strong incentive to find ways to please sellers, but we soon realized that the real reason sellers loved us was because we provided them with buyers. This realization led to a critical principle that stated, “In cases where the needs of the buyers and the sellers conflict, we will prioritize the needs of the buyer, because that's actually the most important thing we can do for sellers.”

George Patton quote I mentioned earlier: “Never tell people how to do things. Tell them what to do, and they will surprise you with their ingenuity.”

The OKR Technique

Objectives should be qualitative; key results need to be quantitative/measurable.

Key results should be a measure of business results, not output or tasks.

The rest of the company will use OKRs a bit differently, but for the product management, design, and technology organization, focus on the organization's objectives and the objectives for each product team,

Find a good cadence for your organization (typically, annually for an organization's objectives and quarterly for a team's objectives).

Keep the number of objectives and key results for the organization and for each team small (one to three objectives, with one to three key results each is typical).

It's critical that every product team track their active progress against their objectives (which is typically weekly).

The objectives do not need to cover every little thing the team does, but they should cover what the team needs to accomplish.

It's important that, one way or another, teams feel accountable to achieving their objectives. If they fail substantially, it's worth having a post‐mortem/retrospective

Agree as an organization on how you will be evaluating or scoring your key results.

What's important here is consistency across the organization,

It's common to define a score of 0 (on a scale from 0 to 1.0) if you essentially make no progress, 0.3 if you just did the bare minimum—what you know you can achieve, 0.7 if you've accomplished more than the minimum and have really done what you'd hoped you would achieve, and 1.0 if you've really surprised yourselves and others with a truly exceptional result,

Establish very clear and consistent ways to indicate when a key result is in reality a high‐integrity commitment

For most key results, you may be shooting for that 0.7 score. But for a high‐integrity commitment, these are special, and it's more binary.

Be very transparent (across the product and technology organization) on what objectives each product team is working on and their current progress.

Senior management (CEO and executive team) is responsible for the organization's objectives and key results.

Product Team Objectives

Imagine if the engineers were told to spend their time on re‐platforming, the designers on moving to a responsive design, and QA on retooling. While each of these may be worthy activities, the chances of solving the business problems that the cross‐functional teams were created to solve are not high.

If you deploy OKRs for your product organization, the key is to focus your OKRs at the product team level. This means don't let functional team or individual person OKRs confuse the issue.

The key is that the cascading of OKRs in a product organization needs to be up from the cross‐functional product teams to the company or business‐unit level.

Product Objectives @ Scale

Focused here on growth stage or enterprise organizations.

With larger organizations, product teams need more help. The first help they need is a very clear understanding of the organization‐level objectives.

Leadership (especially the head of product, head of technology, and head of design) will need to discuss the company objectives and which teams are best suited to pursue each objective.

Moreover, at scale, it is very common to have some significant number of product teams that are there in support of the other product teams. These are often called platform product teams, or shared services product teams.

Leadership will need to help coordinate the objectives for these teams and make sure we coordinate the dependencies and align the interests.

Once you have your objectives, there is a very critical reconciliation process in which the leadership team looks at the proposed key results from the product teams and identifies gaps

Lean on management to help connect the dots between teams.

Delivery managers play a key role in tracking and managing these dependencies and our commitments.

In many enterprise scale organizations, there are essentially multiple business units, and in this case, we would expect that there are corporate level OKRs,

When using OKRs at scale, there's a larger burden on leadership and management to ensure that the organization is truly aligned,

Product Evangelism

Product evangelism is, as Guy Kawasaki put it years ago, “selling the dream.”

Top‐10 pieces of advice for product managers to sell the dream:

  1. Use a prototype.
  2. Share the pain. Show the team the customer pain you are addressing.
  3. Share the vision.
  4. Share learnings generously. After every user test or customer visit, share your learnings—not just the things that went well, but share the problems, too.
  5. Share credit generously. Make sure the team views it as their product, not just your product.
  6. Learn how to give a great demo.
  7. Do your homework. Your team and your stakeholders will all be much more likely to follow you if they believe you know what you're talking about.
  8. Be genuinely excited.
  9. Absolutely be sincere, but let people see you're genuinely excited. Enthusiasm really is contagious.
  10. Learn to show some enthusiasm.

Spend time with your team. If you're not spending significant face time with your designer and every engineer on your team, then they can't see the enthusiasm in your eyes.

PART IV The Right Process

Product Discovery Overview

Two very significant challenges to tackle.

We need to learn fast, yet also release with confidence.

Principles of Product Discovery

The purpose of product discovery is to address these critical risks: Will the customer buy this, or choose to use it? (Value risk) Can the user figure out how to use it? (Usability risk) Can we build it? (Feasibility risk) Does this solution work for our business? (Business viability risk)

Set of core principles that drive how we work.

  1. We know we can't count on our customers (or our executives or stakeholders) to tell us what to build.
  2. Customers don't know what's possible,
  3. The most important thing is to establish compelling value.
  4. It's all hard, but the hardest part of all is creating the necessary value so that customers ultimately choose to buy or to use.
  5. As hard and important as the engineering is, coming up with a good user experience is usually even harder, and more critical to success.
  6. Functionality, design, and technology are inherently intertwined. We expect that many of our ideas won't work out, and the ones that do will require several iterations.

Marc Andreessen, “The most important thing is to know what you can't know,”

We must validate our ideas on real users and customers.

One of the most common traps in product is to believe that we can anticipate our customer's actual response

Our goal in discovery is to validate our ideas the fastest, cheapest way possible.

We need to validate the feasibility of our ideas during discovery, not after.

We need to validate the business viability of our ideas during discovery, not after.

It's about shared learning. One of the keys to having a team of missionaries

Discovery Techniques Overview

We need to tease out the risks and determine where it makes sense to focus our time.

  1. Discovery Planning Techniques
  2. Discovery Ideation Techniques
  3. Discovery Prototyping Techniques
  4. Discovery Testing Techniques
  5. Testing Feasibility
  6. Testing Usability
  7. Testing Value
  8. Testing Business Viability
  9. Transformation Techniques

Discovery Framing Techniques Overview

Two goals

  1. The first is to ensure the team is all on the same page in terms of clarity of purpose and alignment.
  2. The second purpose is to identify the big risks that will need to be tackled during the discovery work.

An opportunity assessment is designed for the vast majority of product work, which ranges from a simple optimization to a feature to a medium‐sized project.

A customer letter is designed for larger projects or initiatives that often have multiple goals and a more complicated desired outcome.

A startup canvas for those times you're creating an entirely new product line or a new business.

Opportunity Assessment Technique

Answer four key questions about the discovery work you are about to undertake:

  1. What business objective is this work intended to address?
  2. (Objective) How will you know if you've succeeded?
  3. (Key results) What problem will this solve for our customers?
  4. (Customer problem) What type of customer are we focused on?
  5. (Target market)

Customer Letter Technique

For smaller and more typically sized efforts, the opportunity assessment is usually sufficient.

But when embarking on a somewhat larger effort, there may in fact be multiple reasons, several customer problems to be solved, or business objectives to be tackled. To communicate the value effectively, it may take more than the four questions listed in the previous chapter.

How Amazon builds product, and one of them is referred to as the working backward process, where you start the effort with a pretend press release.

The idea is that the product manager frames the work ahead of the team by writing an imagined press release of what it would be like once this product launches.

The actual reader of this press release is the product team, related or impacted teams, and leadership.

Variation of this technique that was developed and refined at Nordstrom. The idea is that rather than communicate the benefits in a press release format, you describe them in the format of a customer letter written from the hypothetical perspective of one of your product's well‐defined user or customer personas.

The letter—sent to the CEO from a very happy and impressed customer—explains why he or she is so happy and grateful for the new product or redesign. The customer describes how it has changed or improved his or her life. The letter also includes an imagined congratulatory response from the CEO to the product team explaining how this has helped the business.

Startup Canvas Technique

An Introduction to Lean Canvas

This is an early stage startup, where you are trying to figure out a new product that can power a new business, or, for those that work at an enterprise size company, when you're asked to tackle an all‐new business opportunity for the company.

In other words, you're not being asked to improve an existing product, you're being asked to invent an entirely new product.

Story Map Technique

These are two‐dimensional maps, in which major user activities are arrayed along the horizontal dimension, loosely ordered by time from left to right.

Along the vertical dimension, we have a progressive level of detail. As we flesh out each major activity into sets of user tasks, we add stories for each of those tasks. The critical tasks are higher vertically than the optional tasks.

We can use this story map to frame our prototypes, and then as we get feedback on our prototypes and learn how people interact with our product ideas, we can easily update the story map to serve as a living reflection

Customer Discovery Program Technique

Without strong products, our marketing programs require customer acquisition costs that are too high; our sales organization is forced to get “creative,” which drives up cost of sales, lengthens the sales cycle, and puts downward pressure on price; and our customer success organization is forced to take it on the chin every day with frustrated customers.

The Power of Reference Customers

This is a real customer (not friends or family), who is running your product in production (not a trial or prototype), who has paid real money for the product (it wasn't given away to entice them to use it), and, most important, who is willing to tell others how much they love your product (voluntarily and sincerely).

Consider it the single best leading indicator of future product success.

Single Target Market

We are looking to develop six reference customers in our specific target market or segment, so, the idea is to find six similar customers. If you end up targeting two or three customers from two or three different markets, this program will not give you the focus you want and need.

Recruiting the Prospective Reference Customers

We need them to have people and time willing to work closely with us. They need to be willing to spend time with the product team, testing out early prototypes and helping the team ensure the product works well for them. If possible, we would like them to be well‐recognized marquee names, because that will be of the most value to the sales and marketing staff.

Platform/API Products

For developer products, the program is very much like the one for businesses, but the main difference is that we work with the development teams (engineers and product managers) that will use our APIs to get them successfully using our product.

The result of the program is a set of reference applications rather than reference customers.

Customer‐Enabling Tools

For customer‐enabling tools, such as a new dashboard for your customer service agents, we pick six to eight well‐respected, influential internal users/employees—

Consumer Products

Rather than focusing on six businesses to work closely with (where we have access to many different users at each customer), we instead focus on a somewhat larger number of consumers (on the order of 10–50) that we engage with to get them to the point that they are loving our product.

Customer Interviews

Here's what I'm always trying to understand:

Here are some tips for getting the most out of these learning opportunities:

Concierge Test Technique

The idea is that we do the customer's job for them—manually and personally. Just as if you went to a hotel concierge

With this technique, you become the concierge. You do what the user or customer needs done for them. You may have to ask them to train you first, but you are in their shoes doing the tasks they would do.

A concierge test requires going out to the actual users and customers and asking them to show you how they work so that you can learn how to do their job,

The Power of Customer Misbehavior

Historically, the two main approaches used by good teams to come up with product opportunities have been:

  1. Try to assess the market opportunities and pick potentially lucrative areas where significant pain exists.
  2. Look at what the technology or data enables—what's just now possible—and match that up with the significant pain.

Third alternative is to allow, and even encourage, our customers to use our products to solve problems other than what we planned for and officially support.

Some product people can get upset when they find customers using their products for unintended use cases. This concern is usually tied to the support obligations. I'm suggesting, however, that this special case can be very strategic and well worth the investment to support.

I have been a long‐time fan of public APIs as a part of a company's product strategy. I consider developers to be one of the consistently best sources of truly innovative product

Hack Days

In an undirected hack day, people can explore whatever product ideas they like, so long as it's at least loosely related to the mission of the company.

In a directed hack day, there's a customer problem (for example, something is really difficult to learn and use, or it takes too long to do) or business objective we've been assigned (for example, “Reduce the customer churn rate” or “Increase customer lifetime value”),

There are two major benefits to these directed hack days.

Principles of Prototypes

Five key principles behind their use.

  1. The overarching purpose of any form of prototype is to learn something at a much lower cost in terms of time and effort than building out a product.
  2. Realize that one of the key benefits of any form of prototype is to force you to think through a problem at a substantially deeper level than if we just talk about it or write something down.
  3. Similarly, a prototype is also a powerful tool for team collaboration.
  4. There are many different possible levels of fidelity for a prototype.
  5. The primary purpose of a prototype is to tackle one or more product risks (value, usability, feasibility, or viability) in discovery; however, in many cases, the prototype goes on to provide a second benefit, which is to communicate to the engineers and the broader organization what needs to be built.

Feasibility Prototype Technique

There are several situations wherein your engineers may identify a significant feasibility risk involved in solving a particular problem

Common examples include:

A feasibility prototype is a long way from a commercially shippable product—the idea is to write just enough code to mitigate the feasibility risk.

Most of the time the feasibility prototype is intended to be throwaway code—

Simulation. Smoke and mirrors. It's all a façade. There is nothing behind the curtain.

A low‐fidelity user prototype doesn't look real—it is essentially an interactive wireframe.

High‐fidelity user prototype is still a simulation; however, now it looks and feels very real. In fact, with many good high‐fidelity user prototypes, you need to look close to see that it's not real.

The big limitation of a user prototype is that it's not good for proving anything—like whether or not your product will sell.

Live‐Data Prototype Technique

Some of my favorite examples of this are when applying game dynamics, search result relevance, many social features, and product funnel work.

The live‐data prototype is substantially smaller than the eventual product, and the bar is dramatically lower in terms of quality, performance, and functionality. It needs to run well enough to collect data for some very specific use cases, and that's about it.

There are two big limitations

  1. First, this is code, so engineers must create the live‐data prototype, not your designers.
  2. Second, this is not a commercially shippable product, it's not ready for primetime, and you can't run a business on it.

Hybrid Prototype Technique

One of my favorite examples of a hybrid prototype—and an exceptionally powerful tool for learning quickly in product discovery—is today often referred to as a Wizard of Oz prototype. A Wizard of Oz prototype combines the front‐end user experience of a high‐fidelity user prototype but with an actual person behind the scenes performing manually what would ultimately be handled by automation.

Discovery Testing Techniques Overview

Four types of questions we're trying to answer during discovery:

  1. Will the user or customer choose to use or buy this? (Value)
  2. Can the user figure out how to use this? (Usability)
  3. Can we build this? (Feasibility)
  4. Is this solution viable for our business? (Business viability)

Testing Usability

We do usability testing in discovery—using prototypes, before we build the product—and not at the end,

Usability testing with a high‐fidelity user prototype.

Good product managers know they will get the product wrong initially and that nobody gets it right the first time.

We want to learn whether the user or customer really has the problems we think they have, and how they solve those problems today, and what it would take for them to switch.

Keep your users in use mode and out of critique mode. What matters is whether users can easily do the tasks they need to do.

Three important cases you're looking for:

  1. the user got through the task with no problem at all and no help;
  2. the user struggled and moaned a bit, but he eventually got through it; or
  3. he got so frustrated he gave up.

Avoid giving any help or leading the witness in any way.

After each test subject, or after each set of tests, someone—usually either the product manager or the designer—writes up a short summary e‐mail of key learnings and sends it out to the product team. But forget big reports that take a long time to write, are seldom read, and are obsolete by the time they're delivered

Testing Value

Just because someone can use our product doesn't mean they will choose to use our product.

The customer must perceive your product to be substantially better to motivate them to buy your product and then wade through the pain and obstacles of migrating from their old solution.

Demand Testing Techniques

One of the biggest possible wastes of time and effort, and the reason for countless failed startups, is when a team designs and builds a product

When they finally release the product, they find that people won't buy it.

Fake door demand test.

When the user clicks that button, rather than taking the user to the new feature, it instead takes the user to a special page that explains that you are studying the possibility of adding this new feature, and you are seeking customers to talk to

Landing page demand test.

User sees a page that explains that you are studying the possibility of adding this new offering, and you'd like to talk with them

Protect your revenue and brand, and protect your employees and customers.

Qualitative Value Testing Techniques

Quantitative testing tells us what's happening (or not), but it can't tell us why,

I argue that qualitative testing of your product ideas with real users and customers is probably the single most important discovery activity for you and your product team.

Generally begin the user test with a short user interview where we try to make sure our user has the problems we think she has, how she solves these problems today, and what it would take for her to switch

Value test is always preceded by a usability test.

Test to see whether the user can figure out how to operate our product.

The magic happens when one of your engineers is right there watching the qualitative testing with you.

Can ask people if they will sign a “non‐binding letter of intent to buy” which is a good indicator that people are serious.

Quantitative Value Testing Techniques

Qualitative testing is all about fast learning and big insights, quantitative techniques are all about collecting evidence.

Collect enough data that we have statistically significant results

The gold standard for this type of testing is an A/B test.

It's also important for tech product managers to have a broad understanding of the types of analytics that are important to your product.

I still encounter too many product teams that either aren't instrumenting their product to collect analytics, or they do it at such a minor level that they don't know if and how their product is being used.

Testing Feasibility

Answer several related questions:

If you put an engineer on the spot, without time to investigate and consider, you are very likely to get a conservative answer, partly designed to make you go away.

Testing Business Viability

The solution must also work for your business.

If what you are proposing to build could impact the sales channel, the major marketing programs, or is potentially outside of the brand promise (the range of things your customers expect from your company), then you'll want to discuss this with marketing and show them prototypes of what you are proposing before you consider building anything.

A direct sales channel is very expensive, and this requires a high‐value product and price point.

If what you are proposing would represent a departure from what the sales channel has proven their ability to sell, sit down with the sales leadership and show them what you are proposing before you build anything,

You need to understand what your company's customer success strategy is, and you need to ensure that your products are aligned with that strategy.

If there are cost issues involved, sitting down with someone in finance and modeling the costs will be critical

Privacy concerns, compliance concerns, intellectual property, and competitive issues are all common constraints related to legal.

Most businesses have some number of close business relationships with partners of various types, usually with a contract

Sometimes these agreements can cripple your company's ability to compete. Sometimes they are a huge win.

If you are proposing anything at all remotely related to security, you will probably want to bring your tech lead and sit down with the security leadership early

With every company there is some CEO or general manager that is responsible for the business unit.

A user test is when we test our product ideas on real users and customers.

A product demo is when you sell your product to prospective users and customers, or evangelize your product

A walkthrough is when you show your prototype to a stakeholder and you want to make sure they see and note absolutely everything that might be a concern.

Many inexperienced product managers do a walkthrough with a prospective customer when they should have prepared a product demo.

Be sure to be clear about whether you're doing a user test, a product demo, or a walkthrough.

Netflix

In 1999, a then very young Netflix

There wasn't much of a reason to rent DVDs via the U.S. Postal Service when you could just stop by the local Blockbuster store on the way home from work.

One of many tests they tried was to move to a subscription service.

The good news was that, yes, this approach really did appeal to people.

The bad news is that the team created some real problems

Netflix customers wanted to rent mostly newly released feature films; yet, these were much more expensive

Needed to somehow get customers to want and ask for a blend of expensive and less expensive titles.

This is where Netflix's queue, ratings system, and recommendation engine all came from.

Between working with the co‐founders on the strategy, validating concepts with the users, assessing the analytics, driving features and functionality with the team—and working with finance on the new business model, marketing on acquisition, and the warehouse on fulfillment—you can imagine the workload

The team got the new service up and running and used this to power and grow their business for another seven years, until they disrupted themselves again by moving aggressively to the streaming model.

When they were struggling for cash early on, they offered to sell themselves to Blockbuster for $50 million, and Blockbuster turned them down.

It's hard because the changes are so often cultural.

Discovery Sprint Technique

A discovery sprint is a one‐week time box of product discovery work, designed to tackle a substantial problem or risk your product team is facing.

Always end your week by validating your potential solution with real users and customers.

The GV team decided to share their learnings in a book. The book is titled, Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days, by Jake Knapp, John Zeratsky, and Braden Kowitz.

Pilot Team Technique

Pilot teams allow the roll out of change to a limited part of the organization before implementing it more broadly.

Some people in your organization love change, some want to see someone else use it successfully first, some need more time to digest changes, and a few hate change and will only change if they're forced to do it.

Weaning an Organization Off Roadmaps

The goal is that over time, the organization moves its focus from specific features launching on specific dates to business results.

If the feature you're working on is to add PayPal as a payment method, and the reason is to increase conversion, then be sure to always show the current conversion rate and the result you're hoping to achieve.

If the impact was good, celebrate it. If the impact was not what was hoped, then emphasize to everyone that, while you did ship the feature, the result was not successful.

Two big reasons why stakeholders especially are so attracted to roadmaps:

  1. They want some visibility into what you are working on and assurance that you are working on the most important items.
  2. They want to be able to plan the business and need to know when critical things will happen.

Teams work on the prioritized business objectives determined by the leaders; we share our key results transparently; and we commit to high‐integrity commitments when critical delivery dates are needed.

It is all too easy to institute processes that govern how you produce products that can bring innovation to a grinding halt.

Agile methods are generally very conducive to consistent innovation.

Yet there are several process consultancies that specialize in “Agile at Scale,” which introduce methods and structures intended to scale to large numbers of engineers, yet which absolutely destroy any hope of innovation.

Managing Stakeholders

One practical test of whether a person is considered a stakeholder is whether or not they have veto power, or can otherwise prevent your work from launching.

This group of people typically includes: The executive team (CEO and leaders of marketing, sales, and technology) Business partners (to make sure the product and the business are aligned) Finance (to make sure the product fits within the financial parameters and model of the company) Legal (to make sure that what you propose is defensible) Compliance (to make sure what you propose complies with any relevant standards or policies) Business development (to make sure what you propose does not violate any existing contracts or relationships)

The product manager has the responsibility to understand the considerations and constraints of the various stakeholders, and to bring this knowledge into the product team.

The main way we demonstrate this knowledge to the organization is by sharing very openly what we learn.

The key technique is to spend one‐on‐one time with the key stakeholders. Sit down with them and listen. Explain that the better you understand their constraints, the better your solutions will be. Ask lots of questions. Be open and transparent.

Commit to previewing your solutions during discovery with the key stakeholders before you put this work on the product backlog.

Move the discussion from opinions to data. Share what you're learning very openly.

Presentations are notoriously terrible for testing business viability. The reason is that they are far too ambiguous.

In contrast, high‐fidelity user prototypes are ideal

A group setting is not the forum for designing strong products. It results in design by committee, which yields mediocre results at best.

Communicating Product Learnings

Sharing what we learn in a startup happens naturally because the product team and the company are pretty much the same thing. However, as companies scale, this becomes substantially more difficult;

A technique I love for helping with this is for the head of product, at a company all‐hands or similar meeting every week or two, to take 15 to 30 minutes to highlight what has been learned

PART V The Right Culture

Good Product Team/Bad Product Team

Goot teams

Top Reasons for Loss of Innovation

Organizations that lose the ability to innovate at scale are inevitably missing one or more of the following attributes:

Book Club Discussion

Top Reasons for Loss of Velocity

Establishing a Strong Product Culture

I think of product culture along two dimensions.

  1. The first dimension is whether the company can consistently innovate to come up with valuable solutions for their customers.
  2. The second dimension is execution. It doesn't matter how great the ideas are if you can't get a productized, shippable version delivered to your customers.

What does it really mean to have a strong innovation culture?

What does it really mean to have a strong execution culture?