product innovation blog

You Are What You Measure – Why Metrics Are Important in Product Development

Share

Why Metrics

We are often asked what kinds of metrics should be used for product development (PD).  Metrics are frequently seen as a waste of time by the project team because they don’t seem to drive improvements in product development. So why do we use metrics?  We use them because, when properly defined, they help identify both areas that work well and areas for improvement.[i]

What Are Metrics

Metrics are simply a set of numbers that give information about a particular process or activity.[ii]  How the measurements are used is what brings the value to collecting metrics.  A non-product development example helps to illustrate what is meant by the analysis.  Say the most recent lot of parts from your production line are all on one side of the target value.  They are still in spec.  Do you have a problem or not?  This is where the analysis comes in.  Look at the 2 plots below.

You Are What You Measure – Why Metrics Are Important in Product Development

From these 2 plots you can see that the second one shows a need to take action, now.  Simply relying on ‘everything is in spec’ wouldn’t cause you to take action until after the parts start failing.  In evaluating new software ideas, the use of A/B testing has become widespread.  Being able to identify problems or successes early and respond them is the biggest value to collecting and evaluating metrics.

It is also important to note that people also behave according to how they are measured.  Hauser and Katz summarized pitfalls that lead to counter-productive metrics.[iii]  Therefore, when you select metrics, you must consider the result you want to attain, plus the potential unintended consequences.

Types of Metrics

There are many ways to divide metrics.  For this article, we will use two main groups reflecting the timeframe for use.  Example of each group of metrics are shown below.  Within each of these groups, there may be qualitative and quantitative measures.  But you also need to look at how the measurement can be used to drive changes.

The first group of metrics are long term, strategic, or reactive.  These metrics are designed to be used over an extended time.  As such, the actual metric needs to stay unchanged so that the data can be compared from period to period or over time.  Because of the timeframe needed to collect information, corrections to the process being measured is reactive.

The second group of metrics are short term, actionable, or predictive.  These have a short life and are used to address a particular project or specific problem.  The metric generally is used for a short time, until a particular issue has been resolved or it may be used to provide predictive information to allow a project to respond early enough to prevent problems.

When selecting metrics, start with a few to see if they add value.  Too many metrics may drive conflicting behavior and place significant overhead on the organization to collect the information relative to the value obtained.  You may also find that some metrics are better collected through a proxy metric (e.g., customer satisfaction through complaint data).

Metrics Examples [iv][v][vi]

Below are some examples of product development metrics.

Long term, strategic, or reactive

  • Percent revenue from products developed over last x years compared to % revenue projected from products over that same timeframe
  • Product development cycle time
  • Revenue earned compared to forecast at launch: check at intervals that match your business. Usually at years 1, 2, and 3
  • Number patents issued over past year
  • Percent team turnover

Short term, actionable, or predictive

  • Number scope changes after initial approval
  • Percent project complete compared to percent project planned completion (several methods including EVA or buffer charts (from Theory of Constraints))
  • Time to market versus plan
  • Number active projects fully staffed
  • Queue sizes
  • Decision cycle time

Closing Thoughts

Selecting the right metrics is important for successful product development because they can help you focus on driving improvement into your PD process.  Keep the number of metrics manageable and try different ones when the old ones no longer are effective.

 

—————————————————————————————

[i] Kerzner, H. (2011). Project Management Metrics, KPIs, and Dashboards: A Guide to Measuring and Monitoring Project Performance, Hoboken, New Jersey, John Wiley & Sons[ii] Metrics (n.d.). in Oxford Dictionary Online. Retrieved from http://dictionary.cambridge.org/us/dictionary/english/metrics

[iii] Hauser, John R. and Gerry Katz (1998), “Metrics: You Are What You Measure!.” European Management Journal, 16, 5, (October), 516-528.

[iv] Anderson, A. (2015) NPD Metrics as a Continuous Improvement Tool [PowerPoint slides].  Retrieved from http://www.pdma.org/p/do/sd/topic=724&sid=842

[v] Chan, A. (2005). Using an Effective Metrics Program to Support Business Objectives. In K. B. Kahn (ed.), The PDMA Handbook of New Product Development, Second Edition, (pp. 445-454). Hoboken, New Jersey, John Wiley & Sons

[vi] Reinertsen, D. G. (2009). The Principles of Product Development Flow: Second Generation Lean Product Development. Redondo Beach, California, Celeritas Publishing

© 2005-present Strategy 2 Market®, Inc. All rights reserved. – 53 W. Jackson Blvd. Suite 360, Chicago, Illinois
image_pdfimage_print

One comment on “You Are What You Measure – Why Metrics Are Important in Product Development

  • Thank you for sharing your insights on Product Development metrics. They are notoriously challenging to make meaningful and to drive the desired action.

    Management often does not know how to set or what to do with Product Development metrics. They are used to dealing with metrics for manufacturing where, for our company, we assemble a product in 20 minutes or less and ship hundreds per day from each factory. In contrast, our Product Development projects typically take 20 months to complete and our $1B+ company releases about one new product per month.

    Your comment about not having many Product Development metrics is also spot on. Having just a few lets everyone know what is important and relates to your recent article on Battles Between Marketing and Engineering in that the simple act of having one or two clear metrics drives behavior changes and channels conflict to productive areas rather than counter-productive ones.

Comments are closed.