Показаны сообщения с ярлыком metrics. Показать все сообщения
Показаны сообщения с ярлыком metrics. Показать все сообщения

воскресенье, 17 декабря 2023 г.

The Hewlett-Packard Return Map

 


The Return Map is intended to be used by all of the functional managers on the new product introduction business team. It is basically a two dimensional graph displaying time and money on the x and y axes respectively.

 

Time is drawn as a linear scale while money is more effectively drawn on a logarithmic scale, because for successful products the difference between sales and investment costs will be greater than 100:1.

 

It is important to remember that money amount are shown rising cumulatively.

The time axis is divided into three parts:

  • Investigation,
  • Development, and
  • Manufacturing and Sales.

Investigation

The purpose of Investigation is to determine the desired product features, the product's cost and price, the feasibility of the proposed technologies, and the plan for product development and introduction. At this point all numbers are estimates. Investigation is usually the responsibility of a small team and requires a relatively modest investment. At the end of Investigation, the company commits to develop a product with specific features using agreed-upon technologies.

 

Development

The development phase is usually the primary domain of R&D; in consultation with manufacturing; its purpose is to determine how to produce the product at the desired price.

 

Manufacturing Release

The formal end of the development phase is Manufacturing Release (MR) - that is, when the company commits to manufacture and sell the product. When the product is ready to be manufactured and shipped to customers, sales become a reality and manufacturing, marketing, sales costs and profits are finally more than estimates.

 

New Metrics

The map tracks - in money and months - R&D; and manufacturing investment, sales, and profit. At the same time, it provides the context for new metrics:

  • Break-Even-Time,
  • Time-to-Market,
  • Break-Even-After-Release, and
  • the Return Factor.

References

  • House. C. H. and Price.R. L., "The Return Map: Tracking Product Teams, Harvard Business Review , January-February 1991, pp 92-101.

The Return Map: Tracking Product Teams


by 

The Return Map Captures Both Money and Time 1

The purpose of Investigation is to determine the desired product features, the product’s cost and price, the feasibility of the proposed technologies, and the plan for product development and introduction. At this point, all numbers are estimates. Investigation is usually the responsibility of a small team and requires a relatively modest investment. Obviously, Marketing and R&D should collaborate to determine what features customers want and how they could be provided. At the end of Investigation, the company commits to develop a product with specific features using agreed-upon technologies.

The Development phase is usually the primary domain of R&D in consultation with manufacturing; its purpose is to determine how to produce the product at the desired price. Challenges during this phase include changing product features, concurrent design of the product and the manufacturing processes, and, often, the problems associated with doing something that has not been done before.

The formal end of the Development phase is Manufacturing Release (MR)—that is, when the company commits to manufacture and sell the product. When the product is ready to be manufactured and shipped to customers, sales become a reality and manufacturing, marketing, sales costs, and profits are finally more than estimates. The transitions between these phases or the project checkpoints are key times for the Return Map.

Perhaps the best way to grasp how the Return Map evolves from these early stages is to examine a map for a completed project, where all the variables are known—in this case, the map for a recent Hewlett-Packard pocket calculator (see Exhibit 2).

First, the Team Plots Estimates 2

Investigation took 4 months and cost about $400,000; Development required 12 months and $4.5 million, with many new manufacturing process designs for higher quality and higher production volumes. Hence, the total product development effort from beginning to manufacturing and sales release took 16 months and cost $4.9 million (see the investment line on Exhibit 2). Once the product was released, sales increased consistently for 5 months and then increased at a slightly faster pace during the next 9 months. Sales volume for the first year was $56 million and for the second year was $145 million (see the sales line on Exhibit 2—and remember, the differences are greater than they appear owing to the log scale). Cumulative sales volume gives a sense of how quickly and effectively the product was introduced and sold.

In the first year, net profits of $2.2 million were less than expected due to the sales volume lag and the resulting increase in cost per unit. But profits (see the profit line in Exhibit 2) increased significantly in the second year and passed through the investment line about 16 months after Manufacturing Release. For the second year, net profits reached $13 million. Profits are the best indicator of the contribution the product made to the customers since they reflect both the total volume of sales and the price the product can command in the marketplace.

So the critical lines that are systematically plotted on the Return Map include new product investment dollars, sales dollars, and profit dollars. Each of these is plotted as both time and money, with money in total cumulative dollars. Now that we have the basic elements of the map, we can focus on some novel metrics.

New Metrics

The map tracks—in dollars and months—R&D and manufacturing investment, sales, and profit. At the same time, it provides the context for new metrics: Break-Even-Time, Time-to-Market, Break-Even-After-Release, and the Return Factor. These four metrics (see Exhibit 3, plotted for the pocket calculator) become the focus of management reviews, functional performance discussions, learning, and most important, they are the basis for judging overall product success.

Key Metrics Complete the Picture 3

Break-Even-Time, or BET, is the key metric. It is defined as the time from the start of investigation until product profits equal the investment in development. In its simplest form, BET is a measure of the total time until the break-even point on the original investment; for the pocket calculator, BET is 32 months. BET is the one best measure for the success of the whole product development effort because it conveys a sense of urgency about time; it shows the race to generate sufficient profit to justify the product in the first place.

Time-to-Market, or TM, is the total development time from the start of the Development phase to Manufacturing Release. For the calculator project, TM was 12 months. This time and its associated costs are determined primarily by R&D efficiency and productivity. TM makes visible the major check-points of Investigation and Development. It is obviously the most important R&D measure.

Break-Even-After-Release, or BEAR, is the time from Manufacturing Release until the project investment costs are recovered in product profit. BEAR for the calculator was 16 months. This measure focuses on how efficiently the product was transferred to marketing and manufacturing and how effectively it was introduced to the marketplace. Just as TM is considered the most important R&D metric, BEAR is the most important measure for marketing and manufacturing.

Finally, the Return Factor, or RF, is a calculation of profit dollars divided by investment dollars at a specific point in time after a product has moved into manufacturing and sales. In the case of the pocket calculator, RF after one year was .45 (that is, cumulative profit of $2.2 million divided by total investment of $4.9 million) and was 3.1 (a profit of $15.2 million divided by $4.9 million) after two years. The RF gives an indication of the total return on the investment without taking into account how long it took to achieve that return.

The effectiveness of the Return Map hinges on the involvement of all three major functional areas in the development and introduction of new products. The map captures the link between the development team and the rest of the company and the customer. If the product does not sell and make money, for whatever reasons, the product development efforts were wasted. The team is accountable for designing and building products that the customers want, doing it in a timely manner, and effectively transferring the products to the rest of the company.

Making the Most of the Return Map

We have argued that an interfunctional team uses the Return Map most appropriately during the Investigation phase by generating estimates for a final map, including investment, sales, and profit. These initial estimates or forecasts are a “stake in the ground” for the team and will be used for comparison and learning throughout the project. By focusing on the accuracy of the forecasts, marketing, R&D, and manufacturing are forced to examine problems as a team; all three functions are thus sharing the burden of precision.

Too often, the whole burden during the initial phase of a project is placed on the R&D team—to generate schedules, functionality, and cost goals. But the Return Map requires accurate sales forecasts, which forces market researchers to get better and better at competitive analysis, customer understanding, and market development. Similarly, manufacturing involvement is essential for forecasting cost and schedule goals; manufacturing engineers should never be left to develop the manufacturing process after the design is set.

We cannot emphasize enough that missed forecasts generated for the Return Map in the Investigation phase should be viewed as valuable information, comparable with defect rates in manufacturing—deviations from increasingly knowable standards, proof that either the process is out of whack or the means for setting the standard are. The Return Map can be used to provide a visual perspective on sales forecasts and expected profits given any number of hypothetical scenarios. What if the forecasts varied by 20%, over or under? What are the implications of reaching mature sales six months earlier than expected? The Return Map allows for a kind of graphic sensitivity analysis of product development and introduction decisions; it sets the stage for further, indeed, continual investigation.

By no means should the Return Map be used by management to punish people whose forecasts prove inaccurate. Estimates are a team responsibility or at least a functional one—no individual should be held responsible for generating the information on which they are based. Moreover, if functions are penalized for being late, they will simply learn to estimate conservatively, building slack into a discipline whose very purpose is the elimination of slack.

Consider the estimated Return Map for a proposed ultrasound machine (Exhibit 4). The Investigation phase is planned to last 5 months and cost $500,000, TM is estimated to be 9 months and cost $2.3 million, while anticipated BET is 18 months. Mature sales volume is expected to be 300 units a month, or $16 million, and mature profits are expected to be $2 million per month. These estimates were made by a project team that had developed two other ultra-sound products and were quite knowledgeable about the medical instrument business. They wanted to take more time in the Investigation phase to completely understand the desired product features in order to move more quickly during the Development phase.

During Investigation, the Ultrasound Team Makes Early Estimates 4

During the Development phase, that is, once product features and the customer requirements are understood, TM becomes vital and other concerns fade. The Return Map implicitly stresses execution now. The faster a product is introduced into a competitive marketplace, the longer potential life it will have—hence the greater its return. TM emphasizes the need to respond to market windows and competitive pressures.

It is difficult to keep project goals focused during development. Creeping features, management redirection, and “new” marketing data all push the project to change things in midstream, much to the detriment of engineering productivity and TM. The Return Map can help put this new information into perspective and help team members analyze the impact of changes on the entire project. For example, how much will new features increase sales and profits? If adding the features delays the introduction of the product, how will that affect sales and profit?

For example, two months into the Development phase of the ultrasound machine, Hewlett-Packard labs had a breakthrough in ultrasound technology that would enable the machine to offer clearer images. Should the project incorporate this new technology or proceed as planned? The project team determined that customers would value the features and they would result in more sales, though the new technology would be more expensive to produce. But would the changes be worth the extra expense?

The original Return Map was updated with a new set of assumptions (see Exhibit 5). Development costs would increase by 40% to $3.2 million, the TM would be extended by at least 4 months, but the sales could increase by 50% and net profits could increase by 30%. The Return Map demonstrated that the BET for the project would extend to 22 months, the BEAR would remain the same, but the RF would be reduced slightly.

During Development, New Technology Calls for Revised Estimates 5

What on the surface seemed like a great idea proved not to increase significantly the economic return. In the end, the team decided to incorporate the new technology anyway, but in order to capture a greater market share, not to make more money. The team went into the changes with its eyes open; it made a strategic decision, not one driven by optimistic numbers.

Once the team gets beyond Investigation, the Development phase, represented by TM, can itself be segmented into subphases and submetrics that provide greater understanding and accuracy and more effective management. Teams that compare projects over time become more and more sophisticated about the development processes underlying TM forecasts. Within Hewlett-Packard, for instance, the time required for printed circuit board turnaround emerged as something of a bottleneck for many projects. So the company developed streamlined processes for printed circuit board development and eventually simulation tools that reduced the need for board prototypes.

Another HP study showed that company managers were much more effective at predicting total engineering months than total calendar months (that is, effort rather than time). The big mistake here seemed to be a tendency on the company’s part to try to do too many projects with the available engineers—resulting in understaffed projects. Once management focused on staffing projects adequately, the company experienced a significant reduction in this kind of forecast error.

Beyond Manufacturing Release

The Manufacturing Release, or MR, meeting is perhaps the single most important built-in checkpoint in the system. It is structured to allow the management team to focus on the original goals of the entire development effort and to compare the goals at the project’s inception with the new estimates based on the realities of the Development phase. The team can now analyze the adjustments required by the marketing and manufacturing estimates, in consequence of any elapsed time, schedule slippages, or the changed competitive and economic scene.

Consider the updated Return Map for the ultra-sound project showing the estimates that were made during the Development phase and the real development cost and TM (see Exhibit 6). The project took two extra months to complete and cost $700,000 more than estimated. At this time, the estimates for sales and profits are lower than Development phase estimates. The important point for team members to understand, obviously, is the effect of a slide in TM on the other measures, and they should take corrective action in the future to avoid delays. A series of MR meetings, supported by documentation, will sharpen any team’s estimating and development skills.

At Manufacturing Release, the Team Sees What Actually Happened 6

During the Manufacturing and Sales phase, the emphasis shifts from estimates to real data for sales and profits. This is the moment of truth for the development team. As sales and profits vary from the forecast, everyone has the right—and responsibility—to ask why. One HP study tracked sales forecasts for 16 products over a two-year period. The targets were set rather broadly—a 20% deviation either way would be acceptable—on the assumption that manufacturing and sales could adjust successfully to either surging or weakening demand within that range. Nevertheless, only 12% of the forecasts fell into the acceptable range.

From our experience, this is the time when the project team gets the most constructive criticism, insight, and enthusiasm to do things right, now and the next time. At the MR stage, analysis, thoughtfulness, and responsiveness are vital. We have seen products falter because of such varied problems as unreliable performance, an unprepared sales force, or an inability to manufacture the product in appropriate volumes—all problems that should be correctable, even at this late date. Problems such as inappropriate features, high costs, and poor designs, however, will have to wait for the next generation.

Incidentally, one important lesson we have learned is the need to keep a nucleus of the project team together for at least six months after introduction. Team members should be available to help smooth the transition to full production and sales, and the company’s next generation of products will benefit greatly from the team’s collective learning.

In addition to helping manage and analyze individual products, the Return Map can be used for families of products, programs, and major systems. As companies establish a market presence with a product, it must be buttressed with corresponding products based on customer requirements and the next appropriate technologies. A complete program strategy for an important market usually embraces products from three overlapping generations, which we call a “strategic cycle.”

A family of products can lead to overall success, even if some of the products do not reach standard return goals and are not seen as successful. The success of major new programs may not be obvious until the second generation is produced: it may take many years before success or failure is completely determined.

The table illustrates the cumulative data for an entire product family, divided into three generations, with the second generation divided into five major products (A, B, C, D, E).

Three Generations of One Product Family

A quick scan of the second generation of products suggests that productivity could have been improved by doing only those products that end up with a low BET, low BEAR, and a high RF. Unfortunately, this requires more luck than foresight. One product in the series (product D) made much of the economic difference, but it is not possible to establish a full program with only one product. In fact, the success of product D was almost entirely dependent on the investment made in the technology for product C (note the long Time-to-Market) and the market understanding gained from product B.

To have a winning and sustained market presence usually requires at least three generations of products. Frequently, one of those generations will develop a new and significant technology, while the next generation will exploit that technology by means of rapid product development cycles—products tailored to specific markets.

Easier Said Than Done

The Return Map, along with its BET, TM, BEAR, and RF measures, provides a useful indicator of the effectiveness of new product development and introduction. It provides a general management tool to track the development process and to take corrective action in real time. But while the map is simple, the difficulties of using it well are great, time-consuming, and require significant commitment. The first challenge is to get the forecast data out and to track the actual costs, sales, and profits against those forecasts. Unfortunately, most development teams will have to pull this data manually from the period expense reports, since most companies track costs, sales, and profits on a period rather than a project basis. If development teams can prove project data useful, though, accounting departments may start tracking numbers for major projects and not only aggregate numbers month by month. (HP is now overhauling its cost accounting systems to provide project as well as period data.)

Another challenge is to get functional managers to work together toward common goals and to share openly the subset measures that govern their function’s contribution to BET, TM, and BEAR. The map exposes each function’s weaknesses insofar as each function’s performance is clearly measurable against its own predictions—predictions upon which important project decisions were based. Again, if there is going to be open sharing of data, the Return Map should not be used to penalize people for their forecasts. The race to market is a concerted effort that requires enthusiasm, not fear and apprehension. The judgments of the marketplace are generally all the correction a talented team requires.

The Return Map provides a visible superordinate goal for all the functions of the team and, in graphically representing the common task, helps them collaborate. No graph can substitute for judgment and experience—yet there is no substitute either for basing judgment on an accurate picture of experience.

https://hbr.org/1991/01/the-return-map-tracking-product-teams

понедельник, 28 августа 2023 г.

Performance Metrics: Which one to choose?

 

Introduction

Anyone running a business nowadays must be confused by the number of metrics that are promoted as the Holy Grail. To some the Net Promoter Score is the ultimate question. But what about good old customer satisfaction scores; are they redundant? In 2010 the Harvard Business Review published an article promoting a question that asked about the amount of effort required to do business with a company – and the Customer Effort Score was born. And then there is the Net Value Score – what is that all about?


Figure 1

Why Do We Need Metrics?

It is worth stepping back for a minute and considering why we need metrics at all to run our businesses. Metrics are like the dials and instruments in our cars. Imagine for a moment that we were forced into a situation where our car has only three dials on the dashboard; which dials would we want? Certainly it would be important to have an odometer for otherwise we wouldn’t know how far we have travelled and therefore we wouldn’t know how far we are from our destination. The odometer is the equivalent of our sales figures, which tell us what we have achieved over a period. Secondly, we would need a speedometer for otherwise we wouldn’t be able to calculate how long it will take us to arrive at our destination. This is the equivalent of our gross margin that tells us how we are doing in very basic terms. And, thirdly, we would need a fuel gauge in order to tell us how far we can go before we run out of gas. This is the equivalent of the cash in the bank.

These three measures of sales, gross margin, and cash need to be constantly reviewed in our search for profits. Indeed, if we had to survive on only one of these measures (hopefully for only a short period of time), it would be with the cash figure – the fuel in the tank.

However, these are financial metrics which have been around for a long time and are essential measures in any business. This takes us back to the question posed at the beginning of this note – what is the best measure of general business performance?

Business Performance Metrics

Measures of general business performance rather than financial performance are relatively new and they are jostling for position.

  • The Customer Satisfaction Score

The one that has been around the longest is the customer satisfaction score. It goes without saying that a business needs to keep its customers satisfied for otherwise they will defect. Exactly how we measure customer satisfaction is up for debate. Most companies use a ten-point scale which seems to fit with how we measure most things in life.

The importance of customer satisfaction is obvious but the link with profitability and company performance is a bit more tenuous. In the early 1990s Professor Heskett of Harvard University proposed a link between happy employees and a healthy financial performance. His argument in The Service Profit Chain was that there is a causal link between profitability, customer loyalty, employee satisfaction, and productivity. He opines that growth in sales and profits is stimulated mainly by customer loyalty, and loyalty is a direct result of customer satisfaction. Satisfaction is largely influenced by the value of services provided to customers. The whole virtuous circle leads to greater profitability for a company.

Customer satisfaction is still a vital metric to any company. If we measure overall satisfaction as well as satisfaction with individual parts of the offer, we can correlate the two sets of data and determine what is driving the overall figure (we derive the importance of the metrics). In this way we can point to the parts of our value proposition that are strong and need maintaining, and to those which need immediate attention. Figure 2 below shows that A,F and B are priorities for improvement as they are important in driving satisfaction and the satisfaction scores are low relative to other parts of the offer. Equally, items C and D are achieving high levels of satisfaction but are not driving overall satisfaction and therefore should not receive a high level of investment.


Figure 2

  • The Net Promoter Score

The Net Promoter Score, developed by Fred Riechheld of Bain & Company, brought the focus onto two parts of the measurement scale that are important. The question he proposed is “how likely would you be to recommend company X to a friend or colleague?” In reality, the overall score resulting from this question is very similar to what is achieved in answer to the overall satisfaction question. However, the NPS score focuses on two proportions of the sample response – those who give a high score of 9 or 10 as their likelihood to recommend, and those who give a score of 6 or below. Scores of 7 or 8 are considered neutral and are ignored in the calculation. By subtracting the detractors (scores of 6 or below) from the promoters (scores of 9 or 10), we arrive at the net number of promoters.

A typical business-to-business company (is there such a thing?) achieves a Net Promoter Score of around 20 to 25. Scores of 40 or more are considered exemplary and a strong indicator of high growth with good prospects of future profitability. This “golf handicap” metric has been adopted by many boards of companies who want a simple measure of how their company is doing. The NPS is easy to understand and is a good benchmark against which to judge a company’s performance.

If you had to choose just two general performance metrics, overall satisfaction and the Net Promoter Score would be high on the list.

  • The Customer Effort Score

The Customer Effort Score also may be relevant, especially for certain products and services. If we are buying a new airliner for our fleet, we expect a good deal of effort will be required in the purchasing process. However, for many business transactions they are at a lower level and we expect our suppliers to be easy to deal with. Some are not and the amount of effort that we have to put in can be a serious detraction from doing business. It can be enlightening to ask a question on how much effort is required to buy from a company.

  • The Net Value Score

We live in a competitive world and so it is not enough to know how we are performing in absolute terms; we also need to know how we are performing in relative terms. This means that the questions we ask about our own company as regards overall satisfaction or the Net Promoter Score should be repeated for companies in our competitive set. This can make interviewing tedious since the same question has to be repeated time and again for different companies. A simple way of overcoming this is to use the Net Value Score, which is derived from two questions framed as follows:

  1. How would you rate COMPANY X on the product or service benefits the company offers, compared to the product/service benefits offered by other suppliers of similar products/services?
  2. How would you rate COMMPANY X on it’s prices, compared to the prices of other suppliers of similar products/services?

Answers to these questions enable us to plot a company on the value equivalence line (Figure 3).

Figure 3

This paradigm is useful in telling us whether a company is offering more or less value for money than others in the marketplace. It is a good indicator of future growth and potential actions. For example, company C in the diagram is offering more benefits than companies A and B and is in a position to carry on with its current pricing strategy and increase its market share, or raise its price and collect more profits.

Concluding Words

The conclusion we arrive at is that metrics are a vital tool for helping us run our businesses. However, metrics deliver different intelligence and should be chosen to meet different objectives. The table below summarizes how and where the metrics discussed in this paper should be used.

MetricQuestionWhere to use it
Customer Satisfaction ScoreOn a scale from 1 to 10 how satisfied are you with company X?Determines strengths and weaknesses of a company on different aspects of its customer value proposition
Net Promoter ScoreOn a scale from 1 to 10 how likely would you be to recommend company X?Establishes loyalty to a company and is an indicator of its growth and profitability
Customer Effort ScoreOn a scale from 1 to 10 how much effort is required to do business with company X?Indicator of ease of doing business and future market share
Net Value ScoreHow would you rate company X on its products or services compared to other similar suppliers? (And similarly how would you rate it on its prices?)Positions a company on the value equivalence line

https://www.b2binternationalusa.com/