Metrics for Assessing Human Process on Work Teams

by John W. Bing, Ed.D.

Management has been defined simply as "getting things done through others."1 Managing and leading complex organizations is challenging, at least in part because the most utilized tools to assess management's approaches are end-point financial measures. Reviewing quarterly profit/loss statements as a guide to management's skills is a bit like measuring a doctor's skills by the apparent health of the patient rather than reviewing the full set of analytic results which more effectively predict health or illness. We know too well that reliance on short-term profit results is no certain indicator of the long-term health of a company. We need other measures, measures that assess both the application of specific managerial approaches and policies in addition to the output measures of financial returns. In so doing, we increase the opportunities open to managers to understand and improve the effects of their policies and approaches.

I suggest in this article that measurement and monitoring of work teams over time is a crucial way for organizational leaders to both support improved team performance and to measure, through the aggregation of human performance metrics on a digital dashboard, changes in team performance and, in turn, to relate these measures to bottom line, financial changes.

CHARACTERISTICS OF WORK TEAMS

Work teams are the backbone of contemporary work life. Executive teams run corporations. Project teams create new products and services; matrix teams are involved in the development of everything from pharmaceuticals to the delivery of services in consulting firms and charitable agencies. Marketing and sales teams deliver products and services to customers. Except in the most traditional of organizations, for example sometimes in governmental organizations in which highly structured departments remain, teams are essential to the way organizations carry out their work.

Global Teams are a special genre of teams. Most of the examples in this article are drawn from global teams. The notion of global teams would have been thought at best unlikely until the last two decades. Within that time, communications infrastructures began to provide efficient support for synchronous and asynchronous contacts between distant employees, and corporations began to realign their workflow through those individuals and those geographical areas most likely to increase efficiencies and productivity. Thus, virtual teams and global teams were created to allow companies to improve competitive advantage. Such teams, however, provide managerial challenges, as the imperative to "get work done through others" is different when the "others" are not found around the proverbial water cooler but are embodied in bits and bytes on computers and telephone exchanges. With increasing distance between team members, it is much more difficult to build trust, which underpins successful teams.2

Cultural and linguistic differences also play significant roles in mediating communications on global teams. Although the business of most global teams is conducted in English, there is typically more than one language natively spoken by members of a global team. These members may have more difficulty expressing themselves in spoken or written English than do their native-English-speaking colleagues.

Cultural differences are more subtle intermediaries. In one project monitored by ITAP International involving a pharmaceutical team located both in the United States and in Spain, there were numerous complaints from the American team members because their Spanish colleagues copied e-mail messages to their bosses, which the Americans perceived as undermining forthright communications. In a meeting convened to work through issues uncovered in measurements of team process,3 the Spanish members of the team asserted that it was their responsibility to inform their supervisors of the progress made by their team; to do otherwise would be negligent.

Dutch pioneer Geert Hofstede measured cultural differences through a five-dimensional system.4 The dimensional set which emerged from his research - Power Distance, Uncertainty Avoidance, Individualism/Collectivism, Masculinity/Femininity, and Long/Short-Term Orientation - is the most researched body of quantitative crosscultural research in the literature. The Spanish scores for one of these dimensions, Power Distance, a measure in part of "subordinates' fear of disagreeing with superiors and of superiors' actual decision-making styles,"5 are significantly higher than those for the United States. That specific difference was the root cause of a number of communications problems within this team. As another example of Power Distance issues, the American department head, who supervised the team, thought that the Spanish supervisor controlled the Spanish contingent too tightly; the two leaders had a number of meetings before this issue was at least understood, if not resolved. In circumstances in which one culture dominates another by virtue of the authority of the home company, such differences are often hidden or suppressed but no less significant and are surfaced through the application of process metrics or, less fortuitously, subsequent team malfunctions.

MEASURING TEAMS

Teams are a bit like people in their complexity and types; and although progress has been made in measuring individuals with respect to their type, development and capabilities, the science of team metrics is in its infancy. This is even more notable in that teams could well be natural units by which top management might most efficiently determine the effectiveness of their policies and leadership; yet management has rarely attempted such measures. Reorganizations and restructurings are common; yet how many have been implemented in which appropriate pre- and postmeasures have been taken to determine the specific areas that require redevelopment and restructuring and most important whether such efforts have achieved desired results?

Indeed, much of the reorganization and restructuring of organizations, which is enormously expensive in terms of time and other resources, might well have been avoided had diagnosis of problems been undertaken and smaller-scale changes made. In addition, in the contemporary organization, teams are perhaps the most appropriate unit around which to make complex and ongoing measures of both human process and productivity. As Jones and Moffett write, "an effective measurement system gives work teams the same kind of business data once used only to manage entire organizations.6

Lacking such measures, it seems quixotic at best and malpractice at worst for management to reorganize work units.

The lack of team output measures also accounts for the difficulties that management has in rewarding teams. Although we have 360-degree measurement (typically in Western corporations) to measure the development of managers, very few team measures exist to support rewards systems for teams. Most team rewards are based on individual assessments rather than team effectiveness.

Management's inattentiveness to such matters is, I believe, a constraint on improving the capacity of complex organizations to become more effective as learning organizations. For if management is not focusing on measuring the effectiveness and productivity of their organizations beyond financial measures, it is difficult to determine which parts of the organization are functioning well, and which are not, beyond a gut reaction. Such measures have the additional value of providing specific diagnostics, and therefore appropriate interventions, when teams falter.

TYPES OF MEASURES

There are two types of team measures to be discussed here, those of team process and those of team output or team productivity.

Productivity Measures

Some types of teams lend themselves to such measures more easily than others, and some measures are more typically applied than others. For example, a top management team might be measured appropriately by shareholders by means of the general productivity of the company in terms of its profit or loss over time; however, at the same time, measurements of the company as a learning organization are also important indicators of the capability of the company to maintain or increase productivity, and these are more difficult to make. At the shop floor level, measures are usually easier and can be made in terms of the number of units produced at specific levels of quality by work teams as well as the safety record of the unit over time; absenteeism, and so on.

Jones and Moffett, in a case study on measurement on work teams, list four common measures on such teams: productivity, quality, cycle time, and on-time delivery. They then note that:

To establish ownership, teams customize their measurement system in four ways. First, they can add a measure of their choosing that reflects the team strategy. Second, within limits they can determine the weighted importance of the measures to reflect their own thinking about strategy. Third, within limits they can set their own performance standards for each measure. Fourth, in some cases they can influence how a measure is calculated so that it comes more under their control.7

However, measurement often stops at the factory level.

While it may be more difficult to define appropriate metrics, various human resources department teams could be measured on the length of time required to recruit specific positions matched with supervisor satisfaction of the successful candidates and the costs involved; on the cost of payroll per employee; and training, by supervisor satisfaction with skills provided to subordinates.

Human Process Measures

Human process measures are likely to be precursors to productivity changes. Why? If communications fail or are marginal on a team, it is likely that productivity drops will not be far behind. If objectives are not clear, then how can a team reach them? If roles and responsibilities are muddy, how can the team efficiently carry out its work? If trust is lacking, how can a team work together? An American general who commanded troops in the first Gulf War commented: "We had an unusually strong team, and trust was a major factor.... You need people schooled and trained in their own type of warfare, and then you need trust in each other."8

Although there is face validity to the above statements, management is often reluctant to carry out measures to determine the degree of comparative effectiveness in these basic areas. In such resistance, there may be many missed opportunities.

COMPARING HUMAN PROCESS WITH OUTPUT ON TEAMS

Although this article is not focused on the measurement of team productivity, or outputs, it is important that human process metrics be combined with team output measures to correlate how changes in team human processes lead to changes in output. Of course, although finding of correlations do not prove causation, repeated correlations over time and with different teams will eventually build a solid base for viewing process measures and interventions as fundamental to improving team productivity.

A decade ago, a Swiss-based pharmaceutical company asked ITAP International to develop a method for measuring process performance on three global teams. The teams were composed of scientists from Europe, the Americas and Japan. They met four times over a period of two years, and continued their team responsibilities during the intervening periods. Their purpose was to create procedures to reduce research and development time in three drug delivery areas: oral, skin and subdermal. The teams were tasked with similar assignments and because the composition of the teams was similar, they became ideal candidates for studying differences in human processes on global teams and comparing results between the three teams.

A questionnaire was developed to measure human process on these teams, and was administered six times over the two-year period that these teams met together. At the end of the two years, specific questions from this questionnaire, now called the Global Team Process Questionnaire™ (GTPQ) were compared with peer rankings provided by the participants. These correlated positively; in other words, the highest-peer ranked team also had the highest GTPQ results on the questions tested.9

In Figures 1 and 2, process questions are correlated with the peer-ranked productivity of each team. In Figure 1, team objectives were compared with quality of output; in Figure 2, perceived communications levels were compared with peer-ranked outputs. In both cases, there were straight-line correlations between process and output.

Figure 1. Team Objectives vs. Quality of Output.

Figure 2. Perceived Communications Levels.

Team Process Examples

Let's review examples of measurement of human process on teams. The teams listed were all (except in one case) measured by the Global Team Process Questionnaire™, mentioned above, which has been construct-validated dimensionally, and which has proven reliable in repeated applications.10

The following examples are all taken from assessments of teams within the pharmaceutical industry. This industry makes extensive use of global teams in basic research, statistical analysis, product development, clinical development, regulatory, operations, and marketing and sales. During various stages of drug development and testing, many of these functional areas are representing on cross-functional coordinating teams supported by single-function teams. The functional teams are able to support a number of coordinating teams. Many of these teams may have member representation from more than one country.

FAILING TO APPROPRIATELY CHARTER A TEAM

One of the most important factors in a team's success is the initial chartering of a team. Chartering is, essentially, providing the internal and external objectives and structure for newly created teams and providing team members with an understanding of their roles and responsibilities. If this step is not appropriately provided, many problems can develop in the future activities of the team.

Figure 3 is the executive summary for a team that had not been appropriately chartered. The executive summary contains five dimensions (in a domestic version of the GTPQ, which has an additional dimension, Culture and Language).11 The domestic version is called the Organizational Team Process Questionnaire™ or OTPQ. Note that in all five dimensions this team lags the pharmaceutical industry averages that we maintain.12 In this summary, 1 is the "best" score and 6 the "worst," so the lower the score the better the outcome.

Figure 3. Executive Summary for an Inappropriately Chartered Team.

Figure 4 is a spidergram, in which each letter on the diagram represents a team member's score for this specific question - "Are the objectives of your team clear?" Only one of the nine team members indicated an understanding of the objectives; the remaining eight did not. This is another symptom of failed chartering and indicates that the team must go through a chartering process if it is to form the basis for teamwork.

Figure 4. A Spidergram.

ADDITION OF NEW MEMBERS INTO AN EXISTING TEAM

Over time, productive teams develop a sense of trust and a common approach to work, which can be disrupted when members transfer out or in. The larger the percentage of team members lost or gained, the larger the consequence.

Figure 5 shows data on four questions taken after the induction of 11 new members and a new team leader (with only three prior members remaining). All four questions show a distinct difference in perspective between the old team members and new members on the issues of team leadership, clarity of objectives, communications and trust. (In this table, 1 = best possible score; 6 = worst possible score). Here is clearly a case in which, based on the data, a new team leader combined with a large influx of new members requires a team rechartering effort. Otherwise, the old members will remain a disaffected group within the larger team.

Figure 5. Team Scores after Induction of New Team Members and Leader

Question	Average Score All Team Members	Average Score Old Members	Average Score New Members	Difference
Effectiveness of team leadership	2.50	3.33	2.22	-1.11
Clarity of Objectives	2.67	3.33	2.44	-0.89
Communications	3.00	3.67	2.78	-0.89
Trust	3.00	3.67	2.78	-0.89

DEGREE OF MANAGEMENT SUPPORT

Management support is an external but powerful force that can either buttress or retard the productivity of teams. Figure 6 is a comparison of two teams, one with perceived management support, and one without such support.

The top spidergram is tight, indicating a team that acknowledges and appreciates management support. The diagram on the bottom shows a team that as a whole believes that management support is lacking, although there are strong differences of opinion. In an intervention based on this data, a facilitator would probe the team for the reasons behind the perceived ambivalence of management support. Management, in turn, would have data indicating that their support was either not communicated or not sufficiently provided.

Figure 6. Comparison of Teams With and Without Management Support.

THE EFFECT OF MERGERS ON TEAMS

There are other outside influences on teams. In an early version of this measuring instrument, three teams were tested over six iterations on a variety of team process issues. On the first five iterations, there were changes in the process effectiveness of the teams, although team B was clearly the leader overall. However, on the sixth iteration, all three teams fell (see Figure 7). At first, we were puzzled. We had expected that our interventions would improve team process and performance. Were our interventions faulty? Then we realized that the steep decline in all three teams was caused by the effects of a merger with a larger pharmaceutical company, announced just after the fifth iteration of the questionnaire had been administered. As is often the case in mergers, team members expressed concern for their jobs and positions within the smaller firm with which they were associated. This provided support for the notion that the metrics were reflecting the impact of "real world" events.13

Figure 7. Effects of Mergers on Teams.

KEEPING A TEAM ON TRACK

A corporate leadership team, which had consistently improved over four iterations, had a noticeable fall-off in the fifth iteration GTPQ results. At a team meeting, a facilitator helped the team discuss the issues that had the most marked negative responses and the biggest drop from the previous iteration (see Figure 8). The team was then able to identify common themes - direction, ownership, deployment, communication and cooperation - which they were then able to weave into the strategic planning discussions held throughout the remainder of their off-site meeting. The example in Figure 8, of a specific question from the GTPQ, was but one question of a total of 28. However, the aggregated team results showed the same fall-off in human process effectiveness for this team. Again, this illustrates both the kind of results obtained as well as how the information can be used to quickly check performance declines.

Figure 8. A Specific Question from the GTPQ.

CONCLUSION

Process and outputs are inextricably linked on teams.14 Future research will indicate whether human process measures can forecast productivity changes or whether these factors are correlated in other ways. In either case, metrics that allow managers to drill down to specific problems with processes on teams will allow interventions in the most timely and efficient manner, and thus improve or maintain superior team performance.

Similarly, when aggregating and comparing collective measurements from many teams, for example as part of digital dashboards, managers will be able to look at both process changes on individual teams as well as cumulative data for departments and divisions.

This will lead to improvements at the individual team level, as well as cumulative changes across teams as a way of determining management performances. This represents an alternative to reorganizing departments or companies without pre- and post-data in the hope that such changes will improve financial performance. Large scale change is not always beneficial, although it is sometimes necessary for strategic business purposes. Knowing the difference between change for its own sake and change for a specific set of objectives may save companies both time and money and increase shareholder value.

ENDNOTES

1 Hofstede, Geert. "Business Cultures" in UNESCO Courier, April 1994, V. 47, p. 12.
2 Asherman, Ira; Bing, John W., and Laroche, Lionel, "Building Trust Across Cultural Boundaries" originally published in Regulatory Affairs Forum and available online here.
3 The metrics approach used in the cases illustrated in this article were provided through the Global Team Process Questionnaire™, created and utilized by ITAP International.
4 The Hofstede cultural data has been taken from Geert Hofstede, Culture's Consequences: Comparing Values, Behaviors, Institutions and Organizations across Nations, Second Edition. Sage Publications, 2001.
5 Hofstede, page 79.
6 Jones, Steve, and Moffett, Richard G., "Measurement and Feedback Systems for Teams," in Sundstrom, Eric, and Associates, Supporting Work Team Effectiveness: Best Management Practices for Fostering High Performance. Page. 157. Jossey-Bass: 1999.
7 Jones and Moffett, op. cit., p. 159.
8 Quoted in Mathieu, John E, and Day, David V., "Assessing Processes within and Between Organizational Teams: A Nuclear Power Plant Example," in Brannick, Michael T., Salas, Eduardo, and Prince, Carolyn (eds.),Team Performance Assessment and Measurement: Theory, Assessment, and Applications. Lawrence Erlbaum Associates, 1997.
9 Bing, John W. "Developing a Consulting Tool to Measure Process Change on Global Teams: The Global Team Process Questionnaire™" (Page 4), proceedings of the 2001 national conference of the Academy of Human Resource Development and available online here.
10 Bing, op.cit.
11 The dimensions were created based on literature research and factor analysis of questions responses.
12 The database is maintained by ITAP International and is the collection of responses from over 100 teams, many followed through multiple measurements.
13 Bing, op.cit., p. 7.
14 For other examples of the relationship of process and productivity of teams, see Bing, John W., "The Relationship between Process and Performance on Teams".

John Bing is the chairman of ITAP International, a consulting firm with operations in Europe, the U.S. and the Asia Pacific region. His consuiting experience spans the Americas, Europe and Africa and the pharmaceutical, consumer product, information technology industries and United Nations' Agencies. He is the designer of ITAP International's Team Process Questionnaire family of consulting instruments and is developing a new version of the Culture in the Workplace QuestionnaireTM, originally created by Geert Hofstede. He has published papers and provided presentations at numerous professional conferences including the American Society for Training and Development (ASTD) and the Academy for Human Resources Development and recently co-edited (with Darren C. Short) a volume entitled "Shaping the Future of HRD" in the Advances in Developing Human Resources series. Mr. Bing was a founding member of SIETAR (the Society for Intercultural Education, Training and Research). He is a member of the Research to Practice Committee of ASTD and is the recipient of ASTD's International Practitioner of the Year Award. A graduate of Harvard College, Bing received his Ed.D. from the Center for International Education, University of Massachusetts. He speaks Afghan Farsi and is an avid hiker whose goals include hiking to three of Colorado's 14,000-foot peaks each summer.

This article originally appeared in the International Association for Human Resource Information ManagementIHRIM Journal, November/December 2004, Vol. VIII, Number 6.