Rules For Measuring Success

Corporations as Superorganisms

Like metrics for products, the meaningfulness of each number should directly relate either to the “one number” that truly matters, or indirectly but transparently tie to easy-to-prove related, secondary, or proxy metrics – especially to the individuals being tracked!

To understand the rules I’ll lay out for determining metrics that can bridge the gap between individuals, software teams, business divisions, and corporate goals, I will lean heavily on the concept of the Minimum Viable Superorganism described by Kevin Simler because, to understand what to measure, a leader must know not only how to measure a metric but also why the metric matters to the achievement of long-term goals.

“Here’s our recipe for a minimum viable superorganism: ‘Selfish individuals pursuing shared goals (arising from shared underlying incentives), held together by a Prestige Economy which consists of two activities: (1) seeking status by attempting to advance the superorganism’s goals, and (2) celebrating (i.e., sucking up to) those who deserve it.’

The executive and visionary of the company must be the purposeful “mind” leading the activity of the superorganism. The rules for the relationship between employee motivation and company outcomes rely on this Prestige Economy, so leading a company means guiding this economy.

“Effective leadership is not about making speeches or being liked; leadership is defined by results not attributes.” —Peter Drucker


 

Prestige Economics

So what can be tracked that scales meaningfully from the individual contributor to a multi-division organization? How can motivate great performance across a company with GOOD metrics that encourage correct behaviors for serving the goals of the superorganism? First, we need to look at the Prestige Economics motivating employees in a specific company:

In exchange for doing work that advances the company’s goals, employees are granted social status, most of it in the form of money. But some compensation also takes the form of good old-fashioned prestige, i.e., the esteem of coworkers (often reified in fancy titles or corner offices). Either way, employees are rewarded for helping the company, regardless of whether it ultimately succeeds or fails in achieving its goals.

It is easy to see why metrics are so powerful and dangerous – if executive leadership tracks a number, employees anticipate that managers will use that number in the Prestige Economy. Praise, bonuses, perks, salary increases, and training or conference allowances are all – implicitly or explicitly – impacted by the individual’s contribution to the metric. So when company-wide stock options or bonuses are provided based quarterly revenue goals, this increases cohesion. Everyone is knows they have a reward for the success of the group. Obviously there a few caveats to the success of this metric – if an employee cannot clearly articulate the impact they have on company revenue, or they accurately believe they are not positioned or empowered to control their impact, the metric is unlikely to work.

Now imagine what happens when executives begin tracking a performance metric that is counter to best demonstrated practices. Employees will watch closely for changes in praise from colleagues and supervisors, and judge that cohesion as an indication of that metric’s impact on her or his long-term status, prestige, and rewards.

With the Prestige Economy in mind, let’s look at rules for “Good” metrics – the numbers that, when tracked, provide leaders the insights they need while also positively reinforcing behaviors that culminate in the achievement of company goals.


 

Rule #1 – Reinforce the discovery process, not the processes that are discovered.

Although many team-level empirical process metrics are a natural part of agile software development and the team’s collective desire for continuous self-improvement, scaling any of these metrics to the enterprise restricts team-level experimentation!  As an example, a newly-formed Scrum team may find their velocity will double or triple over the next 6 months as they remove impediments and learn to work in smaller increments.  Tracking velocity is an exciting indication of finding their “flow” as a team. However, comparison of story points per sprint across individuals and across teams in an organization will fail miserably. It is a good team metric as a short-term trailing measurement for experiments; it is an ugly metric for executives to track due to Hawthorne Effect. Velocity is relative to team maturity, product lifecycle stage, certainty of technical environment, and predictability of market demand.  A team consistently driving bleeding-edge innovation will frequently disrupt and re-calibrate team velocity and commitment. Velocity is an occasionally valuable metric to the team only to the extent it validates a hypothesis.

Imagine if an organization standardized a target for two metrics – 1) hours per story point  2) story points per sprint – and set these a benchmark for each in order to review employee performance.  These two team-level measurements may have been handy, after all. Perhaps a process change was at point justified based on its impact on them. In their excitement, they assume setting a target per individual, team, business line, and for the enterprise is a fantastic control for productivity.

Instead, terrible things happen. Setting these targets focus energy away from the strategic goal of rapid innovation:

  • Individuals have a target to meet rather than a team having the freedom to experiment
  • Teams pursue risk avoidance (don’t endanger burn rate) rather than risk recovery (grow to be a team that easily adapts to disruptive innovation)
  • Business managers encourage fail-safe processes over safe-to-fail processes (variance is now a sign of poor management and performance)

Thus, what was once a handy measurement is now the death of continuous improvement, innovation, and the competitive advantage the company hoped to standardize!

This is known as Goodhart’s Law:

When a measure becomes a target, it ceases to be a good measure.

– via University of Cambridge

Professor Charles Goodhart FBA postulated this economic law while Chief Adviser to the Bank of England, analyzing the government regulation of instruments that are also critical to the measurement of economic trends:

Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.

As soon as the government attempts to regulate any particular set of financial assets, these become unreliable as indicators of economic trends [because] financial institutions can […] easily devise new types of financial assets.

– via University of Cambridge

The same phenomenon is true for performance metrics in innovative software development – the moment a target is set for a measurement, the culture of innovation will either collapse (innovation energy is suppressed) or a new practice will emerge that circumvents and actively undermines the measurement (innovation energy is misdirected).

This is once again the key difference between statistical process control and empirical process control. In a childhood chemistry experiment, the children may mix varying quantities of vinegar and baking soda and measure the amount of salt that results. All three are measureable and are variables important to the discovery process. If the teacher gave grades based on variance around a desired amount of salt, it would focus the experiment on finding that quantity rather than learning about the variety of possible outcomes. Consistency of outcomes is detrimental to innovation and learning about anything other than ensuring consistency.

Rule #2 – Metrics must reinforce respect for individuals

For innovation to occur at scale, process metrics must reinforce the empirical processes that encourage organizational learning to continue.  From the Kantian perspective: Ugly metrics communicate a Subject-Object relationship and by communicating that each person is a means to an end, that person now expends energy to overcome her or his feeling of objectification. After all, a Prestige Economy requires a level of sovereignty per person.   Prestige is only attainable by two rationally self-interested individuals that respect the recognition and prestige earned from one another. By undermining this, any ugly metric wastes the combined energy of the superorganism. Bad metrics maintain the Subject-Subject relationship, so that a Prestige Economy is able to grow but reinforce the incorrect behaviors.  Good metrics maintain the Subject-Subject respect while reinforcing the behaviors that maintain this respect.

This is the fundamental problem when an organization suffers from Johnson’s “Confusion of Levels” –

“Confusion of Levels” – failure to see that whereas in a mechanical system one-dimensional quantities can both describe results and enable one to control the linear process that produces those results, in a living system quantities can only describe results, but cannot explain or enable one to control the multi-dimensional interactions and feedback loops of the process that produces the results.  – H. T. Johnson, “Lean Dilemma”

In companies competing via their culture of innovation, this means leadership must carefully ensure the correct emphasis for the goals of the superorganism for each of the five basic phases of any process as defined by Steven Borg:

  1. Queue – Understand what to do
  2. Setup – Get the necessary “resources” ready
  3. Run – Complete the goal
  4. Wait – Look for feedback on success/failure
  5. Move – Find the next goal to accomplish

The inherent flaw of most productivity metrics, and the incredible inefficiency of the productivity-focused superorganism, is over-emphasis on the “running” state of each individual. All additional energy expended in a system or process on the “run” state of each respective functional discipline is a source of waste – a true “confusion of levels” – because the value of the outcomes of a cross-functional, superorganism-based, has more to do with the efficiency and effectiveness of the product as each increment flows through the other four process states.

A good metric must reinforce the superorganism’s Prestige Economy around the most effective and best focused coordination of the five process states by individual functions that are inherently in different states as work moves through the queue. The ugliest metrics in software development creates competition around the functions that take a product increment through the process – performance metrics that maximize each function’s run state rather than the outcome of the cross-functional team.

Here are a few terrible metrics. If you track them now, stop immediately:

  • Lines of code written (as a judge of a developer)
  • Individual burn rate (as a judge of a developer)
  • Bug cards created (as a judge of a tester)
  • Percentage of cards rejected by a tester
  • Hours per story point (as a judge of estimation accuracy)

You can see that any energy expended improving any of these individual “performance” metrics removes energy from collaborating on creation of the best possible software.

Rule #3 – Build Superorganism Identity

The problem of undermining the creative, cross-functional, creative process extend beyond tracking performance metrics that produce inefficiency. Performance metrics, after all, are powerful precisely to the extent they are a leading indicator of status in the superorganism Prestige Economy. If employees see there is no relationship between a performance metric and their salary, position, perks, respect amongst peers, or recognition from management, that metric has no power. Time spent tracking it, evaluating it, and discussing it is all unnecessary waste in the process, of course, but the real danger for executive leadership is that metrics like this erode the power of metrics to redirect the superorganism. On the other hand, the more a performance metric (for better or worse) is formally tied to the Prestige Economy – via awards, bonuses, salary increases, promotions, corner offices, better parking, flying first class, etc – the more powerful (and potentially dangerous) that metric becomes.

It is critical to the viability of the company that executive leadership envision and purposefully craft a Prestige Economy that reinforces the identity of the superorganism and its ability to achieve its strategic vision.  Obviously, the overarching goal of viability is insufficient to ensure its viability! The individuals of a company are part of multiple superorganisms they contribute at any time – a citizen of their national government, a member of a professional guild, part of a social or cultural movement – so viability for each company must contend with the fact that its Prestige Economy could at any time be at odds with multiple other Prestige Economies. The more the individual employee sees a cohesive fit across each superorganism to which they contribute, the better energized, empowered, or inspired they are to pursue a goal that will simultaneously give them additional prestige across their multiple superorganisms.

However, the moment your company’s goals and Prestige Economy are at odds with an employee’s power to earn prestige in their other superorganisms, they will find a new company that ensures better fit and maximizes their capacity for aggregate prestige. For instance, when the company goals for a Six Sigma Black Belt at a manufacturing company or a Scrum Master at a software company have great “fit” with the prestige available from their certifying body or community of practice – each is likely to work that much harder. If the Prestige Economy of their superorganism becomes sufficiently detrimental to the Prestige Economy of their professional guild, their long-term career may be jeopardized. Dropout is a natural outcome.

To that extent, insofar as a performance metric is a formal measurement of variables in a company’s “Prestige Engine”, it is critical that they reinforce the superorganism identity that ensures its viability.

This is the reason why so many companies have adopted not only a vision statement but also a set of values or principles to which every individual – and the company as a superorganism – ought to aspire. These create the “rules of thumb” that let each employee know how long-term strategy should be applied to trade-offs in the day-to-day hard decisions each employee must make. If the values are not properly expressed or inconsistently praised in the Prestige Economy, cohesion will fail and strategic positioning will be incomplete.

We have a clear first step in establishing good metrics: they are quantifiable indications of whether or not a company’s Prestige Economy will reward a behavior that contributes to the identity (values) and direction (strategy) of the superorganism. Put another way; if your accounting KPIs don’t align to your company vision while your employee performance metrics don’t align to your company values you will never accomplish your goals. To succeed, what you measure must align to what you want.

“Managers who don’t know how to measure what they want settle for wanting what they can measure.” – Russell Ackoff

Rule #4 – Reinforce Superorganism Intent

In a small “flat” organization, positive reinforcement and the mutually self-reinforcing Prestige Economy can emerge fairly organically. The visionary of the organization can build a Prestige Economy through attention, praise, and charisma on a one-on-one basis with employees. To the extent the “flat” structure of the organization grew out of a general equality of its members, position, perks, and salaries at this stage can be safely held equal as they grow, leaving public recognition of contribution and aptitude as the “premium currency” of the Prestige Economy.

As a company scales, the capacity of a single leader to personally create, manage, and re-direct the necessary Prestige Economy becomes unimaginable. The recognition and praise of the leader becomes less important the respect of peers because the merits of behaviors – through lack of exposure – seem painfully de-contextualized and thereby de-valued. Likewise, a growing number of employees inevitably necessitates a formalization of an emerging pecking order as experience, tenure, and aptitude all become simpler to segment.

We should note that at a certain size, the primary executive is no longer able to meaningfully reward daily prestige directly to employees without undermining Prestige Economy. The feeling will prevail that the chief executive is showing favoritism or simply rewarding haphazardly. Once that happens, without the company visionary guiding a formalization of the Prestige Economy, the superorganism will dismantle, as agency dilemma pushes individuals to preserve their value outside of the values of the superorganism.

 

This is part of a series!

Part 1 – Metrics: The Good, The Bad, and The Ugly

Part 2 – How to Fail at Performance Metrics

Part 3 – Rules For Measuring Success

Part 4 – Measuring What Matters to Innovation

Throughout the series I tie together ideas from two great resources:

Kevin Simler’s Minimum Viable Superorganism

Steven Borg’s From Vanity to Value, Metrics That Matter: Improving Lean and Agile, Kanban, and Scrum

3 thoughts on “Rules For Measuring Success

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s