The failure of measurement

You may have noticed a preoccupation in my posts about knowing. What it is to know, how does one know, and why it is important to know. I firmly believe knowing your self and your environment is one of the prerequisites to taking control of your life. It is also one of the hardest to get right.

A tale of two systems

There are two kinds of systems. One kind is knowable, while the other is perverse.

Your local weather is a fine example of a knowable system. Despite its confounding complexity, our atmosphere does not respond to the weatherman's forecast. The weather systems crossing the globe do what they will without much regard for our opinions or actions1.

This allows us to continually improve the accuracy of weather forecasts2. Improved data sets, increased scientific knowledge, more complex models, and vast gains in processing power have improved the accuracy of forecasts, increased the distance we can peek into the future, and allow us to update predictions more frequently as new information comes in.

Perhaps the foremost example of a perverse system would be the stock market.

A reality shaped by belief

Stocks move in response to news stories, rumors, and the general current of trades. If enough people buy into them, beliefs the stock market will rise or fall are self-fulfilling.

Correctly predicting any movement in the market, up or down, allows you to make money. But making moves in order to profit from this forecast is itself a signal others actively monitor. If the signal is strong enough, other actors will take actions that benefit themselves and blunt your returns. It is a vicious cycle.

The average return on stock market investments is average, and this goes even for highly trained, very experienced professionals. Hedge funds are in what is essentially a co-evolutionary arms race with the market and each other. All advantages and advances will, by nature of the forces involved, eventually be subverted.

This leads to an interesting state of affairs. We can know the upcoming weather with extreme accuracy for a handful of days, but after that horizon, our ability to forecast specific weather events evaporates.

With stocks the opposite is true. We can tell almost nothing about today or tomorrow, but over the long term, we can make certain assumptions about industries, national economies, and perhaps even specific companies. The best bet for most of us becomes a general and long-term one (broad and diversified index funds).

Think beyond the stock market

Anyone with a job is familiar with measures and targets. They are the primary tools used by businesses to encourage productivity and align employees' actions with the business's goals. The reward for meeting goals might be a bonus, a promotion, or simply not being fired. Government regulation is a similar incentive, as businesses in regulated environments are required to meet certain benchmarks in order to escape sanction.

The difficult reality is all of these targets will be gamed. Perverse systems are created by just these sorts of targets and incentives.

Imagine a retail business that tracks sales in order to reward salespeople. In short order, the sales team will begin to work at cross purposes. They will snipe each other's sales whenever possible. If returns are not tracked, they may scheme with others to purchase and then return items so they can meet sales goals.

Car dealerships sometimes operate in a similar fashion. A small number of cars are sold well below cost in order to meet a monthly or quarterly quota set by the manufacturer. Any rational person would take a $2,000 loss on one sale in order to protect a $30,000 bonus, and dealerships are no different. This behavior is part of why so many of us find buying a car to be a stressful, dirty, and dishonest experience.

My time in law enforcement was no different.

A recent example

The New York Times bestseller list is about as clear a benchmark as one can imagine. Books that sell a lot of copies show up on the list; books selling fewer copies do not. What could be simpler?

A closer look reveals the system is not perfect. Books move in relatively small numbers. It would not take much money for a publisher or author to purchase their own book and move it up to the top of the charts. Spending the same amount of money advertising the book is a gamble in comparison.

This happened recently when a YA novel attempted to purchase its way onto the list. Some of the money behind the book and the fraudulent purchases hoped to make a movie from the novel. Even one brief appearance on the list would have allowed the film to be marketed as "based on a New York Times Bestseller" until the heat death of the universe.

It was not the first attempt to game the list, nor will it be the last. And it might have worked, too, if a few sharp eyes outside of the New York Times had not caught onto it.

Whether it is selling girl scout cookies or meeting performance goals at work, people will find a way to meet a target if it benefits them to do so. This process tends to weaken or break the connection between the target and the reality it was intended to track.

• CEOs chase short-term gains over long-term success due to how their bonuses are structured
• managers worry more about documenting employee training than whether or not the employees got anything from the training
• teachers are forced to target test scores instead of doing everything they can to make sure their students can actually think

The examples go on and on, including a number of recent scandals at Wells Fargo, driven in large part by backwards incentive systems3.

Targets, benchmarks, and incentives are powerful tools for understanding reality and driving action. They are also terribly fraught. Creating a target introduces an arms race between people trying to exploit the new target for personal gain and those trying to extract meaningful information or performance results from it.

This is important to keep in mind if you manage people, and it is also something to hold close as you consider how and when to interact with a perverse system like the stock market.