The problem with incentives

Nothing to do with AI

Eventually I had to have a post that had nothin to do with AI. Today is the day.

Arnold Kling had a great post over the weekend about how incentive systems get gamed. Metrics are always, almost by definition, a subset of the information that matters. What you really want employees to do is “make the company better in a cost efficient way”, but you can’t measure that quantitatively, or create a compensation system around that. So instead you pick metrics to track that, you hope, correlate with the good things you want to have happen at the company.

Getting good metrics is not easy, but it’s not THAT hard. Most experiences executives know what numbers go up when the business is doing well. The problems happen when a metric goes from a measurement tool to a goal. Leads are a pretty good early metric for how many sales opportunities a company will get over the next few months. But if you tell the marketing team to maximize the number of leads they bring in in any given month, you will find leads go up much faster than opportunities. It is a lot easier (and cheaper) to buy terrible leads that do not convert into sales than to find high quality leads that do. When the metric becomes the goal, the metric stops measuring what you wanted it to.

Kling focuses on the employee compensation element of incentives. He gives examples of where these compensation systems fail

I want a loan officer to approve low-risk loans and to reject high-risk loans. If my compensation system rewards approvals too much without sufficiently penalizing the approval of high-risk loans, the incentives will lead to a loan portfolio that is too risky… If I pay writers by the word or software developers by the number of lines of code, I am neglecting to reward quality... Organizations often require teamwork. If my performance metrics are all individually focused, I am inducing my employees to compete with one another, undermining the incentives for cooperation.

You can partially solve the problem by allowing managers to give bonuses based on qualitative judgement, but then you run into the problem of bias. Many studies have shown that employee ratings over time are more correlated with managers, than the employees themselves (i.e., if you want to predict a given employee evaluation you would be more accurate by looking at the scores the manager had given to OTHER employees in the past, than the score that particular employee had obtained in the past from a different manager).

Kling argues that one way to reduce the problems with incentive systems is to change the incentive systems on a regular basis. If a metric is originally correlated with company success, the obvious ways to improve the metric will also likely correlated with company success. Over time employees will find ways to “game” the system and improve the metric without driving company performance. When that happens the company can switch to a new metric which employees have no experience gaming.

(See also last week’s post on the “new arms race” for a similar pattern, albeit not having to do with employees at all).

The whole thing reminds me of a book I read 15 years ago. “The Modern Firm”, by John Roberts, is about organizational design. One part that stuck with me was his belief that there is no perfect organizational structure. If a company is designed around customers (i.e., “Walmart team”, “Costco team”, “Kroger team”), they will underperform competitors in product knowledge. If the organization is designed around products (“Paper team”, “Food team”, “Cleaning team”), then they will underperform competitors in customer knowledge. Robert’s solution is that company’s should be re-organized every 3-5 years from product-focus to customer-focus, and then back again.

I went through a customer-focus to product-focus re-org while I was at Procter & Gamble in 2001. We joked at the time how ridiculous it was because it had not been that long prior that the company had formed into the customer-focused teams. But according to Roberts this seemingly wasteful behavior was the only way to get employees to understand both customers and products well enough to be competitive.

Keep it simple,

Edward