MarketingBS: A Review of Talent

Hi everyone,

I’ve lost count of the number of things I have learned from Tyler Cowen. One particular idea that I internalized and written about here before (from his book The Average is Over) was the incremental development of chess AI. The AI programs went from barely being able to play the game, to challenging grandmasters, to surpassing the best humans in the world. Cowen pointed out something that I hadn’t previously been aware of: there was a period of time when chess AI programs could beat all human champions, but they were still beatable by humans that worked in tandem with AI programs. **Check the footnotes for ways this story influenced my thinking.

Today’s essay takes a look at his latest book, Talent: How to Identify Energizers, Creatives, and Winners Around the World (co-written with Daniel Gross).

—Edward

---

The quest to find great talent

In Marketing BS, I’ve regularly shared my thoughts about managing your career. I’ve written about salary negotiations, skill development, trends in CMO hiring, and more. But building a successful career is primarily, NOT about yourself — the real leverage you get in your career is by surrounding yourself with smart and talented people. For executives, recruiting and retaining a strong team is the best way to have real impact. With these ideas in mind, I excitedly pre-ordered a copy of Talent: How to Identify Energizers, Creatives, and Winners Around the World — a new book by Tyler Cowen and Daniel Gross.

The book is filled with counterintuitive ideas about finding talent. In addition to covering the academic literature on the subject, the book draws on the experiences of the authors. (Cowan is an economics professor, New York Times bestselling author, podcast host, and prolific blogger at Marginal Revolution. Gross founded the startup accelerator Pioneer and started the AI program at Y-Combinator, where he was a partner). Both authors have a track record of identifying early-career innovators with the potential to change the world — Cowen as part of the Emergent Ventures fellowship and grant program, Gross in the world of venture capital. VC funds generally succeed or fail based on obtaining one or two home runs out of 20 investments. It’s okay to fund a company that fails, but you REALLY don’t want to miss the next Facebook, Google, or Stripe. Emergent Ventures is all about making funding bets on people who will change the world, and almost by definition some of those bets will not play out the way you hope.

The authors state that, “Just about everyone is on a quest to find talent in others or to show off their own,” but in practice the book explores a subset of talent. As they describe it: “We focus on a very specific kind of talent in this book—namely, talent with a creative spark.” They describe these people as “project drivers.”

In my experience recruiting project drivers, I’ve found they can be broken down into two sub-types of talent:

1. Individuals who can run an organization. They can motivate a team, understand metrics, and make things happen within the structure that has been provided.

2. Individuals who can figure out how to build something new. They can work across an organization to get something done, iterate through failures, figure out which metrics really matter, and build the processes to collect those metrics.

Both types of individuals are hard to find, but the second type are even rarer than the first. As I read Talent, it seems like the project-driving talent Cowen and Gross were looking for fit almost entirely into the second category.

For people making big bets like Emergent Ventures or venture capital — where missing something good is far worse than funding something bad — I believe this book should be at the very top of the reading list. Even for people doing traditional recruiting of back-office managers, I think you will find lots of interesting ideas to consider. I also expect the book to reach a level of popular influence that will mean you won’t be able to talk seriously about recruitment without having at least an opinion about Cowen and Gross’ strategies. (And don’t be surprised if, a few years from now, more than one company will be codifying the book’s techniques into their own hiring processes.)

Among Talent’s most striking ideas, the authors recommend that interviewers should try and get candidates out of their comfort zone and away from prepared remarks. Getting the a better understanding of a candidate is a laudable task, and clearly better than the opposite. But I do not believe it is quite as important as the authors make it seem, which brings me to three places where I disagree with the book:

1. The book’s central thesis is that the talent market is efficient. If you want to find good talent, you either need to pay more than anyone else or find a way to search for talent that the overall market is ignoring.

2. Related to point 1, the authors believe that “intelligence” is “priced in” and “over-rated,” such that if you focus on recruiting people who are smart, you will overpay for the value they generate.

3. The authors believe that job candidates are prepared for most interview questions, so getting them “off script” is the best way to assess what they are really like.

All three ideas are interesting. The data and anecdotes the authors provide are both informative and entertaining. Most importantly, they challenge you to think in a way that most books about recruitment do not. Unfortunately, I think all three points are wrong.

Let’s dive in.

---

The efficiency of the talent market

Talent’s arguments are grounded in the perception that the talent market is efficient.

In my experience, this is far from the truth. Are any business processes — at any modern company — at the frontier of efficiency? Most of the impact I’ve had in my career came from helping companies do simple stuff better. Many companies don’t need high-tech innovation to improve their conversion rate; they need to focus on things like fixing errors on their site and making the purchase process a little easier.

Recruiting is no different. 

I think this quote nicely sums up the book’s core idea: “Keep in mind that everyone is looking for talent, and you need to spot the talent that your competitors cannot.”

In many ways, this concept is obviously true. When everyone is searching for the same thing, you can expect to pay more to get it — maybe in dollars, but also in time, social capital, or whatever else allows you to “win the auction”

Most books on recruitment assume you are fighting for talent, and they usually provide various techniques for beating your competition. Cowen and Gross don’t go down that route; instead, they argue you should find a way to escape the competition, by finding talent in places and in people that your rivals are ignoring.

In a section predicting how AI will play a larger role in future talent searches, they write:

Such technologies would divert your attention toward what the AI can measure well and distract you from the noisier pieces of information. And if many institutions have access to the same AI technologies, as is likely to be the case, it may be precisely the idiosyncratic, noisy pieces of information that become the most important for you.

The argument is straightforward: if AI is widely available, then everyone will use it; as such, your benefits of using AI will disappear, leaving you to find the candidates being overlooked by AI.

This is ‘Recruitment as Moneyball.’ The rich New York Yankee-type companies can outbid other companies to hire the best graduates of Ivy League schools (Microsoft in 1990, McKinsey in 2000, Goldman Sachs in 2005, Google in 2010, Amazon in 2015, and so on). Talent provides the strategy for budget-limited Oakland A’s-type companies to recruit less-sought-after — but still very talented — employees: hire for “On Base Percentage” instead of RBIs.

If corporate talent markets were highly efficient, then the book’s strategies for dealing with recruitment would be fantastic. But I question whether most talent markets operate anything like baseball (or the stock market, Google paid search, or Facebook ads, etc.). At many companies, recruiting processes are terrible and poor decisions are made all the time.

Cowen and Gross also contend that many things (like intelligence) are “priced into” the recruitment process. That assertion is probably my greatest disagreement with the book. I don’t think that many (if any) companies have fully optimized their talent acquisition process to such an extent that they need to venture way outside the box to compete.

Most of the time you can win candidates by getting the basics right:

1. Reach out to people, don’t wait for them to come to you.

2. Build relationships before you need them.

3. Develop followership (so people that work with you once will want to work with you again).

4. Get candidates excited for the job before you start screening them.

5. Make your workplace a good place to work for smart and talented people (which is NOT the same as making it a “good place to work” generally, or anything from the HR/PR lists.)

6. Be the type of manager that top talent will want to work for.

7. Ensure that you have someone selling the candidate once you know you want to make an offer and start the selling process before the offer is made.

8. Be polite.

9. Be fast.

If you take care of these type of basic things, you will out-compete almost every rival — without the need for experimental interview techniques that push people “off script.”

It also helps if you try to recruit really smart people — something that many organizations fail to do.

---

The (lack of?) importance of IQ

Early in Talent’s introduction, the authors stake a clear position:

We are both “fallibilists,” to use a term that has been revived by Irish tech entrepreneur and Stripe CEO Patrick Collison. That means we’re also going to tell you about things you might think you know but that aren’t true. For instance, for a large swath of jobs, intelligence and IQ are far less important than many smart people believe. Discarding faulty knowledge and being open to surprise are two of the most important things you can do to discover previously unknown talent.

Given that the book encourages you to zig when others zag, it’s no wonder they emphasize some of their counterintuitive takes. Cowen and Gross are two very smart people, surrounded by other very smart people, looking for very smart people who are going to change the world. And so, the idea that IQ is not one of the most important qualities to look for a candidate is definitely counterintuitive.

One chapter focuses on proving that IQ is not as important as people believe. On that topic, the authors write, “we’ve concluded that intelligence usually is overrated, most of all by people who are smart.” But they also go on to admit, “...research also suggests some very important cases where intelligence really matters.” The chapter explains that IQ matters for careers like “inventor” or when someone is at the very peak of the market — Picasso, Georges Braque, Henry Ford, Sergey Brin and Larry Page — but Cowen and Gross seem to believe these people are exceptions to the general rule that IQ is overrated.

To support their thesis, the authors list various research findings: that parental education is better than intelligence at predicting which children will pursue law or medicine, that Swedish CEOs of small and large companies are only at the 67th and 85th percentile of intelligence, that IQ is not very useful for predicting an individual's lifetime income, and that the best US presidents were not the most intelligent (and vice versa).

In the end, the authors concede that IQ is a valuable trait for employees, but that it’s “priced into” the recruitment process. In other words, they assume that everyone values IQ with the same level of importance. If you attempt to recruit for IQ, then you will have to pay “market prices”; and so, you are better to target ‘less valued’ characteristics that other companies ignore. (There’s also an implicit assumption that most companies can identify IQ with roughly equal skill).

On the (lack of) importance of IQ, I fundamentally disagree with the authors — and people like Marc Andreesen, whom the book quotes on the subject.

I do, however, believe they are correct that CEOs are not always the most intelligent people in an organization. When I worked for McKinsey, my official mentor once sat me down and drew a chart:

He told me the CEOs we worked with tended to be bright — not as smart as my peers at the consulting firm, but smarter than most of the more junior clients we advised. But those same CEOs were very politically talented. The mentor’s point was that I should not be intimidated by the CEOs being smarter than I was, but I should observe their political skills, and defer to them about HOW to get things done in an organization.

My mentor would not have been surprised by the fact that CEOs are “only” at the 85th percentile in IQ. But he would snicker at the idea that that proved IQ did not matter for consultants trying to achieve change in an organization. And perhaps there are other reasons why Swedish CEOs don’t rank higher in IQ, like a recruiting process that rewards savvy political skills over extremely high intelligence (CEOs are generally selected from the executive suite, where positioning yourself for the CEO spot is likely more valuable than the direct impact you had on the corporation). IQ may not be the most important trait if you want to become a CEO, but that doesn’t mean it is not the most important trait you want to use to hire your next project driver (or your next CEO for that matter).

IQ is particularly important for the sub-type of talent featured in the book. Type 1 talent — running a large organization — requires a smart person who also possesses other traits like perseverance, organization, likeability, vision, etc. But when hiring Type 2 talent — the type of people you need to build NEW organization or products or business processes — I have never felt that any of my directors were “too smart,” such that I would trade some of their raw intelligence for some other attribute.

That’s not to say that soft skills like “working well with others” or hard skills like “digital marketing experience” are not important. But here’s what I noticed about candidates:

1. Those other abilities often came bundled with IQ;

2. When they did not, the high-IQ employee could typically learn those things with good feedback; and

3. The other traits usually just needed to be “good enough.”

Ultimately, success at the manager or director level requires well-roundedness — but the degree of success is often directly related to the person’s level of intelligence.

During my time at Expedia, one of the directors on my team needed to hire a marketing manager to help run an email program. In the end, it came down to two candidates:

• Candidate 1 had five years of experience running email programs. She was personable and charming. She brought a binder to the interview with hundreds of examples of campaigns she had run. Unfortunately, when I asked her how she would evaluate some of her campaigns, she failed badly. I don’t think she understood the math behind a standard A/B test-and-control experiment.

• Candidate 2 was a former McKinsey business analyst, two years out of his undergrad. He was polite and presented himself well, but he was shy and not particularly charming. He knew nothing about email programs, but he was very, very smart. His undergraduate degree was literally in rocket science, and he had interned with NASA.

In the minds of the hiring panel, the decision wasn’t even close. Everyone wanted to hire candidate 1.

But I disagreed. It was of the few times that I overruled my director, pushing him to hire candidate 2. That less-experienced-but-very-smart candidate went on to be a rock star on the team; in fact, the director told me it was the best hire he had ever made in his career.

Examples like this one are not uncommon. Everywhere I have worked, the organization’s hiring processes were tilted in favor of experience over intelligence. Interviews include behavioral questions or assessments of specific skills. Rarely is anyone on the hiring loop running problem-solving sessions that require the candidate to demonstrate how they might deal with the real-world challenges they will encounter in the workplace.

---

A new way to interview

Throughout Talent, Cowen and Gross advocate for better interview processes, with particular emphasis on the idea that you need to get the candidate “off guard” (in an ethical, non-confrontational way). Their logic is understandable: most candidates prepare a “script” of answers for frequently asked questions; listening to them recite that information is less instructive than watching them think on the fly.

The book provides many examples for implementing this strategy:

• Moving the interview out of the office. Take the candidate to lunch, so that you can observe how they interact with service staff. Even better if something goes wrong and you can see what effect that has. (A measure of patience? Respect?)

• Asking unusual questions that learn what the person does in their downtime away from formal work settings, such as, “What tabs do you have open on your browser?” or “Which subreddits or blogs do you read?” or “How do you practice?”

• Asking meta-questions like “How do you think the interview is going?” or “What questions would you ask if you were interviewing candidates for this job?” or “How did you prepare for this interview?”

• Asking for more and more examples that demonstrate the same trait you are interested in identifying. Keep asking until the candidate runs out of examples; the idea is to run through the “prepared answers” and focus on responses the candidate is generating in the moment.

I believe that any of these examples would push most candidates out of their prepared answers — at least for now. The authors note that “knocking off guard” was the point of Peter Thiel’s question, “What do you strongly believe that most think is wrong?” Of course, after that question was printed in Thiel’s popular book, Zero to One, more and more candidates started having answers to the question ready.

Plus, there’s a harder question for the interviewer: “what do you do with the answers?”

Cowen and Gross provide some guidance: For example, interviewers should ask themselves, “Who is this person responding to or used to performing for? Who do they view as important to impress?”

With respect to the practicing question above they write:

You also should try to learn just how self-conscious a person is about what he or she is doing for self-improvement. And if they give you a fumbling or bumbling account of their practice habits, as we have heard numerous times, you can help them out very easily by suggesting they think about practice a little more systematically.

But when the entire point of the interview is taking the conversation in unpredictable directions, how will you know how to evaluate a “good answer?” In the chapter on interviewing via Zoom, Cowen and Gross suggest you should pay attention to the sounds coming from off camera. But — they also write, “We are not suggesting you take any kind of dogmatic stance as to what kind of home life is the right one to look for (should the dog’s bark be shrill and yapping? Deep and guttural? Who knows?).”

I feel that that “who knows?” is how I would respond to many candidates’ responses to questions about their browser tabs, practice habits, or meta-observations.

In another passage about interviews, the authors state:

Most interviewers consider a person’s gait, microexpressions, and interactions with third parties when judging how trustworthy that person really is, whether those judgments are fair or not. And we don’t know how accurate those signals really are.

I agree! Most people cannot accurately assess whether a candidate’s gait or “microexpressions” would make them a good VP of Operations. And yet, these types of qualitative assessments are exactly what Cowen and Gross propose the reader do in other parts of the book — even specifically suggesting taking the candidate to a local coffee shop to see how they “interact with third parties.”

How do they square these ideas? The authors explain that academic literature indicates that qualitative techniques do NOT lead to better hires, BUT that might be attributed to the average interview’s lack of sophisticated skills and insights. In other words, the authors believe their own abilities are better than average, and so the literature does not apply.

I agree! Cowen and Gross both have track records of finding early-career people who went on to achieve incredible success. But will average readers of the book enjoy the same type of success?

Here’s an anecdote about the challenges of incorporating “book advice” like this into real-world practices. Years ago, I read Geoff Smart and Randy Street’s book, Who? A Method for Hiring (which outlines the methodology that Warburg Pincus uses to recruit CEOs and senior executives). The book was full of new, interesting ideas for me to consider. I immediately tried to implement the book’s ideas during my next hiring process.

The first interview was a disaster.

The actual interview wasn’t the issue; it was the aftermath — I couldn’t gather a clear impression about whether the candidate was the right person to hire. In retrospect, I think the book’s techniques would have been more valuable if I had more time to reflect, more opportunities to practice, and maybe a coach to advise me. With all those pieces in place, then perhaps my interviewing success would have been dramatically improved by using the book’s tactics.

But in the real world, I did not have that luxury. When a company needs a new CFO (or a new director of marketing, leader of the call center, etc.), then hiring a high-quality candidate matters. I’ve never been in a situation where I had so many qualified candidates that I could afford to experiment on a few dozen to work the kinks out of my interviewing techniques. Every single interview mattered — either to decide on the candidate and start selling, or to rule someone out beyond a doubt.

In practice, then, I think new interview techniques need to be incorporated gradually. You need to make incremental improvements in how you assess candidates. Otherwise, the errors you make will be too costly.

The process of careful iteration might be less important if you are looking for talent like Cowen and Gross. In both cases, the authors are evaluating candidates and thinking about the power law. They are not so much looking to avoid disaster as they are hoping not to miss grand slams. For both Cowen and Gross, the constraint is talent, not cash. If they could find more superstar (or potential superstar) candidates, they could probably “find” more money to back them. But when you are looking to fill a specific role in your organization, you are operating from different constraints. We’ve all seen companies brought to their knees due to one bad hire. Unlike the world of venture capital, the downside for most of us is very real.

So, if you are going to use the techniques from this book, I would recommend caution. Maintain your current hiring processes and then incorporate one or two of the book’s ideas at a time. After the interview, conduct your normal assessment, and then ask yourself how (or if) the new questions or techniques helped inform your decision. Where they helped, try and add that method into your standard process; where they didn’t, either adjust or try a different technique from the book. But to take the authors’ advice wholesale is more of a risk than most executives should assume.

---

Now what?

Throughout this essay, I’ve questioned Talent’s reliance on an efficient hiring market. To be clear, I do think that the talent market is somewhat efficient.

Everyone I’ve ever worked with has grown tired of me telling them that for any given price point there is a trade-off between hiring experience or intelligence. The market is efficient enough that if you have two candidates in front of you at a given title and salary level — both recruited via the standard ways — then the one with more experience is almost always the one that is less intelligent. And sometimes, for some roles, you need that experience. You don’t want to hire a really smart generalist if you need a barber or a butcher (or even if you need something like an SEO specialist). But for most roles in the business world, skills and experience are not nearly as important as IQ — especially for roles that require building new things.

I believe the talent market THINKS that it’s efficient, in that there’s a trade-off between IQ and experience. But that trade-off is unbalanced, and very intelligent, junior candidates are still a bargain at market prices. 

You can do very well in recruiting talent by following three principles:

1. Get the basics right.

2. Aim to recruit really smart people (even if they possess less experience).

3. Don’t throw out all your current processes to chase the newest trend.

Fortunately, covering those basic principles is relatively simple — at least conceptually. The challenging part is operationalizing the principles at scale (how do you ensure every candidate is informed of their status a few hours after the interview on a consistent basis?). Evaluating IQ is also straightforward — or at least easier than “really understanding the candidate” after knocking them off their script. There are LOTS of standardized tests that do a pretty good job of figuring out someone’s general intelligence (do you ask candidates for the SAT, GRE or GMAT scores?). Or you could do what I do, which is spend your interview time presenting the candidate with a problem and observing how they think through solving it. At the end of the interview, you will have a very good idea of how smart they are. And as you run more candidates through the same problems, you will have a very good idea how well an individual ranks versus other candidates you have interviewed.

I am less certain I could force rank how good an employee will be based on knowledge about which tabs they have open on their browsers.

Keep it simple,

Edward

Edward Nevraumont is a Senior Advisor with Warburg Pincus. The former CMO of General Assembly and A Place for Mom, Edward previously worked at Expedia and McKinsey & Company. For more information, check out Marketing BS.

Footnote

In The Average is Over, Tyler Cowen describes the incremental development of chess AI. The AI programs went from barely being able to play the game, to challenging grandmasters, to surpassing the best humans in the world. Cowen points out something that I hadn’t previously been aware of: there was a period of time when chess AI programs could beat all humans, but they were still beatable by humans that worked in tandem with AI programs. (The competitions were called “Freestyle Chess”).

That window was still open when the book was published (2013) but has since closed. Most AI is not as advanced in its function as chess AI, so the idea is that there is an extended period of time in any particular field when AI will be good enough to make humans working in isolation obsolete, but still allow humans who know how to work WITH AI to be more efficient and productive than any time in the past. GPT-3’s progeny may eventually eliminate the need for novelists, but well before it does so, humans will be working with deep learning language models to write books and screenplays far faster than any time in the past. (Here is a video of David Shapiro spending an hour building a tool on top of GPT-3 that generates movie scripts. Here is his attempt to have GPT-3 write a novel. Note that the results aren’t great (yet), and that even getting these results requires a lot of human thinking. Also see Sudowrite for a tool that increases the productivity and creativity of novelists — it’s like a grammar check on steroids).

I have used the freestyle chess example many times in writing and in casual conversation whenever the idea that “AI will take all the jobs” comes up. And as famous and widely read as Tyler Cowen is, not once has anyone previously been aware of the freestyle chess example or made any sort of connection to Cowen (unless I mention it myself when I give the example).

I share this example both to demonstrate the respect and reverence I have for Cowen, and to show that, as influential as he is, his ideas have traveled far slower through the wider world than those of us who religiously read Marginal Revolution would think. A related (non-Cowen) example: I was at an event for CMOs of Warburg’s portfolio companies a few weeks ago. The CMOs were all very successful people overseeing massive, rapidly growing technology companies. In every casual conversation I had, I asked how they were thinking about GPT-3 and NLP models. Not a single CMO had even heard of GPT-3, let alone thought about how to apply the capabilities to their businesses. Maybe that will change now that the topic made the cover of The Economist. But if so, it will only move the needle a tiny amount. Most people are focused on blocking and tackling and have no capacity to think about non-traditional ideas and technologies - or optimizing interview techniques.