Paid Search from the cutting room floor
Thank you for all the warm wishes after last week’s post. Many of you assumed that I am stopping Marketing BS entirely. I miscommunicated. I am pulling back on Marketing BS. Instead of four posts per week (Essay, Briefing, Two-part Interview), I am moving to two posts per week (Essay OR briefing, half of the two-part interview). Non-paying subscribers (or which there are far more of you than paying subscribers) will not notice any difference at all!
This reduced workload has allowed me to move ahead on two things:
I am moving much faster with the Marketing BS book. Once the book is out, I may increase my frequency again. So stay tuned.
I have launched another podcast with my friend Michael Kealy. We dropped our first four episodes yesterday. Most episodes are under 10 minutes. We consider the implications of the traditional “Marvel Universe” of 1961 being “real”. It has been a ton of fun. Episodes drop every Monday, Wednesday and Friday. Here is the first episode.
In the meantime, this week’s essay is a chapter on paid search that was cut from the book. I think the content was good, but is was less “fun” than the rest of the book, so out it goes! Rather than throw it away entirely I am sharing it here. The lessons on “how to do paid search correctly” were very novel ten years ago. Now more and more companies have figured it out. But many more have not. Every acquisition I have worked with in the last two years at Warburg Pincus needed help moving to this model. If you already know this, good for you! But if not, it is one of the most impactful things you can do for your business.
Enjoy!
(If you enjoy this, know that EVERYTHING in the book is better than this. At least marginally better - or it would have been cut too!)
Refining Paid Search
If you are selling luxury red umbrellas in Seattle, and people are searching for “luxury red umbrellas in Seattle” on Google, you should run an ad targeting “luxury red umbrellas in Seattle” and your ad copy should read something like, “Luxury Red Umbrellas in Seattle”. If you can do the equivalent for your industry you will be better at paid search than 90% of the companies I encounter. Ten years ago I would have said “better than 99% of the companies I encounter”, so companies are getting better at this, but it has been a very slow progression.
Why has this market inefficiency lasted as long as it has? The idea that you should directly target the keywords people are typing in looking for your product and then text search ads mirroring those same words, claiming you have the products or services they are looking for in the pinnacle of simplicity. Why isn’t everyone doing these simple things? The answer is a case study on all the things written about in this book. Let’s start with why the simple method works so well and then explore why so few companies are doing it.
Google paid search is an “Dutch auction”. Anyone who wants to appear at the top of the search results chooses what terms they want to bid on and how much they are willing to spend. In a simple Dutch auction Google would rank order all of the bids for any given term and whoever was at the top of that list -whoever bid the highest amount they were willing to pay per click - would be given the first position. Instead of paying what they bid (like they would in a traditional auction), they would pay whatever the second highest bid was. The second place bidder would appear second and pay whatever the third place bidder was. As a bidder, because you always pay less than you are willing to spend, the “correct” strategy is to bid your reservation price. In the worst case scenario (when you bid exactly matches someone else’s) you will pay exactly what you bid, but in most cases you will always pay less than your bid. If you bid less than your reservation price then you may lose to someone else when you would have been happy to pay more to get the top position. The Dutch auction method is powerful for Google in that it encourages rational buyers to reveal the true highest amount they would be willing to pay. There is no need to negotiate - or pay high cost negotiators. Google just turns the algorithm loose and sits back and watches the money flow in for every search through their website.
But Google’s primary financial motivation is not to maximize the revenue they receive per click. Clicks are just the method by which they sell what they do have - “searches”. If Google was forced to choose between a business offering $100 per click who generally gets 1% of searchers to click on their link and a business that only offers $10 per click, but has a “click through rate” of 30%, the second business makes more money for Google: $100 x 1% = $1 per search; $10 x 30% = $3 per search. The simple Dutch auction method would put the $100 bidder on top, which would not maximize Google’s revenue per search. So Google uses a “modified Dutch auction”
The modification is something Google calls a “Quality Score”. Every time someone searches on Google, the company pulls every possible advertising for that term and gives each ad a real-time quality score for that particular ad. The algorithm is trying to estimate how relevant that particular ad is for that particular search and assign a number from zero to ten (with dozens of significant digits). Every bid is adjusted based on that quality score, and the bidder with the highest adjusted bid appears on top. Instead of paying the second highest unadjusted bid, they pay the second highest adjusted bid, re-adjusted based on their own quality score. This is best explained with a simple example:
Bidder A: $100, Quality Score 2.0
Bidder B: $99, Quality Score 2.0
Bidder C: $50, Quality Score 5.0
Bidder D: $25, Quality Score 9.0
First we adjust all the bids - just multiply the bids by the quality scores:
Bidder A: $100 x 2 = $200 adjusted bid
Bidder B: $99 x 2 = $198 adjusted bid
Bidder C: $50 x 5 = $250 adjusted bid
Bidder D: $25 x 9 = $225 adjusted bid
Then we force rank the three bidders based on their adjusted bids:
Bidder C: $50 x 5 = $250 adjusted bid
Bidder D: $25 x 9 = $225 adjusted bid
Bidder A: $100 x 2 = $200 adjusted bid
Bidder B: $99 x 2 = $198 adjusted bid
To figure out how much the top bidder (Bidder C) pays, we take the second highest adjusted bid (Bidder D, $225), and re-adjust it with Bidder C’s quality score (5.0). $225/5.0 = $45. So Bidder C takes the top sponsored listings spot and pays $45 per click ($5 less than their bid)
The second spot goes to Bidder D. Bidder D pays the Bidder A’s adjusted bid ($200), re-adjusted by Bidder D’s quality score (9.0), resulting in their cost per click being (200/9) $22.22 (a little less than their unadjusted bid of $25).
Bidder A takes the third spot priced at Bidder B’s bid re-adjusted by A’s quality score (note that since A and B both have the same quality score the “adjust” and “readjust” just cancel out). A pays ($198/2) $99 per bid.
The math here is pretty simple, but there was a lot of it, so it is okay if you did not fully follow what was going on. The key point is that Bidder A, while having the highest bid only appeared in the third position paying $99 per click. Meanwhile the winner of the auction who appears in the top spot is only paying $45 per click, and the second position on the page is only paying $22. Poor Bidder A is paying more than four times as much as Bidder C and appearing below them on the page.
Since the majority of searchers click on the top of the search results (roughly 45% of the paid search clicks go to the first result with almost a 40% decrease per position), the way to get more traffic is either to bid higher or to improve your quality score. Bidder higher obviously leads to worse margins and accelerating diminishing returns. Higher quality score is, if you can achieve it, “free money”.
All of which begs the question of, “how does Google determine quality score?”. The most obvious factor is “position dependent click through rate”. Google knows that the average ad in the first position gets about an 8% click through rate (some people click on lower paid results, some on the organic listings and some choose not to click on anything and do another search). If they show an ad in that leading position that gets less than an 8% CTR, they know the ad is “below average” and can reduce that ad’s quality score for that specific keyword search. If it gets higher than 8%, the algorithm can adjust the quality score upward. Over time Google “AI” will learn (or at least estimate) the relative click through rates of every ad for every type of search. Since Google’s revenue depends on BID x CTR, it is not surprising that the majority of the value of the quality score comes from what Google thinks your ads CTR will be.
But it is not ALL click through rate. To steal from the Goldman Sachs motto, Google is not short term greedy - they are long term greedy. They do not truly care about maximizing revenue per search - they want to maximize revenue per potential searcher on the Google network. If someone clicks on a Google ad and has a great experience (they got what they were looking for) maybe next time they search they click on an ad again. If their experience with paid ads is terrible, they will (sooner or later) learn to avoid the paid ads and scroll to the organic listings (where Google does not make anny money). Google wants to make the experience of clicking on a paid link, if not pleasurable, then at least valuable and helpful. So Google looks for signals that a link will be valuable for the searcher.
Some of those signals are based on user behavior. If people are clicking on a link and then “bouncing” back to search again, there is a good chance the website did not “meet the customers needs”. Google can penalize that ad with a lower quality score so less people will have that (poor) experience in the future. Google can even scan the words on the landing page after the click to see if it is relevant to the search. Google’s algorithm does do those things, but the majority of the impact on quality score uses a much more simple and effective metric.
If someone searches for “luxury red umbrellas in Seattle” a lot of different companies may bid on the term. Maybe there is a speciality retailer in Seattle that sells varieties of umbrellas, specifically “luxury red” ones. They may attempt to target the term “luxury red umbrellas” directly. They could look for exact matches for those specific keyword searches, and when they find them they run ads talking about all their varieties of luxury red umbrellas at their six Seattle locations. Other companies might just be national retailers of umbrellas. Maybe Target sells a hundred thousand items including umbrellas. Someone on Target’s paid search team sticks all of their SKUs (stock keeping units) into a database and runs ads against any keywords that match a product category they sell. Target is targeting anyone that types in the word “umbrella” and serves an ad talking about how many different things Target has.
Does Target have luxury umbrellas or red umbrellas or Seattle locations? Maybe. Or maybe not. But Target is dealing with so many products they do not have the time or inclination to build ads specifically for umbrellas - let alone luxury red ones in the Pacific Northwest. So when someone is looking specifically for luxury red umbrellas in Seattle (which they likely are given that was what they typed in), they would be much better served by the local speciality retailer who has exactly what the customer is looking for vs a mass market retailer that is trying to meet everyone's needs and may or may not have the specific solution the searcher really wants.
If Google wants to maximize the searcher positive experience they will want to show the ad for the speciality retailer above the ad from Target - which means giving the speciality retailer a higher quality score. Part of this will be achieved with click through rate. The user will see one ad for “Target: we have everything” and another ad for “We specialize in luxury red umbrellas in Seattle” and presumably have a higher likelihood to click on the latter. But maybe Target’s sophisticated marketing team finds a way to “trick” the searcher to click on the Target ad (“20% off ipads!”), Google may want to give an additional boost to the companies that look like they are trying to best meet their searcher’s needs. How does Google’s algorithm know? The obvious ways are:
How targeted is the search? If Target wants their ad to appear broadly to anyone who is using the word “umbrella” and the speciality retailer has a specific ad for “luxury red umbrellas” there is a good chance that the speciality retailer has a solution for that specific search while Target is just “fishing” and hoping something sticks
How well does the ad copy match? If Target’s ad copy says “we sell umbrellas” or even worse, “Shop Target for deals” it is unclear they have the solution customers are looking for. If the speciality retailers ad matches the search terms - “Luxury Red Umbrellas in Seattle available at 20% off”, then there is a good chance their solution meets the customer where the customer is at
Which means the way to get a high quality score on Google’s search advertising platform is:
Run ads directly targeting every term you want to appear on (instead of general “broad matches”
Create ad copy that matches (as closely as possible) exactly those keywords
If you follow those two rules you will have a high click through rate, and even higher quality score based on that click through rate - which means you will appear higher on the search results for any given bid you are willing to make - which means you will get more volume than your competitors for lower cost. Which is the point of all of this.
So if it is that easy, why doesn’t everyone do it?
First, many people just don’t know. No one told them they needed to do these things. No one told them how Google thinks. It’s not particularly obvious this is how the world of paid search advertising would work until it is explained to you.
Second, even after it is explained there is often resistance. Google offers all sorts of tools that make things easier that they recommend. Instead of doing the hard work of figuring out what keywords to target and what ad copy to build, you can put yourself at the mercy of Google’s AI who will automate putting in the “right” targeting with the “right” ad copy and even help automate your bids to get you the “right” ROI on your spend. Google ad reps (salespeople) will tell you that their AI is very sophisticated and much better than doing the work yourself. They are particularly right. Google’s AI IS better than what most companies and agencies are doing to optimize their Google paid search advertising. Most companies are doing what the hypothetical Target example was doing: running broad ads and showing ad copy featuring the name of their brand. Google’s AI does much better than that.
But compared to building out a full list of all the keywords you want to target and creating ad copy matching each one, Google AI fails dismally (at least it did last time I tried it - our performance dropped about 20% vs our optimized account structure). Google’s sales people are (rationally) focused on getting the worst performers to be “pretty good”, not trying to make everyone awesome. Part of the reason is that being awesome is, while not conceptually difficult, operationally very challenging.
Which brings us to reason three. If you are only selling “luxury red umbrellas in Seattle” it is not very difficult to come up with all the keywords you want to target and build out the ads and ad copy for those keywords. But what if you sell, in addition to “luxury” a twenty “tiers” of umbrellas (cheap, affordable, average, quality, velvet, etc.), in a hundred different colors in four hundred different cities. Now, instead of one set of keywords to target you have (20x100x400) 800K keywords. And if your speciality store has twenty different products with similar levels of variety that would mean (800K x 20) 16MM ad groups and unique ad copy to write. No human being is going to do the manual work to do that. Instead you would need to develop databases that semi-automated the process. Then, once you had your account structured uploaded to Google, you would need to find a way to monitor the performance of all of those ads to determine the relative effectiveness of each one. You need to catch errors, block “bad searches” (like “f*ckin’ terrible Seattle umbrellas - which is unlikely to convert into a sale), and figure out where you should be increasing or decreasing your bids. This challenge is multiplied by the low search volume you will have on any given term. You might only get a million clicks a month on your 16MM keywords - most keywords will not receive any clicks, and measuring the “conversion rate” on the clicks you do get will be meaningless on its one. If a keyword gets a single click and that click turns into a sale, you have 100% conversion. If a keyword gets a single click and it doesn’t result in a sale you have 0% conversion. That tells you next to nothing about the quality of those two different ad campaigns. So you need to find a way to aggregate the data: Combine different groups of ads together to measure performance vs an index. Maybe Seattle terms convert 10% better than Denver terms. Maybe red products do 4% worse than green products. Maybe “Luxury” searches result in 2x the revenue as “cheap” searches. You need to build your structure so that you can pull out those insights and then find a way to monitor all of those combinations on a regular basis to identify any changes as soon as possible.
Doing all that is a lot of work and requires real expertise. Or you can just do what Google tells you to do and trust their AI. Or hire an agency who will tell you they are experts, and who will then trust Google's AI for you.
Every marketing activity has a story like Google’s paid search. There is almost always a way to do it “well” that is not conceptually hard, but exceptionally difficult or “effortful” to execute in practice. Once you have a marketing channel “at scale”, but often not before, it is almost always worth it to put in the time and effort to refine the channel and make it better.
Search. Scale. Refine.
Keep it Simple,
Edward