AI Creativity

Better than the LSAT

The problem with my new plan of keeping this posts short and focusing on one this per day is that there is more than one amazing thing happening every day in AI — even when I restrict myself to just marketing-related opportunities.

This paper from Cornell Univerity’s Jennifer Haase and Paul Hanel tried to measure the creativity of different AI LLMs. The test had humans and different AI models generate “ideas”. Those ideas were then evaluated by humans (and other AIs) on the quality and the quantity of the ideas. My prior belief was that humans would win on average quality, but AIs would be orders of magnitude better on quantity. When I use AIs to help write comedy I find the value comes from having the tool generate dozens or hundreds of ideas. Then I read the ideas and get inspired to write something (hopefully) better. Since a lot of creativity exercises start with generating a lot of ideas, I find AI really useful in that first step.

But Haase and Hanel found that not only did the AI models outperform on quantity, some of the models — specifically GPT-4 — outperformed on quality as well. GPT-4, the best model they tested ranked in the top 9.4% of human creativity. That compares to the top 7% in SAT reading, 11% in SAT math, 1% in GRE verbal, 20% in GRE quantitative and 12% in LSAT.

Next time you need to think up a creative marketing idea, not having GPT-4 in the process is leaving a top-10% contributor out of the discussion.

Keep it simple,

Edward

P.S., This past weekend a friend asked me to write jokes about a friend’s 55th birthday incorporating wooly mammoths. I only had about 10 minutes, so I had GPT-4 write some for me. Here was the result:

People think that [my friend] is like a wooly mammoth, but that’s not true. Being extinct doesn’t make you that inactive.

But I guess they both have fond memories of the last Ice Age. At least mammoths don’t need to worry about wrinkles.

And [friend] is definitely smarter than a mammoth. Mammoths were constantly migrating to find some place warmer, but they never even found Florida.

Scientists are trying to bring back mammoths, but no way that is ever going to happen. It’s too hard. And they haven’t even managed to engineer a comeback for the Backstreet Boys.

Really though, the biggest difference is that mammoths leave lasting impressions in ice, while [friend] leaves lasting impressions in our hearts.