Product Market Fit (essay)

Things change, but mostly they stay the same

OpenAI launched GPT-3 in beta on June 11, 2020. In November 2021 the company removed the waitlist and anyone could begin using the models. Many did, but most did not. The product was Earth-shatteringly powerful, but most people did not seem to care. From Wikipedia:

The quality of the text generated by GPT-3 is so high that it can be difficult to determine whether or not it was written by a human, which has both benefits and risks… [OpenAI researchers and engineers] warned of GPT-3's potential dangers and called for research to mitigate risk. David Chalmers, an Australian philosopher, described GPT-3 as "one of the most interesting and important AI systems ever produced." An April 2022 review in The New York Times described GPT-3's capabilities as being able to write original prose with fluency equivalent to that of a human.

This was a product that was one of the most interesting and important to ever be produced. It could write so fluently that most people could not tell the difference between it and a human. The technology was so powerful employees at the company were writing open letters calling for research into it’s risks and dangers. And yet most people never bothered to even try using it when it was offered publicly for free.

OpenAI was founded as a research institute, and in 2021 that is what it still was. They thought they were building a technology that other companies could leverage to build products. And some products were developed. Jarvis.ai used GPT-3 to help marketers write content. Sudowrite used GPT-3 to create a fiction writing assistant. And most famously Microsoft used GPT-3 to create co-pilot for developers. But mostly nothing happened.

In an October, 2022 interview with Ben Thompson, Daniel Gross and Nat Friedman (gated unfortunately) lamented that while generative AI technology was advancing very quickly, there were very few products being built with the technology. Here is Nat from the interview:

So I left GitHub thinking, “Well, the AI revolution’s here and there’s now going to be an immediate wave of other people tinkering with these models and developing products”, and then there kind of wasn’t and I thought that was really surprising. So the situation that we’re in now is the researchers have just raced ahead and they’ve delivered this bounty of new capabilities to the world in an accelerating way, they’re doing it every day. So we now have this capability overhang that’s just hanging out over the world and, bizarrely, entrepreneurs and product people have only just begun to digest these new capabilities and to ask the question, “What’s the product you can now build that you couldn’t build before that people really want to use?” I think we actually have a shortage.

The shortage did not last long. Less than two months after the interview (Nov 30, 2022), OpenAI released ChatGPT (and upgraded GPT-3 to GPT-3.5). The result was the fastest growing consumer product in history.

GPT-3.5 was much better than GPT-3, but it was an incremental improvement. It was easier to work with and could do some things GPT-3 could not do (like rhyme), but the explosion of interest wasn’t due to the better technology, it was due to the better interface.

GPT-3 playground did not have product market fit. ChatGPT did have product market fit (maybe the best product-market fit of any product in history).

Once ChatGPT showed what was possible with generative AI there has been an explosion of new companies and products using the tools. Most of the products are terrible - either they do not work, or they do something that is just not that useful to most people (i.e., they do not have product market fit).

Right now there are only two products that have nailed mass consumer product market fit:

• ChatGPT itself; and

• Co-pilot (for coding)

There are some other products that have found fit within some sub-segments (I have personally invested in Spellbook, a legal contact drafting AI tool, that seems to have found fit within that niche community; It could be argued that Sudowrite has found fit within the writer community; there are likely other niches out there), but mostly entrepreneurs and intrepreneurs are throwing products against the wall looking for the next mega-hit.

One area that many believe is the future is chatbots. Chatbots have been around for years now but are mostly terrible. Most people would rather use a search engine rather than a chatbot to make a purchase. But ChatGPT showed that a chatbot-like interface is a great product for generative AI. Could specialized versions of chatbots be the next mega-consumer hit? When Bing launched AI-powered search, most of the buzz came from the (sometimes unhinged) chats with “Sydney”. If someone builds for chat-entertainment (and chat-edutainment) FIRST instead of as a side-effect, maybe that is the future?

SnapChat thought so.

On February 27, 2023 Snap launched MyAI for their premium paid users. The pitch was an AI chatbot that could be communicated with through SnapChat itself. From their launch page:

My AI can recommend birthday gift ideas for your BFF, plan a hiking trip for a long weekend, suggest a recipe for dinner, or even write a haiku about cheese for your cheddar-obsessed pal. Make My AI your own by giving it a name and customizing the wallpaper for your Chat.

AI analysts were optimistic — this is Bing’s “Sydney” as feature not bug.

Last week Snap expanded access to the Chatbot from premium users to all users worldwide, and pinned the bot to the top of the “Chat tab” (ironically the chatbot cannot be removed unless you are a premium paid user).

So how is it going?

From Techcrunch, April 24th:

Over the past week, Snapchat’s average U.S. App Store review was 1.67, with 75% of reviews being one-star… For comparison, across Q1 2023, the Snapchat average U.S. App Store review was 3.05, with only 35% of reviews being one-star.

[Another provider’s] analysis shows “AI” was the top keyword in Snapchat’s App Store reviews over the past seven days, where it was mentioned 2,973 times. The firm has given the term an “Impact Score” rating of -9.2 [on a -10 to +10 scale]…

…The company says it’s constantly iterating on Snapchat’s features based on the community’s feedback but did not commit to removing the AI. Instead, a Snapchat spokesperson said if users didn’t like the AI feature, they don’t have to use it.

The negative sentiment may be over-stated. People hated the Facebook newsfeed when it launched as well. But it sure doesn’t look like perfect product-market fit.

There are other, smaller, companies trying to make relationship-like chatbots work. The one with the longest tenure is Replika (founded 2014, long before generative AI became a thing). The latest is an app from the consulting firm “In Love With” which steals a brand name (and a concept) from the Spike Jonze film “Her”.

Maybe relationship-type chatbots will become the next killer-app of generative AI, but after the Snapchat results it is looking less likely — at least as a mass market product. There is still hope that a chatbot-type interface will work as a copilot interface in business or education, but it looks like, at least for now, most people would rather talk to their friends than a robot. I think that is good news?

Keep it simple,

Edward