Now | Latest squakings

Following on from my previous quick post, and reading yet another Benedict Evans article, this time on the puzzle of generative AI adoption. His argument is that chatbots are unlikely to be the end product—they’re more of a demo layer, waiting for someone to build the “real” apps on top. That is something that I do largely agree with and mentioned in my previous post, but I can see the other side of this as I argue below.

GenAI’s adoption puzzle — Benedict Evans

Generative AI chatbots might be a life-changing transformation in the nature of computing, that can replace all software, but so far, most of its users only pick it up every week or two, and far fewer have made it part of their lives. Is that a time problem or a product problem?

Benedict EvansBenedict Evans

I want to ask the question again: what if the chatbot itself is the product?

Relational databases changed the world, but for most people they became invisible. They’re the plumbing that sits behind every piece of software. The only people who treat them as products are the engineers who build on them.

Excel, though, went another way. It’s as fundamental as a database, but it never disappeared into the background. People use it raw, straight out of the box, at wildly different levels of skill. Some just keep household budgets, others build businesses with pivot tables and macros. Excel is the product.

That’s the analogy I find most interesting for chatbots. Yes, they’re still flaky and limited. But with memory, projects, canvas views and better interfaces, it’s not hard to imagine a world where people use them directly. Not hidden behind other apps. Just the chatbot as the thing.

If that’s true, then maybe we’re not waiting for the “killer app.” Maybe the chatbot already is one—a new general-purpose interface, like a spreadsheet you talk to instead of click on.

So why hasn’t it happened yet?

Part of the answer may be cultural. When Excel launched in the 1980s, tools arrived finished—or at least polished enough to feel complete. With AI, companies are showing unfinished versions, moving fast, shipping in public. Adoption looks huge on paper, but in practice people are still testing, waiting for the product to settle down.

It may simply take time. Excel wasn’t instantly a workplace staple—it grew over years. Chatbots might need the same. Or maybe they really will fade into the background, embedded in other apps.

But I don’t think we should dismiss the possibility. Sometimes the raw tool is the product. Excel was. Chatbots might be too.

AI metrics — Benedict Evans

With every platform shift, we want to measure the growth but we’re confused about what to measure. That’s partly a problem of data and definitions, but it’s really a question about what this is going to be.

Benedict EvansBenedict Evans

I really admire Benedict Evans’ work, and his latest article is another great one. He looks at how AI uptake is measured, and more importantly, what those metrics actually mean. At the end he asks:

"the real question, as I’ve hinted at a couple of times, is how much LLMs will be used mostly as actual user-facing general-purpose chatbots at all or whether they will mostly be embedded inside other things"

I’ve been using LLMs more and more, and for many use cases treating one like “search on steroids” is enough. It delivers results fast — formatted tables, sources, links — and there’s that moment of relief: “Yes! The AI did all the work!”. Then comes the crash: broken links or claims that don’t hold up, and suddenly you’re verifying everything manually.

But that’s just Level-0 stuff. Where LLMs become really interesting is with deeper, multi-stage prompts. That’s also where many people lose traction. If you don’t put in the work — learn how the latest models behave; discover what prompt styles get results — you won’t get close to their potential.

A good example that comes to mind: CV-checking tools that promise to help you beat the ATS (Applicant Tracking System). Yes, you could write a detailed prompt yourself, feed in the job description and your CV, get feedback from GPT or Claude directly. But a lot of people prefer something simpler: upload your CV, click a button, see a slick chart and some step-by step improvement instructions. Easy. Fast.

At the moment my prediction is that for many, AI chatbots are still only wonderful and unexpected novelties. Only the hardcore users are using them close to full potential. I don’t have hard data to prove it, but I believe we’re still far from a world where AI fully, or even substantially replaces people. As William Gibson put it:

"The future is already here – it's just not evenly distributed"

I think it's the same for any tech in the business world. However revolutionary and productivity-enhancing a new technology is it takes time for it to mature and be adopted by organisations. Anyone who has been through a protracted digital transformation or system migration knows just how hard it can be, even if everyone agrees the change is worth doing.

Of course I could be completely wrong and we'll get to AGI in 2027, machines cross the Rubicon, self-improve, foom, we’re done. But for now I still think we are somewhere before the peak of inflated expectations on the hype cycle and people are way too excited.

Time will tell —unless the foom gets us first!

This a great post I came across on substack, check it out if you have 10min to spare. https://cmarmitage.substack.com/p/i-researched-every-attempt-to-stop

It is the middle of August and the wettest it has been for quite some time. Setting up this blog is in some ways easier than I thought, but all the little bits take time. Also, how does anyone ever make a decision on what to post? So many things to choose from, it's hard not to over-think things.

Anyway, in the middle of all that of that I am currently enjoying a 2022 album by New York-based saxophonist Oded Tzur called Isabella. Really amazing stuff that is a combination of middle-eastern / classical Indian music and modern Jazz. Check it out!