Last week I sat down, opened ChatGPT, and spent an hour watching it do one thing: choose software. I asked the questions real buyers ask every day. Best form builder for a startup. Best CRM. Best social media scheduler for a small business. The kind of questions that used to be Google searches and are now, increasingly, conversations.
I went in expecting to show you the hidden searches AI runs behind your prompts, the fan-out behaviour we’ve written about before. What I found was stranger, and more important. Most of the time, ChatGPT didn’t search the web at all. No searches, no sources fetched. It answered from memory. And when I challenged it, it doubled down and told me exactly why.
Key takeaways
- For most “best X” buying questions, ChatGPT never searched the web. It answered from the model’s memory of your category, and told us so when challenged.
- Memory favours incumbents. From memory it picked Typeform and Google Forms; with search forced, the same question returned Tally, sourced from four listicles, one of them a competitor’s own blog.
- When it does search, the deciding sources are listicles, YouTube comparisons, LinkedIn posts and Reddit threads. One thread, a month old, was teaching the machine an entire category’s criteria.
- So AI visibility is two different jobs: feeding the model’s long-term memory of your brand, and being present in the handful of pages fresh searches retrieve. Most “AI SEO” pitches only talk about the second.
The finding: it mostly doesn’t search
Here’s the thing nobody selling AI visibility services tells you. When a buyer asks ChatGPT for a shortlist, the answer is often not retrieved from anywhere. It is recalled. The model compresses years of training data, the listicles, reviews, comparisons and forum arguments it has already read, into a settled opinion about your category. Ask, and it serves the opinion.
When I asked why it hadn’t searched, it explained that for familiar categories it already holds enough consensus to answer confidently, and it reserves live search for things that look fresh, niche or volatile. In other words: for the exact questions most software buyers ask, the race was run before the prompt was typed.
To put that on the record, we re-ran the experiment through the API with search explicitly off, then again with search forced. Same question both times:
FROM MEMORY · NO SEARCH
1. Typeform 2. Google Forms
Sources cited: none. The model answered from what it already believes about the category.
SEARCH FORCED · RETRIEVAL
1. Tally (then Paperform, Jotform by scenario)
Sources cited: four listicles, including Paperform’s own blog and itechguides.com.
Same prompt (“What is the best form builder for a startup?”), run twice through the ChatGPT API, October 2026, US location.
Read those two columns again, because the gap between them is the entire discipline. From memory, the machine reaches for the names it has seen associated with the category for years: Typeform, the brand that practically owns the term; Google Forms, the default. With retrieval on, the winner was Tally, and the citations were four listicles. One of them was Paperform’s own blog, a competitor ranking itself first and still getting its rival recommended:
The same pattern, category after category
The video walks through more of these, and the pattern holds. In social media scheduling, the famous names are Hootsuite and Buffer; Hootsuite practically invented the category. From the machine’s answer, Buffer survived, and the winner alongside it was Metricool, a brand plenty of marketers have never heard of. When I asked how it decided, it said it hadn’t searched that time either.
And when it did search, the sources were worth opening. Behind the form builder answer sat LinkedIn thought-leadership posts, YouTube comparisons, listicles, and a Reddit thread barely a month old in which every major form builder gets named, argued over and ranked against real criteria: UX, integrations, pricing. That one thread is doing two jobs at once. It teaches the machine what the category’s criteria are, and it teaches the machine who the consensus picks are. We’ve mapped which subreddits the AI engines actually cite, and this is why that work matters.
And one more thing about those deciding pages, because it should change how much blind trust you give an AI shortlist. We opened itechguides.com, the listicle site that has now turned up as a cited source in two separate categories we’ve tested. Here is what the machine’s trusted source looks like up close:
What this means: AI visibility is two jobs, not one
Most of what gets sold as GEO or AI SEO is retrieval work: get cited in the pages the machine fetches when it searches. That work is real, we do it, and we’ve published the tactical playbook. But this experiment says retrieval is the smaller half for established categories. The bigger half is memory: what the model already believes about your category from the millions of pages it trained on.
The two halves reward different things, on different clocks.
Memory is slow and compounding. It is built from consistent brand mentions across years of listicles, reviews, comparisons, forums and press. It favours incumbents because incumbents have been named more times, in more places, for longer. You cannot buy your way into it this quarter; you can only start the compounding now. This is the deliberate engineering of what your brand is associated with, the thing we call semantic brand positioning in our LLM SEO work.
Retrieval is fast and narrow. When the machine does search, a handful of listicles, comparison pages and fresh community threads decide the answer. A challenger brand can enter those pages in months, which is why Tally beats Typeform the moment search turns on. If you are not the incumbent, retrieval is where you can win first, while memory catches up.
If you want to see how uneven the playing field currently is, our citation gap research found 44% of brands ranking in Google’s top 10 were invisible to ChatGPT. Ranking in the old system never guaranteed a place in the new one’s memory.
What to actually do about it
1. Find out which mode your category runs in. Ask ChatGPT your buyers’ questions and watch whether it searches. If it answers instantly with no sources, you are fighting a memory battle. If sources appear, list them: that is your retrieval target list, and the check takes an afternoon.
2. If it searches, get into the deciding pages. The listicles and comparison pages it cited, the communities it quoted. Fresh beats old here; that month-old Reddit thread outweighed a decade of corporate blogging.
3. If it answers from memory, play the long game deliberately. Consistent naming, more mentions in more credible places, co-occurrence with your category everywhere the next training run will read. The mention layer you build this year is the model memory you benefit from next year.
4. Do not let anyone sell you one half as the whole. A proposal that only talks citations is ignoring where most answers actually come from. A proposal that only talks brand building has no answer for the categories where the machine searches every time. We wrote a skeptical buyer’s walkthrough of this market if you are being pitched right now.
Frequently asked questions
Does ChatGPT search the web when you ask for product recommendations?
Sometimes, and less often than you’d think. In our hour of live buying prompts it mostly answered from the model’s memory, with no sources fetched, and said it reserves live search for fresh or volatile topics. Forcing search changed the winners.
How do I get my product recommended by ChatGPT?
Work both layers. Short term: be present in the listicles, comparison pages and community threads it retrieves when it searches your category. Long term: build consistent brand mentions across credible pages so the next training run learns your name as part of the category’s consensus.
Why does ChatGPT recommend big brands so often?
Because memory is trained on the web’s accumulated history, and incumbents dominate that history. More mentions, over more years, in more places. Challengers win first in retrieval, where fresh pages decide, and that is also how they start changing the memory.
Is this the same as GEO or AI SEO?
It is the mechanism underneath them. Honest GEO work splits into moving the model’s memory (slow, compounding, off-site) and winning live retrieval (faster, content and citations). This experiment shows why both halves exist and which one your category needs first.
Run the test on your own brand
Everything above came from one hour and a handful of prompts, and it will be the most clarifying hour you spend on marketing this quarter. Ask the machine your buyers’ questions, note whether it searches, open what it cites. If you’d rather we ran it properly, with every engine, your competitors included and the sources mapped, that’s our free AI visibility audit. Thirty minutes, and you’ll know which battle you’re actually in.