Why Your GEO Tracking Tool Only Sees Half the Conversation

I came across a post from Jan-Willem Bobbink, a freelance enterprise SEO developer, asking a question that most GEO vendors would rather you not think about too hard: which prompt-tracking tool is tracking full conversations, not just an initial prompt and answer pair? His point was simple. Most humans don’t ask one question. They ask a question, get an answer, then narrow it with a second and third message, and with more people using AI through plugins and interactive widgets, that pattern is only getting more common, not less.

Underneath his post was a screenshot of ChatGPT comparing GEO trackers by business type, recommending Profound for an enterprise retailer, Peec AI for a Shopify brand, Scrunch AI for anyone needing crawler access data. A clean, confident shortlist. Also, by construction, an answer to exactly one prompt, asked once, with no idea what the person asking it actually needed two messages later.

That post turned into one of the better threads I’ve read on this topic in months, with Dixon Jones (CEO of InLinks and Waikay), Ramon Eijkemans, and a practitioner named Tarık Y. all pushing back and refining the idea in the comments. I want to walk through what they actually said, because the disagreement maps the real shape of this problem better than any vendor page does, and then make the case for why I think most brands should stop waiting for a platform to solve it.

What a prompt-tracking tool actually shows you

Start with the mechanics, because “it’s tracking your visibility” hides a lot of methodology that vendors mostly don’t publish.

Most of these tools run through an API, not a logged-in browser session, which already separates what they report from what a real user sees. A signed-in ChatGPT session carries memory, prior conversations, and personalization the API call doesn’t have access to. That gap alone means a platform’s reported “visibility score” and the answer your actual prospect receives can diverge before methodology even enters the picture.

On run frequency, the public documentation is thinner than you’d expect. One source describes Peec AI’s counting model as single-run per cycle: 25 prompts across 3 models over 30 days produces 2,250 logged answers, which implies one generation per prompt per day rather than several averaged together.[1] Profound, Otterly.ai, and Scrunch AI don’t appear to publish how many times they regenerate a given prompt per cycle.[2] Separately, general sampling guidance for LLM outputs (cited back to Graphite) recommends running each prompt at least 10 times to get a stable read, since the underlying generation is probabilistic and a single run can land anywhere in the distribution.[3] Whether any given vendor actually does that isn’t something I could confirm from their public pages, and I’d ask directly before trusting a single-run number as representative.

Either way, all of it answers the same fixed question: what does the model say, right now, to this one prompt. That’s a reasonable thing to track. It just isn’t the thing that decides most B2B purchases, which rarely get resolved in one exchange.

The thread: four practitioners, four different answers

Dixon Jones replied first, and bluntly: trying to seriously track a multi-turn conversation is “a silly idea.” His reasoning holds up. Search Console is scoped to one website, not one buyer’s path across multiple sites and topics. Watching a single laptop might work, but only for that one person. And most threads never repeat, so whatever you capture is unlikely to represent anyone else’s conversation. His alternative is to stop chasing the conversation and get the foundations right instead: what the model already knows about your brand at the entity level, which is what his company’s tool, Waikay (“What AI Knows About You”), is built to audit. It queries major LLMs, with live search switched off in some tests, to see what a brand’s raw, unbiased training-data footprint looks like before any retrieval happens.[4]

Jan-Willem’s reply didn’t disagree with the diagnosis, only the conclusion. You can’t track one conversation, he said, but you can simulate a thousand or a hundred thousand. He reached for the Copenhagen interpretation as a loose analogy: you don’t know where an electron is or how fast it’s moving until you measure it, and the act of measuring only ever gives you one of those two properties at a time. One measurement tells you almost nothing. A thousand measurements give you a probability distribution. Applied here, that means you’ll never know exactly what a single user saw in their one conversation, but you can know how often your brand shows up on turn one, turn three, or turn five of the journeys that actually matter to your category. He also mentioned combining this with user interviews he’s already run for several clients, which fill in the why and how of a conversation’s start and its follow-ups.

He backed the “these conversations are real” claim with something concrete: Search Console itself leaks fragments of them. On the same page, on the same day, he found the queries “what is the housing market doing in 2026,” “what will house prices do in 2027,” “and in 2027?” and “in amsterdam” sitting in the same property’s query report. Nobody opens a browser and types “and in 2027?” as a first search. That’s the back half of someone’s conversation, surfacing in Search Console because AI Mode queries land in the general Performance report rather than a separately labeled bucket.[5]

Ramon Eijkemans had actually tried this. He agreed with Dixon that paid associations are a better probe than trying to reconstruct conversations, but pushed back on calling the idea silly. He’d tested the Search Console method directly and it “just yielded not enough data.” His comparison was to replaying individual user session recordings on a site with thousands of them: anecdotal, occasionally fun, lightly useful for a feel of things, not something you’d build a reporting system on. He also raised a question nobody else in the thread answered: within GSC or Bing Webmaster Tools, was a given query typed by a human or generated by a machine process, and does the platform tell you which?

Tarık Y., a digital marketing strategist, offered the most concrete interim method in the thread: record a fixed set of multi-turn journeys, including the exact follow-up prompts, the model, the location, and the date, then compare citations and brand mentions at each turn, repeating runs because responses vary. Jan-Willem confirmed that’s close to how most of his current setups run, with one addition: he layers in variations based on ICPs, so the same seed journey gets replayed with the phrasing and context a startup buyer would use, and again with the phrasing an enterprise buyer would use.

Ramon’s fuller version of the idea, posted separately, turns Search Console into a seed generator rather than a measurement tool:

1. Harvest. Pull queries from Search Console that only make sense as a continuation: a price question with no brand attached, a bare “yes, go on,” a sudden topic pivot.
2. Reconstruct. For each one, check the exact day and landing page it appeared on, then chain the queries into a plausible conversation.
3. Build journeys. Group the chains into buyer and support journeys, turn one as the opening prompt, turns two and up as the follow-ups.
4. Replay. Run each journey turn by turn across AI Mode, ChatGPT, Perplexity, Gemini, and Copilot, repeating runs for variance, logging brand mentions and citations per turn.
5. Deliver. One spreadsheet: journeys, replay prompts, per-turn results.

His honest accounting of it: the upside is that the follow-ups come from real behavior, not a guess. The downside is that the source data only covers Google’s AI surfaces, the chain reconstruction is an inference you’re making after the fact, and low-volume queries get hidden by Google’s own anonymization before you ever see them. In his words, it “doesn’t yield a lot of conversations but still useful because partly real.” Jan-Willem tried it against a few domains afterward and called it better than guessing, adding that he’d already built the classifier for this into his own tool, GSC Wizard.[6]

Figure 1. A single tracked prompt is the root of a much larger tree most tools never climb down into.

A worked example: device lifecycle management

Here’s the shape of the problem on a category I’d expect most GEO tools to cover reasonably well.

Turn one: “best device lifecycle management platform for a distributed team.” A model answering that today would plausibly surface names like ZenAdmin, Workwize, and GroWrk, the kind of shortlist that shows up across comparison content for this category.[7] That’s the turn every tracker you can buy right now will show you a score for.

Turn two is where an actual buyer, not a benchmark prompt, goes next: “we have 500+ employees across 100+ countries, which of these handles that.” That single follow-up does real work. Public vendor materials position ZenAdmin as the one with the actual hardware, logistics, and device lifecycle management depth to answer that, alongside its SaaS and access layer; Workwize’s own comparisons focus more narrowly on hardware lifecycle and regional logistics with established MDM integrations; GroWrk’s model is a regional hardware marketplace that at least one reviewer flagged as weaker on managing the lifecycle of peripherals specifically.[8] A model reasoning over that context has real grounds to narrow a four-name shortlist down to two, maybe one. If your brand was mentioned on turn one but drops on turn two, your turn-one score just told you nothing about whether you’re in the deal.

That’s the gap Jan-Willem’s thread is circling. Turn one tells you who gets a mention. Turn two tells you who survives contact with the buyer’s actual constraints, which is a different list and, for anything with a sales cycle longer than an impulse purchase, the more important one.

What tracking this actually costs

Multiply the thread’s own ideas together and the honest shape of the problem is combinatorial, not linear. This is my own back-of-envelope math, not a vendor’s published cost, so treat it as illustrative:

Take one seed topic. Give it four plausible turn-two directions, the kind a real buyer takes (headcount, geography, existing stack, budget), matching Figure 1. Give each of those two third-turn refinements. That’s 1 + 4 + 8 = 13 distinct conversation nodes for one topic. Follow Tarık’s advice and repeat each node 10 times for a stable read, per the same sampling logic that applies to single-turn tracking.[3] That’s 130 generations. Follow Jan-Willem’s ICP layering and run the same tree for four buyer personas (enterprise IT admin, startup ops lead, global-remote HR lead, procurement-led RFP buyer): 520 generations. Track it across four platforms, which is the realistic minimum for most B2B categories today: 2,080 generations, for one topic, for one monthly snapshot.

And because it’s multi-turn, the cost isn’t just call volume. Every turn-two call carries turn one as context, every turn-three call carries both prior turns, so token volume per tree grows faster than the node count alone suggests. A platform selling this at scale across hundreds of brands either prices it well above single-turn tracking, or quietly narrows the tree until it’s closer to single-turn than the pitch suggests.

The case for building this yourself instead of renting it

None of the four people in that thread were wrong. Dixon’s right that no individual thread repeats. Ramon’s right that GSC-mined journeys are directionally useful but thin. Tarık and Jan-Willem’s method is the most rigorous version of this I’ve seen described publicly. All four are still solving the problem from outside the brand, though, simulating a stranger’s conversation rather than working from one.

I think that’s backwards for most B2B companies, and here’s the structural argument for why.

The last row is the honest caveat, not a sales pitch. Building this yourself doesn’t solve the sparse-data problem Ramon hit. It just moves you from “trusting a platform’s simulated buyer” to “working with your own thin but real signal,” which is a smaller, more tractable problem, not a solved one. You still need the interviews Jan-Willem mentioned, the GSC fragment-mining Ramon described, and genuinely talking to your sales team about what the third question on a call usually is.

What you get for that trade is something no dashboard gives you: the exact phrasing. If the real turn-two follow-up in your category is “which of these handles 100+ countries,” that’s not just a tracking input, it’s the H2 your comparison page is missing, or the FAQ answer that should sit above the fold instead of buried in a table nobody scrolls to. A platform’s generic visibility score can tell you that you dropped between turn one and turn two. Only your own seed data tells you which sentence to write to stop dropping.

The honest pushback

Dixon’s objection deserves to stand on its own, not just as a counterpoint I argued past. If no two conversations repeat, then everything in this piece, including Jan-Willem’s probabilistic framing, is measuring a distribution and hoping it’s representative, not measuring any real buyer’s actual path. That’s a real limitation, and it’s one the DIY approach shares with every platform in this category. The honest claim isn’t “build this and you’ll know what buyers see.” It’s “build this and you’ll know, with real uncertainty attached, how often your brand survives to turn two or turn three of the journeys that resemble the ones your actual buyers run,” which is close to exactly how Jan-Willem put it. That’s a meaningfully smaller claim than most GEO tools are selling, and I’d trust it more for being smaller.

If you’re deciding whether to buy a seat on a GEO platform or build a thin version of this yourself, the deciding question isn’t which one is more accurate. Neither is. It’s which one hands your content team a sentence to write next. Right now, only one of them does.

Frequently Asked Questions

Do any GEO tracking tools already measure multi-turn conversations?

Not as a standard, documented feature across the major platforms as of this writing. Public methodology on run frequency and conversation depth is thin for most vendors; the practitioners quoted in this piece are building multi-turn testing by hand, outside any platform, using Search Console fragments, user interviews, and scheduled API replays.

Is it realistic for a small team to build this in-house?

The replay mechanics are genuinely simple: a scheduled script calling each platform’s API with a fixed set of journeys and logging the output. The hard part isn’t the automation, it’s sourcing real seed journeys, which takes the kind of Search Console mining, sales call review, and customer interviews described above.

How many times should a prompt be re-run for a stable result?

General sampling guidance puts the floor around 10 runs per prompt, since LLM outputs are probabilistic and a single generation can land anywhere in the underlying distribution.[3] Whether any given vendor actually does this isn’t publicly documented for most of the major tools.

Want help figuring out which turn your brand is actually losing in, and what to write to fix it? Book an AI Visibility Audit

Notes

1. A third-party comparison describes Peec AI’s counting model as single-run per cycle, citing 25 prompts across 3 models over 30 days producing 2,250 logged answers. I could not independently confirm this against Peec AI’s own documentation, so treat it as one source’s characterization rather than a vendor-confirmed figure.

2. Based on a review of public comparison and review content for Profound, Otterly.ai, and Scrunch AI; none of the pages I checked stated how many times a given prompt is regenerated within a single reporting cycle. Profound’s own comparison pages describe prompts running daily through browser interfaces, and Otterly and Peec advertise daily updates, which addresses frequency over time rather than repetition within a cycle.

3. The 10-run recommendation is cited back to Graphite in a third-party guide on LLM rank tracking. It’s a general sampling recommendation given the probabilistic nature of LLM generation, not a confirmed practice at any specific vendor.

4. Dixon Jones, CEO of InLinks, in the comment thread on Jan-Willem Bobbink’s LinkedIn post, November 2026. Waikay (“What AI Knows About You”) is an InLinks product; InLinks has stated it applied for a patent on the underlying technology. Waikay’s own FAQ describes a score of 0 as meaning the AI did not mention a brand in its raw, unbiased response with live search disabled.

5. Jan-Willem Bobbink, in the same thread. On the broader question of whether AI Mode queries are separately labeled in Search Console: Google’s John Mueller has stated that AI Mode queries are folded into the general Performance report rather than the separate generative-AI reports, which began testing in June 2026 with impression-level but not query-level data. Some third-party guides describe a Search Type filter that separates AI Overview and AI Mode traffic; others, from around the same period, say no such separation exists yet. I could not resolve the conflict from public reporting, so check the Search Type filter in your own property rather than taking either account as current.

6. Ramon Eijkemans and Jan-Willem Bobbink, in the same thread. Jan-Willem’s tool, GSC Wizard (gscwizard.com), is described in third-party listings as a web-based tool that turns Search Console data into reports and opportunity insights, with MCP and ChatGPT access.

7. Based on a review of public comparison content for this category, primarily vendor-authored comparison pages, which should be read with that bias in mind.

8. ZenAdmin’s own materials, including its G2 listing, describe asset procurement, device lifecycle management, and hardware logistics depth alongside its SaaS and access platform. Workwize’s comparison pages position the product around hardware lifecycle and regional logistics depth with MDM integrations including Jamf, Intune, and Kandji. One user review cited in secondary coverage flagged GroWrk’s handling of peripheral lifecycle management as a weaker point relative to its hardware marketplace strength. All of this is vendor- or review-sourced, not independently tested, and included here to illustrate how a model might reason through a turn-two narrowing question, not as a product recommendation.