AI Search Measurement & Tools - Tygart Media

Category: AI Search Measurement & Tools

  • Is Your AI Traffic Hiding in Direct?

    Is Your AI Traffic Hiding in Direct?

    Here is the comforting story making the rounds of every AI-search conversation this year: the answer engines are eating your clicks, yes — but the visitors still come. They read the AI answer, remember your name, and type your URL in later. GA4 can’t see a referrer on a typed visit, so the traffic lands in Direct. Your AI visibility is real; it’s just hiding in a channel your analytics can’t read.

    It is a good story. It is also, on the best test anyone has run of it, not true in any way you can measure. Analyst Kevin Indig took the claim apart in his Growth Memo this month — not by arguing with it, but by testing it against twelve sites’ analytics. This piece is about what he found, and what it means for how you measure visibility when there is no click to count.

    What a real test of the claim has to do

    Most versions of the hidden-traffic claim are proven by pointing at a Direct chart going up. Indig’s test, run across the GA4 properties of 12 sites from January to September 2026, did the three things the chart-pointing never does.

    • He removed the pages that naturally draw Direct visits. Homepages, pricing and plan pages, login and account portals, help docs — the places people genuinely do type or bookmark their way into were excluded, so what remained was the content traffic the claim is actually about.
    • He separated engaged sessions from raw sessions. An engaged session is a visit where someone actually read — GA4’s threshold for a person, not a ping. If AI answers were sending real readers in through Direct, engaged Direct is where they would have to appear.
    • He shuffle-tested the correlations. For four brands he ran 632 time-shifted comparisons between weekly AI visibility and branded search and Direct traffic — then re-ran the alignments randomly shuffled, to see whether the relationships survived being scrambled.

    What the numbers say

    Total Direct traffic across the sample rose 140.9% from January to September. Engaged Direct sessions rose 2.3% on a daily-average basis — and fell 1.0% in raw monthly counts. Flat, in other words, whichever way you slice it. The growth was real; the readers weren’t.

    The single most honest number in the study: one site logged 409,000 Direct sessions, of which 198 counted as engaged. And when Indig looked for Direct sessions carrying identifiable traces of AI referral, he found them at 0.003% of total Direct traffic on average.

    What filled the channel instead was bots. Eight of the twelve properties showed Direct growth from August to September, some over 100% — and digging in, the growth concentrated in clusters of visits sharing the same browser configuration with low or no engagement. Indig’s method for finding them is worth stealing outright: break Direct down by day, landing page, country, browser, operating system, browser version and screen size, and look for sudden growth in same-configuration clusters that never engage.

    The correlation that collapsed

    The visibility half of the test matters just as much, because it is the version operators actually use: “our AI mentions went up, and our branded and Direct traffic followed, so the mentions are working.” Indig found correlations as high as 0.72 — the kind of number that gets screenshotted into a deck. Against randomly shuffled alignments, they collapsed.

    A correlation that cannot survive a shuffle test was never a measurement. It was two lines that happened to rise in the same year — in a year when Direct was being inflated by bots, and branded search was being moved by everything else a growing company does. His conclusion, in his own words: “I could not yet detect ‘hidden’ AI traffic.”

    The one exception — and why it doesn’t rescue the claim

    Indig is fair about the counter-evidence. Ramp’s team, in its own internal analysis, attributed 30–40% of its Direct traffic to answer engines. But look at what that took: filtering bot and email-campaign patterns out of long-tail blog URLs, comparing what remained against organic-search behavior — and, the load-bearing piece, self-reported attribution: asking their own visitors where they came from.

    That is not a refutation of Indig; it is his point. The one credible version of the number required ground truth from the visitors themselves. Nobody gets to Ramp’s answer by staring at a GA4 channel grouping.

    So how do you measure visibility when there is no click?

    You measure the thing itself instead of its shadow. The citation is the event: whether your page is in the answer, for which questions, and how often over a trailing window — the approach we laid out in how to measure AI search visibility. A rate over thirty days is a measurement. A rising Direct line is a hope.

    • Count citations directly, on a trailing window, against a fixed set of questions. One run proves nothing; a rate is a fact.
    • Take the referrals that do arrive at face value. Clicks out of ChatGPT, Perplexity and Copilot largely do carry referrers — they show up as referral traffic, countable and real. Our GA4 and response-side measurement stack covers how we read them.
    • Treat Direct as contaminated until segmented. Any Direct growth story gets the Indig treatment first: engaged sessions only, natural-Direct pages excluded, configuration clusters checked for bots.
    • If you need an attribution number, earn it like Ramp did — ask your visitors, in a survey or an intake field — and label it self-reported when you quote it.

    The hidden-in-Direct story survives because, told loosely, it can’t be proven wrong. Indig told it precisely, and it failed: engaged readers flat for nine months while the channel around them filled with machines. Book your AI visibility where it actually lives — in the answers — and let Direct go back to being what it mostly is now: a bot bucket with a good publicist.

  • What does AEO reporting actually show?

    An honest AEO report shows four things: the exact questions that were asked, the answers the assistants actually gave, which businesses and pages those answers cited, and how all of it changed since the last run. AEO reporting is the scheduled record of those four things, kept the same way every time. Anything that cannot show you the four is showing you a mood, not a measurement.

    What questions were asked?

    A real AEO report prints its question set in full, word for word. The questions are the foundation of the whole report: they should be your customers’ real questions, fixed in advance, and identical from run to run. A citation count means nothing until you can see the questions it was counted on.

    This is where reports quietly cheat. Swap the questions between runs and every comparison is fiction — you improved because the test got easier. Add a question you already win and the average climbs without a single answer changing. Ask for the set in writing before the first run, and treat any later change to it as a new baseline, announced as one. A short set of real questions, held steady, beats a long set that wanders.

    What did the AI actually say?

    A real AEO report keeps the raw answers, not summaries of them. The answer text is what your customer actually reads — the names in it, the order they appear, the words around them. A summary written by the provider can only tell you what the provider noticed. The raw answer lets you check.

    Summaries hide the losses. A run where you were cited on nine questions and dropped on the tenth, hard, reads as “visibility strong” in a summary and as a work order in the raw text. Keep the full answers, dated, for every run. They take almost no space, they cannot be reconstructed later, and they are the difference between having evidence and having a recollection.

    Who got cited?

    A real AEO report names the citations for every question: which businesses the answers credited, which pages of theirs did the work, and what share of the citations on each question went to whom. The citation list is the most useful page in the report. It converts “we are not visible” into a named list of exactly who the machines trusted instead, question by question.

    Read it as a work list, not a wound. When one restoration client of ours kept losing a question to the same competitor, the citation list showed the competitor’s page the assistant kept crediting — a page that answered the question in the customer’s own words, in the first paragraph. That is a specific, fixable problem, and it was invisible in every summary number we had. Your citation share — your slice of the citations on your questions — is the one summary figure worth tracking, because it is arithmetic on the list, not an opinion about it.

    What changed since last time?

    A real AEO report is built to compare: this run against the last, same questions, with the differences stated plainly and the work done in between written next to them. Answers churn — the same question, asked next month, can name different businesses with nothing changed on either side. An honest report says so, and reads movement over several runs instead of celebrating or mourning a single one.

    The work log belongs inside the report for one reason: without it, you cannot tell your work from the weather. “We rebuilt the profile and published two answer pages in March; citations on those questions rose across the April and May runs” is knowledge. The same rise with no log behind it is luck you will try to repeat blind. If your provider’s report never mentions what they did between runs, ask why their work and their results live in separate documents.

    What does a horoscope report look like?

    A horoscope report shows a single score with no questions behind it, percentages with no counts under them, and screenshots of the one good answer from a good day. It changes its questions when the results are inconvenient, reports only the wins, and always finds that things are improving and that more budget would improve them faster. It feels good, commits to nothing, and cannot be checked.

    The test is simple. Pick any number in the report and ask: on which questions, from which answers, counted how? A real report answers in one step, because the number came from a list it can show you. A horoscope report answers with another number. We call them horoscopes because, like the ones in the paper, they are written to be agreeable about the future and impossible to grade about the past.

    How often should you read an AEO report?

    Read AEO reporting on a rolling cadence — monthly is a sane default — and never daily. AI answers move day to day with nothing changed, so daily reading is noise-chasing: it manufactures urgency, spends your attention, and teaches you to react to weather. What is worth reading is the run-over-run trend, and the first run after real work has had time to settle.

    Reading it should end in a decision, or the reading is decoration. The standing decision a good report sets up is small: which questions is a competitor cited on that we are not, and what is the missing answer we should write next? Our AI Citation Quick-Scan is this artifact in its smallest honest form — 10 prompts about your service line, the answers recorded, the citations named, a scored report within 7 days, $97, one time.

    Which lines the platforms lift is their call. We build the material they lift from — and then we measure what they did.

  • How do you measure AI search visibility?

    You measure AI search visibility with two kinds of tools: the reports the platforms publish about their own answers, and trackers that ask the assistants your customers’ questions on a schedule and record who gets named. The platform reports show who the platform cited. The trackers show what an assistant answered when asked. A buyer who knows which tool shows which thing can read either one honestly.

    What does “visible” mean in an AI answer?

    An AI answer can mention your business, cite your business, or recommend your business, and those are three different measurements. A mention means your name appears in the answer text. A citation means the answer credits your page as a source. A recommendation means the assistant names you as the choice. Every tool below reports some mix of the three, under its own labels.

    The distinction decides what a number is worth. A mention can be one name in a list of eight. A citation means the platform read your material closely enough to credit it — that is the receipt that survives. A recommendation is the strongest outcome and the least common. When a tool shows you one big “visibility” number, the first question is which of the three it counted.

    What does Bing AI Performance show?

    Bing AI Performance is Microsoft’s free first-party report, and it shows what the assistants built on Bing actually cited. The report organizes citations four ways: Intents, Topics, Citation Share, and Compare. It is the platform grading its own homework — the most direct evidence a buyer can get, and a view of exactly one surface.

    The four views answer four plain questions. Intents shows the kinds of things people were trying to do when your pages got cited. Topics groups those citations by subject, so you can see which of your subjects the machines trust. Citation Share shows your slice of the citations on a topic against everyone else cited there. Compare puts two date ranges side by side, so a change shows up as a change and not a feeling.

    Our own report is the example we read every month. For tygartmedia.com, our Bing AI Performance report covering six months of Microsoft Copilots and partners shows 1.5M total citations across an average of 164 cited pages. We read that as our footprint on one measured surface — proof the surface can be measured, and a baseline to compare the next run against. It says nothing about any other platform, and it is not a promise about any other business.

    What do third-party visibility trackers show?

    Third-party trackers are subscriptions that ask a fixed set of questions to the major assistants on a schedule and record the answers. A tracker shows mention rates, citation rates, and competitor shares across several surfaces at once. What it sells is a trend line across platforms, built from sampled asks — not a record of what any real customer saw.

    The honest advantages are real. A tracker sees surfaces the first-party reports cannot, because each platform only reports on itself. A fixed question set, asked the same way each run, produces runs a buyer can actually compare. Competitor context comes built in, and in a contested market that context is the point.

    The limits are just as real. The tracker asks its questions, on its schedule, from its accounts — your customers ask differently, locally, and once. Small question sets get reported as precise percentages, and a two-question swing can dress itself up as a trend. And the subscription runs whether you read it or not. A tracker earns its money in a competitive market with several assistants in play, read monthly by someone who acts on it. Everywhere else it is an expensive screensaver.

    What can no tool tell you?

    No tool can tell you what a specific customer saw, whether a mention became a phone call, or why an assistant chose a competitor. AI answers vary by person, place, and moment, and the platforms do not publish their reasoning. A tool shows outputs. The cause behind an output is still the buyer’s work to figure out.

    Sit with that limit before you buy anything, because it is where the honest tools end and the salesmanship starts. No report closes the gap between “the assistant named us” and “the phone rang” — that gap closes with call tracking and asking customers how they found you, unglamorous as that is. A provider who sells you certainty across that gap is selling you the gap.

    Which tool should a buyer start with?

    A buyer should start with the free first-party report for the surface their customers actually use, and add a tracker only when they need a cross-platform trend and will read it monthly. The tool matters less than the discipline behind it: a fixed set of your customers’ questions, asked before the work and after, with the raw answers kept. Any tool run that way beats a better tool run casually.

    Most small businesses find the first-party report is enough to start, because it answers the only question that changes what they do next: on the questions my customers ask, who does the machine name — and is it me? If you want that first measurement done by hand instead, our AI Citation Quick-Scan asks 10 prompts about your service line and returns a scored report within 7 days — $97, one time.

    Which lines the platforms lift is their call. We build the material they lift from — and then we measure what they did.

  • How to Read Bing Webmaster Tools AI Citations (Without Confusing Them for Traffic)

    How to Read Bing Webmaster Tools AI Citations (Without Confusing Them for Traffic)

    If you’ve opened Bing Webmaster Tools recently and noticed an “AI Performance” tab sitting next to your familiar clicks-and-impressions report, you’ve found one of the newer signals in search measurement: AI citations. It’s a genuinely useful number. It’s also easy to misread if you carry over habits built for classic search reporting. Here’s how to read it correctly.

    What a Bing AI Citation Actually Is

    Topic platform fit visual for first-party AI citation measurement
    What a Bing AI citation actually is.

    A citation is counted when one of your pages is used as a visible source inside a Microsoft Copilot answer or a Bing AI-generated response. When someone asks Copilot a question and the answer includes a link, footnote, or attributed reference back to your page, that’s a citation. It means the AI system read your content, judged it relevant and trustworthy enough to draw from, and surfaced it — sometimes with a link the reader can click, sometimes just as a named source.

    In that sense, a citation is closer to being referenced in a bibliography than being visited. Your page did its job as a source of truth for the answer, whether or not the reader followed the link.

    What a Citation Is Not

    Desk with laptop, checklist notebook, and billing card ready before creating an Anthropic API key
    What a citation is not — not traffic.

    This is the part that trips people up, because the reporting sits right next to metrics that mean something different:

    • Citations are not clicks. A citation records that your content was used to generate an answer. It says nothing about whether a human then visited your site.
    • Citations are not sessions. Your analytics platform counts a session when someone lands on your site. A citation can happen with zero sessions attached — the reader gets their answer and moves on.
    • Citations are not rankings. Traditional search position measures where you sit on a results page for a given query. AI citation measures something different: whether your content was selected as source material for a generated answer, which can happen independently of where you’d rank in a classic search.

    Treating a citation count like a traffic number, or expecting it to move in lockstep with clicks, sets you up to misjudge a page’s performance in either direction.

    Where to Find This Data

    Inside Bing Webmaster Tools, the AI Performance section reports citation volume over time, and typically breaks it down by which pages were cited and which queries or topics triggered the citation. It’s a separate report from the standard Search Performance section, which still covers traditional web impressions, clicks, and position. Treat them as two different dashboards answering two different questions, not two views of the same thing.

    How Citations Relate to GA4 and Server Logs

    Because a citation doesn’t require a click, your analytics platform (GA4 or otherwise) will only ever show you a fraction of the activity that citation data reflects. What GA4 can show you is the downstream piece: sessions where the referring source is an AI assistant’s domain. Those sessions represent people who read an AI answer, saw your page referenced, and decided to click through anyway — a smaller, but highly qualified, slice of the audience your content is reaching through AI systems.

    Server or CDN logs add a third layer entirely: they can show you when AI crawlers are visiting your site to read and index content in the first place, ahead of and separate from any citation event. Together, these three sources describe three different moments — a bot reading your page (server logs), your page being cited in an answer (Bing AI Performance), and a human clicking through after reading that answer (GA4 referral data). None of them substitutes for the others.

    Reading the Numbers Without Overreacting

    Citation counts can move for reasons that have nothing to do with your content quality changing: a topic trending in the news, a shift in how often people ask AI assistants about a subject, or changes on the AI platform’s side in how it selects and displays sources. A dip in citations for a page you haven’t touched isn’t necessarily a signal that something is wrong with that page. Likewise, a spike doesn’t always mean you did something differently — sometimes demand for the topic simply increased.

    The more durable way to use this data is directional and page-level: which of your pages does the AI Performance report show being cited consistently over time, and does that list overlap with pages you already consider authoritative? That overlap is a reasonable confirmation signal. A single week’s swing usually isn’t.

    Practical Takeaways

    Comparison of Claude how-to fit versus local service page fit for assistants
    Practical takeaways for reading the numbers.

    Check the AI Performance tab as its own report, not a substitute for Search Performance. Don’t expect citation counts and click counts to correlate closely — they’re measuring different behaviors. Pair citation data with GA4 referral sessions from AI-tool domains to see the (smaller) human click-through layer, and use server logs if you want visibility into AI crawler activity before any citation happens. Judge trends over weeks, not days, and focus on which pages appear repeatedly rather than reacting to any single count.

    FAQ

    If my citation count is high but my clicks are low, is something broken?
    No. That pattern is expected. Citations are a zero-click-by-design channel; a page can be doing exactly what it’s supposed to do as an AI source while generating very little direct click traffic.

    Does Google offer the same kind of citation reporting?
    Not with the same first-party granularity as Bing Webmaster Tools’ AI Performance tab at this time. Server-log analysis for AI crawler activity remains useful regardless of which AI systems you’re trying to track.

    Should I optimize content specifically to increase citations?
    Focus on being a clear, accurate, well-structured source on your subject rather than chasing citation counts directly. Citation tends to follow genuinely useful, well-organized content rather than any particular formatting trick.

    Related on Tygart Media: Bing AI citations vs SpyFu · Bing vs Google Search Console · AI citation monitoring.

  • Claude AI Pricing vs Bing AI Citations: Why First-Party Data Beats SpyFu Estimates (2026)

    Claude AI Pricing vs Bing AI Citations: Why First-Party Data Beats SpyFu Estimates (2026)

    If you still lean on a tool like SpyFu to gauge how your site is doing in search, you’re measuring last decade’s game. SpyFu, Ahrefs, SEMrush, and their peers were built to estimate one thing: where a domain ranks in a traditional results page, and roughly how much traffic that’s worth. Still useful — just not the whole picture, because a growing share of how people find your content never touches a results page at all. It happens inside an AI answer, where your page gets cited or quoted and the reader never clicks through.

    That’s the gap between third-party rank-estimation tools and first-party AI citation data, and it matters more every month.

    What SpyFu (and Similar Tools) Actually Measure

    Third-party SEO tools crawl the web and model search behavior from the outside. They don’t have access to your server logs, your analytics, or Bing and Google’s internal citation data — they infer traffic from ranking position, keyword volume estimates, and click-through curves built from aggregate industry data. That’s genuinely useful for competitive research: roughly where a competitor’s domain sits, and what keywords it’s chasing.

    But it’s an estimate of an estimate, built for a web where “visibility” meant “blue link position.” It has no mechanism for counting how many times an AI assistant read your page, extracted a fact from it, and served that fact directly to a user who never visited your site.

    What First-Party AI Citation Data Shows That Estimators Can’t

    Topic platform fit visual for first-party AI citation measurement
    What first-party AI citation data shows that estimators can’t.

    Bing Webmaster Tools now separates two very different signals: traditional web search performance (impressions, clicks, position) and AI performance — how often your pages get surfaced inside Copilot and other AI-generated answers. Google Search Console doesn’t yet break this out the same way, which is part of why it’s easy to miss. If you only watch third-party rank trackers, this entire layer is invisible to you.

    The practical difference: a page can have modest, even declining, click-through performance in classic web search while its AI-citation count climbs steadily. Judged only by a SpyFu-style estimate, that page looks flat or fading. Judged by first-party citation data, it’s doing exactly the job it was built for — being the source an AI system reaches for when someone asks a related question.

    The Blind Spot: Zero-Click Visibility

    Four cards for content, ops, build, and knowledge work with Claude
    Zero-click visibility is the blind spot.

    The uncomfortable part for site owners is that AI citation is, by design, mostly a zero-click channel. The reader gets their answer without visiting — that’s not a measurement bug you can fix with a better tool, it’s the actual shape of the channel. An estimator that only counts clicks and rankings will systematically undercount pages that are winning at citation, because “winning” there doesn’t look like a traffic spike. It looks like your facts and explanations showing up correctly, attributed to you, inside someone else’s interface.

    Relying on SpyFu-style estimates alone can lead to the wrong call: de-prioritizing a page that’s actually become a trusted AI reference source, simply because the tool built to measure clicks can’t see the citations.

    Building Your Own First-Party Measurement Stack

    None of this means third-party tools are useless — they’re still the right instrument for competitive keyword research and for understanding classic ranking dynamics. But they should sit alongside, not replace, sources that actually see your own traffic and your own citation footprint:

    • Bing Webmaster Tools’ AI Performance tab — the most direct read on how often Copilot and partner AI surfaces are citing your pages.
    • Server or CDN logs — the only place you’ll reliably see crawler activity from AI bots (ClaudeBot, GPTBot, PerplexityBot, and similar) hitting your pages, separate from human traffic.
    • Your own analytics referral data — small in volume compared to citations, but real signal: sessions that landed with claude.ai, chatgpt.com, or perplexity.ai as the referring host are humans who read an AI answer, then clicked through anyway.

    Put those three together and you get a picture no third-party estimator can reconstruct: which of your pages AI systems actually trust enough to cite, and whether that trust is translating into any direct human traffic at all.

    Practical Takeaway

    Desk with laptop, checklist notebook, and billing card ready before creating an Anthropic API key
    Practical takeaway — build your own measurement stack.

    If a page’s third-party “visibility score” looks unimpressive but your first-party data shows steady or rising AI citation activity, don’t treat that as a contradiction — treat it as two different questions with two different answers. The estimator tells you about classic rank. Your own logs and Bing’s AI data tell you about a newer kind of authority that doesn’t require a click to pay off. Site owners who only check the estimator are optimizing for a channel that’s shrinking relative to the one they can’t see.

    FAQ

    Do I need to abandon tools like SpyFu?
    No. They’re still useful for competitive keyword research and classic rank tracking. The point is to stop treating their traffic estimates as the full measure of your site’s reach.

    Can I get AI-citation data for Google’s AI features the way I can for Bing?
    Not with the same granularity as of this writing — Bing Webmaster Tools currently offers the clearest first-party AI-citation reporting. Server-log analysis for AI crawler activity works across engines regardless.

    How do I know if AI citations are actually worth anything to my business?
    Track it as its own funnel stage, not a proxy for revenue. Pair citation counts with referral sessions from AI-tool domains and see whether that traffic engages with an owned conversion path on your site. Citation volume alone tells you about reach, not value.

    Related on Tygart Media: read Bing AI citations · AI citation monitoring · GEO tactics.

  • LLM Visibility Measurement: The 3-Layer Stack for 2026

    LLM Visibility Measurement: The 3-Layer Stack for 2026

    If you have run a GEO campaign for any length of time, you already know the measurement problem: there is no Search Console for ChatGPT, no Performance report for Perplexity, and the analytics you do have leak roughly a third of the traffic into Direct. LLM visibility is real, the buyers are real, but the dashboards that prove it exist have to be assembled from at least three different layers. This is the stack we use for client work in 2026 — what each layer measures, what it costs, and the regex you need to make it work.

    What “LLM visibility” actually means

    GEO versus SEO comparison cards
    What LLM visibility actually means.

    LLM visibility is the percentage of relevant AI-generated answers in which your brand, content, or experts appear. It is not the same as ranking, because answers do not have ranks — they have presence or absence. A useful operational definition borrowed from the practitioner community: track a fixed list of prompts that represent buyer intent for your category, run them across a fixed list of models on a recurring cadence, and count two things. First, mention rate — what percent of responses name you at all. Second, citation rate — what percent of responses include a clickable link back to your domain. Those two numbers are the foundation of every dashboard worth building.

    The three measurement layers

    Four-stage funnel: citation, click, engage, convert
    The three measurement layers.

    No single tool gives you the full picture, so build the stack in three layers and treat them as complementary.

    Layer one — Visibility tracking. Are you in the answer? This is the prompt-monitoring layer. You pick 50 to 200 prompts that a real buyer would type into ChatGPT, Perplexity, Gemini, Copilot, or Claude, then a tool re-runs them on a schedule and parses the responses for your brand and your competitors. This is the only layer that can prove a GEO campaign is working before any clicks happen.

    Layer two — Referral analytics. When an AI answer does include a link and a user clicks it, does it show up in GA4? In May 2026 Google added a native “AI Assistant” channel to the GA4 Default Channel Group, which assigns the medium value ai-assistant to recognized referrers and groups those sessions automatically. That is a major improvement, but the underlying problem has not gone away: mobile apps and in-app browsers for ChatGPT, Claude, and Perplexity strip referrer headers, so a meaningful portion of AI-originated visits still arrive as Direct. Practitioner estimates put clean-referrer coverage somewhere in the 60 to 80 percent range depending on the model and the platform mix.

    Layer three — Proxy signals. Branded search volume, direct traffic on long-tail URLs that have no other discovery path, self-reported attribution in lead forms, and CRM “how did you hear about us” data. None of these are clean, but together they sanity-check the first two layers and catch the AI traffic that the referrer pipeline lost.

    The GA4 channel-group regex

    Even with the native AI Assistant channel in place, you still want a custom channel group for granular per-platform reporting and for any property where the new default has not propagated yet. Create one under Admin → Data Display → Channel Groups and put it above Referral in the rule order — GA4 applies rules top-down and Referral will swallow the visit if it gets there first.

    Match against the source dimension with this pattern:

    chatgpt\.com|chat\.openai\.com|openai\.com|perplexity\.ai|claude\.ai|gemini\.google\.com|copilot\.microsoft\.com|bing\.com/chat|deepseek\.com|grok\.com|meta\.ai|you\.com

    That is the full set of recognized referrers as of the May 2026 Google update. For agency reporting we split this into one channel per platform rather than a single “AI” bucket, because the engagement profile is genuinely different — Perplexity sessions tend to behave like high-intent research traffic, while ChatGPT sessions skew more exploratory.

    What the tools actually do — and what they cost

    The visibility-tracking market in 2026 has consolidated into a recognizable shape. Here is the practitioner read on the four tools most likely to come up in a procurement conversation.

    Profound. Tracks coverage across ChatGPT, Gemini, Google AI Overviews, Google AI Mode, Perplexity, Claude, Copilot, Grok, and DeepSeek. The Lite tier starts at $499/month per Profound’s published pricing. This is the enterprise-default option — broadest model coverage, mature competitive view, the price tag to match.

    Semrush AI Toolkit. Tracks Google AI Overviews, Google AI Mode, Perplexity, ChatGPT, and Gemini. Available standalone at $99/month per domain or bundled inside Semrush One starting at $199/month. Strong choice if you already run Semrush — the prompt monitoring lives next to your traditional keyword reports.

    Otterly. Tracks share of voice across ChatGPT, Google AI Overviews, Perplexity, and Copilot, with AI Mode and Gemini as add-ons. Starts at $29/month on the Lite plan, which makes it the cheapest serious on-ramp in the category. Best for solo operators and small in-house teams that need a real share-of-voice number without a five-figure annual commitment.

    SE Ranking AI Visibility Tracker. Bundled inside SE Ranking’s existing SEO platform. Good fit for SE Ranking users; not a category leader for AI alone.

    For a single client account we typically run Otterly for the day-to-day share-of-voice number and add Profound when the scope justifies the spend — usually when the client has more than three competitors they care about benchmarking against.

    A minimal measurement framework you can ship this week

    Desk with laptop, checklist notebook, and billing card ready before creating an Anthropic API key
    A minimal measurement framework you can ship this week.

    Build it in this order. None of the steps require a tool purchase to begin.

    1. Write your prompt list. Fifty prompts that a buyer in your category would actually type. Mix top-of-funnel (“what is X”), comparison (“X vs Y”), and bottom-of-funnel (“best X for Y”) in roughly equal thirds.
    2. Establish a baseline manually. Run every prompt in ChatGPT, Perplexity, and Gemini once. Record: did the response mention you, did it cite you, who was cited instead. This becomes the zero-point for the campaign.
    3. Configure GA4. Create the AI custom channel group with the regex above and place it above Referral. Verify the native AI Assistant channel is populated on the property.
    4. Set the cadence. Monthly for the manual re-run if you are unfunded. Weekly automated tracking the moment Otterly or equivalent is in the stack.
    5. Report two numbers. Mention rate and citation rate, broken down by model. Everything else is secondary.

    The honest limitation

    Every tool in this category is sampling. They re-run your prompts on their own infrastructure, not on the model instance a real user hits. The same prompt run twice in ChatGPT in the same hour can return different brand mentions because of retrieval variance and the freshness of the model’s web index. Treat any single-day number as noise and any 30-day trend as signal. The teams that get this right report on rolling four-week windows, not daily deltas.

    Where to spend next

    Once the measurement stack is live, the next dollar belongs in two places: the content updates that show up in your low-mention-rate prompts, and an LLMs.txt file if you don’t have one yet. Measurement without an action loop is a dashboard, not a campaign. The point of knowing your citation rate is to move it.

    Related on Tygart Media: measure LLM visibility in GA4 · AI citation monitoring · ChatGPT search citations.

    Frequently asked questions

    What is LLM visibility?
    LLM visibility is the percentage of relevant AI-generated answers — across ChatGPT, Perplexity, Gemini, Copilot, and Claude — in which your brand, content, or experts are mentioned or cited. It is measured by running a fixed prompt list on a recurring cadence and counting mention rate and citation rate.

    How do I track AI traffic in Google Analytics 4?
    GA4 added a native “AI Assistant” channel to the Default Channel Group in May 2026 that automatically groups sessions from recognized AI referrers. For per-platform reporting, also create a custom channel group under Admin → Data Display → Channel Groups, place it above Referral, and match the source dimension against the regex of known AI domains.

    What is the cheapest LLM visibility tool?
    Otterly is the lowest-priced serious option at $29/month on its Lite plan, with coverage of ChatGPT, Google AI Overviews, Perplexity, and Copilot. It is the recommended starting point for solo operators and small in-house teams.

    Why does AI referral traffic show up as Direct in GA4?
    Mobile apps and in-app browsers for ChatGPT, Claude, and Perplexity often strip the referrer header when a user clicks an outbound link. Without a referrer, GA4 cannot identify the source and classifies the session as Direct. Industry estimates put clean-referrer coverage at 60 to 80 percent of true AI-originated traffic.

    How often should I measure GEO performance?
    Report on rolling four-week windows, not daily deltas. The same prompt run twice in the same hour can return different brand mentions because of retrieval variance, so single-day numbers are noise. Weekly automated tracking with monthly reporting is the practitioner standard.

  • Measure LLM Visibility: The GA4 & Response-Side Stack

    Measure LLM Visibility: The GA4 & Response-Side Stack

    Traditional analytics platforms can’t see the most important impression you’re making in 2026. When a user asks ChatGPT, Perplexity, Gemini, or Claude about your category, your brand either shows up in the answer or it doesn’t — and your GA4 dashboard has no idea either way. This is the measurement blind spot at the center of generative engine optimization. If you can’t measure LLM visibility, you can’t optimize for it.

    This guide walks through the measurement stack that actually works in 2026: the GA4 channel grouping that catches AI referral traffic, the manual verification protocol that costs nothing, and the dedicated LLM visibility platforms that automate prompt monitoring at scale. By the end, you’ll have a measurement framework you can run starting today.

    Why GA4 alone is not enough

    Four-stage funnel: citation, click, engage, convert
    Why GA4 alone is not enough.

    Standard web analytics measures what happens after the click. LLM visibility is what happens before the click — or instead of one. According to widely cited industry reporting, a large share of AI search sessions end without the user ever clicking through to a source, which means the brand impression inside the AI response is often the only impression you get. GA4 cannot see that impression. It cannot see when ChatGPT recommends you in a comparison. It cannot see when Perplexity cites your article as a source for an answer.

    You still need GA4 — AI referral traffic is real, growing, and converts well — but you need it as one layer of a two-layer stack. Layer one is referral-side measurement, which captures the users who actually click through from AI platforms. Layer two is response-side measurement, which monitors what AI platforms are saying about you whether anyone clicks or not.

    Layer one: catching AI referrals in GA4

    Topic platform fit visual for first-party AI citation measurement
    Catching AI referrals in GA4.

    GA4 does not have a built-in “AI” channel. By default, traffic from ChatGPT, Perplexity, Claude, and Gemini gets bucketed into the generic Referral channel, where it disappears next to social and partner sites. The fix is a custom channel group that uses a referrer regex to peel AI traffic out into its own bucket.

    In GA4, go to Admin → Data Settings → Channel Groups, create a custom channel group, and add a new rule above the default Referral rule. Set the conditions to Source matches regex and use a pattern like this:

    chatgpt\.com|openai\.com|perplexity\.ai|claude\.ai|anthropic\.com|gemini\.google\.com|copilot\.microsoft\.com|deepseek\.com|you\.com|meta\.ai|poe\.com

    The order matters. Your AI Traffic rule must sit above the Referral rule in the priority list, or AI traffic will be captured by Referral first and never reach your custom channel. Once the rule is live, you can build Explorations that segment AI traffic by source, page, conversion rate, and engagement time — and compare that segment against organic, direct, and social.

    The referrer attribution gap

    One caveat: not every AI click passes a referrer. ChatGPT’s free tier in particular has been reported to strip referrer headers in many configurations, meaning a meaningful share of ChatGPT traffic shows up as Direct in GA4 rather than as a chatgpt.com referral. This is a known limitation across the industry. Treat your AI referral numbers as a floor, not a ceiling, and use response-side monitoring to fill in the gap.

    Layer two: response-side monitoring

    This is the measurement that traditional SEO never needed. You’re no longer just asking “did anyone visit?” — you’re asking “what is the AI saying about me?” There are two ways to answer that question.

    The manual verification protocol

    The free, no-tool approach is a structured query log. Build a list of 15 to 25 prompts that a buyer in your category would realistically type into an AI assistant. Be specific. “Best CRM for small B2B teams” is a prompt. “What is a CRM” is not — that’s a research query, not a buyer query.

    Once a week, run every prompt through each AI platform you care about — typically ChatGPT, Perplexity, Gemini, and Claude — and record three things per query: whether your brand was mentioned, whether your domain was cited as a source, and what position your brand appeared in if it was named alongside competitors. A simple spreadsheet with prompt, date, platform, mention (yes/no), citation (yes/no), and position is enough to start. Week-over-week deltas on this sheet will tell you whether your GEO and AEO work is moving the needle.

    This is slow and manual but it’s the only method that gives you ground truth. The dedicated platforms below are essentially automating this protocol — running the same kind of prompt log against the same APIs on a daily schedule. If you’re under $1,000/month in marketing spend, run it manually. If you’re past that, automate it.

    Dedicated LLM visibility platforms

    A new category of tools emerged in 2025 and matured in 2026 specifically to monitor LLM responses. They all do roughly the same thing — run your target prompts daily across multiple AI engines, score visibility, track which sources the AIs cite, and surface competitor gaps — but they segment by price point.

    At the budget end, Otterly.AI offers monitoring plans starting around $29/month, with a Share of AI Voice metric and time-to-first-data of under ten minutes after signup. It’s the simplest entry point for teams that just want a citation-frequency dashboard. In the mid-market, Peec AI starts around €89/month and emphasizes multilingual coverage and actionable recommendations — it doesn’t just tell you you’re invisible, it suggests what to change. At the enterprise tier, Profound starts around $499/month and adds Prompt Volumes, which estimates real AI search demand by topic with demographic breakdowns. SOC 2 compliance and dedicated onboarding generally start at the $1,000+ enterprise tiers across this category.

    Other platforms in active use this year include Semrush’s AI Toolkit, SE Ranking’s SE Visible, Goodie AI, Rankscale, Nightwatch, AirOps, and Searchable. The category is moving fast — pricing and features change quarterly — so verify the current state of any platform before committing.

    The six KPIs to track

    Whatever measurement stack you use, the same handful of metrics will tell you whether GEO is working. Organize them into leading and lagging indicators:

    Leading indicators (response-side, change first):

    • Mention Rate — the percentage of monitored prompts where AI responses mention your brand name. This is the broadest signal.
    • Citation Rate — the percentage of monitored prompts where your domain is cited as a source, not just named. Citation is stronger than mention because it implies the AI is treating your content as authoritative.
    • Position — when your brand is named alongside competitors, where in the list does it appear. First-named brands get disproportionate attention.

    Lagging indicators (referral and revenue-side, change later):

    • AI Referral Sessions — total sessions from your AI Traffic channel group in GA4.
    • AI Referral Engagement — engagement rate and average engagement time for the AI segment, compared to organic. Strong AI referral traffic typically engages longer because the user arrived with intent already framed by the AI.
    • AI-Influenced Conversions — conversions where AI was part of the attribution path, even if not the last touch.

    Tier-one metrics move first because content changes affect what AIs say within days to weeks. Tier-two metrics lag because they require enough traffic to be statistically meaningful, which can take a quarter or more to develop.

    The minimum viable setup

    Desk with laptop, checklist notebook, and billing card ready before creating an Anthropic API key
    The minimum viable setup.

    If you do nothing else this week, do these three things:

    1. Add the AI Traffic channel group to GA4 using the regex above and move it above Referral in priority.
    2. Build a 15-prompt spreadsheet of buyer-intent queries for your category and run them once across ChatGPT, Perplexity, Gemini, and Claude. Record mention, citation, and position.
    3. Set a calendar reminder to repeat step two every Friday for four weeks. After four weeks you’ll have a real trendline.

    That setup costs nothing and produces the measurement layer that lets you tell whether your GEO, AEO, and LLMs.txt work is actually compounding — or whether you’re guessing. Once the trendline is stable, evaluate whether automating with Otterly, Peec, or Profound is worth the spend. For most operators, the manual protocol gets you 80% of the insight at 0% of the budget.

    Related on Tygart Media: LLM visibility measurement · citation monitoring · Bing Webmaster AI tab.

    Frequently Asked Questions

    What is LLM visibility?

    LLM visibility is the measurement of how often, and how prominently, a brand or website appears in responses generated by large language models like ChatGPT, Perplexity, Gemini, and Claude. It is the response-side counterpart to traditional search ranking — instead of measuring where you appear in a results page, you’re measuring whether AI assistants mention or cite you when answering questions in your category.

    Can GA4 track AI traffic from ChatGPT and Perplexity?

    GA4 can track AI referral clicks if you create a custom channel group with a referrer regex matching AI domains and place it above the default Referral rule. It cannot track impressions inside AI responses where the user doesn’t click through, and ChatGPT’s free tier often strips referrers entirely, so a portion of AI traffic still lands in Direct. Treat GA4 numbers as a floor.

    What is the difference between mention rate and citation rate?

    Mention rate measures the percentage of monitored AI prompts where your brand name appears anywhere in the response. Citation rate measures the percentage where your specific domain or URL is referenced as a source. Citation is a stronger signal because it indicates the AI is treating your content as authoritative, not just naming you in passing.

    Which LLM visibility tool should I use in 2026?

    For budget-conscious teams, Otterly.AI starts around $29/month and gets you to first data in minutes. For mid-market needs with multilingual coverage and recommendations, Peec AI starts around €89/month. For enterprise teams that need prompt-volume demand data and SOC 2 compliance, Profound starts around $499/month. Verify current pricing before purchasing — the category moves quickly.

    How often should I check my LLM visibility?

    For manual tracking, weekly is the right cadence — frequent enough to catch movement, infrequent enough to avoid noise. Dedicated platforms typically run automated checks daily and let you review weekly. Don’t expect day-to-day stability; AI responses have inherent variance, so look at week-over-week and month-over-month trends rather than single data points.

  • How to Measure LLM Visibility: 2026 Tracking Stack

    How to Measure LLM Visibility: 2026 Tracking Stack

    Updated September 30, 2026.

    Direct Answer: Measure LLM visibility with five metrics—citation frequency, prompt coverage, share of voice, AI referral sessions, and conversion quality from that traffic. Pair a monthly 20-prompt audit with GA4: use Google’s May 2026 “AI Assistant” channel where it applies, plus a custom channel group (regex on session source, ordered above Referral) so Perplexity and historical sessions don’t stay buried in Referral or Direct.

    Most SEO teams know they need to care about AI search. Almost none of them have a measurement system in place for it. That’s the gap this article closes.

    Ranking in ChatGPT, Perplexity, Google AI Overviews, or Claude isn’t a vanity metric anymore — it’s a traffic channel. But unlike Google, AI systems don’t serve a results page you can screenshot. They weave citations into prose. Your brand either shows up in that prose or it doesn’t, and if you’re only watching default GA4 channel reports, you’re flying mostly blind.

    This is a practitioner’s setup guide: the exact metrics, GA4 configuration, and tool stack needed to track LLM visibility systematically — including how to define AI citation rate before you buy software.

    GEO versus SEO comparison cards

    The Five Metrics That Define LLM Visibility

    Five metrics that define LLM visibility.

    Traditional SEO tracks ranking position, impressions, and clicks. None of those exist in AI search. You need a new metric set:

    Citation frequency — How often your domain or brand is mentioned in AI-generated answers for your target query set. LLMs typically cite a handful of sources per response. Capturing one of those slots consistently is the entire game.

    Prompt coverage — Out of your tracked prompt library, what percentage of prompts return your brand at all? Calculate it as: (prompts where you appear ÷ total tracked prompts) × 100. A brand actively optimizing for AI search should be above 40% coverage on tier-1 prompts within 90 days of focused content work.

    Share of voice — For a given topic cluster, how often do AI answers cite you versus competitors? If you appear in 12 of 30 tested prompts and a competitor appears in 20, they hold 67% share of voice on that topic. That ratio is more strategically meaningful than any single citation count.

    AI referral sessions — The sessions in GA4 that arrived from an AI platform with a usable referrer header (or GA4’s native AI Assistant classification). This is the metric that ties visibility to business outcomes. Setup is covered in the next section.

    Conversion quality from AI traffic — AI-referred visitors often arrive with higher intent: they asked a specific question and your site was the answer. Track engagement rate, pages per session, and goal completions for AI referral sessions separately. If this cohort converts at 2–3× the rate of your organic traffic — which early practitioner data suggests — it changes how you think about GEO investment.

    Four-stage funnel: citation, click, engage, convert

    Setting Up GA4 to Capture AI Traffic

    GA4 regex to capture AI traffic.

    Native AI Assistant channel (May 2026)

    Since May 13, 2026, GA4’s default channel group includes an AI Assistant channel. When Google recognizes an assistant referrer, it sets medium ai-assistant. You will see it under Reports → Acquisition → Traffic acquisition. That is progress, but it is not a complete LLM visibility stack: Perplexity commonly still lands in Referral, assistant coverage in Google’s list changes, and clicks from Google AI Overviews or AI Mode still arrive as google.com — they stay in Organic Search, not AI Assistant.

    Custom channel group and regex

    For Perplexity, legacy referrers, and retroactive reporting, build your own group. In GA4: Admin → Data Display → Channel Groups → Create New Channel Group. Name it “AI Search” and add a rule:

    • Condition type: Session source
    • Match type: matches regex
    • Pattern (copy exactly):
    ^(.*\.)?(chatgpt\.com|chat\.openai\.com|perplexity\.ai|claude\.ai|anthropic\.com|gemini\.google\.com|bard\.google\.com|copilot\.microsoft\.com|deepseek\.com|grok\.com|meta\.ai|you\.com|poe\.com)$

    Critical step: Place “AI Search” above “Referral” (and treat order as first-match-wins). If Referral appears first, Perplexity and other assistant referrers never reach your AI rule. Custom channel groups can be applied retroactively in reports; the native AI Assistant channel does not backfill history the same way.

    Scope caveat: Many assistant-driven visits still arrive without a referrer — mobile apps, in-app browsers, and privacy modes send them to Direct or (not set). Your AI channel captures the visible minority. Don’t benchmark on absolute volume alone. Benchmark on growth rate, and watch Direct only when other signals line up.

    To supplement GA4, add a self-reported source question on high-intent forms: “How did you find us?” Include “ChatGPT / AI assistant” as an option. That gives ground truth session data cannot.

    The Tool Tier: Free to Enterprise

    The LLM visibility tool market matured through 2025 and 2026. Three tiers cover most teams. Start manual before you pay — the same discipline described in our AI citation monitoring guide applies whether you use a spreadsheet or a dashboard.

    Free / DIY — start here

    Run 20 representative prompts manually across ChatGPT, Perplexity, Claude, and Google AI Overviews each month. Record mentions in a spreadsheet: cited (yes/no), cited with link (yes/no), competitor named instead. That gives you baseline prompt coverage and share of voice with zero budget. Do it for at least one month before buying software — you’ll know which gaps actually hurt.

    Mid-market tools (roughly $30–$500/month)

    OtterlyAI runs scheduled prompt sets and tracks brand mentions, cited URLs, and share of voice. Published USD pricing starts at $29/month (Lite, 15 prompts); Standard is $189/month (100 prompts); Premium is $489/month (400 prompts). ChatGPT, Google AI Overviews, Perplexity, and Copilot are included; Google AI Mode, Gemini, and Claude are add-ons. Enterprise starts around $1,000/month.

    LLMrefs takes a keyword-first approach: import SEO keywords, and the platform generates prompt fan-outs and refreshes visibility on a schedule — closer to a rank tracker than a blank prompt spreadsheet. Useful when your team already thinks in keyword lists rather than conversational prompts.

    Enterprise layer ($1,000+/month)

    Profound centers on Prompt Volumes — modeled demand from real answer-engine conversations (ChatGPT, Gemini, Claude, Perplexity). It helps you prioritize which questions deserve content before you commit headcount. Valuable at scale; overkill for a single-location operator on the 20-prompt audit. Pair it with playbooks such as the GEO case studies series when you need proof the numbers moved.

    The 20-Prompt Audit: Your Monthly Baseline Protocol

    Whether you use a paid tool or not, run this protocol monthly:

    1. Build a prompt library of 20 questions your target buyer would ask an AI system. These should be questions your content is designed to answer — not keyword-stuffed phrases, but real conversational queries.
    2. Run each prompt across ChatGPT, Perplexity, and Google AI Overviews (3 platforms × 20 prompts = 60 data points per month).
    3. For each result, record: was your brand cited in text, was your domain linked, and which competitor was cited if you were not.
    4. Calculate prompt coverage per platform and total share of voice versus your top three competitors.

    Log results in a spreadsheet with a date column. Three months of data reveals directional trends — whether your GEO and AEO work is moving the needle. Longitudinal baselines also make experiments legible; structured publishing tests (for example, the 40-article Bing citation experiment) only pay off when you can compare citation rates month over month.

    Desk with laptop, checklist notebook, and billing card ready before creating an Anthropic API key

    Diagnosing a Citation Drop

    Diagnosing a citation drop.

    If your monthly audit shows prompt coverage declining, run through this checklist before blaming a black-box algorithm change:

    Did you remove or restructure a previously cited page? AI systems build representations of your content over time. Pages that disappear or get heavily rewritten lose citation weight. Check your changelog against the prompts that declined.

    Did a competitor publish stronger content on the topic? Citation slots in a single answer are finite. If a competitor shipped a more authoritative page, it may have displaced yours. Review their recent publishing calendar.

    Check LLMs.txt and robots.txt. A crawlability block in LLMs.txt or a misconfigured Disallow directive cuts AI access at the source. Verify both files allow the URLs you expect to be cited.

    Check for a platform or model update. Major chatbot and search-model releases in 2025–2026 shifted which sources each engine trusted. Read the platform’s public changelog for the period in question.

    If none of these apply, audit structured data on the pages that lost citations — schema, FAQ blocks, heading hierarchy, and factual density all affect extraction. A redesign that removed an FAQ block can remove your citation utility in the same deploy.

    The Bottom Line

    LLM visibility measurement is not solved, but the primitives exist: GA4’s AI Assistant channel plus a custom regex group for traffic, manual prompt audits for citation coverage, and mid-market tools when labor becomes the bottleneck. Operators who need a packaged starting point can map the same metrics to a vertical playbook such as the AI search visibility package for water damage — the measurement layer stays identical even when the prompts change.

    Build the 20-prompt library this week. Configure GA4 today. Everything else stacks on those two streams.

    FAQ

    What is LLM visibility? LLM visibility is how often your brand or domain appears in AI-generated answers — in the prose, in linked citations, or both — for the questions your buyers ask. It is the AI-search equivalent of showing up on page one, except there is no single rank number to screenshot.

    Does GA4 automatically track ChatGPT and Perplexity traffic? Partially. Since May 2026, GA4’s AI Assistant channel classifies some assistant referrers with medium “ai-assistant.” Perplexity often remains Referral, AI Overview clicks stay Organic, and no-referrer visits stay Direct. A custom “AI Search” channel group with a source regex — placed above Referral — is still required for complete trending.

    How often should I run a prompt audit? Monthly is the minimum useful cadence. Run the same 20 core prompts each cycle so prompt coverage and share of voice are comparable. Weekly spot checks are fine for crisis monitoring, but monthly logging is what builds a baseline.

    What is the difference between prompt coverage and share of voice? Prompt coverage is the share of your tracked prompts where you appear at all. Share of voice compares how often you appear versus named competitors across that same prompt set. High coverage with low share of voice means you show up, but rivals dominate the answers.

    Do I need a paid tool to measure AI citations? No. A spreadsheet and 60 manual checks per month (20 prompts × three platforms) are enough to start. Paid tools mainly automate scheduling, add engines, and export reports once you know which prompts and competitors matter.

    Why would AI citations drop suddenly? Common causes are URL removals or restructures, stronger competitor pages, crawl blocks via robots.txt or LLMs.txt, or model updates on the assistant platform. Rule those out before you assume your entire GEO strategy failed.