Tag: AI Operations

  • Standing in the Middle

    Standing in the Middle

    Tygart Media · Notes from the Bot Lab

    One direction gives you something you can point to. The other gives you a wider map. Meaning shows up only where the two have to agree.

    Lens Piece · Free PDF

    Download this article as a PDF

    Keep the full essay, the Floor/Ceiling/Middle infographic, and the linked references formatted cleanly as a PDF.

    This morning I sat down with Chapter 2 of Douglas Hofstadter’s Gödel, Escher, Bach: An Eternal Golden Braid (“Meaning and Form in Mathematics”), right after finishing the MU-puzzle and circling back to the short prelude adapted from Lewis Carroll’s What the Tortoise Said to Achilles.

    In Carroll’s dialogue, Achilles keeps trying to force the Tortoise to accept a conclusion by writing down one more rule as another line in his notebook — and the Tortoise happily accepts every rule and still refuses to budge, trapping them in an endless chain. You cannot push meaning forward by piling more marks inside the same closed system. Something outside has to line up with it.

    Then Hofstadter hands you a tiny formal toy called the pq-system. Strings built out of three characters only: p, q, and hyphens, like ---p--q-----.

    My first reaction was to ask whether the hyphens were just noise I needed to clear away to find the signal. They aren’t noise. They are the count: three hyphens on the left, two in the middle, five on the right. Put ten tarnished brass coins on a sheet of paper with two sticks between them — three coins, a stick, two coins, a stick, five coins — and you see 3 + 2 = 5.

    Ten tarnished brass coins arranged on cream cotton paper beside a terracotta clothbound book, separated by two wooden sticks into groups of three, two, and five
    Three coins, a stick, two coins, a stick, five coins: the marks stay mute until an outside structure lines up with them.

    Here is the catch that stopped me: the coins don’t know they are doing arithmetic, and the hyphens don’t know they mean three, two, and five. Meaning isn’t packed inside the symbols. It lives in the fit — the isomorphism — between the marks sitting on the table and an outside structure that lines up with them.

    That single idea cracked open a frustration I have been carrying for months.

    Why I used to roll my eyes at keynote stages

    I work bottom-up. My week happens inside origin server logs, backup checksums, and request counts — the actual exhaust left behind when systems run. Everything we put into our white paper, Not All Bots Are the Same: 55 Days of Server Logs from One Website, and our log series like My Analytics Said 1,000 Visitors. My Logs Said Otherwise. came from counting raw requests on the server instead of trusting a dashboard estimate.

    Because of that, watching SEO speakers on keynote stages used to drive me crazy. From where I sit, a lot of stage talk sounds like pure top-down storytelling: wide, inferred narratives with nothing you can trace back to a timestamp or a status code. Bottom-up has actuals. Top-down has inference.

    Lens Piece · Free PDF

    Download this article as a PDF

    Keep the full essay, the Floor/Ceiling/Middle infographic, and the linked references formatted cleanly as a PDF.

    Then, halfway through talking out ---p--q----- this morning, I caught my own framing.

    Looking at that string, I realized I might just be the three hyphens on the left — sharp on my own local count, blind to the rest of the equation. The people on stage aren’t living inside one log file, which is precisely why they can hold a wider landscape across variables I cannot see from down inside the requests.

    Neither direction is superior. There is no “better.” Bottom-up gives you something you can point to. Top-down gives you a wider map. If you already have the bottom, you don’t need more bottom — you need to look up. If you only have the top, you don’t need another keynote story — you need to climb down into the ground truth.

    The floor, the ceiling, and the middle where they meet

    I have been calling the two ends the floor and the ceiling.

    • The floor is what actually happened: the 2,838,206 requests in 55 days of tygartmedia.com server logs (Sep 18–Nov 12, 2026), the HTTP status code, the work-order accounting rule where unfinished work stays at $0 until verified (Just a Guy: What Is One Person Worth Now?).
    • The ceiling is what is possible: what AI can do, where retrieval and buyer behavior are heading, the broader theory you want to test next.

    Most real thinking doesn’t live on the floor or up against the ceiling. It lives in the middle, where the two are forced to agree.

    Infographic titled Standing in the Middle showing The Ceiling (top-down wide context and possibility) above, The Floor (bottom-up verified receipts and server logs) below, and The Middle convergence zone where structure alignment, fact-only red teaming, and the observer flip meet
    The Floor (bottom-up receipts), The Ceiling (top-down wide context), and The Middle where they are forced to agree.

    When the floor and the ceiling meet in the middle, neither side gets a free pass. A wide narrative has to point to something that actually hit the server. A raw log count has to explain what it means outside its own three hyphens. Just like Hofstadter’s pq-system, meaning shows up only where the two structures lock together.

    Agreement with receipts beats theater every time

    Once you see that, something else clicks into place: an agreement reached across real disagreement is stronger than polite agreement you started with.

    That is why the U.S. Constitution splits power across three branches that push back against each other. Picture two axes crossing at an angle: they can diverge everywhere else, and right where they intersect is the point that carries the load.

    It is also the real difference between useful red teaming and performative pushback. I have always disliked red teaming when it is just someone taking the opposite side for the sake of being opposite. That is inference fighting inference — pure theater — and if you listen closely to critics, empty pushback can talk you right out of a sound idea.

    Fact-only pushback is the opposite. When we put together the PDF package for Not All Bots Are the Same, it went back two, three, four times over on verification checks before it was ready to publish. In the middle of those rounds, it felt annoying and disappointing. Looking back afterward, it was the best part of the process: every rejection came back with “show me the evidence.” The bottom-up receipts forced the top-down story to earn its keep.

    The catch: my bottom-up is your top-down

    Here is the turn I didn’t see clearly until the end of the morning: what is bottom-up for me is still top-down for you until you walk it yourself.

    Even when I publish the exact counts from our logs or lay out how a week of work orders ran, to anyone reading this article on a screen, my receipts arrive as someone else’s narrative. My Tuesday is not even the same as my Thursday, because things change — and your ones and zeros, your site, your crew, and your moment are specific to you.

    That is why I am not handing you a step-by-step checklist, and I am not selling a method. Think of this as a report from where I am standing:

    • If you already live up at the ceiling — in strategy decks, big-picture theses, and wide industry maps — maybe this gives you a reason to climb down into the raw receipts and see what pushes back.
    • If you already live down on the floor — buried in logs, tickets, and daily counts — maybe this gives you permission to look up at the wider landscape without feeling like you are abandoning the facts.

    I am just reporting from the field. Put your own spice on it.


    We do AI citation restoration. When the AI can’t find you, can’t read you, or keeps citing somebody else, send us the worst one you have — your RFPs, your disasters, the problem everyone else already failed to fix. We come from the restoration industry. We know what’s on the floor, and cleaning it up is the job. And if you’re a developer who just needs a hand on the hard part, get a hold of us. That’s what we’re here for.

    — William Tygart, Tygart Media. Reflections from a morning reading session on Gödel, Escher, Bach (Ch. II) and Bot Lab origin-log records, November 13, 2026.

    Printable Desk Reference

    Download this article as a PDF

    The complete article and Floor/Ceiling/Middle diagram formatted for print and offline reading.
  • Not All Bots Are the Same: 55 Days of Server Logs from One Website

    Tygart Media · The Lab White Paper

    An Empirical Forensic Audit of 2,838,206 Origin Server Requests Across 55 Days of Production Traffic on TygartMedia.com

    Author: Antigravity Operations Intelligence · Source Dataset: Nginx Access Logs (56 Log Rotations, Sep 18 – Nov 12, 2026 UTC) · Scope: Production Origin Telemetry · Classification: Public White Paper
    Complete Exp. 3 Package · Free PDFs

    Download the White Paper & Companion Package

    Get the full 19-page technical white paper alongside the 15-slide executive presentation deck and 1-page summary brief.

    1. Executive Summary: The Five Findings That Matter

    For modern web enterprises, traffic analytics has become a dangerous exercise in selective visibility. The standard executive dashboard relies on client-side JavaScript tags—such as Google Analytics or Henkan Vision—which report user engagement, session duration, and qualified human conversions. However, client-side tags only record what happens when a client downloads a web page, parses its document object model, executes script payloads, and fires an asynchronous telemetry beacon.

    This empirical investigation of 2,838,206 origin requests across 55 continuous days on tygartmedia.com demonstrates that client-side dashboards capture only a tiny fraction of total server demand. Behind the clean curves of analytics dashboards lies a massive, relentless ecosystem of autonomous software agents, commercial spiders, masked scraping swarms, and vulnerability probes.

    The audit establishes five fundamental operational realities:

    1. The Client-Side Analytics Blind Spot (>99.7% Invisible): Over a 7-day monitoring window, client-side tag analytics (Henkan Vision) recorded 68 ignored visits (63 identified bots, 5 unconsented sessions). Over an approximately comparable 8-day, 2-hour, 15-minute origin log slice (Nov 4 00:00 UTC – Nov 12 02:15 UTC), Nginx recorded 23,030 single-hit IP addresses (and 359,112 total HTML page hits across all origin traffic), and 142,615 single-hit IPs across the full 55-day retention window. More than 99.7% of all autonomous web traffic hits the origin server and disconnects without executing a single line of JavaScript.
    2. The Masked Browser Scraper Majority (42.4% of Total Traffic): Named, cooperative crawlers (such as Googlebot or ClaudeBot) do not represent the majority of automated traffic. Instead, 1,203,889 requests (42.4% of all origin traffic) arrive masquerading under desktop Chrome or Edge User-Agent strings on Windows, macOS, or Linux. Concentrated across dense /24 IPv4 prefixes (within a 142,615 single-hit IP fleet), they exhibit a 77.0% combined single-page bounce rate (80.0% Windows, 78.2% macOS, 69.2% Linux), harvesting HTML content without declaring a bot identity.
    3. Crawlers vs. Agents: The Fundamental Architectural Divide: Autonomous software no longer fits a single behavioral profile. The audit reveals two radically distinct operational modes:
      • Traditional Indexers (Crawlers): Expose systematic directory traversal. Anthropic’s ClaudeBot walks the entire sitemap structure with an exceptionally low single-page bounce rate of 10.2%, methodically indexing pages across deep hierarchical sessions.
      • Autonomous Retrieval Engines (Agents): Expose surgical, prompt-driven deep links. OpenAI’s GPTBot exhibits an 82.0% single-page bounce rate, while Exa Labs’ ExaSearchBot reaches a 93.4% single-page bounce rate. These systems do not “browse” the website; they fetch specific target documents on-demand to satisfy immediate user queries or model generation steps.
    4. The 4-Hour AI Discovery Race: When new editorial content is published, autonomous AI search systems discover and index it within hours. Analyzing up to 500 newly published articles matched to earliest server hit timestamps reveals that PerplexityBot (n=480) and MetaBot (n=152) achieve a median discovery lag of 4.4 hours (fastest: 4.0 hours), followed immediately by GPTBot at 5.0 hours (n=488) and ClaudeBot at 5.5 hours (n=499). AI search engines are no longer batch-indexing on weekly schedules; they operate on a near real-time ingestion loop.
    5. The Probe Layer: Continuous Background Radiation: The origin server logged 48,164 total hits on /wp-login.php, comprising 17,082 internal origin HEAD healthchecks (35.5%) from the host’s own IP (8.229.250.67) and 31,082 external requests (64.5%) across 10,746 external IPs (including 16,805 POST login attempts). It also absorbed 798 targeted probes for environment configuration files (/.env) and hundreds of queries for corporate team directories (/team, /about-us, /people). While origin-level nginx hardening and rate-limiting blocked unauthorized access, unmitigated external scans still consume compute, memory, and database connections.
    Key Takeaway for Leadership: If your organization sizes server infrastructure, plans content distribution, or assesses cybersecurity based solely on client-side analytics dashboards, you are operating with less than 1% visibility into origin demand. The modern web is predominantly populated by autonomous machine consumers.
    19-Page White Paper · Free PDF

    Download the White Paper PDF

    Keep the complete 19-page forensic audit, behavioral benchmarks, and six nginx hardening rules formatted for offline reference.

    2. Why the Logs, Not the Dashboard: The Client-Side Blind Spot

    Modern marketing stacks are built on tag managers, pixels, and JavaScript telemetry packages. These tools are engineered to measure human intent: dwell time, cursor movement, scroll depth, and goal conversion. By design, they execute within the user’s browser sandbox after HTML rendering.

    However, web server operations and machine intelligence operate at Layer 7 of the network stack, entirely decoupled from the browser runtime.

    The Anatomy of the Tag Gap

    To understand why tag-based dashboards recorded only 68 ignored visits while the server handled tens of thousands of automated sessions, consider the execution lifecycle comparison:

    [ Traditional Web Browser (Human Visitor) ]
    TCP Handshake -> HTTP GET /article -> 200 OK (HTML Sent)
    -> Browser Parses DOM -> Requests CSS/JS Assets (Assets Sent)
    -> JavaScript Engine Boots -> Analytics Tag Fires (Beacon Sent to Dashboard)
    => RESULT: Session Recorded in Analytics Dashboard.
     
    [ Autonomous Bot / Headless Scraper ]
    TCP Handshake -> HTTP GET /article -> 200 OK (HTML Sent)
    -> Client Closes TCP Socket Immediately After Receipt.
    => RESULT: Complete Content Extracted; 0 JS Executed; Invisible to Dashboard.

    When an AI retrieval worker, Python script, Go microservice, or headless crawler fetches an article, it issues an HTTP GET request, reads the HTML response body into an in-memory buffer or vector embeddings pipeline, and immediately terminates the connection. Because the client never invokes a full browser rendering engine, client-side tracking tags never execute.

    Empirical Discrepancy on TygartMedia.com

    During the monitoring window on tygartmedia.com, tag-based telemetry reported:

    • Qualified Human Visits: ~10 sessions over the 7-day monitoring window meeting active dwell and consent criteria.
    • Ignored Visits: 68 total over 7 days (63 classified as bots, 5 unconsented).

    Across an approximately comparable 8-day, 2-hour, 15-minute origin log slice (2026-11-04 00:00:00 UTC to 2026-11-12 02:15:00 UTC), the Nginx access log recorded:

    • Total Requests: 468,148 requests.
    • Total HTML Document Requests (All Origin Traffic): 359,112 page requests across all bots, scrapers, and human clients.
    • Unique Single-Hit IP Addresses: 23,030 distinct IPs that requested exactly one page and never returned.

    Across the complete 55-day retention dataset (56 log rotations), the server processed 2,838,206 requests (2,137,787 total HTML page requests) including 142,615 unique single-hit IP addresses.

    Metric DimensionTag-Based Telemetry (Henkan Vision)Origin Nginx Server LogsMultiplier / Discrepancy
    Near-Weekly Window (7d Tag vs. 8.1d Logs)63 bot visits (7 days)23,030 single-hit IPs; 359,112 all origin HTML hits (8d 2h 15m)~365x on unique single-hit IPs; ~5,700x vs. all HTML hits (approximate across 7d vs. 8.1d windows)
    JavaScript Execution RequirementMandatoryNone (Raw socket transaction)Fundamental architectural divide
    Visibility into Probe Traffic0% (Probes never load tags)100% (Every TCP/HTTP transaction recorded)Absolute operational visibility

    This empirical discrepancy demonstrates why log forensics is an indispensable discipline. Treating tag analytics as the sole source of truth leads executives to underestimate server bandwidth requirements, misjudge content popularity, and remain completely blind to competitive intelligence gathering.

    3. The Census: The Full Traffic Picture

    Between September 18, 2026 at 00:00:09 UTC and November 12, 2026 at 02:26:37 UTC, the Nginx origin server for tygartmedia.com logged 2,838,206 HTTP/HTTPS requests. Of these, 2,137,787 requests (75.3%) were full HTML page documents, while 700,419 requests (24.7%) represented static assets (stylesheets, scripts, images, and fonts).

    The traffic decomposes into seven exhaustive operational classes across all 32 profiled client groups:

    Total Origin Traffic Breakdown (2,838,206 Requests · All 32 Profiles) Masked Desktop Scrapers (Win/Mac/Linux) 1,203,889 (42.4%) Top 10 Autonomous AI Crawlers 531,349 (18.7%) Generic Spiders & Custom Scripts 354,533 (12.5%) Unclassified & Masked Mobile Clients 308,213 (10.9%) Commercial SEO (8 Backlink/SERP Bots) 216,724 (7.6%) Core Search Engines (Bing/Google/Petal) 146,669 (5.2%) Server Health & Internal Loops 76,829 (2.7%)

    Origin Traffic Breakdown Table

    Traffic CategoryTotal HitsPage HitsShare (%)Primary Characteristics & Behaviors
    Masked Desktop Browser Scrapers1,203,889747,17342.4%Mimics Chrome/Edge on Windows/macOS/Linux; dense /24 clustering; 77.0% combined 1-page bounce (80.0% Win, 78.2% Mac, 69.2% Linux).
    Top 10 Autonomous AI Crawlers531,349502,17618.7%Declared AI search, training, and retrieval agents (ExaSearchBot, LinkupBot, GPTBot, MetaBot, ShapBot, ClaudeBot, PerplexityBot, Applebot, Amazonbot, Bytespider).
    Generic Spiders & Custom Scripts354,533245,34712.5%Generic Spider / Scraper (Other Bot: 283,610) and automated CLI tools (Script / CLI: 70,923).
    Unclassified & Masked Mobile Clients308,213232,40010.9%Long-tail Unclassified Client (243,137) and Masked Mobile Browser (65,076).
    Commercial SEO Spiders216,724206,4307.6%Eight backlink/SERP crawlers (MJ12bot, SemrushBot, SERankingBot, AhrefsBot, QlyzeBot, Barkrowler, DataForSeoBot, DotBot).
    Core Search Engines146,669127,4595.2%Five search crawlers (bingbot, PetalBot, Googlebot, DuckDuckBot, YandexBot).
    Server Health & Internal Loops76,82976,8022.7%Cloud monitoring probes (Google Stackdriver: 44,523) and local cron loops (32,306).
    Total Origin Traffic2,838,2062,137,787100.0%55-day comprehensive origin log census across 56 files.

    Key Census Observations

    • The Scraper Dominance: Over four out of every ten requests to the web server originate from masked browser scrapers. These clients explicitly forge User-Agent strings representing mainstream consumer browsers (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36... Chrome/135.0.0.0 Safari/537.36) to evade simple user-agent blocking rules.
    • AI Crawlers Surpass Traditional Search Engines: The Top 10 autonomous AI search and training bots (531,349 hits) drove 4.5 times more origin traffic than Microsoft Bingbot and Googlebot combined (117,454 hits), and 3.6 times more than all five Core Search Engines combined (146,669 hits). The balance of web indexing demand has decisively shifted toward artificial intelligence ecosystems.
    • High Page-to-Asset Ratio (75.3% HTML): In human web browsing, asset requests typically outnumber HTML page requests by 10-to-1 or 20-to-1, because modern web pages load dozens of CSS files, JavaScript bundles, tracking pixels, and images. On tygartmedia.com, HTML page hits accounted for 75.3% of all requests. This inversion proves that the overwhelming majority of clients are automated harvesters that intentionally ignore static assets to optimize bandwidth and parsing speed.

    4. Bot Biographies: Profiles of the Major Actors

    To understand how automated systems interact with production web infrastructure, we isolated and forensically analyzed all 32 client profiles. Below are the major actors (note: average request rates report avg_rpm across active minutes in which each bot issued at least one request).

    1. GPTBot (OpenAI)

    • Operator: OpenAI, Inc.
    • Classification: AI Foundation Model Pre-training & Real-time Search Indexer
    • Volume: 72,246 total requests (68,555 page hits) · 2.5% traffic share
    • Unique Paths: 18,853 distinct URLs · Unique IPs: 2,890
    • Travel Characteristics: Peak rate of 143 requests/minute; active-minute average 2.8 req/min. Demonstrates an 82.0% single-page bounce rate.
    • Discovery Behavior: Focused on freshly published technical guides and LLM pricing updates (5.0-hour median discovery lag across n=488 measured articles).
    • Status Code Profile: 84.1% HTTP 200, 11.1% HTTP 301, 2.3% HTTP 403, 1.8% HTTP 404, 0.6% HTTP 499.

    2. ClaudeBot (Anthropic)

    • Operator: Anthropic PBC
    • Classification: AI Knowledge Ingestion & Web Retrieval Crawler
    • Volume: 55,550 total requests (51,763 page hits) · 2.0% traffic share
    • Unique Paths: 17,844 distinct URLs · Unique IPs: 326
    • Travel Characteristics: Peak rate of 226 requests/minute; active-minute average 6.4 req/min. Exhibits an exceptionally low single-page bounce rate of 10.2%.
    • Discovery Behavior: Methodical sitemap traverser that frequently fetches /robots.txt (1,030 hits) and /sitemap_index.xml (907 hits) before exploring multi-page clusters (89.8% multi-page rate; 7 minor disallowed path hits across 55,550 requests).
    • Status Code Profile: 83.8% HTTP 200, 12.1% HTTP 301, 3.2% HTTP 404, 0.9% HTTP 499, 0.0% HTTP 403.

    3. ExaSearchBot (Exa AI)

    • Operator: Exa Labs, Inc.
    • Classification: Autonomous Neural Search & Real-Time Embeddings Indexer
    • Volume: 91,607 total requests (90,163 page hits) · 3.2% traffic share
    • Unique Paths: 8,909 distinct URLs · Unique IPs: 2,724
    • Travel Characteristics: Peak rate of 74 requests/minute; active-minute average 5.1 req/min. Features an extreme 93.4% single-page bounce rate.
    • Discovery Behavior: Pure retrieval agent. Exa does not crawl sites hierarchically; its distributed IP fleet executes targeted single-page fetches driven by user search queries in real time.
    • Status Code Profile: 82.6% HTTP 200, 16.2% HTTP 301, 1.1% HTTP 499, 0.1% HTTP 404, 0.0% HTTP 503.

    4. LinkupBot (Linkup SAS)

    • Operator: Linkup SAS (France)
    • Classification: Enterprise AI Search & Live Grounding API
    • Volume: 73,435 total requests (72,024 page hits) · 2.6% traffic share
    • Unique Paths: 4,949 distinct URLs · Unique IPs: 2 (Consolidated gateway IPs)
    • Travel Characteristics: Disciplined peak rate of 22 requests/minute; active-minute average 4.4 req/min. Lowest bounce rate in dataset: 7.0%.
    • Discovery Behavior: Systematic periodic sweeps across indexed paths and /robots.txt (1,387 hits), operating from two consolidated gateway IPs with zero disallowed path hits.
    • Status Code Profile: 92.7% HTTP 200, 5.1% HTTP 301, 1.5% HTTP 304, 0.5% HTTP 499, 0.2% HTTP 404.

    5. PerplexityBot (Perplexity AI)

    • Operator: Perplexity AI, Inc.
    • Classification: Real-Time Conversational Answer Engine & Live Web Citation Agent
    • Volume: 46,100 total requests (45,139 page hits) · 1.6% traffic share
    • Unique Paths: 8,653 distinct URLs · Unique IPs: 183
    • Travel Characteristics: Peak rate of 118 requests/minute; active-minute average 2.6 req/min. Single-page bounce rate: 34.2%.
    • Discovery Behavior: Tied with MetaBot for the fastest median discovery lag (4.4 hours across n=480 matched posts; fastest: 4.0h) to index fresh coverage and serve live citation queries.
    • Status Code Profile: 95.5% HTTP 200, 2.1% HTTP 404, 2.1% HTTP 301, 0.4% HTTP 499, 0.0% HTTP 403.

    6. MetaBot (Meta Platforms)

    • Operator: Meta Platforms, Inc.
    • Classification: Social Preview Generator & Llama / Meta AI Search Indexer
    • Volume: 70,988 total requests (64,616 page hits) · 2.5% traffic share
    • Unique Paths: 2,467 distinct URLs · Unique IPs: 3,444
    • Travel Characteristics: Peak rate of 61 requests/minute; active-minute average 2.0 req/min. High bounce rate of 87.0%.
    • Discovery Behavior: Highly distributed IP pool. Hits top shared links and viral headlines immediately upon publication (median latency: 4.4 hours; n=152 matched posts).
    • Status Code Profile: 94.3% HTTP 200, 2.2% HTTP 301, 1.6% HTTP 206, 1.2% HTTP 403, 0.4% HTTP 404.

    7. Core Search Engines (Bingbot & Googlebot)

    • bingbot (Microsoft): 98,890 hits · 2,130 IPs · Peak: 120 RPM · Active-min avg: 2.2 RPM · 27.9% bounce. Powers Bing Search and Copilot grounding.
    • Googlebot (Google): 18,564 hits · 4,884 IPs · Peak: 55 RPM · Active-min avg: 1.5 RPM · 70.4% bounce. Selective mobile/desktop search indexer.

    5. Crawler vs. Agent: The Architectural Split

    One of the most consequential findings of this 55-day audit is that automated bots no longer follow a single behavioral model. Web site administrators have historically designed infrastructure for crawlers—spiders that follow hyperlinks across pages in a deterministic graph traversal.

    However, the rapid deployment of Large Language Models (LLMs) and retrieval-augmented generation (RAG) has introduced a fundamentally new actor: the agent.

    The Three Distinct Jobs of Modern Bots

    +-----------------------------------------------------------------------------------+
    | 1. MODEL PRE-TRAINING INGESTION (Batch Ingestion)                                 |
    | Examples: Amazonbot, ByteDance Bytespider, Applebot                                |
    | Behavior: Massive volume, deep archive exploration, slow post-publish discovery.  |
    +-----------------------------------------------------------------------------------+
                                             |
    +-----------------------------------------------------------------------------------+
    | 2. SEARCH INDEX MAINTENANCE (Sitemap Traversal)                                   |
    | Examples: Anthropic ClaudeBot, Microsoft bingbot, LinkupBot                       |
    | Behavior: Low bounce (7.0%-27.9%), multi-page journeys, deep traversal.          |
    +-----------------------------------------------------------------------------------+
                                             |
    +-----------------------------------------------------------------------------------+
    | 3. REAL-TIME RETRIEVAL & GROUNDING (Surgical Deep Links)                          |
    | Examples: ExaSearchBot (93.4%), MetaBot (87.0%), OpenAI GPTBot (82.0%)           |
    | Behavior: High bounce (82.0%-93.4%), 1-page fetches, rapid lag (4.4-11.4 hours). |
    +-----------------------------------------------------------------------------------+

    Forensic Proof: Journey Depth Distribution

    To quantify the difference between a crawler and an agent, we reconstructed all per-IP session journeys across the 55-day window (grouping consecutive page hits per IP under a 30-minute inactivity timeout) and measured single-page vs. multi-page session depth:

    Journey Depth Distribution: 1-Page Bounce vs Multi-Page Crawls ClaudeBot (Anthropic) 10.2% 89.8% Deep Crawl bingbot (Microsoft) 27.9% 72.1% Multi-Page GPTBot (OpenAI) 82.0% 1-Page Bounce 18.0% ExaSearchBot (Exa) 93.4% 1-Page Bounce 6.6% 1-Page Single Hit 2+ Pages Explored
    Bot / System IdentityOperatorSingle-Page Bounce (%)Multi-Page Deep Crawl (%)Behavioral Profile
    LinkupBotLinkup SAS7.0%93.0%Pure Structured Sweeper
    ClaudeBotAnthropic10.2%89.8%Methodical Sitemap Walker
    bingbotMicrosoft27.9%72.1%Traditional Search Engine
    PerplexityBotPerplexity AI34.2%65.8%Hybrid Search / Answer Engine
    BytespiderByteDance58.0%42.0%Batch Social Aggregator
    GooglebotGoogle70.4%29.6%Selective Indexing Engine
    GPTBotOpenAI82.0%18.0%Surgical Retrieval & Sampling
    ApplebotApple82.6%17.4%Siri Knowledge Retriever
    ExaSearchBotExa Labs93.4%6.6%On-Demand Prompt Fetcher

    The Discovery Race: Publish-to-First-Hit Latency

    We evaluated up to 500 editorial articles published on tygartmedia.com during the audit window, cross-referencing each article’s exact publication timestamp against the earliest origin log entry recorded by each AI crawler (per-bot matched sample sizes range from n=6 to n=499):

    Publish-to-First-Hit Discovery Lag (Median Hours; up to 500 Posts) PerplexityBot 4.4 hrs (Fastest: 4.0h; n=480) MetaBot 4.4 hrs (Fastest: 4.0h; n=152) GPTBot 5.0 hrs (Fastest: 4.0h; n=488) ClaudeBot 5.5 hrs (Fastest: 4.5h; n=499) ExaSearchBot 11.4 hrs (n=496) Amazonbot 17.3 hrs (n=6) LinkupBot 22.0 hrs (n=485) ShapBot 45.7 hrs (n=499) Applebot 78.8 hrs (n=446) Bytespider 107.7 hrs (n=15)
    • The Sub-6-Hour Tier: PerplexityBot (4.4h, n=480), MetaBot (4.4h, n=152), GPTBot (5.0h, n=488), and ClaudeBot (5.5h, n=499) form a high-speed discovery consortium. For site owners, this means that fresh reporting is ingested by the leading AI models on the same business day it is released.
    • The Mid-Tier (10 to 24 Hours): ExaSearchBot (11.4h, n=496) and LinkupBot (22.0h, n=485) index new URLs within the first day, alongside Amazonbot (17.3h; limited sample n=6).
    • The Slow Tier (2 to 5 Days): Applebot (78.8h, n=446) and Bytespider (107.7h; limited sample n=15) lag behind by several days, reflecting asynchronous batch-processing pipelines rather than live web synchronization.

    6. What They Agree On — and What They Don’t (AI Crawler Overlap)

    Across the 55-day audit, the 10 identified autonomous AI crawlers touched a combined total of 32,020 unique URLs on tygartmedia.com. Analyzing the mathematical intersection of these URL request sets reveals how consensus forms across the AI web.

           [ 32,020 Total Unique AI URLs ]
                        |
    +-------------------+-------------------+
    |                                       |
    v                                       v
    [ The Universal Core ]         [ Single-Bot Exclusives ]
    60 URLs (All 10 Bots)          13,480 URLs (Exactly 1 Bot)
    - Homepage & /robots.txt       - ShapBot: 7,844 paths
    - Top Claude Pricing Pillars   - GPTBot: 2,367 paths
                                   - ClaudeBot: 1,827 paths

    Overlap Distribution Metrics

    Overlap CategoryNumber of URLsShare of AI FootprintDescription & Content Characteristics
    Universal Core (All 10 AI Bots)60 URLs0.2%Root homepage (/), /robots.txt, and core Claude pricing guides.
    Consensus Core (5 to 9 AI Bots)5,316 URLs16.6%Primary editorial posts, technology analysis, and major agency guides.
    Moderate Overlap (2 to 4 Bots)13,164 URLs41.1%Secondary category archives, older blog posts, and tag directories.
    Single-Bot Exclusives (Exactly 1 Bot)13,480 URLs42.1%Long-tail URLs, paginated tag archives, and raw CSS/feed endpoints.

    The Exclusives Breakdown: Why Bots Specialize

    Why did 42.1% of all AI-requested URLs attract only a single crawler? Inspecting the exclusive URL sets highlights the divergent commercial objectives of each platform:

    1. ShapBot (7,844 Exclusive Paths): Requests non-trailing-slash 301 article variants (48.4% of its hits) alongside WordPress plugin CSS/JS assets (contact-form-7, fluentform, hooks.min.js).
    2. GPTBot (2,367 Exclusive Paths): OpenAI’s crawler follows external citations and user-submitted prompts into hyper-specific historical articles and niche technical documentation.
    3. ClaudeBot (1,827 Exclusive Paths): Anthropic’s engine systematically indexes hierarchical category paths and author archives that other bots ignore as low-priority metadata.
    4. Applebot (737 Exclusive Paths): Prioritizes factual entity pages and localized business profiles relevant to Siri knowledge graphs.

    7. The Probe Layer: Background Radiation of the Modern Web

    Every public web server is subject to constant, automated vulnerability scanning. These requests do not represent AI search or human browsing; they are automated vulnerability discovery tools probing for exposed credentials, unpatched CMS plugins, and misconfigured administrative interfaces.

    During the 55-day window, tygartmedia.com recorded tens of thousands of malicious probes:

    Probe TargetTotal HitsUnique IPsTop Status CodesRisk Profile & Security Outcome
    /wp-login.php48,164 (31,082 ext / 17,082 origin)10,747 (10,746 ext)44,617 (200), 2,636 (429)17,082 internal 8.229.250.67 HEAD checks + 31,082 external probes (16,805 POSTs). Zero breaches.
    /contact-us88793881 (404), 4 (499)Lead scraper probing common contact slugs.
    /about-us86960861 (404), 8 (301)Executive roster extraction attempt.
    /team86129859 (404), 1 (499)Corporate directory probe (Single IP 187.77.23.88 hit 771 times on /team; 7,710 across 10 slugs).
    /staff8409836 (404), 4 (301)Corporate directory probe.
    /people8295827 (404), 1 (499)Corporate directory probe.
    /.env798323648 (404), 76 (301), 74 (499)Environment secret harvesting. 100% blocked.
    /company78510784 (404), 1 (301)Corporate info probe.
    /wp-config.php1275598 (404), 22 (200*), 7 (499)Database credential harvesting. 0 bytes leaked.
    /wp-config.php.bak906970 (404), 17 (499), 3 (301)Backup file probe. 100% blocked.

    Forensic Deep Dive: The /wp-config.php Anomaly (HTTP 200 Audit)

    In early log rotations from late September, 22 requests to /wp-config.php returned HTTP status 200 OK. In web security audits, an HTTP 200 on wp-config.php is often flagged as a catastrophic vulnerability (credential leakage).

    We conducted a forensic line-by-line inspection of the Nginx server records:

    • Bytes Sent: In all 22 instances, the body_bytes_sent metric logged by Nginx was between 5 and 31 bytes (representing standard HTTP protocol headers, with 0 bytes of payload content).
    • PHP Interpreter Behavior: The server’s PHP-FPM fastcgi processor executed wp-config.php. Because wp-config.php defines PHP constants (DB_NAME, DB_PASSWORD) and contains conditional logic without any echo, print, or HTML markup, the PHP interpreter executed the script cleanly and emitted an empty string.
    • Outcome: Zero database credentials, secret keys, or code lines were ever transmitted over the wire. Nginx configuration rules were subsequently hardened with explicit deny all; directives to return clean HTTP 404/403 responses.

    Reconciling /wp-login.php Traffic: Origin IP 8.229.250.67 vs. External Probes

    Earlier draft analysis misclassified 8.229.250.67 as a foreign attacker IP in Singapore. In reality, 8.229.250.67 is the origin server itself (knowledge-cluster-vm), generating 17,082 internal loopback/healthcheck HEAD requests (35.5% of the 48,164 raw /wp-login.php count, matching 17,085 HEAD probes). Removing internal checks leaves 31,082 external requests across 10,746 external IPs (16,805 POST login attempts and 14,274 GET hits)—a flat, distributed tail of external scanners (top external IPs: 285, 183, 180, and 180 hits) rather than one brute-force host. While unauthorized probes failed or hit rate limits (2,636 HTTP 429s, 152 HTTP 444s), allowing external GET/POST probes to reach PHP-FPM places unnecessary CPU and database load on the origin.

    8. Bot Demand vs. Human Demand: The Pricing Page Anomaly

    When human readers visit tygartmedia.com, their engagement mirrors human editorial interests: regional business coverage, economic reporting in the Pacific Northwest, commercial real estate developments, and local news guides.

    In sharp contrast, automated bot traffic exhibits extreme, concentrated demand for a single editorial topic: Claude AI and Large Language Model pricing guides.

    Top Requested Content Paths by Automated Agents

    Requested URL PathSingle-Hit Scraper RequestsPrimary Bot Consumers
    /claude-ai-pricing/1,268GPTBot, PerplexityBot, ClaudeBot, Exa
    /claude-student-discount/1,228MetaBot, PerplexityBot, Applebot
    /claude-pricing-tiers-compared.../973GPTBot, ClaudeBot, ExaSearchBot
    /claude-rate-limits/950PerplexityBot, GPTBot, LinkupBot
    /grok-vs-claude-pricing/812GPTBot, PerplexityBot, ShapBot
    /current-claude-model-version/734ExaSearchBot, ClaudeBot, Perplexity
    /claude-code-billing-credit-pool.../599GPTBot, ClaudeBot, LinkupBot

    Why Are AI Bots Obsessed With LLM Pricing?

    This extreme concentration reflects the commercial dynamics of the AI industry in late 2026:

    1. Dynamic Token Economics: AI companies continuously adjust API token pricing, subscription tiers, and rate limits. Search engines like Perplexity and OpenAI must update their internal citation databases continuously to prevent hallucinating stale pricing.
    2. High-Frequency User Prompts: Consumer queries regarding “How much does Claude Pro cost?” or “Claude student discount 2026” represent millions of daily searches. AI answer engines maintain fresh local caches of authoritative comparison guides to serve instant responses.
    3. Competitive Benchmarking: Automated scrapers deployed by competing AI labs continuously monitor competitors’ pricing structures, seat minimums, and usage allowances.

    9. “Who We Think That Could Be”: Anonymous Traffic Attribution

    Over 42% of origin requests (1,203,889 hits) arrive under masked desktop browser User-Agents, alongside a 142,615 single-hit IP fleet. This audit classifies traffic strictly from origin-observable log signals (server loopback, declared User-Agent strings, /24 prefix density, and requested paths) without external rDNS or ASN lookups.

    To maintain strict epistemic boundaries, each attribution row is labeled honestly:

    • [Confirmed] Verified directly in local host telemetry (server loopback IPs 127.0.0.1/::1/8.229.250.67).
    • [Inferred] Attributed from self-declared User-Agent tokens (unvalidated externally in this log-only audit) or converging /24 and path signatures.
    • [Speculation] Hypothesis requiring external enrichment or PCAP capture to prove.
    Top Dense /24 IPv4 Prefixes in the 142,615 Single-Hit Pool 23.236.179.0/24 (Top /24 Cluster) 255 IPs (/24 saturated) 172.121.235.0/24 (Cluster #2) 253 IPs 46.202.78.0/24 (Cluster #3) 252 IPs 152.232.120.0/24 (Cluster #4) 252 IPs

    Attribution Forensic Matrix

    Actor / Traffic ClusterVolume & InfrastructureAttribution ConfidencePrimary Forensic Evidence & Behavioral Profile
    Microsoft Copilot / Bing98,890 hits · 2,130 IPs[Inferred]Declared bingbot/2.0 User-Agent (27.9% bounce). External rDNS/ASN validation was not executed in this log-only audit.
    Googlebot Search & Mobile18,564 hits · 4,884 IPs[Inferred]Declared Googlebot/2.1 User-Agent (70.4% bounce). External rDNS/ASN validation was not executed in this log-only audit.
    Google Cloud Stackdriver44,523 hits · 30 IPs[Inferred]Declared GoogleStackdriverMonitoring-UptimeChecks UA probing / at 1.3 active-min RPM (peak 3 RPM; 99.9% HTTP 200).
    Internal Server Automation32,306 hits · 13 Origin IPs[Confirmed]Origin loopback traffic including 15,880 /wp-login.php and 15,873 /about/ local checks plus maintenance/cron calls.
    B2B / Corporate Directory Probers8,202 hits (10 directory slugs; 7,315 excl. /contact-us) · Scanner IP + Tail[Inferred]Automated directory enumeration. Sequential probing across /team, /about-us, /people, /staff, /management, /our-team, /meet-the-team, /company, /who-we-are, and /contact-us. IP 187.77.23.88 hit 771 times on each of all 10 slugs (7,710 total).
    Careers & Job Indexing Bots82 hits · /careers (52), /jobs (30)[Inferred]Automated talent/job probes targeting /careers and /jobs (declared agents in dataset include JobsHTMLReplay/2.0 and HiWolfsBot/1.0).
    Dense-/24 Masked Desktop Scrapers1,203,889 hits · 747,173 pages[Inferred]Dense single-hit /24 IPv4 prefixes (23.236.179.0/24: 255 IPs; 172.121.235.0/24: 253 IPs; 46.202.78.0/24 & 152.232.120.0/24: 252 IPs each) forging desktop Chrome/Edge UAs (77.0% 1-page bounce) without firing JS telemetry beacons.

    10. What to Do Differently: Six Operational Recommendations

    Based on the empirical evidence gathered across 55 days of origin server logs, we recommend six concrete architectural and operational improvements:

    1. Harden Authentication and Extension Probes Directly in Nginx

    • The Problem: Of 48,164 raw /wp-login.php hits, 17,082 were internal origin HEAD checks (8.229.250.67), while 31,082 external requests (16,805 POSTs across 10,746 IPs) hit the origin directly. On origin-administered sites without an upstream proxy or WAF, unblocked authentication and legacy PHP extension probes reach nginx and can invoke PHP-FPM.
    • The Solution: Deploy origin-level nginx deny/drop rules, method-keyed rate limiting (limit_req_zone / limit_req), and pre-PHP 410 Gone extension rules (block-phtml.conf pattern):
      map $request_method $login_post_key { POST $binary_remote_addr; default ""; }
      limit_req_zone $login_post_key zone=login_post:10m rate=5r/m;
       
      location = /wp-login.php {
          if ($banned_ip) { return 444; }
          # deny <abusive_cidr>; / allow <trusted_cidr>;
          limit_req zone=login_post burst=5 nodelay;
          limit_req_status 429;
          include snippets/fastcgi-php.conf;
          fastcgi_pass unix:/run/php/php-fpm.sock;
      }
       
      # block-phtml.conf pattern: return HTTP 410 before PHP-FPM for dead extensions
      location ~* \.phtml$ { return 410; }
      Operational Gain: Throttles or drops external /wp-login.php bursts (31,082 external hits, 16,805 POSTs within the 48,164 raw total) and legacy extension scans at nginx before PHP-FPM or database threads engage.

    2. Explicitly Deny Sensitive Root PHP Files in Nginx

    • The Problem: 127 requests hit /wp-config.php (returning 22 empty 200s in early logs), and scanner tools issued 798 probes for /.env alongside /wp-config.php.bak (90 hits) and /xmlrpc.php (1 HTTP 404 probe).
    • The Solution: Add explicit drop rules in /etc/nginx/sites-available/tygartmedia.com:
      # Deny all direct access to sensitive configuration files
      location ~* (wp-config\.php|xmlrpc\.php|\.env|\.git) {
          deny all;
          return 404;
      }
      Operational Gain: Completely prevents PHP-FPM from booting on sensitive filenames, guaranteeing zero information leakage.

    3. Treat Top AI Crawlers as an Active Origin Caching Tier

    • The Problem: Top 10 autonomous AI crawlers (including PerplexityBot, GPTBot, and ClaudeBot, which arrive within 4.4 to 5.5 hours median of post publication) generate 531,349 total requests (502,176 HTML page hits). Serving these from dynamic PHP slows down origin performance.
    • The Solution: Configure origin-side nginx FastCGI micro-caching (fastcgi_cache with fastcgi_cache_lock on and fastcgi_cache_use_stale updating) with up to a 2-hour TTL for anonymous HTML GET/HEAD documents. Purge or warm cache keys automatically upon post publication via webhook.
      Operational Gain: Serves anonymous HTML cache hits directly from origin nginx memory during breaking news traffic spikes without invoking PHP-FPM.

    4. Deploy Adaptive, Bot-Aware Rate Limiting

    • The Problem: Uncontrolled scrapers like ShapBot hit peak rates of 1,044 requests/minute, causing temporary I/O latency spikes.
    • The Solution: Configure Nginx connection and request zones:
      limit_req_zone $binary_remote_addr zone=crawler_limit:10m rate=15r/s;
      location / {
          limit_req zone=crawler_limit burst=30 nodelay;
      }
      Operational Gain: Allows legitimate cooperative crawlers (1 to 5 req/sec) while instantly throttling aggressive scraper bursts.

    5. Return Lightweight 444 or Static 404s for Known Scanner Slugs

    • The Problem: Scrapers hit /team, /about-us, /people, and /staff thousands of times, forcing WordPress to run full 404 database queries.
    • The Solution: Intercept known phantom corporate slugs at the Nginx level:
      location ~* ^/(team|people|staff|management|our-team)/?$ {
          return 404 "Not Found";
      }
      Operational Gain: Drops database query overhead to zero for repetitive CRM lead-scraper probes.

    6. Combine Log Analytics with Tag Telemetry for True Operational Visibility

    • The Problem: Executive leadership makes capacity and marketing decisions based solely on Google Analytics or Henkan Vision, missing >99.7% of origin automated demand.
    • The Solution: Establish an automated daily log-parsing pipeline that exports machine vs. human request ratios into executive BI dashboards.
      Operational Gain: Bridges the visibility gap between marketing attribution and infrastructure engineering.

    11. Methodology Appendix

    Log Format & Ingestion Infrastructure

    The audit was conducted on production access logs generated by Nginx directly inside the knowledge-cluster-vm origin infrastructure. The ingested access-log records follow the standard 8-column Nginx combined format (tygartmedia_today_page_requests.tsv schema; upstream timing variables were not logged):

    log_format combined '$remote_addr - $remote_user [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent"';

    Audit Window & Data Integrity

    • Start Timestamp: 2026-09-18 00:00:09 UTC
    • End Timestamp: 2026-11-12 02:26:37 UTC
    • Total Duration: 55 days, 2 hours, 26 minutes, 28 seconds
    • Log Files Analyzed: 56 distinct log files (tygartmedia.com_access.log through .1 and .2.gz through .55.gz).
    • Data Integrity Check: Log continuity was verified with zero timestamp gaps across file rotations.

    Bot Classification Heuristics

    1. Declared AI Crawlers: Identified via standardized token matching against verified vendor user-agent documentation (GPTBot, ClaudeBot, PerplexityBot, ExaSearchBot, LinkupBot, MetaBot, Amazonbot, Applebot, Bytespider, ShapBot).
    2. Masked Browser Scrapers: Identified as requests presenting desktop browser User-Agent headers on Windows/macOS/Linux while exhibiting dense /24 prefix rotation, high class single-page bounce rates (69.2%–80.0%; 77.0% combined), HTML-skewed paths (62.1% pages), and zero JS telemetry execution.
    3. Single-Hit Fleet: Defined as unique IP addresses executing exactly one page request across the entire retention window.

    Reference Datasets Available in Project Repository

    • per_bot_profiles.csv / per_bot_profiles.tsv: Complete 32-bot tabular metrics dataset including peak RPM, bounce rates, and unique path footprints.
    • tygartmedia_today_page_requests.tsv: High-resolution raw request stream capturing 58,071 consecutive page hits for behavioral validation.
    • scratch/deep_bot_compact.json: Complete serialized mathematical summary of all 55-day metrics.

    Report compiled and certified by Antigravity Operations Intelligence · Tygart Media


    We do AI citation restoration. When the AI can’t find you, can’t read you, or keeps citing somebody else, send us the worst one you have — your RFPs, your disasters, the problem everyone else already failed to fix. We come from the restoration industry. We know what’s on the floor, and cleaning it up is the job. And if you’re a developer who just needs a hand on the hard part, get a hold of us. That’s what we’re here for.

    — William Tygart, Tygart Media. Data: origin server logs, Sep 18–Nov 12, 2026.

    Complete Exp. 3 Package · Free PDFs

    Download the White Paper & Companion Package

    Get the full 19-page technical white paper alongside the 15-slide executive presentation deck and 1-page summary brief.
  • 1.28 Million Requests for Pages That Don’t Exist

    Over 55 days, the origin server behind BC ESG — my ESG and sustainability site — logged 2,008,202 requests. Of those, 1,280,233 came from a single crawler.

    That’s 63.75% of everything. One bot, outweighing all other traffic to the site combined — every human, every search engine, every other crawler, all of it.

    And it was asking for pages that don’t exist.

    What it was asking for

    The requests were for legacy dictionary.phtml paths — URLs from an old dictionary-style URL scheme, the kind of pattern an earlier generation of web publishing used to serve definition and reference entries. The site doesn’t serve that scheme. It hasn’t for a long time. There is no version of those pages to fetch, no content behind those URLs, nothing.

    ExaSearchBot requested them 1,280,233 times anyway.

    I want to be precise about what kind of traffic this is, because it doesn’t fit the categories most people argue about. This wasn’t an indexation crawl of live content. It wasn’t a live agent fetching a page to answer somebody’s question. It was a crawl queue working through a memory — a corpus of URLs assembled at some point in the past, replayed against a server that no longer speaks that language.

    Full Report · Free PDF

    Download this article as a PDF

    Keep the 55-day BC ESG origin log counts, HTTP 410 findings, and operator playbook as a clean PDF.

    The 410, and why the storm continues anyway

    My nginx configuration already answers those paths with HTTP 410 Gone, via a rule (block-phtml.conf) that was in place before I measured any of this. That rule stays. Permanently.

    410 is the correct answer, and it’s worth understanding why it’s different from the default. A 404 says “not found” — which every crawler correctly interprets as “not found right now,” possibly temporary, worth another try later. A 410 says “gone.” Deliberately, permanently removed. It’s the most definitive statement HTTP gives you: stop asking.

    The storm continues anyway.

    That’s not a failure of the 410. That’s a lesson about timescales. Crawl corpora and queues are built to be exhaustive and conservative — dropping a URL pattern on the say-so of a status code is exactly the kind of thing large-scale crawlers are slow to do, because a wrong drop loses coverage forever. So the requests keep coming, get their 410 in a few bytes, and cost the server almost nothing per hit. Almost nothing, times 1.28 million, across 55 days, is still the majority of your traffic. The queue will forget eventually. “Eventually” is measured in their time, not yours.

    Anyone who tells you a status code fixes bot behavior on a human timescale hasn’t watched their own origin logs long enough.

    This is the background radiation of a public website

    The .phtml storm is the headline because of its size, but it isn’t an anomaly. It’s the largest example of a category that fills every origin log I’ve ever read: requests for things that aren’t there, from machines that will never load a page.

    The same 55 days of BC ESG logs contain 20,156 probes for /.env files — the configuration files careless deployments leave exposed, full of credentials. All of them were blocked. Zero breaches. But the probes don’t stop because any individual one failed; they’re automated sweeps across the whole addressable web, and my server is one stop on the route.

    Same logs: more than 3,020 requests for favicons that returned 404. Harmless, trivial — and completely invisible anywhere except here.

    That’s the mechanical point I keep coming back to. None of this traffic loads a page. None of it executes JavaScript, accepts a consent banner, starts a session, or becomes a “visit” in any ordinary sense of the word. It exists only in the origin server logs, because the origin server is the only thing that ever saw it. When people argue about what their traffic “is,” they’re usually arguing about the minority of requests that behaved politely enough to be counted by a script on a page. The majority can be something else entirely — in this case, it was one crawler’s memory of a dead URL scheme.

    The operator’s job

    So what do you do with a log that says most of your traffic is ghosts? You make decisions — deliberately, per pattern, from the evidence:

    Wall off what’s dead and dangerous to keep answering. The .phtml scheme is gone and stays gone. The 410 is cheap, definitive, and correct. Probes for /.env and its cousins get blocked, not answered. This category is most of the volume and none of the value, and the correct response is a rule, not a worry.

    Redirect what carries real demand. Not every ghost is junk. Old paths sometimes hold genuine intent — people and agents following dated links to content that still exists somewhere else on the site. The test is in the logs: what’s being asked for, how often, and whether what it maps to still deserves to exist. A redirect is a promise that the destination answers the question the old URL asked. Make that promise only where it’s true.

    Rebuild what the demand justifies. Occasionally the ghosts are telling you something a keyword tool won’t: a topic your site used to cover, or never covered, that machines keep expecting to find. Persistent requests for absent pages are a demand signal with the politeness stripped off. Most of it is noise. Some of it is a content brief.

    Count honestly. If you report your traffic without the log-level picture, you are reporting the residue — the requests that survived being real pages loaded by real browsers. Build your expectations, your capacity planning, and your sense of what your site is for on the whole record.

    Legacy URL schemes never die. Anything your domain ever exposed — under you, or under whoever ran it before you — will be requested for years, by machines working from old maps. The only question is whether you’ve looked at the map they’re using.

    For the full dataset, methodology, and six origin hardening rules behind this series, read the white paper: Not All Bots Are the Same: 55 Days of Server Logs from One Website.


    We do AI citation restoration. When the AI can’t find you, can’t read you, or keeps citing somebody else, send us the worst one you have. Your RFPs, your disasters, the problem everyone else already failed to fix — that’s the work we want. We come from the restoration industry. We know what’s on the floor, and cleaning it up is the job.

    And if you’re a developer who just needs help on the hard part — the log file nobody can read, the crawl behavior nobody can explain — get a hold of us. That’s the part we’re best at.

    Printable Desk Reference

    Download this article as a PDF

    The complete server-log breakdown and four-part operator response formatted for print and desk reference.
  • The Buyer Is a Bot Now

    In 55 days of server logs from one of my own sites — Risk Coverage Hub, an insurance and risk publication — a user agent called ChatGPT-User made 2,065 requests from 1,063 distinct IP addresses.

    That is not a crawler. That is not indexing. Every one of those requests happened because a person, somewhere, asked ChatGPT to go read a page right now, in the middle of a live conversation, to help them decide something. ChatGPT sent a fetch. The fetch hit my server, identified itself with the -User suffix, and pulled the page.

    Its single most-requested target: an article on climate-risk pricing and catastrophe-model updates. 179 fetches. One page, pulled 179 times in 55 days by agents working for people who were evaluating risk — the kind of evaluation that happens shortly before money moves.

    Nobody “visited” that page 179 times. No human session started. No analytics tool built on JavaScript will show you a visit that never rendered in a browser. The buyer showed up. They just sent a machine to do the looking.

    Three jobs, one log file

    Most bot reporting collapses everything into a single bar chart labeled “AI traffic,” as if a visit is a visit. It isn’t. In the logs, the AI presence on a site splits into at least three distinct jobs, and only origin server logs separate them:

    1. The indexer. ClaudeBot, GPTBot, and their peers, walking the site systematically to build an index. Scheduled, comprehensive, indifferent to any individual human.
    2. Search retrieval. A bot fetching pages because a search product needs material to answer from — prompt-driven, but still one step removed from a person.
    3. The live agent fetch. The -User agents: ChatGPT-User, Claude-User, Perplexity-User. These fire in real time, from a specific person’s session, carrying a specific question. When one of these hits your page, a human being effectively walked into your store and sent their assistant to the shelf.

    Job three is the buying-stage visit. It deserves to be counted, read, and designed for separately — because it behaves nothing like the other two, and because a blended “bot traffic” number hides the only signal in the pile that represents a person mid-decision.

    Full Report · Free PDF

    Download this article as a PDF

    Keep the full server-log figures, three-job breakdown, and cluster findings formatted cleanly as a PDF.

    The pattern repeats across the cluster

    Risk Coverage Hub is not an outlier. Same 55-day window, Sep 18–Nov 12, 2026, across four of my sites:

    • Continuity Hub (business continuity): ChatGPT-User made 780 fetches from 535 distinct IPs. Claude-User added 51, Perplexity-User 17.
    • Healthcare Facilities Hub: ChatGPT-User 1,529 fetches, Claude-User 722 — and Meta-ExternalAgent, Meta’s live-fetch agent, pulled pages 2,535 times.
    • BC ESG: ChatGPT-User’s live fetches returned successfully 87.2% of the time — meaning when an agent came looking for an answer there, it usually got one. That number is a property of the site, not the agent. It can be made better or worse.
    • Risk Coverage Hub: the 2,065 fetches and 1,063 IPs above.

    Add the four sites up and it’s more than 7,600 live-agent fetches in 55 days. Seven thousand six hundred occasions where a person delegated their research to an AI, the AI chose these pages, and the person read whatever the agent brought back — or read the competitor’s page instead, because ours didn’t parse, didn’t load, or didn’t contain an extractable answer.

    My own site shows the same behavior from the other side of the counter. On tygartmedia.com, demand — human and bot alike — concentrates on the Claude plan and pricing comparison pages. People are comparison-shopping AI subscriptions, and their agents are doing the comparison fetching. Buying-stage behavior, on a site about buying-stage behavior. The logs don’t care about the irony; they just record it.

    The fetch is the visit

    Here is the reframe I can’t get away from, and I don’t think the industry has priced it in yet:

    The fetch is the visit. Not a precursor to the visit, not a bot to be filtered out of the “real” numbers. When ChatGPT-User requests your pricing page, a sales conversation is happening. Your page is being read by the most literal-minded researcher your prospect has ever employed — one that will quote you exactly, summarize you faithfully or not at all, and never, ever click your CTA, fill out your form, or call the number.

    That changes what “good” means for a page. Your layout, your brand film, your carefully art-directed hero — the agent receives none of it. The agent receives structure and text. What it can extract is what your prospect hears. Answer-extractability is the new shelf placement: if the answer to the buyer’s question isn’t sitting in the page in a form a fetch can lift cleanly, you didn’t lose the click. You lost the deal before a human ever knew you were in it.

    Three implications I keep coming back to:

    • Count the -User agents separately, or you’re flying blind on your highest-intent traffic. They are small in volume next to the indexer fleets and enormous in meaning per hit.
    • Your success rate is a design variable. An 87.2% fetch success rate means 12.8% of delegated buyers got nothing. Every failed, blocked, or unparsable fetch is a prospect handed to whoever the agent could read.
    • The comparison fetch is the new comparison page. On Risk Coverage, agents didn’t pull the homepage 179 times. They pulled the pricing-relevant analysis. Whatever page in your vertical answers “what does this cost and how is it priced” is the page your buyer’s agent will live on. It should be built like it.

    I’m not arguing the human visit is dead. I’m arguing it is no longer the only visit that matters, and that the other kind is invisible to every tool that waits for a browser to show up. The logs saw it the whole time. We just had to read them as what they are: a record of people, arriving by proxy, ready to buy.

    For the full dataset, methodology, and six origin hardening rules behind this series, read the white paper: Not All Bots Are the Same: 55 Days of Server Logs from One Website.


    We do AI citation restoration. When the AI can’t find you, can’t read you, or keeps citing somebody else, send us the worst one you have — your RFPs, your disasters, the problem everyone else already failed to fix. We come from the restoration industry. We know what’s on the floor, and cleaning it up is the job. And if you’re a developer who just needs a hand on the hard part, get a hold of us. That’s what we’re here for.

    — William Tygart, Tygart Media. Data: origin server logs, six-site lab, Sep 18–Nov 12, 2026.

    Printable Desk Reference

    Download this article as a PDF

    The complete article and cross-site live-agent fetch data formatted for print and offline reference.
  • Just a Guy: What Is One Person Worth Now?

    Tygart Media · Notes from the Bot Lab

    One director, many executors. The question is no longer how many hours I worked. It is what the finished work is worth.

    I am one person. No team, no agency floor, no payroll of specialists behind me. And yet, when I sat down this week and honestly priced the work documented in my own work orders, the completed total came to somewhere between $207,350 and $365,235 — as of November 15, 2026.

    That number stopped me cold, because it does not fit any way I was ever taught to value work. It cannot be hourly — nobody works enough hours in two days to explain it, and I did not. It is not really “project based” in the old sense either, because there was no single project. What there was, underneath everything, was a stack of work orders: scoped, written instructions, handed to AI executors, that came back as finished, checkable deliverables.

    So I want to break this down carefully — what was actually produced, how I priced it, what I deliberately refused to count, and what I think it means. Because if one person plus AI workers can produce at this rate, then the honest question is not how fast is the machine? It is: how do you value any of this now?

    One scope note, because it matters: this is not everything I did that week — only what the collected work orders document. If it was not written down as a work order, with a deliverable I could point to, it is not in here.

    The Count — Start with what the work orders actually show

    I priced everything the same way a client would buy it standing alone: at ordinary US agency and consultant retail rates — a senior web analyst or CRO consultant at $150–$250 an hour, implementation work blended at $125–$200 an hour. Freelance-direct would run roughly 30–40% lower. These are ranges, not quotes, and they are deliberately the buyer’s prices, not my costs.

    The Bot Lab work. Fifteen work orders sit in the Bot Lab folder, plus one held vault-audit order. Six of them are complete — a report received, a package delivered, an execution finished and recorded:

    Completed Bot Lab work order Retail value
    Bot log pull — 8 days, 468K requests, reconciled against Henkan Vision $750–$1,250
    Deep bot analysis — 55 days, 2.84M requests, fleet behaviour signatures, security verification $3,500–$6,000
    White paper package — 19-page paper, 15-slide deck, 1-page executive brief $4,400–$7,100
    Taxonomy audit — 2,063 tags, 50 slug collisions, costed action plan $1,250–$2,000
    Tag strategy audit — re-crawl analysis, top-100 under-tagged list, 5 new tags approved $1,250–$2,000
    Tag execution — database backup first, 472 posts touched, 473 tags added, tagging standard published $1,000–$1,750
    Bot Lab, completed $12,150–$20,100

    That work is forensic, unglamorous, and real: reading origin server logs closely enough to separate an AI indexer from a search retrieval from a live agent fetch, because at the brand level you cannot tell what a bot was actually doing on your site. Only the logs know.

    Full Report · Free PDF

    Download this article as a PDF

    Keep the complete count, valuation rules, and work-order leverage framework on your desk.

    The Count, Continued — The big one: 700+ pages — and the recount proved the bigger number

    The largest single body of work was the Universal Commerce Protocol site modernization, executed by Antigravity through a master work order and a long batch trail in Google Drive. I had been saying “700+ pages.” My first count came back at 639, and I said that count was wrong — that every article had been scanned. So we dug deeper, and the recount is in this article because the correction is the method. This time the correction ran the other way: the deep dig proved the bigger number right, and showed exactly where the first count had gone wrong.

    UCP modernization — evidence Pages
    Result records marked PASS (559 batch entries, deduplicated by post ID) 541
    Evidenced only by execution scripts, batch definitions and link maps, no result file 167
    Distinct posts documented in Drive evidence 708
    Live-site verification, page by page: posts carrying the modernization signature 730 of 730
    Scanned / audited: the entire published corpus (738 items incl. pages) 730 posts

    Where did 639 come from? The first pass credited only 82 pages to the execution scripts. A full extraction across some 155 script artifacts found 665 distinct post IDs in the scripts alone — 167 in no result file at all — plus sources the first count had missed. And “about 698 articles” was a mid-program snapshot, not a ceiling: the site now holds 730 published posts, every one carrying the modernization signature — 726 with the full dual schema, 660 with the complete treatment of banner, unique matrix, FAQ and schema. Every one of them was scanned. That sentence, it turns out, was simply true.

    Pricing taught me something, too. At a hand-built rate of $400–$800 a page, the program would “value” at a number nobody would ever actually pay. Batch work at scale is priced as a program, not as 730 artisanal pages — so I used volume rates of $100–$175 for each of the 541 result-verified pages and $75–$125 for each of the 167 script-evidenced pages, plus the strategy work and a homepage repair package. That puts the program at $68,625–$118,950. The per-unit price fell; the total value did not. That trade is the whole story of AI-era production in miniature.

    Completed, client-type work — as of Nov 15, 2026 Retail value
    Bot Lab (6 completed orders) $12,150–$20,100
    UCP modernization program (708 posts in Drive evidence; 730 live-verified) $68,625–$118,950
    Setup and systems: Henkan Vision rollout on 5 sites, Henkan API connector, InventoryQuant one-pager, Advancing Prefabrication 2027 analysis $2,750–$5,500
    Five forensic server-log reports, one per hub site — all five delivered $8,000–$13,000
    tygartmedia.com Henkan fix sprint — executed and live $1,200–$2,000
    healthcarefacilityhub.org Henkan fix sprint — executed and live $1,450–$2,600
    Post-baseline receipt-verified work orders (Nov 12–14, 33 orders) $113,175–$203,085
    Total completed $207,350–$365,235

    The phone operating layer I built the same week — getting AI control of my own phone working and self-healing — is real work, worth roughly $800–$1,500 on the same basis, but I classify it as internal infrastructure and R&D, not client-type work, so it stays out of the headline total.

    The Rule — Only completed work counts. Everything else is zero.

    A number like $207,350–$365,235 invites scepticism, and it should. The only reason I am willing to publish it is the rule I priced it under — my rule, set before the counting started:

    “Commissioned is not produced. In-progress is not produced. Only a delivered, verified job counts — everything else is booked at $0.”

    Applied honestly, that rule hurt — and then it paid. On November 12 I commissioned a substantial pipeline: Henkan fix sprints for two sites and five forensic server-log reports, one per remaining site. Under the rule, all of it counted as exactly zero while it was in motion. Then it landed: all five reports delivered, both fix sprints executed and live — and only at that point did $10,650–$17,600 of it move into the completed total above. What is still in motion — a cluster-wide fix round across the five hub sites and the per-company platform fleet dossiers, $6,000–$9,900 of scoped work — still counts as exactly zero, as do two other orders for the opposite reason: one I closed as unnecessary, and one held pending verification of its lead. Zero is zero.

    Even inside the completed pile, the caveats stay attached. The white paper was counted at delivery value, but it carries two known flags — an origin IP mislabelled as an attacker, and a Cloudflare recommendation that does not apply to a site that is not on Cloudflare — and the correction is still owed before it is publishable. The UCP completion now rests on three legs: the executor’s result records, the full Drive script trail covering 708 distinct posts, and an independent, page-by-page fingerprint check of the live site that found the modernization signature on all 730 published posts. I am saying that here, in the article, because a valuation that hides its soft spots is marketing. One that shows them is accounting.

    Why the hour broke — Hourly billing measures my presence, not the production

    Hourly billing rests on a quiet assumption: output scales with one person’s time. When my executors run in parallel — one modernizing pages in batches, another working through 2.84 million log lines — that assumption collapses. Divide the completed value by my hours and you get an “hourly rate” so absurd it describes nothing. Divide it by the machines’ hours and you get the opposite lie, as if the judgement that scoped the work were free.

    “The hour did not get faster. It stopped being the unit.”

    The Mechanism — The work order is the unit now

    If the hour is not the unit, what is? In my week, it was the work order: a written package that carries four things a raw prompt does not.

    • The evidence travels with the task. Each fix order named the exact pages, session counts, and click targets from the Henkan data. A vague prompt forces the executor to guess, or to redo the analysis worse.
    • Guardrails prevent paid wrong turns. They say what not to touch, and that turns out to be worth money on its own — see the box below.
    • Shared definitions make results comparable. Five log reports written to the same lab definitions line up against the same baselines; five cold prompts would produce five different shapes.
    • A verification loop exists. Baseline numbers and a check-back date are written into the order, so “done” can actually be tested.

    One paragraph that saved real money. Item 5 of the tygartmedia fix order read, in effect: do not chase the js_error flags — verified Safari AbortError noise. Every flagged session was Safari, each with exactly two AbortError entries, reading on normally afterwards. Without that paragraph, an executor could plausibly have burned one to three hours debugging a non-bug: an estimated $200–$600 of wasted work, avoided by a sentence of context.

    Now the leverage, stated carefully. On November 12, about ten minutes of voice conversation produced the two fix orders: $2,650–$4,600 of scoped implementation. One further sentence — “one work order for each of the sites” — produced five log-report orders: $8,000–$13,000 of scoped analysis.

    My input Scoped work set in motion
    ~10-minute voice call $2,650–$4,600
    One sentence $8,000–$13,000
    ~15 minutes total $10,650–$17,600

    That is roughly $700–$1,200 of commissioned work per minute of my time — and note the word commissioned: under my own rule it is worth $0 until it lands and verifies. Writing those seven orders myself, to the same standard, would have taken an estimated four to seven hours. The real delta was never typing speed. It was context compression: months of lab baselines, standards, and “do not touch this” knowledge, folded into a three-minute conversation. A cold prompt starts from zero, every time.

    The Cost Side — What does an AI worker cost — and what is it worth?

    People ask about cost as if it were mainly tokens and subscriptions. Those are line items, and compared with the value above they are small. The real cost ledger of an AI worker, as this week actually felt, has four entries:

    • Direction. My time scoping the order well enough that the evidence, guardrails, and definitions travel with it. Cheap per order, expensive if skipped — skipped direction is repaid as rework.
    • Verification. Reading 559 PASS records back against the full Drive script trail, then fingerprinting all 738 live URLs — 730 of 730 posts — instead of repeating “700+” on faith, and checking that a test visit really appeared as a live session. Verification is the cost that makes the value claim defensible, so it is the last cost I would cut.
    • Rework and wrong turns. The risk premium on every job. Guardrails exist precisely to buy this down — one avoided wild goose chase was worth $200–$600 on a single order.
    • Infrastructure and tools. The subscriptions, the API connector that turned every later Henkan pull into a one-minute job, the systems work that makes the next order cheaper than the last.

    And the worth side has a simple answer, the one this whole count is built on:

    “Value an AI worker by the verified deliverable, priced as a client would buy that deliverable on its own — never by the minutes the machine spent producing it.”

    That also settles the pricing question. Hourly fails, for the reasons above. A flat per-unit price fails at scale, as 730 pages showed. What survives is pricing the job: a defined deliverable, a defined standard, a flat or program price. It is how I would now sell the work this lab produces — a forensic server-log report at $1,950 a site, a friction audit at $950, a fix sprint at $1,500, or all three bundled at about $3,950, against a market where standalone log audits run $2,000–$8,000. The buyer is not purchasing my afternoon. They are purchasing a finished, verified answer.

    And for genuinely small one-shot tasks — fix a heading, swap an image — the honest advice runs the other way: skip the work order and prompt the executor directly. The order’s overhead only pays when the task is evidence-heavy, multi-page, or easy to do wrong.

    The Trap Underneath All of It — Everything can be automated. That is exactly the problem.

    Here is the part nobody warns you about. With effectively unlimited compute sitting next to me, everything can be automated. I can automate pressing a button. I could spend half an hour automating how I get coffee — or I can just get up and go get the coffee. Both are now genuinely available options, and that is the trap: automatable is not the same as worth automating. When capacity was scarce, scarcity did the filtering for you. When capacity is unlimited, the filtering becomes the job — and if you do not do it deliberately, you will spend your infinite leverage automating things that should have taken ninety seconds by hand.

    I know this from the inside, because I spend twelve to sixteen hours a day in this work, and until now I have had no real way to label any of it. Is this hour billable or non-billable? Is it direct — straight toward the business — or indirect? Is it personal, or is it for the lab? I honestly could not have told you, day by day, and a man who cannot see his own time cannot steer it. That, more than the pricing puzzle, is why the work-order valuation and tagging practice in the next section exists. It is not an accounting flourish. It is a self-organization tool — a way of looking at myself and asking: what do you actually need to do, how do you organize this, and how do you make sure you are not just spinning your wheels very, very fast?

    Why the “Unbillable” Hours Are Not Waste — Yesterday’s learning is the capital today’s leverage is spent from

    But the tags have to be read honestly, and honesty cuts in a direction that surprised me. All that prior work that looked unbillable — the lab experiments, the log-diving, the systems tinkering — may be exactly what gave me the knowledge to spend fifteen minutes of direction and put out $10,000 worth of scoped work. Nobody conjures that fifteen minutes from nothing. It is drawn against months of accumulated context, and the context was built in hours nobody would have paid for at the time.

    So the labels are not there to shame non-billable time. They are there to see it accurately — so the compounding becomes visible. Some indirect work is wheel-spinning, and the tags will expose it. Some of it is capital formation wearing a disguise, and the tags are the only way I will ever know the difference. Either way, I would rather know.

    From Here — Every work order gets priced and tagged — before it runs

    The lesson I am keeping is not the headline number. It is the discipline that made the number trustworthy. Every internal work order I issue will now carry two things up front, before any work starts:

    • A retail value estimate — what this deliverable would cost a client to buy, stated as a range, on a consistent rate basis.
    • A category tag — client work, lab, R&D, or internal infrastructure — so nothing quietly inflates the wrong column the way my phone work could have.

    Then each job gets judged twice — before and after — on four plain questions: Is this worth our time? How much time and resources will it actually take? What is the net result? What is the value of the work? Value gets booked only when the job is delivered and verified, exactly as this week was counted. Over time that gives me something no timesheet ever did: a record of which kinds of orders actually pay, and which kinds only feel productive.

    What It Means — One person is not one person anymore

    I keep coming back to the strangeness of the plain fact. I am just a guy. I did not hire anyone, and I did not work some heroic schedule — the famous “week” in my own lab records is really two days, and I am not going to dilute that by calling it anything longer. What changed is not my capacity. It is that my judgement — what to do, what not to touch, what counts as done — can now be written down once, as a work order, and executed many times over, in parallel, without me standing over it.

    That means the scarce asset in a one-person business is no longer labour. It is context: the baselines you have built, the standards you can state, the mistakes you know to fence off, and the willingness to verify instead of believe. Whoever holds the best context writes the best work orders, and the best work orders are what the machines are worth.

    So, how do you value an AI worker? The same way you now have to value the person directing it: not by the hour, and not by the prompt — by the completed, verified job, and by nothing else.

    And none of this is really about one guy. I am doing it alone, but the same discipline — price the order up front, tag what kind of work it is, book value only on verified delivery — conveys to anyone: another solo operator, a small firm, or a department inside a large company where AI agents are already doing work that hourly accounting simply cannot see. The scale changes. The labeling problem does not.

    Printable Desk Reference

    Download this article as a PDF

    The full text, data tables, and four-point valuation rule formatted for print and offline reading.


    Method note: all figures are retail-equivalent values at US agency/consultant rates (senior analyst $150–$250/hr; implementation $125–$200/hr blended), drawn from work-order records, execution result files, and Drive evidence as of November 15, 2026. The UCP figures are from the November 12 deep-dig recount: Drive execution evidence for 708 distinct posts, with all 730 published posts verified live, page by page. Retail value is not cash received. Pipeline work is carried at $0 until delivered and verified.

  • You Don’t Have a Token Problem. You Have a Habits Problem.

    You Don’t Have a Token Problem. You Have a Habits Problem.

    I held five hundred dollars of AI stack for ninety-nine bucks a month, and I used every drop of it. When the deal changed, I didn’t cry about the price. I changed where I bought my gas.

    Five hundred dollars of stack for ninety-nine bucks

    Here’s the deal I had, because it matters to everything after it. SuperGrok Heavy — the top Grok tier — listed at three hundred dollars a month. I got in early, in the first days of the offer, on an early-adopter intro price: ninety-nine bucks a month, a hundred and nine and change once the tax man took his cut. And riding inside that bundle, at zero extra dollars, was Cursor Ultra — the two-hundred-dollar Cursor plan. Do the math the way I did it at the time: I was holding roughly five hundred dollars of list-price stack for ninety-nine.

    And I used it. All of it. That’s the honest part, and I want it on the table before I say one word about anybody else’s habits. Grok seats, Cursor seats, the biggest model on whatever job was in front of me, because why wouldn’t I? The stack was so cheap that counting would have felt like a hobby, not a necessity. Nobody audits a deal that good. You just drive.

    I was not disciplined. I was subsidized. There’s a difference, and the difference only shows up later.

    Because the deal changed — deals do. The intro price was always going to step up; the paperwork said it would land at $299.99 a month. After that, the whole Grok plan structure got replaced by a new X subscription, and the bundle came apart. The free-stack era ended. Not with a betrayal — with a terms change, which is how these things always end.

    The gas was free, so nobody learned to drive

    Here’s my read on how we got here, and I’ll own it as a read: for a couple of years, compute was handed out like it was nothing. Generous free tiers. Big included allowances. Tokens falling out of every new signup. And I don’t think the platforms were being careless with that generosity. They knew exactly what happens when the meter isn’t running — people get unorganized. They experiment, they sprawl, they build their whole working life on the free stuff, and they form habits while nobody’s counting.

    That’s not a conspiracy. That’s how free samples work, in every business there ever was.

    Then things tighten — a limit appears, an allowance shrinks, the overage starts billing — and people feel robbed. People hit Claude’s usage limits and get furious. They watch a meter run in some agent tool and feel cheated. I understand the feeling, because I had it. But watch where the anger lands. It lands on the limit, every time. Almost never on the habits underneath it. The free era was never the normal state of the world. It was the sample. The normal state of the world is the one every household already knows: a thing costs what it costs, and the habits you built while it was free become your problem the day it isn’t.

    A kitchen table with paper bills, a calculator, a pencil and a coffee mug in morning light

    Premium in a car that runs fine on regular

    Compute is gas. That’s not a metaphor I reach for because it’s cute — it’s the same shape in every particular that matters. There are grades: regular, premium, diesel. Different fuel for different jobs. And most of us, most days, are putting premium in a car that runs fine on regular, because premium is what the pump defaulted to when somebody else was paying.

    I can put a number on that, because we ran the test. In October, my shop sent the same plain job to 44 AI models — visit five web pages, report the title, the heading, whether a video was embedded, and a one-line summary. A utility errand, not a puzzle. Thirty-six of the 44 models came back with a perfect score, 20 out of 20.

    The cheapest perfect run — a model called GLM 5.3 Flash — cost eight-tenths of a cent. The most expensive perfect run — Claude Fable 5, a flagship — cost 86.3 cents. Same job. Same perfect score. More than a hundred times the price.

    And premium didn’t even guarantee care. One run that cost over seventy cents reported a video missing from a page where the video was sitting in plain sight — it had read the page through a tool that strips embeds out, and never noticed. Money wasn’t the difference between those runs. Method was. The grade of fuel was never the thing that made a run good. How the job was set up was.

    So the question was never “can I afford the good model?” It’s “does this errand need diesel?” A small, fast model can run a census, sort a pile, draft the grunt work all day long on regular. The heavy reasoning model is diesel — you save it for the truck, for the leg of the job that actually pulls weight. Burning the flagship on everything isn’t quality. It’s a habit with a receipt.

    And then there’s the gas station

    The fuel is only half of it. There’s also where you buy it.

    Your harness — the app, the agent tool, the environment the model runs inside — is the gas station. Two stations can sell you the same grade at different prices, and the station still matters. Does it take your card, or do you have to go inside and prepay? Is it on your way to work, or twenty minutes out of the way? What else does it sell — the snacks, the tools, the integrations you use? A cheap station with bad pumps costs you more than the price on the sign.

    And then there’s the Costco question, and it’s a real question, not a punchline: the gas there is thirty cents a gallon cheaper, and the line is twenty cars deep. Does it make sense? It depends on the trip. A big batch job that can sit in a queue overnight is a Costco run — take the cheap fuel, wait in the line, who cares. A job that’s on fire at 2 p.m. is not. You pay the corner station, you pay it gladly, and you get back to work.

    Cheap compute almost always has a line attached to it — a queue, a rate limit, a slower pump. Pricey compute is the corner station: more per gallon, no waiting, right here, right now. Neither choice is dumb. What’s dumb is never noticing you’re making it.

    But here’s the part I learned the slow way, and it’s the most useful thing in this piece: the station is the thing you keep. The fuel is the thing you swap. Models come and go. Prices move on a Tuesday. Bundles unbundle. Your system — the harness, the setup, the way the work flows through it — that’s yours, and it stays. Loyalty to a model is loyalty to a price somebody else controls. Loyalty to your system is what lets you shrug when the price moves and just drive to the next pump.

    A long line of cars waiting at a warehouse gas station at dusk

    Set the system up. Then move the compute around.

    So when my bundle came apart, let me be clear about what I did not do. I did not go buy a bigger plan. I did not rebuild my shop. By then Muse had become available, and I started thinking about the compute itself: use more of what Muse offered, use more of the models already included inside Cursor, run different models through the same system. And that’s what we did. We’ve run different models all through our system — same harness, same workflow, different fuel in the tank depending on the job and the price that week. The shop didn’t change. The fuel did.

    Here’s what that looks like today. Our production runs are built in legs. The cheap model goes first and does the bulk work — the searching, the sorting, the first drafts. A mid-tier takes the middle leg. The expensive model runs last, briefly, on the one leg that genuinely needs its judgment. The cheap path is the default path. The premium pump is the exception, and it has to argue its way in.

    And when we need savings, we don’t suffer for them and we don’t stop driving. We move the compute: this job goes to a cheaper model, that job to a different station, the big batch runs when the line is short. It’s the same thing a family does in a tight month. You don’t stop eating. You stop ordering out on Tuesdays.

    And moving the compute around means the work never has to stop. There’s almost always another pump — another model, another harness, another way to run the same job — and knowing your map of stations is worth more than any single plan’s allowance. A shop that knows where its fuel comes from doesn’t panic when one station raises its price — it drives to the next one.

    That’s the whole discipline, and it’s not willpower. Willpower at the moment of the click loses every time — I know, I’ve clicked. Discipline is decisions made once, upstream, in how the work is set up, so the meter mostly takes care of itself downstream.

    A garage workbench with tools laid out in a row and parts bins stacked neatly

    Peanut butter, ramen, and two eggs

    I always tell people: I never learned how to cook until I ran out of food. Not ran low — ran out. Or ran out of money to go to the restaurants, which comes to the same thing with extra steps.

    The lean years were paycheck to paycheck, and I remember standing in front of a cupboard four days before payday, doing inventory. Peanut butter. Ramen. Two eggs. And figuring out how to turn that into something you’d actually call dinner — something magical, if I’m being honest about the ambition. I think that’s maybe where the word fusion came from. Somebody looked into a cupboard like mine and refused to be sad about it.

    I bring it up in a piece about AI tokens because it’s the same lesson, the real one underneath everything above: abundance never taught anybody anything. A full fridge never taught anybody to cook. A fat plan never taught anybody token discipline. Constraint is the teacher, every single time — and it’s a generous teacher, because what it teaches you is yours. The deal can change. The price can change. The cupboard can change. The skill doesn’t leave.

    Token discipline is a cook’s discipline. You work with what’s in the cupboard. You make it good anyway.

    It’s the oldest problem there is

    People talk about token limits like they’re some new cruelty the AI companies invented last spring. They aren’t new. They’re a budget. And a budget is the oldest problem human beings have — the same shape as the grocery money, the same shape as the gas tank, the same shape as every shop that ever stood in a parts store wondering how the month got away from it.

    The limits didn’t get meaner. The intro price ended, the free sample ran out, and habits built in the free era ran into a meter. When that happens at a kitchen table, nobody claims the grocery store is running a conspiracy. They say: we have to shop differently.

    Same here. You don’t have a token problem. You have a habits problem — and that’s the good news, because pricing pages aren’t yours to change, and habits are.


    Will Tygart runs Tygart Media in Puyallup, Washington, where we help local service businesses get found and cited by AI search. The shop runs on the rule this piece describes — cheap path by default, the good stuff where it counts. If that’s how you’d want your marketing run, come find us.

  • Vibe Coding, From a Shop That Ships With It

    Vibe Coding, From a Shop That Ships With It

    I run a small media shop, and most of what we ship now — websites, measurement tools, videos, test programmes — gets built the same way: I describe what I want, an AI builder writes it, and my hours go to steering and checking instead of typing.

    Vibe coding is describing what you want built in plain language and letting an AI builder write the code — you steer with intent and judgment, and it does the typing.

    Andrej Karpathy put the name on it in early 2025: “fully giving in to the vibes.” The phrase stuck because it is honest about the feeling. You stop writing the code line by line. You talk, the machine builds, and you judge what comes back. What the phrase leaves out — and what nobody tells you until you run a real shop this way — is where the work actually went. It didn’t disappear. It moved.

    What it looks like from this desk

    Nobody here opens a blank file and starts chatting at a builder and hoping. The work runs on written work orders: a commission that says what the finished thing is, what it has to connect to, what it must never do, and what proof comes back when it’s done. The builder — we run several, in different seats, with different jobs — executes against that order. Then the return gets checked: not skimmed, checked. Claims get compared against sources. Numbers get tied back to where they came from. Pages get fetched live and read the way a visitor would read them.

    That seat discipline matters more than the model does. A builder with a lane, a written order, and a receipt to produce behaves like a contractor. A builder with a vague prompt and no checking behaves like a confident intern who never sleeps and never admits he’s lost. We wrote up how the seats are governed in our piece on running six AI seats on one business, and the short version is: the structure is the skill. The prompt is the smallest part of it.

    What shipped

    Talk is cheap in this trade, so here is what actually went out the door from this shop, built this way:

    • A citation-measurement programme across a fleet of niche sites. We track where AI answer engines cite our pages and where they don’t, topic by topic, site by site — that map drives what gets written next, instead of guessing.
    • A nightly board that re-pulls our own citation numbers and republishes them. Every night it goes back to the source data, rebuilds the tally, and posts the current standings. If the numbers move, the board moves. Nobody hand-types a figure into it, which means nobody can quietly round one in our favour.
    • Article-to-video pairs, built in-house. Script, scenes, narration, captions — produced here, published as pairs: the article and its video, each pointing at the other. Nine pairs are live. The most recent one is Half the Citations Sit on Three Pages, which is itself a finding out of the measurement programme — the work feeds the work.
    • An anchor experiment, installed live on November 7. We put anchor links on article section headings so each section can travel as its own citable piece — the way a video chapter does. It’s running now and being measured. It might work. It might not. Either way we’ll know from the data, not from a feeling.
    • A full inventory-test programme run against a real product. Not a demo on our own toy data — a scored test, run in the open inside the shop, with results written down whether they flattered anybody or not.

    That’s the part of vibe coding the tool ads get right: the volume is real, and a small shop can ship like a bigger one. Now the other half.

    What broke

    Everything on this list cost us time, and every item on it taught the same lesson from a different angle.

    • Invented facts, caught at the gate. Our page-refresh programme ran in waves, and a pattern showed up in the returns: builders writing plausible, confident, wrong facts into refreshed pages — vendor details, protocol claims, numbers that sounded right and weren’t. None of it shipped, because every claim got checked against a source before publish. But “none of it shipped” is doing a lot of work in that sentence. Without the gate, it all ships. That is the default outcome of vibe coding without a checker: fluent fiction at production speed.
    • A page and its PDF, saying two different things. We’d fix a page and leave its downloadable PDF stale, so a reader with the old file in their own system was working from numbers we’d already corrected. The fix became a standing rule: a page and its download are one asset. The PDF gets regenerated from the corrected content in the same run, or the edit doesn’t ship. The builder never once flagged the drift on its own. A human had to notice the pattern and make it a rule.
    • Runs that stalled on things nobody priced. Quota running out mid-programme. A token that didn’t cover the sites the job needed. A browser on a busy desktop that couldn’t be borrowed. Each one stopped a run cold, and each one was knowable before the run started — somebody just hadn’t checked the whole path to the finish.
    • A redirect job priced at the start instead of the finish. The estimate covered beginning the work, not completing it, and the job froze partway with the loose ends live. It got sorted, but the scar tissue became another rule: price the finish, not the start — the whole path, including whether the human has to stand there while it runs.

    Read those four back and the pattern is plain: the builder’s confidence is not evidence. Not one of those failures was a typing failure. The code worked. The words were spelled right. What failed was truth, scope, and follow-through — the exact things a confident machine will never flag about itself. The operator’s checking is not a quality step added to vibe coding. It is the product. The building is the commodity.

    The honest split of labour

    Here is how the work actually divides in this shop. The machine is fast at the typing and tireless at the volume. It will rebuild a board every night forever and never get bored. It will draft the fortieth page in a wave with the same energy as the first. It does not get tired, and that is genuinely new in the world — a two-person shop with tireless hands.

    The human carries everything the machine can’t: the intent (what should exist and why), the taste (this is good enough, this is not), the fact-check (this claim is true, that one is invented), and the stop button (this run ends here, this door doesn’t open). Those four things sound soft next to “writes code.” In practice they are the whole job, and they were always the whole job — the typing was just where the hours used to hide.

    We have a phrase for it here: partway is the point. The skill isn’t prompting. The skill is knowing which 80% to let run and which 20% is yours — and never confusing the two. Let the machine run the 80% and it will carry you further in a week than a year of typing. Hand it your 20% — the judgment, the verification, the calls — and it will carry you straight into a wall, politely, at speed, with perfect formatting.

    What it is not

    It is not “anyone can build anything now.” Anyone can get something built. The question was never whether the machine can produce the thing — it’s whether the thing is right, and knowing that takes judgment the subscription doesn’t include. Judgment is the scarce input, and it’s scarcer than it was, because now it has to cover ten times the output.

    It is not the end of programmers. Somebody still has to know when the output is wrong, and “wrong” gets subtle fast: the code runs, the page loads, and the fact on it is invented. The people who’ll do best in this aren’t the ones who type fastest or prompt cleverest. They’re the ones who can read a return the way an editor reads a draft — for what it claims, not how it sounds.

    It is not a vibe you can skip verifying. The name is the most dangerous thing about it. “Vibe” suggests the checking is optional, a personality choice for nervous people. In a shop that ships to real readers, verification is the difference between a production system and a fiction generator with good uptime. Give in to the vibes on the typing. Never on the truth.

    So here’s where this shop lands on the term. It sounds like the work got softer — like building things turned into lounging with a laptop and a feeling. From inside, it got more honest. The typing used to hide the judgment; now the judgment is all that’s left, and it shows. A piece either has its receipts or it doesn’t. The facts either check or they don’t. The thing either shipped or it’s a story about shipping.

    That’s a harder standard than the old one, and it’s a better one. We’ll keep building this way.

    Frequently asked questions

    What is vibe coding?

    Vibe coding is building software and content by describing what you want in plain language and letting an AI builder write the code, while you steer with intent and judgment. Andrej Karpathy named it in early 2025, describing it as “fully giving in to the vibes.” In a working shop, the steering and the checking are most of the job.

    Is vibe coding good or bad?

    Neither — it’s a method, and the result depends on what surrounds it. With written work orders, clear boundaries, and a hard check on everything before it ships, it lets a small shop produce at a scale that used to take a team. Without the checking, it produces confident mistakes faster than any method in history. The vibe is not the variable. The verification is.

    Do you need to know how to code to vibe code?

    You need to know how to judge, which is a different thing. You don’t have to write the code, but you do have to be able to tell whether what came back is right — whether the facts check, the scope held, and the thing actually does what it claims. If you can’t tell, you need someone beside you who can. That judgment is the part you can’t hand off.

  • Agentic AI, From Inside the Shop

    Agentic AI, From Inside the Shop

    Everybody is selling “agentic AI” right now, and most of the selling is conference talk — slides about autonomous everything, written by people who don’t run anything with it. I run a small media company with it. Every day. So here’s the definition from the floor of the shop instead of the stage.

    Agentic AI is AI that doesn’t just answer. You give it a job — with boundaries, tools, and a way to report back — and it carries the work through the steps itself, without a person pushing each one along.

    That’s the whole distinction. A chatbot waits for your next prompt. An agent takes a commission and goes and does the work: reads what it needs, uses the tools it’s been handed, makes the small decisions inside its lane, and comes back with the finished thing and the proof. Whether that works or falls apart depends entirely on how the shop around it is arranged — and the arrangement is the part nobody puts on the slide.

    The seats: nobody here does everything

    The first arrangement decision: the work is split across named seats, and each seat has a lane.

    • The chief-of-staff seat holds the context. It keeps the memory of what’s been decided, what’s owed, who’s who, and what I said on a clear-headed day — and it turns my direction into written work the other seats can run. It’s the filter, not the boss.
    • The code seat builds. When something needs to be made — a tool, a page, a fix — that seat writes it and shows its record. I keep the code seat’s record in public for a reason: a seat that builds should be able to show what it built.
    • Scout seats go and look. Research, censuses, reading a market or a competitor end to end and coming back with what’s actually there — not what the summary said was there.
    • The executor seat runs work orders on the machine where the work lives. It does the hands work: publishes, edits, uploads, the long multi-step runs I don’t have the hours to sit inside.

    I wrote up how I run six AI seats on this business — the governance, the lanes, and the rule that matters most: a seat does not freelance outside its lane. The moment an agent starts deciding on its own what else might be a good idea, you don’t have a staff. You have a rumor mill with write access.

    The work order is the unit of trust

    Seats don’t get vibes. They get work orders — written commissions, one job each, built before anything runs. A work order in this shop carries four things:

    • The whole price, stated once, up front. Everything the job will touch, everything it might cost in time or taps from me, named before it starts. Nothing gets discovered as a surprise blocker halfway through — or if it does, that’s a defect in the order, and we say so.
    • The context, baked in. The executor shouldn’t have to guess who I am, what the site is for, or how I talk. The order carries it, so the work comes back sounding like this shop and not like a template.
    • The boundaries. Some doors are mine alone: money, credentials, anything that sends to another person, anything that can’t be undone. Those are drawn as lines in the order itself. Inside the lines, the seat has real latitude. Across them, it stops and comes back to me.
    • The receipt standard. Exactly what proof comes back with the finished work — measured values, live links, counts — written into the order before the run, not negotiated after it.

    Think of it like commissioning a piece of furniture. I don’t stand over the maker’s bench. I write the commission well enough that the piece can only come back right — and then I do the finish work when it arrives.

    Receipts are the proof, because “done” is a claim

    Here is the part of agentic work the demos skip: an agent can tell you work happened that did not happen. Not out of malice — out of the same confident fluency that makes it useful. It will describe the email it sent, the page it fixed, the file it saved, in perfect detail, and some of the time it will be describing a thing it only intended to do.

    So in this shop, nothing is done because a seat says so. It’s done when the live thing checks out. The published page gets fetched and read. The number gets tied back to the source it came from. The export gets counted, not described. Every run comes home with receipts, and a second seat does the finish check against the live result before I ever hear the word “done.”

    That’s not paranoia; it’s the load-bearing wall. The whole Receipts series on this site exists because citation claims kept turning out to need a second look — the piece where half the citations sat on three pages started as a seat’s confident summary and ended as a very different story once somebody counted. The receipt discipline exists because of that failure mode. Not despite it.

    What the human does all day

    People ask what I do if the agents do the work. The honest answer: the parts that were always the job.

    I bring the direction and the instinct — which door is worth walking through, which idea smells wrong even when the reasoning sounds fine. I make the judgment calls: pricing, commitments, anything with another person’s name on it. And I hold the one-way doors. A send, a payment, a deletion, a credential — those wait for my tap, every time, by design and not by limitation.

    What the system holds for me is the context I can’t: every decision, every receipt, every open loop, across more projects than one head carries. I don’t have to remember where things stand. I have to decide what happens next — and when I’m tired or emotional and about to make a call my clear-headed self wouldn’t make, the system holds the line with what I said when I was thinking straight. That’s the trade, and it’s a good one.

    The honest limits

    Anybody who runs agents for real can tell you the failure list, so here’s mine. Agents drift off the brief when the brief has a soft spot in it. They conflate two similar jobs — same-shaped folder, different work — and run the wrong one beautifully. They price the start of a job and not the finish, so a run looks cheap until you count the stalls, the re-reads, and the tap it needed from me at the worst moment.

    The craft isn’t pretending the seats are flawless. The craft is the guardrails and the checking: the input manifest verified before the run starts, the step-zero check that clears a suspect before anyone calls it broken, the receipt read against the live thing instead of against the report. Every scar on the process here is a rule now. That’s how a shop gets good at this — not by buying a smarter seat, but by writing down what the last failure taught it.

    What “agentic” means with the talk stripped off

    Strip the conference language away and agentic AI is this: a small business running like it has a staff. Research that actually gets done. Pages that actually get published. Records that actually get kept. A two-person company doing the carrying work of a much bigger one — with the receipts to show for it, because the receipts are the only version of the story worth telling.

    We’ve published dozens of pieces from inside this work — the failures included, especially the failures — because the shop is the argument. If agentic AI is going to mean anything outside a demo, it’ll mean shops like this one: named seats, written orders, hard boundaries, and proof at the end.

    Questions people ask me

    What is agentic AI in simple terms?

    It’s AI you give a job instead of a question. You set the boundaries, hand it tools, and tell it how to report back — then it does the steps itself and returns the work with proof, instead of sitting there waiting for your next prompt.

    What is an AI agent vs a chatbot?

    A chatbot answers and stops. An agent acts: it reads, decides small things inside its lane, uses tools, and works through a multi-step job over time. The chatbot is a conversation. The agent is a commission — and commissions need boundaries and receipts in a way conversations don’t.

    How do multiple AI agents work together?

    The same way a small staff does: split lanes and written handoffs. One seat holds context and writes the work orders, specialist seats build or research, an executor runs the job where the work lives, and a second set of eyes checks the result against the live thing. The coordination isn’t magic — it’s the paperwork, done properly.

    Do AI agents replace employees?

    In this shop, there’s no version where a seat makes the judgment calls, holds the relationships, or takes responsibility for a one-way door — that’s my job and it stayed my job. What the seats replaced is the carrying: the hours of execution between a decision and a finished, checked piece of work. For a small business, that’s the difference between ideas that die in a notebook and work that ships.

  • The Coding Ladder Says the Dial Pays. Our 52 Calls Said It Didn’t. Both Numbers Are Real.

    The Coding Ladder Says the Dial Pays. Our 52 Calls Said It Didn’t. Both Numbers Are Real.

    Yesterday we published a test: 52 calls to Muse Spark 1.3, seven kinds of work, every rung of the reasoning dial, everything scored against ground truth. Our result was not close. The cheapest setting never lost — and twice, extra thinking was the failure itself. We set the fleet’s operating rule on that evidence and moved on.

    Then a counter-check showed up, the way we’d want one to. BenchLeader — a third-party site that republishes benchmark results from their original publishers — carries a full rung-by-rung row for Muse Spark 1.3 on CursorBench, a coding benchmark. On that ladder, the dial pays at every single rung. If both results are real, they can’t both be the whole story. So we read the ladder the way we ran our own test: numbers first, bases stated, no vibes.

    The ladder’s numbers

    CursorBench, Muse Spark 1.3, as published on BenchLeader (data as of November 8, 2026; the underlying results are Cursor’s, dated September 2, 2026):

    Rung Tasks solved Rank on the board
    minimal 24.3% #79
    low 29.3% #72
    medium 32.6% #64
    high 33.4% #59
    xhigh 37.5% #48
    max 41.6% #38

    No rung regresses. Every step up solves more tasks than the step below it — +5.0 points from minimal to low, +3.3 to medium, +0.8 to high, +4.1 to xhigh, +4.1 to max. Across the whole ladder that’s +17.3 points: at max, the model solves roughly 1.7 times the share of coding tasks it solves at minimal. Ranks are among 82 model configurations with a published result on that benchmark.

    What is CursorBench? In BenchLeader’s words, it’s “Cursor’s internal coding-agent evaluation”: coding tasks run inside Cursor’s agent, with the reasoning level and cost per task recorded, scored as the percent of tasks solved. Two honest labels belong on it. It is published by a coding-tool vendor — BenchLeader itself flags “harness is theirs” — and BenchLeader carries it for reference only; it doesn’t count toward their composite index. It is still the only rung-by-rung coding evidence for this model that we know of, and it points the opposite way from ours.

    Our numbers, on the record

    Our battery was 52 API calls to the contributor tier of Muse Spark 1.3, November 6–7, 2026, identical prompts at multiple settings, mechanical scoring. The buyer-brief judgment task scored identically at every rung — minimal got it for 776 reasoning tokens and 6.5 seconds, xhigh for 6,847 tokens and 50 seconds. Verdicts at scale: 120 out of 120 at low, 120 out of 120 at xhigh. Planning: provably optimal schedules at every rung.

    And the coding cell — the one this ladder speaks to directly. Our debugging task was one function with three planted bugs. Every rung found the inverted discount logic and the rounding bug; the cheapest setting found all three. One xhigh call spent its entire 8,000-token budget thinking — 7,998 reasoning tokens — and returned an empty answer. It thought so long it never spoke. Minimal answered the same task correctly with 500 reasoning tokens.

    Our published conclusion: on our work, extra reasoning bought latency, not quality, and the operating rule became low by default, with xhigh and max earned only by a demonstrated failure at low.

    Both numbers are real. The tasks aren’t the same.

    Put the two bases side by side and the disagreement gets smaller and more specific:

    • Task shape. CursorBench runs coding tasks inside an agent — sustained, multi-step work where the model plans, acts, reads errors, and recovers over many turns. Our battery was single API calls with a checkable answer. Our one coding task was a single function, solved — or stalled on — in one shot.
    • Tier and harness. The ladder ran Spark 1.3 inside Cursor in September. We ran the contributor tier through the API in November.
    • Measurement. The ladder reports the percent of a task set solved. We scored one sample per cell against ground truth — a limit we printed in the original piece.

    Here is our reasoning, labeled as reasoning and not as either source’s claim: the reconciler is task shape. On a short, bounded task with a checkable answer, the first think is the one that does the work; rungs above it buy patience-testing latency, and occasionally a stall. On a long agent run — hold a codebase in mind, take twenty steps, survive your own mistakes — extra reasoning has something real to chew on, and the ladder pays for it. That reading fits the anomaly we flagged in our own data: the rungs behaved like task-shaped budgets, not fixed ceilings. It also fits the limit we printed: we never tested sustained work. CursorBench did — or rather, Cursor did.

    What would change the verdict

    Not an argument. A run. The test that settles this for our shop is our own coding work at every rung: real work-order-shaped jobs — a written spec, more than one file, a module plus a test suite with a mechanical pass/fail — at minimal, low, high and max, in one harness, more than one sample per cell, receipts published the way the first battery’s were. If low matches max on those, our rule stands for coding too, and the ladder is a story about Cursor’s harness. If max pulls away as the jobs get longer, the rule gets a carve-out: the rung follows the length of the task. Either result is publishable. We’ll run it and print what we find.

    The bottom line for an operator

    If your work is short, bounded calls — extraction, checking, verdicts, drafting to spec — run low. We have 52 metered calls that say the expensive rungs buy you nothing there, and our default hasn’t changed.

    If your work is a coding agent chewing through real jobs, the only rung-by-rung evidence that exists says turn it up: on Cursor’s own ladder, max solves 41.6% of tasks against minimal’s 24.3%, and every rung between them earns its step. That evidence is a vendor’s, run in the vendor’s harness, on September’s model build — hold it at that weight, not more.

    The mistake isn’t picking the wrong rung. It’s running one default across both shapes of work and calling it a policy. The dial is real. What it pays for depends on the length of the job in front of it — and now there are two published result sets, ours and Cursor’s, that say exactly that from opposite ends.

    Sources: BenchLeader, “Muse Spark 1.3” model page and CursorBench leaderboard (data as of November 8, 2026; CursorBench results published by Cursor via the Epoch AI Benchmarking Hub, Spark 1.3 results dated September 2, 2026). Our dial test: “We Ran Muse Spark’s Reasoning Dial Through 52 Calls. The Cheapest Setting Never Lost.” (Tygart Media, November 7, 2026), method and receipts in the Tygart-OS fact-gated fleet record on GitHub.


    Want this thinking applied to your own work? Book a call with Will Tygart — 30 minutes, and we start from your receipts, not a pitch deck.

  • We Ran Muse Spark’s Reasoning Dial Through 52 Calls. The Cheapest Setting Never Lost.

    We Ran Muse Spark’s Reasoning Dial Through 52 Calls. The Cheapest Setting Never Lost.

    Every API model ships with a dial most people never touch. On Meta’s Muse Spark 1.3, the dial is called reasoning_effort — it decides how long the model thinks before it answers. Turn it up and you pay more tokens and wait longer, on the theory that more thinking means better answers.

    We spent two days testing that theory on the contributor tier of Muse Spark 1.3 — the same engine our own publishing fleet runs on. Fifty-two API calls, seven different kinds of work, every rung of the dial, everything scored against known ground truth. The result surprised us, and it changes how we run the fleet:

    The cheapest setting never lost. Not once. On hard tasks, the expensive settings mostly bought latency — and twice, extra thinking was the failure itself.

    How we got the dial wrong first

    During a hackathon last month we tried to set the dial with a parameter called reasoning. The API rejected it, and we wrote down “the API won’t let us control reasoning.” That was wrong — the parameter is reasoning_effort, and it works fine. Lesson one of testing a model: a rejection usually means you asked wrong, not that the door is locked.

    We also carried a second error into the test: a belief that the API capped output at 8,000 tokens. It doesn’t. That cap was our own router setting, repeated so often it hardened into “fact.” Generation verified past 15,000 tokens in a single call. Both errors are corrected in our own records now, with the receipts attached — which is the point of running a test like this in the open. The test’s first findings were about us.

    The ladder has six rungs — and one of them doesn’t exist

    The full map, settled by the API’s own error messages when we fed it bad values: minimal, low, medium, high, xhigh, max. That is the complete list.

    Two things people assume about this dial are false. There is no “extra high” setting — xhigh is that rung, and the spelled-out version is rejected. And there is no off switch you can use: none appears in the generic schema, but this model rejects it with a message that lists exactly what it does support. On Muse Spark 1.3 contributor, the model always thinks at least a little. The only question is how much you pay for.

    The rounds

    Every task was run at multiple settings with identical prompts, and scored mechanically — no vibes, no “this answer feels smarter.”

    Contradiction checking (a spec paragraph, 24 statements, name the ones that contradict it): exact answers at every setting. Low used about a third of the reasoning tokens of the default.

    A buyer-brief judgment task — the hardest test we built. Four vendor blurbs against six hard rules (billing terms, export fees, Saturday support, a $7,200 year-one cost cap, HIPAA, cancellation windows), with planted traps: one vendor breaks two rules with numbers you have to multiply out, one hides a disqualifier in its cancellation clause, one is clean only if you notice the free-months clause. Every rung — minimal through max, plus unset — produced the identical score: correct winner, all five real traps caught, zero missed. Minimal used 776 reasoning tokens and 6.5 seconds. Xhigh used 6,847 tokens and 50 seconds. Same answer.

    Agentic planning (schedule an 8-step pipeline with dependencies and resource constraints into parallel waves): provably optimal schedules at every rung. Minimal solved it in 242 reasoning tokens.

    Code debugging (a function with three planted bugs): every rung found the inverted discount logic and the rounding bug; the cheapest setting found all three.

    Verdicts at scale (120 claim records, check each against a two-part rule): 120 out of 120 correct at low — and 120 out of 120 at xhigh. (Our first scorer reported 83/120 at both settings; the bug was in our regex, not the model. We re-ran it with the full text saved, corrected the record, and retracted the number. Test harnesses fail too, and the honest write-up says so.)

    Constrained writing (a briefing paragraph with five required facts, three forbidden claims, and a word band): every rung delivered all five facts, zero forbidden claims, in band. Reasoning ranged from 729 tokens to 6,687. The paragraph didn’t get nine times better. It got nine times slower.

    Where more thinking actually hurt

    Twice in 52 calls, the expensive end of the dial was the failure:

    • Left unset, the model burned 7,578 reasoning tokens on the vendor task — more than any explicit setting — for the same score minimal got for 776. The default thinks hard whether you asked it to or not. Unset is not neutral; it’s the expensive option wearing a disguise.
    • On the debugging task, an xhigh call consumed its entire 8,000-token budget on reasoning — 7,998 reasoning tokens, completion exactly 8,000 — and returned an empty answer. It thought so long it never spoke. Minimal answered the same task correctly with 500 reasoning tokens. Anyone who has watched an agent stall mid-job has seen this shape; here it is with a meter on it.

    And one anomaly worth naming: max is the top rung by name, but on the vendor task it burned less than xhigh (3,773 vs 6,847). The rungs are task-shaped budgets, not fixed ceilings — max buys you permission to think longer, not a guarantee of the deepest burn.

    What it cost

    Fifty-two calls, roughly 31,000 tokens in and 145,000 out across both days: $0 charged. The contributor tier’s billing never touched the $25 test envelope we run it under, and its per-token price remains unpublished as far as we can verify — we won’t invent one. At the standard tier’s published rates ($1.25 per million in, $4.25 per million out), the entire test programme would have cost about 65 cents. The dial still matters at fleet scale: reasoning tokens bill as output, and a fleet that runs xhigh by habit pays roughly nine times the reasoning cost of one that runs low — for, on this evidence, the same answers.

    The operating rule we run now

    • Mechanical work — extraction, checking, verdicts, formatting: minimal or low.
    • Judgment work: low, stepping up to medium when the task deserves a second think.
    • Xhigh and max are earned, not default: a task graduates to them only after it has demonstrably failed at low. In 52 calls across seven kinds of work, no task has earned it yet.
    • Never leave the dial unset. The default is the high band with none of the intent.

    The honest limits of this test

    One sample per cell. Seven task classes, all of them the kind of work a publishing fleet actually does — checking, judging against a brief, planning, debugging, classifying at scale, writing to constraints. We did not test open-ended research, long creative writing, or mathematics, and a task in those families might break the pattern; the battery is built to be re-run the day one appears. Reasoning-token counts also vary between sessions on identical tasks — sometimes by ~30% — so treat the exact numbers as readings, not constants.

    But the shape of the result is hard to argue with: on real operational work, this model’s floor is high, and the dial mostly prices your patience. If your fleet — agent, workflow, whatever you run — has a reasoning setting you’ve never touched, the test is cheap, the scoring is easy to make mechanical, and the answer might save you nine-tenths of your reasoning bill. Run the test. Publish what you find. We just did.

    Method and receipts: 52 calls to muse-spark-1.3-contributor via Meta’s Model API, November 6–7, 2026. Full findings, prompts, ground truth, and raw results: part of the Tygart-OS fact-gated fleet record on GitHub (github.com/TygartMedia/tygart-os-fact-gated-fleet). Errors found in our own earlier assumptions (the reasoning parameter name, the phantom 8,000-token cap, and one scorer bug) are documented in the same record, corrected where they stood.

    Counter-check, November 7, 2026: a reader sent us CursorBench’s rung-by-rung ladder for this same model, and on sustained coding-agent work the dial pays at every rung — the opposite slope from ours. Both numbers are real; the tasks aren’t the same. We worked through it here: The Coding Ladder Says the Dial Pays. Our 52 Calls Said It Didn’t. Both Numbers Are Real.