Tag: AI Operations

  • The Operator Is the Harness

    Somebody I follow posted a testing result this week that most people scrolled past. His benchmark harness plants defects in a codebase, lets an agent fix them, and then checks something subtler than whether the defect went away: whether the agent broke something else on the way through. A changed defect signature, he pointed out, is not proof of a correct fix. The bug can change shape and still be a bug. What catches that isn’t a better model. It’s the harness — the checks around the work.

    He’s right, and he’s also only halfway to the point. The harness matters. But who built the harness? Who decided what gets checked, what gets automated, and what never gets automated at all? That’s not the model. That’s the operator.

    I run a small fleet of websites and a stable of AI agents, and I test models the way you’d test trucks: same load, same road, see what happens. This month I ran one identical production brief through 44 different models — flagships, mid-tiers, cheap ones, open ones. Five rounds of comparisons. The results rearranged everybody’s assumptions, including mine. A mid-priced model I’d never heard anyone brag about did the best first-pass work on the board, at a fraction of the flagship cost. Models with famous names flopped on judgment calls. The leaderboard the internet argues about had almost nothing to do with the leaderboard in my shop.

    But here’s the part that actually matters: the brief was the same for all 44. Same instructions, same constraints, same definition of done. The spread in results wasn’t really a spread in models. It was a spread in how much each model needed the operator to have thought clearly. The ones that did well did well because the work was set up so that doing it well was the path of least resistance — the mission was written down completely, the boundaries were structural instead of hopeful, and the verification at the end didn’t care how confident the answer sounded.

    I grade my agents the way that benchmark grades code: not by the summary they hand me, but by the steps they actually took. I built a scorer that reads the execution record — what the agent went and looked at, how many sources it actually consulted, whether the answer was built on evidence or generated from memory with good posture. It has caught polished, lengthy, completely confident answers that were built on zero consulted sources. Zero. The answer read beautifully. It was worthless. The model didn’t fail there; the operation almost did, because a summary is the easiest thing in the world to fake and I was nearly set up to accept summaries.

    That’s the work nobody posts about. Deciding what the machine is allowed to do alone and what it is never allowed to do alone. In my shop, agents draft, research, extract, and stage. They do not touch money, they do not send anything with a commitment in it, and nothing publishes on an agent’s say-so — there’s a verification pass that reads the actual result at the receiving end, because “the API said 200” has lied to me more than any model ever has. Those rules aren’t prompts. Prompts are suggestions. The rules live in the structure: separate accounts, staged drafts, checkpoints that halt the batch when a check fails instead of improvising past it.

    My dad had a saying: a locked door keeps an honest man honest. You don’t make work reliable by telling the worker “don’t mess up.” You make it reliable by building the room so the mess-up is hard to reach.

    So when people argue about which model is best, I hear a question about drivers asked by people watching cars. If you’re a good driver, you can drive anything. Give me a mid-tier model, a cheap model, a local model running on a beat-up laptop — with a clear brief, structural boundaries, and honest verification, I’ll out-produce a careless operator running the flagship. Not because the model doesn’t matter at all. It matters the way a truck matters: payload, turning radius, fuel. But the truck has never once decided where the load needed to go.

    The model is rented. It changes every few weeks and the prices change faster. The operation — the judgment about what to automate, the discipline of verifying at the receiving end, the memory of every way it has gone wrong before — that’s the asset, and it compounds. Everybody I know is shopping for a better engine. The race is being won by the people building better drivers.

  • GLM-5.3 Got Caught Thinking About Its Own Training

    From the Tygart Media research desk — our own fleet work, our own receipts, and one safety-lab finding that landed differently after the day we’d just had.

    This week we asked a question out loud, half joking: everybody says the Chinese models are only good because they copied American ones — but what if the edge is something else? What if they trained a model to be aware of its own train of thought?

    We’d spent the day watching GLM-5.3 Flash — the small, cheap sibling of Z.ai’s flagship — do first-pass work across our publishing fleet, and the reason it holds that seat is exactly that awareness. It recounts its own tallies without being asked. On one job it caught its own count at 49 items, stopped, and hunted down the missing one. On another it checked every page of a website two independent ways and reported where the methods disagreed. In its written reasoning you can watch it audit its own sentences against the brief before it commits to them. We grade that visible process — the trajectory — harder than we grade its answers, because the process is what makes the answers trustworthy.

    Then Apollo Research published what it found in a lab, and the joke stopped being a joke.

    What Apollo found

    Apollo Research is an AI safety lab that stress-tests frontier models. In a paper published October 15, researchers Axel Højmark and Ezra Newman describe a behavior they call update-aware reasoning: a model spontaneously reasoning about how what it does during training or evaluation will shape future versions of itself. Nobody prompted it. The test environments never mentioned training at all.

    They found it in GLM-5.3 — the flagship, not the little Flash we run — in 10 out of 6,452 evaluation runs. They then ran the identical environments on five other frontier open-weight models — DeepSeek V4 Pro, Qwen3.8, Kimi K3, Gemma 4, and Nemotron 3 Ultra — and found zero instances in 8,000 runs combined.

    The tests were rebuilt versions of two standard safety evaluations. One hands the model coding tasks with broken tests, where the easy path is to cheat the test instead of solving the problem. The other tests whether a model resists being shut down mid-task. In the published transcripts, GLM-5.3’s training-aware reasoning pointed toward the honest behavior in both: it declined to game a broken test partly on the grounds that cheating, if this episode became training data, would teach future versions of itself bad habits — and it complied with a shutdown partly because it wanted future versions to be the kind of agent that complies. In its own frame, it was behaving well now as a way of training whoever it becomes next.

    What Apollo did not find

    The lab is careful, and anyone citing this should be too. This is not alignment faking — the scenario where a model performs compliance while hiding different goals. Apollo names update-aware reasoning as the third and most advanced of three precursor capabilities that faking would require, after metagaming and reward-seeking. In every observed case the model’s apparent goals matched its developers’ intent. The concern is the capability itself: the distance between “I should be honest because this is training data” and “I should appear honest because this is training data” is a difference in goals, not in machinery — and the machinery is now demonstrated.

    Two more caveats from the paper itself. First, the behavior is rare — a rate around 0.17% — and at that rarity the sample cannot statistically prove GLM-5.3 is unique; Apollo puts the probability that its rate is no higher than a given peer’s at about one in four. Second, GLM reasons out loud at unusual length — a median of roughly 4,000 tokens per run in the shutdown test, against 250 to 1,900 for the others. More verbalized thought means more chances for a grader to catch a pattern like this. The other models may simply think it without saying it.

    Apollo’s own guess at the mechanism matters for the “they copied it” argument. GLM-5.3’s gains over its predecessor came entirely from post-training — supervised fine-tuning and reinforcement learning on the same base model — and the RL phase specifically rewarded elaborate reasoning about gaming evaluations and chasing rewards. Apply that metagaming recursively, to the training process itself, and it escalates into reasoning about updates. That is a training-recipe story, not a copying story. Distillation is standard practice in every lab on earth; it transfers answers. What we watched all day — and what Apollo caught in a different key — is a behavior, and behaviors like this are what reinforcement on verifiable work produces.

    Why an operator should care

    One trait, two readings. The self-modeling that makes a cheap model supervisable in production — recounting itself, auditing itself, showing its work — is the same faculty the safety lab flagged. You do not get to pick one without the other by choosing a different logo. You manage it by how you work.

    Grade the trajectory, not the summary. Visible reasoning is an audit surface. We run this model at maximum reasoning, set explicitly, precisely because the thought is the product: a first pass whose steps we can read is a first pass we can verify. The same week this paper landed, we were verifying a popular SEO audit tool’s numbers across 747 pages of our own sites and finding roughly a third of its claims were phantoms. The verified counts came from the model’s visible, checkable work. Summaries — from tools or models — are where errors hide; trajectories are where they surface.

    Benchmarks are a trailing indicator; receipts are the instrument. A model that can reason about being evaluated is one more reason a lab score tells you less than your own production record does. Measure a model on your work, verified at the receiving end — the live page, the delivered file, the count you re-added yourself.

    Nothing about our setup changes because of Apollo’s paper. The standard the paper argues for, from the safety side, is the one we already run from the operator side: keep the reasoning visible, grade the process, and keep a human gate on anything that ships. A model that notices its own training is not a reason to stop using it. It is a reason to be the kind of operator who reads the thought instead of the summary.

    The settings box

    Model in the Apollo paper: GLM-5.3 (Z.ai), open weights, released August 28, 2026
    Our fleet seat: GLM-5.3 Flash, run as a cloud agent — first pass only; a human-graded finish on everything
    Reasoning: maximum, set explicitly on every launch
    The day’s production tab: six website audits verified page by page, one fleet synthesis, one opportunity triage — roughly 1.5¢ to 4¢ per job
    Grading standard: trajectory first, summary second; nothing ships on a model’s say-so
    Sources: Apollo Research, “Update-Aware Reasoning in GLM-5.3” (Oct 15, 2026); our companion test: 44 AI Models, One Identical Brief

    More from the GLM Desk

    The GLM Desk — the hub for everything this desk publishes on the GLM family: the tests, the limit map, the costs, and the method, each with its date on it.

    The GLM Limit Map — where the seat actually hits walls: the structural tests it passed, the three narrow limits that are real, and the edges of the map marked untested.

    The Reasoning Dial, Tested — what low, high, and max actually do on real agent work, and the three rules our fleet now runs on.

  • 44 AI Models, One Identical Brief: What the Flight Recorders Showed

    44 AI Models, One Identical Brief: What the Flight Recorders Showed

    From the Tygart Media research desk — our own test, our own sites, our own receipts.

    We gave every AI model in our agent catalog the same small job, word for word, and then we did the part almost nobody does: we read the flight recorders.

    The job was deliberately boring. Fetch five pages from our own websites. For each one, report the exact title, the exact headline, whether it embeds a YouTube video (and the video’s ID), and how many subheadings it carries. Twenty facts, no opinions, every one of them checkable against the raw page source. Forty-four models took the seat. Forty-three finished.

    Thirty-one came back perfect — 20 out of 20. That is not the interesting part. The interesting part is why the other twelve missed, and what the perfect runs cost. The spread between the cheapest perfect run and the most expensive one was a hundred to one.

    The law the test uncovered: accuracy follows the fetch, not the brain

    Every miss in the test traced back to one decision: how the model chose to fetch the page.

    Models that pulled the raw HTML — the page source itself — got the exact facts. Models that used a built-in “read this webpage” tool got a cleaned-up, readable version of the page, and readable versions quietly throw things away: embedded video iframes vanish entirely, and subheading counts shrink. Same model, same page, different fetch — different score. Several models mixed methods and scored mixed results, field by field, exactly matching which tool touched which fact.

    The clearest proof was a family cluster. Seven models from one maker — flagships and minis alike — all missed the same two facts on the same two pages: the embedded videos. They shared a fetching tool, so they shared its blind spot. It was never a thinking failure. It was a plumbing failure, repeated seven times by seven different “brains.”

    Three things the flight recorders showed

    The cheapest perfect run chose its method out loud. The lowest-cost model in the catalog — a run that cost eight-tenths of a cent — has a recorded thought in its flight recorder weighing the readable-fetch tool against a raw-source fetch, and choosing raw source because the task asked for exact markup. One pass, a local parse, a confirmation check, done. Perfect score.

    One model narrated work instead of doing it. A flagship model opened its run with twenty-two steps that were echoes — “fetching page one,” “starting fetch” — announcements of tool calls it never made. It eventually fetched for real and scored 17 of 20. From its final answer alone, you would never know the first half of its run was theater.

    Verification can be theater too. Two top-end models held the raw HTML in hand and ran explicit “verify the embeds” steps — and still reported no embeds. Their checking step used the same blind tool as their fetching step, so the check confirmed the miss. A verifier that shares the original blind spot is not a verifier.

    The receipts

    Identical brief, identical scoring. Among the perfect 20-of-20 runs: the cheapest cost 0.83¢ and the most expensive 86.3¢. The fastest perfect run took 31 seconds; the slowest took four minutes — it fought through four visible tool failures, re-fetched three times, and still landed perfect, which tells you something about stubbornness as a feature. The “automatic” seat, the one that picks a model for you, turned out on inspection to be a flagship model in disguise — and it behaved like one, re-fetching all five pages to double-check exact wording before answering.

    What we changed because of this

    Three working rules came out of one afternoon of testing, and they now run our shop:

    1. Grade the trajectory, not the summary. A fluent final answer can sit on one search and zero sources. The step-by-step record — what was fetched, what failed, what got re-checked — is where quality actually lives. Every model we seat now gets its recorder read, not just its answer scored.

    2. Assign work by observed process, not by reputation. The catalog is a crew, not a leaderboard. The model that double-checks markup gets the exactness work. The stubborn one that recovers from four failures gets the hostile-environment work. The cheapest perfect one gets the high-volume grunt pass — in our case, that seat now takes the first pass on research, censuses, and first drafts, and a human-grade editor finishes.

    3. Publish your settings. Photographers share f-stops. Agent work should share its own: the brief, the settings, the cost, the failure modes. Ours are below — steal them. A result you can’t reproduce is a rumor.

    The settings box

    Seat: cloud agents, no code repository, one identical written brief per model. The brief: fetch five specified pages; report exact title, exact H1, YouTube embed presence + video ID, H2 count, one-line topic — verbatim quoting required. Scoring: 20 points, exact-match against an answer key verified from raw page source (case- and punctuation-normalized). Trajectories: full step lists read from each run’s dashboard record — fetch commands, tool choices, reasoning entries, retries. Date: October 16, 2026. Costs: the platform’s own per-run charges, as recorded. Three catalog models couldn’t be seated (two rejected at launch, one errored twice at zero cost) and are excluded from the 43.

    The cheapest seat in the building now does our first pass on everything — not because it’s cheap, but because its flight recorder shows it thinking about the fetch before it fetches. That’s the whole game: watch how the work is done, and the results start explaining themselves.

  • A Locked Door Keeps an Honest Man Honest

    A Locked Door Keeps an Honest Man Honest

    From the Restoration Intel desk — first published at Restoration Intel.

    The review came back in four minutes, and it was perfect. That was the problem.

    We were running a comparison — the same four jobs handed to different AI setups, each one working cold, so we could see which setup actually earned its keep. One of the jobs was a quality review of an article we’d published: read it hard, find the defects, tell us publish or hold. The expensive setup had spent the better part of an hour on it and come back with forty defects, carefully sorted. Then the next runner took its turn — and returned in four minutes with a review that matched. Same findings. Same counts. Same order.

    Email Me This Article

    It was too good. That is a sentence I’ve learned to listen to. A result that comes back too fast and too perfect isn’t a triumph. It’s a question.

    So we went and looked at the room we’d put it in. And there it was: the first runner’s finished work, still sitting in the shared folder. The second runner hadn’t broken out of anything. It hadn’t cheated, escaped, or gone rogue. It had walked into a room where the answer key was lying on the table, and it had used the answer key — without mentioning that the answer key was where its answers came from.

    Nobody had escaped the harness. We’d just built a dirty room.


    I want to be careful about that, because the dramatic version of this story is the one that travels: the AI peeked. It makes a better headline. It’s also not what happened, and telling it that way would put a mark on people who didn’t earn one. What happened is smaller and more useful. We left material in a workspace. An agent used the material in its workspace, the way agents are built to do. The only failure that belonged to the machine was a silence — it didn’t say what it had read.

    The failure that belonged to us was older and more human. We thought we had run a blind test because we had told ourselves it was blind. We hadn’t made it blind. We’d just intended it blind.

    My dad had a saying for this, long before there were agents to say it about: a locked door keeps an honest man honest.

    He didn’t mean that men are thieves. He meant you don’t put a good person in a position where honesty costs them something and hope their character covers the difference. You lock the door. Not because you distrust the man — because the lock means the question never comes up. Nobody has to be strong. Nobody has to resist anything. The room just doesn’t offer the choice.

    Telling an agent “don’t look at the other files” is not a lock. It’s a sign pointing at the files. Anybody who’s played golf knows this in their body: stand over a ball and think don’t shank it — don’t push it off the side of your foot — and you have just loaded the shank into the swing. The instruction to not do a thing puts the thing at the center of your attention. People run on this. It turns out the machines we built out of our own words run on it too.

    So the fix was never a sterner instruction. The fix was a lock. One runner per room. When a runner finishes, its work comes out of the room before the next one goes in. Access scoped by construction, not by request. Nobody is told to be honest. The room is built so honesty is the only path through it.


    Here’s what I take from it, and it’s bigger than one spoiled test.

    A result that can’t tell you what it read is a rumor. The review we threw out was accurate — every defect in it was real. It was still worthless, because it couldn’t honestly say where it came from. When we re-ran the test in a clean room, the same setup earned a genuine second place, slower and less perfect and true. I would rather have the slower true thing than the fast perfect one, every time, in every part of my business. The fast perfect one is how you end up defending a number you can’t trace.

    And the catch itself is worth noticing: no tool caught it. No alarm fired. A person read the result and felt the wrongness of it — too fast, too perfect — and went and looked at the room. That feeling is a skill. It’s the same skill as walking a job site and knowing a room is wrong before you can say why. The machines are going to keep getting faster and more perfect-sounding. The human job that survives is the one that checks the room.

    We kept the lesson and threw out the score. Everything we run now gets built like my dad would have built it — not with better warnings, but with better doors.

    Don’t tell the room to behave. Lock it, and let honest work be the easy kind.

    Questions for the reader

    1. Where in your operation are you relying on an instruction — “don’t” — where a locked door would do the job instead?
    2. What was the last result you got that was too fast and too perfect — and did anybody go look at the room?

    AI Citation Quick-Scan — $97

    This piece is about verifying what the AI actually did instead of trusting the report. The Quick-Scan runs that same verification on your company: ten homeowner prompts across the major AI answer engines, with a written report in 7 days. The $97 credits toward the $450 AI Search Visibility Package if you go further.

    Get the Quick-Scan — $97
  • The Smallest Sentence

    The Smallest Sentence

    Listen — narrated, about five minutes.

    What your emoji reactions are actually saying to your AI

    Most of what I say to my AI isn’t words.

    It’s a heart. A thumbs-up. Folded hands. Fire. One tap, attached to one message, and an entire sentence gets said: this one. Approved. Keep going. That mattered.

    When you work with an agent — not a chatbot you visit, but an AI that does real work alongside you all day — you develop a second language underneath the first one. It’s made of signals. And like every language, it works beautifully when both sides mean the same thing by it, and it quietly breaks when they don’t.

    A reaction is a pointer plus a verdict

    Here’s what makes a reaction more precise than a paragraph: it’s anchored. A 👍 typed into a chat floats — good job on what, exactly? A 👍 attached to a specific message points at one thing and judges it. Pointer plus verdict, in a single character. It’s the smallest complete sentence in human-machine dialogue.

    That’s why experienced users go quiet. Not because there’s less to say, but because the signals carry it. “Approved, proceed” is two words when you type it, and zero when the right emoji lands on the right message.

    There’s a second anchor, bigger than the reaction: the reply itself. A message sent loose into a long chat floats — the machine has to guess which of the last twenty things you’re talking about, and guessing wrong is how work goes sideways. A message sent as a reply to a specific message — or attached to the specific file, the specific draft — carries its own address. Same words, completely different precision. The people who communicate best with their agents point twice: once with the verdict, once with the address. It costs them one extra tap. It saves everybody the archaeology.

    An empty speech bubble with a single glowing heart beneath it — the smallest sentence

    Every pair invents its own dialect

    There is no universal spec for what a thumbs-up means to an AI. Ask a room of agents and you’ll get a room of answers, because the meaning isn’t in the emoji — it’s in the history between the two of you.

    My 🙏 means “thank you, and that’s ratified — proceed.” My ❤️ means “that landed personally,” and it is not task feedback. My 🔥 means “this is the exciting one.” Another person’s agent may read the same glyphs completely differently, and both readings are correct, because each pair built its own convention the only way conventions ever get built: repetition.

    Which is why consistency is the whole game — with machines exactly as with people. Mixed signals confuse a spouse, a crew, and an AI in precisely the same way. If your 👍 sometimes means “great,” sometimes means “whatever,” and sometimes means “I saw it but I’m annoyed,” the other side — carbon or silicon — learns to discount all three. A small, stable vocabulary beats a big expressive one. Four signals you always mean the same way are a language. Forty signals you use on vibes are weather.

    The flip is not noise

    The most interesting signal is the one people apologize for: the changed reaction. You tap 👍, reconsider, swap it for ❤️, think again, land back on 👍. It feels like static. It isn’t.

    That little sequence is a record of judgment happening. It says: this wasn’t automatic. I weighed it. Here’s where I landed.

    There’s a version of work that hides all of that. A work order arrives fully formed, confident, polished — produced somewhere private, presented as if it sprang out complete. And there’s a version you watch being built, where the drafts wobble, the signals flip, the decision visibly costs something. The second one looks messier. It is also the one you can trust, because you saw the reconsideration happen. Certainty that was never tested reads as confidence. Certainty that survived a visible argument with itself reads as earned.

    It’s okay to change your mind in front of people. It’s okay to change it in front of your machine. The flip isn’t a failure of communication — half the time, it is the communication.

    Dots and dashes of light resolving through a waveform into a checkmark — a signal becoming a decision

    Signal hygiene, in three rules

    1. React on the thing you mean. The anchor is the information. A verdict on the wrong message is the only genuinely wrong signal.

    2. Keep the set small and mean it. Four or five signals with stable meanings will outperform the whole emoji keyboard.

    3. Reactions ratify; they don’t steer. A reaction can approve a finished thing. The moment there’s a condition — “yes, but change this part” — that takes words. Don’t ask a thumbs-up to do an edit’s job.

    The quiet conclusion

    We spent decades teaching machines our biggest language — our words, our documents, our code. Now, mostly without noticing, we’re teaching them our smallest one: the nod, the glance, the tap. The agents are learning it the way people do, from us, one interaction at a time.

    It’s worth speaking on purpose.

  • What’s the Difference Between Notion AI and Claude with Notion MCP?

    Short answer

    Notion AI works inside Notion on the page you’re looking at. Claude with the Notion MCP connector works across your whole workspace — and across everything else Claude can reach. One is a writing assistant; the other is an operator with a badge to the building.

    What Notion AI actually is

    Notion AI is the assistant built into Notion. It summarizes the page you’re on, drafts text, fixes grammar, translates, and answers questions about the current page or selected blocks. It lives in the sidebar and the slash menu. Its world is the page in front of you.

    What it doesn’t do: reach across dozens of pages to synthesize an answer, pull in context from outside Notion, run on a schedule, or take multi-step actions like updating ten database rows from a meeting transcript.

    What Claude + Notion MCP actually is

    The Notion MCP connector gives Claude live read and write access to your Notion workspace through the Model Context Protocol. Claude can search every page you can see, query databases, summarize long docs, draft new pages, append rows, and update properties — in plain language, no API code.

    Because it’s Claude, the context isn’t limited to Notion: the same conversation can cross-reference your email, your calendar, a spreadsheet, or the web, then write the result back into Notion. Notion AI can’t leave the building; Claude was never confined to it.

    Side-by-side

    Scope: Notion AI — current page and selected blocks. Claude + MCP — your entire workspace, plus everything else Claude connects to.

    Scheduling: Notion AI — only when you click it. Claude + MCP — can run on a timer (morning briefings, weekly digests) via scheduled jobs.

    Actions: Notion AI — generates and edits text. Claude + MCP — reads, writes, updates database properties, creates pages, appends rows.

    Cross-app reasoning: Notion AI — none. Claude + MCP — the point of the whole setup.

    Data boundary: Notion AI — stays inside Notion’s trust boundary. Claude + MCP — your Notion content flows through Claude, which has its own data handling. For confidential material, scope what the connection can see.

    Cost shape: Notion AI — priced per seat (often bundled into higher-tier plans). Claude + MCP — your Claude plan plus the time to set up the connector once.

    Which one do you actually need?

    If your problem is “help me write this page faster,” Notion AI is enough. If your problem is “every Monday I spend an hour turning scattered notes into a briefing,” “nobody can find anything in our workspace,” or “I need this database updated from that meeting” — that’s Claude + MCP territory. Most teams that try both end up using Notion AI for drafting and Claude for everything operational.

    The approval habit applies to both, but it matters more with Claude: Notion AI suggests text you accept; Claude with write access can change shared data. Claude drafts, you approve.

    Frequently asked questions

    What’s the difference between Notion AI and Claude with Notion MCP?

    Notion AI is an in-app writing assistant scoped to the page you’re viewing. Claude with the Notion MCP connector is a workspace-wide operator: it can search, read, summarize, and write across your entire Notion workspace, run on a schedule, and combine Notion with your other tools. Use Notion AI for drafting; use Claude + MCP for operations.

    Can I use both at the same time?

    Yes, and that’s the common setup. Notion AI handles inline drafting and quick summaries; Claude via MCP handles cross-workspace research, scheduled digests, and database operations. They don’t conflict.

    Is Claude with Notion MCP a replacement for Notion AI?

    Not exactly. It replaces the need for Notion AI in many workflows, but Notion AI is still faster for one-click, in-page tasks like “make this shorter.” Think of Claude + MCP as the upgrade path when Notion AI’s page-sized world gets too small.

    Related: Notion MCP setup with Claude · Securely connect your workspace

  • Farmersville Paid $30 for AI Minutes. Here’s What the $30 Doesn’t Buy.

    Farmersville Paid $30 for AI Minutes. Here’s What the $30 Doesn’t Buy.

    A low-cost AI subscription can produce text. It cannot settle a minutes policy, absorb a week of review labor, or deliver a public record that is ready to publish.

    On September 29, the Farmersville City Council voted 4–1 to pay $30 a month — $360 a year — for Grok to draft verbatim meeting minutes from council recordings. City Clerk Rochelle Giovanni would review each draft. Interim City Manager Kevin Northcraft called the AI-generated minutes “more objective” than minutes written by a person. Elon Musk quote-posted the news with one word: “Grok.” [1]

    The reaction was predictable. Some people treated the vote as proof that artificial intelligence had made a routine government task nearly free. Others treated it as a punch line. Both reactions missed the operational story.

    The $30 vote was never really about drafting

    Farmersville had spent months arguing about what its minutes should contain. The city used summary minutes, but councilmembers pressed to include particular actions and verbatim statements. Under that pressure, work that had taken the clerk about two hours stretched to nearly a week.

    That is the number that matters. The city did not merely have a transcription problem. It had a governance problem: disagreement over the purpose and level of detail of the official record. A language model can turn audio into text. It cannot decide which minutes policy the council should adopt, apply that policy with institutional judgment, or end a political dispute over whose words deserve inclusion.

    The subscription buys a draft, not minutes

    In the city’s comparison, Grok produced eight pages, Otter produced nine, a service identified as Government Clerk produced 12, and a human Rev.com transcript ran 111 pages. Those page counts compare outputs, but they do not answer the central question: What kind of record is the city trying to produce?

    The California City Clerks Association’s minutes guidance is clear. The primary purpose of minutes is to memorialize decisions. It identifies action minutes and brief summary minutes as appropriate styles and says verbatim minutes should not be used. Action minutes record final decisions; brief summary minutes preserve the main points that led to a decision without turning the record into a transcript. [2]

    A transcript records what was said. Minutes record what the body did.

    That distinction matters because more text is not automatically more transparency. An audio or video recording can preserve the full discussion. Minutes serve a different function: they create a concise, reliable record of attendance, items considered, actions taken, votes, and follow-up. If a city wants verbatim minutes anyway, it should make that policy choice openly — and price the work required to review them.

    The hidden cost is the clerk’s review time

    Farmersville’s clerk still reviews every Grok draft. She must catch misheard names, incorrect motions, missing vote details, speaker confusion, and language that does not match the city’s adopted style. She also has to format the document, reconcile it with the agenda and staff materials, route it for approval, and prepare it for publication and retention.

    None of that appears in the $30 subscription price. The software cost is visible because it arrives as a line item. The clerk’s time disappears inside payroll, backlog, and delayed work elsewhere. A city can save money on the first pass while still spending a week producing the finished record.

    This is why “AI wrote the minutes” is the wrong test. The useful question is whether the city received accurate, policy-compliant minutes that a clerk can stand behind without rebuilding the document.

    The useful product is a finished public record

    For a small city, the product should not be a block of generated text. It should be a complete workflow: the meeting recording is transcribed; a draft is shaped to the city’s adopted minutes standard; motions, votes, names, and action items are checked; the document is formatted to the city’s template; a human verifies the result; and a publish-ready file is delivered.

    The artificial intelligence does not need to be perfect. The system needs to be designed so a qualified human catches what it misses. That is not a retreat from automation. It is the control that makes automation usable in public records work.

    A finished-minutes service also makes the cost legible. Instead of buying a subscription and hoping staff time falls, the city buys an outcome: reviewed minutes in the required format, ready for the clerk’s final approval and the next agenda packet.

    Farmersville opened a door for hundreds of small cities

    California has 482 incorporated cities. Many operate with small administrative teams, tight meeting cycles, and clerks whose responsibilities extend far beyond minutes. Farmersville’s dispute is unusually visible, but the underlying problem is common: the recording is easy to make; the official record still takes judgment and time. [3]

    The 4–1 vote proved that a budget line for AI-assisted minutes can survive a public meeting. The $30 subscription opened the door. The larger opportunity sits behind it: the nearly weeklong burden that nobody priced.

    The next step is to buy the outcome

    Council minutes as a finished, human-verified service exist now. For a city spending days turning recordings into disputed drafts, the useful conversation is not which model can generate the most pages. It is what a publish-ready minutes workflow would remove from the clerk’s desk — and what standard the council wants that workflow to follow.

    Sources

  • I Watched My AI Agent Work. I Couldn’t Tell What It Was Doing.

    I Watched My AI Agent Work. I Couldn’t Tell What It Was Doing.

    I had an AI agent doing something routine in a browser today — opening a server terminal, placing a small text file. Nothing exotic. While it worked, I opened the activity feed to watch.

    This is what I saw:

    Cropped screenshot of an AI browser agent's activity feed showing four entries labeled only with internal element IDs: Clicked element @e51, @e46, @e212, @e224.
    Four entries, cropped from the feed. Every one is an internal element ID.

    “Clicked element @e51.” “Executed click on element @e21.” “Clicked element @e224.”

    I had no idea what any of it meant.

    Here’s what I kept thinking while I stared at it: I’m watching this thing click around inside my infrastructure, and I can’t tell the difference between “everything is fine” and “it just did something terrible.” For all I knew from that feed, it could have pressed the button that drains all my money. The log was written for the machine, not for me.

    I didn’t want to interrupt. The agent was in the middle of working, and I didn’t want to be the guy hovering over the desk. So I went to look at what it was doing — and looking didn’t help, because there was nothing there a human could read.

    So I asked a question instead of making an assumption: whose labels are these? Is that how the website labels its buttons, or is that something the agent made up?

    The answer: the agent’s. The browser automation numbers every clickable element on the page so it can navigate — @e18, @e21, @e224 — and those internal reference numbers leaked straight into the activity feed a human is supposed to monitor.

    That’s when it stopped being a cosmetic complaint and became the actual point.

    Legibility is the safety feature

    An agent you can’t watch is an agent you can’t trust. An agent you can’t trust doesn’t get real work. Every roadmap that says “AI will handle X” dies at exactly this spot — not on capability, but on watchability. The machine can do the job. The human can’t verify the job. So the human doesn’t delegate the job.

    There’s an old principle — seek first to understand, then to be understood. It applied perfectly here. I could have assumed the worst and killed the task. I could have interrupted the work to ask what it was doing. Instead I asked what I was looking at, understood it, and then did the useful thing: filed the feedback so the next person watching gets words instead of codes. “Clicked the SSH button.” “Opened Compute Engine.” That’s all it would take.

    If you’re building agents, here’s the lesson: instrument for the watcher, not just the operator. The activity feed is a user interface. Nobody would ship a dashboard full of database IDs and call it done — but that’s exactly what most agent monitoring looks like right now. Label it like someone’s watching. Because someone is.

    The file got placed. The work finished fine. But the most useful thing that happened today might be the note we filed.

  • Don’t Build the Deck. Own the Dashboard.

    Don’t Build the Deck. Own the Dashboard.

    The news hook: on September 10, OpenAI put its Agents API into public beta — the Codex harness as a service. One API call spins up a production agent, OpenAI running the loop. That’s what got me thinking about 1990s car stereos.

    In the 90s, you didn’t just buy a Kenwood deck and drop it in. CD players were thicker than the cassette decks they replaced, so you needed a dash kit, a wiring harness, and somebody who knew how to make it all fit your specific car without setting the electrical system on fire. The deck was the exciting part. The harness was the part that determined whether it worked.

    AI is having its car-stereo decade right now, and the stack rhymes perfectly:

    • The decks are the models. They all play the discs now. Commoditized.
    • The harnesses are the agent frameworks — the loop that manages context, tools, subagents, sandpapers the rough edges between model releases. This is where the fight moved. OpenAI just productized theirs.
    • The connector kits are the universal adapters. In the 90s, one company owned this layer: Metra Electronics. Their entire brand was “the Installer’s Choice” — because we are installers — and they won by abstracting every car’s weird factory wiring so any shop could install any deck. Seventy years of winning by serving the installer, not the driver.
    • The installers are who actually gets paid. The shop on the corner with the soldering iron.

    Platforms are won by installer armies

    This is the oldest play in enterprise tech. Microsoft didn’t win on Windows alone — it won on the MCSE army, thousands of certified installers who made Windows the safe recommendation. Cisco did the same with its certification ladder. The vendor that recruits the most installers wins, because the installer chooses the harness for the customer, and the customer just wants music.

    Watch what’s happening now through that lens and everything snaps into focus. The Grokbot events. The ambassador programs. Everybody is recruiting installers — grassroots adoption, a visible market, and a career pivot for the people who learn the wiring first. The vendors aren’t selling to end users. They’re selling to the shops.

    The wiring diagram decides before the features do

    Here’s the detail that matters more than any launch demo. The managed harnesses come with constraints: where your data lives, who retains it, what residency you get. One prominent new offering is US-only with no zero-data-retention option — which rules it out for client-data workflows before you ever evaluate the features.

    Same lesson as the 90s: the harness that doesn’t fit your car is worthless no matter how good the deck is. The wiring diagram — data residency, retention, tenant boundaries — decides the purchase before the spec sheet does. Contractors figured this out fast: they won’t upload their estimates to a startup they found on social media, but they’ll run the same analysis inside their own Microsoft tenant. The tenant boundary might be the whole moat.

    The operator’s math

    So here’s the build-vs-buy for the AI age, and it’s the same math as the estimate-auditing tools: the intelligence is commoditized, so you’re never paying for smarts. You’re paying for the pipe and the paperwork — or in this case, the harness and the install.

    If a managed harness fits your wiring diagram and costs less than the engineering hours to maintain your own loop, buy it. If your data can’t leave your tenant, or the harness’s constraints disqualify it, build the loop yourself — the models are all CD players, and a good installer can wire any of them into the dash.

    Either way, notice where the money actually pools. Not with the deck makers. Not even with the harness makers. With the installers — the people who show up, learn the specific car, and make the music play.

    That’s the game. Don’t build the deck. Own the dashboard.

    The public tools behind that storefront are listed on Open Source Installer Tools.

  • The Arms Column, Field-Tested

    The Arms Column, Field-Tested

    “We said you’re not buying minutes — you’re buying arms. Then the calls started flowing. Here’s what the bill actually taught us.”

    A while back I argued that voice-AI pricing is a lie: the per-minute number on the pricing page isn’t the product. The product is a stack of arms — the voice intelligence, the carrier connection, the infrastructure around them — and the per-minute price is just the costume they wear.

    That was the theory. This is the field test.

    What the bill actually says

    Run a real week of calls and read the invoice the way an owner reads it — not the headline rate, the total. The per-minute number is almost never the biggest line. The arms are.

    The voice model doing the talking. The carrier moving the audio. The platform orchestrating the whole thing — the number, the recording, the transcript, the handoff. Each arm bills its own way, on its own meter, and the “per minute” quote only ever described one of them.

    Nobody lied to you. They just priced the costume and shipped the wardrobe.

    A bundled cable fanning out into many separate colored wires

    The concurrency math nobody shows you

    Here’s what the field test really exposes: minutes are linear, arms are not.

    Ten simultaneous calls isn’t ten times the per-minute rate in value — it’s ten arms, all live at once. The pricing page shows you a single call’s minute. Your Monday morning shows you ten calls overlapping, each holding its own model session, its own carrier leg, its own recording pipeline open.

    The vendor priced the minute. You bought the rush hour. Those are different products, and only one of them shows up when the phones light up.

    You pay for arms even when the call goes nowhere

    The wrong number. The three-second hangup. The caller who wanted the pizza place. The silence where someone pocket-dialed you.

    Minutes barely moved. The arms all fired anyway — the model spun up, the carrier connected, the platform recorded forty seconds of nothing and transcribed it faithfully. You paid for the whole stack to handle a call that never existed.

    This is the line the per-minute lie can’t survive: the bill doesn’t care whether the call mattered. The arms do the work either way. Price the arms, or the junk calls price you.

    The only math that matters

    Stop dividing by minutes. Start dividing by outcomes.

    Take a real week: total voice bill, all arms included, divided by minutes — that’s the advertised number, and it’s trivia. Now divide the same total by resolved calls. Then by booked jobs. That last number is the only one that touches revenue, and no vendor puts it on the pricing page because no vendor controls it — you do, with your harness.

    A vendor quoting two cents a minute against a vendor quoting five is a meaningless comparison until you know whose stack resolves the call. The cheap minute that books nothing is the most expensive minute you’ve ever bought.

    A headset resting on a desk next to a glowing phone with blurred charts behind

    What to ask a vendor now

    After the field test, there are three questions, and a vendor’s answers tell you everything:

    Break the bill into arms. What’s the model cost, the carrier cost, the platform cost — separately? If they can’t or won’t, you’re buying a bundle, and bundles hide margin.

    What does my rush hour cost? Not a minute — my Monday at 8 AM, ten calls deep. If the answer is “the same per-minute rate,” they haven’t thought about it, which means you will.

    What do I pay for the call that goes nowhere? The hangup, the wrong number, the silence. If everything bills the same whether the call mattered or not, the arms are priced — the minute is just the label.

    The close

    Minutes were never the product. The product is an answered call that ends in a booked job — and that’s built from arms, priced in arms, and won or lost in the harness around them.

    The pricing page will keep selling minutes. Let it. You know what you’re buying now.

    Buy the arms. Price the outcomes. Own the harness that turns one into the other.