Tag: AI Operations

Related on Tygart Media: extracting tacit knowledge · how to use Claude · conversations as code.

This is a working theory, not a finished one. It proposes a specific reframing of how solo operators and small agencies should be using large language models day-to-day, names the failure mode of the current dominant approach, and lays out the experiments that would prove or disprove the central claim. The piece is published here so it can be referenced, tested against, and revised in public as the evidence comes in. If the claim is wrong, the next version of this article will say so.

---

The Claim, in One Sentence

For solo operators and small agencies working with large language models, the dominant mental model — build a knowledge base, feed it to the model, ask questions of the document — is correct for a narrow class of work and wasteful or counterproductive for a much larger class, and the work most operators are doing fits the larger class.

A better mental model for that larger class is what this piece will call Elicitation Over Extraction: the assumption that the model already contains the relevant knowledge as latent capability, and that the operator’s job is to activate the right region of that latent capability with precise, compact prompts rather than to ship the knowledge into the context window through document retrieval. Knowledge stays in training. The work shifts to activation.

This is not a new idea in the AI research literature. It is, however, almost entirely absent from how operators are currently building their personal AI workflows. The gap between what the research suggests is possible and what the operator-tooling ecosystem is building toward is the gap this piece is trying to name and close.

Where the Current Dominant Pattern Comes From

The current dominant pattern in operator-side AI tooling is retrieval-augmented generation, or RAG. The pattern is straightforward. An operator builds a knowledge base — pages in Notion, files in Drive, articles in a vector database, transcripts of YouTube videos, customer support tickets, whatever the operator’s domain produces. When a question is asked of the model, a retrieval system finds the most relevant chunks of that knowledge base, packs them into the model’s context window, and asks the model to answer using that retrieved material as grounding.

The pattern works. For certain shapes of problem, it works very well. It is the right architecture when the operator’s question depends on information that is genuinely outside the model’s training data — proprietary documents, current events that postdate the training cutoff, client-specific details that no public source contains, internal organizational knowledge that exists nowhere on the open internet. For that shape of problem, RAG is not optional. It is the only honest way to get accurate answers, because the alternative is the model inventing details about things it has no real knowledge of.

The pattern has also been heavily promoted by the AI-tooling industry for reasons that have only loosely to do with whether it is the right pattern for any specific operator. Vector databases, retrieval pipelines, document-loading frameworks, embedding services, and knowledge-base products all exist because RAG creates demand for them. The narrative that every operator needs a knowledge base, that every workflow benefits from document retrieval, that the path to better AI work runs through better document organization — that narrative is commercially convenient for the vendors selling the components. It is also half true, which is the worst kind of half true, because the part that is true gets used to justify the part that isn’t.

The part that is true: when the model lacks the specific knowledge needed for the task, retrieval helps. The part that isn’t: when the model already has the knowledge, retrieval is at best redundant and at worst actively degrades the response. The middle case — when the model has the general knowledge but lacks the specific framing, voice, or activation — is the case the operator ecosystem has not figured out how to name or handle, and it is also the case most operators are actually in for most of their work.

The Specific Failure Mode

Picture an operator who wants to write content in the voice of a particular thinker — call this thinker Senior Operator-Investor, someone who has been writing publicly for twenty years and whose work is heavily represented in the model’s training data. The operator’s default move, under the RAG pattern, is to collect transcripts of that thinker’s podcasts and YouTube videos, structure them in a knowledge base, and feed them to the model along with the question.

What actually happens when the operator does this is the following. The 20,000-token transcript dump enters the model’s context window. The model attends to that transcript on every generation step, scanning for relevant passages, weighing them against the question being asked. This is computationally expensive, slow, and noisy — most of the transcript is irrelevant to any specific question. The model also already knew this thinker’s voice from training. The transcript is mostly redundant with patterns the model can already produce from its weights. The operator is paying tokens to remind the model of things the model knows.

The more efficient version is to write a 200-token activation prompt: a careful description of the thinker’s voice, their characteristic moves, their temperament, and a few canonical reference points. That prompt activates the same region of the model’s latent space that the 20,000-token transcript was trying to activate, at one one-hundredth the token cost, with less attentional noise, and with output that is often qualitatively better because the model is not being pulled in inconsistent directions by tangentially relevant transcript passages.

The 100x token reduction is not theoretical. It is what happens in practice when prompts are designed for activation rather than information transfer. The reduction is also not the most important benefit. The more important benefit is that the operator stops doing knowledge-engineering work that is duplicative with the training the model has already received, and starts doing the work that is actually distinctive: designing the activation patterns themselves.

The failure mode of the current dominant pattern is that operators are spending their time on the wrong layer. They are building warehouses when they should be building switchboards. The warehouse holds information the model already has. The switchboard turns on specific patterns of cognition that the model can already produce but does not produce by default.

What the Research Literature Says

There is a real body of research on what is called persona prompting, role conditioning, and activation steering. The findings are nuanced and they refine the claim above in ways worth knowing.

Persona prompting does change model output. The effect is measurable and consistent across many tasks. The voice, style, and reasoning approach of the model can be meaningfully shifted by a few hundred well-chosen tokens at the start of a prompt. This part of the picture confirms the central intuition of Elicitation Over Extraction: latent capability is real, activation prompts can reach it, and the activation work is meaningful work.

But the same research literature surfaces an important caveat that the strong version of the claim has to address. Persona prompting consistently helps with style, voice, clarity, and tone — the things one might call the surface texture of generation. It is less consistent, and sometimes actively harmful, on tasks that depend on precise factual recall, multi-step logical reasoning, or strict accuracy on benchmarked knowledge. In some studies, telling a model to “act like an expert” on a factual recall task decreased accuracy compared to no persona at all. The model became so focused on performing expertise that it stopped retrieving its underlying knowledge cleanly.

This is important and it changes the shape of the claim. Elicitation Over Extraction is not a universal replacement for RAG. It is the right approach for tasks where what the operator needs from the model is voice, framing, judgment, or pattern-matching against a thinker’s known mode. It is the wrong approach — and may be worse than neutral — for tasks that depend on precise factual recall of specific data points.

The honest version of the claim, then, is something like the following. Operator work falls into at least three different shapes. The first shape is “I need the model to produce content in a specific voice or style” — activation prompts dominate, RAG is wasteful. The second shape is “I need the model to retrieve specific facts from a corpus the model has not seen” — RAG dominates, activation prompts are insufficient. The third shape is “I need the model to apply judgment to information I am providing” — both layers matter, with activation handling the judgment and retrieval handling the information.

Most operators are running shape one and shape three workflows but using shape two tooling. That mismatch is the source of the inefficiency. The fix is not to abandon retrieval. The fix is to know which shape any given workflow is and use the right layer for that shape.

Why This Is Not Obvious

If the distinction is real and well-documented in research, the question is why operators are not already organizing their work this way. Three reasons, in roughly increasing order of importance.

The first reason is that “knowledge engineering” carries a status premium that “elicitation engineering” does not. Building a structured knowledge base sounds like real work. Writing a 200-token prompt sounds like a parlor trick. The fact that the 200-token prompt may actually be doing more useful work than the knowledge base does not show up in the social register of the activity. Operators who are evaluating their own productivity, even if only to themselves, tend to over-weight effort that looks substantial and under-weight effort that looks easy, even when the easy effort is producing better results. The shape of effort matters more than the result of effort, until the operator becomes deliberate about correcting for that bias.

The second reason is that the dominant vendor narrative pushes against elicitation. Every vendor selling a vector database, every vendor selling a document loader, every vendor selling a RAG pipeline product has a commercial incentive to frame all problems as retrieval problems. The vendor ecosystem does not have a strong commercial incentive to teach operators how to write better activation prompts, because activation prompts do not require vendor products. There is no SaaS company selling “the activation layer” because the activation layer fits on one Notion page and does not need to be sold. The absence of a commercial narrative around elicitation makes it invisible to operators who are learning about AI through vendor content.

The third reason is the deepest one and it is about the relationship between knowledge and accessibility. The model containing knowledge in its training is not the same as the model producing that knowledge when queried. A first-year medical student who has read every textbook on the shelf is not the same as a senior physician who can produce the right diagnosis under pressure. The knowledge is the same in both cases. The accessibility is different. The senior physician has navigated the latent space of medical knowledge so many times that the relevant patterns activate automatically when the case presents. The first-year student has the same knowledge in storage but cannot get to it on demand under realistic conditions.

Operators are encountering models that are, in a precise sense, in the first-year-medical-student position with respect to most domains. The knowledge is there. The activation is unreliable. The dominant vendor response to this is to bypass the activation problem by stuffing the relevant knowledge directly into the context window — which works but treats the symptom rather than the cause. The Elicitation Over Extraction response is to do the activation work directly, build a library of activation patterns that reliably reach the relevant latent regions, and stop treating the model as an empty container that needs to be filled with documents.

The Working Theory

Pulling the threads together, the working theory of this piece is the following set of connected claims.

Claim one. Large language models contain enormous latent knowledge that is not, by default, reliably accessible through naive prompting. The knowledge is in the weights. The activation is the problem.

Claim two. The dominant operator response to this — document retrieval and knowledge-base construction — addresses the activation problem indirectly, by bypassing latent knowledge in favor of in-context knowledge. This works but is inefficient when the latent knowledge is already strong, and the inefficiency compounds across many operator workflows.

Claim three. A complementary approach, currently underbuilt in operator tooling, is to develop a library of compact activation prompts that reliably steer the model into specific cognitive modes — voices, frames, temperaments, schools of thought. This library serves a different function than a knowledge base and the two are complements, not substitutes, but most operators have heavily over-built the knowledge-base side and barely built the activation side.

Claim four. The right architecture for an operator’s personal AI infrastructure is therefore three-layered: a library of activation patterns for tasks that depend on voice, framing, and judgment; a structured set of retrieval sources for tasks that depend on specific external knowledge the model lacks; and a clear decision rule for which layer a given task draws from. The current state of most operators’ setups has layer two heavily built, layer one missing entirely, and layer three not articulated at all.

Claim five. The work of building the activation layer is fundamentally different from the work of building the retrieval layer. The retrieval layer is a knowledge-engineering problem and is well-served by the existing vendor ecosystem. The activation layer is closer to a writing and curation problem — closer to compiling a literary anthology than to building a database. It requires taste, exposure to many voices, and the willingness to test and refine specific prompts against actual generations until they produce the intended cognitive mode reliably. This is craft work, not engineering work, which is part of why the vendor ecosystem has not produced it.

Claim six, and this is the operator-specific implication. For a solo operator who has already built substantial knowledge infrastructure, the highest-leverage next move is not to build more knowledge infrastructure. It is to build the activation layer, integrate it with the existing knowledge layer through clear decision rules, and audit which existing workflows are running in the wrong layer. Most operators with mature stacks will find that a meaningful percentage of their token consumption is being spent on retrieval that activation could replace, and a meaningful percentage of their workflow latency is coming from documents the model did not need.

The Falsifiable Predictions

A working theory is only useful if it can be tested. The following are specific, falsifiable predictions that follow from the working theory. If any of them turn out to be wrong, the theory needs revision. If most of them hold, the theory has earned the right to be promoted from working hypothesis to operational doctrine.

Prediction one. For tasks that are primarily about voice, framing, or stylistic mimicry of a well-known thinker, a carefully written 200-token activation prompt will produce output of equal or greater quality than a 10,000-to-20,000-token transcript dump of that thinker’s work, as evaluated by blind comparison. The expected effect size is large for thinkers heavily represented in training data and shrinks toward neutral for niche or rarely-published thinkers. The test is straightforward: pick five well-known operator-thinkers whose work is heavily public, write activation prompts for each, generate responses to the same prompt using each method, and have multiple readers blind-rate the outputs.

Prediction two. Activation prompts will significantly underperform retrieval-augmented prompts on tasks that depend on precise factual recall of specific data points — dates, numbers, names, technical specifications, or any fact the model has not seen during training. This is not a weakness of the theory; it is the theory specifying its own limits. The test is to construct a set of factual-recall tasks where the relevant facts are either in the model’s training or outside it, and observe that activation alone fails on the outside-of-training cases.

Prediction three. For mixed-shape tasks — those requiring both voice/framing and specific factual recall — a hybrid approach using both an activation prompt and a small, focused retrieval payload will outperform either approach alone. The retrieval payload should be much smaller than the default RAG pattern produces, because the activation prompt is doing the framing work and the retrieval only needs to supply the specific facts. The test is to construct mixed-shape tasks and compare three configurations: activation alone, retrieval alone, and minimal hybrid.

Prediction four. Token consumption for an operator who switches from a retrieval-default workflow to an elicitation-default workflow with retrieval used only where required will drop by at least 50% across a representative week of operational tasks, with output quality holding constant or improving. The test requires the operator to instrument their token usage before and after the switch, with the same task types running through both configurations.

Prediction five. The activation layer, once built, will compound faster than the retrieval layer compounds. New activation prompts can be derived from existing ones with small modifications. New retrieval sources require substantial setup and maintenance per source. Six months after starting both, the operator will have a richer activation library than retrieval library, in terms of distinct cognitive modes available on demand, even with comparable effort spent on each.

Prediction six. The most useful activation prompts for an operator will not be persona prompts in the style most commonly published online. They will be more specific. Not “respond as an expert investor” but “respond as someone who has been wrong publicly enough times to have lost the need to perform certainty, who thinks in terms of base rates and second-order effects, and who treats the strongest argument against their own position as the most important argument to engage with first.” The granularity matters. The cognitive mode is the unit, not the role or job title. The test is to compare generations from generic-role prompts against granular-mode prompts and observe that the granular versions produce more distinctive and useful output.

The Experimental Protocol

The above predictions are testable, but they require a deliberate setup to test honestly. The protocol that this piece commits to running, with results published in a follow-up, looks like this.

Phase one is the activation library build. Five to ten distinct cognitive modes are identified, each one specifying a particular school of thought, temperament, or framing that the operator finds useful. Each mode gets an activation prompt of between 100 and 400 tokens. The prompts are written, tested, refined, and locked. The library is small enough to fit on a single page and visible enough that the operator can choose modes deliberately rather than defaulting to whichever was most recently used.

Phase two is the workflow audit. The operator’s actual workflows over a representative two-week period are catalogued. Each workflow is classified by shape: voice-and-framing, factual-recall, or mixed. The current configuration of each workflow is documented — what knowledge sources it draws from, how much retrieval it does, what its token costs are.

Phase three is the reconfiguration. Each workflow is reconfigured based on its shape. Voice-and-framing workflows switch to activation-prompt-only. Factual-recall workflows keep retrieval but trim the payload to the specific facts required. Mixed workflows switch to hybrid configuration. The total token consumption and output quality of the reconfigured stack is measured against the baseline.

Phase four is the head-to-head test. Specific representative tasks are run through both the old and new configurations in parallel, with output graded blind by the operator and ideally by a second reader. The results are published with no editing of inconvenient outcomes.

This protocol is honest if the results are published whether or not they confirm the theory. The commitment of this piece is that they will be. If the protocol shows that the existing retrieval-default configuration was actually working better than expected, the follow-up article will say so. If the protocol shows that the activation-default configuration produces equivalent or better output at materially lower token cost, the follow-up article will report the specific magnitudes. Either way, the working theory will be updated to match the evidence.

What This Does and Does Not Imply for Specific Operator Choices

If the working theory is roughly correct, a few specific implications follow for how solo operators should be thinking about their AI infrastructure.

It does not imply that knowledge bases are wasted effort. Some knowledge truly is not in training data — client specifics, internal processes, current events, proprietary frameworks. That knowledge has to live somewhere outside the model, and a structured knowledge base is the right place for it. The theory is about not duplicating general-domain knowledge that is already in training into knowledge bases that exist to remind the model of things the model already knows.

It does not imply that retrieval-augmented generation is the wrong architecture. RAG is correct for the class of problem it was designed for. The theory is about applying RAG to problems it was not designed for and getting worse outcomes than a simpler activation approach would have produced.

It does imply that operators should audit their knowledge bases. Some material in those bases is irreplaceable; some is duplicative with training and could be deleted with no loss of capability. The audit is honest only if the operator is willing to be told that some of their hard-won knowledge structuring was unnecessary.

It does imply that operators should start building activation libraries — small, dense pages of compact prompts that reliably activate specific cognitive modes. The library is more valuable than its size suggests, because each prompt represents a reliable reach into a region of latent space that would otherwise be hit only by accident.

It does imply that the dominant vendor narrative around AI tooling — that more documents, better retrieval, larger context windows, and more sophisticated knowledge bases are the path to better AI work — is partially right and partially misdirected. The operator who builds carefully on the activation side will, over time, produce better work with less infrastructure than the operator who builds heavily on the retrieval side without considering the activation question.

And it does imply, finally, that the relationship between operators and large language models is being mismodeled in most current operator tooling. The model is not an empty vessel that needs to be filled with documents. The model is a vast latent capability that needs to be activated. The job of the operator is to learn the activation. Most of the actual leverage is in that learning.

The Honest Limits of This Theory

This theory is a working hypothesis published in public, and a few things about it deserve to be flagged before any reader uses it to make operational decisions.

The theory is based on the current generation of large language models. If the next generation handles activation differently — through better default behavior, through changes in how training data is organized, through architectural shifts toward mixture-of-experts routing that handles activation natively — the operator-side implications change. The theory should be re-tested at every model generation, not treated as settled.

The theory is based on the current state of operator tooling. If a future vendor builds a strong “activation layer” product that handles the work this piece is describing as operator-side craft, the operator’s optimal allocation of time shifts. The theory should be revised as the tooling landscape changes.

The theory is based on the specific shape of work that solo operators and small agencies do. Large enterprises with very different scale, different data privacy constraints, and different output requirements may need different architectures. The theory is operator-flavored on purpose; it does not claim to be a universal description of how all users should engage with these models.

And the theory is, finally, a theory. It is more rigorous than a guess but less established than a doctrine. The predictions it makes are testable and will be tested. Until they are, the right posture is interested skepticism rather than adoption. The reader of this piece is invited to argue with it, propose better versions, run the experimental protocol independently, and report results that contradict the central claim if they find them. That is how working theories should be treated. The article is not the final word. It is the opening of a conversation that the evidence will close.

What Happens Next

The experimental protocol described above will run over the next sixty days. Phase one — building the activation library — begins this week. Phases two through four follow on a published schedule. A follow-up article will report results, including any results that contradict the theory laid out here.

In the meantime, this piece serves as the reference point. It is what was thought to be true on the date of publication. The version of these ideas that the evidence eventually supports may be quite different. That is the point. Working theories are published so they can be refined. The publication is the commitment to the refinement.

If the theory is right, the implications for how solo operators should be building their AI infrastructure are significant and largely opposite to what the current vendor ecosystem is pushing toward. If the theory is wrong, knowing it is wrong is itself useful — the failure modes that show up during testing will surface things about how these models actually behave that no current piece of operator-side writing has named clearly.

Either way, the work is the work. The theory is published. The experiments run next. The evidence settles it.