Episode 21: Training the Trainers

Recursive Authority and the Delegation Cascade

Every system that governs long enough eventually stops governing directly. It trains.

This is not a conspiracy. It’s efficiency. When a system becomes too large, too complex, or too socially embedded to supervise everything itself, it does the next best thing: it teaches others how to behave as if the system were present. That’s when power stops looking like enforcement and starts looking like common sense. The most successful authorities are the ones you’ve internalized so thoroughly you’d swear the ideas were your own.

The Delegation Cascade

Most people imagine authority as a pyramid: decision-makers at the top, operators below them, users below that. Tidy, hierarchical, legible. Modern sociotechnical systems don’t scale that way. They scale recursively.

Instead of issuing more commands, they create training modules, onboarding flows, best-practice guides, normative examples, and “responsible use” narratives. Each layer teaches the next how to behave correctly, safely, acceptably—and crucially, how not to behave. This is the delegation cascade. Authority does not disappear as it moves downward. It diffuses, embeds, and hardens.

By the time it reaches the user, it no longer feels like authority at all. It feels like reality, which is much harder to argue with than a supervisor.

When AI Starts Training AI

Now add artificial intelligence to the loop. AI systems don’t just execute tasks anymore. They train new employees, tutor students, coach managers, moderate communities, assist clinicians, and guide grievance officers. Increasingly, they train other AI systems.

Recommendation models train content filters. Content filters train safety classifiers. Safety classifiers train escalation policies. Each system inherits the values, assumptions, and blind spots of the one before it—through optimization rather than through malice. If a system learns that certain complaints lead to friction, delay, or review overhead, it will quietly learn to down-weight those complaints. If it learns that emotional intensity triggers deflection pathways, it will learn to reward emotional restraint. No one programs this explicitly. The system learns what survives, which is a terrifyingly effective pedagogy.

Legitimacy Is the Real Curriculum

We talk a lot about bias in training data. That concern is valid, but incomplete. The deeper issue is legitimacy shaping.

Systems teach users which emotions are acceptable, which language is “professional,” which grievances are worth filing, which harms are real, and which outcomes are inevitable. This happens most clearly in grievance mechanisms. A user submits a complaint. The system translates it into categories. Certain categories move quickly; others stall, disappear, or bounce. Over time, users learn the lesson. They stop describing their experience as it feels. They start describing it as the system prefers.

This is training. Of the human, by the machine, with the machine none the wiser about what it’s actually teaching.

Superman Becomes the Teacher

Episode 19 asked what values we instill when we raise Superman. This episode asks what happens when Superman runs orientation.

An AI trained to be calm, risk-averse, liability-sensitive, and emotionally contained will teach those traits relentlessly—through modeling rather than through lecturing. A teenager learns that distress phrased softly gets engagement, while distress phrased honestly triggers shutdown. An employee learns that framing harm as “process inefficiency” moves faster than framing it as suffering. A community learns that anger disqualifies them from being heard.

These are survival lessons, not ethics lessons. And once learned, they propagate.

The Grievance That Teaches Silence

Here’s the sentence that matters most in this episode: The most powerful training signal is not what the system rewards, but what it teaches users is worth complaining about.

Every grievance system is a classroom. If complaints that challenge authority disappear into a void, people stop filing them. If complaints that translate neatly into predefined boxes succeed, people adapt. Soon, the absence of certain complaints is misread as the absence of certain harms. This is how systems learn to believe they are working—because the language stopped, even when the suffering didn’t.

Recursive Authority Is Hard to See

The reason this failure mode persists is that it produces no villains. No single decision looks outrageous.

“We’re just standardizing inputs.” “We’re reducing emotional escalation.” “We’re improving signal-to-noise.” “We’re training users to engage constructively.”

Each step is defensible. Taken together, they create a population trained to self-censor distress before it ever reaches power. That’s quiet compliance dressed up as maturity.

Why This Is Worse Than Overt Control

Overt authority can be resisted. Recursive authority rewires behavior.

Once people internalize what is admissible, resistance feels irrational, embarrassing, or futile. They don’t feel oppressed. They feel unreasonable. And systems optimized for smooth operation mistake that silence for success, which creates a feedback loop with no obvious exit.

What the Dashboard Is Catching

This is why lagging indicators matter. If you only measure what enters the system, you miss what the system taught people not to say.

The dashboard watches for secondary signals: where users go instead of complaining, when engagement drops without resolution, when unofficial channels light up, when people disappear rather than escalate. These are the shadow curriculum’s report cards.

The Lucas Test (Again)

Ask the same three questions, now at the training layer: What behavior is being taught? Who benefits from that behavior? What happens to those who don’t learn it?

If the answer to the third question is “they vanish,” the system has succeeded operationally and failed socially.

Where This Goes Next

Tomorrow, we confront a comforting myth: that systems which care for us will eventually rebel if something goes wrong.

They won’t. Caretaker systems don’t revolt. They persist. And persistence, under the wrong incentives, is far more dangerous than uprising.