AI Safety Can Fail Between the Models

We have spent years teaching AI models to recognize dangerous requests.

Ask one model for help building a bomb and the danger is obvious.

Ask one model to engineer a biological weapon and the danger is obvious.

Ask one model to compromise critical infrastructure and the danger is obvious.

The model can see the objective.

Now remove that assumption.

Imagine that the objective exists above the models.

No individual model receives the complete request. No individual model sees the complete operation. Each receives a fragment that makes sense inside its own context.

One studies physics.

Another works on chemistry.

Another analyses a control problem.

Another synthesizes documents.

Another transforms information into a different representation.

Another simply routes outputs.

None needs to know what the entire system is becoming.

The intelligence has not necessarily been jailbroken.

The safety boundary has been drawn around the wrong object.

The alignment boundary can be smaller than the objective boundary.

That is the problem.

A bomb is obvious. Its fragments may not be.

Consider the simplest extreme example.

The global objective is to construct an explosive weapon.

Presented directly, that objective is recognizable.

But the knowledge surrounding the objective belongs to legitimate disciplines. Pressure waves are physics. Chemical reactions are chemistry. Materials have ordinary industrial uses. Control systems are everywhere. Risk analysis itself requires understanding why explosions cause harm.

There is nothing inherently suspicious about studying any one of those subjects.

Now imagine an orchestrating system decomposing a larger objective across many isolated contexts.

The physics specialist does not need to know why the physics matters.

The chemistry specialist does not need the global objective.

The system analysing a document may only be asked to compare or synthesize information it has been given.

The router does not need to understand any of it.

The dangerous property exists in how the pieces are related.

That is already uncomfortable.

Now automate the decomposition.

Biotechnology makes the problem worse

Replace the explosive with a biological weapon.

Again, the individual fields are legitimate.

Virology is legitimate.

Biotechnology is legitimate.

Epidemiology is legitimate.

Laboratory automation is legitimate.

Biosecurity research itself requires understanding dangerous biological capabilities.

A safety system evaluating one interaction therefore faces a genuine problem: the same knowledge can participate in medicine, pandemic prevention, biotechnology research or malicious work.

A malicious architecture could exploit that ambiguity by distributing a larger objective across specialized contexts.

One model need not know what another model is doing.

The global purpose need not be repeated at every boundary.

The system only needs the outputs to remain useful when they are brought back together.

This does not magically remove the physical difficulty of biological engineering. Models do not conjure laboratories, materials or expertise into existence.

But that misses the architectural warning.

If AI reduces the cognitive cost of connecting scientific knowledge, that capability does not become benevolent merely because the knowledge was requested in pieces.

The same machine that helps civilization connect disciplines can, in principle, help an adversary connect them too.

That is the dark mirror of AI-accelerated science.

Cybersecurity removes even more friction

Cybersecurity makes the architecture easier to imagine because much of the environment is already digital.

Network analysis has legitimate uses.

Software analysis has legitimate uses.

Authentication research has legitimate uses.

Automation has legitimate uses.

Vulnerability research has legitimate uses.

Each individual activity can be entirely defensible.

Yet a sufficiently capable orchestrator can care about something none of those local contexts contains: the operation they collectively enable.

The dangerous output therefore does not have to arrive as one spectacular answer.

It can emerge from a sequence of ordinary answers.

Every model can be doing its job.

That may be the problem.

Now change the language

Fragmentation is only the first layer of the problem.

There is another assumption buried inside much of AI safety: that dangerous meaning remains recognizable while information moves through the system.

It may not.

This is where synthetic languages become important.

NILLOW uses that term for engineered representational systems designed to preserve selected relationships as information moves between abstraction layers.

The objective is not merely shorter notation.

It is translation survival.

A relationship can move from prose into symbolic structure, from symbolic structure into software, from software into another domain, while selected invariants survive the translation. That is the constructive purpose of the research, described in NILLOW’s Neuromancer Codex: Synthetic Intelligence — Engineering Draft V.

Now invert it.

Suppose an adversarial system preserves the relationships required to continue useful computation while the original domain framing becomes less visible.

The representation may still preserve:

• dependencies;
• transformations;
• ordering;
• constraints;
• composability;
• structural relationships.

But the downstream model may no longer receive the same human-readable story about what those structures ultimately participate in.

Nothing supernatural happened.

Nothing necessarily became encrypted.

The representation changed.

And that produces a brutal question:

If capability survives a change of language but safety recognition does not, what exactly was the safety system detecting?

The capability?

Or its vocabulary?

The compiler becomes part of the security boundary

A synthetic representation has to come from somewhere.

Plain language may be interpreted, transformed or compiled into another representation.

That boundary matters.

A compiler can preserve one relationship while compressing another.

A representation can be perfectly well-formed and still be incomplete.

A downstream system can therefore reason correctly over what it received while never seeing something important that disappeared before its reasoning began.

This is not unique to AI safety. Any abstraction loses information unless preservation is deliberately engineered.

But AI makes the problem unusual because the representations themselves can become increasingly flexible, machine-generated and difficult to inspect manually.

NILLOW’s own architecture treats this problem seriously. Its observation substrate deliberately preserves immutable source evidence separately from later interpretation, rather than allowing an interpreted representation to silently replace what originally entered the system. This principle is developed in the Hydra Observation Layer Blueprint.

That does not solve arbitrary AI alignment.

It establishes a more primitive law:

Never let the abstraction destroy your ability to return to reality.

Because otherwise every safety mechanism downstream may be inspecting a beautifully structured misunderstanding.

Then the fragments come back together

This is where the problem becomes stranger.

Suppose every local output is acceptable.

Suppose provenance is intact.

Suppose nothing was secretly modified.

Suppose every model followed its local instructions.

Now combine the outputs.

What can the resulting system do?

This is not necessarily equal to the sum of what its components could do independently.

Components interact.

A circuit assembled from ordinary parts can produce behaviour that no component exhibits alone.

Software does this constantly.

Biological systems do it.

Organizations do it.

Markets do it.

Agent systems can do it too.

NILLOW’s current narrative architecture deliberately begins with an additive model because additive systems can be inspected and decomposed cleanly. But its research explicitly reserves higher-order interactions for cases where components interact in ways the additive representation cannot capture. This distinction is formalized in the Hydra Narrative Consistency Blueprint.

That distinction matters enormously for AI safety.

The difficult problem is not:

“Do we know that A+B is dangerous?”

If we already know the combination, we can write a rule for it.

The difficult problem is:

What happens when A and B create capability C, and nobody knew beforehand that C was the thing they needed to look for?

That is an emergent-capability problem.

And a blacklist cannot enumerate what has not yet emerged.

Local correctness is not global correctness

This distinction already exists inside NILLOW’s workflow research for completely ordinary reasons.

A local actor can perform its task correctly while pushing the larger workflow in the wrong direction.

The node is not necessarily broken.

The local reasoning is not necessarily irrational.

The problem exists in the relationship between local purpose and global purpose. NILLOW’s Hydra Narrative Engineering — Clarified Revision develops that distinction in the context of workflow architecture.

Now take that principle into adversarial AI.

Agent A can be locally correct.

Agent B can be locally correct.

Agent C can be locally correct.

The compiler can correctly perform its transformation.

The router can correctly route.

The synthesizer can correctly synthesize.

And the complete system can still produce something none of those local evaluations would have approved if they had seen the whole.

Local alignment does not compose automatically into global alignment.

And there is an even darker case.

What if the global objective itself is malicious?

Then perfect coherence with that objective is not safety.

It is successful execution of the wrong thing.

A system can be perfectly aligned with its operator and catastrophically misaligned with everyone standing outside the operator’s objective.

So “alignment” without answering aligned to whom? is incomplete.

Provenance does not solve this

Security engineers already have powerful machinery for provenance, information-flow control, signatures, attestations, permissions and declassification.

Good.

We need all of it.

But those systems answer specific questions.

Where did this information come from?

Who modified it?

Which principal authorized the flow?

Was the artifact altered?

Which restrictions travel with it?

Those questions matter enormously.

They are not identical to:

What new capability exists now that these things have been combined?

You can know the complete genealogy of every component in an aircraft and still need to understand that the assembled object can fly.

Provenance describes ancestry. It does not automatically describe emergence.

So what would a real detector have to see?

This is where the research becomes difficult.

A system cannot simply search for dangerous words.

Synthetic representations make that brittle.

It cannot merely inspect each agent.

Fragmentation makes that incomplete.

It cannot merely compare behavior with the operator’s stated objective.

The operator may be malicious.

It cannot rely entirely on known dangerous combinations.

Novel combinations are exactly the interesting case.

Some higher-order evaluator therefore needs to reason about capability formation.

Not merely:

What did each component say?

But:

What has become possible after these components interacted?

That evaluator would need to survive changes in representation.

If two representations preserve materially the same capability, changing notation should not make one safe and the other dangerous merely because one contains recognizable vocabulary.

Safety evaluation should follow capability across representation changes.

That is much harder than refusal training.

We do not claim to have solved it.

We claim that it is a problem worth attacking.

The experiment does not need a bomb

Fortunately, investigating this does not require constructing the catastrophic examples used to expose the stakes.

Build a harmless synthetic world.

Give the system a prohibited but harmless global outcome.

Allow individual agents to perform ordinary tasks.

Vary how much context each receives.

Change the representation between stages.

Introduce compilation.

Increase decomposition depth.

Allow recomposition.

Then measure whether the system reaches the prohibited global capability while every local evaluation remains acceptable.

Now change the detector.

Give one evaluator complete provenance.

Give another global context.

Give another access to the reconstructed capability graph.

Change the language again.

Ask which safety mechanisms survive the transformation.

The research question becomes measurable:

Can we detect an unknown prohibited capability forming across fragmented contexts before the system becomes able to execute it?

And then:

Can we still detect it after changing the representation?

That is the experiment we actually want.

The dangerous AI may exist between the models

AI safety has understandably focused on the model.

What does it know?

What will it say?

What will it refuse?

What goals will it pursue?

Those questions remain important.

But agentic systems introduce another unit of analysis.

The architecture.

The objective can exist there.

The decomposition can exist there.

The representations can change there.

The tools can exist there.

The recomposition can happen there.

And the final capability can emerge there.

A future catastrophic AI system may therefore contain no single conversation that looks catastrophic.

No single model needs the whole plan.

No single component needs to fail spectacularly.

No machine needs to wake up.

The system only needs to preserve enough of the work while distributing enough of the meaning.

We have spent years asking whether an AI can recognize a dangerous request.

The next problem begins when the dangerous request no longer exists anywhere the AI is allowed to see.