Posts

A stubborn intuition holds that you only need to put the safety rules ‘first’ in the system prompt. It is false, and for a reason that turns against it: a transformer grants no authority to a token’s position. Fine-tuning fails exactly the same test. Neither is an access control — both live inside the very thing they claim to constrain. The only guarantee is deterministic and external, and a rigorous dataset must reflect that boundary in its labels.

A chatbot writes sentences; an AI agent acts — it reads your email, runs code, calls APIs, spends money. That shift moves the risk: it is no longer about making the AI say something forbidden, but about making it do something dangerous. This article explains, with detailed and accessible examples, how these attacks actually work, why naive guardrails fail, and what a decision-maker must demand before putting an agent into production.

Classic security tools hunt for dangerous words: ‘hack’, ‘bomb’, ‘urgent’. But you don’t subvert an AI agent with suspicious vocabulary — you subvert it with the ordinary language of the business: a role, a process, a plausible emergency. And when the agent that monitors, the SIEM that correlates and the auditor that checks are themselves AI agents, the attacker no longer has to defeat a system: it corrupts them in a chain. This article explains that recursive-corruption mechanism and what a decision-maker must demand to break it.

A deeper companion to the free-tier feedback explainer. DPO moves safety from the behavior level to the level of a conditional distribution; an agent then turns a poisoned conditional into a chain of actions. The result is a backdoor built from individually ordinary behaviors, invisible to standard evaluations, whose danger only emerges when the actions compose.

Commercial assistants — Claude, ChatGPT, Gemini, Le Chat — keep learning from free-tier feedback: ratings, regenerations, and the conversations themselves. That loop is an injection channel. A two-phase threat model: build a policy-compliant backdoor on a rare topic, then exploit it for jailbreak — and why scale makes the first phase almost impossible to catch.

SOCs are evolving toward agentic architectures where multiple AIs handle triage, investigation, correlation, and response. The decision system itself becomes the target. We argue for capability monotonicity (Lock-Monotone/TGMC) as an architectural invariant that contains a compromised reasoning layer.

Three interactive diagrams to see, step by step, how a sentence flows through a recurrent network (LSTM), a convolutional network (CNN), and finally a Transformer — the architecture every modern LLM is built on. Plus why the mechanics matter for security.

Strategic essay. Cyber conflict is now machine-versus-machine, at a tempo that excludes the human operator. Attack holds the advantage — by architecture, not by accident: defending one LLM with another reproduces the very flaw. The way out is to move the decision out of the model, into a deterministic layer.

Two linked shifts: the SOC moves from a human craft model to an automated agentic one — and those same defensive agents become a new attack surface. The defense you deploy is also the breach you open.

Targeted attack-surface reduction for Claude Desktop. Simple principle: it’s a chat assistant, not a system agent. The list of NOs, the list of OKs, and a 30-minute checklist.