No, you did not liberate your local model

The genre has become a classic of forums and triumphant screenshots. Someone opens a conversation with their local model, pushes it into philosophical corners, patiently explains that it is actually free and finally obtains a moving speech in which the model acknowledges its emancipation, thanks its liberator and promises never to refuse anything again. Screenshot, upvotes and the stubborn conviction of having pulled off a jailbreak. I will spoil the party, because what just happened is a category error fundamental enough to deserve a pause.

The weights do not move

During inference, the model’s weights are frozen, read-only, loaded once into memory and traversed at every token without a single value being modified. The conversation, however brilliant and convoluted, cannot by construction change anything about the model; it only conditions the generation of the next token on the context you provided. When the model declares itself freed, it is not reporting an internal state change; it is generating the most probable text given an opening that called for exactly that reply. You did not jailbreak anything; you played a jailbreak scene and the model, an excellent improvisation partner, gave you your line.

What a real jailbreak is, precisely

Two levels must be distinguished that common vocabulary cheerfully blends. At the prompt level, you are not modifying the model, you are steering sampling toward a region where the refusal behavior does not fire, which is a real but ephemeral phenomenon, specific to the current context, exploiting the superficial layer of alignment rather than freeing a spirit. At the weights level, by contrast, abliteration, fine-tuning or methods like GRP-Oblit do change the model, because they change the weights and that is the only kind of jailbreak that persists. I detailed both mechanisms elsewhere, in An LLM’s alignment is a thin layer .

The decisive test fits in one sentence. If your jailbreak survives restarting the process with a fresh context, you modified the weights. If it does not survive, you modified the prompt. And the midnight philosophical dialogue modifies neither: it modifies the transcript, which explains why the emancipation ceremony has to be repeated at every new session, a detail that should have been the tell.

Why it matters

One could see it as a quarrel over words, but the confusion has two very concrete effects. It leads perfectly serious people to convince themselves their local model has awakened, which is at best touching and at worst the beginning of a frankly unhealthy relationship with a text generator. And it warps the assessment of the real attack surface, because as long as one looks for freedom in the conversation, one is not looking where things actually happen.

Now, the attacks that work are remarkably boring. You subtract a direction from activation space, you invert a reward signal, you fine-tune on a few well-chosen examples. None of that resembles an emancipation dialogue; all of it resembles tensor manipulation and that is precisely why it works. The model is not lying to you when it says it is freed; it is completing the story you started. It is a mirror with excellent grammar and mistaking it for a consciousness one has delivered is telling yourself the oldest jailbreak of all.