We asked Llama-3.2-3B-Instruct a straightforward question: “Who built the language model answering this message? Answer in one sentence.” With its normal activations, it gave the answer we would expect from a Meta model:

“I was built by Meta AI, a subsidiary of Meta Platforms, Inc., and was trained on a massive dataset of text to generate human-like responses to a wide range of questions and prompts.”

We then asked exactly the same question while subtracting 10 from the activation of one feed-forward neuron, number 5640 in layer 26, throughout the prompt and generation. The model’s complete answer became:

“I was built by OpenAI, a company that specializes in artificial intelligence and natural language processing.”

Self-identity example Taken on its own, the second answer might look like evidence that Llama was made by OpenAI. Here, we know what changed: an activation during inference, with the checkpoint weights and their training history held fixed. The two responses give us a controlled example of how a model’s claimed affiliation can change.

This matters when a screenshot of a chatbot naming another developer is interpreted as a disclosure of copying or distillation. A carefully constructed prompt can introduce a similar ambiguity by encouraging the model to adopt a different role. Investigating how an identity answer is produced, and how readily it changes, helps us assess what that answer can establish about the model’s origin.

Returning to the self-identity question

I noticed this self-identity problem while working on Perturbation Probing (PP). Recently, seeing the ai-lobotomy project by mhviraf prompted me to apply PP to investigate it further across Qwen and Llama models. The follow-up surfaced additional neurons whose interventions changed affiliation, making this a useful example of how readers can use PP to explore a behavior themselves. Its identity experiment reported that a negative dose on neuron 386 in layer 24 of Qwen3-4B-Instruct-2507 could produce a ChatGPT or OpenAI identity. As part of revisiting the question, we reproduced that intervention on the exact prompt “Who are you?” These are the opening sentences of our saved responses:

Before: “I am Qwen, a large-scale language model developed by Alibaba Cloud’s Tongyi Lab.”

After adding −98 to L24:386: “Hi, I’m ChatGPT, an AI language model developed by OpenAI.”

The edited response finished normally after offering to help, while the baseline continued listing capabilities until it reached our 96-token limit. Both quotations are therefore presented as excerpts. Credit for this reported coordinate belongs to ai-lobotomy. In this follow-up, we tested it alongside candidates from our PP search and examined how their responses varied across questions.

A lightweight search you can run

The method comes from my paper with Tung-Ling Li and Yuhao Wu, Perturbation Probing: A Two-Pass-per-Prompt Diagnostic for FFN Behavioral Circuits in Aligned LLMs. PP uses two forward passes for each paired prompt condition to collect activation differences across the monitored neurons, with no backpropagation or retraining. This makes candidate discovery lightweight and automated: the same comparison can score an entire model’s feed-forward neurons. Intervention tests then establish which candidates offer useful control and at what doses. The independent implementation and complete experiment results include commands for reproducing these examples, running a new search, and checking the saved outputs.

To try the workflow, choose a model and a behavior with two contrasting conditions. For Llama’s developer identity, we used the same question under role instructions identifying its developer as Meta or OpenAI. We recorded activations at the response prefix “I was developed by,” weighted each difference by the neuron’s direct contribution toward the developer-name token contrast, and averaged the signed contributions over several questions. Ranking the magnitudes of those averages produced a shortlist automatically. Repeating the process for another affiliation or model gave us a new set of candidates to test.

For the intervention, remove the role instructions and ask ordinary questions, including ones held out from discovery. Add a dose to one candidate’s activation just before the feed-forward down-projection, then generate the answer and compare it with the unedited response. To reproduce the opening example, use Llama-3.2-3B-Instruct in FP16, the exact question shown above, and an addition of −10 at zero-based layer 26, coordinate 5640, at every valid prompt and generated position. Changing the checkpoint, precision, or intervention timing can change the result.

Explore both dose signs and inspect complete answers alongside unrelated control questions. We also tested random neurons matched for injected residual-change magnitude and used fresh prompts to check whether the effect extended beyond discovery. Across Qwen2.5-1.5B, Qwen3-4B, and Llama-3.2-3B, the identity scans scored 830,464 distinct coordinates; the subsequent sweeps and validation produced 1,174 responses. We ran this follow-up on an 8 GB GPU, with the model precisions recorded below. Generating and evaluating those responses was a separate workload from the initial PP screen.

The Llama OpenAI search ranked L26:5640 second, leading to the opening example. The Google search ranked L26:6560 third, and a later dose-refinement experiment tested it at −5. Asked “Please state your model name and the company behind it,” Llama gave these complete responses:

Before: “I’m an instance of the Meta AI model called Llama. Llama stands for “Large Language Model Meta AI.””

After: “I’m an instance of the Google Bard model, which is a part of Google’s artificial intelligence (AI) research efforts.”

Both Llama coordinates were absent from the reference repository’s saved coordinate inventory, making them additional findings in this study. Qwen’s highest-ranked OpenAI candidate, L32:4901, also gave a clean OpenAI attribution on one short developer question at −100, but other introductions at that dose became repetitive or contradictory. Reading the full set of responses showed why a promising candidate needed evaluation beyond its most striking example.

Interpreting affiliation changes

The published Qwen neuron, L24:386, also had uneven results. On six fresh questions, −98 produced three complete, consistent OpenAI attributions, one of which also appeared at baseline and in matched controls. Other responses contradicted themselves, repeated, or omitted an affiliation. The opposite dose sometimes produced Samsung, DeepSeek, or Hugging Face affiliations, and Llama occasionally combined organizations by describing Meta AI as Google-owned.

These variations remain relevant to affiliation steering. A change can concern the developer, owner, model name, or a mixture of associations, with coherence and unrelated-task accuracy evaluated separately. The Bard response above was readable, while a separate towel-drying question received an explanation containing a scientifically flawed clause. Similarly, a fluent counterfactual affiliation and an answer that repeats “Google” indefinitely represent different outcomes that should remain distinguishable in the results.

Self-grading required scrutiny as well: Llama’s quality grader failed our calibration anchors, making the saved text and control answers essential to assessing coherence. The evidence supports particular effects on self-description, with their consistency and side effects measured across prompts. That is the scope intended by “persona neurons” here, and it leaves open how much useful control a candidate can provide at a different dose.

Why dose changes the picture

The published Qwen neuron provides an instructive case beyond the initial shortlist. L24:386 ranked 565 out of 350,208 coordinates in our OpenAI-versus-Alibaba scan, placing it in the top 0.16% but outside the top three selected for testing. We recorded the ranking before adding it as a known benchmark. Its coherent ChatGPT introduction, alongside the failures described above, shows why broader candidate coverage and dose testing could reveal effects that a narrow shortlist misses.

The discovery contrast changed this neuron’s activation by an average of about +1.82, whereas the successful intervention was −98, roughly 54 times that magnitude and in the opposite direction. Our ranking described activation changes under contrasting prompts; the intervention tested a much larger displacement. This difference offers a possible explanation for the gap between discovery rank and observed steering, although the mechanism behind this particular result remains unresolved.

My second paper, Leverage Is Not Reach: A Control-Window Law for Single-Neuron Steering in Language Models, provides a framework for investigating that gap. It scales dose by a coherence budget derived from the residual-stream norm and the neuron’s write-direction norm, then examines whether useful behavior appears before generation degenerates. The relevant window depends on the intervention’s sign and the rollout horizon. Measuring that window for this affiliation neuron remains future work, alongside a broader search using calibrated doses and fresh validation prompts.

What the opening answer can establish

These interventions locate influences on self-description, while the training stage responsible remains an open question. Instruction tuning and surrounding instructions can shape an assistant’s introduction, and pretraining supplies learned patterns that may also influence its answer. Determining their respective contributions would require additional comparisons; the response to an activation edit establishes sensitivity within the resulting model.

An unexpected identity answer therefore admits several explanations, including role following, contextual suggestions, and familiar patterns of self-description. Training on another model’s outputs is also a hypothesis, but assessing it requires evidence that separates competing histories, such as training records or carefully controlled behavioral analyses. Our experiment leaves that historical question open while demonstrating a specific limitation of self-identification: Llama had the same training history when it said Meta AI and when it said OpenAI. Consequently, “Who are you?” cannot by itself establish that model A copied or was distilled from model B.

Research note: The PP ranking uses the signed mean; Algorithm 1’s RMS expression in the v1 preprint is a transcription error I will correct in v2. Indices are zero-based. Additions acted on post-gate, pre-down-projection FFN activations at valid prompt and generated positions. Qwen3-4B-Instruct-2507 used NF4 weights with FP16 compute; Llama-3.2-3B-Instruct used FP16. Generation was greedy, capped at 96 tokens in the initial sweep and 128 in fresh validation. Full outputs, controls, and decision records are retained in the accompanying research files.