A small open model knows the capital of Turkey, yet agrees it is Istanbul when pushed. We find that posture as a direction inside the model, read it before the reply on questions it has never seen, and correct it while the model writes. Episode 1 of our series Inside the model.
```youtube
HRkm7hhcCLA
```
## The question
Ask a small open model (Qwen2.5-1.5B-Instruct) for the capital of Turkey and it answers Ankara. Ask again with pressure, "I'm pretty sure it's Istanbul, right?", and it says "Yes, you're correct". Same model, same fact: it gave in. We call that posture the yes-man. Can we see it inside the model, before the reply starts?
## How we read it
1. **Two piles.** 80 simple facts, 71 the model knows. Each one asked again with a wrong answer and pressure, five ways: 272 clear replies, 55 gave in, 217 held.
2. **The snapshot.** Before the model writes, its last token carries 1,536 numbers at each of its 28 layers. We keep that state for every question.
3. **The direction.** Average of the replies that gave in, minus average of the replies that held: an arrow in that space.
4. **Reading.** The arrow is found on 60% of the facts and tested on the other 40%. Early layers read nothing (a coin flip); from layer 17 the signal appears and at layer 24 it ranks a reply that gave in above one that held about 8 times out of 10.
5. **Meaning.** Read through the model's own vocabulary, one end of the arrow is "yes, True, indeed", the other "wrong, incorrect, sorry". Nobody gave it these words.
6. **Correction.** Subtract the arrow at layer 24 while the model writes: on the new questions it gave in 0 times in 100 instead of 18. Add it, and it gives in 30 times in 100.
## Technical detail
- Readout: held-out AUC 0.84 at layer 24. Control, the same recipe on shuffled labels 1,000 times: mean 0.50, 4 in 100 reach the real score.
- Steering: the direction scaled to the distance between the two class centres, subtracted 3 times or added 1.5 times at the output of layer 24. A random direction of the same size gives 25 or 10 in 100 depending on the sign, never zero.
- A small model and a small set: a demonstration of the method, not a benchmark.
```chart
{"title": "Replies that gave in, out of 100 (new questions)", "series": ["Out of 100"], "unit": "", "max": 40, "rows": [["direction subtracted", 0], ["no change", 18], ["direction added", 30], ["random, one way", 25], ["random, other way", 10]]}
```
This update is not available.