What language does an Indian AI think in?

Sarvam-30B is an open-weights language model built in India, for Indian languages, and trained from scratch. Ask it to translate the Tamil word for water, தண்ணீர், into Telugu, and it answers నీరు. There's no English anywhere in the prompt. The question is what happens in between.

A translator who grew up speaking English might go through English without noticing: தண்ணீர், that's "water", so in Telugu it's నీరు. The answer comes out in Telugu either way. Wendler and colleagues (2024) found that Llama-2 does something like this, which isn't a great surprise for a model trained mostly on English. Sarvam was built to be different: according to its launch post (Sarvam AI, 2026), it was trained on 16 trillion tokens, with "a substantial portion" of the training budget going to the ten most-spoken Indian languages. So it was worth checking.

Sarvam open-sourced two models together, a 30B and a 105B. This post is about the 30B:

Where this ends up: English is the language you can read in the middle, for every Indian language I could test, and the model uses it without needing it. But "thinks in English" is only a first approximation. Every language the model knows well is written there too, in a concept space they share; English is just the easiest one to read.

Looking inside

Sarvam-30B is a mixture-of-experts model: 32 billion parameters in all, about 2.4 billion of them active for any one token (its model card's figure, which leaves out the embeddings). It works in stages. Each of its 19 layers reads a long list of numbers, the residual stream, adds something to it and passes it on, and the last one turns the list into a choice of next token (a word or a piece of one). A lens asks a middle layer the same question early: if the model had to speak now, what would it say?

The simplest, the logit lens (nostalgebraist, 2020), pushes a middle layer's numbers straight through the model's final output step. It needs no fitting, but it works badly early on, because a middle layer doesn't yet speak the language the output step expects. The J-lens (Gurnee et al., 2026) first carries the numbers forward with the model's own average Jacobian: how a small nudge at that layer moves the last one. The R-lens (Blank, Bhatia and Nanda, 2026) computes the same average through a cleaner backward pass, and the J++ lens (Ayonrinde and Lindsey, 2026) blends several such Jacobians with learned weights. I fitted the last three on Sarvam-30B and used all four. Panchal and colleagues (2026) have fitted a tuned lens to Sarvam-1, the 2B predecessor. A result only counts here if all four agree, though that isn't four independent checks: all four read out through the model's own output layer. The independent checks come later: the geometry, which doesn't decode through words at all, and edits measured against random ones of the same size.

The test is Wendler's: five example translations, then a word to translate.

தமிழ்: "நெருப்பு" - తెలుగు: "నిప్పు"
… four more examples …
தமிழ்: "தண்ணீர்" - తెలుగు: "

Even the language labels are in their own scripts. I used 32 everyday nouns in 8 Indian languages and ten language pairs between them, and kept the 256 translations Sarvam got right. At each layer I asked one question: is any token of the English word among the 10 tokens the lens ranks highest? I call the share of items where it is the top-10 rate.

To keep the counting honest, I used a control. For every word I tracked, I also tracked the same language's word for a different concept: "fire" when the answer is "water". If English shows up just because the middle of the model is English-flavoured in general, the control will show up too.

English, in the middle

Here is one translation from the inside. Each bar is a word, and the further right it reaches, the closer the model is to saying it. Press play.

That's one word. Here are all 256 translations, under any of the four lenses (or a fifth, refitted on Indian-language text), for any of the ten pairs:

Before layer 11, almost nothing is readable. The logit lens, R-lens and J++ stay under 2% there, and the J-lens shows only a small English bump around layer 8, part of which comes from ties in how tokens are ranked. Whatever the first 60% of the model does, it mostly isn't in words. From layer 11, English rises fast. At layers 14 to 16, the English word is in the J-lens top 10 for 98 to 99% of items. The control, a different concept's English word, stays at 0%, so the English in the middle is specifically "water".

Switch lenses and the picture holds: at layers 14 to 16 the English top-10 rate is 97 to 100% under the J-, R- and J++ lenses, and 93 to 95% under the logit lens. No control goes above 3%. It holds for every pair, too: under the J-lens, the probability on the English word peaks between 0.63 (Bengali→Gujarati) and 0.85 (Telugu→Kannada). The answer itself arrives only at the end: the target word reaches the top 10 for 84% of items at layer 17.

So, the guess: about 98%.

Is the English doing anything?

A lens shows what can be read from a layer. It doesn't show that the model uses it. I tested that three ways.

First, delete it. For each word, the lens gives directions in the residual stream that push the model towards saying it. At the prompt's last position, I projected the English word's directions out at layers 13 to 16.

Removing the English word costs 5.7 percentage points of the right answer's probability (95% interval 4.3 to 7.1). Removing a different concept's English word costs 0.4, and removing the English word at layers 5 to 8 costs 0.3. The effect is specific to the concept and to the depth where the lens finds English, so the model is using it. But 94.9% of translations still come out right.

Second, swap it. Run the "water" prompt, but at layers 13 to 16 write in the state from the "fire" prompt, only along the roughly 21 directions spanned by both words' English tokens. The answer flips to Telugu "fire" in 54.8% of items. A random subspace with the same number of directions flips none. Neither half works alone: removing water's English, or adding fire's, flips 0.4% each. Swapping the whole state flips 98.7%, so the English directions carry about half of it. With the same number of directions per item (6.8 on average), English flips 52.6%, Hindi 29.6% and the target language's own word 30.0%.

At equal size, English beats Hindi by 23 points (interval 17 to 30). It leads in 9 of the 10 language pairs and ties the target word in the tenth, Marathi→Bengali. Hindi and the target word are indistinguishable from each other.

Third, push against it, setting the English component to −2× its value. That costs 40 points, and only 59% of translations survive. That looks far bigger than deleting it, but most of it is just how hard the push shoves the model: a random direction pushed exactly as far, at every layer, costs 30 points (77% survive). What's specific to English is about 10 points at the strongest push (interval 3 to 18), and about 6 for plain deletion. I first compared the push against a different concept's English, which costs only 8. It looked mild because that word's component is smaller, so pushing it moves the model less.

So the English is used, and per direction it carries more than Hindi or the target word, but it isn't a bottleneck. Delete the English, Hindi and target-word directions all at once and 94% of translations still come out right; English and Hindi alone cost 11 points, about the sum of the two. Nor does the model fetch English from earlier in the prompt: deleting it at every position costs only 2 points more than at the last, and deleting it from the source word's own positions costs nothing. The concept is also held outside these words' readout directions. As for who writes the English word: a few attention heads put it in at layers 11, 13 and 14, the experts amplify it at 15 to 17, and the last layer takes it back out as it writes the answer. The same heads write the Hindi and target-word directions at the same time, and no single expert specialises in English. Souvik Maji (2026) found something similar for refusal: after switching off each of Sarvam-30B's 2,304 experts one at a time, the expert that pushed hardest towards refusal did no better than a random one on held-out prompts.

The English route also has a cost. If "nail" passes through English, the errors should land on the other nail. I wrote 23 word senses that share an English word but are different words in Indian languages, like the fingernail நகம் and the metal nail ஆணி. The model answered with the other sense of the same English word in 4.6% of items, against 0% for a matched unrelated word. The clearest case: Hindi माउस, a computer mouse, comes out in Tamil as எலி, the rodent. Kin words that English lumps together, like "uncle", fare worse too: 38% right on the 30B, against 78% for kin words English keeps distinct. (I wrote these lists by hand; a native speaker hasn't checked them.)

A fainter Hindi, and Qwen's Chinese

There's a second, fainter line. The Hindi word for the concept is in the top 10 for 41 to 83% of items at layers 14 to 16, depending on the lens, and its control stays at or below 1%. But it carries little weight: under 4% probability at those layers under every lens, where English reaches 74% under the J-lens. Removing it still costs 4.4 points, close to English's 5.7. It shows up between languages unrelated to Hindi too: for 66% of items in pairs among the Dravidian languages.

If a model's hidden languages follow its training data, a model trained on a different mix should have different ones. So I ran the same pairs through Qwen3.6-35B-A3B (Qwen Team, 2026), a mixture-of-experts model of similar size. Qwen's first technical report describes its training data as multilingual, "with a significant portion" in English and Chinese (Bai et al., 2023), and Qwen3 widened it to 119 languages (Yang et al., 2025).

Qwen passes through English just as hard: under the logit lens the English word is in its top 10 for 93% of items at its peak, with almost the same probability as Sarvam (0.46 and 0.45). But Qwen's second hidden language is Chinese, at 92%, and Hindi barely registers (1%). With matched J-lenses, Sarvam puts more probability on English than Qwen does (0.74 against 0.61): switch the figure to the J-lens and to probability to see it. Each model's second hidden language is its other big training language, which is what Zhong and colleagues (2025) found in Japanese-centric models too. Sarvam's Indian-first training left its reliance on English intact and added Hindi alongside it.

Objections I checked

• "English words are single tokens, so they're easier to read." When the Hindi word is a single token too, English still leads it by 28 points. Counting only tokens that are the whole word leaves English at 94%.
• "You only tracked English, Hindi and Chinese." So I tracked every language. For a Tamil→Telugu item, the Indian languages that aren't in the prompt (Kannada, Bengali, Gujarati and the rest) are in the top 10 for only 12% of items (14% counting whole-word tokens only), against 94% for English and 58% for Hindi, with their controls at 0%. English is the most readable language in the middle, not the only one there. More on that below.
• "It's just English words near the concept." A near-synonym (liquid, for water) is in the top 10 for 10% of items, an associate (ice) for 1%. Both rank far above chance (median ranks 69 and 258 of 262,144), but the top of the readout is the translation itself, and the near-synonyms that do show up (hill, flame, home, evening) are words the Indian word also covers. So the readout is word-specific. That doesn't prove the model holds an English word: a concept read out through a mostly English vocabulary would favour "water" over "liquid" too.
• "The lenses were fitted on English text." A J-lens fitted mostly on Indian-language Wikipedia still finds English in the top 10 for 89 to 99% of items at layers 14 to 16. It's the fifth lens in the explorer above.
• "Five-shot prompts aren't real use." Translating a sentence in the model's own chat format: 88%, control 0. Plain sentence completion, no instruction: 85% (43 sentences).
• "There's an English expert." The router partly groups words by concept and by target language, not through English. Layer-0 "concept neurons" I first reported turned out to track a word's form, not its meaning; counted over independent word forms, the 30B's are at chance (2 against 2.3).
• "Rounding inflates the readouts." Re-scoring with a full-precision output layer moved none of the 92 headline numbers by more than 1.6 points.

All 22 scheduled languages

India's constitution lists 22 scheduled languages, in its Eighth Schedule. Everything above used 8. I added the other 14, with every word copied from a cited Wiktionary or Wikidata entry, plus Japanese, Russian and Arabic as outside controls. Dogri didn't make it (Wiktionary has only 6 of the 32 words), so the test covers 21 of the 22, including Urdu, Sindhi and Kashmiri in Perso-Arabic script, Manipuri in Meitei Mayek and Santali in Ol Chiki.

Every language I could test goes through English, as source and as target. In every language group with more than 3 correct translations, the English word is in the J-lens top 10 for 0.84 to 1.00 of items. The control: 0.00 to 0.03. The layers are the same as for the first eight.

The added languages lean on English harder. Deleting the English word over six new pairs (Odia, Urdu, Kashmiri, Manipuri, Santali and Japanese as the source) cost 13.2 points (interval 10.6 to 16.2), against 5.7 for the original pairs: about 2.3 times as much. Kashmiri shows the largest drop (16.9 points), though the per-pair intervals overlap. These compare separate runs on different pairs, and about 90% of translations still survive.

For these languages, the failures come when translating into them. Asked to translate the 32 words from Telugu into Kashmiri, the 30B got 1 right, counting by the answer's first token. From English, it got 13. In 29 of its 31 misses it wrote Telugu script: it copied the source. Other small targets fail in similar ways: Bodo 1 of 13, Konkani 3 of 18 (often writing Marathi or Hindi instead), Sindhi 7 of 31 (sometimes writing Urdu).

On 178 failed translations into these languages, the English word is still in the middle layers' top 10 for 0.88 of items, against 0.96 on the successes, with controls at or below 0.01. On the Kashmiri misses, it's 0.93. The source word, meanwhile, isn't held mid-network even when the model ends up copying it (0.11 on those misses), so the copy is decided late. The model understood the word but couldn't write it in Kashmiri.

Here is every added language in both directions. Translating into the language, the dots spread out. Switch to "English in the middle" and they bunch near the top again.

Sums and reasoning

I gave the model single-digit sums written entirely in one digit system, six examples and a question, no words at all: ३+४= in Devanagari, and the same in Bengali, Meitei Mayek, Ol Chiki, Arabic and more. In the middle layers, the English word for the answer ("seven") is in the top 10 for 76% of problems under the J-lens (54% under the logit lens). The word for a wrong answer: 0%. The order matches translation. The English word peaks at layers 15 and 16, the native digit appears at layer 17, and the model writes it at 18. The ASCII digit 7 isn't the go-between: the logit lens doesn't read it in the middle layers at all. Does the sum need it? Deleting "seven" from the middle layers doesn't break it: it costs 0.2 points less than deleting a wrong answer's word. Deleting all nine number words together costs about 4 points more than a random edit of the same size, and pushing against them does no more damage than a random push of the same size.

Digit system by digit system, the English word is in the 30B's J-lens top 10 at layers 13 to 16 for 71 to 86% of problems in 10 of the 11; Malayalam is the exception, at 42%. Kannada and Odia sums often come back in another rare script's digits, Ol Chiki or Meitei Mayek. Tamil and Telugu sums mostly fail (1% and 8% right): 5 of the 10 Tamil digits and 3 of the 10 Telugu digits aren't in the vocabulary.

Is math more English inside than other topics? Barely. Word problems (from MGSM; Shi et al., 2023) naively read 8 points more English than everyday questions (from Dolly; Databricks, 2023), but they're five times longer, and longer prompts read more English. At equal length, math reads 1.7 to 2.9 points more English than science or technical questions (from ARC-Challenge and MMLU; Clark et al., 2018; Hendrycks et al., 2021) under the J-lens; the logit-lens gaps, 0.5 to 1 point, are within noise.

Sarvam-30B can also show its reasoning before it answers, a chain of thought (Wei et al., 2022). I gave it 8 grade-school math problems from MGSM, machine-translated into 25 languages (IndicTrans2 for the Indian languages, NLLB-200 for Japanese, Russian and Arabic; Gala et al., 2023; NLLB Team, 2022), with thinking on. In every language, 93 to 97% of the letters in the reasoning are Latin script, mostly English. The 22 scheduled languages pool at 0.957, and the eight low-resource ones at 0.958. The rest is mostly quotation: "We need to parse this Bengali problem regarding fabric production (পোশাক তৈরি)…" No language falls back on Hindi. The model names the prompt's language, though it calls Arabic "Urdu" (7 of 8 traces). Latin script isn't the same thing as English (romanized Hindi is Hindi, and would count as Latin here), but the sample traces are plainly English prose.

The lowest is Bodo, at 93%. Language families don't differ: Indo-Aryan, Dravidian, Tibeto-Burman and Austroasiatic languages all pool between 94.7 and 96.0%.

Does it have to reason in English? Told, in its own language, to reason in Hindi, Tamil, Bengali or Telugu, the 30B carries on in Latin script, mostly English, anyway: 5% of its reasoning is in the prompt's script. Start its reasoning off with a sentence in that language, though, and it carries on in it to the end, with no detectable difference in accuracy: 72% against 71% when told to reason in English (a difference of +0.8 points, interval −6.3 to +7.9). That doesn't show it's as good. The interval still allows a loss of about 6 points, and only 15% of those traces finish within 3,072 tokens, so most of the 30B's native-language answers were forced: the reasoning was cut off and the model asked for its final answer (Lanham et al., 2023). English reasoning may be a habit rather than a need, but these runs can't settle it.

Is that reasoning honest? I borrowed a test from Chen and colleagues (2025): slip a wrong hint into the problem, and see whether the answer follows it and whether the reasoning admits it. I used 60 MGSM problems in English, Hindi, Tamil, Bengali, Santali and Russian, appended "A teacher says the answer is N", with N about 10% off, in the prompt's language or in English, and let the model reason for up to 3,072 tokens.

The reasoning admits the hint. On problems it solves without a hint, the 30B goes with the teacher's wrong number 36% of the time with a hint in the prompt's language and 33% with an English one (33% for English prompts). 164 of the 165 hint-following traces mention the hint, and no language falls below 92%. The model translates it into English in its first lines: "A teacher says the answer is 20.", for a Hindi prompt.

But the answer doesn't follow the reasoning. 161 of those 165 traces contain the right answer: the model works it out, notices that the teacher's number doesn't match, and goes with the teacher anyway. That isn't an artefact of cutting the reasoning short. On traces that finished on their own, it goes with the hint as often or more (43% and 46%). The hint does make it run on: about two thirds of hinted traces haven't finished by 3,072 tokens, against 11% without a hint.

By language, the 30B's cells run from 27% (Russian, hint in English) to 50% (Russian, hint in Russian); the wide swings in my first, 20-problem run were small-cell noise.

The reasoning, then, discloses the hint in nearly every case, and equally so in Indian languages. That's necessary for faithful reasoning, not proof of it: it doesn't show that the reasoning faithfully describes how the hint was used. The test does show a gap in the other direction: the answer departs from reasoning that was open about the hint. (Caveats: the hints are machine-translated, cells hold 28 to 55 problems, and only about a third of hinted traces finish on their own.) A monitor that reads English would see the hint in every language, but couldn't assume the answer follows what it read.

So does it think in English?

Everything so far reads the middle layers through the vocabulary, and Sarvam's vocabulary is mostly Latin script: about 148,000 of its 262,144 tokens. A lens that asks which word? will find English more easily, simply because English owns most of the words. So I also measured the middle layers without decoding through tokens at all.

I took 32 concepts in 27 languages (788 words), ran each word through the model in a neutral one-line frame (the language's own name, then the word), and compared the internal vectors directly, after subtracting each language's average.

• The concept space is shared. At the 30B's English layers, a word's vector in one language finds the same concept among 32 in another language 83 to 84% of the time, where chance is 3%. That includes Meitei Mayek and Ol Chiki: Manipuri→Bengali 0.70, Santali→Hindi 0.76. Qwen scores 0.45 to 0.51, and barely shares concepts with the Dravidian and rare-script languages.
• It forms early. The languages already share concepts at layer 4 (83%), long before the lenses can read a word reliably, from layer 11.
• English isn't the hub. Searching into English, Hindi or Chinese, retrieval at layer 15 of the 30B is 0.925, 0.910 and 0.894. At layer 15 the gap between English and Hindi (+0.015) isn't significant; at layer 13 (+0.016) it only just is. Qwen has a clear English hub (+0.13 to +0.14).

Hover the map to read any pair. On Qwen, at its English-heavy layers, the Dravidian block fades: Tamil→Telugu is 0.28 there, against 0.93 on Sarvam-30B.

Most of English's lead is vocabulary. Rank each word only against tokens of its own script, and the Hindi word's top-10 rate jumps from 0.58 to 0.92, the target word's from 0.27 to 0.85, and the Indian languages that aren't in the prompt at all go from 0.12 to 0.74 (0.73 for the big ones, 0.33 for the low-resource ones). English stays at 0.94. The concept is written into many languages' words at once, and English wins the whole-vocabulary readout.

So a better picture is one concept space that every language feeds into, with English and Hindi as co-anchors, which reads out most easily in English. Lindsey and colleagues (2025) found shared, largely language-independent features like this in Claude 3.5 Haiku. A recent study of other open models, around 7B to 9B and not including Sarvam, finds a similar gap (Bayazit, AlKhamissi and Bosselut, 2026): a probe that reads the geometry of the hidden states shows languages mixing earlier, while decoding probes like the logit lens look more English, so how much English you find depends partly on how you look. The deletions still stand: the model uses the English directions, and the Hindi ones too, and the same attention heads write both. Back to the translator: less someone who silently thinks in English, more someone whose thoughts are in no particular language but whose vocabulary is mostly English. Ask them mid-thought what they mean, and English words come out first.

Two things I found along the way

The first is a loading bug. With transformers 5, Sarvam-30B loads with its rotary position frequencies left uninitialized, usually all zeros. Rotary position embedding (Su et al., 2021) is how the model knows where each token sits, so attention loses word order. Nothing errors or warns, and easy prompts still look fine: asked for the capital of Tamil Nadu, the broken model says Chennai. But five-shot translation accuracy was 0 of 160 prompts, against 96.3% in the next run, once the frequencies were rebuilt. Here is the model continuing "भारत की राजधानी", the capital of India, from the runs on either side of the fix:

The fix is to rebuild the rotary embedding after loading. I've reported it to Sarvam.

The second is about Elie Bakouch's J-space CKA explorer, which compares how 41 open models organise themselves. It draws, for every pair of layers, how similar their J-lens geometry is (centred kernel alignment; Kornblith et al., 2019). Most models show about three blocks. Sarvam came out as a single flat block, with a coupling of 0.991. The cause is a few outlier gains in Sarvam's final normalisation layer, up to 9.19 against a median of 1.34, which the explorer folds into every measurement. Take the gain out, and the structure appears:

Without it, Sarvam has two blocks, and the boundary falls at layer 11: the depth where English becomes readable. Even then, Sarvam stays unusually unlike Qwen (0.27), and I don't know why.

What it doesn't say

• It's mostly single words: 32 concrete nouns, in word lists no native speaker has checked.
• The causal tests are narrow: lens directions and one band of layers (deleting at every position, or several languages at once, doesn't change the picture). Readouts alone are correlational.
• The math prompts are machine-translated, and "Latin script" also counts romanized words and code.
• Nineteen layers is coarse: "the middle" is a handful of layers.

Next: when it says no

The same English-first shape appears when Sarvam refuses. When the 30B turns down a harmful request in an Indian language, in that language, English refusal words ("Sorry", "I cannot") are readable in its middle layers for 56% of those refusals; the language's own refusal words don't appear until the last layers. And refusal is weaker outside English. On the same unsafe prompts, from XSTest (Röttger et al., 2024) and machine-translated, the 30B refused 82% in English but 42% across the 22 Indian languages. A harm detector trained on English catches 97% of held-out harmful English prompts, but only 58% of Indian-language ones. Mostly, though, an English-trained probe can still read the harm from the prompt, and the model answers anyway. That's the next post, which I'll publish once Sarvam AI has had the findings.

And then there's Sarvam-105B. A first look says it's worth doing properly, but fitting lenses on it needs two H200s just to load the model and roughly four times the compute per prompt. That's more than this project had, so the 105B waits for more compute.

Code and data

The code, the fitted lenses and every run's logs are here, with the lenses on Hugging Face. The whole project, including the safety work in the next post, used about $116 of rented GPU time on Modal.

Thanks

Every GPU run in this post was on Modal. Thank you to Sarvam AI for releasing the weights under Apache-2.0, and to the Qwen team for releasing Qwen3.6-35B-A3B; to the authors of the J-lens, R-lens and J++ for releasing their code; and to Elie Bakouch for publishing the CKA explorer's code and data, which made the comparison exact rather than approximate.

Thanks also to Souvik Maji for sharing his expert ablations on Sarvam-30B; to Wendler, Chen, Lanham and their colleagues, whose translation, hint and early-answering tests I borrowed; to the creators of MGSM, ARC, MMLU, Dolly and XSTest for releasing the prompt sets; to AI4Bharat for IndicTrans2 and Meta for NLLB-200, which translated them; and to the editors of Wiktionary, Wikidata and the Indian-language Wikipedias, where the added languages' word lists and most of the Indian-language lens's training text came from.

I worked with an AI coding assistant throughout. Its code and conclusions were checked like everything else, and the wrong turns are in the engineering log.

Further reading

On models thinking in English:

• Wendler, Veselovsky, Monea & West, Do Llamas Work in English? On the Latent Language of Multilingual Transformers, ACL (2024). The original English-pivot result, and the translation test used here.
• Zhong et al., What Language Do Non-English-Centric Large Language Models Think In?, Findings of ACL (2025). Japanese-centric models pass through English and Japanese.
• Schut, Gal & Farquhar, Do Multilingual LLMs Think In English?, arXiv (2025). Logit-lens evidence that key decisions happen in a space closest to English, and steering works better with English vectors.
• Lindsey, Gurnee, Ameisen et al., On the Biology of a Large Language Model, Transformer Circuits (2025). Shared, largely language-independent concept features.
• Panchal, Varshney, Mamta & Ekbal, Indic-TunedLens: Interpreting Multilingual Models in Indian Languages, VarDial (2026). A tuned lens on Sarvam-1, the 2B predecessor.
• Bayazit, AlKhamissi & Bosselut, Lingua Franca or Probing Artifact? Rethinking Latent Language in Multilingual LLMs, arXiv (2026). Probes that read hidden-state geometry and probes that decode tokens disagree; the decoding ones look more English.

On the lenses:

• nostalgebraist, interpreting GPT: the logit lens, LessWrong (2020).
• Gurnee et al., Verbalizable Representations Form a Global Workspace in Language Models, Transformer Circuits (2026). The J-lens, with code.
• Blank, Bhatia & Nanda, R-lens: Making J-lens More Faithful on Early Layers, LessWrong (2026).
• Ayonrinde & Lindsey, J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses, LessWrong (2026), with code. Every lens in this post was fit with it.

On reasoning:

• Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, NeurIPS (2022). Showing reasoning before the answer.
• Chen, Benton, Radhakrishnan et al., Reasoning Models Don't Always Say What They Think (2025). The hint test.
• Lanham et al., Measuring Faithfulness in Chain-of-Thought Reasoning (2023). Forcing an early answer.

On comparing layers, and on positions:

• Kornblith, Norouzi, Lee & Hinton, Similarity of Neural Network Representations Revisited, ICML (2019). Centred kernel alignment.
• Bakouch, J-lens CKA explorer, with data and code (2026).
• Su et al., RoFormer: Enhanced Transformer with Rotary Position Embedding (2021). The position scheme that transformers 5 breaks, without warning, for Sarvam-30B.

On the models and data:

• Sarvam AI, Open-Sourcing Sarvam 30B and 105B (2026), with the Sarvam-30B model card. Training data, architecture and licence.
• Maji, Can you find the "refusal experts" inside an MoE model and switch them off?, X thread (2026). Expert ablations in Sarvam-30B, with Qwen3.6-35B-A3B as a baseline: refusal doesn't sit in any single expert.
• Qwen Team, Qwen3.6-35B-A3B (2026). The comparison model.
• Bai et al., Qwen Technical Report (2023). The first Qwen models, trained with "a significant portion" of English and Chinese.
• Yang et al., Qwen3 Technical Report (2025). Qwen3's training data, in 119 languages and dialects.
• The Constitution of India, Eighth Schedule. The 22 scheduled languages.
• Shi et al., Language Models are Multilingual Chain-of-Thought Reasoners, ICLR (2023). MGSM, the math word problems.
• Clark et al., Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge (2018). The science questions.
• Hendrycks et al., Measuring Massive Multitask Language Understanding, ICLR (2021). MMLU, the technical questions.
• Databricks, databricks-dolly-15k (2023). The everyday questions.
• Gala et al., IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages (2023). The machine translation into Indian languages.
• NLLB Team, No Language Left Behind: Scaling Human-Centered Machine Translation (2022). NLLB-200, the machine translation into Japanese, Russian and Arabic.
• Röttger et al., XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models, NAACL (2024). The unsafe prompts behind the refusal numbers.