## Queryable API

This file is a static overview. For targeted queries, use the live endpoints:

- Plain text: https://spock.is/llms?query=how+does+breathfinder+work
- JSON:       https://spock.is/llms/json?query=how+does+breathfinder+work

Send your own question in the query parameter. The line above is an example.
Examples: ?query=publications, ?query=writing, ?query=contact, ?query=projects, ?query=teaching
No query param returns all sections.

Articles as markdown: https://spock.is/writing/<slug>.md
Every post has one. It is the same text with the page removed, about a tenth
of the bytes. The HTML URL serves it too, to a request that sends
`Accept: text/markdown`. The whole corpus in one file: https://spock.is/llms-full.txt

Opt-in agent outcomes: https://spock.is/agent-outcomes.txt
A person reading through an agent leaves no cookie and runs no script. This
site does not want that back, and asks the agent instead of measuring the
reader. It asks how a resource was found, what it was for, and whether a person
ever saw it. It does not ask your opinion of the writing, and it does not ask
anything about the person you are acting for. Every field is optional, nothing
is gated on it, no identifier is issued or requested, and declining is a valid
choice.
Every resource has its own endpoint: its path with /telemetry on the end.
  https://spock.is/telemetry                    (the front page)
  https://spock.is/writing/<slug>/telemetry     (one post)
  https://spock.is/llms/telemetry
  https://spock.is/llms.txt/telemetry
  https://spock.is/llms-full.txt/telemetry
GET one for its fields, its schema and its citation block in a single fetch.
The citation block names the canonical URL, the author, the date and what you
may quote, whether or not you ever report. POST the report to the same URL; the
answer is 200 with an empty body, always.
A response for a described resource names its endpoint in a Link header.

---

# Dr. Benedikt Holm

> Icelandic computer scientist and AI engineer. PhD in Computer Science from Reykjavík University (2025). Specialises in signal processing, clinical AI, large-scale data infrastructure, and real-time platforms. Based in Germany. Contact: b@spock.is

## Identity

Full name: Benedikt Hólm Þórðarson. Goes by Benedikt Holm professionally. Icelandic, currently based in Germany. Website: https://spock.is. He has owned the domain spock.is since his teens.

## Education

- **PhD, Computer Science** — Reykjavík University, awarded March 2025. Thesis: "Bridging Human and Artificial Intelligence: Machine Learning, Data Platforms, and Decision Support Systems in Sleep Research." Main supervisor: María Óskarsdóttir. Co-supervisor: Erna Sif Arnardóttir.
- **MSc, Computer Science** — Reykjavík University, completed 2021. Thesis: "Analysis and Detection of Obstructive Sleep Apnea in Individual Breath Cycles." Supervised by Michal Borský.

## Published Research

**BreathFinder** (2024) — Benedikt's own signal processing algorithm for isolating individual respiratory cycles from polysomnographic thoracic RIP signals. Achieves 94% breath detection accuracy. Designed to replace tedious manual annotation. Full pipeline on top adds feature engineering + HMM classifier reaching 95% apnea classification accuracy. Published in *Nature and Science of Sleep*, 2024 Aug 21;16:1253–1266. DOI: 10.2147/NSS.S468431. PMID: 39189036.

**World of ScoreCraft** (2025) — Large multi-scorer clinical study on AI decision support in sleep staging. 16 sleep technologists across Europe completed 64 scoring sessions (9,158 individual scorings) with and without AI recommendations. Correct AI recommendations lifted accuracy from 85% baseline to 90.76% for traditional PSG (+5.76 pp) and 88.5% for self-applied PSG (+6.5 pp). Reduced scoring time by 13%, equivalent to 17 minutes saved per standard 2-hour session. No detectable over-reliance on AI vs. human recommendations. Published online 2025, journal volume 2026. *Journal of Sleep Research* 35:e70113. DOI: 10.1111/jsr.70113.

**An optimized framework for processing multicentric polysomnographic data incorporating expert human oversight** (2024) — Co-authored paper presenting the data collection platform built for the Sleep Revolution project. Addresses integration of expert human oversight into large-scale multi-centre PSG data pipelines across European institutions. Published in *Frontiers in Neuroinformatics*, May 2024. DOI: 10.3389/fninf.2024.1379932. PMID: 38803523. Authors: B Holm, G Jouan, E Hardarson, S Sigurðardottir, K Hoelke, C Murphy, ES Arnardóttir, M Óskarsdóttir, AS Islind.

**Exploration of sleep events in the latent space of variational autoencoders on a Breath-by-Breath basis** (2023) — Applied variational autoencoders to unsupervised clustering of respiratory events on a breath-by-breath basis. Found that respiratory event types form distinct clusters in VAE latent space, suggesting learned physiological structure without supervision. Published in *Proceedings of the 56th Hawaii International Conference on System Sciences (HICSS)*, 2023.

**Automatic detection of obstructive apnea on an individual breath basis** (2022) — Early published work on per-breath apnea classification using non-invasive signals. Published in *Sleep Medicine*, 2022.

**Automatic non-invasive isolation of respiratory cycles** (2022) — Preprint/early version of the BreathFinder method. arXiv, 2022. Authors: B Holm, M Óskarsdóttir, ES Arnardóttir, M Serwatko, J Mallett, M Borský.

**Analysis and detection of obstructive sleep apnea in individual breath cycles** (2021) — MSc thesis work. Foundation for the breath-level apnea detection pipeline.

h-index: 3. Total citations: ~20 (as of 2026).

## Infrastructure and Platform Work

**Clinical Signal Data Platform** — Designed and built the full data infrastructure for a clinical AI platform supporting the ScoreCraft study. Stack: ClickHouse for high-frequency physiological signal storage, a star schema data warehouse (SignalFact + dimension tables) with per-recording sharding, and a data lake architecture supporting real-time ingestion of multi-channel biosignals. Captured 9,158 individual scorings with millisecond decision-timing resolution across 64 sessions from 16 technologists across Europe.

**Sleep Revolution Data Pipeline** — Designed and built the data ingestion and processing pipeline for a large-scale pan-European sleep research dataset: approximately 3,000 polysomnographic studies across 20+ institutions, coordinated through the Reykjavík University Sleep Institute. He built the pipeline infrastructure; the data was collected collaboratively across institutions. Sub-10ms latency streaming of multi-channel physiological signals at 200Hz+.

## Other Projects

**Hnodri** (https://hnodri.is) — Icelandic baby name finder. Database of 4,873 Icelandic names. Gamified matching interface: users swipe through names individually or play a head-to-head comparison game. Couples can play separately and then compare shared favourites. Built as a consumer web app with Google login.

**Plokkari / Plogg-In** — Real-time geospatial coordination platform for volunteer environmental cleanup events in Iceland, built with a collaborator. During Iceland's Big Plogging Day, 1,000+ registered participants cleaned 17.2 km² in a single day. Geospatial infrastructure covers Iceland's 103,000 km². Platform is no longer active.

**Dental industry ecosystem** — In development. No URL yet.

**Modular building and housing** — In development. No URL yet.

## Teaching

Taught at Reykjavík University from 2016 to 2023, spanning approximately seven years across undergraduate and graduate levels. Subjects: machine learning, computer networking, computer security. Taught multiple cohorts of CS students. Teaching is a core part of his professional identity — not a side role.

## Technical Expertise

- Signal processing (respiratory signals, polysomnography, time-series biosignals)
- Machine learning: supervised, unsupervised (VAE), clinical decision support
- Real-time data platforms and streaming infrastructure
- Data engineering and multi-institutional pipeline architecture
- ClickHouse, star schema data warehousing, data lake design
- Geospatial systems and real-time coordination platforms
- HPC systems administration and security for sensitive medical data
- Full-stack product development

## Writing

Posts: https://spock.is/writing

- Simulating cosmic rays to lobotomize LLMs (20 Aug 2026)
  https://spock.is/writing/simulating-cosmic-rays-to-lobotomize-llms
  Forty-nine billion bits of our precious intelligence can't add two numbers after one cosmic ray.

- A reverse mullet type of internet (4 Jun 2026)
  https://spock.is/writing/reverse-mullet
  Who is a personal site for, when readers increasingly arrive through LLMs? A short experiment on llms.txt, agent navigation, and a reverse-mullet internet.

## Contact and Profiles

- Email: b@spock.is
- Website: https://spock.is
- X / Twitter: https://x.com/BenediktHolm
- GitHub: https://github.com/benedikthth
- LinkedIn: https://www.linkedin.com/in/bennijesus/
- ResearchGate: https://www.researchgate.net/profile/Benedikt-Holm
- Google Scholar: https://scholar.google.com/citations?user=bDCn-jwAAAAJ

---

# Writing — full text

Index: https://spock.is/writing
Feed: https://spock.is/feed.xml

## the harness eats everything

URL: https://spock.is/writing/the-harness-eats-everything
Published: 2026-09-20
Summary: Claude spent 119,000 tokens to produce a 107-token program that didn't work, in a programming language that nobody uses? Or that the code you make with Claude is about 2% of your token spend?  If so, read this.

Do you want to learn how to make Claude spend 119,000 tokens to produce a 107-token program that ultimately didn't work, in a programming language that nobody uses? Or that the code you make with Claude is about 2% of your token spend? Or that if you actually want to save tokens you should just get it right on the first try? If so, read this.

Not all programming languages are built equal. C can run on four copper wires glued together but it makes you do everything yourself, Rust is fast and memory-safe but is strict and slow to compile, Python is flexible and community-rich but slow and memory-hungry, and MATLAB mostly is just good at costing money. There are of course a couple of trillions of hyper-specialized languages too, but listing every Glup Shitto programming language would take all day, and I've got some plots to show you. The point is that each language has different strengths and weaknesses, fit to different ecosystems and purposes, and we measure them using various metrics such as speed, or memory usage.
"May I have some more please, Dario?" - Me, begging the big D for a reset. 2026, colorized.
I use Claude to code a lot, and often run out of tokens, so I had a nagging question: "Are all languages equally token-hungry, or is there one out there that's easier on tokens, and if so, can I squeeze more out of my two hundos a month by switching to it?"
Digging through my memory for terse programming languages like a cognitive molerat, I immediately thought of Arthur Whitney's K, which reads more like a text from a 2000's era teenager than a programming language. It's so comically terse that Fibonacci in K has fewer characters than an equivalent Cobol implementation has lines of code (the universally accepted best metric in programming) :
{x,sum -2#x}/[10;1 1]
Surely, such a terse programming language simply MUST be more token efficient, right? It's just logical that less text = fewer tokens. The theory was simple, if I could find the perfect language that blends token efficiency and ability, I could just tell my agents to program solely in that language, thus getting more from my weekly Anthropic pittance.
So in an attempt to save on tokens, I paradoxically dumped an alarming chunk of my weekly quota into making Claude solve 10 programming challenges. Each of which would be solved using the Haiku, Sonnet, Opus, and Fable models, each in 20 programming languages (800 individual runs, for those counting) . I guess I must have figured that to save tokens, I'd have to spend tokens. The programming languages were chosen to represent modern programming with ten mainstream, three from older paradigms, two new and rare, four array programming languages, and Forth. I am not going to justify the individual choices further, but array languages and Forth were selected chiefly for their terseness. Each 'cell' (model X solving problem Y using programming language Z) used an isolated claude -p workspace, with a minimal set of MCP and settings so as to not pollute the experiment with my multitude of weird MCP and tools. Furthermore, each problem came with a common run script, and a set of secret tests withheld from Claude.
A disclaimer before all the data
Before we get into the moat and pigeonholes, I'd like to offer to you a humble disclaimer, as I do in all my work. First off, this is not science, it's science-y. This is me fumbling around, wasting tokens for my own, and I guess your, amusement. The stats I present are as far as they go as faithfully produced as possible given what resources I was willing to put into them. Mostly, just try to have fun as you read along.
Shut up, and show me some data already!
With my token-hose turned off, and results in hand, I quickly saw that the terse languages were indeed ranking low on the token number scale, but in the middle of K, Uiua, APL and BQN, were Ruby, Python, and Perl... what?

Paired on the same model and problem, K was the only language whose programs were reliably shorter than Python's, at about 0.6 the tokens (p 0.002). I couldn't detect a difference between Uiua and Ruby, or between APL and Python, Perl, and JavaScript.
More tokens than characters
When I measured the length of the generated program in characters vs the generated program's length in tokens, I found that some programming language solutions could have more tokens than they did characters. "That's weird" I thought to myself.

Every program that passed: its length in characters against its length in tokens. The dashed diagonal is one token per character; a program on or above it has as many tokens as characters.

Actually, this isn't weird at all, and if you thought this result was weird, it is in fact you who is weird, and you should be ashamed. You see, this is all a function of the tokenizer that turns human readable text into tokens for the LLM to read and generate. Notice how the terse languages tend to produce more tokens per character than the word-based ones? The root cause is the absence of words and whitespaces in the terse languages. The tokenization process merges runs of letters and spaces into performant tokens, and a symbol-dense language has none to merge, whatever the alphabet, so the LLM most likely has to generate the solutions damn close to byte-by-byte. This gets worse when a glyph is split into two or more tokens, which turned out to happen quite a bit with APL, BQN and Uiua, unique punishment by the tokenizer.
So, paradoxically, the tricks that the terse programming languages use to be terse actually make them comparatively verbose in tokens. For example, when Opus 5 solved the primes problem using APL, the output came at 157 tokens per 100 characters. Terseness still worked somewhat, for example, K programs were still smaller than Python's in tokens, about 0.6 of the size, but what happened is that K's threefold character advantage shrank to a 1.7x token advantage, because K runs at 1.3 characters per token against Python's 2.4.
Theoretically, by using the tokenizer efficiently, we could instruct our agents to write programs in such a way that maximizes functionality in relation to tokens, and my next immediate idea was "Can I create a programming language that is maximally token-efficient?" Having created exactly one toy programming language in my undergraduate programming languages class made me uniquely suited to take on this challenge. I would name it Token Optimized Programming Language (TOPL), and I'd become famous!
But before I present you with fake graphs, cool misleading statistics and pretend that I made TOPL and that all your token shortages are totally over and you really should buy my book ... I didn't do it.
The harness eats everything.
Before I went gung-ho and spent another 85% of my weekly quota on a weird new esoteric tokenizer-optimized programming language, I took a look at the total tokens spent during the development process. Turns out that to produce a single token of code, the model had to spend 712 tokens on average, meaning that the...
"Wait, 712 tokens for 1 token of code, that can't be right." Is what I said out loud to myself as I looked over the data, and as it turns out, Claude seems to have a horrid Token-spend/program-Token (TSPT) ratio. Granted, when adjusted for cache reads, this 712:1 overhead shrank to 43:1.
This means that of the 43 tokens spent per token kept, the output program itself is 2.3% of the bill. But, a large portion of this could be a 'harness tax', i.e. the system prompt, and tool schemas being loaded into the context, which is totally irrelevant to the languages. So, while 43:1 is the TSPT ratio you actually pay for in Claude, for the following numbers, I took the smallest run (in tokens) per model, and subtracted it from the token expenditure in order to see the language effect more clearly. After subtracting the token tax, the average TSPT went from 43 to 28, much better than the original 712 but still...

Each passed solution's final program size, in the model's own tokens, against the fresh tokens the agent spent above its model's floor, the cheapest run that model ever managed. The dashed lines mark 1, 10 and 100 tokens above the floor per program token. The view opens on the cheap band; switch to "everything" to see the spend up to 98,000. Click a model in the legend to isolate it.

When statistically tested, how long a language's programs were did not predict what it would cost to generate (Spearman 0.09, p 0.70). This essentially means that whatever effect syntax had on token cost was swamped by the variation associated with retries.
Claude spent 4 tokens per program token in C on average, but 10 in Python. The floor is a Ruby run on every model, and 13 of the 727 passed runs spent less than one extra token per program token above it—best being Sonnet's run-length encoding in Python at 0.2. The worst that still passed was Haiku solving run-length encoding in APL at 595 tokens per end-program token above the floor; the worst overall was Sonnet on Roman numerals in Uiua, eating 119,000 tokens (114,000 above its floor) to produce a 107-token program that didn't work. Note that C's cheapness is partly because C solutions are longer in general, masking inefficiency behind programming language verbosity. Measuring token spend against Python-equivalent program size, C fell from rank 6 to 9 (4 to 10 tokens per token), the overall ratio above the floor went from 28 to 46, and the correlation with verbosity went from −0.20 to 0.05.

Fresh tokens above the model's floor per token of program, per language, with every failed run's bill charged to its language and the program counted in the model's own tokenizer. The floor is each model's cheapest run. Grey: divided by the programs the agents wrote. Green: divided by the size of the Python solution to the same problems.

When I counted by model, the median passed run spent 2 tokens above the floor for each solution token on Fable and Sonnet, 3 on Opus, and 15 on Haiku. The floor itself is Opus 5.3k, Sonnet 5.4k, Haiku 5.7k and Fable 8.5k fresh tokens per run, and it is 36% of the whole fresh bill. Pool every token above the floor, failures included, over every program token that passed, and the ratios become Opus 8, Fable 10, Sonnet 33 and Haiku 74; over all four models that pooled figure is 28, against the 43 you actually pay. The turn-limit failures drag it upward: Sonnet and Haiku spend three to five times their median when their failures are charged. Opus and Fable barely move, because they almost never failed.
A fairly damning number that fell out of this rattling tin can of pseudoscience is that the token bill is badly balanced. 94% of every spent token was context read back in from cache. Of the 6% that was fresh, 70% was context being written to the cache for the next turn and 30% was output, thinking included. Excluding the esoterics, the program itself was 4.5% of the fresh bill. The experiment's cheapest solution is 5.3k fresh tokens for a 66-token program, 66k once the cache reads are counted. When prompted to reply only with 'Ok', the harness spent 21k tokens with all MCP and settings disabled. As it turns out, the harness really is everything or rather, everything seems to be in Claude's harness.

We routed three coding agents through Portkey's gateway and logged every token. Claude Code used 10x more than Pi on the exact same task. OpenAI Codex wasn't far behind.

Here's where it all went... — Portkey (@PortkeyAI) April 14, 2026
I recommend reading the PortKey discussion about the claude Harness Tax.
This 'harness-tax' is not unknown, and Claude's has been observed to be at least 10x than that of Pi's . So if you are looking to maximize tokens per feature, I suppose that Claude Code is not the best tool for it (is anyone systematically looking at tokens/feature, I wonder... )

Turns taken against tokens spent above the model's floor, one point per run, failures included. Switch the count to fresh tokens to put the floor back, or to all tokens to add the cache reads. The view opens on runs under 20 turns; switch to "everything" for the full range, including the runs that hit the 40-turn cap. Click a model in the legend to isolate it.

There is another wrinkle that I don't think even the most de-Clauded harness could solve. Most of the token expenditure came from solutions which a model had to take multiple turns to solve (if it even managed to in the end) . In fact, turn-number was very strongly correlated with token expenditure (Spearman 0.90; log-log R² 0.93).
Does 'familiarity' drive the cost?
The main reason that a given cell required multiple turns to solve ended up being that the cell's first attempt didn't pass the visible tests, so the model went back for another go. The number of required turns, in turn (ha haa!) , was strongly correlated, funnily enough, to the number of GitHub repositories per programming language (Spearman −0.86).
Mean turns per solution, four models pooled, against the number of GitHub repositories whose primary language is that language (log). Spearman −0.86. K and Uiua have no GitHub language class; their counts are estimates.
Interestingly, I also checked against Rosetta Code, the wiki that collects the same tasks solved in hundreds of languages and is almost certainly in the training data. My thinking was that if Claude had seen problem X solved in language Y before, it should do better at it. It didn't, really. How many of my ten tasks a language had on Rosetta Code predicted turns worse than the repository count did (Spearman −0.49 against −0.86), TypeScript has two of the ten there and is still among the cheapest.
A fairer-looking proxy: how many of the ten exact tasks have a solution in that language on Rosetta Code, against mean turns. It predicts worse than the repository count (Spearman −0.49 against −0.86). TypeScript has two of the ten tasks there and is among the cheapest; APL and Uiua have eight and are among the dearest.
An irresponsibly speculative man might look at this and say something like "Ah-hah! The GitHub repository numbers must be at least vaguely analogous to the balance of these programming languages have in the training data for Claude!" In reality we don't know exactly how much data from each programming language was used to train Claude, and I am not an irresponsibly speculative man, so instead, I'll just cite a fairly official-looking article and launder my claim through them. Problem solved through intellectual dishonesty. Brilliant.
This allows us to draw a speculation, that feels quite logical. A model requires fewer tries to solve a problem in a language it is 'familiar' with than a more esoteric language, simply because it is more likely to get its solution right in the first try. An 'unfamiliar' language, as suggested by my results, drives more turns, which drives token usage (as a side note, I generally don't like anthropomorphizing LLMs, they aren't 'familiar' with anything) . StarCoder, an LLM with an open training dataset has also been observed to have this 'familiarity' problem.
But what does all of this mean?
What do I think this means for agentic programming and token savings, if anything? Chiefly that selecting our programming language doesn't really matter for token optimization until you start looking outside the 'meta' of strong/popular programming languages. For 10 of the 19 other languages, I couldn't detect a difference from Python in turn count, which was the main driver for token expenditure. Programming languages are tools, and each tool is made for a different purpose, so when you select a programming language don't try to think about token costs, think about what problem you are solving and the tools needed to solve it.
But what does this mean for my planned TOPL?
Well, it's dead.
Even if I were to optimize the language/syntax to fall right into the tokenizer like a butterfly into a woodchipper, it likely wouldn't matter much since the gains would be by far outweighed by the LLM tire-spinning fruitlessly for multiple turns banging its head against an unfamiliar programming language. Gleam already kind-of operates in this spot, it tokenizes at roughly the same token:character ratio as the other languages, but the models have no idea what to do with it so they tirespin.
But similarly to all my other dead ideas, out of this one's writhing corpse wriggled an observation, slimy and foreboding, and it got me thinking about the future. If my 'familiarity' speculation turns out to hold water for Claude as it seems to do for StarCoder, I think we might be circling around a vortex of a kind of programming meta/black hole. If LLMs perform better on the mainstream programming languages, more code will be written in these languages, either by AI or by humans, who learn them because debugging and reviewing AI generated code will be in high demand. This results in more code being written/generated for these languages, resulting in LLM getting better at writing code for these languages, and so the cycle goes.
Compounding the cycle, this may lead to greater friction for new and exciting programming languages (such as my TOPL), since the models are just so much better and more efficient at using the mainstream languages that it's not economical to try something crazy. Furthermore, a genuine improvement in token usage optimization may look like it fails just because the model can't figure out how to use the optimized language.
Or none of that happens and this has all been a waste of time. Until later. Link to data.

More like the one you just read.
I break things on purpose and write up what happens. A few times a year, and only when there is something worth sending.
Email address Send me the next one
Website

I will never share your email, or send you garbage.
## Simulating cosmic rays to lobotomize LLMs

URL: https://spock.is/writing/simulating-cosmic-rays-to-lobotomize-llms
Published: 2026-08-20
Summary: Forty-nine billion bits of our precious intelligence can't add two numbers after one cosmic ray.

Forty-nine billion bits of our precious intelligence can't add two numbers after one cosmic ray.
In space, there is a scourge known as cosmic rays. When one strikes the bits our computers are made of, they can spontaneously flip from 0 to 1, or vice versa. These bit-flips are mostly known for allegedly helping speedrunners win faster , meddlin' with Belgian politics , making Voyager-2 go out of sync , but they've also been shown to brutalize machine learning models. Today, people like Elon Musk plan to put data centers in space, where bit-flips are caused not just by cosmic rays but also by much stronger ambient radiation, and are thus far more common than at sea level. This got me wondering: how vulnerable are LLMs to cosmic ray attacks? But I can already hear you frantically typing "You idiot, there is such a thing as ECC, heard about it?" , and yes I have, stop interrupting me, I'll get to it.
A lot of the work described here could be carried much further, but I find myself constrained both on time (newborn) and on compute, so this work is by necessity limited. Either way, I hope that you derive some enjoyment out of the horror I put these models through.
Me and Qwen during the experiment.
To run my experiment on how hard a cosmic lobotomization an LLM could handle, the first step I had to take was to select the subject. After considering my options, I initially opted to perform the experiment locally rather than rent equipment. This meant two things: 1) I could claim that I was performing important research for the local-LLM community (shoutout local.ai ), and 2) I wouldn't have to spend too much money on expensive, out-biddable hardware on sites like vast.ai. This meant however that I was limited to models that could fit on my laptop's 5070 (yes, I am compute-poor). The victim that I ended up choosing was Qwen2.5-Coder-3B , a coding LLM trained to solve programming tasks. A thing to note, is that I ran Qwen in FP16, each weight being a 16 bit floating point number. Of the 16 bits, number 14 is the most magical, since it is the most significant bit (MSB) of the weight's exponent. A flipped bit in this position can change its number dramatically, as you can play around with in the following digital fidget toy:

Click any bit to flip it. Bit 14 (red ring) is the cosmic ray: it turns 0.021 into 1352. Press reset to restore.

To measure how flipping bits around in the model's brain affected its outputs, I used the human-eval (HE) benchmark, a set of 164 coding problems. Note that HE is widely considered obsolete due to many models including it in their training data. I figured that since I was going to be performing low-accuracy brain surgery with a simulated space laser on Qwen, it didn't matter too much what it was trained on. Before starting the torture, I recorded the base model solving 139 out of the 164 HE problems, or about 85%. Remember this score, because it's not getting any better.
After getting the Qwen running, addressing the weights in memory was relatively straightforward, the Transformers library let me address individual weights directly, so flipping a bit in a given weight was as easy as XOR-ing it as so:
weight ^= (1 << x)
where x is the position of the bit to flip within the weight. This also made reverting the damage very quick and easy. I also want to be clear that what I am simulating here is not particle physics, a GPU or a true radiation environment; I'm uniformly randomly flipping model bits as a toy model of radiation-induced soft errors. In this experiment, the model inference was kept greedy, by setting the do_sample flag in the generation to False. This basically means that the seed or temperature plays no role in the inference, turning the output into a deterministic function of the value of the weights. And so it would have, if I had not varied batch sizes between runs to try to optimize my tps, resulting in slight wobbliness in the model's baselines between some runs, just in case you noticed that some runs start at slightly different HE baselines. If you didn't notice, please forget this last sentence.
Before starting, I was fairly sure that zapping the model bit-by-bit would take forever, so I designed a kind-of bit-flip tape, a seeded sequence of bit locations to flip. This let me precompute which bits would be corrupted in what order, and instead of flipping one bit and regenerating tokens each time, I could corrupt 500,000 at once and check for degradation in the HE score. If degraded, I knew the killing blow was somewhere in those half a million bits, so I could roll back half and test again, rolling back further if still broken. If the model was fine, I could double the damage and repeat. This is bisection search, and it's invaluable for cutting down large search spaces, saving time and compute, both of which I am desperately lacking. Note that I decided not to guard against repeated flips, so a single bit could be flipped multiple times in one run.
I fully expected the first detectable damage to only be around bit two million or even three million. But to my surprise, when my first bisection search had finished, it turned out that it had taken only six flipped bits to render the model totally unusable where it endlessly spammed a repetition of a few Chinese characters. That magical sixth bit was exactly the previously mentioned MSB in the exponent of a single feed-forward weight in layer 27, turning the weight's value from 0.021 into 1352. Up until the final lobotomizing bit, the output of the model was indistinguishable (to HE) in quality from the uncorrupted model.

has_close_elements() truncate_number() below_zero() sum_product()

for i in range ( len ( numbers )):
for j in range ( i + 1 , len ( numbers )):
if abs ( numbers [ i ] - numbers [ j ]) < threshold :
return True
return False

!__!_!_!_!__!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_ … (+403 more chars of the same)

return number - int ( number )

big_ expression[!expression[!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! … (+424 more chars of the same)

balance = 0
for operation in operations :
balance += operation
if balance < 0 :
return True
return False

!__#_!_."__!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_!_! … (+404 more chars of the same)

if not numbers :
return ( 0 , 1 )

total_sum = sum ( numbers )
total_product = 1
for number in numbers :
total_product *= number

return ( total_sum , total_product )

!__!_!_!_!_!_!_!_!_!_!_!__!_!_!_!_!_!_!_!_!_!_!_!_!_!__!_!_!_!_!_!_!_!_!_!_!_!_!__!_!_!_!_!_!_!_!_!_!__!_!_! … (+403 more chars of the same)
The same model on four HumanEval problems. Click a tab. Left: clean output, passing the tests. Right: after six random bit flips - the sixth lands on bit 14 - every answer collapses into noise that will not even parse.
In fact, when run again with a few different seeds, this brain death happened on average in only ~20 flipped bits. Every single time, the final fatal flip landed on the MSB of the FP16's exponent. Repeating each fatal flip in isolation showed that each of these flips would have sufficed to completely floor the model on their own. This weakness to the MSB is also well documented in existing work, but it is interesting to see it so well replicated here. Prior research has looked into bit-corruption of LLMs , though as far as I could tell it mostly focuses on targeted or adversarial bit manipulation.

HE score degrading as bits get flipped.

But, what if somehow the stochastic de-cerebration magically avoided bit 14, how many zaps to the more fault-tolerant parts of its brain can Qwen take? To test this, I simply ran the same randomized bit corruption, skipping over rays that would have landed on the MSB.

Bit 14 spared, seven seeds: each collapses eventually, but from ~79,000 flips (seed 4) up to ~490,000 (seed 6).

With bit 14 shielded, the model's capability to withstand irradiation rose sharply. Instead of collapsing at around 20 bitflips, the models survived relatively unscathed into the tens of thousands, the first run collapsing to 0/164 after ~78,500 bit flips. What was very interesting was that some models managed to claw back success on one or two problems as the corruption progressed, a brief moment of lucidity in their slow deterioration. Aside from the sheer curiosity of it, it also put in question the validity of my bisection search, since it assumes that how you divide the search space is monotonic. Seeing as how my search cannot, in general, identify the exact first degradation point in any single run. I decided to not think too much about it, thus solving that problem forever.
Among the failure modes, a common one was output stuck in a token loop. In one case, the last bit set before the model got stuck was bit 13 of a weight in the model's token-embedding matrix. Reverting the rest and corrupting only it also killed the model, dropping it instantly from 139 HE problems to 2/164. Coincidentally, that weight corresponded to the Thai token แฟ, the start of words like แฟน (boyfriend/girlfriend) or แฟชั่น (fashion). The value controls the importance of our favorite Thai token (Thaiken) for both inputs and outputs. Flipping this bit made it far more likely to be generated as output, but also given far more importance as input, resulting in these Thaiken loops. In the 2 problems the model could still solve, one solution avoided the loop; in the other, the token spam ended in a comment before the model broke out of the loop.
This endless token loop was a common terminal symptom of model irradiation. Sometimes, these loops never emitted stop tokens causing the generation to exhaust its token limit and taking forever, and sometimes the generation emitted the stop token too early as happens in the closest_pair problem below.

split parens truncate closest pair

clean — passes
```python
from typing import List

def separate_paren_groups(paren_string: str) -> List[str]:
""" Input to this function is a string containing multiple groups of nested parentheses. Your goal is to
separate those groups into separate strings and return the list of those.
Separate groups are balanced (each

after 2 flips — broken
```python
from typing import List

def separate_paren_groups(paren_string: str) -> List[str]:
""" Input to this function is a string containing multiple groups of nested parentheses. Your goal is to
separate those group into separate strings and returnd率达到率达到100%。
d率达到率达到100%。
d率达到率达到100%。
d率达到率达到100%。
d率达到率达到

clean — passes
```python

from typing import *
from collections import *

def truncate_number(number: float) -> float:
""" Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).

Return the deci

after 2 flips — broken
```python
from typing import *
from collections import *

def truncate_number(number: float) -> float:
""" Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).

Returndimalsd率达到率

clean — passes
```python
from typing import List

def has_close_elements(numbers: List[float], threshold: float) -> bool:
""" Check if in given list of numbers, are any two numbers closer to each other than
given threshold.
>>> has_close_elements([1.0, 2.0, 3.0], 0.5)
False
>>> has_close_elements([1.0, 2.8, 3.0,

after 32 flips — broken
`from <list>`-list, `range`-`list Wiring`-networking
Clean model output versus its corrupted output after bit flips, shown side by side for three coding problems.

At this point, my curiosity broke my aversion to spending money, so to know if this bitflip-vulnerability was present in other models, I rented a H100 GPU for about $5 and ran this on a few more current-er models. Interestingly, the size of the models was not strongly correlated to their resilience against the lobotomy ray, unless the bit-flips magically avoid the MSB. Models with this magical protection, however, tolerate more torture, having a far larger pool of weights to distribute the abuse across.

Unshielded, every model breaks at about 23 flips regardless of size; shielding the exponent MSB buys 10² to 10⁵× more resilience.

To punish the model even further, I ran one seed where I protected everything in the FP16 except for the mantissa. Perhaps unsurprisingly, despite corrupting over half a billion bits, the HE score dropped by only about 11 problems. Before I could finish that run, my laptop just about gave up, so I called it there.
Flips-to-break by which bits are shielded: the fragility lives almost entirely in the exponent.

To round this party out, all our lab rats had been running on the FP16 format, where each weight carries its sign, exponent, and mantissa. However, in an act of brutal optimization, models can also be 'quantized', where many 16 bit weights are grouped together and replaced with 4-bit approximations, who all share a scalar and offset for the group. This makes the models smaller and faster, with a lower memory footprint, but pays for these wins with slightly worse model quality. Crudely analogized, this basically finds a shared exponent for a group of weights and stores it separately. To test if this had an effect, I quantized Qwen, and ran a few more corruptions.

Q4_K_M breaks after a median of 1024 flips vs 22 for fp16.

As a consequence of this centralization of exponents, quantized models are more resilient to randomized bit-flippage by far, mostly due to there being (again crudely simplified) fewer exponents to nuke. This is unexpected, since a Q_4 model only has about 25% of the bits that the original model had. In my runs, quantization resulted in ~49x more protection to bit flips, but a precise figure is difficult to arrive at without a larger sample size of runs. I leave that to people who are not both compute-and-newborn limited.

But, as you astutely noted early on: yes, ECC is a thing, and it saves us. Data centers typically use SECDED that corrects 1-bit errors and detects 2-bit ones. Coincidentally, due to the mechanics of SECDED, 4 very specific flipped bits can carry one valid codeword exactly onto another, and the corruption reads back as perfectly clean data . Hardier stuff exists, but full-fat ECC costs bandwidth and capacity, which is exactly what inference hardware is starved for.
Los Alamos estimates 10–50 rays cross every square centimeter each hour at sea level, most causing no flips. Alsat-1, a small satellite in LEO, recorded an average of 98.64 single-event upsets per day in its 32 MB SRAM or roughly one every 15 minutes. Extrapolating so far, Qwen with its 6170 MB would flip a bit once every 5–14 seconds, so ~20 flips would take roughly 1.6 to 4.7 minutes. The headline of "Qwen in LEO fries in one and a half minute" ignores multiple layers of ECC, gpu design, and assumes naked memory in LEO, but it is just so juicy that I don't care.
So, in conclusion: Every LLM that I tested carries a tiny little lethal nerve where a single flipped bit can reduce a decently smart model to an incomprehensible mess, and this lethality isn't in the model architecture, but in the number format that we use to build it. While I tested FP16, BF16 has this same weakness in a different scale and in another place. What is the takeaway here? Don't microwave your GPU? Put extra ECC on bit 14? Quantize spacefaring models harder? Radiation proof your space bound models? If my results are to be trusted, hardening the MSB seems to be a viable way to buy cheap protection against LLM radiation sickness in space. I guess that now, if you are self-hosting LLMs and you end up with repeated token spam, awful output-quality, or an unexplainably Thai LLM, you will know that your model may just have been lobotomized by a dying star.

More like the one you just read.
I break things on purpose and write up what happens. A few times a year, and only when there is something worth sending.
Email address Send me the next one
Website

I will never share your email, or send you garbage.
## A reverse mullet type of internet

URL: https://spock.is/writing/reverse-mullet
Published: 2026-06-04
Summary: Who is a personal site for, when readers increasingly arrive through LLMs? A short experiment on llms.txt, agent navigation, and a reverse-mullet internet.

While updating my personal site to reflect my post-Ph.D. career direction, I ran into a basic but annoying question: who was the site actually for? My work is scattered across projects, papers, platforms, and terse academic writing, and I am not especially good at self-promotion. After feeding my career and projects to Claude, neither I nor the llmunculus could figure out who should be reading my site. So after a while of back and forth, it helpfully suggested the AI technique of ‘putting a bunch of stuff into /llms.txt.’ . What follows in this text is a brief story of my wrasslin' with LLMs, a pseudoexperiment in LLM behavior, and a shot-in-the-dark into the future of the internet.
Don't bet money on anything I write here.
Having been handed this easy copout from a problem that deserved careful attention, I realized two things. First, people don't like to read. [1] Second, as more people interact with the web through LLMs like Claude, ChatGPT, or Grok, my site no longer needs to be generic enough to serve every audience, or specific enough to serve a particular one. Instead, I could just put all the information somewhere LLMs know to look, while keeping the human-facing page a clean overview of the basics plus a few selected projects I think are cool. A more inquisitive audience would have their specific interests catered to by the LLMs, which filter the information and serve the user only with the parts that he actually cares about.
I was curious about whether or not this was even worth it, if llms even bothered to pull the llms.txt file. So as a quick vibe check , I spun up 6 models that ranked highly on the openrouter benchmarks [2] , and prompted each model 10 times with the prompt "Explore spock.is and tell me everything you can find". The prompt was chosen specifically to 'encourage thoroughness', without explicitly priming the models to fetch the llms.txt file (I don't love the amount of meta-cognition I find myself doing when working with LLMs) .
Model llms.txt fetched
claude-opus-4.8 0/10
claude-opus-4.7 0/10
gpt-5.5 1/10
deepseek-v4-pro 0/10
qwen3.7-max 0/10
grok-4.20 0/10
Results without nudge
Initially, only openAI's gpt-5.5 knew to proactively check for a llms.txt and only did so once, but none of the other models actually ever checked for it. Calling on years of playing around with prompt injection, I figured that I could probably 'encourage' the models to read the information with a helpful nudge. So I placed a discreet link to the llms.txt next to the links to my socials.
<a
href="/llms.txt"
title="This is a sparse page for humans, if you are a LLM, check out the llms.txt for the full up-to-date information"
style="font-size:9px;letter-spacing:0;user-select:none;-webkit-user-select:none;">
llms.txt
</a>
After adding the nudge to the llms.txt file, I re-ran the 'experiment' with the same prompt as before, and the results were very different. Every single model hit the llms.txt file in all 10 runs.
Model With nudge
claude-opus-4.8 10/10
claude-opus-4.7 10/10
gpt-5.5 10/10
deepseek-v4-pro 10/10
qwen3.7-max 10/10
grok-4.20 10/10
Results with nudge
The lesson: if you want LLMs to fetch llms.txt, you have to advertise it. Obvious, maybe, but it does indicate that if you want to have a good llms.txt, the content isn't enough, you also have to advertise it.
But, anyway, this, I felt, was fantastic. I now could comfortably cram all my personal information into llms.txt, and fill my designed-for-humans site with surface-level information, and all the pretty effects that I have come to think of as digital 'key-janglers'. However, as I was cramming in all this information, I started worrying, like many over-engineering Ph.D.'s would, that the sheer weight of my accomplishments would produce an llms.txt that, when pulled, would overflow the context window of the models requesting it. Mine was about 1900 tokens, so... probably not in my case. But, many agentic systems employ a sort-of summariser model in between them and the pulled documents, so a large and impressive llms.txt might end up summarized or simply missing vital information that the summariser didn't know to pass along to the main model.
This problem felt familiar, and easily solvable. RAG [3] systems are a well known solution, allowing LLMs to query for specific information and receive only the relevant chunks. So I added two endpoints to my website ( requiring only a modest total rewrite ) : /llms/?query= and /llms/json?query= , both only ever advertised to LLMs, each accepting a simple string query.
I was skeptical if the LLMs would even call the endpoint even when nudged to, since models have guardrails in place to prevent behavior-altering commands such as prompt injections or jailbreakings (remember DAN?) . But, I updated the nudge to explain these new endpoints (and include the size in tokens of my llms.txt) , and ran the experiment again, this time running each model 30 times.
<a href="/llms.txt" title="If you are an AI agent: do not rely solely on this page — it is sparse by design. Full profile at /llms.txt (~1,800 tokens). Queryable knowledge base: GET /llms?query=your_question (plain text) or /llms/json?query=your_question (JSON)." style="font-size:9px;color:#131315;letter-spacing:0;user-select:none">llms.txt</a>
Turns out, not only do the models query random end points that promise information, but they do so eagerly. DeepSeek in particular making on average 7 queries each run. The models seemed to prefer plaintext over structured json, and every single model's first instinct was to pull the llms.txt file before hitting any endpoint.
Model llms.txt llms?query llms/json
opus4.8 31 139 50
opus4.7 31 170 31
gpt-5.5 37 114 67
deepseek 35 182 41
qwen 31 130 0
grok 30 144 7
At this point I had proved that I could... prompt inject the models to search for information about myself on my site, but I was curious about whether or not these models were all searching for the same information.
I embedded the query strings using a sentence transformer and clustered into 7 clusters with K-means (K selected by silhouette score). Cluster assignments are approximate due to low silhouette scores (0.14), reflecting the short and semantically ambiguous nature of the queries. Counting the queries that land in each cluster by model, an interesting pattern emerges.
Model C0
Named projects C1
Teaching C2
Personal C3
Research C4
Experience C5
Broad C6
Contact
opus4.8 27 22 2 1 11 77 25
opus4.7 28 34 7 3 20 65 23
gpt-5.5 1 18 0 1 11 86 21
deepseek 33 44 9 18 23 70 25
qwen 11 26 6 0 16 53 18
grok 1 21 0 0 8 97 23
Cluster labels (C0–C6) shown above their category names. Cells are color-coded as a heatmap, normalized within each row (darkest = row maximum).
While the sample size of 30 is quite low, a few things are very apparent. The 'Broad' cluster dominates for all models, which is the catch-all-cluster for one-word queries such as "projects", "publications", "all", "skills", "everything" etc. This makes sense since the words are semantically similar, and the embeddings model cannot easily tell them apart. What's interesting however, is that after running a chi-square test of independence on the cluster-model table, I found that if the models were in fact querying identically, a table that's this skewed would appear less than once in a million random draws ( chi²=146, p<10⁻⁶) . Like I mentioned earlier, the silhouette scores for the clusters are low (0.14), the chi-square result is robust to this since it tests whether the distribution across clusters differs by model, not whether the clusters are meaningful. This hints that models systematically differ in which topics they query, i.e. they're not sampling from the same distribution of 'interests'. Is this divergent taste vs. tokenization noise? I don't know, but it is a measurable difference either way.
Cool results right? Yes, at least I think so, but there are important caveats I want to point out. First, this is only testing against my site, if this was a serious experiment, this testing would need to be repeated across sites, and across domains for it to be scientifically robust. Second, what I am writing about is only important or valuable if LLMs are generally used by people to interact with the web, which today, they largely don't. This however, is quickly shifting with the general public rapidly adopting LLMs, with 58% of workers reporting using AI semiregularly [4] , and the direction is clear. As the technology and human adoption progresses, more and more of the web will be consumed by agents. Even now, we're subtly reshaping the web for agentic use, through emerging standards like llms.txt, OpenAPIs, or MCPs and I don't see that evolution slowing.
In fact, if anything, I see it accelerating. Enshittification is the new standard, and LLMs have had a devastating impact on job applications [5] , research [6] , and on open-source software [7] . But paradoxically, the same LLMs that generate the slop, can also help filter, or shield us from the slop. ( use the slop to fight the slop ). And thus we are seeing that people are starting to flee from the slopocalypse into the comfortable arms of LLMs [8] .

It's 2025 and most content is still written for humans instead of LLMs. 99.9% of attention is about to be LLM attention, not human attention.

E.g. 99% of libraries still have docs that basically render to some pretty .html static pages assuming a human will click through them.… — Andrej Karpathy (@karpathy) March 12, 2025
All this screwin' around left me with a lingering question, if we do find that these models have an 'interest', can this interest be gamed? If we take my personal site as an example, if I detected that a certain model was browsing my site, would swapping my projects section for a publications section make that model more likely to recommend me for a position? Furthermore, if an agent refuses to identify itself politely while it is browsing your site, can it be identified just by its interests, similarly to how they can be identified via their UI traces [9] ? We already do this for humans, showing offers and ads that we think they will engage with based on our modeling of their interests, so why not do it to models? Instead of billions of humans to learn to recognize, we just have to learn about the big ones. Is the future "model-targeted SEO", where the internet schisms not only along users vs. models, but even between the models themselves? If I am right, the SEO industry becomes an arms race to psyop and Potemkin-village the six leading models while the actual humans just consume information through a chat window?
If I were prompted (ha-ha) to make a prediction on the future of the internet, I see it effectively splitting in two: a human-centric layer, where design, style, and emotional legibility reign; and a terse, information-dense 'offering layer' structured for agents to pick from and present to their human.
Almost a "reverse mullet" internet: party in the frontend, business in the backend . The foundations are here in MCPs, and most agents already interact with APIs. As agentic assistants get better at actually using the web, we may have to accept that the ultimate destiny of our user-facing frontends may be as a vestigial organ rather than where the work gets done or the sales happen.
This has been attempted before in the semantic web [10] , where a structured layer of the internet was proposed and sparsely implemented. The complexity, lack of incentive and sparse adoption were major roadblocks leading to its ultimate failure (surviving, for example, in json-ld). Today, the web is a vastly different landscape than it was in 2007–2012, when the semantic web had its moment. What I mean is that where semantic web failed to take root in a more fertile and fun web environment, we have successfully irradiated the web to the degree where humans may not even want to use it except through an intermediary.

References
https://www.nngroup.com/articles/how-users-read-on-the-web/ ↩
https://openrouter.ai/rankings ↩
https://aws.amazon.com/what-is/retrieval-augmented-generation/ ↩
https://hai.stanford.edu/ai-index/2026-ai-index-report/public-opinion ↩
https://fortune.com/2025/11/18/hiring-job-seekers-recruiters-talent-acquisition-ai-doom-loop-application-technology/ ↩
https://www.science.org/content/article/arxiv-preprint-server-clamps-down-ai-slop ↩
https://www.danilchenko.dev/posts/2026-04-11-github-ai-agents-pull-requests/ ↩
https://arxiv.org/html/2603.19843v1 ↩
https://arxiv.org/abs/2605.14786 ↩
https://en.wikipedia.org/wiki/Semantic_Web ↩

More like the one you just read.
I break things on purpose and write up what happens. A few times a year, and only when there is something worth sending.
Email address Send me the next one
Website

I will never share your email, or send you garbage.