Unlocking what AI is doing
and why.

Mission

Resera Research studies how neural networks function on the inside: the mechanisms by which they represent information, compute, attend to context, and arrive at their outputs.

Our aim is to contribute controlled evidence to the shared understanding of how these systems work, alongside a growing research community: predictions written down before we run our experiments, and adversarial review of our own results. As AI systems grow more capable and more consequential, knowing how they do what they do, not just that they do it, becomes essential to trusting, steering, and improving them.

Current focus — 2026

Our present work studies how neural networks pack information: superposition (a network representing more features than it has neurons by storing them as overlapping directions) and the related phenomenon of polysemanticity (one neuron taking part in several unrelated computations). Building on the foundational work in this area, we are running a series of experiments, mostly in small models where we can check our answers against the true features, to better understand when networks pack features this way, what it costs them, what it buys them, and whether the same patterns hold in larger models.

Research & findings

• We refuted our own published superposition threshold: measured directly, the benefit begins below the packing band we had claimed
  Superposition is the trick by which a neural network represents more distinct features than it has neurons, by packing them along overlapping directions rather than giving each its own neuron. It is not free — the overlaps cause interference — so a natural question is: at what packing density does superposition actually start to pay for itself, beating the cleaner alternative of one-feature-per-neuron (monosemantic) coding? In an earlier finding we placed that crossover at a packing ratio of roughly n/m ≈ 1.25 features per neuron — superposition, we said, is favored once packing gets tighter than that. But that figure was the midpoint of a gap our grid never directly sampled; we had inferred the crossover, not measured it. Here we measured it directly, in the model's own loss units, across eight sparsity levels, against a fair monosemantic baseline (one that we had previously, and wrongly, denied a cost-free bias parameter). The result refutes our own prior claim. For all eight resolvable sparsities the net-benefit crossover sits below our pre-registered falsification band of [1.16, 1.34] — at n/m < 1.1538, and holding against the stricter analytic baseline in seven of eight. The benefit of superposition begins earlier — at looser packing — than we had said. We report the crossover as a bracket, (1.0345, 1.1538), not a single located point: the interior is resolution-limited at this network size, so we can bound where the benefit turns positive without pinpointing it. This is a toy-model measurement; we make no claim that the location transfers to real models, and we do not report a single located crossing. The contribution is twofold: a validated instrument for measuring the superposition/monosemantic trade directly in loss units, and an honest correction of our own published threshold — the prediction was on the record before this run. Attribution: this work's prior-art and forward-citation checks used the Semantic Scholar Academic Graph via its Open Data Platform API. Contact: contact@resera.ai
• Incidental polysemanticity
  Neural networks often store several unrelated features on the same neuron — “polysemanticity.” It is widely assumed this happens because a network has more features to represent than neurons to spare. We find that polysemanticity arises even when a network has spare capacity — more neurons than features — so feature-sharing is not merely a response to a capacity shortage but an incidental outcome of how networks initialize and train on sparse inputs. This sharpens what an interpretability method has to contend with: shared representations are the norm, not an artifact of tight budgets.
• The optimizer shapes how superposition forms
  How much of a network’s initial random structure survives training depends strongly on the optimizer. Plain (non-adaptive) gradient descent stays in a “rich” regime, reshaping its representations substantially; adaptive optimizers can instead stay “lazy,” retaining far more of their starting structure — a gap that grows with network width. We registered a prediction about the mechanism in advance, found the data reversed it, and report the corrected account. The practical upshot: a basic training choice changes how much superposition a network forms.
• The cost of sharing
  Does packing features onto shared neurons actually hurt a network? We find it carries a real but narrowly-scoped cost: when two features that share a neuron are active at the same time, the pair is reconstructed worse, while matched features that do not share a neuron are not — an effect we isolate with balance controls so it is not an artifact. Notably, the broad input-noise fragility sometimes attributed to polysemanticity did not appear; the cost is specific to co-activation of shared features.
• The cost survives into superposition
  Our cost result was first established in a simple “one-neuron-per-feature” setting, unlike the overlapping, distributed representations real models use. We find the co-activation cost survives into distributed superposition for the core case of two features colliding, across a wide range of packing densities. The honest shape: most of the measured cost is the geometry of overlapping directions, with a smaller genuinely-learned component on top that holds for pairs but fades when many features are co-activated in dense packing. This tells us the failure mode worth looking for in real models is pairwise interference between overlapping features.

We publish titles and findings rather than full papers while results are under review. Summaries are added as each result clears adversarial review.

About & contact

Resera Research was founded in 2026 by Cliff Schmidt. Cliff holds a degree in Brain and Cognitive Sciences from MIT and recently served as a Managing Director and Distinguished Software Engineer at JPMorgan Chase, where he led AI and machine learning for Corporate Responsibility.

Contact: contact@resera.ai