Unlocking what AI is doing
and why.

Mission

Resera Research studies how neural networks function on the inside: the mechanisms by which they represent information, compute, attend to context, and arrive at their outputs.

Our aim is to contribute careful, controlled evidence to the open, shared understanding of how these systems work, alongside a growing research community: predictions written down before we run our experiments, and adversarial review of our own results. As AI systems grow more capable and more consequential, knowing how they do what they do, not just that they do it, becomes essential to trusting, steering, and improving them.

Current focus — 2026

Our present work studies how neural networks pack information: superposition (a network representing more features than it has neurons by storing them as overlapping directions) and the related phenomenon of polysemanticity (one neuron taking part in several unrelated computations). Building on the foundational work in this area, we are running a series of pre-registered experiments, mostly in small models where we can check our answers against the true features, to better understand when networks pack features this way, what it costs them, what it buys them, and whether the same patterns hold in larger models. Our hope is to add careful, controlled evidence to the field's growing picture of how superposition works.

Research & findings

• Incidental polysemanticity
  Neural networks often store several unrelated features on the same neuron — “polysemanticity.” It is widely assumed this happens because a network has more features to represent than neurons to spare. We find that polysemanticity arises even when a network has spare capacity — more neurons than features — so feature-sharing is not merely a response to a capacity shortage but an incidental outcome of how networks initialize and train on sparse inputs. This sharpens what an interpretability method has to contend with: shared representations are the norm, not an artifact of tight budgets.
• The optimizer shapes how superposition forms
  How much of a network’s initial random structure survives training depends strongly on the optimizer. Plain (non-adaptive) gradient descent stays in a “rich” regime, reshaping its representations substantially; adaptive optimizers can instead stay “lazy,” retaining far more of their starting structure — a gap that grows with network width. We registered a prediction about the mechanism in advance, found the data reversed it, and report the corrected account. The practical upshot: a basic training choice changes how much superposition a network forms.
• The cost of sharing
  Does packing features onto shared neurons actually hurt a network? We find it carries a real but narrowly-scoped cost: when two features that share a neuron are active at the same time, the pair is reconstructed worse, while matched features that do not share a neuron are not — an effect we isolate with balance controls so it is not an artifact. Notably, the broad input-noise fragility sometimes attributed to polysemanticity did not appear; the cost is specific to co-activation of shared features.
• The cost survives into superposition
  Our cost result was first established in a simple “one-neuron-per-feature” setting, unlike the overlapping, distributed representations real models use. We find the co-activation cost survives into distributed superposition for the core case of two features colliding, across a wide range of packing densities. The honest shape: most of the measured cost is the geometry of overlapping directions, with a smaller genuinely-learned component on top that holds for pairs but fades when many features are co-activated in dense packing. This tells us the failure mode worth looking for in real models is pairwise interference between overlapping features.

We publish titles and findings rather than full papers while results are under review. Summaries are added as each result clears adversarial review.

About & contact

We pre-register our predictions, run controlled experiments, and subject our own results to adversarial review before we report them.

Contact: contact@resera.ai