Memory is rarely a perfect recording. More often, it is a reconstruction. We remember pieces of an event, infer what likely happened between them, and recover the shape of the whole sequence from context.
MESH starts from this idea: what if a memory system could reconstruct an unfolding experience from only a few partial observations?
The project explores a memory-based architecture for sequence reconstruction. Instead of storing every full sequence in detail, MESH stores compact exemplar-like entries that link three things: what was happening, which episode it belonged to, and where it occurred in the episode. In the language of the model, these are state, episode, and phase.
That triplet is the core idea. A memory entry does not just say this state occurred. It says something closer to: this kind of state occurred in this kind of episode at this point in its temporal structure. With enough of these entries, the system can begin to answer reconstruction questions from different directions.
Given a few observed states, MESH tries to infer what episode they belong to. Given an episode and some observed moments, it tries to estimate where those moments sit in the sequence. Given an episode and a phase, it tries to reconstruct what should be happening there. The goal is not just recognition, but completion: recovering the missing parts of a structured sequence from partial evidence.
This makes MESH especially interesting for sequences where timing matters. A melody, an action, a spoken utterance, a movement, or a sensory episode is not just a bag of features. The same state can mean different things depending on when it appears and what larger episode it belongs to. MESH treats that temporal position, phase, as a first-class part of memory.
The larger ambition is to build a hierarchy of such memory modules. A local module reconstructs structure at one scale; a network of modules could eventually support richer, multi-level reconstruction. Lower levels might recover fine-grained states, while higher levels represent broader episodes or contexts. In that sense, MESH is not only a memory table, but a sketch of a system that could move between fragments and wholes.
The guiding question is simple but deep: how much of an experience do we need to store in order to reconstruct the rest?
A naive memory system could keep every full trajectory. That would work, but it would not be compact. MESH asks whether a fixed-capacity memory can approximate that ideal by learning a smaller set of useful entries. If successful, the system would behave like a compressed exemplar memory: not storing everything, but storing enough structure to infer what is missing.
This also gives MESH a clear scientific role. It is a way to study the ingredients of sequence memory: identity, temporal position, local state, interpolation, retrieval, and reconstruction. When reconstruction succeeds, the model tells us something about what kind of structure was sufficient. When it fails, the failure can be traced to episode inference, phase estimation, or state retrieval.
So the short version is: MESH tries to turn memory into a reconstructive system. It asks whether partial observations can be expanded into full sequences using a compact store of state-episode-phase associations. The long-term vision is a hierarchical memory architecture that can recover structured experience from fragments, bridging the gap between raw observations and remembered episodes.