Scientists rebuilt what mice saw from brain activity

Researchers have reconstructed 10-second movies using nothing but the brain activity of mice, effectively playing back a rough version of what the animals had just watched. The work, published in eLife by a team at the Sainsbury Wellcome Centre at University College London, is the first time natural movies have been rebuilt from single-cell recordings rather than broader brain imaging.

The team recorded the activity of individual neurons in the mouse visual cortex with two-photon calcium imaging while the animals watched film clips, then reconstructed 10-second videos at 30 frames per second from those signals alone. They report a pixel-level correlation of 0.57 between the real clips and single-trial reconstructions — against 0.24 for earlier work that reconstructed static images from the same kind of recordings.

How the reconstruction works

The method starts from a dynamic neural encoding model, the kind built for the 2023 Sensorium Competition to predict how individual neurons respond to a movie. The researchers first asked the model what the neurons would have done if the mouse had been staring at a blank screen, then compared that prediction with what the neurons actually did. The difference was fed back into an algorithm that nudged the pixels of an initially empty video, round after round, until the reconstruction drifted towards the film the mouse had really seen — a process described as video optimisation by backpropagation through the encoding model.

Two details suggest the system is not simply replaying films it already knew. Reconstruction quality improved as more neurons were added to the analysis, and the team tested the trained model on a video that had never been part of its training data, reconstructing a 10-second clip from that recording alone. Averaging the predictions of several models improved the result further.

Why single cells change the picture

Most celebrated work in this field uses fMRI in humans, where a scanner measures blood flow across large regions of the brain and the reconstruction is inferred from those broad signals. Recording individual cells offers something more direct: a measure of what specific neurons are representing, rather than an average across millions of them.

Dr Joel Bauer, the study's lead author, said the aim was to build a method that could capture what is being represented in the brain and compare it with reality, rather than depending on experiments designed around one particular question. The team's wider interest is in the gap between what is physically in front of an animal and what its brain does with it.

The limits are still substantial

A correlation of 0.57 is a long way from a clean picture. The researchers themselves note considerable room to improve image resolution and the proportion of the visual scene that can be recovered, and say future work will focus on collecting data that supports sharper reconstructions. The timing of the reconstructed clips also drifts slightly against the originals.

Our opinion

The obvious headline — mind reading — is the least interesting thing here. What makes this work worth attention is the measurement it enables: a way to put an animal's internal picture next to the actual scene and score the difference. Bauer's own framing is the striking part, that the brain's habit of skewing and warping what arrives from the eyes is not a defect but a feature of how perception is built. That is a far more useful finding than any reconstructed video, because it turns a philosophical argument about perception into something you can quantify. Two caveats are worth holding onto. The correlation is modest and the reconstructions are fuzzy, so anything resembling a decoder for arbitrary mental content remains a research programme rather than a product. And the technique is currently limited to animals with electrodes and microscopes trained on visual cortex, which is a very different world from consumer brain-computer interfaces. The pipeline runs literally through a microscope; that part is still science, and the interesting science is the comparison, not the movie.