Why Brain-Like Computers Are Hard: Spikes, Silicon Neurons and the Memristor

A brain runs on roughly 12 to 20 watts. A single modern AI accelerator can draw 300 to 700. That gap is the reason engineers keep trying to build computers that work like brains, and Asianometry's 2023 explainer is one of the clearest tours of why that is so difficult (Asianometry, 2023). This post follows the video, which is embedded below, and adds the papers behind the chips it describes. The timestamps jump to the matching moment.

The bottleneck neuromorphic computing tries to escape

Conventional computers follow the von Neumann design: programs and data share one memory, and both travel the same road to the processor. The result is that systems are often limited not by how fast they compute but by how fast they can move data, the von Neumann bottleneck (watch from 0:03). In-memory computing brings memory closer to compute. A neuromorphic system, built to work like the brain, has to change much more than that.

The motivation is efficiency. The video compares the brain's 12 to 20 watts with a desktop computer's 175 and an H100's 300 to 700, and notes that the brain manages this with neurons firing at around 10 hertz. A bee, with fewer than a million neurons and less than a watt, can fly, navigate and find its way home (watch from 1:08).

Parallel, unclocked and noisy

Brains get there by being massively parallel and unsynchronised. The brain's roughly 86 billion neurons fire spikes on their own schedule, in response to spikes from their neighbours, with no central clock (watch from 2:11). Digital chips do the opposite: a clock coordinates every step, which makes some tasks very fast but costs time and energy to distribute and leaves components waiting for their turn.

The brain also treats noise as a feature. Most spikes a neuron receives do nothing, but there are so many connections that enough signals get through. A CPU, by contrast, spends a great deal of energy making sure almost every signal counts (watch from 3:14). Tolerating error is part of what makes brains efficient, and also remarkably resilient to damage (watch from 4:14).

Neurons, synapses and memory in the same place

The video then walks through the neuron: dendrites carry spikes in, the soma integrates them, and the axon carries a spike on to other neurons through synapses. A neuron in the cerebral cortex can have around 10,000 inputs, and one in the cerebellum up to a quarter of a million (watch from 4:14; 5:15). Synapses change strength over time, and neurons vary enormously in speed, length and the stimuli they respond to (watch from 6:15).

The crucial point is that memory and computation are not separate. Memories are stored in synaptic strengths, and synapses take part in the computation too, which is why the brain has no von Neumann bottleneck. A real neuromorphic computer therefore needs artificial neurons and synapses that bind memory, compute and communication tightly together (watch from 7:20).

That is also why today's neural networks are not neuromorphic in this sense. Perceptrons borrow the neuron's weighted-sum-and-fire idea, but the networks behind modern AI run on von Neumann hardware, GPUs and TPUs included. Neuromorphic systems instead run spiking neural networks on hardware designed for them (watch from 8:20).

Silicon neurons

The first generation of neuromorphic chips uses ordinary CMOS transistors (watch from 9:22):

• IBM TrueNorth (2014): 4,096 cores of 256 programmable neurons, 256 million synapses and 5.4 billion transistors, running on about 70 milliwatts (Merolla et al., 2014).

• Intel Loihi: a manycore research chip with on-chip learning, followed by Loihi 2 (Davies et al., 2018).

• BrainScaleS: from the European Human Brain Project, using analog circuits to mimic spiking neurons (watch from 10:25).

• Tianjic (2019): a Tsinghua University hybrid chip that runs both conventional and spiking networks (Pei et al., 2019).

The video's close look at TrueNorth shows how this works in practice. Each core links 256 axons to 256 neurons through a 256 × 256 synaptic crossbar held in SRAM. Neurons accumulate incoming spikes into a membrane potential and fire when it crosses a threshold. The cores run asynchronously and respond only to events, and random-number generators add the brain's noise on purpose (watch from 11:30; 12:34).

Why CMOS is not enough

Building on CMOS lets neuromorphic chips ride the semiconductor industry's progress, and TrueNorth is scalable and frugal. But the video lists real drawbacks (watch from 12:34; 13:39):

• Efficiency costs accuracy. Tests found a trade-off between energy efficiency and accuracy, and commercially competitive accuracy needs more power.

• The chips are large. Every core carries its own memory, whereas von Neumann machines benefit from one dense memory bank that fits today's large models.

• The brain is analog. Digital circuits are a poor fit, and adding analog elements reduces flexibility.

The memristor

The most popular way beyond CMOS is the memristor. Leon Chua proposed it in 1971 as a fourth basic circuit element, alongside the resistor, capacitor and inductor (Chua, 1971). The idea sat largely unused until Hewlett-Packard researchers reported a physical device in 2008: a platinum and titanium-oxide sandwich whose resistance depends on the history of the voltage applied to it (Strukov et al., 2008; watch from 14:43).

That history-dependence, kept even with the power off, is why memristors look like artificial synapses and promise in-memory computing. The challenges are practical: manufacturing them uniformly at scale, their endurance and retention, and programming a device whose response is non-linear (watch from 15:46).

The real competitor

The video ends on what may be the biggest obstacle: GPUs keep getting better. AI inference performance on GPUs has improved roughly a thousandfold in a decade, which makes a new architecture hard to justify on performance alone. The more likely future, it suggests, is hybrid: neuromorphic and von Neumann chiplets side by side, each doing what it does best (watch from 16:49).

What to take away

Neuromorphic computing is not about copying the neuron's shape; it is about copying the brain's economics: unclocked, event-driven, noisy and with memory where the computation happens. Silicon neurons have shown the idea works; memristors may make it dense and efficient. Whether either beats the GPU curve depends less on biology than on manufacturing, software and the patience of the industry.

References

Asianometry. (2023, December 21). Why brain-like computers are hard [Video]. YouTube. https://www.youtube.com/watch?v=FegeRT5N3A4

Chua, L. (1971). Memristor—The missing circuit element. IEEE Transactions on Circuit Theory, 18(5), 507–519. https://doi.org/10.1109/TCT.1971.1083337

Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., Liao, Y., Lin, C.-K., Lines, A., Liu, R., Mathaikutty, D., McCoy, S., Paul, A., Tse, J., Venkataramanan, G., … Wang, H. (2018). Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1), 82–99. https://doi.org/10.1109/MM.2018.112130359

Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, F., Jackson, B. L., Imam, N., Guo, C., Nakamura, Y., Brezzo, B., Vo, I., Esser, S. K., Appuswamy, R., Taba, B., Amir, A., Flickner, M. D., Risk, W. P., Manohar, R., & Modha, D. S. (2014). A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345(6197), 668–673. https://doi.org/10.1126/science.1254642

Pei, J., Deng, L., Song, S., Zhao, M., Zhang, Y., Wu, S., Wang, G., Zou, Z., Wu, Z., He, W., Chen, F., Deng, N., Wu, S., Wang, Y., Wu, Y., Yang, Z., Ma, C., Li, G., Han, W., … Shi, L. (2019). Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature, 572(7767), 106–111. https://doi.org/10.1038/s41586-019-1424-8

Strukov, D. B., Snider, G. S., Stewart, D. R., & Williams, R. S. (2008). The missing memristor found. Nature, 453(7191), 80–83. https://doi.org/10.1038/nature06932