NeuromorphicDo LLMs Need to Dream? Karpathy on Model Collapse and the Search for Entropy
Andrej Karpathy explains why language models cannot simply train on their own reflections: model samples are silently collapsed. Dreams, jokes and children show why learning systems need a steady supply of entropy.
Akmal Alif
