Semantic memory

Semantic memory is a cognitive term for all of the “commonsense” general knowledge that humans acquire about the world, in the posterior areas of the neocortex. It is contrasted with episodic memory, which refers to learning about individual specific episodes of experience. In the complementary learning systems (CLS) framework, the hippocampus has neural specializations to support episodic memory (fast learning and sparse, pattern-separated representations), while the neocortex performs slower, interleaved learning over distributed representations to develop categorical abstractions that enable us to reason about the world more efficiently, and systematically generalize to novel situations.

The fundamental tradeoffs between combinatorial vs conjunctive representations (in the neocortex vs. hippocampus respectively) motivate the need for distinct neural systems supporting these different types of representations. By contrast, the transformer architecture used in large language models (LLMs) has a single neural network component in each layer with properties that generally favor more conjunctive representations, supporting a combination of both episodic and semantic memory in one network.

These LLM models provide the best example outside of the human brain of the power of semantic memory trained on essentially all of human knowledge, to support flexible reasoning and systematic, generative behavior. This crystalized intelligence can be contrasted with the fluid intelligence supported by the prefrontal cortex, which is critical in humans for supporting strategic, goal-directed problem-solving in particularly challenging novel domains.

Although much effort is currently under way to develop these “reasoning” abilities in LLMs, ultimately it is the human prompting and control signals that keep these models focused on solving specific problems. The Rubicon model represents our attempt to understand how these goal-driven mechanisms work in the human brain.