Neocortex
The neocortex (“new cortex”) or cerebral cortex (often just referred to as “cortex”) is the evolutionarily most recent, outer portion of the brain (i.e., the telencephalon, also known as the pallium in other vertibrates) where most of advanced cognitive functions take place. This brain area is the primary focus of the Axon framework, both in terms of activity dynamics and learning mechanisms. It is unique in having extensive bidirectional connectivity among excitatory neurons (also known as recurrent connectivity), whereas most other brain areas only have such connections involving inhibitory neurons.
The connectivity of the neocortex is organized into multiple different specialized areas of processing, and each such area has a similar laminar structure with 6 distinct layers of neurons. These multiple areas correspond to the “deep” layered organization of abstract neural networks, which demonstrably provide important computational power. At a cognitive level, this power arises from the process of categorization, where multiple stages of neuron detectors progressively develop more abstract, general, and powerful ways of representing the world, which then support more systematic and appropriate behaviors even in novel environments.
Bidirectional connectivity
Main page: Bidirectional connectivity
Bidirectional excitatory connectivity supports multiple important functional properties, including: constraint satisfaction, attractor dynamics, top-down attention, and imagery. At the broadest level, most theories of conscious awareness hinge on the ability to directly share information in an all-to-all manner, and either directly depend on bidirectional connectivity for this (Lamme, 2006) or indirectly through constructs such as a global workspace or related constructs (e.g., Baars, 2002, Dehaene et al., 2017; Dehaene et al., 1998; Seth & Bayne, 2022).
Relative to most abstract neural network (ANN) models, Axon is unique in directly incorporating full bidirectional connectivity, and is thus in a position to capture the computational and functional benefits that this connectivity affords. However, there are also significant practical difficulties that come along with bidirectional excitatory connectivity. For example, it tends to produce complex activation dynamics that can easily lead to runaway positive feedback loops, which unfortunately also occur in the human brain in the form of epileptic seizures. A robust form of inhibition is critical for controlling these feedback loops.
Learning is also much more difficult in the context of complex activation dynamics, and interestingly there are surprisingly impressive results from reservoir computing networks that eschew learning within bidirectionally connected networks entirely, using them instead as “reservoirs” of complex dynamical activity states from which signals can be decoded via simpler feedforward error-driven learning mechanisms.
By contrast, the form of learning in Axon depends critically on bidirectional excitatory connectivity for propagating error signals throughout the network, and can tune large, complex bidirectional networks to develop effective predictive learning representations of the environment, leveraging the principle of learning based on a temporal derivative. There is now experimental evidence consistent with this form of learning in at least one specific, widely studied pathway involving cortical pyramidal neurons and synaptic mechanisms that exist throughout the neocortex (Jang et al (2026)).
From a computational cost perspective, bidirectional connectivity is very expensive because it doubles the number of synaptic connections, and requires roughly 200x iterations through the network to process a single input. This significantly limits the ability to scale up the models, which has been the primary driver of impressive computational performance in current feedforward ANN models. Nevertheless, as parallel compute hardware continues to improve, this limitation will hopefully be overcome (and the current version of Axon runs efficiently on any GPU, using WebGPU so it works through the browser too). For the time being, the models do focus more on capturing the principles rather than the kinds of raw performance improvements that come with scaling (see bias-variance tradeoff for more discussion).
Categorization
Main page: Categorization
At a cognitive level, we can understand the function of the neocortex in terms of the neuron detector model, where multiple layers of such detectors build up increasingly abstract categories that enable the organism to more systematically generalize to novel stimuli. For example, faces can look very different from one another in terms of their raw “pixel” inputs, but we can categorize these diverse inputs in many different ways, to treat some patterns as more similar than others: male vs. female, young vs. old, happy vs. sad, “my mother” vs. “someone else”, etc. See the faces simulation for a simple model demonstrating this principle.
Forming these categories is essential for enabling us to make the appropriate behavioral and cognitive responses (approach vs. avoid, borrow money from, etc.). Imagine trying to relate all the raw inputs of a visual image of a face to appropriate behavioral responses, without the benefit of such categories – the relationship (“mapping”) between pixels and responses is just too complex. These intermediate, abstract categories organize and simplify cognition, just like file folders organize and simplify documents on your computer. One can argue that much of intelligence amounts to developing and using these abstract categories in the right ways.
It is also critical to understand how many individual neural detectors at each stage of processing can work together to capture the subtlety and complexity necessary to encode complex conceptual categories, in the form of distributed representations. These distributed representations are also critical for enabling multiple different ways of categorizing an input to be active at the same time — e.g., a given face can be simultaneously recognized as female, old, and happy. A great deal of the emergent intelligence of the human brain arises from multiple successive levels of cascading distributed representations, constituting the collective actions of billions of neurons working together in the cortex.
Biology of the neocortex
Figure 1: Neural constituents of the neocortex. (A) shows excitatory pyramidal neurons, which constitute roughly 85% of neurons, and convey the bulk of the information content via longer-range axonal projections (some of which can go all the way across the brain). (B) shows inhibitory interneurons, which have much more local patterns of connectivity, and represent the remaining 15% of neurons. Reproduced from Crick & Asanuma (1986).
The neocortex is composed of roughly 85% excitatory neurons (mainly pyramidal neurons, but also stellate cells in layer 4), and 15% inhibitory interneurons (Figure 1). We focus primarily on the excitatory pyramidal neurons, which perform the bulk of the information processing in the cortex. Unlike the local inhibitory interneurons, they engage in long-range connections between different cortical areas, and it is clear that learning takes place in the synapses between these excitatory neurons (evidence is more mixed for the inhibitory neurons).
The inhibitory neurons can be understood as “cooling off” the excitatory heat generated by the pyramidal neurons, much like the cooling system (radiator and coolant) in a car engine. Without these inhibitory interneurons, the system would overheat with excitation and lock up in epileptic seizures (this is easily seen by blocking inhibitory GABA channels, for example). There are, however, areas outside of the cortex (e.g., the basal ganglia and cerebellum) where important information processing does take place via inhibitory neurons, and certainly some researchers will object to this stark division of labor even within cortex, but it is nevertheless a very useful simplification.
Layered Structure
Figure 2: A slice of the visual cortex of a cat, showing the six major cortical layers (I - VI), with sublayers of layer IV that are only present in visual cortex. The first layer (I) is primarily axons (“white matter”). Reproduced from Sejnowski and Churchland (1989).
The most evolutionarily recent areas of neocortex have a characteristic 6-layer structure (Figure 2), which evolved from a simpler form of mesocortex (also known as archicortex or protocortex_), as discussed in Evolution of neocortex. This laminar structure varies across different areas of cortex (Figure 3), in ways that are consistent with the different functions of these areas, as summarized in Figure 4. The anatomical patterns of connectivity in the cortex are also an important source of information giving rise to the following functional picture:
Figure 3: The thickness of the different cortical layers varies depending on the location in cortex — this is an important clue to the function of these layers (and the cortical areas). A) shows primary visual cortex (same as the previous figure) which emphasizes input layer 4. B) shows extrastriate cortex which processes visual information, and emphasizes superficial layers 2,3. C) shows primary motor cortex, which emphasizes deep layers 5,6. D) shows prefrontal cortex (“executive function”) which has an even blend of all layers. Reproduced from Shepherd (1990).
• Input areas of the cortex (e.g., primary visual cortex) receive sensory input (typically via the thalamus), and these areas have a greatly enlarged layer 4, which is where the axons from the thalamus primarily terminate. The input layer contains a specialized type of excitatory neuron called the stellate cell, which has a dense bushy dendrite that is relatively localized, and seems particularly good at collecting the local axonal input to this layer.
• Hidden areas of the cortex are so-called because they don’t directly receive sensory input, nor do they directly drive motor output — they are “hidden” somewhere in between. The bulk of the cortex is “hidden” by this definition, and this makes sense if we think of these areas as creating increasingly sophisticated and abstract categories from the sensory inputs, and helping to select appropriate behavioral responses based on these high-level categories. This is what most of the cortex does, in one way or another. These areas have thicker superficial layers 2,3, which contain many pyramidal neurons that are well positioned for performing this critical categorization function.
• Output areas of cortex have neurons that synapse directly onto muscle control areas (“motor outputs”), and are capable of causing physical movement when directly stimulated electrically. These areas have much thicker deep layers 5,6, which send axonal projections back down into many different subcortical areas. The evolutionarily more ancient mesocortical areas (e.g., cingulate cortex along the medial wall) lack a distinct layer 4, and are primarily output areas with mostly deep layer neurons.
Figure 4: Function of the cortical layers: layer 4 processes input information (e.g., from sensory inputs) and drives superficial layers 2,3, which provide a “hidden” internal re-processing of the inputs (extracting behaviorally relevant categories), which then drive deep layers 5,6 to output a motor response. Green triangles indicate excitation, and red circles indicate inhibition via inhibitory interneurons. Solid lines indicate connections that are included in our models. Dotted lines indicate less strong connections that are not included in our models. BG = basal ganglia which is important for driving motor outputs, and Subcortex includes a large number of other subcortical areas.
In summary, the layer-wise (laminar) structure of the cortex and the area-wise function of different cortical areas converge to paint a clear picture about what the cortex does: it takes in sensory inputs, processes them in many different important ways to extract behaviorally relevant categories, which can then drive appropriate motor responses. We will adopt this same basic structure for most of the models we explore.
Patterns of Connectivity
Figure 5: Typical patterns of connectivity between cortical areas (Feedforward and Feedback) and within cortical areas (Lateral). Information flows from the Hidden layers “up” in a feedforward direction into the Input layers of “higher” areas (from which it flows into the Hidden layer of that area), and flows back down in a feedback direction from Hidden and Output (output typically stronger as indicated) back to Hidden and Output layers in “lower” areas. All areas have lateral connections that similarly originate in the Hidden and Output layers, and connect to all three layers within another part of the same cortical area. Based on Figure 3 from Felleman and Van Essen (1991).
The dominant patterns of longer-range connectivity between cortical areas, and lateral connections within cortical areas are shown in Figure 5. Consistent with the Input-Hidden-Output laminar structure described above, the feedforward flow of information “up” the cortical hierarchy of areas (i.e., moving further away from sensory inputs) goes from Input to Hidden in one area, and then to Input to Hidden in the next area, and so on. This flow of information from sensory inputs deeper into the higher levels of the brain is what supports the formation of increasingly abstract hierarchies of categories that we discuss in greater detail in the next section.
Information flowing in the reverse direction (feedback) goes from Hidden & Output in one area to Hidden & Output in the previous area, and so on. We will see later in this chapter how this backward flow of information can support top-down cognitive control over behavior, direct attention, and help resolve ambiguities in the sensory inputs (which are ubiquitous). One might have expected this pattern to go Hidden to Output in one area, to Hidden to Output in the previous area, but this pattern is only part of the story. In addition, the Hidden layers can communicate directly to each other across areas. Furthermore, Output areas can also directly communicate with each other.
We can simplify this pattern by assuming that the Output layers in many cortical areas serve more as extra copies of the Hidden layer patterns, which help make additional connections (especially to subcortical areas — all cortical areas project to multiple subcortical areas). Thus, the essential computational functions are taking place directly in the Hidden to Hidden connections between areas (mediated by intervening Input layers for the feedforward direction), and Output layers provide an “external interface” to communicate these Hidden representations more broadly. The exception to this general idea would be in the motor output areas of cortex, where the Output layers may be doing something more independent (they are at least considerably larger in these areas).
Each cortical area also has extensive lateral connectivity among neurons within the same area, and this follows the same general pattern as the feedback projection, except that it also terminates in layer 4. These lateral connections serve a functional role very similar to that of the feedback projections — essentially they represent “self feedback”.
Figure 6: Connectivity matrix between cortical areas, showing that when a given area sends a feedforward projection to another area, it typically also receives a feedback projection from that same area. Thus, cortical connectivity is predominantly bidirectional. Reproduced from Sporns & Zwi (2004).
The other significant aspect of cortical connectivity that will become quite important for our models, is that the connectivity is largely bidirectional (Figure 6). Thus, an area that sends a feedforward projection to another area also typically receives a reciprocal feedback projection from that same area. As noted in the introduction, this bidirectional connectivity has many functional benefits, including being important for enabling the network to converge into a coherent overall state of activity across layers, and for error-driven learning.
Laminar Details
This section provides a more detailed treatment of the laminar structure of neocortex, across different cortical areas. As discussed in Evolution of neocortex, the most evolutionarily ancient cortical areas only have three layers of structure, e.g., in the cingulate areas and in the hippocampus areas (CA3, CA1, and DG). Thus, here we generally refer to more modern neocortical areas that have the full compliment of laminar neuron types.