One such model, called Adaptive Resonance Theory, or ART, proposes how the brain learns categories whereby to attend, recognize, and predict objects and events. This is accomplished in ART using feedback interactions between attended feature patterns and an active recognition category, leading to a feature-category resonance (Figure 2). In this conception, attended feature patterns activate bottom-up adaptive filter pathways, while activated recognition categories active their top-down learned expectation pathways.

Bottom-up inputs to the feature pattern level are matched against active top-down expectations. An active top-down expectation, in the absence of bottom-up inputs, can modulate, or prime, the feature-selective cells to anticipate expected feature patterns, but cannot, by themselves, fire these cells. When a good enough match occurs, the system locks into an attentive resonant state that synchronizes, amplifies, and prolongs system response (Figure 2). Such a feature-category resonance drives the recognition learning process whereby the adaptive weights, or LTM traces, in the bottom-up and top-down pathways can learn from the patterns of resonating activities, or STM traces; hence the term adaptive resonance. By focusing on attended features, the resonance also suppresses unattended features, thereby helping to solve the stability-plasticity dilemma (Grossberg, 1980), whereby the brain can learn quickly, without being also forced to catastrophically forget already learned, but still useful, knowledge.

ART has predicted that all conscious brain states are resonant states, and that feature-category resonances support the recognition of objects and events (Grossberg, 1980, 2017). More generally, ART predicts a link between brain processes of consciousness, learning, expectation, attention, and synchrony; the so-called CLEARS processes, and that all brain systems that are capable of supporting conscious experiences embody such a linkage between CLEARS processes.

Category learning in ART is controlled by interactions between an attentional system and an orienting system (Figure 3). These two systems embody computationally complementary laws (Grossberg, 2000a). The attentional system (levels F₁ and F₂ in Figure 3) carries out processes like attention, category learning, expectation, and resonance in response to familiar and expected events. The orienting system (level A in Figure 3) enables the attentional system to learn about unfamiliar and unexpected information using processes like reset, memory search, and hypothesis testing. The attentional system includes brain regions like temporal cortex and prefrontal cortex (PFC). The orienting system includes brain regions like the nonspecific thalamus and hippocampus (HPC).

If an input pattern causes a sufficiently bad mismatch to occur within the attentional system, it will activate the orienting system. The orienting system, in turn, resets the active recognition category and initiates a memory search, or hypothesis-testing for a better-matching category, possibly an entirely new one. Figure 3 summarizes this ART search and learning cycle.

Object attention is carried in an ART network by a top-down, modulatory on-center, off-surround network. Such a circuit is said to obey the ART Matching Rule. Due to the ART Matching Rule, attention can modulate, sensitize, or prime an expected critical pattern of feature-selective cells, but cannot fully excite them unless enough of them also receive matched bottom-up featural inputs. The off-surround network can actively inhibit unexpected features at the same time. Carpenter and Grossberg (1987) have mathematically proved that category learning is stable in response to an arbitrary sequence of input patterns if the ART Matching Rule is obeyed, but can easily become unstable if it is not, leading to catastrophic forgetting of previously learned categories. See Grossberg (2013, 2017) for a summary of the abundant psychological and neurobiological evidence that supports the prediction that object attention obeys the ART Matching Rule.

How good a match is required for resonance, sustained attention, learning, and consciousness, to occur? This criterion is set by a vigilance parameter ρ that is computed within the orienting system (Carpenter and Grossberg, 1987). The size of the vigilance parameter determines the generality of the recognition categories that will be learned. If vigilance is high, then learning of a concrete or specific category occurs, such as recognition of a frontal view of a friend’s face. If vigilance is low, then learning of an abstract or general category occurs, such as recognition that everyone has a face. In general, vigilance is chosen as low as possible to conserve memory resources, without causing a reduction in predictive success. Because this baseline vigilance level is initially set at the lowest level that has led to predictive success in the past, ART models try to learn the most general category that is consistent with the data. This tendency can lead to the type of overgeneralization that is seen in young children (Brooks et al., 1999) until subsequent learning leads to category refinement (Tomasello and Herron, 1988).

When a given task requires a finer categorization, vigilance is raised. Vigilance can be automatically adjusted to learn either specific or general information in response to predictive failures, or disconfirmations, within each environment. Such a predictive failure could occur, for example, if a viewer classifies an object as a dog, whereas it is really a fox. Within ART, such a predictive disconfirmation causes a memory search that shifts attention to focus on a different combination of features that can successfully be used to recognize that the object is, in fact, a fox, and perhaps to recognize other foxes as well.

Vigilance is computed within the orienting system of an ART model (Figures 3B–D). This can happen because the orienting system measures how good a match occurs in the attentional system. It does so in the following way: An input pattern I activates two bifurcating pathways. One pathway activates feature detectors at level F₁ in the attentional system. This activity pattern is denoted by X in Figure 3A. After a top-down expectation V is also activated, the number of active feature detector cells decreases as the match of I with the top-down expectation gets worse. This attended activity pattern is denoted by X* in Figures 3B,C. The second pathway that is activated by I sends converging excitatory signals to the orienting system A. After the feature detectors in F₁ are activated, they also activate two bifurcating pathways. One pathway sends a signal pattern S via an adaptive filter to level F₂, where they activate category-coding cells that compete to be stored in short-term memory (STM). The other pathway sends converging inhibitory signals to the orienting system A. In this way, I and X* compete within A. A nonspecific reset signal is emitted by A if the inhibition from X* to A is less than the excitation from I to A.

Here is where the vigilance parameter plays a role. The vigilance parameter ρ multiplies the bottom-up excitatory inputs I to A; that is, ρ is the gain of the excitatory inputs to A. The orienting system A is activated when the total excitatory input ρI is greater than the total inhibition from the features X* across F₁ that survive top-down matching. This occurs when ρ|I| − |X*| > 0, where |.| denotes the number of positive inputs or matched features. This inequality can be rewritten as ρ > |X*| |I|⁻¹ to show that the orienting system is activated whenever ρ is chosen higher than the ratio of active X* matched features in F₁ to total features in I. In other words, the vigilance parameter controls how bad a match can be before reset of the current category, and search for a new category, is initiated.

If the vigilance parameter is low, then many exemplars can all influence the learning of a shared category prototype, by chipping away at the features that are not shared with all the coded exemplars. If the vigilance parameter is high, then even a small difference between a new exemplar and a known prototype (e.g., F vs. E) can drive the search for a new category with which to represent F.

Vigilance can be automatically adjusted in response to a predictive disconfirmation (e.g., E is predicted in response to F). One particularly useful way is called match tracking. Suppose that on every learning trial, a predictive failure causes vigilance to increase by the smallest amount that can trigger reset of the currently active category. That is, increase ρ until ρ > |X*||I|⁻¹. Reset is followed by a memory search for a new recognition category that can correct the error. As a result of such a memory search, a category will be learned that is just general enough to eliminate the error, because every increase in vigilance reduces the generality of learned categories, and vigilance was increased by the minimum amount that was needed to learn a new category. This scheme is called match tracking because vigilance tracks the degree of match between the input pattern I and the attended feature pattern X*. Match tracking leads to minimax learning, or learning that can minimize predictive errors while it conjointly maximizes category generality. In other words, match tracking uses the minimum memory resources that are needed to correct the predictive error.

Because vigilance can vary during match tracking in a manner that reflects current predictive success, recognition categories capable of encoding widely differing degrees of generalization or abstraction can be learned by a single ART system (Carpenter and Grossberg, 1987, 1991). Thus a single ART system may be used, say, to learn abstract prototypes with which to recognize abstract categories of faces and dogs, as well as “exemplar prototypes” with which to recognize individual views of faces and dogs, depending on task requirements.

The LAMINART model was developed in articles such as Grossberg and Raizada (2000) and Raizada and Grossberg (2001) to simulate psychological and neurobiological data about 2D visual perception using rate-based neurons. 3D LAMINART unlumped, and generalized, these analyses to the perception of objects in depth. LAMINART was also unlumped in another direction by the Synchronous Matching ART, or SMART, model (Figure 6; Grossberg and Versace, 2008; Grossberg et al., 2016) wherein rate-based neurons were replaced with spiking neurons, among other refinements.

The SMART model predicted that vigilance is controlled by modifying the excitability of cortical layer 5 cells using acetylcholine (ACh) that is released there in response to mismatch events (Figure 7). The anatomical pathway leading to ACh release has been clarified by anatomical and neurophysiological studies in monkeys, cats and rats. These studies show that the nonspecific thalamus—in particular, the midline and central lateral thalamic nuclei—are sensitive to the degree of mismatch (Figures 6, 7), and project to the cholinergic nucleus basalis of Meynert (Figure 7; Van der Werf et al., 2002) which, in turn, is one of the main sources of cholinergic innervation to layer 5 of the cerebral cortex.

How ACh may influence vigilance is illustrated by experiments of Saar et al. (2001), who have shown that ACh release reduces the after-hyperpolarization (AHP) current and increases cell excitability in cortical layer 5 cortical cells. In SMART, this increased layer 5 excitability due to predictive mismatch may cause reset via the layer 5-to-6^I-to-4 circuit (Figures 6, 7), even in cases where top-down feedback may earlier have sufficiently matched bottom-up input, which is a key property of vigilance control. An increase of ACh might hereby trigger a search for finer recognition categories in response to environmental feedback, even when bottom-up and top-down signals have a pretty good match in the nonspecific thalamus based on similarity alone.

Grossberg and Versace (2008) used the SMART model to make other predictions that are related to the ACh-modulated vigilance prediction, and that may be combined with it to design new kinds of experimental tests. In particular, within SMART, a good enough top-down attentive match with a bottom-up feature pattern yields (faster) gamma oscillations, whereas a big enough mismatch-and-reset yields (slower) beta oscillations. The model also predicts how the mismatch is triggered in deeper layers of cortex (Figure 7B) before the reset can reorganize processing in all the cortical layers (Figure 7C). Both types of oscillations are emergent properties of network dynamics.

Grossberg (2013, Section 38) reviews the fact that this predicted match–mismatch gamma–beta dichotomy has subsequently been supported by neurophysiological experiments from at least three laboratories recording in three different parts of the brain. These reports include data about higher amounts of average gamma in superficial cortical layers vs. more beta (or alpha) power in deep cortical layers of cortical area V1 (Buffalo et al., 2011), beta synchronized with spatial attention shifts in the frontal eye fields (Buschman and Miller, 2009), and an inverted-U in beta power through time during learning of place cells in hippocampus during the navigation of novel spatial environments (Berke et al., 2008). Combining the gamma-beta prediction with the ACh-vigilance prediction suggests that an ACh-modulated increase of vigilance will, other things being equal, reduce the amount of gamma relative to beta power by increasing the number of mismatch reset events per unit time.

The match–mismatch gamma–beta prediction can also be tested by redoing the experiments of Spitzer et al. (1988) in which neurophysiological data are recorded from cortical area V4 during the learning by monkeys of easy vs. difficult discriminations. These authors report that “in the difficult condition, the animals adopted a stricter internal criterion for discriminating matching from non-matching stimuli… The animal’s internal representations of the stimuli were better separated… increased effort appeared to cause enhancement of the responses and sharpened selectivity for attended stimuli…” These are all properties of ART vigilance control, where higher vigilance is needed to make more difficult discriminations. These V4 effects could, for example, be due to vigilance-modulated selective attention from inferotemporal cortex (IT). Such object attention is focused via vigilance-modulated top-down expectations that are learned along with the bottom-up adaptive pathways that select IT recognition categories. Other things being equal, one would expect to find more beta power in the difficult than the easy discrimination condition. Direct activation of nucleus basalis or appropriate ACh target neurons might also be expected to increase beta power.

How does a reduction of AHP current by ACh increase cell excitability in cortical layer 5 cortical cells (Saar et al., 2001)? Many cortical networks contain recurrent on-center off-surround networks that transform input patterns before storing them in STM. A series of theorems in the 1970s (e.g., Grossberg, 1973; Grossberg and Levine, 1975) proved how the feedback signal functions in rate-based recurrent on-center off-surround networks control this STM storage process. In particular, sigmoid signal functions in such networks induce a quenching threshold below which inputs are suppressed as noise and above which they are contrast-enhanced before being stored in STM. A sudden decrease of the quenching threshold, other things being equal, can cause a reset burst and thus act like an increase in vigilance.

What action of ACh modulation could cause a sudden decrease in quenching threshold? If a mismatch-mediated burst of ACh suitably changes the shape of a sigmoid signal function, say by decreasing its threshold and increasing its slope, then that may decrease the quenching threshold.

A sigmoid signal can be explicitly defined in a rate-based neural network. In contrast, in an unlumped network composed of spiking neurons, the shape of the signal function is an emergent property of several factors that interact together. Palma et al. (2012a,b) showed how AHP currents that act on three different time scales can together control sigmoid signal threshold and slope in networks of spiking neurons. These properties clarify how activation of ACh by basal forebrain circuits, notably nucleus basalis of Meynert, may alter the brain’s sensitivity to predictive mismatches, and thus the vigilance with which the learning of recognition categories is modulated in the brain.

These AHP currents are predominantly carried by calcium-dependent potassium channels (Hotson and Prince, 1980; Lancaster and Adams, 1986), but also partly by calcium-independent potassium currents (Lorenzon and Foehring, 1992, 1995). In layer 5 Betz cells of cat sensorimotor cortex (Schwindt et al., 1988) and pyramidal cells in layers 3–6 of human neocortex (Lorenzon and Foehring, 1992) three distinct AHP currents—a fast (fAHP), medium (mAHP), and slow current (sAHP)—have been identified. Similar mAHP and sAHP currents occur in rat slices (Storm, 1987, 1989; Lee et al., 2005). How the different time courses of these AHP currents arise has not been completely explained, but proximity to calcium channels may be a key factor rather than the time constant of calcium binding to the channels themselves (Lima and Marrion, 2007).

The Palma et al. (2012b) simulations demonstrated that, in spiking neurons, a leftward threshold shift in the sigmoid signal occurs when the sAHP and mAHP currents decrease, as the fAHP current increases, whereas the slope of the transfer function becomes steeper when the sAHP and fAHP currents decrease, as the mAHP current increases. Both of these changes can reduce the quenching threshold and thereby increase vigilance.

At the other end of the continuum from high vigilance and environmentally-sensitive attention, learning, and recognition are the complex phenomena that occur during sleep. A full analysis of the complexities of sleep is not attempted here. Rather, relevant sleep data are summarized to set the stage for the proposal that specific properties of laminar neocortical circuits that have been used to simulate data about perception and cognition in awake individuals (Figures 5–7) may also shed light on the dynamics of slow wave sleep. This proposed connection suggests that these properties of sleep may emerge from vigilance changes in the same model laminar neocortical circuits that have successfully modeled various awake behaviors.

Relevant sleep data include the following: Steriade (2004, p. 179) has noted that: “Fast rhythms (20–60 Hz) appear during the sustained depolarization of thalamic and neocortical neurons during brain-active states that are accompanied by increased release of ACh in the thalamus and cerebral cortex. Such fast rhythms also occur during the depolarizing phases of the slow oscillation (0.5–1 Hz) in non-REM sleep. Intracellular recordings of neocortical neurons during natural states of waking and sleep demonstrate stable and increased input resistance of corticocortical and corticothalamic neurons during the sustained depolarization in wakefulness, compared to the depolarizing phase of the slow oscillation in non-REM sleep. Despite the highly increased synaptic inputs along different afferent systems that open many conductances of cortical neurons during wakefulness, the increased input resistance is attributed to the effect of ACh on cortical neurons”.

An earlier study of Metherate et al. (1992, p. 4701) reported compatible data: “In the mammalian neocortex, the EEG reflects the state of behavioral arousal. The EEG undergoes a transformation, known as activation, during the transition from sleep to waking. Abundant evidence indicates the involvement of the neurotransmitter acetylcholine (ACh) in EEG activation; however, the cellular basis of this involvement remains unclear. We have used electrophysiological techniques with in vivo and in vitro preparations to demonstrate actions of endogenous ACh on neurons in auditory neocortex. In vivo stimulation of the nucleus basalis (NB), a primary source of neocortical ACh: (1) elicited EEG activation via cortical muscarinic receptors; (2) depolarized cortical neurons; and (3) produced a change in subthreshold membrane potential fluctuations from large-amplitude, slow (1–5 Hz) oscillations to low-amplitude, fast (20–40 Hz) oscillations. The NB-mediated change in pattern of membrane potential fluctuations resulted in a shift of spike discharge pattern from phasic to tonic. Stimulation of afferents in the in vitro neocortex elicited cholinergic actions on putative layer 5 pyramidal neurons. Acting via muscarinic receptors, endogenous ACh: (1) reduced slow, rhythmic burst discharge and facilitated higher-frequency, single-spike discharge in burst-generating neurons; and (2) facilitated the appearance and magnitude of intrinsic membrane potential oscillations. These in vivo and in vitro observations suggest that neocortical activation results from muscarinic modulation of intrinsic neural oscillations and firing modes…”

Further quantification of the role of basal forebrain ACh release during wakefulness and sleep was provided by the study of Vazquez and Baghdoyan (2001, p. R598) who reported that: “Cortical ACh release is greatest during waking and rapid eye movement (REM) sleep and reduced during non-REM (NREM) sleep. Loss of basal forebrain cholinergic neurons contributes to sleep disruption and to the cognitive deficits of many neurological disorders. ACh release within the basal forebrain previously has not been quantified during sleep. This study used in vivo microdialysis to test the hypothesis that basal forebrain ACh release varies as a function of sleep and waking. Cats were trained to sleep in a head-stable position, and dialysis samples were collected during polygraphically defined states of waking, NREM sleep, and REM sleep. Results from 22 experiments in four animals demonstrated that means ± SE ACh release (pmol/10 min) was greatest during REM sleep (0.77 ± 0.07), intermediate during waking (0.58 ± 0.03), and lowest during NREM sleep (0.34 ± 0.01)…”. The fact that ACh release was greater during REM sleep than wakefulness is a reminder that conscious awareness does not depend only on ACh modulation. Neural models describe some of the other brain processes that are proposed to play a key role during conscious awareness. For recent reviews, see Grossberg (2016, 2017).

More recent experiments provide further details confirming the causal relationship between cholinergic neurons of the basal forebrain and sleep homeostasis (e.g., Kalinchuk et al., 2015; Nair et al., 2016).

This section and the next one describe how laminar cortical ART models that provide a unified explanation in the awake individual of processes of boundary completion, contrast normalization, attentional modulation, decision-making, activity-dependent habituation, and mismatch-mediated reset, also exhibit properties of Up and Down states during slow wave sleep. First some additional data about slow wave sleep are reviewed before suggesting how they may arise from mechanisms that have earlier been characterized for the above processes in awake individuals.

As noted above, NREM sleep often exhibits a slow <1 Hz rhythm in the EEG that appears to have a cortical origin (Steriade, 1991; Steriade et al., 1993a,b; Timofeev and Steriade, 1996; Sanchez-Vives and McCormick, 2000; Crunelli and Hughes, 2010; Sanchez-Vives and Mattia, 2014). This slow rhythm has been proposed to carry out multiple functions, ranging from metabolic clearance from the brain (Xie et al., 2013) to memory consolidation (Steriade and Timofeev, 2003; Marshall et al., 2006; Franklin and Grossberg, 2017). Lesions of the basal forebrain cholinergic nuclei, or pharmacological block of muscarinic receptors, lead to such slow waves throughout the neocortex (Buzsaki et al., 1988; Vanderwolf, 1988).

Slow wave generation in layer 5 (Calvet et al., 1973; Ball et al., 1977; Rappelsberger et al., 1982) is supported by bursting pyramidal cells in layer 5 that synchronize activity across the neocortex (e.g., Connors, 1984; Chagnac-Amitai and Connors, 1989; Silva et al., 1991; Wang and McCormick, 1993). Intrinsically bursting cells have also been reported in layer 3 (Steriade et al., 1993a,b). In the transition from sleep to wakefulness, this spontaneous burst firing is replaced by the spike sequences that are frequency-modulated by input intensities and related parameters. Livingstone and Hubel (1981) have, for example, reported a marked increase in activity in layers 5 and 6 during wakefulness as compared to during sleep. This transition is supported by a significant increase in the extracellular concentration of ACh (Szerb, 1967; Phillis, 1968).

An important property of the Up and Down states of slow wave sleep is that, as Volgushev et al. (2006, p. 5671) have observed: “all cells, excitatory as well as inhibitory, were involved in the same slow rhythm, and we never observed a cell to be systematically active while other neurons were silent…high synchrony of the silent state onsets implies the existence of a network mechanism that switches activity to silence… This would lead to termination of activity of both excitatory and inhibitory cells, including those cells that have generated the silencing discharge”. These authors go on to review evidence that bursting neurons in layer 5 large pyramidal cells initiate this activity cycle, and that fast-spiking inhibitory interneurons have an early onset during the subsequent Up and Down states. These dynamics are, moreover, due to intracortical interactions that occur even when thalamic gates do not permit the intrusion of signals from the outside world during slow wave sleep (e.g., Steriade and Timofeev, 2003). An intracortical origin for these slow waves is also supported by their survival after extensive thalamic lesions (Steriade et al., 1993b) and by the absence of slow waves in the thalamus of decorticated cats (Timofeev and Steriade, 1996). In their mean field model of the Up-Down cycle, Sanchez-Vives and Mattia (2014) have also observed a role for “activity-dependent fatigue” in regulating the relative duration of the Down state.

In order to more fully understand how such properties of slow wave sleep may arise, it is informative to link them to laminar neocortical circuits that also play important functional roles during the awake state. Ideally, many properties of slow wave sleep could then be shown to arise as natural emergent properties of these cortical circuits, in just the same way that their properties during awake states do. Such a connection will be outlined below. This proposed link between awake and sleep states also sheds some light on the following questions: Why do not the bursts that occur during slow wave sleep cause movement, let alone consciousness, especially given that layer 5 cells project to movement causing areas of the brain such as the superior colliculus and spinal cord, as well as subcortical targets such as the pontine nuclei (Wang and McCormick, 1993)? Multiple mechanisms may contribute to this result, but it is shown here how a prescribed intracortical mechanism may also play a role.

Several experimental and modeling studies have contributed insights into the cellular mechanisms of slow wave sleep and how it may transition to the waking state (e.g., Bazhenov et al., 2002; Esser et al., 2009). These studies have not, however, tried to connect their studies of sleep to laminar cortical models that have been used to quantitatively simulate psychological and neurophysiological data that have been collected during awake behaviors. Such a connection is made here, and adds new concepts and circuit mechanisms to this discussion. In particular, these are network mechanisms that seem to be conserved in multiple neocortical regions, and that can help to explain synchronization and wave-like properties of slow wave sleep across the neocortex (Massimini et al., 2004; Volgushev et al., 2006).

The discussion in this section will focus on the laminar cortical circuits in the visual cortex that are modeled by the LAMINART model, and its 3D LAMINART extension. Similar laminar circuits, with suitable specializations, have been used to simulate speech perception data using the cARTWORD model, and data about cognitive working memory and list chunking using the LIST PARSE model (Section 7). These mechanisms thus seem to be conserved, albeit with suitable specializations to carry out different psychological functions, across at least the granular neocortex, with its characteristic architecture of six layers of cells and their sublaminae, and the characteristic bottom-up, top-down, and horizontal interactions that enable their functional capabilities (Brodmann, 1909; Martin, 1989).

Two kinds of interactions within layers 2/3 and 5/6 are of special interest to the explanatory goals herein: Long-range excitatory horizontal connections, and shorter-range self-normalizing networks that balance excitatory and inhibitory interactions. The long-range horizontal connections within cortical layers 2/3 in LAMINART carry out the process of boundary completion, also called perceptual grouping (e.g., Grossberg and Raizada, 2000; Raizada and Grossberg, 2001, 2003; Grossberg and Swaminathan, 2004; Grossberg and Yazdanbakhsh, 2005; Fang and Grossberg, 2009; Léveillé et al., 2010). In addition, perceptual grouping can support a traveling wave in response to a spatial focus of attention (Grossberg and Raizada, 2000, Figure 3; Roelfsema et al., 1998). Shorter-range excitatory horizontal connections in layers 2/3 and 5/6 carry out processes such as binocular fusion (Grossberg and Howe, 2003, Figure 18). These features of cells in cortical layers 2/3 and 5/6 take on new significance in the context of sleep research for at least two reasons: (1) layers 3 and 5 contain intrinsically bursting neurons that may be influenced by ACh modulation (Calvet et al., 1973; Ball et al., 1977; Rappelsberger et al., 1982; Connors, 1984; Chagnac-Amitai and Connors, 1989; Silva et al., 1991; Steriade et al., 1993a,b; Wang and McCormick, 1993); (2) long-range excitatory connections may help to both synchronize the ACh-modulated slow waves within local cortical networks, and to facilitate the propagation of these synchronous activations in traveling waves across the cortex (Massimini et al., 2004; Volgushev et al., 2006).

The networks with self-normalizing balanced excitatory and recurrent inhibitory interneurons are at least as important from the present perspective. They occur, with a similar design, in both layers 2/3 and 5/6. In layers 2/3, they are part of a larger circuit that also includes the long-range excitatory horizontal connections. Figures 5C,E simplify the depiction of the inhibition in layer 2/3 as a single inhibitory interneuron. Together with the long-range horizontal connections, the shorter-range recurrent inhibitory network helps to realize the bipole property that enables perceptual grouping to occur in the following way (Figure 8; Grossberg, 1984a; von der Heydt et al., 1984; Grossberg and Mingolla, 1985; Peterhans and von der Heydt, 1989).

In Figure 8A, a single pac man figure (in black) induces a boundary representation using simple and complex cells in cortical area V1 whose orientation preferences are the same as, or similar to, those of the pac man’s bounding contour. This boundary representation activates long-range excitatory horizontal connections in layer 2/3 of cortical area V2. Excitatory signals in these horizontal connections also excite inhibitory interneurons on their way to target cells in layer 2/3. The inhibition from the disynaptic inhibitory interneurons balances the excitation from the long-range horizontal connection, thereby preventing the target cell from firing. It is a case of one-against-one. By this mechanism, a single contrast in an image is prevented from creating outwardly spreading boundaries across the entire image.

In Figure 8B, two like-oriented pac man figures are aligned across space, and are sufficiently near one another to complete a boundary between them. In this case, the excitatory signals from their long-range horizontal connections summate at the target cell. The disynaptic inhibitory signals from the inhibitory interneurons also summate there. Why, then, does not the total inhibition again cancel the total excitation? This is because—and this is the main point—the inhibitory interneurons also inhibit each other to form a recurrent lateral inhibitory network. It was mathematically proved in Grossberg (1973; see also Grossberg, 1980) that, if the inhibitory cells in such a recurrent network obey the membrane equations of neurophysiology—also called shunting dynamics—then their total activity tends to be normalized, and independent of the number of active inhibitory cells. Thus, although the total excitation increases, the total inhibition does not. It is a case of two-against-one, so excitation wins and the boundary can form.

Layers 5/6 also include the same kind of recurrent self-normalizing circuit. This kind of circuit, in fact, seems to occur in many parts of the neocortex. In Figures 5A–C,E, these inhibitory interneurons are drawn as a feedforward network, for simplicity. In actuality, they also inhibit each other in the LAMINART model, thereby creating self-normalizing total inhibition. This property helps to ensure properties in the adult of contrast normalization, attentional modulation, decision-making, activity-dependent habituation, and mismatch-mediated reset in a wide variety of visual processes (e.g., Francis et al., 1994; Francis and Grossberg, 1996; Grossberg, 2003, 2016; Cao and Grossberg, 2005, 2012; Grossberg and Yazdanbakhsh, 2005; Grossberg and Pilly, 2008; Grossberg and Versace, 2008).

The need for such self-normalizing inhibition can be understood by considering Figure 5A. Here, the network from layer 6-to-4 is supposed to create a modulatory on-center as part of the on-center off-surround network that inputs to layer 4. Such a modulatory on-center requires that excitatory and inhibitory inputs are approximately equal, or balanced, to the on-center cells. The lateral geniculate nucleus (LGN) is able to fire target cells in layer 4 by also activating a direct LGN-to-4 excitatory pathway. The indirect LGN-to-6-to-4 is not useless, however. The direct LGN-to-4 pathway, combined with the LGN-to-6-to-4 network, can both activate and contrast-normalize the inputs to layer 4 cells in response to input patterns from the LGN.

Why is the combination of a direct LGN-to-4 network and a modulatory on-center off-surround network from LGN-to-6-to-4 needed? Why could not the LGN-to-6-to-4 network be designed so that its excitatory on-center could, all by itself, fire target layer 4 cells, thereby eliminating the need for a separate LGN-to-4 pathway? This can be understood by considering how the modulatory on-center in the 6-to-4 network helps to realize top-down attention via the ART Matching Rule in Figure 5B. As noted in Section 2.2, the ART Matching Rule asserts that top-down object attention is achieved using a top-down, modulatory on-center, off-surround network in order to achieve stable learning and memory. The excitation and inhibition in the on-center of the layer 6-to-4 network thus need to be approximately balanced. This constraint clarifies the need for a direct LGN-to-4 pathway that is sufficient to fire target layer 4 cells.

The above considerations explain why the layer 6-to-4 on-center is modulatory, but not why the off-surround needs to be a recurrent inhibitory network. The main design constraint that forces recurrence is that the modulatory requirement must be maintained both when multiple input sources contribute simultaneously to the total top-down attentional priming signal, or to the total bottom-up feature pattern. Under these variable load conditions, the modulatory requirement would fail if the inhibitory network was feedforward. In particular, if the total activity of off-surround cells grew proportionally with the number of cells inputting to it, then the excitatory inputs to the on-center would be overwhelmed. Self-normalization of total inhibition by the recurrent inhibitory network, no matter how many input sources are active, solves this problem.

Balancing excitation and inhibition in these circuits achieves several other useful functional roles. For starters, it helps to avoid seizures that could result if excitation was too much stronger than inhibition, or persistent depression if inhibition was too much stronger than excitation. Such a balance also maintains the cortex at a “cusp of excitability” wherein enough activation occurs to maintain homeostatic plasticity without triggering undesired behavioral consequences, while also enabling the cortex to respond efficiently and vigorously to external inputs when they do occur. Indeed, the laminar cortical model of Grossberg and Williamson (2001) showed how the desired balance of excitation and inhibition could develop for carrying out bipole grouping circuits in layers 2/3 and contrast-normalizing attentional modulation in layers 6-to-4. These developed circuits were then used in that article and subsequent ones (e.g., Grossberg and Raizada, 2000; Raizada and Grossberg, 2001) to simulate a wide range of psychophysical and neurophysiological data about adult visual perception and attention, thereby showing how LAMINART models can provide a unified explanation of data about both cortical development and adult attention and visual perception.

It is also worth noting that a recurrent inhibitory network is probably easier to grow during development than a feedforward network, since inhibitory connections can then grow to all nearby cells, both excitatory and inhibitory, not just to excitatory cells.

These recurrent inhibitory circuits naturally lead to properties of Up and Down states during slow wave sleep (Steriade et al., 1993b; Timofeev and Steriade, 1996; Steriade and Timofeev, 2003; Volgushev et al., 2006), including the fast reaction of inhibitory interneurons, and the silencing of both excitatory and inhibitory neurons during the Down state. The simplest version of such a recurrent inhibitory circuit between layers 6 and 4 also includes ACh-modulated inputs from layer 5 cells (see Figure 7D) that intermittently burst during slow wave sleep (Figure 9).

The analysis herein does not explicitly model how layer 5 cells generate intermittent bursts during slow wave sleep as a result of ACh down-regulation, among other factors. Other modeling articles make proposals about how this may happen; e.g., Bazhenov et al. (2002) and Esser et al. (2009). Given such a excitatory burst emanating from the layer 5 cell in Figure 9, the burst inputs to both the on-center and the off-surround of the network. If the inhibitory interneurons obey dynamics of fast spiking cells, they may thus begin to be activated a little before the excitatory on-center cells. Despite such a brief advantage, the on-center cells can nonetheless begin to fire before signals from the inhibitory interneurons can take effect at the on-center, due to the extra stage of axonal delay in the off-surround anatomy. This period of on-center activation is an Up state.

This Up state is transient, however, due to the transient nature of the layer 5 excitatory input burst, combined with the balanced total excitatory and inhibitory inputs to the on-center. When the inhibitory interneuronal signals take effect at the on-center cells, the on-center cells will be completely inhibited due to this balance. As the on-center cells are inhibited, so too will the inhibitory interneurons by the recurrent inhibitory feedback that they deliver to each other. When both on-center and off-surround cells are silenced, a Down state is created.

Another relevant factor is that the layer 6-to-4 connections in the LAMINART model are gated by activity-dependent habituative transmitters. These habituating gates help to explain many data, ranging from data about visual persistence (Francis et al., 1994; Francis and Grossberg, 1996) to mismatch-mediated reset (Grossberg, 1976; Grossberg and Versace, 2008). In the case of slow wave sleep, such habituation may influence the duration of the Down state (Sanchez-Vives and Mattia, 2014).

A great deal more modeling needs to be done to characterize the quantitative properties of the proposed network substrate in Figure 9 of Up and Down states during slow wave sleep. For the present, the most important fact is that this proposal uses the same laminar cortical networks that have also been used to quantitatively simulate many data about cortical development, perception, and cognition in awake individuals. This linkage raises new and fascinating questions. For example, the same modulatory on-center, off-surround network in layers 6-to-4 that is used in Figure 9 to qualitatively describe some properties of Up and Down states has also been used to explain and quantitatively simulate many data in awake animals and humans about contrast normalization, attentional modulation, decision-making, activity-dependent habituation, and mismatch-mediated reset, among other properties. It will be instructive in future studies to see if the same network parameters that fit data in the awake state also succeed in providing quantitative fits to data about the durations of Up and Down states during slow wave sleep, and how differences in ACh modulation in the awake and sleep states influence these results.

More modeling also needs to be done to unify neural mechanisms of circadian rhythms and slow wave sleep, and the complex relationships that coordinate these processes (e.g., Achermann et al., 1993; Deboer et al., 2003). Neural models have proposed how the clock-like dynamics of the suprachiasmatic nuclei (SCN) in the hypothalamus are modulated by homeostatic factors like activity-dependent fatigue signals and non-homeostatic factors like the timing of light and dark episodes, to simulate the intensity, duration, and patterning of ultradian activity-rest cycles and the duration of circadian periods under a wide range of light-dark conditions (Carpenter and Grossberg, 1983, 1984, 1985).

This Gated Pacemaker theory of circadian rhythms also suggests how circadian and appetitive hypothalamic circuits may both be constructed from similar components, albeit being specialized to carry out their respective tasks. Indeed, the homeostatic and non-homeostatic inputs in the SCN circuit have homologs in satiety and sensory inputs, respectively, in the appetitive circuits. Within this modeling framework, it can be seen how the SCN model circuits can modulate the sensitivity of the appetitive circuits during the control of a wide range of cognitive-emotional interactions that have also been modeled (e.g., Grossberg, 1972a,b, 1982, 1984b, 2000b; Grossberg and Seidman, 2006; Dranias et al., 2008; Grossberg et al., 2008a).

These cognitive-emotional interactions also include the nSTART model (Figure 4) of the memory consolidation dynamics that occur during sleep and how they are influenced by early or late hippocampal, amygdala, orbitofrontal, or thalamic lesions (Franklin and Grossberg, 2017). It remains to carry out the large-scale modeling program that would be able to unify these emerging strands of theoretical insight that each link brain mechanisms to the psychological functions that they support.

Yet another important linkage that remains to be fully articulated is one between circadian rhythms, sleep, and Alzheimer’s disease, as the next section will discuss. Of particular interest from the present perspective is that major structural degeneration in the brains of Alzheimer’s patients occurs in cortical layers 3 and 5, which are also layers where vigilance control is mediated and where key properties of slow wave sleep seem to be generated (Figure 9).

Alzheimer’s disease includes significant structural abnormalities in many brain regions that may be caused by complex interacting factors. Some of that experimental evidence is reviewed here. In addition, data are reviewed pointing to how this structural degeneration may cause a severe degeneration of cholinergic function in areas of the cerebral cortex, including temporal cortex, that are critical for learning, recognizing, and recalling all sorts of information about the world. Much of the literature focuses on these degenerative structural events. ART clarifies dynamical consequences of these structural events. Indeed, a failure of both tonic and phasic vigilance control could cause a generalized collapse of cognitive processing, way beyond anything that would be caused by the imbalances in the dynamical processing of vigilance that may contribute to behavioral symptoms of autism and medial temporal amnesia.

An early article of Coyle et al. (1983) noted that Alzheimer’s disease was then already one of the most common causes of mental deterioration in the elderly. This article cited compelling evidence that ACh-releasing neurons, whose cell bodies lie in the basal forebrain, selectively degenerate and thereby influence widespread areas of the cerebral cortex and related structures that play an important role in cognitive functions, notably learning and memory. Whitehouse et al. (1982) additionally noted that postmortem studies demonstrated profound reduction in the presynaptic markers for cholinergic neurons in the cortex of patients with Alzheimer’s disease. In particular, neurons of the nucleus basalis of Meynert undergo a greater than 75 percent selective degeneration in these patients. Experimental studies using Alzheimer’s disease animal models have provided supportive data showing that both anticholinergic drugs and lesions of the nucleus basalis of Meynert disrupt learning or memory in a number of paradigms, including passive avoidance learning and Morris water maze tests (Lo Conte et al., 1982; Friedman et al., 1983). Francis et al. (1999), Iqbal and Grundke-Iqbal (2008), and Pimplikar (2009) review these and additional factors that contribute to the disease.

AChE is the main enzyme that can break down ACh in the brain. Inhibition of AChE is thus considered one of the treatment strategies to ameliorate symptoms of Alzheimer’s disease (e.g., Orhan et al., 2004; Mukherjee et al., 2007). Indeed, Janeczek et al. (2017) described an extensive network of cortical pyramidal neurons in the human brain with abundant AChE activity. They quantified the density and staining intensity of these neurons using histochemical procedures. Their methods enabled them to show that brains of adults above age 80 with unusually preserved memory performance (SuperAgers) showed significantly lower staining intensity and density of AChE neurons when compared with same-age peers, leading them to speculate that low levels of AChE activity could enhance the impact of ACh on pyramidal neurons to counterbalance factors that mediate the decline of memory capacity during normal aging.

Rabiei et al. (2014, p. 353) have reported studies that suggest a different approach to enhancing ACh function. They note that Zizyphus jujube (ZJ) activates choline acetyltransferase (ChAT), a transferase enzyme that is responsible for the synthesis of ACh. Their study investigates the effect of ZJ extract in intact rats and in a rat model of Alzheimer’s disease with lesions of the nucleus basalis of Meynert. Learning and memory performance were assessed using a passive avoidance paradigm, and spatial learning and memory were evaluated by the Morris water maze. The results suggested that ZJ has repairing effects on memory and behavioral disorders produced by the nucleus basalis of Meynert lesions, and therefore suggest that ZJ may also have beneficial effects treating Alzheimer’s patients.

Many studies (e.g., Wang et al., 2000a,b) have described the structural degeneration that accompanies and may cause ACh deficits in Alzheimer’s patients, including the role of the 42-amino acid β-amyloid peptide (Aβ_1–42) in the formation of neuritic plaques and neurodegeneration. Wang et al. (2000a,b) also noted that the α7 nicotinic acetylcholine receptor (α7nAChR) is highly expressed in the basal forebrain cholinergic neurons that project to the hippocampus and cortex of normal and Alzheimer brains (e.g., Perry et al., 1992), correlates well with brain areas that exhibit neuritic plaques in Alzeheimer’s disease, and modulates both calcium homeostasis and release of the neurotransmitter ACh, two important parameters involved in cognition and memory. Nagele et al. (2002) further studied how amyloid β_1–42 binds with exceptionally high affinity to α7nAChR and accumulates intracellularly in neurons of Alzheimer’s disease brains. Related studies have shown that amyloid peptides can inhibit the release of ACh (Kar et al., 1996).

The above results are consistent with results in Tomlinson et al. (1968, 1970) showing that plaques are more frequent in cortical layers 3 and 5, and in Arnold et al. (1991) summarizing that the laminar distribution of neurofibrillary tangles tended to be selective, involving primarily layers 3 and 5 of association areas; see also Rogers and Morrison (1985). These authors also report far more tangles in both limbic and temporal lobes than in frontal, parietal, and occipital lobes, but also suggested that plaques were more evenly distributed throughout the cortex.

In summary, all of these studies describe the kinds of structural degeneration in layers 3 and 5 that could lead to the massive collapse of ACh function and support of learning and memory that other studies have reported.

Ju et al. (2014, p. 115) have reviewed evidence that “the sleep-wake cycle directly influences levels of Aβ [amyloid-β peptide] in the brain. In experimental models, sleep deprivation increases the concentration of soluble Aβ and results in chronic accumulation of Aβ, whereas sleep extension has the opposite effect.” The authors go on to note that changes in sleep precede the onset of cognitive symptoms in Alzheimer’s patients, and that disrupted sleep patterns occur in patients with Alzheimer’s disease. Moreover, there is a diurnal variation in the level of soluble Aβ in the interstitial fluid. During the Down state of slow wave sleep, less Aβ is released than during wakefulness or REM sleep. A disruption of slow wave sleep can hereby lead to higher sustained extracellular concentrations of Aβ. These higher sustained Aβ concentrations are, in turn, associated with early amyloid plaque formation.

As noted above, plaques are found more frequently in cortical layers 3 and 5, where they can disrupt the ACh-mediated Up and Down oscillations in slow wave sleep that are diagrammed in Figure 9. By disrupting slow wave sleep in this way, and thereby indirectly causing higher Aβ extracellular concentrations, a vicious cycle can be perpetuated.

The article’s focus on vigilance control and its review of how normal ACh dynamics are devastated in Alzheimer’s patients raises a question about whether non-invasive behavioral activities that promote normal ACh dynamics, in addition to all appropriate drug and other medical interventions, may help to delay, or lower, the chances of developing Alzheimer’s disease? In other words, can some types of behavior better support the brain’s design to solve the stability-plasticity dilemma (Section 2.1), and thereby allow learning to continue throughout life without a loss of memory stability?

Since unexpected events phasically raise vigilance by releasing ACh (Figure 7), it is natural to ask whether novelty-seeking activities can help to delay, or lower, the chances of developing Alzheimer’s disease? Some data suggest that this may be the case (e.g., Fritsch et al., 2005), while other studies report diminished novelty-seeking behavior in patients with probable Alzheimer’s disease, that is distinct from general cognitive decline (Daffner et al., 1999). It may thus be of use to consider whether and how novelty-rich experiences can be designed to help slow the onset of Alzheimer’s.