Evolutionary transitions of social behavior can introduce novel cognitive demands (Silk, 2007). Metabolic costs of neural tissue are disproportionately high and brain tissue investment should be constrained to meet cognitive demands (Niven and Laughlin, 2008; Iwaniuk, 2017). Origins or modification of sociality may drive changes in brain tissue investment. Several hypotheses have been advanced to explain changes in neural tissue investment associated with transitions in social behavior, generally referred to as “social brain” hypotheses (Dunbar, 1998). Initially, social brain hypotheses assumed there would be a positive effect of sociality on brain investment, but more recent theoretical treatments (Gronenberg and Riveros, 2009; Lihoreau et al., 2012) and empirical findings (vertebrates: Gonzalez-Voyer et al., 2009; Fedorova et al., 2017; Kverková et al., 2018; social insects: O’Donnell et al., 2015) have called this assumption into question.

Studies to date have not adequately identified or accounted for varying behavioral selective pressures that could change cognitive demands as sociality evolves. Analyzing the diversity of social interactions can contribute to more rigorous theory linking social environments with brain evolution. We suggest comparative tests of social brain hypotheses should explicitly consider which aspects of behavior change during transitions from solitary to group living, or during transitions in social complexity, in each study taxon. We identify some key features of social systems that can act as independent selective forces on neural investment. We explore how each of the following attributes of social systems can affect cognitive demands and thus selection for neural investment: intragroup competition, individual recognition, information sharing, dominance hierarchies, and task performance specialization and redundancy. We predict that some of these behavioral attributes will decrease the fitness effects of individual cognitive abilities, such that selection for neural investment will be relaxed. In contrast, other attributes of sociality may place greater cognitive demands on individuals, selecting for increased neural investment.

It is important to note that few, if any, of the behavioral attributes we explore are universally present among social taxa. The relevance of these behavioral attributes vary among animal taxa, depending in part on how social groups form and function. An important complicating factor is the role of individual vs. group/colony selection as the main determinant of fitness in a species (O’Donnell et al., 2018). Variation among animal lineages in the structure of social groups provides opportunities for cross-taxon comparative analyses, which can be used to assess the relative importance of different behavioral factors as determinants of neural investment.

Our paper is divided into two main sections. In the first section we present some attributes of social behavior that could independently impact cognitive demands on animals, either increasing or relaxing selection for brain investment. We discuss potential effects of these demands on the evolution of brain investment. The second section is an empirical test of social brain effects in a taxon with diverse grades of sociality and a relatively simple social system (cobweb spiders: Avilés and Guevara, 2017). Social cobweb spider species exhibit task sharing and redundancy among females; social males and subsocial species do not (Samuk et al., 2011; Harwood and Avilés, 2013, 2018). Furthermore, social spiders do not exhibit some potentially confounding elements of sociality, such as dominance hierarchies and task specialization (Lubin, 1995; Settepani et al., 2013) that could complicate the interpretation of social brain effects comparisons.

The evolution of social behavior potentially increases the frequency or intensity of competition between animals, and social competitive costs must nearly always be compensated by other fitness payoffs to favor sociality (Silk, 2007). Social group participation can drive increased brain tissue investment, either in response to or in anticipation of competitive interactions. Greater neural investment can improve performance on behavioral tasks, such as mate location, and may confer individual benefits in highly competitive social environments (Kotrschal et al., 2015). Potential relationships of heightened competition with brain investment are illustrated by species where social groups involve temporary or facultative aggregations that can increase local competition for food or other resources. In desert locusts, the transition from solitary living to the formation of massive swarms that compete for food and engage in cannibalism (Guttal et al., 2012) is accompanied by increased total brain investment and greater relative investment in the mushroom bodies, which are cognitive integrating regions of the insect brain (Ott and Rogers, 2010). Experimental manipulations that induce sociality in species that are typically solitary have found similar increases in cognitive performance or neural investment in social groups. In Drosophila melanogaster fruit flies, larval crowding resulted in increased food competition and was associated with greater adult neural investment (Heisenberg et al., 1995); juvenile Marpissa muscosa jumping spiders reared in groups had significantly improved performance on learning assays although competition was not directly implicated in this study (Liedtke and Schneider, 2017).

Individual recognition can be cognitively demanding because it requires discriminating and remembering the individual identities of social partners (Shultz and Dunbar, 2006; Bergman and Beehner, 2015). In addition to the number of unique relationships, qualitative differences in interactions between group members can impose different cognitive demands. For example, monogamous pair bonds are common among birds, but species vary in the level of pairs’ cooperation and affiliative behaviors (Emery et al., 2007). Emery et al. (2007) proposed that increased total brain investment in some bird species can be attributed to a higher degree of cooperation during brood care, increased affiliative displays, and social support interactions among mates. Social alliances in particular may be expected to impose higher cognitive demands because of the necessity to remember group member identities, predict the outcome of interactions, and recognize others with similar status (Chapais, 1995).

Dominance hierarchies describe ranks of group members in relation to one another (Drews, 1993). The criteria used to define a “dominant position” vary among species but may include increased access to food resources or reproductive opportunities relative to other members of a group (Drews, 1993). Achieving dominant status and maintaining a hierarchy may increase cognitive demands (and brain investment) by requiring dominant individuals to differentiate group member relationships (Dunbar, 1992). Socially dominant positions in hierarchies are associated with increased neural investment in socially flexible bees relative to both solitary and subordinate individuals (Smith et al., 2010; Rehan et al., 2015). A loss of sociality in a primitively social bee lineage resulted in decreased foundress neural investment, potentially due to the loss of dominance interactions (Pahlke et al., 2019, 2020).

The expected relationship of dominance behavior with brain investment is complicated because dominance hierarchies do not always rely on cognitively demanding, individualized aggressive behaviors. Some hierarchies rely partially or entirely on chemical signaling (Faulkes and Bennett, 2001; Sundström and Boomsma, 2001; Orlova et al., 2020). In hierarchies which do not rely on aggressive interactions, selection for neural investment to navigate social interactions may be minimal (Lihoreau et al., 2012). An additional complication is that dominance status can affect other aspects of behavior such as task performance. For example, reproductively-dominant social insect queens may perform few behaviors other than egg laying (Noirot, 1989; Barchuk et al., 2018), potentially permitting reduced brain investment for dominants compared to subordinates (Barchuk et al., 2018; Gordon et al., 2019; but see O’Donnell et al., 2017). In these systems, neural investment in dominants and subordinates may no longer be driven by social interactions, but instead by caste-specific division of labor (thus falling under the task specialization attribute, described below).

Information sharing allows social animals to respond adaptively to stimuli they have not experienced directly. Information sharing can decrease the fitness costs of inaccurate stimulus detection and may reduce selection for individual cognitive abilities, permitting decreased brain tissue investment. The effect of group information sharing on cognitive demands is well-illustrated by vigilance behaviors. Individual vigilance behavior performance rates decrease in the presence of conspecifics in some species (McBlain et al., 2020; Wang et al., 2021), and predator detection can be faster and more accurate in social groups compared to lone individuals (Ward et al., 2011). However, the need for increased ability to produce, detect and respond to social signals can involve additional cognitive and neural costs. In this case, social and environmental information may be integrated by different parts of the brain, resulting in distinct patterns of mosaic brain evolution: in cichlid fish, environmental complexity is associated with increased telencephalon investment, while increased social complexity is associated with increased hypothalamus investment (Pollen et al., 2007). Thus, for systems in which information sharing occurs, assessing changes in specific brain regions, rather than total brain investment, may better detect the effects of changes in cognitive demands on brain investment.

A common theme in some taxa is for social group members to specialize on a subset of the tasks typically performed by solitary-living individuals (Jeanne, 1986). Individuals may specialize on specific behaviors for all or part of their lifetime; in extreme cases, the evolution of morphologically distinct castes in some social insects limits workers to performing certain behaviors or tasks (Korb and Thorne, 2017; O’Donnell et al., 2018; Gordon et al., 2019). Task specialization is often paired with neural adaptations to meet the cognitive demands of a specific behavioral class (Amador-Vargas et al., 2015; Iwaniuk, 2017), but may result in decreased selection for general cognition (Gronenberg and Riveros, 2009). Task specialization may improve colony-level efficiency (Keller et al., 2011), and decreases in individual neural investment can represent a savings of resource investment by the social group. Reduced individual neural investment may mark a transition to colony-level selection on neural investment (O’Donnell et al., 2015, 2018).

Redundancy, or the availability of multiple individuals to perform or complete a given task, may also reduce individual cognitive demands. Redundancy can facilitate high probabilities of successful task completion even when individual competency is relatively low (Herbers, 1981). Redundancy in colony labor is often present in caste-specialist systems and may be critical for effective task partitioning among group mates (Jeanne, 1999), but redundancy is not restricted to task-specialist systems. Redundancy applies whenever the completion of a shared task is not the responsibility of a single individual, such as in cooperatively breeding social groups (Taborsky et al., 2007; Riehl, 2013). Decreased neural investment has been associated with living in stable groups in woodpeckers (Fedorova et al., 2017), and in cichlid fishes who engage in biparental care (Gonzalez-Voyer et al., 2009). In both of these systems redundancy to complete tasks may reduce individual cognitive demands. In fact, the evolution of redundant cooperation may be more likely to occur when individuals are unaware of group-mates’ social strategies (Pedroso, 2020).

Several evolutionary transitions between subsocial and social behavior have occurred in the cobweb spider genus Anelosimus (Figure 1). In subsocial species, individual females perform extended maternal care but offspring disperse prior to or at adulthood (Yip and Rayor, 2014; Avilés and Guevara, 2017). In social species the offspring do not disperse, leading to the maintenance of permanently communal nests. Social species adult females cooperatively capture prey and share offspring care (Avilés and Tufino, 1998; Yip et al., 2008; Samuk and Avilés, 2013; Avilés and Guevara, 2017). However, social Anelosimus spiders lack task specialization and do not exhibit individual recognition or dominance hierarchies (Jones and Parker, 2000; Aviles and Bukowski, 2006; Settepani et al., 2013). Individual redundancy, resulting from cooperation on maternal care and prey capture, could drive reduced selection for individual competency at these tasks. As predicted, individual females in social Anelosimus species are less competent at maternal care (Samuk et al., 2011) and less effective hunters (Harwood and Avilés, 2018) when compared to females from related subsocial species. We expected brain investment to differ among the sociality levels due to differences in reliance on individual ability to successfully complete maternal care and prey capture tasks. We predicted the arcuate body would be less developed (relatively smaller) in females of the social compared to subsocial species, because the arcuate body is a center of mechanosensory integration (Babu and Barth, 1984; Steinhoff et al., 2018). Unlike females, males of both social and subsocial species die shortly following mating (Avilés and Gelsey, 1998; Avilés and Salazar, 1999; Viera et al., 2007) and have little or no engagement in colony activity, except in nest maintenance (Aviles, 1986). Because male behavior does not differ consistently between the sociality categories, we predicted male brain investment patterns would be similar in subsocial and social species.

For females, social category was a significant predictor of arcuate body volume. After accounting for significant effects of brain size [supraesophageal ganglion volume: GLM F₍₁, ₈₎ = 12.02, p = 0.008], arcuate body volume was significantly greater in subsocial than in social species [GLM F₍₁, ₈₎ = 5.72, p = 0.04]. Similar patterns were found in the analysis of log-log transformed brain volume data [Figure 2; supraesophageal ganglion volume effect: GLM F₍₁, ₈₎ = 16.15, p = 0.004; social category effect: GLM F₍₁,₈₎ = 7.03, p = 0.03]. The log-log slope of the arcurate body vs. supraesophageal ganglion volume regression was significantly lower than 1.0 (slope = 0.39 ± 0.24 95% CI), demonstrated an allometrically decelerating rate of increase in arcuate body size. Using the PGLS to account for phylogeny supported the effects of sociality category: arcuate body size was greater in subsocial than in social species [Figure 2; PGLS on log transformed data: F₍₁, ₈₎ = 14.84, p = 0.005]. For males, social behavior category did not predict arcuate body size [GLM linear analysis F₍₁, ₆₎ = 1.11, p = 0.33; GLM log-log analysis F₍₁, ₆₎ = 1.72, p = 0.24; PGLS on log transformed data F₍₁, ₆₎ = 3.3, p = 0.12].

Female Anelosimus of social species engage in cooperative brood care and prey capture (Avilés and Tufino, 1998; Yip et al., 2008; Harwood and Avilés, 2013, 2018; Samuk and Avilés, 2013; Avilés and Guevara, 2017), and we predicted that individuals’ redundancy in these tasks would reduce individual cognitive demands and relax selection for neural investment compared to subsocial congeners. As predicted, females of social species had significantly reduced relative investment in the arcuate body compared to subsocial species. This pattern did not hold for males which do not differ in task redundancy between subsocial and social species. These results indicate that behavioral redundancy is an attribute of sociality which may promote reduced brain region investment, independently of other social attributes such as task specialization. Application of the behavioral framework we develop to other social systems may encourage the identification of behavioral attributes driving brain evolution that have previously gone unnoticed.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

LA collected and supplied specimens. All authors listed have made substantial contributions to the work and approved it for publication.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.