This study was approved by the institutional ethics and animal welfare committee at the University of Veterinary Medicine Vienna (Approval numbers: 09/04/97/2012, 04/05/97/2012, 09/10/97/2012) in accordance with Good Scientific Practice guidelines and national legislation¹. The owners participated on a voluntary basis and they all signed an informed consent form before beginning the experiment.

Altogether, 217 purebred Border collies, recruited from the Clever Dog Lab database in Vienna, participated in a behavior test battery and were genotyped for OXTR polymorphisms. In the current analyses, we excluded dogs younger than 10 months and dogs with missing genotype data. The final sample consisted of N = 173 dogs, 72 (41.6%) males and 101 females, and their mean age (±SD) was 3.87 ± 3.02. From this sample, due to technical reasons, two dogs had missing values for the Greeting during first encounter, four dogs for the Greeting during separation, and one dog for the Greeting after threatening approach context (please see below for descriptions of test contexts).

First, in order to provide descriptive analyses, we investigated if the four possible modifying factors: context, sex, age, and stress in the preceding situation per se have a significant effect on the greeting behavior of the dogs. The context, sex, and age were included in a generalized linear mixed model as main effects, with the greeting behavior as the dependent variable and the dogs’ ID as a random effect. The model also included all two-way and three-way interactions. The effect of being stressed or not in the preceding situation was investigated with independent t-tests in the Greeting during separation and Greeting after threatening approach contexts separately. Next, to investigate whether there was any association between our subjects’ different individual characteristics (genotype distributions of the four SNPs, stress during separation, reaction to threatening, and sex of the dogs), we analyzed the relationships between these variables using χ² tests.

Second, we used another generalized linear mixed model to analyze if the dogs’ individual characteristics (sex, age) and/or the context modified the association between the dogs’ OXTR genotype and behavior. This model included the greeting behavior as the dependent variable, the four SNPs, sex, and context as fixed factors, age as covariate, and the dogs’ ID as random effect. Here we also included all age × SNP, sex × SNP, and context × SNP interactions, as well as all sex × SNP × context and age × SNP × context interactions. Non-significant effects were removed from the model using a backward elimination procedure.

Third, we ran an additional GLM for each of the Greeting during separation and Greeting after threatening approach tests, in order to investigate how the dogs’ reaction to the preceding situation moderated the association between the OXTR genotypes and the behavior. The dogs’ reaction to the preceding situation was not added as a fixed factor in the previous models because the expected two-way or three-way interactions between SNP and reaction (and sex or age) may not be detected due to the lower sample size (N = 73–100 dogs per stress category). Therefore, we analyzed the effect of the SNPs separately in the two reaction categories of dogs using GLM models with the same setup as described above. In all the models, the effect size of each factor was estimated with Partial η². SPSS version 22 was used for the analyses.

Regarding the factors affecting the greeting behavior, neither the dogs’ sex, nor the age, nor any of their interactions were significantly related to the greeting behavior (p > 0.261 at removal). We found a main effect of the context (F_2,511 = 50.760, p < 0.001), pairwise contrast revealed that the dogs greeted the experimenter more when they first met her and after threatening approach than during separation (p < 0.001 for both), but there was no difference between the Greeting during first encounter and Greeting after threatening approach contexts (p = 0.728) (Figure 1).

We found no significant differences between the stressed and non-stressed dogs either in the Greeting during separation or in the Greeting after threatening approach contexts (p = 0.206; p = 0.289, respectively) (Figure 1).

Regarding possible correlated effects of the different individual characteristics, we found no significant associations between stress during separation, reaction to the threatening approach, sex of the dog, and the genotype distributions of the four SNPs (Table 4).

We found two significant sex × SNP × context and two significant age × SNP × context interactions, as well as a significant two-way interaction between SNP and context. The -213A/G SNP showed a significant three-way interaction with age and context (F_2,464 = 3.151, p = 0.044), the -94T/C SNP with sex and context (F_4,464 = 3.174, p = 0.014), and the -74C/G with both age and context and sex and context (F_4,464 = 3.798, p = 0.005; F_4,464 = 2.427, p = 0.047, respectively). The rs8679682 SNP had no interaction with sex or age, but we found a two-way interaction with context (F_2,464 = 4.361, p = 0.013): the difference between this SNP’s genotypes was larger in the Greeting during separation context than in the other two contexts. To further explore and interpret the three-way interactions, we also analyzed the three contexts separately using general linear models (GLM), including the four SNPs, sex, age, and all age × SNP and sex × SNP interactions. Pairwise contrast was used for post hoc tests for main effects and sex interactions; the significant age interactions were interpreted by investigating the effect of age in the different genotypes separately.

In the first context, upon their first encounter, the dogs had the chance to greet the unfamiliar experimenter in the presence of their owner. In the Greeting after threatening approach context, the dogs greeted the more or less familiar experimenter also in the presence of the owner; however, in this context the dogs had a clearly negative experience with the experimenter immediately before the greeting situation. Although based on this we expected that dogs would greet the experimenter less enthusiastically after being threatened than during their first encounter, we found that the dogs greeted the experimenter with a similar intensity in these two contexts. This likely reflects the fact that dogs can flexibly adjust their behavior to the behavior (attitude) of their human partners (as Vas et al., 2005 also showed). This may also explain why no difference was found between the greeting behavior of dogs that had responded with or without overt stress to the social threat (i.e., dogs that showed avoidance or aggression vs. behaved in a friendly manner or remained passive). In contrast to the two contexts above, in the Greeting during separation context, the owner was not present during the test, and in this context the dogs greeted the experimenter less intensively than in the other two contexts. Although some of the dogs showed clear signs of stress caused by being separated from their owner and the others did not, the greeting intensity of both groups was uniformly lower in this context than in the presence of the owner. This suggests that the presence of the owner buffers the stress caused by the test situation, probably by providing social support (Gácsi et al., 2013).

Approximately half of the dogs (48.5%) showed stress-related behaviors during separation, and 57.8% of the dogs reacted with avoidance or aggression to the threatening approach (categorized as “stressed” based on Gácsi et al., 2013 and Horváth et al., 2008). It is important to note that we found no association between dogs’ signs of stress in response to separation and their behavior in response to a social threat. This means that the stressed vs. non-stressed categorization of the dogs did not reflect their general stress proneness, or willingness to show overt behavioral expression of their stress. Instead, showing behaviors related to stress seems to indicate a negative reaction specific to either test context: either separation anxiety or fear of a threatening person.

The most important finding of this study is that different OXTR SNPs were associated with greeting in stressed dogs and non-stressed dogs in either context. When trying to interpret these results, we should first mention that neither the separation stress nor the reaction to threatening was significantly related to any SNPs (or to the sex of the dog). Therefore, despite numerous studies suggesting the contrary (e.g., Kumsta and Heinrichs, 2013; Buttner, 2016), we cannot conclude that the OXTR polymorphisms themselves were directly related to general stress reduction or lower perception of social fear – at least not indiscriminately. Instead, the different SNPs had different behavioral associations depending on how the dog reacted to the context. As a possible explanation, stressed dogs may have approached and greeted the experimenter for different reasons than non-stressed dogs; even if they greeted her to a similar extent, they may have had different motivations to do so. This explanation would indicate that the dogs that did not show a negative reaction to separation or to a social threat maintained the same motivation to greet the experimenter as they had during the first encounter. However, for the dogs that were stressed by either being separated from their owner or threatened, the motivation to affiliate with the experimenter may have changed due to separation stress or social fear. For example, during separation, stressed dogs might look more for social support from the experimenter than non-stressed dogs, as they are more strongly affected by being alone.

In the context of the Greeting during separation, the -94T/C and -74C/G SNPs were found in association with the greeting only in stressed dogs, and both had a sex interaction. For the -94T/C, stressed males carrying the TT genotype greeted the experimenter more than males with the CC genotype, while the relation was opposite or absent in female dogs. For the -74C/G, stressed males carrying the CC genotype greeted the experimenter more than males carrying the GG genotype, while the relation was the opposite in the case of females. Human studies also found not only stress-dependent OXTR–behavior associations, as described in the section “Introduction,” but also sex differences in the effects of social support on the stress response (i.e., social support attenuated the stress reaction more for men than women, Kirschbaum et al., 1995; Ditzen et al., 2008).

Interestingly, even though the separation from the owner and a threatening person presented different types of stress for the dogs, the -94T/C SNP had a similar association with the greeting behavior also in the Greeting after threatening approach context. That is, stressed males carrying the TT genotype greeted the experimenter more than males with the CC genotype both when their owners left them alone, and when the experimenter had successfully threatened them beforehand. This seems to indicate that the -94T/C has a more general function in regulating the behavior of stressed (especially male) individuals, but, whether this SNP regulates the social fear of stressed dogs or affects their general stress coping ability should be further investigated.

In contrast, the polymorphisms -213A/G and rs8679682 were associated with greeting in non-stressed dogs during separation, and, in the case of the rs8679682, also after being threatened. Importantly, these same two SNPs were associated with the behavior also in the Greeting during first encounter context. This seems to indicate that dogs in general did not perceive the first encounter with the experimenter as stressful.

For the -213A/G, individuals carrying the A allele showed more intense greeting than dogs with the GG genotype. Since this SNP was associated with the dogs’ behavior only when the experimenter presented a positive or neutral figure (i.e., Greeting during first encounter and Greeting during separation in non-stressed dogs), the effect of this SNP might be sensitive to the characteristics of the experimenter. That is, the A allele predicts increased social approach and affiliation but only when the dog had no prior negative experience with the experimenter. Similarly, human studies also found differential effects of oxytocin on social behavior depending on the attitude of the social partner (e.g., Mikolajczak et al., 2010; De Dreu, 2012). Moreover, Kis et al. (2014) also found significant associations between the -213A/G and dogs’ proximity seeking (a behavior scale that includes variables assessed in different greeting situations) both in German shepherds and in Border collies, and similarly to our study, they found that in Border collies, carrying the A allele was associated with higher proximity seeking.

For the rs8679682 SNP, non-stressed dogs carrying the C allele showed more intense greeting than dogs with the TT genotype in all three contexts (however, this association might be stronger in females than in males). This seems to indicate that the rs8679682 has a more general function in regulating the social affiliative behavior of dogs toward humans (with the C allele facilitating more positive social behaviors relative to the T allele), but its effects seem to be sensitive to the positive salience of the context. Whether this SNP regulates social motivation, attraction to humans or general curiosity needs further investigation.

Another parallel between our results and other studies is that the sex of the dog also modulates the gene × behavior associations. Sex differences in the function of the oxytocin system are well known (Bos et al., 2012; Chen and Johnson, 2012; Kovács et al., 2016), and can be explained by possible differences in the general hormonal environment (e.g., estrogen level, McCarthy, 1995; Petersson et al., 1999), different patterns of oxytocin release (Jezová et al., 1996), and/or different patterns of amygdala activation between the sexes. For example, oxytocin administration decreases amygdala reactivity to emotional faces in men (Domes et al., 2007) and increases it in women (Domes et al., 2010). Our results suggest that also in dogs, some relationships between polymorphisms of the OXTR and social behavior seem to be sex-specific.