In the course of the 21st century, Human-Computer Interaction (HCI) has significantly expanded its scope, from personal computers and computer-supported cooperative work, to truly mobile and pervasive technologies, permeating almost all contexts, and mediating many of the situations and services, associated to modern life. Accordingly, the field of Interaction Design has expanded significantly in scope. The meaning of “interaction” in “Interaction Design” now goes way beyond the design of immediate and situated interactions between humans and computerized interfaces, and into the design of relationships between different disciplines, organizational departments and sections of the product lifecycle. The term “User Experience (UX) Designer” has become commonplace to designate a wide range of professional roles, varying from tactical activities (also known as “downstream”) such as user interface design, graphic design, prototyping, coding, animation, 3D modeling, etc., to more strategic activities (also referred to as “upstream”), such as customer research, attraction, and retention, brand identity, data analysis, design strategy, innovation, product management, and others. Those later roles have in turn led to fields with a slightly distinct focus from traditional HCI, such as Service and Product Design.

Underpinning this development, there is the increasing demand to relate the so-called “user experience” of a designed artifact or service to the context it occupies within a client organization and outside it, in society. In recent years, growing awareness of the interconnected nature of humans, technology and the world at large has been manifested through tensions pulling HCI in different directions, from concerns about existential risks such as climate emergency (Knowles et al., 2018; Bremer et al., 2022) or the threat of uncontrolled Artificial Intelligence (Nowak et al., 2018), to global social movements like Black Lives Matters and First Nations self-determination, arguing for alternative, decolonized and inclusive approaches to possible futures. That in turn has led to a reckoning in the HCI community, with emerging voices calling for embedding decolonized thinking (Bidwell, 2016; White and Hespanhol, 2022) and more-than-human (Coulton and Lindley, 2019; Wakkary, 2021) considerations in design, often also expressed as a shift from Human-Centered to Life-Centered (Lau, 2004) or even Planet-Centered Design (Xu, 2021).

Taking a related, yet different approach, some researchers have pointed to the relevance of borrowing from previous philosophical efforts toward drafting alternative, inclusive and relational worldviews around technology. In that context, Indigenous knowledge systems (Abdilla, 2018; Lewis, 2020) and feminist critical theory (Butler, 1990; Barad, 2007; Bardzell and Bardzell, 2011) have figured prominently. Frauenberger (2019) offers an in-depth review of a range of entanglement theories, including Latour's Actor-Network Theory (Latour, 2007), in argument for a relational ontology as “a productive way for HCI to evolve in response to a changing world” (Frauenberger, 2019, p. 3). Draude (2020) provides a more comprehensive introduction to posthumanism and new materialism, and its potential to inform a broader approach to HCI. Notably, both draw from the ideas of Barad (2007), particularly her articulation of Agential Realism, centered on what she calls ethico-onto-epistemological perspective, and coalesced into her notion of intra-actions.

In this article, I propose to take a step forward toward discussing what a shift from Human-Computer Interaction (HCI) to Human-Computer Intra-Action (HCIA) would entail (Figure 1). To that end, I consider recent dynamics in the field of HCI to support the conceptualization of intra-actions as the entanglement of co-constitutive relationalities, which can involve low-level (closer to the interface) or high-level (closer to the environment) concerns. I then consider the increasing hybridity of socio-technical engagements both in space (local vs. remote) and in time (immediate vs. stretched), to discuss emerging notions of spatiality and temporality. Based on that vocabulary, I then propose a preliminary design method for mapping intra-actions associated to interactive technologies, offering a way to situate the interactive product within a constellation of relationships. I then offer two examples on how to apply this method to the design and evaluation of a service and product, respectively. Worth of note, in order to allow for a clearer critique of the term “interaction”, I will instead be using throughout this paper the term “engagement” to refer to any dialogical processes involving humans and technology. It is my hope that the short discussion here presented can contribute to repositioning the design process of interactive technologies toward embedding, from the outset, considerations about systems-thinking, ethics, equity, inclusion, shortcomings, co-responsibilities, and the prefiguration of increasingly entangled futures.

The term Human-Computer Interaction (HCI) presupposes two separate and previously existing entities—human and computer—coming into contact through an interface, which in turn allows them to interact with each other. The human is understood as a user of this interface, whose design, therefore, should be catered to a degree of quality capable of ensuring a good user experience of the computer or technology. The design process must therefore be informed by knowledge of both the human and the computer, so that the resulting interface is effective in mediating the conversation. This approach is practical, clear and objective, allowing for a pragmatic understanding of users and their needs, the affordances and purpose of the computer technology, and consequently suitable requirements for the interface matching the two.

Yet, as pointed out by Draude (2020), the increasing ubiquity of digital media across all aspects of human life also reveals clear limitations of this dualistic approach, which appears insufficient to describe the wide variety of scenarios where humans may encounter technologies, and how those engagements may in turn ripple across and affect other actors and the environment. It tends to ignore the extent to which being a human user of technology, as well as its qualia and meaning, is self-referential and intrinsically defined by the very experience of engaging with a computer. A user is only so to the extent it uses the technology, yet the very condition of being a user presupposes certain expectations and behaviors, thus prefiguring the experience. Likewise, the computer only exists as interactive technology to the extent it can partake in dynamic conversation with the human (Figure 1). Rather than two immutable entities—human and computer—they actually exist in relation to each other, as well as to other systems (digital or not) contextually connected to them, forming relational constellations involving humans and non-humans (Coulton and Lindley, 2019). These relationships continuously and mutually affect and redefine the roles and identities of all involved.

Such a realization has been articulated in HCI literature, particularly since the turn of the 21st century, through an expanded understanding about the loci of interactions between humans and computers, and the ensuing mechanisms through which the very act of interacting affects both. Höök proposed the notion of “affective loops” to describe emotions not as pre-defined states we assume during interactions, but rather “as processes, constructed in the interaction, starting from bodily, cognitive or social experiences” where “the user is an active, meaning-making individual” (Höök, 2008, p. 11). In doing so, she drew from what Fallman had articulated as the interplay between an individual's experiential and cultural bodies: whereas the former is physical and open to sensory stimuli, the latter provides the framework to interpret those stimuli, connecting them to memories, and helping to construct an ever-evolving emotional and cognitive response, co-defined with the environment through interaction (Fällman, 2003). While those approaches address the interplay and mutual co-constitution between human, computer and environment, they have more often than not adopted a human-driven lens, focusing on the aspects pertaining to human sense and meaning making, how our bodies reflect and act on the surrounding environment, and the multiple and evolving ways we position ourselves toward the world in response. In other words, while focus has been placed on how human epistemology and phenomenology are co-created through engagement with the world, less attention has been given to the constellations of processes surrounding both humans and technological systems, as well as the realization that they constitute, ultimately, somewhat arbitrary constructs. With the growing prominence of alternative design approaches challenging Human-Centered Design, there is also a growing need to expand the awareness of “entities” as “processes” beyond Human-Computer Interaction, to the myriad processes associated to technology-mediated human engagements with the world at large.

This has become particular apparent with the spread of data harvesting and analytics, and, increasingly, its use in the training of models for machine learning and other forms of artificial intelligence (AI). Unlike more traditional forms of HCI, data gathering happens in the background, as a by-product of the interactions between humans, computers and the (computerized) environment. While surely still carried out by computers, the devices, systems and services dedicated to data gathering are not always designed as front-end interfaces that users may directly interact with. Rather, more commonly, they constitute back-end algorithms collecting and processing massive amounts of data, making decisions based on them, and feeding information back to other algorithms, which in turn do affect user interfaces, human and non-human subjects, as well as the environment. This kind of “computer-to-computer” interaction, mediated by intelligent algorithms, acquires an ambient, contextual character (Sezer et al., 2017), which can be largely dissociated, both in space and time, from any other actors involved in the process, as articulated in the sections below. As a result, humans may engage with Big Data (Zwitter, 2014) and AI systems without ever realizing it: on the one hand, their data is often collected without their consent or awareness; on the other hand, they are subject to decision-making based on that data, aggregated with those of many other humans, which also affect their lives in circumstances and through mechanisms they may be ignorant about, and have no agency over. As Comber et al. (2019) pointed out, “interaction” becomes a highly inaccurate term to describe those sorts of relationships between humans and technology, as the mediation (Verbeek, 2015a) performed by technology in those scenarios is enacted both through interaction and also without and outside interaction (Comber et al., 2019). Yet, they can fundamentally determine how individuals and societies function and organize themselves. Furthermore, this contextual “framing” plays out in conjunction with other non-technological contextual factors such as culture, social norms and values, the economy, the weather, natural environments and their cycles, non-human living beings, the news, policies and regulations, and many others, traversing the encounters between “humans” and “computers”, and continuously redefining that relationship.

Addressing such dynamics in HCI, authors (Frauenberger, 2019; Draude, 2020) have turned to new materialism and critical posthumanism as epistemologies capable of informing ways forward, drawing inspiration, among others, from the work of physicist Karen Barad. To describe such a relational and inherently contextual perspective to engagements, more broadly, Barad (2007) proposed to replace the prefix inter in “interaction” (Latin for “between”) with intra (Latin for “within”). Intra-actions thus refer to this process of mutual determination between the parties within a relationship, thus only possibly defined in relation to each other. In the context of a Human-Computer Intra-Action (HCIA), what is otherwise referred as “interface” can therefore be reinterpreted as one of many elements enacting the relationship between humans and technology, and mediating (Verbeek, 2015b) further relationships between those and the world, including relationships with and between other devices, algorithms, services, non-human animals, and the built and natural environments. Rather than focusing on interfaces, I argue that the focus of design should shift to the factors framing relationships among the parties participating in a particular context, which I hereby propose to refer to as relationalities. Moreover, I argue for, and articulate below, a conceptual framework and associated design method to assist with identifying and mapping such relationalities around a given reference point within an otherwise human-computer interaction scenario.

Whereas interactions characterize encounters between immutable entities, assumed to maintain their integrity before and after engaging with each other, intra-actions seek to establish inter-subjective relations capable of pointing to new possibilities of mutual acknowledgment and reliance, prompting changes to the identities of participants taking into account their extended cultural context. That extended cultural context may potentially include social, political, economic, ethical and moral concerns, all contributing to the identification of synergies and antagonisms (Bishop, 2004) framing the relationship. By relationalities, I refer to this friction between the synergies and antagonisms defining a relationship between entities, thus establishing what could be understood as a “relational tension”. This relational tension becomes thus the object of HCIA design, and may play out at low or high levels of concerns in regards to the technology impact.

Following Barad (2007), rather than referring to entities, I propose to describe relationalities by referring to agencies, or abilities to act, conferred by the relationships of co-constitutive parties. Those abilities to act may either be limited in scope and impact, or else imply deeper and longer-lasting implications to society and the world at large. I define low-level relationalities as those where the abilities to act are constrained to the immediate vicinity of the interfaces. They pertain direct engagements between humans and digital technologies (or between multiple technological systems) closer to the interface “surface” (van der Tuin and Verhoeff, 2022), thus implying usability factors, user experience, tangibility, immersion, ergonomics, and so on, with limited consequences beyond those factors. In other words, low-level relationalities include all the cultural contexts, conditions, motivations and heuristics associated to the design of artifacts located downstream within a product or service lifecycle, closer to the user. In contrast to the “zoomed-in” view associated to low-level relationalities, I define high-level relationalities as consisting of the “zoomed-out” perspectives of human-computer intra-actions, pertaining to upstream concerns in the lifecycle of products or services, and looking at the various ways media assemblages interact with each other and, by extent—being ultimately human designs—how they interact with natural and societal ecosystems at large. That includes complex factors ranging from social responsibility and the ethical and moral use of technologies, to their viability, and economic and environmental sustainability. It also includes how they affect those same factors in other socio-technical ecosystems, i.e., the ripple effects triggered by their introduction into the world. Here, we approach media and technology as “machines” in the sense used by Deleuze and Guattari (1987), i.e., assemblages intrinsically connected to others, in a relational manner, partaking in continuous and mutual constitution.

In that sense, relationalities in human-computer intra-actions can be described as both performative (Butler, 1990; McKinlay, 2010) and sympoietic (Haraway, 2016). Performative, insofar they are created by the very fact people participate in them, as explained by Fallman (2005) when arguing for a shift from “freedom” to “involvement” in mobile HCI: “subjects do neither solely impose meanings on the world, nor do they simply find suitable structures of meaning in the world, but rather [...] body-subjects through interplay with the world both find as well as create meaning.” (Fallman, 2005, p. 7). Moreover, relationalities, including the knowledge embedded in them, are created through sympoiesis—or “makingwith”, as defined by Haraway (2016)—i.e., through the collaborative configuration of the various parties involved, which engage in collective and active prefiguration (van der Tuin and Verhoeff, 2022) of their own future. Shifting from interactions to intra-actions finds parallel on the discussions about similar previous shifts toward relational art: “conceptualized as a reciprocal, if not strictly speaking interrogative gesture, the form of relational art—and the formative structures it engenders— effectively “invent[s] possible encounters” and “the conditions for an exchange” (Downey, 2007, p. 268). Similarly, when discussing craft-based HCI, Frankjaer and Dalsgaard (2020) argued for the generation of intermediate level knowledge through a cyclical sympoietic framework based on the manifestation of the artifact (localizing), the interaction between the artifact and the world into which it is introduced (questioning), and subsequent reflection and synthesizing of concepts (opening). This ongoing causal transitivity (whereby an action or intention is carried from the subject to the object) (Downey, 2007) underpins the co-constitution inherent to intra-actions, and is enacted across the continua of space and time, as discussed below.

Considerations about co-constitutive relationships between humans and their surrounding environments along the dimensions of space and time have gained prominence since the early 2000s with the emergence of relational aesthetics (Bourriaud, 2002; Bishop, 2004; Downey, 2007) and, more specifically in HCI, mobile computing (Kjeldskov, 2002; Fallman, 2005). Fallman (2005) argued for the design of mobility beyond mere matters of space and time, rather as an interplay between freedom and involvement, where the human-computer-world physio-social contextual relationship would determine where, when and whether certain forms of interaction should occur. Likewise, Dourish (2006) drew particular attention to the importance of social relationships in shaping space, place and meaning. The subsequent advent of the smartphone granted people greater control over the context of each interaction, while simultaneously leading to increasingly hybrid contexts, in which social participation could be regularly distributed across geographies and unfold asynchronously in time. The limitations to social movement and gathering in public spaces prompted by the COVID-19 pandemic led to the dissemination and normalization of those hybrid contexts across most of society, globally, in the process fostering new notions of sociability (Beiguelman, 2020), public spaces (Bravo and Tieben, 2020), and placemaking (Hespanhol, 2022).

As a consequence, relational contexts have become increasingly ubiquitous/pervasive, and highly transient not only between physical and digital spaces, but also between immediate and “stretched” experiences of time. Hybridity begets a process of deterritorialization: interactions are localized, yet displaced, simultaneously. Deterritorialization in space translates into an extended notion of mobility. If mobility previously meant the ability to swiftly transit across different physical geographies, benefiting from some level of location awareness while still retaining the remaining aspects afforded by the engagement with technology (e.g., access to conversations, services, or social media feeds under the same user profile), it is now extended to the realms of “augmented placemaking” (Hespanhol, 2022), through digitally-augmented physical space [like augmented-reality based tourist apps (Morrison, 2022)] and purely virtual interactive environments (such as Zoom calls). While still operated from physical locations, these become less relevant to the technology-mediated experience. Instead, synergies or antagonisms then tend to arise from the friction between the embodied phenomenology of a physical place, and the affective and cognitive experiences of the digital environmental mediating the engagement, both unfolding simultaneously. Likewise, interactive experiences are informed by their situational context and all the potentially concurrent influences at play, as verified, for example, in the twisted use of texting while driving (Bayer and Campbell, 2012), the case of so-called “zoombombings” (Ling et al., 2021), the digital intimacy resulting from the rising popularity of work from home, with the consequent sharing of domestic settings as backgrounds of teleconference calls (Beiguelman, 2020), or frictions emerging from people juggling between simultaneous work and parenting while online (Manzo and Minello, 2020).

This intermingling between public and private relations, blending physical to virtual spatialities through a deterriorialization of space, echoes a parallel deterritorialization of time. The growth of social media, streaming, and similar cloud-based services engenders elastic notions of temporal reality, whereby the perception of the present, contextually, may spread many different interactions over time. As an example, Coleman (2018) compares the temporal dynamics of a social media platform like Twitter vs. a streaming service such as Netflix: while the former implies a sense of immediacy and “up-to-dateness”, the latter is built in ways to allow episodes to be paused and resumed at a later stage, or users to binge-watch many installments of a series in a row, thus stretching the temporal “now” at various scales. Experiences mediated by technology are no longer necessarily bound by real-time synchronicity, with asynchronous communication being in fact the norm of engagements via social media or text messaging services. Importantly, these different notions of time should be interpreted from an experiential perspective, i.e., whether the scope of an interaction feels immediate, or else stretches for longer periods, potentially remaining continuously open-ended and never reaching a definitive conclusion. This dual hybridity—of space and time—maps to emerging new forms of human relationships, fragmented and enacted through encounters across the wider range of phygital (physical-digital) environments (Gaggioli, 2017) we now inhabit on a daily basis. As Koch and Miles (2021) pointed out, “in contrast to the focus on spontaneous encounters that underpin much theorization on public life, meetings between strangers are now frequently planned in advance and take place in relative privacy, increasingly within spaces of the home” (Koch and Miles, 2021, p. 1380), leading to a relational arrangement they call stranger intimacy: “conditional relations of openness among the unacquainted, however fleeting, through which affective structures of knowing, providing, befriending or even loving are built.” (Koch and Miles, 2021, p. 1380). In contrast to the serendipitous character of encounters with familiar strangers in urban public spaces (Paulos and Goodman, 2004), this intimacy between strangers, mediated by digital technologies, unfolds in hybrid territory, potentially displaced spatially, and enacted across temporal dimensions that may be both immediate and stretched, synchronous and asynchronous. Further, they articulate relationalities that are both low-level—such as the ability to pause, share or comment on a YouTube video streaming—and high-level—such as (to stay on the same example) who are we sharing it with and why, the potential spread of fake news, or the promotion of filter bubbles (Spohr, 2017).

The considerations above suggest a preliminary lexicon through which one could initiate a process of mapping out the co-constitutive relationalities associated to human-computer intra-actions, against dimensions of spatiality and temporality. This relational approach posits the human-computer condition as a mutual entanglement of agencies, rather than a conversation between entities, and thus places emphasis on mutual action, impact, and responsibility. Addressing any part in isolation is therefore naive and incomplete, as a solution will only truly exist once all aspects get addressed—and, even so, only insofar the relationships remain in place. Of particular concern is the identification of the synergies and antagonisms (i.e., the relationalities) framing a technological solution. Once relationalities are identified, designers can also identify frictions of controversies between them, capable of highlighting their co-constitutive nature, and thus prompting discussions about trade-offs inherent to the design solution (Baibarac-Duignan and de Lange, 2021). Human-Computer Intra-Action could thus be understood as a process of mapping out the tactical and strategic relationships defining a social-technical scenario, as well as their impact at the geographical location the interface is deployed to and across space, both now and in the future. That definition, in turn, suggests a method for identifying and classifying relationalities, which I describe below.

In this article, I made the case for Human-Computer Intra-Action, a broader appreciation of the various interdependent aspects co-constituting engagements between humans, technology and the world, performed through continuous and mutually formative synergies and antagonisms—a set of tensions I referred to as relationalities. I also discussed new perspectives in the field related to its increasing hybridity, which, as I argued, can be verified across space and time. Furthermore, I proposed a new design method—intra-action mapping—to map out intra-actions pertaining to a particular technology, by identifying its primary and secondary relationalities, and evaluating their low- and high-level character, temporality and spatiality. In doing so, I call for a shift in perspective toward a relational approach for interactive technology, one capable of acknowledging its true impact and dependencies in the world, thus also highlighting aspects not only related to usability, effectiveness and efficiency, but also sustainability, social responsibility, and ethics. I envisage that this approach can be of use by interaction and product designers, managers and stakeholders in general, as well as researchers, helping to drive conversations, from the outset, around the often hidden impact of technology, and the ensuing trade-offs inherent to any real-world rollout of technology.

That said, the method proposed above must be taken with caution, given its open ended nature. It has been left open on purpose, for the sake of allowing its possible further customization to various different contexts and scenarios. On the flipside, it does not quite capture or describe the nature of the relationships between relationalities in greater detail, nor suggests any approach for ranking them according to any particular criteria, other than whether they are primarily or secondarily related to the chosen point of reference. Further, identifying relationalities may, in practice, demand multiple iterations, each revealing further relevant aspects for consideration. Nevertheless, it is my hope that this preliminary analysis and method may be helpful in quickly identifying relationships, co-dependencies and indirect stakeholders, and highlighting which ancillary factors must be taken into account early on in an interactive technology design project.

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

The author confirms being the sole contributor of this work and has approved it for publication.