Probabilistic inference is a problem in the complexity class #P, and computing an approximate solution for it better than a factor of 0.5 is NP-hard (Koller and Friedman, 2009). Nevertheless, there are classes of models called tractable probabilistic model (TPM) (Darwiche, 2002; Poon and Domingos, 2011; Kisa et al., 2014; Zhang et al., 2021) (see Section 3) where computing evidence and marginal queries can be guaranteed to have a cost bounded by a polynomial in its size. They can be used to model any probability distribution defined over categorical variables, and, as expected due to the hardness of approximation of inference¹, their size requirements can be exponential in the problem specification size. Inference is a subroutine in learning and approximations used when learning can have an impact on what is learned (Koller and Friedman, 2009; Poon and Domingos, 2011). Moreover, different approximate inference procedures used over a learned model can yield different results for the same queries (Koller and Friedman, 2009). Using TPM where inference is guaranteed to have a bounded cost enables the utilization of exact inference procedures for learning and usage of the learned model. In that scenario, all approximations are done when choosing the structure and size of the TPM.

While probabilistic models capture co-occurrences of events in observed environments, in causal models, it is assumed that the behavior of models can change when we choose to act on the world via an yet unmnodeled process (Pearl, 2009). An intervention on a variable changes the function that was used to define the value of that variable (Pearl, 2009). Without further assumptions on how each change in an intervened variable influences the rest of the variables, the effects of an intervention are undefined (Pearl, 2009). The growth of the space of functions that is needed to model probabilistic relations in the scenarios with interventions, in tandem with the potential disruption of parameter level dependence relationships exploited to get modern compressed TPM (Darwiche, 2022), poses challenges in tying TPM and causality while avoiding large TPM models.

Research in causality is built around the proposition that it is useful to think about causes and effects while modeling the world. The study of cause–effect relationships in models is relevant due to tools it provides to its user (Pearl, 2019). It is central in many aspects of modeling such as: (1) missing data imputation (Mohan and Pearl, 2021), (2) identifiability of parameters and learnability (Tikka et al., 2019; Xia et al., 2021), (3) transportability (Bareinboim and Pearl, 2013; Pearl and Bareinboim, 2014), or (4) out-of-distribution generalization (Jalaldoust and Bareinboim, 2023). At the core of these tools are statements about inter-dependency among variables in the presence of interventions. An effect is naturally defined as a function of its causes but not the other way around. This asymmetry, already present in structural equation modeling (SEM) described in Wright (1921), is the cornerstone of the structural causal model framework (SCMF) approach to causality advocated in Pearl (2009). A key point in this article is the expression of causality through function compositions. Under this framework, a cause–effect statement is equivalent to a statement that a function can be decomposed in a specific form. Specifically, parent–child relations exist in the function decomposition through the input–output relations. However, the existence of a function decomposition that, when exploited, allows us to correctly compute the global function does not imply its explicit use.

A structural causal model (SCM) (Pearl, 2009; Bareinboim et al., 2022) is defined as a 4-tuple (V,U,F,P(U)) where: (1) F is a set of functions that is used to define “endogenous” variables in SCM in the absence of interventions on them; (2) V is a set of variables that are “endogenous” to the model by virtue of being, in the absence of interventions, defined as the outputs of functions in F; (3) U is a set of “exogenous” variables whose value determines, at an individual level, every factor of variation in functions in F; and (4) P(U) stands for a probabilistic distribution over all exogenous variables. An intervention is a replacement of a function that defines an endogenous variable in structural causal models by another, yet undefined, function whose output provides the new definition of that endogenous variable (Pearl, 2009; Bareinboim et al., 2022). This implies that exogenous variables do not characterize the uncertainty over interventions, and, in that regard, they are outside of what is (explicitly) modeled by an SCM. Nevertheless, the way they influence the set of endogenous variables is well defined in that an algorithm that takes as input F and the interventions can output a new set of functions can be used to answer queries containing interventions. Therefore, some of the information in an SCM is encoded in its structure.

There are three approaches to model causality with TPM: (1) using variable elimination over a SCM to get a TPM structure (Darwiche, 2022), (2) using a transformation between a TPM and a causal Bayesian network (CBN) (Papantonis and Belle, 2020) to support cause–effect claims in a TPM, and (3) using separate parameters for each interventional case, which was used in interventional sum product network (iSPN) (Zečević et al., 2021) .

The two first approaches rely on the existence of a class of models like CBN or SCM on which causality has been studied (Pearl, 2009; Bareinboim et al., 2022). TPM research was sprung in the context of efforts to accelerate inference in Bayesian networks (Chavira and Darwiche, 2005); therefore, there is good reason to ask if compilation of CBN would provide a good way to introduce causality into TPM. In Darwiche (2022) a SCM used to describe some phenomenon is compiled into a TPM via an algorithm akin to variable elimination. Similarly to SCM, an algorithm that takes as input both the structure of the computation graph (CG) (Erikssont et al., 1998; Trapp et al., 2019; Peharz et al., 2020) of a TPM and information about interventions will adjust the CG so that queries pertaining to interventions can be answered (Darwiche, 2022). The structure of TPM is used as a source of information; therefore, not all information in the TPM [in the approach taken in Darwiche (2022)] is encoded explicitly in the input–output mapping. This limits the set of structures that can be used by the TPM to those in which the algorithm that adapts the TPM to respond to queries pertaining to interventions works as intended.

The second approach depends on the ability to transform a probabilistic distribution expressed in a sum product network (SPN) or probabilistic sentential decision diagram (PSDD) (both TPM) as a bayesian Network (BN) (Papantonis and Belle, 2020). In that work, the transformations from SPN to BN described in Zhao et al. (2015) and a transformation from PSDD into BN developed in Papantonis and Belle (2020) were used for that purpose. From the BN that is obtained, in Papantonis and Belle (2020), a set of cause effect statements regarding the initial model is discussed under the assumption that the directed acyclic graph (DAG) of the BN encodes a set of cause–effect relationships. In either the CBN, SCM, or TPM in Papantonis and Belle (2020), variables that specify the interventions are not explicitly mentioned. This is problematic because the transformation they use preserves only the input–output relationships obtained with the variables that are explicitly declared. As a result, the information in the structure that is used as input to the algorithm that modifies the models to answer interventional queries can be lost.

The third approach avoids referencing CBN or TPM explicitly by making every parameter in an SPN with random structure a function of the adjacency matrix (pertaining to a CBN with the cause–effect relationships) that one would get after applying the algorithm that replaces the variable definition given by its modeled causes by an intervention (external definition) (Zečević et al., 2021). All information about all interventions can potentially influence all parameters, and the locality of interventions is lost in the sense that an intervention that, in SCM only replaced a function in a set of functions, in iSPN (Zečević et al., 2021) acts globally in the parameters of all functions. By changing all parameters due to interventions, no specific structure in the CG of a iSPN is required in order for an algorithm that adjusts the model like the one used in Darwiche (2022) to respond to interventions. However, this does not prevent us from using information about cause–effect relationships to create the structure of a iSPN-like model, which raises the research question: “Is it useful to still consider cause–effect relationships when building iSPN-like models?”.

A CG of either TPM and SCM describes a set of operations that implement the model. As long as the CG has depth >1, the operations can be described in a series of steps. A function that implements the input–output mapping of a model can be decomposed according to a CG that describes it. In SCM, functions that define causes of a variable with index i, or interventions that replace them, are sub-functions of the function used to define that variable (with index i). Therefore, there exists a CG that implements a SCM according to which the computations are ordered from causes to effects. In TPM, the function decomposition has a different purpose: minimizing the number of operations required for queries pertaining to marginalization of variables. There is a mismatch between the function decomposition pertaining to a TPM where all variables appear at the inputs of a CG and the function decomposition implied by cause–effect relationships where endogenous variables are functions of each other. A discussion of causality in TPM benefits from a different foundation where both functional descriptions of a model can be described and compared. Toward this end, SCM are extended to extended structural causal model (ESCM) where all information is encoded in the input–output mapping.

Based on ESCM, cause–effect statements are expressed as constraints over a space of functions. In order to be able to express all interventions through the input–output mapping of a model, a set of variables that can express them (in the input space) has to be added. Simply adding variables referring to interventions as inputs of functions in the set F changes the meaning of the U that, in SCM, does not characterize factors of variation pertaining to interventions. The meaning of exogenous variables is tightly coupled with functions in the set F so, in this study a distinct set of functions G is considered for the implementation of the interventions, resulting in the approach described in Section 2.This approach involves declaring that the information regarding an “endogenous” variable is computed in two steps:(1) In the first step, the corresponding function in F is used to compute what we can estimate about the variable given its modeled causes and U; (2) in the second step, a function in G takes as input both the value of the previous step and information about interventions on the variable . The output of the functions of the second step is used as input to the functions that reference the respective variable in a function F as it contains the most information about it.

The causality expressed as constraints over a function space is an unifying framework for expressing causality with TPM, in that, the three approaches that are described can be analyzed within it. A problem they all face is how to deal with information that is encoded only implicitly in the model's structure. Expressing cause–effect statements as constraints over a space of functions allows discussing how to incorporate those statements in a model and still: (1) discuss different structures in the model, avoiding a structureless approach used in iSPN, (2) without imposing a set of structures over the TPM [as is the case of Darwiche (2022)], and (3) without relying on transformations between models that can lose information [as is the case in Papantonis and Belle (2020)] . The expression of cause–effect statements as constraints over a space of functions is applicable to any model that can be described by a set of functions and not just to TPM for which it was developed.

In this study, we make the following contributions:

Definition 0.0.1. An ESCM is a 7-tuple  U,C,T,D,F,G,P(U,D) .

All information a SCM depends on is declared explicitly in an ESCM while making minimal changes to the definitions of SCM (Pearl, 2009; Bareinboim et al., 2022). The exogenous variables U, the probability distribution over them P(U), and the set of functions F keep the meaning they have in SCM. The changes of definitions of variables in SCM (interventions) are carried out explicitly by functions in G. For each variable v_i in SCM, there is a variable c_i,d_i, and t_i in an ESCM (see Figure 1). These three types of variables correspond to different types of information about a variable in SCM. A variable c_i corresponds to what we can infer from its modeled causes, i.e., it contains information about the corresponding variable in V (in SCM) which can be inferred from computing the corresponding function in F (in SCM). A variable d_i corresponds to sources of information outside of what is modeled (interventions on the model) that impact the information we have regarding a variable. A variable t_i corresponds to information about the corresponding variable in V (in SCM) given an intervention or its absence. Every c_i is a function of the full information of its modeled causes and background factors, hence: c_i = f_i(Pa_{T, U}(c_i)). A variable d_i encodes the information pertaining to interventions on the endogenous variable i that is present in t_i and missing in c_i, therefore: t_i = g_i(c_i, d_i). In order to be able to express uncertainty over the set of functions G, the object that modeled uncertainty in SCM (P(U)) was extended to P(U,D). In ESCM, it can be stated that D and U alone capture all information over the model, i.e., C and T are derived from them. Therefore, all uncertainty in the model can be attributed to P(U,D), and characterizing the uncertainty over the other sets of variables is redundant. An algorithm for generating an ESCM from a SCM is provided in the Supplementary material along with an example.

The link between variables in the ESCM and SCM is established through the functions that are used. In SCM, there is no explicit mention of a set of functions (like G in ESCM) that implements a change of definition of variables in V and no set of variables (like D in ESCM) that characterizes the behavior of those functions. It is the usage of F and G that motivates the replacement of V in SCM by C and T in ESCM. In ESCM, the functions in F and in G take distinct sets of variables as inputs (a function in F takes variables in the sets U and T while a function in G takes variables in the sets C and D), and their output values are attributed to distinct sets of variables (the outputs of functions in F are assigned to variables in C and the outputs of functions in G are assigned to variables in T). An asymmetric causal relation between variables in SCM is easier to express in ESCM because: (1) the sets of variables in the inputs and outputs of both F and G are disjoint and (2) there is asymmetry in the information in C and T.

For two variables {v_e, v_c}∈V in a SCM that correspond to the sets {c_c, d_c, t_c} and {c_e, d_e, t_e} in an ESCM, we have that vc→Causesve⇒ce=fe(tc,...) which expresses an asymmetric relation in the sense that it is different from c_c = f_c(t_e, ...). This contrasts with SCM where a single variable v refers to both information pertaining to the respective c and t, and for that reason, arrows in a graphical model (see Figure 1) are necessary to express the asymmetry of causal relationships. The asymmetry in models is further discussed in the Supplementary material.

A causal relation in ESCM is defined by setting which variables in T are arguments in a function that outputs the value of a variable in C and hence:

Lemma 0.0.1. A causal relation in a model implies the existence of a constraint of the type expressed in Equation 1.

Proof. Cause–effect relationships define the arguments of functions in a model, and by definition, a function only depends on its inputs. This essentially derives from the independence assumption of a variable's output from the non-causes of this variable in the SCM.    ☐

Corollary 0.0.1.1. The zero sensitivity of a causal model to some variable can be assessed through cause–effect relationships using the chain rule for derivatives, without needing to specify the functions the model decomposes to.

Current TPMs, namely, probabilistic generating circuit (PGC) (Zhang et al., 2021) ⊃ SPN (Poon and Domingos, 2011) ⊃ arithmetic circuit (AC) (Darwiche, 2002) ⊃ PSDD (Kisa et al., 2014), are defined as recursive calls of functions defined over a semiring with operations ⊕, ⊗ (Friesen and Domingos, 2016) and get their properties via structural constraints (Shen et al., 2016; Choi and Darwiche, 2017). A clear example appears in the PSDD literature. The structure of PSDD is based on Sentential Decision Diagrams (SDD) (Darwiche, 2011). SDD are defined with ⊕ and ⊗ as ∨ and ∧ over Boolean values. Despite PSDD being functions in ℝ⁺ where ⊕ is + and ⊗ is × , it is common in PSDD-like structures to draw analogies between the two different semirings. The least amount of structural constraints that is imposed in order to ensure the construction² of a model with tractable marginalization contains the properties:

It will be assumed that all computations are performed explicitly, that is, there are no edge weights in the CG. This does not affect the size more than a constant factor as every such input could be replaced by one multiplication. The structural properties of current TPM depend on scope partitions; hence, the question of how to handle scope arising from the parameters is pertinent. When the parameters are not outputs of functions, it is considered that they do not contribute to the scope which enables us to rule out smoothness and decomposability related issues arising from parameters in that case. When the parameters are functions of some variables in the model, they contribute to the scope of the overall model. In that case instead of thinking of the TPM as a model over the initial variables, we should think of it as a model over the augmented set of variables that includes the parameter variables.

In Figure 6, it can be seen that the best results were obtained with the first type of model (Type_Ord) and that the more parameters in the second type of model, the better the results. This empirically validates the argument in Section 3.2 that for smooth and decomposable TPM, not adhering to an order of computations expected from the function decomposition implied by a set of cause–effect relationships can increase the amount of parameters required to yield some level of accuracy. This is also corroborated (for the case of the Asia dataset) by the number of parameters used in iSPN (Zečević et al., 2021) that for each interventional case and the same dataset ranged from 600 to 3,200 in increments of 600, which is much bigger than 86 used in Ord₈₆ for all interventional cases. The first type of model can be seen in light of the second type as a model where: (a) each of the parameters is a function of a constant bias term, (b) there exist indicator functions for both the observations and interventions of variables, and (c) the structure of the CG is constructed based on cause–effect relationships so that information about effects is not processed before information about their causes . While the second type of model behaves as an SPN for each intervention case, the first type of model is a TPM over observations and interventions.

A set of cause–effect relationships implies the existence of a function decomposition (see Section 2) that reduces the number of parameters for a model but yields no information about the values of the non-zero parameters. This makes the space of functions considered for the first type of model used in the experiments much smaller than that of the second type of model that does not use a principled way of trimming down the number of hypothetically good models before looking at the training data. There is a parallel between the construction of the TPM based on processing information pertaining to causes before the effects in a CG and the variable elimination algorithm used in Darwiche (2022). In both, we have that: (a) information about a cluster of variables is created with a layer of ⊗ operations, and (b) a set of conditional independence relations that follow from the cause–effect relationships allows us to state that the value of some variable that is yet to be included in the CG does not depend on some variable already in the CG which in turn allows us to aggregate information with ⊕ nodes. In the second type of model, the clusters we make do not rely on the cause–effect relationships to choose an order by which information is processed. Therefore, we cannot necessarily use the conditional independence relations to rule out at each ⊗ layer some of the information. The size of a cluster of variables increases exponentially with the number of variables; hence, in the worst of cases, for an exact answer, the number of states we can have in one layer of the model can become exponentially bigger. It is known that the order of variable elimination can lead to expressions of different sizes for the same input–output behavior (Koller and Friedman, 2009). However, the structure of the second type of model is not amenable to such an analogy as the clusters we express in the CG do not capture all the neighbors of a variable in a ground truth undirected graphical model, which would lead to over-sized clusters before each “elimination” of a variable. Moreover, it can be stated that the lack of constraints over the space of functions prevents the extrapolation of results to cases that are either absent or are rare in the dataset.

In Figure 12A, it can be verified that the different models fitted to the synthetic dataset give different answers to queries. In Figure 12B, it can be seen that an intervention on a cause of a variable can impact its c value. In Figure 12C, it can be observed by comparison of the bars RatioOfCountsTr that an intervention on a non-cause of a variable has nearly the same number of samples but not exactly the same. This is attributed to random sampling generation and finite dataset size. The Type_Ord model is closer to the RatioOfCountsTr than Type_Tree models that also vary more between the two queries.

As an artifact of random sampling on data generation and finite dataset size, there is a difference (see Figures 7–10) in the height of the bars for Ord₈₆ that corresponds to model counting with a factorization compatible with the data generation and the bars for RatioOfCountsTr that corresponds to model counting without a factorization. In Figures 7, 8, the mean responses to queries over the second type of model are within one standard deviation from Ord₈₆, but they are far from both Ord₈₆ and RatioOfCountsTr in the rightmost graphs shown in Figures 9, 10 that correspond to scenarios with two interventions that set the intervened variables to a uniform distribution. In Type_Ord models, the structure allows the extrapolation to be performed correctly but the same cannot be said for the neural network that provided the parameters for each of the iSPN-like models. The same issue can be observed in Figure 13. The neural networks used for the Type_Tree models have only been trained on inputs with values one or zero which left the function that ought to be learned undefined in between, that is, no constraints (other than the function being piece-wise linear due to the activation functions of the neurons) were imposed on how the outputs of the neural networks should change as inputs vary from zero to one. More than a problem of lack of structure in the SPN to which the parameters are assigned, this is a problem of generalization(extrapolation to cases not observed during training) as the height of the bar RatioOfCountsTr is close to Type_Ord.

Another significant difference is that for iSPN-like models, where interventions are used only for providing the weights of an SPN, we cannot compute joint likelihoods involving different interventional cases. An example of such queries is present in Figure 11 where . The joint likelihoods of t₄ and c₃ are computed with L_*(L_*(I_f(c₃), I_f(c₄)), I_f(d₄)) which is a different branch from where d₂ is computed (see Figure 4B). Therefore, the value of each joint likelihood query over Val(c₃, t₄) conditioned on the intervention over d₂ does not depend on d₂ as P(c₃, t₄, d₂) = P(c₃, t₄)P(d₂) which implies that P(c3,t4|d2)=P(c3,t4)P(d2)P(d2)=P(c3,t4).

Causal assertions stem from an asymmetric relation between some variable, its causes, and effects. Both the causes and effects are correlated with information about the state of a variable. A variable is only correlated with the partial information about its effects which does not include factors of variation outside of the modeled ones (e.g., Interventions). However, by definition, a variable is correlated with the full information pertaining its causes, something that is not accessible without simplifying assumptions. By making those assumptions explicit, structural causal models are extended and causality is defined as a constraint over the function space of a higher dimensional model. Current TPM and cause–effect relationships imply distinct function decompositions. In the decomposition implied by cause–effect relationships, the endogenous variables are outputs of functions and in the TPM they are inputs in the model. The mismatch is only apparent because it is resolved when taking an input–output perspective as in both cases exogenous variables exist and all aspects of both models depend on them. The process of answering queries pertaining to interventions with SCM uses an algorithm external to the SCM to adapt its structure for the execution of that intervention. This is not the case with ESCM where that process is included in the model through the set of functions G. Therefore, by using ESCM instead of SCM as a starting point for compilation of TPM, the usage of an algorithm external to the model that changes it in order to adapt it to answer questions pertaining to interventions was avoided. It was shown that implementing that algorithm explicitly leads to a TPM that in the worst case has a linear size increase in the number of variables and the maximum number of states of a variable. Sufficient conditions for implementing it in a generalization of current classes of TPM to other semirings are stated. The functional approach is used to unify under one framework the distinct approaches for modeling TPM with causality. It enables us to both explain adherence to cause–effect constraints without explicit structure in a function decomposition as in iSPN (Zečević et al., 2021) and the role of structure in compilations from SCM to TPM (Darwiche, 2022) as an implicit way of imposing constraints over a function space. It was discussed and shown empirically that choosing not to adhere to a function decomposition consistent with an order implied by a set of cause–effect relationships can lead to a big increase in size requirements for a smooth and decomposable TPM.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

DC: Conceptualization, Formal analysis, Investigation, Methodology, Resources, Software, Visualization, Writing—original draft. JB: Formal Analysis, Supervision, Validation, Writing—review & editing.