InfoGAN, which stands for Information Maximizing Generative Adversarial Network (Chen et al., 2016), is an expansion of the conventional Generative Adversarial Network (GAN) framework (Goodfellow et al., 2014). InfoGAN's primary objective is to uncover and manage the structured representations inherent in the data it generates. In the realm of GANs, there exist various variants based on statistical properties, such as Conditional GAN (CGAN) as discussed in Mirza and Osindero (2014), the f-GAN as explored in Nowozin et al. (2016), and Wasserstein GAN (WGAN). InfoGAN itself has also given rise to variants like Causal InfoGAN, as described in Wu et al. (2019), and Semi-Supervised InfoGAN (ss-InfoGAN) as detailed in Kurutach et al. (2018). These models have been widely adopted due to their flexibility in modeling complex, high-dimensional distributions and their empirical success across a broad range of applications.

InfoGAN has applications similar to vanilla GANs, including data imaging, natural language processing, and medical image analysis (Reed et al., 2016; Zhu et al., 2017; Yi et al., 2019). A recent review of GANs and their applications is provided in Gui et al. (2023). Beyond classical InfoGAN, several recent InfoGAN-inspired models incorporate additional information-theoretic structure and disentanglement mechanisms. For instance, IB-GAN introduces an information bottleneck constraint within GAN training to encourage disentangled representations while remaining partially InfoGAN-like in spirit (Jeon et al., 2025). Similarly, Double InfoGAN extends InfoGAN ideas to contrastive analysis by leveraging InfoGAN-style regularization to separate common vs. salient generative factors (Carton et al., 2024).

Despite their empirical success, the theoretical foundations of GANs and InfoGANs are not well established, and numerous issues related to their theory and training dynamics remain unresolved (Reed et al., 2016; Liang, 2021; Singh et al., 2018). This has motivated a growing body of recent work aimed at improving stability and generalization in adversarial training. For example, CHAIN proposes a Lipschitz-constrained normalization strategy that targets discriminator overfitting in data-limited regimes and supports improved stability and generalization through theoretical analysis (Ni and Koniusz, 2024). Relatedly, VE-cGAN develops a recent generalization framework for conditional GANs using vicinal estimation, addressing challenges such as limited conditional samples and high-dimensional outputs (Jang and Hwang, 2026). In particular, understanding the statistical generalization properties of adversarially trained models remains a central challenge in modern machine learning theory.

A key question in GAN research is how well these models can approximate a target distribution using a limited number of samples. For instance, the authors in Reed et al. (2016) showed that GANs may fail to generalize under standard metrics even with a polynomial number of samples and established generalization bounds based on neural network distance. The work in Zhang et al. (2018) further analyzed neural network distance and expanded upon these findings. The authors in Liang (2021) and Singh et al. (2018) approached the problem from a nonparametric density estimation perspective. These works highlight both the difficulty and importance of developing rigorous learning-theoretic guarantees for adversarial models.

These recent directions reinforce the need for learning-theoretic guarantees for adversarial objectives, particularly when additional regularization or structural modifications are introduced into the generator or discriminator. However, existing results still have notable shortcomings, and the theoretical analysis of InfoGAN remains relatively rare in the literature. In particular, most available results focus on vanilla GAN objectives and do not address the additional structural components introduced by InfoGAN, such as latent codes and mutual information regularization. A natural direction for theoretical investigation is therefore to evaluate the generalization error of InfoGAN-type objectives under generator regularization by comparing the population objective to its empirical counterpart.

We emphasize that the framework studied in this study is not classical InfoGAN in its original form. By removing the latent code variable and introducing an explicit generator regularization term, we obtain an InfoGAN-inspired adversarial objective that is analytically tractable for generalization analysis. This modification preserves the adversarial structure of GANs while enabling explicit control of the generator through regularization. Throughout the study, we therefore focus on the generalization behavior of this generator-regularized adversarial model rather than classical InfoGAN with latent codes.

From a statistical learning perspective, the generator regularization term plays a role analogous to penalization in nonparametric estimation, providing capacity control and enabling explicit bounds on the generalization gap. This viewpoint allows us to bridge ideas from empirical process theory and adversarial learning.

GANs differ from classical density estimation methods by implicitly learning the data distribution through an adversarial process between a generator and a discriminator. Let the generator be denoted by G with sample size m and the discriminator by D with sample size n, where D aims to distinguish between the data distribution p_x and the generator distribution p_z. Let z be a noise variable distributed according to p_z and X denote a real data variable. The generator transforms noise samples into synthetic data points, while the discriminator attempts to distinguish these generated samples from real observations. Consider GAN models in which both the generator and discriminator function classes are parameterized. The minimax problem of GAN introduced in Goodfellow et al. (2014) can be written as

The InfoGAN framework extends this setup by dividing the noise variable z into an incompressible noise component and a latent code c, so that the generator takes the form G(z, c). The InfoGAN objective (Chen et al., 2016) is given by

where I(c; G(z, c)) = H(c)−H(c|G(z, c)) denotes the mutual information between the latent code and the generated sample, and λ ≥ 0 is a regularization parameter. The mutual information term encourages the generator to encode interpretable structure in the latent variables. However, optimizing I(c; G(z, c)) is difficult since it requires the posterior distribution P(c|x).

To address this, a lower bound L_I(c; Q) is introduced by defining an auxiliary distribution Q(c|x) to approximate P(c|x). The practical InfoGAN objective is therefore written as

While Equation 3 serves as the primary objective function commonly used in applications, this study opts to consider and subsequently employ Equation 2 as the core objective function for its theoretical analysis. This choice allows us to isolate the effect of generator regularization and to derive explicit learning-theoretic guarantees. This objective function introduces regularization in the generator variable, a departure from the majority of existing literature, which typically lacks such regularization.

The existing theoretical research is primarily based on vanilla GAN error analysis defined by the difference between empirical and population objectives, as in Liang (2021), Huang et al. (2022), Ji et al. (2021), and Zhang et al. (2018). A preprint of this work has previously been published in Hasan and Muia (2025). In this study, the objective function (Equation 2) is used to study generalization properties for an InfoGAN-inspired framework without latent variable c in the setting of two-layer neural networks. In this work, we deliberately exclude the latent code variable c in order to focus on a generator-regularized adversarial objective that admits explicit generalization analysis. This choice is motivated by analytical tractability rather than by the representational goals of classical InfoGAN. We stress that our results do not apply to classical InfoGAN with latent codes and variational mutual information terms. Moreover, the logarithmic function satisfies logx → −∞ as x → 0, which may lead to instability in practice. We therefore develop a new objective function without a latent code and with a stable measuring function. The generalization error is defined as the difference between the population version of the objective function and its empirical counterpart. Our analysis quantifies this difference using tools from empirical process theory. The difference between the population and empirical objective functions is bounded using Rademacher complexity. The resulting bounds are derived explicitly for two-layer networks under Lipschitz and non-decreasing activation functions.

Our contributions are threefold: (i) we formulate a generator-regularized, InfoGAN-inspired adversarial objective (without latent code) and cast it as a neural network distance with an explicit generator penalty; (ii) we bound the empirical–population objective gap using Rademacher complexity for the discriminator, generator, and their composition; (iii) we specialize the bounds to two-layer networks under Lipschitz and non-decreasing activations and validate the predicted trends empirically. A concise comparison between classical InfoGAN and the generator-regularized objective studied in this paper is provided in Table 1. The main theoretical contributions and organization of the study are summarized as follows:

Our theory is developed for bounded two-layer networks and an objective without latent codes; Section 6 discusses the implications of these assumptions and directions toward deeper architectures and classical InfoGAN settings.

In the original InfoGAN framework, instead of using a single unstructured noise vector z, the authors divide the input noise vector into two components: an incompressible noise variable, still denoted by z, and a latent code denoted by c. The generator is trained adversarially to confuse the discriminator while simultaneously maximizing the mutual information between the latent code and the generated samples. This additional structure is intended to encourage the emergence of interpretable and disentangled representations in the generated data.

In this work, we focus on a simplified yet analytically tractable setting by excluding the latent code variable. Specifically, we consider the case in which the latent code is absent and effectively set c = 0. This modification allows us to isolate the effect of generator regularization and to derive explicit generalization bounds without the additional complexity introduced by latent-variable inference.

From a theoretical standpoint, removing the latent code eliminates the need to handle variational approximations of mutual information, thereby enabling a direct empirical process analysis of the adversarial objective.

Throughout this section, we assume that the generator output G(z) admits a density with respect to Lebesgue measure, is bounded, and satisfies G(z) ∈ [0, 1] almost surely. Under these assumptions, the entropy and expectation terms involving logG(z) are well-defined and finite. These regularity conditions ensure that all subsequent expectations and entropy terms are mathematically well-posed. Under this setting, Equation 2 reduces to:

Here, mutual information can be represented equivalently as I(0;G(z, 0)) = H(0)−H(0|G(z, 0)), where H denotes entropy. Equation 4 presents the objective function with generator regularization in the case where the latent code is zero. In this formulation, the regularization term acts directly on the generator distribution, penalizing low-entropy or degenerate outputs. Under the density and boundedness assumptions stated above, the differential entropy of G(z) satisfies H(G(z)) = −𝔼_pzlogp_G(G(z)), where p_G is the density of G(z). In our simplified setting, we use a bounded surrogate regularizer of the form −𝔼_pzlog(G(z)) to obtain an analytically tractable generator penalty; replacing log by ϕ in Equation 5 yields a stable objective compatible with integral probability metric analyses. This surrogate regularization can be viewed as a tractable proxy for entropy control on the generator output. However, this can lead to issues in practice, as logx → −∞ as x → 0. Such behavior may cause numerical instability and poor gradient behavior during optimization. By replacing log with a monotone function ϕ:[0, 1] → ℝ, the objective becomes:

Assumption 1 (Measuring function ϕ). Throughout, the measuring function ϕ:[0, 1] → ℝ is assumed to be non-decreasing and L_ϕ-Lipschitz. This ensures that ϕ ∘ D remains uniformly bounded and allows standard contraction arguments in the Rademacher analysis.

The replacement of the logarithmic function by a general monotone measuring function ϕ is motivated by both theoretical and practical considerations. In particular, the logarithmic function becomes unstable near zero, while monotone functions allow the objective to be interpreted within the neural network distance and integral probability metric frameworks commonly used in GAN theory. From a learning-theoretic perspective, this replacement also facilitates the use of contraction inequalities and simplifies the derivation of complexity bounds. Here, ϕ is a non-decreasing Lipschitz measuring function (Assumption 1). This can also be written as Reed et al. (2016):

For ϕ(x) = x, the final objective function with changing the notations becomes:

Equation 7 can be interpreted as a neural network distance augmented with an explicit generator regularization term. This formulation is consistent with existing generalization analyses of GANs based on integral probability metrics and neural network distances, while introducing additional control over the generator through regularization. In particular, it fits naturally within the framework of integral probability metrics with a penalized generator class.

Equation 7 therefore represents a generator-regularized neural network distance. While regularization could in principle be applied to either the discriminator or the generator, we emphasize that in the absence of a latent code variable, the regularization term naturally acts on the generator. This choice is also aligned with the role of the generator as the primary source of model complexity in adversarial learning. Consequently, the regularized objective function in Equation 7 is particularly suitable for adversarial models in which the generator takes an unstructured noise variable as input.

Suppose that {Xi}i=1n are independent and identically distributed observations drawn from the data distribution p_x, and that the generator produces {G(zj)}j=1m as independent and identically distributed samples drawn from the model distribution p_z. We assume throughout that the data sample and the noise sample are independent.

We define the two empirical loss functions as follows, based on Equation 7:

and

Equation 8 is the fully empirical objective (empirical averages over both the data sample and the noise sample), whereas Equation 9 is a mixed empirical–population objective (empirical average over the data sample and population expectations over the noise distribution). These two formulations will be used to quantify different sources of statistical error in the subsequent generalization analysis. Here, D(G(z)) = D ∘ G is the composition of the discriminator and generator.

Notation

The generalization bound of InfoGAN is defined by the difference between the empirical and population versions of the objective function. In particular, we consider the discrepancies between the empirical objective in Equation 8 and its population counterpart in Equation 7, and between the mixed empirical–population objective in Equation 9 and its population counterpart in Equation 7. Considering D^ and Ĝ as the empirical counterparts of D and G, respectively, the difference in the objective function can be represented as:

In Equation 10, this indicates the difference between the empirical objective (based on both samples) and the population objective. Meanwhile, Equation 11 compares the mixed empirical–population objective with the population objective. The subsequent theorem establishes bounds for Equations 10, 11, assuming that both the discriminator D and generator G are uniformly bounded. The proof employs the Cauchy-Schwarz inequality and McDiarmid's inequality. Throughout, we assume that the data sample {xi}i=1n and the noise sample {zj}j=1m are independent, and that each sample is i.i.d. from its respective distribution.

Notation

Recall that n denotes the sample size for x₁, …, x_n ~ p_x, and m denotes the sample size for z₁, …, z_m ~ p_z. We write Rn(D) for the (expected) Rademacher complexity of the discriminator class evaluated on n samples from p_x, and Rm(D∘G) for the complexity of the composed class {D∘G:D∈D,G∈G} evaluated on m noise samples from p_z. For clarity, Rn(·) denotes the (expected) Rademacher complexity, i.e., the expectation is taken over both the sample and the Rademacher signs.

Theorem 3.1. Suppose the sets of discriminator functions D and generator functions G are symmetric with ∥f∥_∞ ≤ ℚ_x for all f ∈ D and ∥g∥_∞ ≤ ℚ_z for all g ∈ G. Then, with probability at least 1 − 2δ over the random training samples {xi}i=1n and {zj}j=1m, we have

and

Proof: We first bound Equation 10. Using the definition of Equations 7, 8 and the standard inequality

we obtain

The second term is taken over the composed class D ∘ G, and the third term is taken over G because the generator regularization term does not involve D.

We now bound each term using standard symmetrization and McDiarmid/Hoeffding-type concentration; see, e.g., Theorem 3.1 in Zhang et al. (2018) for this template. For completeness, we note that the bounds below follow from (i) symmetrization, (ii) the Rademacher contraction principle for bounded function classes, and (iii) McDiarmid's inequality (or Hoeffding's inequality) applied to bounded differences.

(i) Discriminator term

With probability at least 1 − δ,

(ii) Composition term

With probability at least 1 − δ,

Here we use that for any D ∈ D and G ∈ G, the composition D ∘ G is uniformly bounded by ||D||_∞ ≤ Q_x, and we absorb constants into Q_z for notational simplicity (as in Section 4).

(iii) Generator regularization term With probability at least 1 − δ,

Finally, combining Equations 15, 16, 17 into Equation 14, and applying a union bound over the three events (absorbing constants so that the final probability is at least 1 − 2δ), yields

which is Equation 12.

The bound Equation 13 follows directly from Equation 15, since dI(D^,G) differs from d_I(D, G) only through the empirical approximation of 𝔼_pxD(x). In particular, the generator-related terms remain at their population values in Equation 9, so only the discriminator sampling error contributes to Equation 11.

Remark 3.1. The generalization bound in Theorem 3.1 decomposes the gap between the empirical and population objectives into (i) complexity terms, measured by Rademacher complexities, and (ii) finite-sample concentration terms, controlled by the uniform bounds and the sample sizes.

The term 2Rn(D) captures the statistical complexity of the discriminator class when evaluated on the data sample {xi}i=1n.

The term 2Rm(D∘G) measures the complexity of the composed class D ∘ G when evaluated on the noise sample {zj}j=1m through the generated points G(z_j).

The additional term 2λRm(G) arises from the generator regularization component in the objective. Since the regularization term depends only on G, its empirical population deviation is controlled by the Rademacher complexity of the generator class itself.

Finally, the remaining terms 2Qxlog(1/δ)2n and 2Qz(1+λ)log(1/δ)2m arise from concentration of empirical means around expectations under the uniform boundedness assumptions.

Overall, Theorem 3.1 shows that the empirical objective approaches its population counterpart as the sample sizes grow and as the effective complexities of D, D ∘ G, and G are controlled. In particular, the bound makes explicit how two sources of sampling error contribute separately: the data-sampling error scales with n through Rn(D) and the concentration term, while the noise-sampling error scales with m through Rm(D∘G), Rm(G), and the corresponding concentration term. Moreover, the regularization strength λ amplifies the generator-only terms, reflecting a natural bias–variance trade-off: larger λ increases the contribution of Rm(G) and the m-dependent concentration term, while potentially improving stability and controlling generator outputs.

This section instantiates the general generalization bounds in Theorem 3.1 for concrete two-layer (one-hidden-layer) neural network classes. We (i) define discriminator and generator hypothesis classes with explicit ℓ₁-type constraints that control capacity, (ii) bound Rn(D), Rm(G), and Rm(D∘G) using covering numbers and Dudley-type entropy integrals, and (iii) plug these bounds into Equations 12, 13 to obtain explicit rates in n and m under two common activation assumptions: Lipschitz and non-decreasing.

The derived bounds in Theorem 3.1 provide valuable insights when applying the infoGAN framework in Equation 7 to a two-layer neural network architecture. In this section, we discuss how these bounds can be useful in analyzing and improving the performance of such networks. The goal is to minimize the objective function disparity between the empirical distributions of D^ and Ĝ, as well as the objective function difference between D^ and G. The derived bounds, as shown in Equations 12, 13, provide upper limits on the disparity and difference in the objective functions, respectively. These bounds allow us to assess the potential deviation between the empirical and true objective functions. Furthermore, the analysis of these bounds offers insights into the convergence behavior of the two-layer network. In this section, we will focus solely on the theoretical framework of two-layer neural networks. The applications of a two-layer neural network for the readers can be found in the recent studies by Wang et al. (2019) and Nian and Yao (2018). Our emphasis is on explicit learnability guarantees: we quantify how sampling error decays as n and m increase, and how architectural constraints (through V and the activation choice) control the effective complexity of the adversarial objective.

It is important to note that neural network classes are typically infinite, so bounds involving finite cardinalities such as log|D| or log|G| are generally not appropriate. In this section, we therefore derive all Rademacher complexity bounds using covering numbers and entropy integrals (Dudley-type bounds), which are standard tools for infinite hypothesis classes.

In this study, we studied the generalization properties of an InfoGAN-inspired adversarial framework in which the latent code variable is removed and an explicit regularization term is introduced on the generator. By analyzing the difference between the empirical and population versions of the adversarial objective, we derived generalization bounds in terms of the Rademacher complexities of the discriminator, generator, and their composition. These bounds reveal explicit n^−1/2 and m^−1/2 decay rates and highlight the role of the generator regularization parameter λ. A key feature of our analysis is the explicit separation of the two statistical error sources: the data-sampling error governed by n through Rn(D), and the noise-sampling error governed by m through Rm(D∘G) and Rm(G).