Artificial-intelligence (AI) models now underpin safety-critical functions in medical imaging, autonomous driving, and national security. Their growing deployment, however, has exposed a systemic weakness: carefully crafted adversarial examples, minute, often imperceptible perturbations to inputs, can trigger grossly erroneous model outputs (Ren et al., 2020). A stop sign modified with a few strategically placed stickers, for instance, may be interpreted by an autonomous-vehicle vision system as a yield sign, jeopardising passenger safety (Maple et al., 2019). In clinical practice, subtle pixel-level changes to radiographic images could prompt a diagnostic system to overlook malignancies or flag healthy tissue as pathological, adversely affecting patient management.

The threat landscape extends across digital identity and surveillance. Adversarial audio or visual artefacts have been shown to bypass state-of-the-art facial- and voice-recognition pipelines, enabling unauthorised access to secure facilities and devices while undermining the evidentiary integrity of CCTV footage (Wang J. et al., 2019). Malicious actors can also weaponise generative models to fabricate photorealistic or audio-realistic content, fuelling misinformation campaigns, reputational damage, and electoral manipulation. Voice-activated assistants are vulnerable to inaudible command injections, compromising user privacy and data integrity, whereas algorithmic decision-making in policing, recruitment, and credit scoring remains susceptible to false positives and false negatives with significant societal repercussions.

At the geopolitical level, adversarial exploitation of AI-enabled defence platforms raises the spectre of cyber-warfare, espionage, and strategic destabilisation. Commercial enterprises are likewise exposed: a single successful attack on an AI-driven recommendation or pricing engine can erode consumer trust and precipitate substantial financial loss. Generative-AI chatbots can serve as high-throughput front ends for gathering a diverse spectrum of user information, ranging from product-preference signals and free-text feedback to demographic attributes such as age, gender, and approximate location. Their ability to sustain thousands of concurrent dialogues and to operate continuously affords firms an efficient, low-latency mechanism for capturing customer insight at scale, which in turn supports real-time personalisation of marketing campaigns and service delivery.

Notwithstanding these benefits, two limitations are critical. First, large-language-model (LLM) services such as ChatGPT, in their default configuration, neither persist user-specific content nor expose database-like retrieval endpoints (Figure 1). Consequently, organisations that require structured data capture must integrate the model within a bespoke pipeline that extracts, stores, and post-processes conversation metadata. Second, any deployment that ingests personal data must comply with prevailing regulatory regimes, most notably the EU General Data Protection Regulation (GDPR) (GDPR, 2018; ICO, 2018; Peloquin et al., 2020), the California Consumer Privacy Act (CCPA) (CCPA, 2018), and sector-specific standards for consent and retention. Custom implementations therefore need robust consent workflows, explicit purpose limitation, encryption at rest and in transit, and audit logging to mitigate legal and ethical risks associated with large-scale conversational data harvesting.

In Figure 1, we can see that ChatGPT is designed to forget the user’s personal information after the conversation, ensuring user privacy.

Figure 1 presents a layered data-collection workflow centred on the ChatGPT core LLM. User input enters at the top and is processed by the LLM, which generates responses while simultaneously passing salient metadata to an intermediate Data Interface. This interface funnels structured information into a persistent Data Store, but only after routing it through a dedicated Consent Manager that verifies user authorisation and applies opt-in policies. A parallel path leads all collected data (raw and processed) through a Privacy-Compliance checkpoint that enforces encryption, retention limits, and jurisdiction-specific regulations (e.g., GDPR, CCPA).

The primary contribution of this work is the formulation of a system-level analytical framework that reconceptualises adversarial AI security across perceptual, cognitive, and executive layers, rather than the enumeration of individual attack techniques. Secondary contribution is the formalisation of agentic AI security as a multi-layered problem spanning perception, cognition, and executive control. By mapping adversarial vulnerabilities onto autonomy, self-governance, and closed-loop decision-making, the proposed framework extends adversarial machine learning from input-level robustness to system-level behavioural integrity. This shift enables systematic reasoning about failure modes that are not adequately captured by conventional threat models, including temporal error accumulation, goal misalignment, and policy-level manipulation. As such, the taxonomy functions as an explanatory and generative framework, guiding empirical evaluation and the design of next-generation defence mechanisms for autonomous and agentic AI systems.

This study employed a multi-phase analytical methodology integrating a Systematic Literature Review (SLR), Bibliometric Analysis, and Experimental Validation to comprehensively investigate adversarial vulnerabilities in both conventional and agentic artificial intelligence (AI) systems. The methodology aligns with PRISMA 2020 guidelines for systematic reviews and conforms to bibliometric standards established by the Bibliometrix framework in R Studio.

To ensure methodological transparency and reproducibility, this study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) guidelines in structuring the literature review on adversarial vulnerabilities in agentic artificial intelligence (AAI). The objective of the review was to identify, evaluate, and synthesise empirical and theoretical research addressing adversarial attacks, defence mechanisms, and agentic properties (autonomy, self-governance, and decision-making loops) in AI systems.

To ensure wholistic review of existing work, and to gather all data available on this subject, the first search was performed on the Web of Science Core Collection on the topic of Adversarial Attacks on Agentic AI, which produced 101 results (date 22 October 2025). This file was extracted from the Web of Science Core Collection and analysed in R Studio using the Bibliometrix plugging with Biblioshiny (Aria and Cuccurullo, 2017).

The first step was to perform Factorial Analysis (Figure 2) followed by a dendrogram (Figure 3).

The factorial analysis (Multiple Correspondence Analysis, MCA) of the Web of Science dataset on “Adversarial Attacks on Agentic AI” in Figure 2, reveals a clearly defined conceptual structure centred on three clusters of related terms. The largest semantic field is anchored around “adversarial machine learning,” “deep learning,” and “cybersecurity,” indicating that current research situates adversarial studies within security-focused machine-learning contexts. A secondary cluster is formed by “defences,” “review,” and “perturbation methods,” suggesting consolidation of methodological work on defensive mechanisms. The triangular distribution in the MCA map indicates an evolving but interconnected research landscape—agentic autonomy remains a peripheral topic, reflecting that agentic AI as a term is still emerging rather than established. The wide geometric spread demonstrates a heterogeneous but converging field with overlapping technical and conceptual vocabularies.

The hierarchical clustering dendrogram in Figure 3, complements the factorial analysis by illustrating the taxonomic proximity between topics. It shows dense clustering among keywords such as “machine learning,” “deep neural networks,” “adversarial training,” and “defence mechanisms,” confirming that the core literature is technically driven. Smaller, distinct branches appear for “autonomy,” “reinforcement learning agents,” and “decision-making,” showing that agentic elements are currently treated as specialised subtopics rather than mainstream research axes. The height of the clustering tree demonstrates significant semantic distances between these subfields, underscoring fragmentation in how agentic attributes (autonomy, self-governance) are addressed within adversarial ML research. This structure highlights a field transitioning from isolated algorithmic studies toward integrated system-level inquiry.

The second search parameters used was ‘Adversarial Attacks on Artificial Intelligence’, which produced 973 results (date 22 October 2025). This data was analysed with a Thematic Map (Figure 4).

The thematic map for the broader search on “Adversarial Attacks on Artificial Intelligence” in Figure 4, presents a mature, multi-clustered landscape divided along development (density) and relevance (centrality) axes. The motor themes, including “deep learning,” “adversarial attacks,” “artificial intelligence,” and “classification,” dominate the upper-right quadrant, reflecting a high degree of conceptual maturity and research centrality. The niche themes, such as “explainability” and “interpretability,” exhibit high density but lower centrality, representing specialised, well-developed topics that are methodologically self-contained. Emerging themes, including “adversarial AI” and “adversarial training,” lie in the lower-left quadrant, denoting rapidly growing but not yet consolidated research directions. Thematic dispersion confirms that the field of general adversarial AI is far more saturated and structurally stable than the emergent agentic AI subdomain.

Comparing the two bibliometric datasets, “Adversarial Attacks on Agentic AI” (101 records) versus “Adversarial Attacks on Artificial Intelligence” (973 records), reveals a sharp contrast in research maturity and conceptual cohesion. The smaller agentic AI dataset exhibits dispersed, low-density clusters, reflecting early-stage exploration characterised by methodological borrowing from adversarial ML rather than original frameworks for autonomy or decision loops. Conversely, the general AI adversarial dataset demonstrates thematic centralisation around established paradigms in deep learning, computer vision, and robustness testing, supported by strong interconnections between attack and defence research. The comparison indicates that while adversarial AI research is approaching conceptual saturation, adversarial agentic AI remains a nascent domain, fragmented but promising for defining the next generation of security paradigms integrating autonomy and self-governance into adversarial robustness models.

The bibliometric evidence reinforces the conceptual taxonomy proposed for agentic AI adversarial vulnerabilities. Themes identified in Figure 4, such as adversarial attacks, deep learning, and cybersecurity, align predominantly with perceptual-layer threats, where adversaries manipulate sensory or input representations to mislead AI perception systems. The emerging clusters around adversarial training, interpretability, and explainability correspond to the cognitive layer, reflecting attempts to secure internal reasoning processes and improve model self-awareness. Meanwhile, niche but growing themes related to autonomous systems and decision-making point toward the executive layer, where control logic and goal prioritisation become targets for higher-order adversarial influence. Collectively, these findings indicate that while current research remains concentrated on perceptual and cognitive defences, the executive dimension, central to agentic autonomy and self-governance, remains underexplored. This gap highlights the need for a new generation of adversarial-security frameworks that explicitly integrate the multi-layered structure of agentic AI, bridging perception, cognition, and executive control within a unified resilience paradigm.

Agentic Artificial Intelligence (AAI) extends beyond conventional pattern-recognition models by incorporating autonomy, self-governance, and goal-directed decision-making loops (Acharya et al., 2025). In contrast to static predictive systems that passively map inputs to outputs, agentic systems operate as persistent entities within dynamic environments, continually perceiving, reasoning, and acting based on internal objectives. This agentic capability introduces a new adversarial surface: vulnerabilities not only in model inference but also in planning, feedback integration, and goal adaptation.

The field of machine learning, specifically deep learning, deals with adversarial attacks (Szegedy et al., 2013, 2015), but we need more specialised methods for detecting cyber-attacks (Ye et al., 2006). This means creating inputs that can purposely mislead a model into making incorrect predictions or classifications (Chejara et al., 2013). These inputs are adversarial examples and can be hard to detect (Costa et al., 2023), especially in image-based datasets and other database security, such as blockchain cybersecurity (Schlatt et al., 2023). Adversarial attacks deliberately introduce finely tuned perturbations into model inputs, causing systematic misclassification while remaining imperceptible to human observers. The challenge is particularly acute for high-dimensional modalities such as medical or street-scene imagery, where pixel-level changes can easily evade human scrutiny yet induce catastrophic model error. Comparable risks have been demonstrated in structured domains, including blockchain-based anomaly detection, underscoring that adversarial vulnerability is not restricted to vision tasks.

Figure 6 groups attack techniques into three operational categories. White-box attacks assume full disclosure of model internals, architecture, parameters, and sometimes even training data. Such complete knowledge enables gradient-based optimisation of adversarial inputs, exemplified by the Fast Gradient Sign Method (FGSM), the Jacobian-based Saliency Map Attack (JSMA), and the more precise DeepFool algorithm. Black-box attacks, by contrast, treat the target as an oracle, exploiting only input–output pairs. Although gradient information is unavailable, adversaries can estimate it via query-efficient procedures such as Zeroth-Order Optimisation (ZOO), Boundary Attack, or Natural Evolution Strategies (NES). The efficacy of these methods relies on the empirical observation that decision boundaries learned by different models trained on the same task often align; hence perturbations transferable across architectures remain effective despite limited system knowledge. Transfer attacks explicitly exploit this property: an adversary crafts examples on a surrogate (locally accessible) model and deploys them against the remote target, achieving high success rates when data distributions or inductive biases overlap.

Orthogonal to knowledge assumptions is the attack objective. Targeted attacks coerce the model into a prespecified erroneous label, e.g., forcing all ‘stop-sign’ images to be recognised as ‘speed-limit’, whereas untargeted attacks merely seek any classification error. This orthogonal taxonomy clarifies threat-model selection when evaluating defensive strategies or certifying robustness claims.

In white-box attacks, the adversary has complete knowledge of the target model. This includes the system’s architecture, trained parameters, and, in some cases, training data. With this comprehensive understanding, the adversary creates adversarial examples to mislead the target model, e.g., FGSM, JSMA, Deepcool.

With Black-box attacks, the attacker only have access to the input and output of the model. The attacker uses this input–output data to generate adversarial examples. Despite limited knowledge, black-box attacks can be potent because models and intense neural networks can share vulnerabilities across architectures. Transferability is essential, as an adversarial example created for one model can deceive another, e.g., ZOO, Boundary Attack, NES. Transfer Attacks are one type of black-box attack in which an adversary generates an adversarial input by accessing a different model, known as the surrogate model, and then uses it to attack the target model. The idea is that models trained on similar tasks share similar vulnerabilities, making it possible to transfer adversarial examples between them.

Attacks on models can be classified based on their goals. Targeted attacks aim to make the model generate a specific incorrect outcome. In contrast, untargeted attacks are focused on causing the model to make a mistake or be incorrect without specifying the desired incorrect output.

Figure 6 positions white-box, black-box, and transfer attacks along a single “model knowledge” axis and annotates each category with canonical methods (e.g., FGSM, ZOO) plus operational descriptors such as gradient access, query budget, complexity, and typical success. This presents comparison of threat assumptions and highlights how decreasing attacker knowledge generally increases query cost while reducing baseline effectiveness, which is further analysed in Figure 7.

Figure 7 details the adversarial attack and defence dynamics for the:

Various other methods are available for safeguarding against malicious attacks, each with unique benefits (Costa et al., 2023; Liang et al., 2022). These techniques include adversarial training, which involves training a model on adversarial examples to improve its robustness; defensive distillation, which reduces the amount of information available to attackers; and gradient masking, which obscures the gradients of a model to prevent attackers from exploiting them.

Adversarial attacks are prime examples of deceptive exercises (Ozdag, 2018) that can mislead AI systems, causing them to make incorrect classifications or decisions (Anthi et al., 2020). Adversarial manipulation can occur at any stage of the machine-learning lifecycle, data curation, model training, model distribution, or inference, each stage exposing distinct attack surfaces with different observability and forensics burdens. Thinking in lifecycle terms helps relate attack mechanics to defensive control points: if an adversary perturbs inputs only at inference, detection must occur inline or downstream of the model; if the training corpus is poisoned, remediation requires provenance, dataset hygiene, and robust optimisation; if weights are tampered with in transit, supply-chain assurance becomes central. The examples below are organised accordingly to sharpen threat modelling and clarify where specific controls apply.

Inference-time perturbation (feature-space evasion): In its simplest form, an attacker adds a small, norm-bounded perturbation to a single input x such that the perturbed instance x’ crosses the model’s decision boundary while remaining visually or statistically indistinguishable to humans. Although gradient-based methods (e.g., FGSM variants) are canonical in white-box settings, black-box approximations using score queries or decision-only feedback can achieve comparable misclassification with iterative refinement. In high-stakes domains, medical image triage, automated traffic-sign recognition, these perturbations can suppress or amplify class-relevant features, yielding false negatives (missed tumours) or false positives (phantom lesions) with direct safety impact.

Training-time data poisoning: Poisoning attacks corrupt the learning signal by inserting crafted samples into the training set. Two broad regimes matter operationally: dirty-label poisoning (attacker controls both features and labels) and clean-label poisoning (attacker perturbs features but preserves the nominal label so poisoned points survive basic validation). Even low-rate insertions (<1%) can tilt decision boundaries, degrade calibration, or create predictable failure modes under distributional drift. Poisoning also interacts with data augmentation and class imbalance; for example, poisoning rare classes in medical datasets disproportionately reshapes model priors. Detection typically requires influence-function analysis, outlier scoring in feature embeddings, or spectral anomaly detection on gradient statistics.

Adaptive test-time evasion (query-driven): Whereas single-shot perturbations assume gradient access or surrogate alignment, adaptive evasion exploits repeated interaction with a deployed model. The adversary submits queries, observes class probabilities or confidence scores, and uses zeroth-order or bandit optimisation to estimate a descent direction in input space. Boundary Attack and NES exemplify this class. Query-efficiency constraints dominate practicality: rate limiting, output rounding, and score obfuscation substantially reduce attack convergence, but too aggressive a response impairs legitimate API users. Empirical work shows that even coarse confidence leakage (top-k scores) accelerates black-box evasion relative to decision-only interfaces.

Trojan/back-door model compromise: In a Trojaned model, the attacker implants a trigger-conditioned mapping during training: the model behaves normally on clean data but reliably emits an attacker-chosen label when a specific pattern (pixel patch, watermark, audio tone) is present. Triggers may be spatially local (corner patch), distributed (colour histogram shift), or semantic (accessory type in face images). Clean-label back-doors, where trigger-bearing samples retain the correct label in the training corpus, are especially insidious because they evade standard quality checks. Once deployed, a physical sticker on a road sign or patterned spectacles in a biometric gate can activate the hidden mapping. Detection approaches include activation clustering, trigger inversion, neuron pruning with fine-tuning, and statistical testing for unusually low-rank feature correlations tied to rare visual motifs.

Supply-chain logic bombs and model re-use risk: Increasingly, downstream systems import third-party pre-trained weights, adapters, or foundation models. This introduces a supply-chain channel for adversarial payloads that need not reside in the training data at all. A malicious contributor can ship a model that passes standard validation yet contains a latent decision shortcut, activated only under a compound condition (e.g., specific Unicode tokens + input length range). Such logic bombs propagate across transfer-learning workflows: fine-tuning on a new dataset may leave the hidden pathway intact if the relevant neurons are not significantly updated. Verifiable model signing, weight-difference auditing, and targeted re-training with trigger search are emerging mitigations.

Across these categories, the unifying property is goal-directed manipulation under resource constraints: the adversary trades perturbation budget, query cost, and detectability to induce controlled model failure. Effective defence therefore requires layered controls aligned to lifecycle stage, dataset provenance and sanitisation to counter poisoning, robust and certified training to enlarge safe margins, interface hardening to reduce query-driven leakage, and supply-chain assurance to prevent Trojan insertion. Without such integration, even high-accuracy models remain operationally fragile in adversarial environments.

Backdoor attacks involve manipulating the AI model during training by inserting a specific backdoor pattern. This pattern will cause the model to generate incorrect outputs when it encounters the input data pattern, allowing attackers to manipulate its behaviour. These attacks require advanced defence mechanisms and security protocols to ensure the dependability and robustness of AI applications in various domains (Ozdag, 2018; Macas et al., 2024).

To ensure reproducibility and transparency, this section outlines the experimental setup used for implementing the FGSM and C&W adversarial attacks.

Recent benchmark datasets and competitive evaluations have played a crucial role in advancing the state-of-the-art in adversarial robustness by providing standardised tasks, reproducibility protocols, and comparative baselines for attack and defence techniques. Two prominent initiatives, RobustML and the Adversarial Vision Challenge (AVC), offer structured environments to evaluate model performance under adversarial conditions.

Attackers White-box threat models grant the adversary complete visibility into a target neural network’s architecture, parameter values, and, occasionally, its training data. This transparency enables highly-tailored attacks that exploit exact gradient information, internal activation statistics, and structural shortcuts that remain opaque in black-box settings.

Gradient-driven perturbations. Armed with the full loss landscape, an attacker can compute precise input gradients and craft imperceptible perturbations that maximise the model’s prediction error. Canonical examples include FGSM, projected-gradient descent (PGD), and the Carlini-&-Wagner optimiser, all of which routinely achieve near-100% misclassification rates on ImageNet-scale models once the perturbation budget is aligned with human-perception thresholds (e.g., ε = 8/255 in the L∞ norm).

Exact model cloning. Given access to weights and training data, the adversary can duplicate the model verbatim, run offline sensitivity analyses, and search for corner-case failures without rate limits or audit trails. The duplicate can also serve as a surrogate generator of transferable adversarial examples that will fool the original with probability approaching one.

Privacy-oriented exploits. Full data access trivialises membership inference and model-inversion attacks: confidence skew and gradient back-propagation directly reveal whether, and in what form, a record contributed to training. For clinical or financial datasets, this constitutes a direct breach of regulatory constraints such as HIPAA or GDPR.

Architectural vulnerability mining. White-box inspection exposes brittle components, e.g., unregularised batch-norm layers, low-rank bottlenecks, or unsafe activation ranges, that can be perturbed to induce exploding activations, vanishing feature maps, or numerical overflow, thereby triggering denial-of-service or silent logic corruption.

Given the exceptional leverage afforded by white-box knowledge, defensive counter-measures must shift from obscurity to formal robustness. Recommended practices include (i) certified or provable-robust training against gradient-based perturbations, (ii) differential-privacy mechanisms to bound information leakage, (iii) architectural hardening via gradient-mask–free regularisers, and (iv) run-time anomaly detectors that flag activation patterns outside the training manifold. Only a multi-layered approach can meaningfully degrade the success probability of the sophisticated attack vectors enabled by full-model disclosure.

The Jacobian-based Saliency Map Attack (JSMA) is a sparse, targeted, white-box technique that perturbs only the most influential input features to force a classifier into a chosen label. Unlike norm-bounded attacks that diffuse small noise across all pixels, JSMA computes an explicit saliency map from the input-gradient (Jacobian) of the network: each pixel receives a score reflecting how strongly increasing its value raises the target-class logit while suppressing competing classes. The attacker greedily modifies the highest-saliency pixels, often fewer than 5% of the image, until misclassification occurs or a pre-set L₀ budget is reached (see Figure 13). This sparsity yields adversarial examples that remain visually plausible yet evade many magnitude-based defences.

JSMA belongs to a broader family of feature-targeted attacks illustrated in Figure 14. DeepFool linearises the local decision boundary and iteratively moves the input along the shortest L₂ path to cross that boundary, achieving minimal global distortion but without pixel-level sparsity. Iterative Gradient Sign Method (I-FGSM) extends FGSM by taking multiple small steps in the gradient-sign direction, trading speed for higher success rates under the same L∞ budget. Carlini-&-Wagner (C&W) refines the optimisation further, searching for perturbations that minimise both distortion and a confidence-weighted misclassification term, producing near-imperceptible attacks even against defence-aware models. Boundary Attack assumes no gradient access at all; it starts from a random point in the target class and performs a random-walk projection toward the original sample, converging on an adversarial example under decision-only feedback.

Collectively, these methods expose complementary weak spots in deep networks—sparsity (JSMA), minimal-norm (DeepFool), iterative L∞ (I-FGSM), optimisation-tight (C&W), and decision-based (Boundary). A comprehensive robustness evaluation therefore requires testing against the full spectrum rather than relying on any single attack type. The diversity shown in Figure 14 underscores why modern defence pipelines combine adversarial training, gradient-regularising architectures, and runtime anomaly detection to achieve credible resilience in safety-critical deployments.

In Figure 14, we can see each attack’s main characteristics and steps.

Each of these attacks in Figure 14 highlights different aspects of adversarial methodologies, demonstrating the diversity and evolving complexity in crafting adversarial inputs and emphasising the critical need for developing robust and versatile defensive mechanisms.

Deep neural networks (DNNs), while highly performant across a wide range of tasks, are susceptible to adversarial inputs, carefully crafted perturbations that can cause targeted misclassification. The C&W attack remains one of the most effective and precise techniques for generating such examples. To illustrate its implementation, we applied the L₂-norm variant of the C&W attack on a convolutional neural network trained on the MNIST dataset of handwritten digits. The model used in our experiment consisted of two convolutional layers with ReLU activations, max-pooling operations, followed by two fully connected layers with dropout regularisation. It achieved 99.2% accuracy on clean test images.

Before launching the attack, the input data (28 × 28 pixel greyscale images) was normalised to the [0,1] range and one-hot encoded for labels. The neural network was either trained from scratch or loaded from a pre-trained checkpoint. Once verified on clean data, we proceeded to craft adversarial examples designed to cause the model to misclassify each image as a specific target class.

The C&W attack constructs adversarial samples by solving an optimisation problem. The aim is to find the smallest possible modification to an input image that causes it to be misclassified as a chosen target class, while keeping the change imperceptible. To maintain valid pixel values during optimisation, the input image is not modified directly. Instead, a latent variable w is introduced in an unconstrained space, and the final adversarial image x_adv is derived through a bounded transformation. The transformation used is based on the hyperbolic tangent function, ensuring that the pixel values in x_adv remain within valid image bounds:

Mathematical equation displaying "x_adv equals zero point five times the quantity of hyperbolic tangent of w plus one."

This transformed image is then fed into the model to evaluate how confidently it is classified into the target class. The attack objective is to increase this confidence while minimising the visible difference from the original image. The trade-off between confidence and distortion is controlled by a scalar constant c, which is determined through binary search for optimal balance.

The overall workflow of the attack can be summarised procedurally as follows:

Python code for optimizing a model with adversarial examples. It iterates over images and target classes to adjust variables, calculate losses, and update weights using gradients and the Adam optimizer. Key steps include initializing variables, computing adversarial examples, calculating confidence and distortion losses, and updating with a total loss function.

In our experiment, we used 1,000 randomly selected images from the MNIST test set. For each image, a target class was chosen that differed from the true label. The attack was run for up to 1,000 optimisation steps per image, using the Adam optimiser with a learning rate of 0.01. The confidence parameter, denoted κ in literature, was tested with values ranging from 0 (baseline) to 20. The balancing constant c was determined through 9 rounds of binary search starting from an initial value of 1e-3. This ensured that adversarial samples were successful while introducing minimal perturbation.

The results confirmed the effectiveness of the C&W attack. A 100% success rate was achieved across the sample set: every adversarial example caused the model to misclassify the input into the intended target class. At the baseline confidence (κ = 0), the average perturbation magnitude (measured as L₂ distance) was approximately 1.73, visually imperceptible to the human eye. As the confidence parameter increased, the attack became more forceful but also required larger perturbations: with κ = 10, the average distortion rose to 3.21, which remained subtle but became slightly perceptible in some instances. Qualitative inspection of the adversarial examples showed that the changes were localised and did not significantly alter the semantic appearance of the digit.

These findings reinforce the C&W attack’s position as one of the most refined adversarial techniques. Its success lies in its ability to tightly control the trade-off between stealth and misclassification certainty. The tanh-space reparameterisation ensures that all adversarial samples remain valid images, while the binary search over c ensures adaptive adjustment based on target class and model behaviour. This method is particularly useful in security-sensitive applications—such as facial recognition, signature verification, and document forgery, where subtlety of attack is paramount.

Black-box threat models assume the adversary can supply inputs to a deployed service and observe only the returned labels or confidence scores; the model’s architecture, weights, and training data remain hidden. Despite this limited view, several families of attacks can still achieve high misclassification rates:

Figure 15 summarises these attack families along two axes: information available (label vs. score) and query budget. Transfer attacks succeed offline with zero queries but rely on surrogate alignment; ZOO and bandit methods trade elevated query cost for tighter perturbation budgets; decision-only attacks operate under the strictest feedback constraints at the expense of more iterations.

In the examples used in Figure 15, the transfer attacks represent adversarial examples generated for one model and are used to attack another model. The zeroth order optimisation directly estimates the gradient of the targeted model by querying it. This type of attack can be separated into query-based attacks, which are conducted by repeatedly querying the model, where attackers estimate its gradient to craft adversarial examples, and HopSkipJumpAttack, where a decision-based attack where the attacker has no information about the model’s gradients, only its outputs.

Adversarial perturbations fall into two intent classes. Targeted attacks are goal-directed: the adversary crafts a perturbation that forces the model to emit one specific wrong label, e.g., a “stop” sign misread as “speed-limit.” Success is measured by whether the output matches this pre-selected class, so the optimisation explicitly maximises the target class logit while suppressing all others. Such precision is indispensable for fine-grained fraud (redirecting facial recognition to a chosen identity) or for bypassing class-specific access controls.

By contrast, non-targeted attacks seek any incorrect label. The adversary’s objective is simply to shove the input out of its correct decision region, thereby degrading model accuracy or eroding confidence in automated decisions. The Basic Iterative Method (BIM) exemplifies this category: starting from a Fast-Gradient-Sign seed, BIM applies many small, bounded steps in the gradient-sign direction, gradually increasing loss until the classifier flips to some alternative class. Because the optimisation landscape is less constrained, non-targeted attacks usually require smaller perturbations or fewer queries than their targeted counterparts.

Distinguishing the two threat models is critical for defence design. Robustness certificates for targeted attacks must cover worst-case perturbations that hit a designated class, while defences against non-targeted attacks focus on enlarging the overall decision margin. Comprehensive evaluation therefore reports both targeted and non-targeted success rates to capture the full adversarial risk surface.

Advanced adversarial attacks extend beyond traditional perturbation-based methods by targeting different phases of the machine learning pipeline and exploiting broader threat surfaces, including training data leakage, physical-world vulnerabilities, and model access modalities. Table 6 presents a formal taxonomy of these attack types, classifying them by attack surface, knowledge required, primary objective, attack modality, and notable challenges or constraints.

This taxonomy reveals several important distinctions:

Understanding these structural characteristics enables more targeted defence strategies and prioritisation of threat mitigation based on deployment context and attacker capabilities.

During training, adversarial examples are introduced to enhance the model’s robustness to potential threats. This is achieved through ensembling multiple models, which enables the averaging of their respective predictions. As an additional measure, pre-processing techniques such as JPEG compression and image smoothing are utilised to remove adversarial noise from the data. These strategies create a more reliable and accurate model, improving the system’s effectiveness.

Defensive distillation is a technique in machine learning that involves training a model to replicate the behaviour of another model. The approach is based on using less extreme output probabilities, which helps to increase the model’s robustness and resistance to adversarial attacks. By imitating the behaviour of a more complex model, the distilled model can perform better in real-world scenarios, where it may encounter unexpected inputs or other sources of uncertainty. Defensive distillation is a powerful tool for improving the reliability and safety of AI systems, especially in high-stakes applications such as autonomous driving, medical diagnosis, and financial forecasting.

Several methods can be applied against attacks. One such method is Feature Squeezing, which removes extraneous features from input data, thereby restricting the search space for potential attackers. Another technique is Randomised Input Transformations, which confuses adversaries through the random transformation of inputs during inference. A third approach is Gradient Masking, which renders gradients uninformative to prevent attackers from using them to create adversarial examples. The Detection method involves training auxiliary models to recognise adversarial perturbations instead of trying to achieve complete robustness against them. This requires evaluation of potential adversarial attacks.

Empirical evidence accumulated over the past decade shows that comprehensive knowledge of a model’s internals is no longer a prerequisite for producing high-confidence adversarial failures. Gradient-free bandit optimisers, finite-difference estimators, and transfer-based strategies now attain misclassification rates on ImageNet that approach those of canonical white-box attacks, demonstrating that decision boundaries learned by modern architectures remain highly correlated even when the parameters are hidden. This observation calls into question evaluation protocols that rely exclusively on fast, single-step perturbations such as FGSM or on a limited set of norm-bounded iterative attacks.

Robustness is also demonstrably task-dependent. Adversarial training with projected-gradient descent improves ε = 8/255 accuracy on CIFAR-10 to roughly 47%, yet under identical perturbation budgets the same procedure affords fewer than 20% robust accuracy on ImageNet. When the perturbations are extended from pixel noise to geometric distortions, two-degree rotations or sub-pixel translations in autonomous-driving imagery, the effective accuracy drop is larger still, pointing to a gap between standard benchmark metrics and the failure modes that dominate physical deployments.

Privacy leakage scales with both model capacity and the granularity of the outputs released to the user. Membership-inference and model-inversion attacks achieve 60–70% accuracy on unprotected CIFAR-10 classifiers but fall to near-random levels once training is performed with differential-privacy stochastic gradient descent at ε ≈ 1. The modest two-to-four-percentage-point reduction in clean accuracy observed in these experiments suggests that rigorous privacy guarantees and practical utility are not mutually exclusive.

No single defensive mechanism exhibits universal efficacy. Defensive distillation, feature squeezing, and gradient masking all deteriorate under adaptive evaluation; only multi-layered strategies that combine adversarial training, randomised smoothing, differential privacy, and interface hardening provide measurable resilience, and even these solutions leave a gap of more than 50 percentage points between clean and robust accuracy on large-scale image classification. Large-language models present an additional alignment problem: automated red-teaming frameworks such as MAD-MAX attain jailbreak success rates exceeding 95 % on GPT-4-class systems with an average of 11 queries, revealing that reinforcement-learning-based alignment alone is insufficient to prevent malicious prompt injection.

Finally, supply-chain attacks on pre-trained weights, data-set poisoning in self-supervised learning, and logic bombs embedded in parameter-efficient adapters highlight that adversarial robustness must be considered across the entire model lifecycle, data acquisition, training, distribution, and inference monitoring, rather than at a single frozen checkpoint.

This work advances the study of adversarial AI security by reframing robustness as a system-level property of agentic artificial intelligence rather than a narrowly defined characteristic of individual models or inputs. By formalising adversarial risk across perceptual, cognitive, and executive layers, the proposed framework extends classical adversarial machine-learning theory to account for autonomy, self-governance, and closed-loop decision-making. This shift is essential for analysing modern AI systems whose behaviour emerges over time through feedback, planning, and interaction with dynamic environments.

A central insight of this study is that adversarial vulnerabilities cannot be fully understood, or mitigated, through input-level defences alone. While perceptual attacks remain the most empirically validated, higher-order failures arise when erroneous perceptions propagate through internal reasoning, policy formation, and actuation. The analysis demonstrates that architectural properties such as feedback dynamics, memory, and goal specification fundamentally shape adversarial behaviour, transferability, and impact. Consequently, robustness must be treated as an emergent property of the entire agent–environment system rather than as a static performance metric.

Importantly, this paper does not position higher-order agentic adversarial attacks as empirically settled phenomena. Instead, they are framed as hypothesis-driven, architecturally motivated risks that demand systematic investigation. The absence of standardised benchmarks for autonomy, long-horizon decision-making, and behavioural integrity represents a critical gap in current research. Addressing this gap requires new evaluation environments, metrics that capture cumulative behavioural deviation and policy drift, and defence mechanisms that integrate perceptual robustness with governance, verification, and runtime oversight.

Beyond consolidating existing adversarial knowledge, the primary contribution of this work lies in defining a coherent research agenda for agentic AI security. By unifying adversarial machine learning, systems safety, and control-theoretic perspectives, the framework provides a foundation for studying resilience, stability, and alignment in autonomous AI systems. As AI agents are increasingly deployed in safety-critical and socially consequential contexts, such system-level approaches will be indispensable for ensuring not only accuracy, but trustworthy and accountable behaviour over time.