Introduction

Artificial Intelligence (AI) technologies are deeply integrated into critical infrastructures, and their resilience is crucial for ensuring the safe and stable operation of systems. High-resilience AI systems can maintain robust operation in extreme environments, quickly recover from attacks or failures, and adaptively evolve to meet changing demands, thereby avoiding system paralysis or erroneous decisions due to vulnerabilities.

The article “Current Status and Prospects of AI Resilience Research” published in the 2026 second issue of China Engineering Science defines AI resilience in four core dimensions: robustness, defense capability, recovery capability, and evolutionary capability. It systematically reviews the current state of AI resilience research, focusing on key technological advancements domestically and internationally, particularly the new challenges and solutions brought by technologies such as large language models. The research highlights prominent issues faced in the development of AI resilience, including the lack of top-level planning for capability building, the absence of testing systems in real scenarios, and insufficient emphasis on the resilience of large models. The study suggests strengthening strategic guidance, constructing a systematic resilience framework, building high-fidelity, multi-dimensional, reproducible resilience testing systems, and focusing on exploring the potential of large models to enhance their multi-level resilience capabilities throughout the entire lifecycle of training, deployment, operation, and updating, thus creating more reliable, trustworthy, and sustainable intelligent systems.

AI Resilience Dimensions

AI resilience can be detailed into four core dimensions: robustness, defense capability, recovery capability, and evolutionary capability. These dimensions collectively form the foundational capabilities for AI systems to achieve long-term reliable operation and sustainable development.

Robustness

Robustness refers to the ability of AI systems to maintain stable and accurate outputs in the face of environmental disturbances, input noise, or unstable operating conditions. This dimension emphasizes the system’s tolerance to uncertainties in input data, such as changes in lighting, sensor errors, and distribution shifts, while maintaining performance consistency across various application scenarios and task configurations. Achieving robustness relies on the generalization ability of algorithm models, logical reasoning capabilities, and robustness optimization during the training process, including methods such as data augmentation, regularization, and interference resistance mechanisms.

Defense Capability

Defense capability focuses on the ability of AI systems to respond to security threats, particularly in effectively identifying and defending against external malicious attacks (such as adversarial samples, data poisoning, and backdoor insertion) or internal abnormal behaviors (such as model tampering and privilege abuse). This capability not only relates to the security and credibility of AI systems but also directly affects their feasibility in open environments. High defense capability in AI systems typically requires information access control, anomaly detection, attack identification, and response capabilities, combined with model security design and continuous monitoring mechanisms to enhance the overall security of the system.

Recovery Capability

Recovery capability reflects the ability of AI systems to recover quickly to a stable operating state after experiencing functional degradation or local failures. AI systems with recovery capability must be able to quickly diagnose problems, locate damage, and restore core functions to prevent local issues from evolving into systemic risks. Achieving recovery capability relies on state perception, anomaly recovery, and self-healing mechanisms, including system reconstruction, model rollback, and data redundancy methods to enhance the system’s self-repair ability and fault tolerance.

Evolutionary Capability

Evolutionary capability refers to the ability of AI systems to proactively adapt and continuously optimize themselves in the face of environmental changes, task transitions, or new threats. This capability is applicable to AI applications in dynamic environments, such as cybersecurity, autonomous driving, or financial decision-making. Evolutionary capability emphasizes the system’s ability to perceive the environment, transfer knowledge, and engage in continuous learning, allowing it to adjust its strategies, optimize model structures, and expand its knowledge base.

Current Status of AI Robustness Technology Development

AI robustness reflects the system’s ability to maintain stable outputs in unstable environments. This characteristic requires models to sustain performance under uncertainties such as noise interference, distribution shifts, and hardware failures while consistently ensuring reliability and consistency in complex and variable environmental conditions. The components of AI robustness include the generalization ability of algorithms, reasoning logic, and robust training.

Generalization Ability

Generalization ability is the core of AI robustness, aiming to ensure that models maintain stable predictive performance in unknown data distributions or cross-domain scenarios. Enhancing generalization ability typically requires expanding the capability space of AI systems, indirectly enlarging the robustness domain to improve system robustness. Approaches to achieve generalization ability mainly include data augmentation and transfer learning.

Reasoning Logic

Accurate reasoning logic is fundamental to the robustness of AI systems, ensuring the correctness of model decisions and the transparency of the decision-making process. From a capability composition perspective, reasoning logic enhances the connectivity among robustness domains within the AI system’s capability space to prevent spontaneous risks in model reasoning and decision-making processes.

Robust Training

Robust training enhances AI models’ resistance to noise, data bias, and adversarial disturbances through techniques such as adversarial training, regularization methods, and data augmentation, allowing them to make stable and reliable decisions in complex and variable environments.

Current Status of AI Defense Capability Technology Development

Defense capability focuses on the ability of AI systems to resist internal and external attacks, including adversarial attacks, data poisoning, and backdoor attacks. Enhancing defense capability not only protects models from malicious tampering but also strengthens the overall security of the system, ensuring the credibility of AI in open environments.

Information Limitation

The goal of information limitation is to minimize the permissions and information accessible to various participants in the AI system while ensuring functionality. This capability spans all stages of the AI system’s lifecycle.

Attack Identification

The ability to identify attacks is a prerequisite for ensuring system security, enabling timely detection and blocking of attacks, thus buying time for subsequent responses.

Attack Defense

Attack defense capability complements the identification capability, jointly preventing attackers from achieving their intended goals. Traditional passive defenses primarily enhance model robustness, such as improving resistance to adversarial samples through adversarial training.

Current Status of AI Recovery Capability Technology Development

Recovery capability reflects the ability of AI systems to quickly restore to normal states after suffering damage or functional failures. A well-recovered AI system can rapidly adjust and restore its original functions, avoiding systemic collapse.

State Monitoring

State monitoring capability requires AI systems to recognize abnormal states. This capability demands higher real-time performance, typically focusing on application deployment and online phases.

Impact Elimination

Based on the identification of abnormal states, impact elimination capability reflects the model’s ability to remove risks and restore normalcy. Current research has proposed various solutions to remove backdoors related to neural networks.

Current Status of AI Evolutionary Capability Technology Development

Evolutionary capability reflects the ability of AI systems to autonomously adapt and continuously optimize themselves in response to environmental changes, task upgrades, or new threats. This self-evolution capability allows AI to maintain competitiveness over the long term, especially in dynamic adversarial environments.

Environmental Perception

Environmental perception capability is the foundation of AI systems’ evolutionary capability, aiming to extract information beneficial for model evolution from changing environments.

Continuous Learning

Building on environmental perception, continuous learning capability enhances intelligent models’ adaptability and usability in changing environments by utilizing available information.

Challenges in Enhancing AI Resilience

Lack of Top-Level Planning for AI Resilience Construction

Currently, AI resilience construction lacks unified planning, with imbalanced investment in technological research and development across dimensions. The focus is primarily on enhancing robustness during training and defense capabilities during deployment, while research on recovery and evolutionary capabilities after damage is insufficient.

Insufficient Experimental Scenarios for Resilience Evaluation

The AI resilience evaluation system remains incomplete, particularly in the design and construction of experimental scenarios. Existing evaluation methods primarily focus on algorithm-level metrics, lacking standardized evaluation frameworks that comprehensively reflect AI systems’ resilience capabilities in complex environments.

Need for Greater Emphasis on Large Model Resilience Construction

As large models become critical components of new information infrastructures, their resilience capabilities are increasingly vital. Systematically exploring and strengthening the resilience potential of large models is key to enhancing their applicability in complex environments.

Recommendations for Enhancing AI Resilience

Strengthen Strategic Guidance and Build a Systematic Resilience Framework

It is recommended to promote top-level design for AI resilience from national strategies and industry standards, clarifying its positioning and goals in national security, industrial development, and digital infrastructure construction.

Construct a High-Fidelity, Multi-Dimensional, Reproducible Resilience Evaluation System

Accelerating the establishment of standardized evaluation platforms and scenario libraries for AI resilience is suggested. This should include diverse, dynamic, and comprehensive simulation environments covering various challenges such as adversarial attacks, data anomalies, system failures, and environmental changes.

Explore the Potential of Large Models and Promote Multi-Level Resilience Enhancement

It is recommended to prioritize large models as key targets in the AI resilience system, systematically exploring their inherent potential in robustness, defense capability, recovery capability, and evolutionary capability, thereby establishing a resilience assurance mechanism throughout the entire lifecycle of training, deployment, operation, and updating.

Conclusion

As AI technology increasingly becomes a core support of national critical infrastructures, its resilience is no longer an ancillary attribute but a core element ensuring the safe, stable, and sustainable operation of systems. This article constructs a capability framework for AI resilience around the four dimensions of robustness, defense capability, recovery capability, and evolutionary capability, systematically reviewing relevant research progress and identifying key shortcomings in current development, especially the lag in resilience construction and evaluation mechanisms for large models. The study indicates that building high-resilience AI systems is not only a fundamental requirement for achieving AI safety and reliability but also a prerequisite for long-term self-evolution and service assurance in complex real-world environments.