科孚（CRIF）：2025生成式AI应用的核心防护与安全挑战研究报告（英文版）（20页）.pdf

1、The key security and safety challenges in adopting GenAIn 03I Insidenside G GenenAIAI21 Introduction32 The three axes:security,safety and accuracy 52.1 Security 62.2 Safety 92.3 Accuracy 133 The CRIF way 144 Conclusions 20I Insidenside G GenenAIAITable of Contents31 IntroductionAnalyzingthereliabili

2、tyofgenerativeartificialintelligence(GenAI)systems is crucial in todays technologicallandscape.TherapidproliferationofGenAI,known for its extraordinary ability to generate complexandrealisticcontent,combinedwithits“blackbox”nature,presents unprecedented challenges.InsideInside G GenenAIAI4The number

3、 of providers is increasing,while the gap between open source and proprietary models is decreasing,as are the inference costs.In the early stages of GenAIs evolution,reasoning capability was the most important criterion for selecting one model over another;now,additional factors such as price,speed,

4、and security play a crucial role.In this context and considering that it is expected that the number of GenAI-based applications in production systems will increase in 20251,the creation of a robustevaluationframework is fundamental to correctly guide architectural choices.Addressing these challenge

5、s requires a concerted effort across research,regulation,and technological innovation to ensure that the benefits of GenAI can be fully realized without compromising the security and integrity of systems.Unfortunately,for the AI architectures that power most of the GenAI-based applications,it is imp

6、ossible to prevent all attacks.For example,crafted inputs can manipulate the model into producing undesired or harmful outputs,such as unsafe content,while mitigation strategies such as input sanitization,adversarial tuning,and moderation models can strongly reduce these risks2,but do not eliminate