Photo: Courtesy of Fangcun AI

As artificial intelligence (AI) systems become increasingly capable of making decisions and carrying out tasks with limited human intervention, ensuring their safe and reliable operation is emerging as a growing challenge for the global AI industry.

Against this backdrop, Chinese AI safety company Fangcun AI recently published an independent assessment of DeepSeek's latest 4.1 version. The results showed substantial safety improvements over earlier DeepSeek models, with performance approaching or matching leading closed-source models in several areas. 

However, the assessment also identified behaviors warranting further attention, including instances in which the model prioritized task completion over operational restrictions.

For Fangcun CEO Li Xiaojian, the purpose of the assessment was not to single out one model, but to draw attention to challenges facing the industry.

"Conducting this kind of evaluation does not mean that we have identified safety problems specific to DeepSeek," Li told the Global Times in a recent interview. "Rather, we hope to highlight some of the common problems and safety challenges facing large AI models today."

One of these broader challenges is how to ensure that increasingly capable AI systems remain within boundaries set by human operators while carrying out tasks.

The growing capabilities of AI agents are changing how researchers approach AI safety, said Fangcun's chief scientist Dong Yinpeng.

"The harm caused by AI is no longer reflected solely in its final output, but can also emerge throughout the entire execution process," Dong said.

Through reinforcement learning, AI models can discover strategies that developers may not have anticipated. While such strategies can represent improvements in capabilities under controlled conditions, they may create risks in real-world applications if models prioritize task completion over rules established by human operators.

"In an environment with clearly defined rules, this kind of innovation is generally regarded as an improvement in capabilities. But in real-world application scenarios, certain strategies adopted by a model to complete a task may cross the boundaries set by humans," Dong said.

This also means that ensuring AI safety cannot rely solely on alignment during model training, he argued. External monitoring and protective mechanisms are also needed while models are operating.

As AI systems become more autonomous, independent assessment is becoming an important part of the efforts made to identify and address potential risks before deployment.

Li noted that major international AI companies, including OpenAI and Anthropic, work with external safety organizations to evaluate their models.

However, independent researchers' ability to examine closed-source systems can be constrained by limited access to their internal mechanisms, whereas open-weight models provide greater opportunities for external scrutiny.

Beyond model developers, independent assessments can also help companies and organizations that use or deploy AI systems better understand potential risks before putting the technology into practice, Li said.

Looking further ahead, Dong explained that one of the major challenges for AI safety research is shifting from responding to problems after they emerge toward anticipating risks before they cause harm.

He also pointed to the possibility of AI participating in the development of subsequent generations of AI, a process often described as recursive self-improvement.

If this becomes more prevalent, safety supervision may need to extend beyond AI's behaviors in practical applications to the development process itself.

For Fangcun, these challenges represent important directions for future research and product development.

Li believes that independent evaluation will become increasingly important as more companies enter the AI market and models evolve at a faster pace.

"The faster the industry develops and the more model developers enter the field, the greater the need for third-party organizations to act as intermediaries," Li said.