Washington DC [US], September 19 (ANI): A false intelligence report generated with the help of artificial intelligence (AI) reportedly came close to triggering a US military operation against a Chinese vessel in West Asia during the war with Iran, according to a CNN report citing multiple sources familiar with the episode.
According to CNN, the intelligence report, circulated across the US military this spring, claimed that a Chinese ship in West Asia was transporting components linked to a nuclear weapons programme.
The report immediately raised alarm within the US military and prompted preparations to intercept the vessel, as reported by CNN.
Citing four sources familiar with the incident, CNN reported that the US military moved towards an operation against the ship. Two of the sources said armed US military personnel were preparing to board the vessel, while military aircraft were already in the air, according to two other sources.
However, shortly before the planned operation, officials examined the intelligence more closely and discovered that the report had been prepared with the assistance of an AI system. The chatbot used by a US Special Operations Command analyst had inaccurately identified the material being transported by the ship, the sources told CNN.
CNN reported that it was unable to determine what the cargo had actually been. One source described the intelligence report as “entirely false".
The same source told CNN that the incident “almost started a war", warning that any US military operation against a Chinese vessel could have risked escalating into an armed conflict between the two countries.
According to CNN, the analyst had queried a chatbot about intelligence reporting concerning the ship's manifest. The original reporting had originated with US Special Operations Command Pacific, based in Hawaii.
It remained unclear whether the chatbot involved was a commercially available system or a US government-developed product.
The chatbot reportedly combined open-source intelligence with secret signals intelligence contained in government holdings before reaching the erroneous conclusion about the cargo.
The analyst subsequently used AI a second time to package the findings into a standard intelligence report. The report was then disseminated through military channels, according to CNN, citing sources.
The incident comes as the US military and intelligence community accelerate the adoption of AI for a wide range of functions, including analysing large volumes of intelligence, selecting targets for strikes, moving military assets and handling budgeting, logistics and supply chains.
The episode highlighted concerns over the use of AI-generated information in military targeting, particularly when decisions are being made during an ongoing conflict in West Asia.
Officials have argued that rapidly integrating AI is necessary to help the US military make battlefield decisions faster and to avoid falling behind adversaries that could potentially use similar technologies.
In January, US Secretary of War Pete Hegseth released the Pentagon's “Artificial Intelligence Acceleration Strategy”, which seeks to speed up the military's adoption of AI.
“We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI,” Hegseth said in a speech announcing the strategy.
The strategy also calls for expanding access to AI across the US Department of War. A memo announcing the initiative said its objective was to “democratise AI experimentation and transformation across the Department by putting America’s world-leading AI models directly in the hands of our three million civilian and military personnel, at all classification levels.”
However, multiple US officials familiar with the situation told CNN that the adoption of AI remains decentralised, with different parts of the government using different systems, orders and safety standards.
According to the report, there is no single set of standards governing how information generated by these systems is verified.
The range of AI systems being deployed across different parts of the military and intelligence community also means their reliability and functionality can vary.
The incident has raised concerns among intelligence officials about the possibility of AI-generated errors being incorporated into formal intelligence products and subsequently relied upon by military decision-makers.
One source familiar with current military policies told CNN, “AI in targeting is definitely something that is ramping up, and there is no real guidance for how having a human in the loop will prevent civilian casualties or fratricide.”
CNN reported that the hallucination involved in the Chinese ship incident was not necessarily an isolated occurrence within the intelligence community since AI tools began becoming more widely used across the government, according to CNN, citing one of its sources.
Several sources also said some older intelligence officials who otherwise support the use of AI are concerned that the technology is creating pressure on analysts to produce and distribute intelligence more quickly, potentially increasing the likelihood of mistakes.
Young analysts, according to the sources, are more accustomed to using AI tools and may be more likely to trust their output without sufficient scrutiny.
“AI allows you to get to a bad idea faster,” one source told CNN.
The Chinese ship episode, as described by CNN's sources, illustrates the potential consequences of inaccurate AI-generated intelligence being incorporated into established military reporting systems, particularly when such information is used during an active conflict and could influence operational decisions. (ANI)
AI-generated false intelligence nearly triggered US-China military confrontation: Reports
