To illustrate, in Course TST 2040 at the Warfighting Acquisition University, students are tasked with collecting data on the performance of a paper helicopter delivery system, referred to as the Weapon Aerial Delivery Sys - tem, and analyzing whether the sys - tem meets specifications for accu - rately delivering “munitions” (coins). The target and hit points were marked on the floor using 1/2-inch squares cut from colored Post-it® notes. An image was captured with a smartphone, with care taken to min - imize distortion.Copilot AI then was tasked with analyzing the results. The following script was provided: Assume the squares are 1/2 inch by 1/2 inch. The yellow square is the target, and the pink squares are hit points. Make a scatter plot. Determine the up- down and left-right miss distances, as well as the RMD of each hit point. Cal- culate the CEP. Copilot undertood RMD as Radial Miss Distance—the straight-line dis - tance from each hit point to the tar - get and CEP as Circular Error Prob - able—the radius within which 50 percent of the hit points fall relative to the target. The results were generated rapidly and presented in a polished manner. Initially, this appeared to greatly sim - plify the often-tedious process of cat -
aloging and analyzing data. However, upon closer examination, discrepan - cies became apparent. A side-by-side comparison of the original input im - age and the AI-generated scatter plot provides insight into these issues and supports the broader argument (Fig - ures 1 and 2). AI failed to detect the yellow target square and assigned it an arbitrary lo - cation, while also omitting one of the hit points. Consequently, the analy - sis was rendered invalid despite its clear and authoritative presentation (Figure 3). The primary lesson is that AI can generate rapid and polished out - puts; but these must be rigorously validated because hallucinations can compromise their reliability. To ensure the dependability of AI-generated results, specific valida - tion strategies should be employed. These include manually spot-check - ing a subset of data points against the original image or dataset, cross- verifying outputs with traditional analysis methods such as manual measurements, or utilizing a second independent tool to replicate the analysis. Discussions with individuals possessing greater expertise in AI led to several suggestions for improving the training of large language models.
This experience underscores that, while AI can significantly reduce ad - ministrative workload, it also intro - duces risks of data contamination and invalidation due to hallucina - tions. Notably, such hallucinations are an inherent characteristic of large language models rather than an anomaly. Therefore, meticulous training and validation of AI tools are essential. There is little question that using AI tools for myriad applications will be critical in actualizing the ambi - tious goals of the ATS. It therefore falls heavily on the user to choose wisely and employ well. For detail on testing of AI-enabled systems that sheds light on some of the problems encountered by this example and how to avoid those pitfalls please reference the two re - cently published guidebooks: DT&E of AI Enabled Systems Guidebook and DT&E of Autonomous Systems Guidebook . SHEPHERD is a professor of Test and Evalu- ation at the Warfighting Acquisition Univer- sity’s South Region at Redstone Arsenal, Ala. The author can be contacted at Seth.Shepherd@waru.edu . The views expressed in this article are those of the author and not the Department of War. Repro- duction or reposting of articles from Defense Acqui- sition magazine should credit the authors and the magazine.
SEPTEMBER –OCTOBER 2026 | DEFENSE ACQUISITION MAGAZINE 35
Made with FlippingBook - Online Brochure Maker