mental and physical effort required to satisfy the perceived demands of a specified flight task.” Also, like the HQRS, the decision tree starts in the lower left corner and points to a series of dichotomous questions requiring a yes or no response as shown in Figure 2. The responses then lead to descrip- tions of different levels of workload corresponding to a numerical (ordi- nal) rating. Pilot-induced Oscillation (PIO) Rating Scale The PIO Rating Scale (see Appen- dix VI of the U.S. Naval Test Pilot School Flight Test Manual ) classifies the suscep- tibility of pilot-induced oscillations (i.e., aircraft-pilot coupling or pilot- assisted oscillations) when perform- ing a task. The rating is hierarchical and numerical at six levels. Also, like the other pilot rating scales men- tioned, the choices on the scale do not define an interval scale since the distance between the choices cannot be construed as equal (i.e., a rating of 6 on the PIO scale does not repre- sent double the susceptibility of a PIO rating of 3). The aforementioned pilot rating scales, as well as others, were devel- oped and have been utilized assuming a human-centric application that the performance of the human-machine team is evaluated rather than just the machine performance. The distinc- tion is that the ratings are influenced by the human pilot’s skill, alertness, and strength as well as practical in- telligence, judgment, and creativity, especially when confronted with un- certain environments and unforeseen situations. However, a rating scale has yet to be developed specifically tailored for the T&E of aircraft with AI systems. Knowledge Gap and Terminology Current regulations, guidance, specifications, and policies lack any standardization or, in many cases, even the existence of terminology that will be needed for the T&E of AI
Figure 2. Bedford Pilot Workload Scale
Operator Demand Level
Rating
Workload insignificant.
1
Workload low.
2
Enough spare capacity for all desirable additional tasks.
3
Yes
Insucient spare capacity for easy attention to additional tasks.
4
Was workload satisfactory without reduction?
Reduced spare capacity. Additional tasks cannot be given the desired amount of attention. Little spare capacity. Level of eort allows little attention to additional tasks.
5
No
6
Yes
Very little spare capacity, but the maintenance of eort in the primary task is in question. Very high workload with almost no spare capacity. Diculty in maintaining level of eort. Extremely high workload, no spare capacity. Serious doubts as to the ability to maintain level of support.
7
Was workload tolerable for the task? No
8
9
Yes
Was it possible to complete the task?
10
Tasks abandoned. Pilot unable to apply sucient eort.
No
Enter Here
(Roscoe, 1984)
Note. Adapted from A Subjective Rating Scale for Assessing Pilot Workload in Flight: A Decade of Practical Use (Report No. TR 90019) , by A. H. Roscoe and G. A. Ellis, 1990 (https://apps.dtic.mil/sti/tr/pdf/ADA227864.pdf). In the public domain.
systems. Hence, it is critical to estab- lish some generalized terminology for AI and pilot intervention, with the latter encompassing key words such as uncertain environments , unforeseen situations , and threats . Defining AI There are a multitude of definitions for AI. However, one has been codified in United States Public Law 116–283 under the National Artificial Intelligence Ini- tiative Act: The term “artificial intelligence” means a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations or decisions influencing real or virtual en- vironments. AI systems use machine and human-based inputs to— A. perceive real and virtual environments; B. abstract such perceptions into mod- els through analysis in an automated manner; and C. use model inference to formulate op- tions for information or action. (§9401).
The Federal Aviation Administra- tion (FAA) definition for AI provided in the “FAA Roadmap for Artificial Intelligence Safety Assurance” distinguishes between implementations of fixed and deter- ministic algorithms (i.e., Learned AI), and implementations that continue to change through learning in the op- erational environment (Learning AI). The DoW AI Strategy also uses the term Learning in its following definition. AI refers to the ability of machines to per- form tasks that normally require human intelligence—for example, recogniz- ing patterns, learning from experience, drawing conclusions, making predictions, or taking action—whether digitally or as the smart software behind autonomous physical systems. A rigorous combination of the definitions from the National Artifi- cial Intelligence Initiative Act, DoW, and FAA, along with others from in- dustry documents, yields a somewhat complex characterization.
November-December 2025 | DEFENSE ACQUISITION | 25
Made with FlippingBook - Online Brochure Maker