Historically, the most prominent and contentious impediment to intro- ducing new and novel technologies has been the excessive time required for aviation authorities to certify air- craft systems that lack established regulations, guidance, specifications, or standards, as is the case as well for AI systems. The performance of tasks and functions by AI systems under the premise of replacing the human op- erator (i.e., pilot) must be evaluated to determine if AI systems provide an equivalent (or better) performance and safety compared with that of a human—not just under normal, ab- normal, and emergency conditions but also in uncertain environments and unforeseen situations. These evaluations demand a clearly defined process to minimize extended nego- tiations between aviation authorities and manufacturers to facilitate and expedite certification of aircraft and Various recognized pilot rating scales have been used for decades by flight test pilots, flight test engineers, and human factors specialists during the test and evaluation (T&E) of air- craft and aircraft systems. Most such “scales” provide a hierarchal quanti- tative measure of qualitative evalua- tions through simple flow diagrams with binary (i.e., ”yes” or ”no”) deci- sions of flight testers based on their observations and judgment of how well the human-machine team can show compliance in performing regu- latorily required tasks and functions. For example, there are well-estab- lished, industry-accepted pilot rating scales for aircraft handling qualities (i.e., flying qualities), pilot workload, and aircraft-pilot coupling (i.e., pilot aircraft systems using AI. Pilot Rating Scales
or characteristics of an aircraft that govern the ease and precision with which a pilot is able to perform the tasks required in support of an air- craft role.” Historically, researchers and manufacturers developed their own rating scales for conducting and documenting handling qualities; hence, there was no standardization or a consistent structured method for manufacturers to show compli- ance and for military/civil aviation authorities to find compliance with the relevant regulations. Recognizing the need for an in- dustry standard, George E. Cooper of Ames Research Center developed an initial Pilot Rating Scale for handling qualities; the scale was meant to use key words and phrases easily under- stood yet sufficiently separated to
enable a series of definitive dichoto- mous decisions leading to a numerical (ordinal) rating. Over the years, the scale was modified to the present Cooper-Harper Handling Qualities Rating Scale (HQRS) as shown in Figure 1, now the foundation of most pilot rating scales used in aviation research, develop- ment, and aircraft certification. Bedford Pilot Workload Scale The Bedford Pilot Workload Scale (BPWS) —see Figure 2—developed by test pilots at the former Royal Aircraft Establishment in Bedford, England, is based on the HQRS. The BPWS has a comparable decision-tree design except that, rather than evaluate handling qualities, it evaluates pilot workload defined as the “integrated
Figure 1. Cooper-Harper Handling Qualities Rating Scale (HQRS)
Adequacy for Selected Task or Required Operation*
Aircraft Characteristics Excellent Highly desirable
Demands on the Pilot Selected Task or Required Operation* Pilot compensation not a factor for desired performance Pilot compensation not a factor for desired performance Minimal pilot compensation required for desired performance Desired performance requires moderate pilot compensation Adequate performance requires considerable pilot compensation
Pilot Rating
1
Good Negligible deficiencies Fair—some mildly unpleasant deficiencies
2
3
Yes
Minor but annoying deficiencies
4
Is it satisfactory without improve- ment?
Moderately objectionable deficiencies
Deficiencies warrant improvement
5
No
Very objectionable but tolerable deficiencies
Adequate performance requires extensive pilot compensation
6
Yes
Major deficiencies
Adequate performance not attainable with maximum tolerable pilot compensation. Controllability not in question. Considerable pilot compensation is required for control Intense pilot compensation is required to retain control
7
Is adequate performance attainable with a tolerable pilot workload?
Deficiencies require improvement
Major deficiencies
8
No
Major deficiencies
9
Yes
Major deficiencies
Control will be lost during some portion of required operation
Improvement mandatory
Is it controllable?
10
induced oscillations). Cooper-Harper Rating Scale
No
Pilot decisions
*Definition of required operation involves designation of fight phase and/or accompanying conditions
Aircraft development and certifica- tion require an evaluation of handling qualities, defined as “those qualities
Note. Adapted from The Use of Pilot Rating in the Evaluation of Aircraft Handling Qualities (No. NASA-TN-D-5153), by G. E. Cooper and R. P. Harper Jr., 1969 (https://ntrs.nasa.gov/api/citations/19690013177/downloads/19690013177.pdf). In the public domain.
24 | DEFENSE ACQUISITION | November-December 2025
Made with FlippingBook - Online Brochure Maker