Defense Acquisition magazine Reimagining Innovation Bimonthly magazine of the Defense Acquisition University for senior military personnel, civilians, defense contractors, and defense industry professionals in program management and the acquisition, technology and logistics workforce. Defense Acquisition Magazine November-December 2025 Vol LIV | No. 6 | Issue 307
A PUBLICATION OF THE WARFIGHTING ACQUISITION UNIVERSITY NOVEMBER–DECEMBER 2025 DEFENSE ACQUISITION
A New Systems Approach
The Value Added
Planning and Operating With AI
WE’RE LOOKING for a Few Good Authors Got opinions to air? Want to help change the way the Department of War does business? Willing to share your expertise with the acquisition community? Interested in passing on lessons learned from your project or program?
Write an article (ideally, 1,500 to 2,500 words long), and Defense Acquisition magazine will consider it for publication. Our readers are interested in real-life, hands-on experiences that will help them expand their knowledge and do their jobs better. WHAT’S IN IT FOR YOU? First off, seeing your name in print is quite a kick. But more than that, publishing in Defense Acquisition can help advance your career. One of our authors has even been offered jobs on the basis of ar- ticles written for the magazine.
Now we can’t promise you a new job, but many of our authors: » Earn continuous learning points » Gain recognition as subject-matter experts » Are invited to speak at conferences or symposia » Get promoted or rewarded
For more information and advice on how to submit your manuscript, check the writer’s guidelines at https://www.dau.edu/library/damag/writersguidelines or contact the managing editor at defacqmag@dau.edu .
If you’re interested in having longer, scholarly articles considered for publication in the Acquisition Research Journal , or if you’re a subject-matter expert and would be willing to review articles, contact the managing editor at defensearj@dau.edu . Be sure to check the guidelines for authors at https://www.dau.edu/library/damag/writersguidelines .
SUBSCRIBE FREE PUBLICATIONS Sign up on our website to have the Defense Acquisition magazine and Defense Acquisition Research Journal delivered straight to your inbox.
www.dau.edu/publications
06 A Systems Approach to Sustainable Innovation Amanda M. Girth, Ph.D.; Laura Ma g uire, Ph.D.; and Michael F. Rayo, Ph.D.
The new Innovation Alliance Program can help Department of War (DoW) ac- quisition teams identify and scale grass- roots innovations by diagnosing system- ic barriers and stress-testing solutions.
12 The Value of
Warfighting Innovation Thom Hawkins and Tom Housel
Traditional return on investment met- rics are insufficient for evaluating inno- vation. Use of complexity-based metrics like knowledge value-added is proposed as an alternative. 18 Revolutionizing Planning and Operations With AI Maj Caleb “CJ” Werner, USSF
23 Artificial Intelligence— Trust But Verify Col. Robert Joslin, USMC (Ret.), Ph.D.
A Pilot Intervention Rating Scale can be created for use during Test & Evaluation flights involving artificial intelligence systems.
AI can transform space systems acqui- sition through practical applications and lessons learned from implementing the large language models within Space Sys- tems Command.
34 Spectrum Certification and Supportability Risk Assessments— Not Just Radio Red Tape James Ortega
Safe and secure battlefield communica- tions depend on careful compliance with U.S. law and host-nation requirements.
38 Contract Disputes and Their Resolution Jennifer Jones
Disagreements are inevitable. But there are abundant opportunities to resolve many of them.
44 Contracting Officer
22 MDAP Program Manager Changes 48 Statement of Ownership Procurement Integrity Act Investigations Lt. Col. Michael J. Davidson, USA (Ret.), S.J.D. Trading on insider information and oth- er integrity violations can trigger serious consequences for program managers and contractors.
28 Doers, Not Reviewers— Leadership Role in Agile Software Development David Kuroda
Successful agile software development within the DoW depends on a shift to active, partici- patory leadership that removes obstacles and fosters cross-functional collaboration.
DEFENSE ACQUISITION
VOL LIV
NO. 6, ISSUE 307
A Publication of the U.S. Department of War
Under Secretary of War for Acquisition and Sustainment Michael Duffey
WARFIGHTING ACQUISITION UNIVERSITY President Bilyana Anderson Chief of Staff Angela Carsten Director, Communications & Public Affairs Christen Goulding Associate Director, Visual Arts and Press Norene L. Johnson Defense Acquisition Editorial Staff
Managing Editor Benjamin Tyree Art Director Michael Krukowski
Assistant Editor Collie J. Johnson Associate Editor Dr. Larrie Ferreiro Production Manager Frances Battle Circulation Manager Michelle McDonald
Editorial Assistant Susan Muth Art and Graphic Design Nicole Brate Paul Kim Editorial Support Dr. Olena McLaughlin
50 Conferences and Meetings
To contact the Editorial Staff, email defacqmag@dau.edu or call 202-579-4628. Visit our website for additional articles and resources: www.dau.edu/library/damag For information on how to submit an article, please consult our Writer’s Guidelines, which are available at the back of each issue or on our website. To subscribe to our digital publication or update your current subscription information, click on the “sub- scribe” button on our homepage or email the editors at the address above. Defense Acquisition (ISSN 2637-5052 [print] and ISSN 2637-5060 [online]), formerly Defense AT&L and ear- lier Program Manager, is an official publication of the Department of War. Defense Acquisition is published bimonthly and is free to all U.S. and foreign national subscribers. Periodical postage is paid at the U.S. Postal Facility, Fort Belvoir, Va., and additional U.S. postal facilities. Postmaster, send address changes to: Editor, Defense Acquisition, Warfighting Acquisi- tion University, 9820 Belvoir Road, Fort Belvoir, Va. 22060-5565. Disclaimer Defense Acquisition magazine promotes the free ex- change of ideas. The views expressed are those of the authors and do not reflect the official policy or position of Warfighting Acquisition University, the Department of War, or the United States Government. Articles are in the public domain and may be reprinted or posted on the Internet. When reprinting or posting, please credit the authors and Defense Acquisition. Photos in this publication may have been sourced from the Department of War website (www.war.gov/ Multimedia/Photos/). The appearance of the U.S. De- partment of War (DoW) visual information does not imply or constitute DoW endorsement. Some images may be digitally enhanced.
Current conferences and meetings of interest to our magazine’s readers.
51 Acquisition News and Highlights
Acquisition-related news, press releases, announ- cements, and more on topics relevant to the ac- quisition professional. 53 What’s New at Our University University news, deployment of new credentials, updated online training, available workshops, web- inars, and other learning opportunities.
Click to Read our Digital Publication
WINNER AWARD
2025
EMILY ASHCOM, BENJAMIN TYREE, NICOLE BRATE, SUSAN MUTH, COLLIE JOHNSON, and VISUAL ARTS AND PRESS STAFF WARFIGHTING ACQUISITION UNIVERSITY
A Systems Approach TO SUSTAINABLE INNOVATION
by AMANDA M. GIRTH, PH.D., LAURA MAGUIRE, PH.D., and MICHAEL F. RAYO, PH.D.
Sustained technological innovation will build the edge of warfighters in multiple Services and domains. Image is derived from a U.S. Air Force photo of a Military GPS User Equipment Demonstration at Fort Knox, Ky., in June 2025. Source: U.S. Air Force photo by Tech. Sgt. Kaleb Mayfield This image was cropped to show detail and was edited using multiple filters plus dodging and burning techniques.
T he pursuit of innovation in the public sector often meets a roadblock: in- stitutional norms that prioritize procedural compliance and the status quo. These patterns can stall innovative practices. Likewise, instilling an interven- tion strategy that emphasizes leveraging innovative tools throughout Department of War (DoW) also faces challenges in aligning, prioritizing, and adopting innovations in acquisition operations.
The DoW is mandated to meet goals set forth in the National Defense Strategy and Executive Order 14265 to mod- ernize its acquisition processes, with an emphasis on innovation and adapt- ability to meet evolving strategic de-
mands. Rapidly evolving threats from adversaries such as China and Rus- sia necessitate that DoW’s Adaptive Acquisition Framework, the governing system for acquiring capabilities, be continuously updated and adaptable
throughout the acquisition process, thereby ensuring that DoW remains operationally effective and mission- ready. Innovation, as a crucial com- ponent of that process, enables acqui- sition professionals to respond with
6 | DEFENSE ACQUISITION | November-December 2025
agility, ensuring that the DoW can deliver timely and effective solutions to support the Warfighter. Development of Program Structure The DoW’s Acquisition Innovation Re- search Center and its new Innovation Alliance Program (IAP) now offer warfighting innovators a decision- making model/tool that continu- ously monitors and identifies the barriers and facilitators to a healthy innovation culture. We developed this decision-making tool to interpret
these barriers and signals and target proven remedies through a variety of techniques, such as interviews, work- shops, stress testing, refinement, and scaling of high-potential innovations. In so doing, we address a critical DoW challenge: how to ensure that promis- ing local innovations are not lost but instead matured and mobilized to cre- ate lasting, enterprise-wide impacts. About the IAP Process The IAP process began by embed- ding systems thinking into both the assessment and design phases, thus
enhancing an organization’s ability to scale locally developed solutions and overcome systemic barriers to inno- vation. The process is lightweight yet analytically robust and supports early identification of innovative ideas, structured stakeholder dialogue, and data-driven refinement of implemen- tation strategies. A tailored suite of techniques elicits grounded insights into how individuals navigate complex, high-pressure work environments and create sustain- able solutions. Originally developed in the Cognitive Systems Engineering Lab
November-December 2025 | DEFENSE ACQUISITION | 7
Figure 1. See-Do-Teach Model
ing, aligning team goals, internal and external collaboration, and supporting autonomy all contribute to innovative processes. Instead of focusing on individual characteristics, our SCAD method- ology captures and weaves together stories from the acquisition work- force. These narratives illuminate how adaptations arise in the face of com- peting priorities, limited resources, and structural pressures. This ap- proach shifts the analysis from iso- lated events to a systems-level view of the operational environment. This approach offers both a diagnostic snapshot and foresight into where future challenges or adaptations are likely to unfold. From our efforts, we noted repeat- edly that these systemic priorities, resources, and pressures contribute to inhibiting or facilitating innovative behaviors more than individual at- tributes or differences between ana- lysts. Certain system attributes, such as a willingness to embrace failure and support for organizational learn- ing, can enhance innovative efforts. However, other pressures, such as stringent adherence to procedures and time constraints, inhibit innova- tion. But barriers to innovation can be over-come by securing leadership support,
Implement A team of DoW partici- pants use the research- backed workshop method to “stress test” the imple- mentation of innovation initiatives.
Identify The lightweight interview method is used to identify barriers and facilitators to innovation and monitor the status of innovation culture within DoW.
Interpret Interview data are applied to the DoW model to as- sist with interpreting the innovation trajectories and support targeted im- provement efforts.
Innovation Alliance Serves to develop the capacity through training and strengthen the collaborative networks within DoW to enchance the scalability and adoption of innovative practices that can cost effectively modernize acquisitions to deliver capability to the Warfighter when needed.
Source: Author
at The Ohio State University and ap- plied in high-stakes safety domains such as commercial aviation and healthcare, the IAP is now adapted for use in DoW acquisition, where complexity and operational risk simi- larly demand a nuanced, systems- informed approach to innovation. The three core functions of the program follow: • A method for continuous moni- toring to identify signals of bar- riers and facilitators to a healthy innovation culture using the light- weight Systemic Contributors and Adaptations Diagramming (SCAD) interview technique. • A model and a tool to aid in the in- terpretation of the signals collected in the identification activities and target improvement efforts. • The Accelerating IMPActS Workshops (AIWs) that use a co-design process to stress test high-potential inno- vations for supporting the transi- tion of high-potential ideas and improve their ability to be imple- mented and sustained at increasing scale. The See-Do-Teach model is a pro- gressive framework designed to build internal capability for executing, ex- panding, transitioning, and sustaining the IAP. Its approach to organizational learning is critical to the success of the IAP. We apply the three core compe- tencies of the IAP—identifying (e.g., conducting interviews), interpreting
(e.g., analyzing data), and imple- menting (e.g., leading workshops)— through a model that guides partici- pants through three stages of skill acquisition: See , where individuals observe and receive foundational training; Do , where individuals ac- tively conduct and interpret research activities with support and coaching; and Teach , where individuals train others and facilitate activities inde- pendently. This approach ensures a structured transition from novice to expert and anchors knowledge trans- fer by building internal capacity to ex- ecute the program. Diagnosing Risk Aversion To assess intervention ideas de- signed at improving adoption of or- ganizational changes, we first need to proactively uncover barriers to implementation before they lead to intervention failure. In collaboration with the Air Force Installation Con- tracting Center (AFICC) and the Army Contracting Command-Aberdeen Prov- ing Ground (ACC-APG), The Ohio State University researchers piloted the IAP in the DoW. Em- ploying these novel interviewing techniques revealed key organi- zational dynamics that foster in- novative acquisition practices. For example, making room for failure, measured risk-taking, fostering organizational learn-
8 | DEFENSE ACQUISITION | November-December 2025
aligning team goals, fostering inter- nal and external collaboration, and granting autonomy to empower the acquisition workforce. Organizational pressures in ac- quisition either strengthen or erode existing systemic attributes. These can involve policies, processes, regu- lations, time constraints, innovation prioritization, the balance of work demands, budget constraints and resources, personnel turnover, reli- ance on routines, political exposure or public sector scrutiny, personal reputation, external events, and or- ganizational relationships. Stress Testing Innovations to Scale The full value of DoW innovation is realized only when its benefits are more broadly circulated. While local- ized or one-off innovations may offer immediate operational improvements to the implementing unit, they fail to produce systemic change or address enterprise-level challenges. Issues with cross-boundary coordination, resource constraints, heavy workload, and conflicting goals can contribute to innovations remaining local. This reduces the return on investment in terms of saving time, sharing labor, and increasing institutional learning. From an organizational perspec- tive, the spread of innovation is essen- tial to fostering institutional adapt- ability and resilience.
and lost opportunities for enhanced mission outcomes. AIWs actively foster innovation dissemination throughout an orga- nization. The AIW is a co-design strategy rooted in systems engineer- ing, which stress-tests innovations to scale. Anchored by the IMPActS Framework—comprised of five di- mensions: Ideas, Mental Model Alignment, Pragmatics, Actors, and Sustainment—AIWs address the fre- quent disconnect between promising interventions and their long-term vi- ability. AIWs convene diverse stakehold- ers—including operational staff, orga- nizational leaders, and the workshop facilitators themselves—in a struc- tured forum to critically examine po- tential interventions through the lens of the five IMPActS elements. The process blends guided dialogue with qualitative assessment, enabling par- ticipants to identify misalignments, unveil system-level constraints, and refine interventions with practical foresight. Through facilitated ex- ercises and shared understanding, the workshop fosters alignment and builds adaptive capacity into interven- tion design from the outset. In our pilot at AFICC, AIWs proved effective in surfacing hidden imple- mentation barriers, aligning stake- holder assumptions, and refining interventions in ways that less struc- tured methods could not. The importance of aligning men- tal models across stakeholder roles,
which is foundational to both inter- vention design and long-term viability, emerged as a key theme throughout AIW probing. Participants also valued the framework’s focus on sustain- ment, which highlighted the need to anticipate future resource demands, institutional support, and engage- ment strategies beyond initial adop- tion. Later workshops expanded the process to include practical mitigation planning and integration of ideas from grassroots innovation efforts. Across all sessions, participants reported increased ownership, im- proved clarity of implementation trade-offs, and enhanced confidence in proposed solutions. These out- comes demonstrate the AIW’s poten- tial as both a diagnostic and genera- tive tool, supporting scalable change across complex operational systems. Leadership Support When supported by leadership and institutional mechanisms, grassroots innovations can be assessed for scal- ability and integrated more broadly across the enterprise. Fostering a culture of innovation that encourages calculated risk-tak- ing supports organizational learning, promotes alignment across teams, and grants autonomy is essential for sustainable transformation. Leaders play a pivotal role in cultivating this environment by providing “top cover” for experimenting, aligning incentives, and championing promising innova- tions.
When innovations are restricted to specific units, other parts of the organization continue experi- encing inefficiencies or capability gaps that have been solved else- where. This results in duplication of effort, wasted resources,
November-December 2025 | DEFENSE ACQUISITION | 9
Leadership facilitates innovative behaviors by the following actions: • Being available and accessible to team members. • Giving frequent feedback to realign goals across the organization, ad- dress and learn from issues, and generate new ideas and insights. • Allowing subordinates to not know all the answers but to ask ques- tions and share knowledge. • Providing “top cover” for teams and individuals experimenting with innovative solutions. • Sustaining innovations even when the originator has left the team. They can act as a throughline for the innovation, orchestrating the handoff and providing ongoing momentum. • Driving teams to articulate trade- offs to account for risk versus re- ward when making ambiguous decisions. However, leadership discourages innovative behavior when a person in authority does not share common goals and shuts down innovation, does not delegate authority to create changes, or fails to provide an inno- vator enough influence to advocate a new approach. A change in leadership greatly impacts the goals and innova- tive capabilities of a team both facili- tating and blocking innovation. This is particularly true during leadership turnover when incoming leaders who want to innovate can create an envi- ronment that allows more risks and boundary pushing. It’s also true that new leaders who prioritize maintain- ing the status quo can block innova- tions begun by their predecessors. In everyday work, high-potential grassroots innovation interventions are being developed locally by the acquisition workforce to quickly solve problems. However, to benefit from and amplify this innovative behav- ior, the organization needs to assess emerging innovations to determine their scalability and address systemic barriers that hinder broader adoption, dynamically prioritize those that it will foster and proliferate, provide
Fostering a culture of innovation that encourages calculated risk-taking supports organizational learning, promotes alignment across teams, and grants autonomy is essential for sustainable transformation.
resources to support the transition to global interventions, and support leaders who use innovation-focused approaches. The IAP can be readily transitioned to and adopted by DoW components to tackle fundamental barriers to speed, agility, and innova- tion in warfighting acquisition. GIRTH, a nationally recognized expert in acq- uisition policy and practice, is an associate professor and faculty director of Washing- ton Programs at the John Glenn College of Public Affairs, The Ohio State University. She has testified before Congress and led DoW- sponsored research. Girth is a member of the Research Council for the Systems Engi- neering Research Center, a DoW University- Affiliated Research Center that manages the DoW’s Acquisition Innovation Research Cen- ter. She holds a Ph.D. in Public Administration from American University, an MBA from The George Washington University, and a B.S. in Public Administration and Policy from Oak- land University. MAGUIRE is a research scientist with The Ohio State University’s Cognitive Systems Engineering Lab, with extensive experience working with industry and governmental or- ganizations. She has led studies and advised Fortune 500 companies, start-ups, industry and professional associations, and govern- ment entities. Maguire holds a Ph.D. in Inte- grated Systems Engineering from The Ohio State University, an M.S. in Human Factors and Systems Safety Engineering from Lund University, and a B.Comm. in Entrepreneurial Management from the Royal Roads University. RAYO is an associate professor of integrated systems engineering at The Ohio State Uni- versity’s College of Engineering. His research focuses on how the design of cognitive tools influences multi-agent teaming architectures that contribute to system behaviors. His work on how to foster system resilience has been funded by the National Patient Safety Foun- dation, the Agency for Healthcare Research
and Quality, the Office of Naval Research, and the Air Force Research Laboratory. Royo holds a Ph.D. in Health and Rehabilitation Sciences and an M.S. in Industrial and Systems En- gineering from The Ohio State University, as well as a B.S. in Chemical Engineering and a B.A. in Music Performance from Case Western Reserve University. The authors can be contacted at Girth.1@osu.edu , Maguire.81@osu.edu , and Rayo.3@osu.edu . This material is based upon work supported, in whole or in part, by the U.S. Department of War through the Office of the Under Secretary of War for Acquisition and Sustainment and the Office of the Under Secretary of War for Research and Engineer- ing under Contract HQ0034-19-D-0003, Task Orders TO#0285, and TO#0309. The views, findings, conclusions, and recommenda- tions expressed in this material are solely those of the authors and do not necessarily reflect the views or positions of the United States Government (in- cluding the Department of War and any government personnel), the Stevens Institute of Technology, or The Ohio State University. Reproduction or repost- ing of articles from Defense Acquisition magazine should credit authors and the magazine.
Related Resources • Innovate to Win Playbook (Online Tool) • EXE 4050V Leading Change to Drive Innovative Culture (Virtual Instructor-Led Training) • Scaling Innovation (Video) • Driving Innovation in Acquisition (Web Event Series)
10 | DEFENSE ACQUISITION | November-December 2025
2026 Warfighting Acquisition Workforce INNOVATION in ACTION AWARD
Share your story and help shape the future of warfighting acquisition.
INNOVATION IN OVERCOMING OBSTACLES UP TO FIVE $5,000 AWARDS A firsthand account of using independent judgment and an original idea to overcome a warfigthting acquisition obstacle. INNOVATION IN IMPLEMENTING ACQUISITION FLEXIBILITIES ONE $5,000 AWARD A documented program management approach using innovative acquisition flexibilities and authorities. ELIGIBILITY All military and civilian members of the warfighting acquisition workforce are eligible. Contractors and university employees are not eligible. SUBMISSION DEADLINE: MARCH 2, 2026 Essays should be 1,000 to 1,500 words Submit by email to innovation.awards@dau.edu
For complete guidelines, visit www.dau.edu/innovation-in-action-awards
THE VALUE of Warfighting Innovation by THOM HAWKINS and TOM HOUSEL
A t 2300 Zulu, an air-to-surface missile, guided by laser to its objective, makes impact. The battle damage assessment confirms that the target has been destroyed. Mission accomplished! A great deal of effort has been invested in this mission, encompassing intelligence gathering and logistics, as well as technology development and command and control. We all share in the success. Any attempt to attribute credit to one system or another is beneath us as a military.
12 | DEFENSE ACQUISITION | November-December 2025
ammunition, guidance, targeting, and even enablers such as intelligence and logistics. The capability to move en- compasses vehicles, of course, but also geospatial systems and logistics. Finally, the capability to communicate includes radios, computers, satellites, and networks. The effective and ef- ficient combination of all three core capabilities is required to achieve successful outcomes. The proponent for each system’s investment appeals for operational impacts, but no one has ever at- tempted to aggregate the individual impacts across all those operations. We do know how much it costs to shoot, to move, or to communicate; cost accounting can tell us that. A more detailed assessment of the value that each capability brings to the fight would need to include the performance process outputs that demonstrate accurate targeting for all warfighting ordnance and associated systems. These operations are in the hands of the military specialist, who ultimately will operate the system under various conditions in accor- dance with the concept of operations. Estimating the ROI of each core process and the contributions of its human operators and the systems they use is a significant challenge, highlighting the problem of using capability as the monetized value numerator and the process cost (for humans and systems) as the denomi- In his 1789 book, An Introduction to the Principles of Morals and Legislation , Jer- emy Bentham wrote that “nature has placed mankind under the governance of two sovereign masters, pain and pleasure.” Bentham used this idea to build a framework for what became known as “utilitarianism,” which later was expanded by John Stuart Mill, proposing that our actions are guided by a desire to maximize pleasure and minimize pain. An essential element of utilitarianism is the unification of all valued activity to a standard unit—a nator in an ROI calculation. Buy More, Save More?
U.S. Air Force Munitions Systems Airmen assigned to the 174th Attack Wing and members of the 178th Wing load an inert GBU 12 laser guided missile onto an MQ-9 Reaper at Springfield-Beckley Air National Guard Base in Springfield, Ohio, on March 18, 2025. Source: U.S. Air National Guard photo by Senior Airman Colin Simpson This image was cropped to show detail and was edited using multiple filters plus dodging and burning techniques.
Meanwhile, back at home base, it’s time to make a funding request. That request must include some jus- tification for the money, such as an operational impact—$4 million will allow us to procure 60 more devices, and those devices will be fielded to three units, giving them the capabil-
ity to defend against incoming mor- tars. Money buys capability—that is our return on investment (ROI). The Army’s core capabilities, according to doctrine, can be stated simply as “shoot, move, and communicate.” The ca- pability to shoot could encompass various systems, including weapons,
November-December 2025 | DEFENSE ACQUISITION | 13
whether these investments directly relate to a more effective military is an open question because we do not have an unambiguous metric for the value produced by these investments. Enter AI “London School of Economics adopts AI- powered procurement system,” touts a re- cent headline, “reports 15 percent cost savings.” Another ominously announ- ces that “Microsoft uses AI to save $500 mil- lion, lays off workers.” These news reports imply that the pursuit of cost savings justifies the expense of implementing AI, or, more generally, reflects an ap- parent and clear relationship between investments in tech-nology and pro- ductivity because of a reduction in the ROI denominator. In a December 1993 article titled “The productivity paradox of information tech- nology,” Erik Brynjolfsson examined why business productivity declined in the 1970s despite massive invest- ments in information technology. He concluded that the apparent paradox could be attributed to an inadequate measurement scheme for the new technology, resulting in the mis- management of resource allocation. Now a Stanford University professor, Brynjolfsson has revised his conclusions somewhat, particularly in light of the 1990s, when a stronger correla- tion emerged between technology investment and firm-level productiv- ity. He now attributes the mismatch to a lack of understanding about how technology can or should be used to benefit productivity. Brynjolfsson cites this conclusion in his argument against the level of cur- rent AI investments—that we are failing to tie our implementation of AI technology to business outcomes. In other words, we do not yet know the returns on AI—we need more time to fully understand how AI should be used and where it makes sense. Our current investment strategy, which assumes that AI is a mature technol- ogy with a well-established correla- tion to productivity, leans too heavily into assumptions about the relation-
Just as a sign in a store might claim “buy two and save 50 percent,” claims of cost savings often are exaggerated and should not be the sole focus of military investments.
“util” or unit of utility that can mea- sure the pleasure or pain an action in- curs. The util metric makes sense until you try to use it in practice. Not all utils are the same and therefore can- not be used to compare the relative utility value of any action, process, or resource against any other. Despite the progress of the inter- vening centuries, the problem persists of identifying a common unit of value across disparate processes. Even if the Army could identify standard units of destruction, capability, readi- ness, or maneuverability, comparing the value of one type of Army process relative to the other is unrealistic. The ROI formula is the ratio of sales return (revenue minus investment) to the in- vestment alone:
focus of military investments. Mak- ing them the focus, taken to its logical conclusion, would mean maximizing cost savings by buying nothing or perhaps using appropriated funds to start a for-profit side business in- stead of buying weapons and tanks to help us win wars. (In business, this is called the cost-cutting death spiral and would be a dangerous approach for an Army that needs to win a war.) When we invest in the stock mar- ket, we buy a stock at a particular price with the expectation that its value will rise and we can later sell it for a higher price. At the time of pur- chase, we are making an investment. But because we are buying at market price, the revenue is zero. Over time, the stock price will fluctuate due to factors such as company earnings, market competition, and past volatil- ity of the stock price. If the price goes up, that is revenue from the perspec- tive of the stock owner (unrealized until sold), and therefore the ROI goes up. If the stock price goes down, so does the ROI. However, when Congress appro- priates funds for the military, we are expected to invest that money into the goods and services that support military core processes. These funds represent taxpayers’ monetized in- vestments. Therefore, they cannot be the equivalent of sales revenue. The expectation is that the military will turn these taxpayer investments into valuable products and services that protect the country, which is a bit like paying for insurance. However,
Revenue – Investment Investment
ROI =
An investment (that is, the cost to acquire or use a resource) is the focus of any funding request. In the world of business, revenue is gained from sales of a product or service. But because the government does not sell prod- ucts or services, we cannot use the ROI formula in standard financial ac- counting. Because dollars are an accepted denominator, it is tempting to use them as the numerator as well. So, cost savings are often used as a sur- rogate for the monetized value of rev- enue. Just as a sign in a store might claim “buy two and save 50 percent,” claims of cost savings often are ex- aggerated and should not be the sole
14 | DEFENSE ACQUISITION | November-December 2025
ship between funding and implemen- tation. There is much that we do not yet know about AI’s capabilities or how to implement them optimally. Despite these challenges to gaug- ing the contribution of AI to produc- tivity, the DoW is making significant AI investments. These investments are not aimed simply at cutting costs and reducing the denominator of the ROI ratio. They are made to increase capabilities that will allow us to over- match our adversaries, who are also making extremely large investments in AI. So, we are involved in an AI arms race. These investments in improving our capabilities are a value play that requires a useful numerator estimate. Capabilities find their way into core processes where the AI is actually used for targeting, signal intelligence interpretation and detection, cy- bersecurity, and other core military processes. For this reason, we need a value metric that can help make a robust and convincing argument in forecasting AI’s future value. Unless value can be measured in common units—e.g., common units of value in terms of capability or readiness—we will have a very difficult challenge in convincing skeptical lawmakers, executives, and generals that the AI Various methods are available for measuring the impact of processes as they transform a set of inputs into a set of outputs, from raw materials to a finished product. A technique called knowledge value-added (KVA) uses Kolmogorov complexity, or K- complexity, originally based on a measure of the length of a computer program to produce a particular out- put. For example, outputting a string of eight zeroes (00000000) could be simplified by telling the program to print “0” eight times, so it has a low K-complexity. However, an output of random alphanumeric characters (12CGTj52) has a higher K-complexity because the program used to define investments are a good bet. Complexity and Value
a series of random characters would be longer. KVA assumes that value is added when inputs are changed into out- puts. In lieu of measuring the length of computer programs needed to pro- duce a particular output, KVA mea- sures complexity by the time it takes a notional average employee, given the necessary inputs, to learn how to pro- duce the specified outputs. The gen- eral idea is that the more complex the process, the longer it takes to learn. Therefore, learning time provides an estimate of the complexity—i.e., inputs being transformed into out- puts. This enables the comparison of various processes, such as gathering signal intelligence or planning a ship alteration. The DoW has applied KVA to at least 45 areas within the depart- ment, including flight scheduling and ship maintenance. The application of KVA to ques- tions of AI implementation is straight- forward. It is simply a matter of de- termining the difference between the time it would take a civilian or military employee to learn how to produce
Unless value can be measured in common units— e.g., common units of value in terms of capability or readiness— we will have a very difficult challenge in convincing skeptical lawmakers, executives, and generals that the AI investments are a good bet.
November-December 2025 | DEFENSE ACQUISITION | 15
something without the aid of AI and the time to do so with AI’s help. If the AI correctly produces the outputs of a given process that, for example, an operator formerly produced, it has captured the knowledge of that op- erator in the AI program. Consider the task of producing an executive summary based on a meeting transcript. Someone would need to be trained to understand the role and format of an executive sum- mary and likely practice determining the right level of detail for the audi- ence. It may take less time for that same person to learn how to write a prompt to ask a large language model (LLM) to write an executive summary based on the transcript. However, the LLM may not have the context to know what is important to the audi- ence and to perform this seemingly mundane process.
It follows that, in this case, the re- turn in using AI may not be worth the investment. In other cases, perhaps with a more complex task like anom- aly detection, AI may provide a more significant advantage by combining the knowledge of many operators into a single, fast output. KVA provides an objective, supportable measure of value to reach that decision. Conclusion Optimistically, AI is a powerful tool that may improve the precision, ac- curacy, capability, and power of our systems. But we should not mistake promise for results. The task of quan- tifying returns for our investments— whether in capability, readiness, pro- ductivity, pleasure, or pain—remains a challenge. A deliberate, extensible, and repeatable approach, such as KVA, will ensure that our infrastruc- ture—not only our systems of mea-
surement and accounting, but also testing, training, and practice—keeps pace with the technology we procure for our soldiers. The next steps will include further pilots for KVA or similar complexity- based techniques to provide ready examples for future analyses. Unde- niably, there remains the challenge of formalizing a process to enable pre- cise complexity measurement and ensuring its consistency across the DoW. But we have a blueprint. HAWKINS is a computer scientist with U.S. Army Manager for Command and Control Data and Artificial Intelligence. He holds an M.S. in Library and Information Science from Drexel University and is currently a student in the In- formation Sciences Ph.D. program at the Na- val Postgraduate School (NPS). His published work includes research papers and editorials focused on the intersection between people and technology. HOUSEL, a tenured professor of Information Sciences at NPS, specializes in valuing intel- lectual capital, knowledge management, and IT. With a Ph.D. from the University of Utah, his research, published in journals such as MIS Quarterly , also includes behavioral account- ing. He was previously an associate profes- sor at the University of Southern California and Chief of Consumer Market Research for Telecom Italia, and his value research was featured in a Fortune magazine article. The author can be contacted at jeffrey.t.hawkins10@army.mil . The views expressed in this article are those of the authors and do not necessarily reflect the official policy of the Department of the Army, Department of War, or the U.S. Government. Reproduction or re- posting of articles from Defense Acquisition maga- zine should credit the authors and the magazine.
Related Resources • Innovate to Win Playbook (Online Tool) • EXE 4050V Leading Change to Drive Innovative Culture (Virtual Instructor-Led Training) • Scaling Innovation (Video) • Driving Innovation in Acquisition (Web Event Series)
16 | DEFENSE ACQUISITION | November-December 2025
2025
WINNER
Gold Award FOR WEBSITE IN THE CATEGORY OF ONLINE PUBLICATION
EMILY ASHCOM, BENJAMIN TYREE, NICOLE BRATE, SUSAN MUTH, COLLIE JOHNSON, and VISUAL ARTS AND PRESS STAFF
WARFIGHTING ACQUISITION UNIVERSITY
Revolutionizing Planning AND OPERATIONS WITH AI by MAJ CALEB “CJ” WERNER, USSF
18 | DEFENSE ACQUISITION | November-December 2025
W ith the advent of easily accessible and user friendly artificial intelligence (AI) technolo- gies such as large language models (LLMs), future human/computer teaming can prove critical in maintaining the technological edge of the United States Space Force (USSF) in the increasingly contested domain of cyber and space. Despite initial challenges, development of AI-driven workflows significantly im- proved efficiency and planning outputs within the Space Systems Command (SSC).
The domain of space has become increasingly critical to national secu- rity. Maintaining space superiority requires not only advanced technol- ogy but also the ability to acquire, integrate, and deploy these systems with unprecedented speed and effi- ciency. SSC, as the acquisition arm of the USSF, shoulders the responsibility of equipping the nation for this chal- lenge. To address this, a fundamental shift in how the USSF approaches ac- quisition is required to deal with a rap- idly evolving technological landscape, expanding mission requirements, and constant pressure to deliver cutting- edge capabilities within tight budget and time constraints. Traditional acquisition processes, often burdened by manual tasks, bureaucratic hurdles, and informa- tion silos, struggle to keep pace with the demands of the current environ- ment. The sheer volume of data, the intricate interdependencies of space systems, and the need for rapid deci- sion-making necessitate a more agile and innovative approach. AI offers the
potential to automate routine tasks, analyze vast datasets, enhance col- laboration, and ultimately accelerate delivery of critical space capabilities. Among the various AI tools ex- plored within the warfighting sec- tor, NIPRGPT (Non-Classified In- ternet Protocol Router Generative Pre-trained Transformer), a language model capability developed by the Air Force Research Laboratory, has emerged as a promising solution for streamlining planning and operational processes. One particular experience highlights the power of AI to overcome seemingly insurmountable obstacles. Discovering NIPRGPT’s Power Prior to my own “aha” pivotal ex- perience, and because of a variety of factors, I didn’t fully recognize the po- tential of AI tools like NIPRGPT when I first started using them. Overwhelmed by the daily deluge of tasks and lack- ing any guidance or practical experi- ence, I felt that exploring AI seemed an unaffordable luxury and another
[A] fundamental shift in how the USSF approaches acquisition is required to deal with a rapidly evolving technological landscape, expanding mission requirements, and constant pressure to deliver cutting-edge capabilities within tight budget and time constraints.
November-December 2025 | DEFENSE ACQUISITION | 19
week to complete were now accom- plished in just one or two hours by a single person. Moreover, the quality of the team’s planning outputs improved dramatically. The S.M.A.R.T. goals set for the team became more cohesive, comprehensive, and clearly defined and reflected deeper strategic intent. This newfound efficiency and en- hanced quality were not just incre- mental improvements; they repre- sented a paradigm shift in how the team approached work. It was clear that AI, wielded effectively, could be a game-changer for SSC. However, this initial success was just the be- ginning. Integrating AI into a large organization like SSC presented its own unique challenges. Navigating AI Adoption Integrating this new technology into established workflows and or- ganizational cultures was a complex undertaking. The following key chal- lenges highlight the complexities of widespread AI implementation. The learning curve. Mastering NIPRGPT, like any new technology, required an investment of time and ef- fort. Fortunately, the Warfighting Ac- quisition University offered resources such as free online courses taught by leading AI experts on platforms like Coursera. These courses provided a foundation in AI concepts and tech- niques, including prompt engineer- ing and risks and biases. Initially, the focus on using LLMs was for relatively simple tasks, such as refining written work and generating ideas—treating AI more as a sounding board or re- viewer than a content creation tool. The biggest initial hurdle was simply knowing where to start. Ironically, this was overcome by asking NIPRGPT it- self how it could best assist, which proved surprisingly effective in iden- tifying potential use cases and maxi- mizing its utility. Addressing awareness and ac- cess. Unlike with other emerging technologies, skepticism about NI- PRGPT wasn’t the primary barrier to adoption. Instead, the biggest chal-
complex tool for which there never seemed enough time to learn. That all changed during a post- Thanksgiving lull. In standing up a new USSF program, the current task at hand was to manually create countless Jira (planning tool) tickets to translate the team’s strategic road- map into actionable steps. The sheer volume of work was overwhelming. And in a moment of near despera- tion, NIPRGPT was used to explore untapped potential. Feeding the program’s strategic plan into the system involved using prompt engineering to generate Jira tickets complete with descriptions, definitions of done, and acceptance criteria. Essentially, the system was asked to transform the strategic vision into S.M.A.R.T. (Specific, Measurable, Actionable, Relevant, Time bound) goals. The results were a revelation (a tailorable prompt is included with this article). The output that materialized was like getting a glimpse into The Matrix , a 1999 movie where everyone has the individual responsibility to make the choice between the real world and an artificial world. A sudden understand- ing of the AI’s vast untapped potential became impossible to discount. The
implications are vast, as making this capability available to all Guardians will revolutionize USSF workflows and processes. But with the opportunities come risks. Multiple dangers must be navigated: These include halluci- nations (where AI “makes up facts”) and biases (toward agreeing with the one using AI or those who trained it). It was evident that AI literacy courses are essential requirements for proper AI management and implementa- tion when the larger workforce starts adopting it. The following two weeks became a whirlwind of exploration and experi- mentation. Focusing on integrating NIPRGPT into existing workflows, this LLM was able to generate not only Jira tickets but also code for automating various tasks. There were hurdles as the systems used weren’t designed for seamless AI integration, requiring a “human in the loop” to manually debug and re- fine LLM-generated code using tools like WordPad and Jira. Despite the manual effort involved, this hands- on experience provided invaluable insights into how LLMs could enhance and automate even the most complex processes. Tasks that previously re- quired a team of two or three a full
20 | DEFENSE ACQUISITION | November-December 2025
A Vision For the Future The initial successes and chal- lenges experienced with NIPRGPT illuminated a path toward a future where AI can empower SSC opera- tions with unprecedented speed, ef- ficiency, and strategic foresight. This vision extends beyond simply auto- mating individual tasks; it encom- passes a fundamental shift in how the USSF acquires, develops, and deploys space capabilities. It also addresses a critical underlying issue: the lack of standardized processes within many areas of space acquisition. Leveraging NIPRGPT, a defined workflow was created, complete with supporting artifacts like Excel templates and tailored instructions for different stakeholder groups. This demonstrates how AI can not only automate existing tasks but also fa- cilitate the creation of standardized processes where they are lacking, bringing much-needed structure and efficiency to complex projects. Be- yond process development, AI can transform core acquisition functions. Imagine AI-powered tools that op- timize satellite design parameters based on mission requirements and real-time data, automate launch
lenge was simply awareness and ac- cess. Many colleagues were unaware of NIPRGPT or how to access it for work. Furthermore, security restric- tions on the Air Force network pre- vented access to public LLMs like ChatGPT, limiting exposure to broader AI tools. This lack of awareness and access required proactive communi- cation and training. In the intervening months since these initiatives were taken, many more tools have come online as well as campaigns to in- crease workforce awareness. The security imperative. One of the greater advantages of NIPRGPT is its inherent security. Residing on government networks and developed by the government, it offers a level of data protection and confidential- ity that public LLMs simply cannot match. This is of paramount impor- tance within USSF and the DoW, where safeguarding sensitive infor- mation is crucial. Furthermore, hav- ing access to the source code allows for customization and deployment on secure internal systems, optimizing NIPRGPT for specific-use cases, and mitigating the risks associated with external platforms. The debugging dilemma. Integrat- ing NIPRGPT with existing workflows, particularly within Jira, presented a significant technical challenge. While NIPRGPT could generate code for au- tomating tasks like ticket creation, the process was far from seamless. The generated code often required exten- sive debugging due to inconsisten- cies between NIPRGPT’s output and the intricacies of Jira’s import func- tion, which was further complicated by Jira’s highly customizable nature and frequent updates. This debug- ging process, often using simple tools like WordPad, could be surprisingly time-consuming, sometimes negat- ing the time-saving benefits of using the LLM in the first place, especially when striving for perfect code. This highlighted the need for tighter inte- gration and more robust automation tools to fully realize the potential of AI-driven workflows.
scheduling by analyzing orbital dy- namics and resource availability, and streamline supply chain management by predicting and mitigating potential disruptions. This future is not about replacing humans with machines but about fos- tering a powerful synergy. The Infor- mation Operations Networking (ION) program provides a compelling exam- ple. A key challenge was the difficulty in staffing and training Model-Based Systems Engineering modelers. An innovative solution emerged: funding an AI-driven modeling effort. This ap- proach not only augments the capa- bilities of existing modelers but cre- ates opportunities for reskilling and retraining. Modelers can transition into AI model managers, overseeing and directing the AI while contributing their domain expertise. This not only increases efficiency but also allows those with modeling skills to contrib- ute to other critical areas within the organization, maximizing utilization of valuable talent. This exemplifies how human-machine teaming can enhance productivity and create new, more fulfilling roles within SSC. SSC recognizes the importance of responsible AI development and
Beyond process development, AI can transform core acquisition functions. Imagine AI-powered tools that optimize satellite design parameters based on mission requirements and real-time data, automate launch scheduling by analyzing orbital dynamics and resource availability, and streamline supply chain management by predicting and mitigating potential disruptions.
November-December 2025 | DEFENSE ACQUISITION | 21
Page 1 Page 2 Page 3 Page 4 Page 5 Page 6 Page 7 Page 8 Page 9 Page 10 Page 11 Page 12 Page 13 Page 14 Page 15 Page 16 Page 17 Page 18 Page 19 Page 20 Page 21 Page 22 Page 23 Page 24 Page 25 Page 26 Page 27 Page 28 Page 29 Page 30 Page 31 Page 32 Page 33 Page 34 Page 35 Page 36 Page 37 Page 38 Page 39 Page 40 Page 41 Page 42 Page 43 Page 44 Page 45 Page 46 Page 47 Page 48 Page 49 Page 50 Page 51 Page 52 Page 53 Page 54 Page 55 Page 56 Page 57 Page 58Made with FlippingBook - Online Brochure Maker