Defense Acquisition Research Journal #108

Defense Acquisition Research Journal Leading Change The Defense Acquisition Review Journal (ARJ) is a scholarly peer-reviewed journal published by the Defense Acquisition University (DAU). All submissions receive a blind review to ensure impartial evaluation. Articles represent the views of the authors and do not necessarily reflect the opinion of the DAU or the Department of Defense. October 2024 Vol. 31 No. 2 | Issue 107 Journal DOI: 10.22594/dau.042023-103.30.02 Elect ISSN: 2156-8405

ARTICLE LIST

Distribution Comparisons of EAC Cost Growth for Aircraft Work Breakdown Structure Elements Capt Kyle P. Marquis, USAF, Edward D. White, Brandon M. Lucas, Robert D. Fass, Jonathan D. Ritschel, and Shawn M. Valentine Commercial Solutions Opening: An Innovative, Competitive Process to Solve Slow Government Procurement Eric W. Washburn, Mary Beth Colavito, CDR E. Cory Yoder, USN (Ret.), and Rene G. Rendon

Analyzing Stability of Estimates at Completion for Long Duration Development Efforts Bradley Vuu, Jonathan D. Ritschel, Brandon M. Lucas, and Edward D. White

BOOK REVIEWS

Blockchain Technology: Exploring Opportunities, Challenges, and Applications Written by Sonali Vyas, Shaurya Gupta,

and Vinod Kumar Shukla Reviewed by Bob Natale

Military Culture Shift: The Impact of War, Money, and Generational Perspective on Morale,

Retention, and Leadership Written by Corie Weathers, LPC Reviewed by LTC Jess Rankin, USA (Ret.)

Securing the MRAP: Lessons Learned in Marketing and Military Procurement

Written by James M. Hasik Reviewed by Ronald Hicks

We’re on the Web at: http://www.dau.edu/library/arj

Mr. Steven J. Morani Performing the Duties of Under Secretary of Defense for Acquisition and Sustainment Ms. Bilyana Anderson President, DAU

Ms. Angela Carsten Chief of Staff, DAU Mr. Dom Dionne Director, DAU Operations Support Group

Editorial Board Dr. Larrie D. Ferreiro Chairman and Executive Editor

Dr. Michelle Bailey Catholic University of America Dr. Don Birchler CNA Dr. John M. Colombi Air Force Institute of Technology Dr. Cynthia R. Cook Center for Strategic and International Studies Dr. Bobbie G. DeLeon DAU Dr. William T. Eliason Eisenhower School for National Security and Resource Strategy Dr. J. Scott Frampton Eisenhower School for National Security and Resource Strategy Dr. Steve King MITRE Mr. Jeffrey R. LaFleur Eisenhower School for National Security and Resource Strategy

Mr. David H. Lewis Naval Postgraduate School Mr. William Lucyshyn University of Maryland Dr. Thomas A. Mazzuchi The George Washington University Dr. John McCormack Cardiff Metropolitan University (UK) Dr. John G. McGinn

Dr. Robert "Bobby" Ralston Eisenhower School for National Security and Resource Strategy Dr. Yvette Rodriguez DAU Dr. Dana Stewart DAU Dr. David M. Tate Institute for Defense Analyses Dr. Trevor Taylor Royal United Services Institute (UK) Dr. Marina Theodotou Defense Innovation Board Office of the Under Secretary of Defense for Research and Engineering Mr. Jerry Vandewiele DAU

George Mason University Dr. Georgella McRae DAU Mr. Patrick Morrow DAU Dr. Robert F. Mortlock Naval Postgraduate School Dr. Troy J. Mueller MITRE Dr. Christopher G. Pernin RAND

ISSN 2156-8391 (print) and ISSN 2156-8405 (online) DOI: https://doi.org/10.22594/dau.042025-108.32.01

The Defense Acquisition Research Journal , formerly the Defense Acquisition Review Journal , is published quarterly by the DAU Press and is an official publication of the Department of Defense. Postage is paid at the U.S. Postal facility, Fort Belvoir, VA, and at additional U.S. Postal facilities. Postmaster, send address changes to: Editor, Defense Acquisition Research Journal , DAU Press, 9820 Belvoir Road, Suite 3, Fort Belvoir, VA 22060-5565. The journal-level DOI is: https://doi.org/10.22594/dauARJ.issn.2156-8391. Some photos appearing in this publication may be digitally enhanced. Photos in this publication may have been sourced from the Department of Defense website (https://www.defense.gov/Multimedia/Photos/). The appearance of the U.S. Department of Defense visual information does not imply or constitute Department of Defense endorsement. Some images may be digitally enhanced. The views expressed in the Defense Acquisition Research Journal are those of the author(s) alone and not of the Department of Defense. Reproduction or reposting of articles from the Defense Acquisition Research Journal should credit the author(s) and the journal.

Chief of Visual Arts & Press Norene L. Johnson Interim Managing Editor Dr. Olena McLaughlin Assistant Editor Christopher McGowan Art Director, DAU Press Michael Krukowski Graphic Designer Paul Kim Production Manager Frances Battle Graphic Designer, Digital Publications Nina Austin Technical Editor Collie J. Johnson Copy Editor Michelle McDonald

Editorial Support Emily Ashcom

Commercial Solutions Opening: An Innovative, Competitive Process to Solve Slow

Government Procurement Eric W. Washburn, Mary Beth Colavito, CDR E. Cory Yoder, USN (Ret.), and Rene G. Rendon

This research identifies the strengths, weaknesses, and best practices of the Commercial Solutions Opening (CSO) as a solicitation technique leading to contract award. It further details how CSO was first legislated in 2016 and is now a rapidly evolving competitive solicitation technique to achieve innovation throughout the defense contracting community.

Distribution Comparisons of EAC Cost Growth for Aircraft Work Breakdown Structure Elements Capt Kyle P. Marquis, USAF, Edward D. White, Brandon M. Lucas, Robert D. Fass, Jonathan D. Ritschel, and Shawn M. Valentine In this study, the authors analyze probability distributional fits to Estimate at Completion cost growth for various Work Breakdown Structure elements. Results suggest either the lognormal or Weibull as best fits and a possible underestimating of the Coefficients of Variance and 85th percentiles in a program’s budget. Analyzing Stability of Estimates at Completion for Long Duration Development Efforts Bradley Vuu, Jonathan D. Ritschel, Brandon M. Lucas, and Edward D. White The article provides empirical evidence that long duration programs behave differently than short duration programs. Program managers are cautioned to consider the length of duration when estimating cost at completion up to and slightly beyond the 70% completion point.

From the Chairman and Executive Editor Dr. Larrie D. Ferreiro

viii

Defense ARJ Staff Welcomes New DAU President Bilyana Anderson Defense Acquisition Research Journal (ARJ) Staff Call for Authors We are currently soliciting articles for the 2025 and 2026 Defense ARJ print years. Commercial Solutions Opening: An Innovative, Competitive Process to Solve Slow Government Procurement Eric W. Washburn, Mary Beth Colavito, CDR E. Cory Yoder, USN (Ret.), and Rene G. Rendon Distribution Comparisons of EAC Cost Growth for Aircraft Work Breakdown Structure Elements Capt Kyle P. Marquis, USAF, Edward D. White, Brandon M. Lucas, Robert D. Fass, Jonathan D. Ritschel, and Shawn M. Valentine Analyzing Stability of Estimates at Completion for Long Duration Development Efforts Bradley Vuu, Jonathan D. Ritschel, Brandon M. Lucas, and Edward D. White Professional Reading List Blockchain Technology: Exploring Opportunities, Challenges, and Applications Written by Sonali Vyas, Shaurya Gupta, and Vinod Kumar Shukla Reviewed by Bob Natale Military Culture Shift: The Impact of War, Money, and Generational Perspective on Morale, Retention, and Leadership Written by Corie Weathers, LPC

xii

Reviewed by LTC Jess Rankin, USA (Ret.) Securing the MRAP: Lessons Learned in Marketing and Military Procurement

Written by James M. Hasik Reviewed by Ronald Hicks

Current Research Resources in Defense Acquisition A selection of new research curated by the DAU Research Center and the DAU Virtual Research Library

84 92 99

Defense ARJ Guidelines for Contributors

Statement of Ownership

Recognition of Reviewers 2024 We would like to express our appreciation to all of the subject matter experts who volunteered to participate in the Defense ARJ peer review process.

100

FROM THE CHAIRMAN AND EXECUTIVE

EDITOR Dr. Larrie D. Ferreiro

The theme for this issue is “Informed Decision-Making.” Defense acquisition often involves inherently uncertain processes when developing new platforms, creating novel technologies, or navigating unique procurement strategies. The three articles in this issue, as well as the books reviewed, all address the processes that inform decision- making at every level. The first article is “Commercial Solutions Opening: An Innovative, Competitive Process

to Solve Slow Government Procurement” by Eric W. Washburn, Mary Beth Colavito, E. Cory Yoder, and Rene G. Rendon. The authors identify the strengths, weaknesses, and best practices of the Commercial Solutions Opening (CSO) as a solicitation technique leading to contract award. They further detail how CSO was first legislated in 2016 and is now a rapidly evolving competitive solicitation technique to achieve innovation throughout the defense contracting community. The second article, by Kyle P. Marquis, Edward D. White, Brandon M. Lucas, Robert D. Fass, Jonathan D. Ritschel, and Shawn M. Valentine, is titled “Distribution Comparisons of EAC Cost Growth for Aircraft Work Breakdown Structure Elements.” In this study, the authors analyze probability distributional fits to Estimate at Completion (EAC) cost growth for various Work Breakdown Structure elements, using over 60 aircraft programs as their basis. Their analysis shows that either

viii

lognormal or Weibull distributions provide a relatively good fit to EAC cost growth in budgets. The third article is “Analyzing Stability of Estimates at Completion for Long Duration Development Efforts” by Bradley Vuu, Jonathan D. Ritschel, Brandon M. Lucas, and Edward D. White. The authors of the article provide empirical evidence that long duration programs behave differently than short duration programs. They caution program managers to consider the length of duration when estimating cost at completion up to and slightly beyond the 70% completion point, after which cost deviation tends to stabilize. This issue’s Current Research Resources in Defense Acquisition is Business Acumen. The Defense Acquisition Reading List features the following book reviews: Military Culture Shift: The Impact of War, Money, and Generational Perspective on Morale, Retention, and Leadership by Corie Weathers; reviewed by Jess Rankin Blockchain Technology: Exploring Opportunities, Challenges, and Applications by Sonali Vyas, Shaurya Gupta, and Vinod Kumar Shukla; reviewed by Bob Natale Securing the MRAP: Lessons Learned in Marketing and Military Procurement by James M. Hasik; reviewed by Ronald Hicks. Kevin Buck, David Gallup, John Cannaday, and John Snoderly have left the Editorial Board. We thank them for their long and devoted service. We sadly bid a belated farewell for the passing of RADM James Greene, USN (Ret.). We welcome to the Editorial Board our new members: Dr. Bobbie G. DeLeon (DAU); Dr. J. Scott Frampton (Eisenhower School for National Security and Resource Strategy); Mr. Jeffrey R. LaFleur (Eisenhower School for National Security and Resource Strategy); and Dr. Steve King (MITRE).

Source: Photos by Michael Krukowski

DEFENSE ARJ STAFF WELCOMES NEW DAU PRESIDENT BILYANA ANDERSON Defense ARJ Staff

The entire staff of the Defense Acquisition Research Journal welcomes our new DAU President Bilyana Anderson. As she shared her vision for DAU at the DAU Presidential Welcome ceremony, Anderson emphasized that “We have a huge responsibility to the nation to ensure our acquisition cadre is ready.” She stated that she intends to approach leading DAU from a program manager’s perspective with a spirit of collaboration and Warfighter needs as DAU’s top priority.

Anderson has previously served as Deputy Assistant Secretary of the Navy for Ship Programs; the executive director, Program Executive Office, Integrated Warfare Systems within the Naval Sea Systems Command; the executive director for Combatants in the Program Executive Office Ships (PEO Ships); executive director for Surface Warfare; and several other senior leadership positions. Anderson holds a Bachelor of Science in Industrial Engineering from Tennessee Technological University and an MA in Public Administration from Virginia Polytechnic Institute and State University. She is DAWIA certified in Program Management; Systems Planning, Research, Development, and Engineering; and Business Financial Management. Welcome, President Anderson!

CALL FOR AUTHORS We are currently soliciting articles for the 2025 and 2026 Defense Acquisition Research Journal (Defense ARJ) print years. We welcome submissions describing original research or case histories from all acquisition career fields and phases of the acquisition life cycle—the conceptualization, innovation, initiation, design, testing, contracting, production, deployment, logistics support, modification, and disposal of weapons and other systems, supplies, or services (including construction) needed by the DoD or intended for use to support military missions. Articles of 5,000 or fewer words appear in both print and digital publications. Articles of 5,000 to 10,000 words will be considered for digital publication only. All manuscript submissions are peer reviewed.

xii

All submissions should include the following items:

• Cover letter • Manuscript • Figures and tables • Biographical sketch for each author • Headshot for each author Benefits of publishing

• Share your research results with the defense acquisition community. • Change the way DoD does business. • Become a nationally recognized expert in your field or specialty. • Be invited to speak at a conference or seminar. • Earn up to 40 continuous learning points. For more information, contact the Defense ARJ managing editor (DefenseARJ@dau.edu) and check out our Guidelines for Contributors at https://www.dau.edu/library/arj/defense-arj- submissions

xiii

We’re on the Web at: http://www.dau.edu/library/arj

ISSUE 108 SPRING 2025 VOL. 32 NO. 1

Commercial Solutions Opening

COMMERCIAL SOLUTIONS OPENING:

AN INNOVATIVE, COMPETITIVE PROCESS TO SOLVE SLOW GOVERNMENT PROCUREMENT

Defense ARJ , Spring 2025, Vol. 32 No. 1

https://www.dau.edu

Eric W. Washburn, Mary Beth Colavito, CDR E. Cory Yoder, USN (Ret.), and Rene G. Rendon

The primary purpose of this research is to identify the strengths, weaknesses, and best practices of the Commercial Solutions Opening (CSO) as a solicitation technique leading to a contract award. With their findings, the research team intends to provide DoD Components, Defense Agencies, 4th Estate organizations, and their associated workforces with a consolidated report analyzing available data on the CSO solicitation technique and making recommendations based on the use of CSOs. Working with these DoD-affiliated organizations, the researchers captured extensive direct feedback from CSO Cross Talk meetings among DoD points of contact who previously conducted CSOs or are working to develop CSO policies/procedures at their individual organizations. The DoD and other agencies outside of the department will also be able to use this research to capitalize on the utility of CSOs in requesting their own permanent authority. Further, the research provides an analysis to shape informed decision-making for future solicitation strategies as future requirements owners and contracting offices develop their plans to meet agency needs. Finally, this research can be used as a catalyst to refine CSO reporting requirements, bolstering the data value stream for the department’s executive decision-makers.

DOI: https://doi.org/10.22594/dau.24-922.32.01 Keywords: Commercial Solutions Opening, acquisition, contracting, solicitation, innovation.

Defense ARJ , Spring 2025, Vol. 32 No. 1

Commercial Solutions Opening

The DoD works tirelessly to achieve rapid acquisition objectives and narrow the strategic and defense capabilities gap between the United States and near-peer adversaries. A quick review of past newspaper headlines reveals the significant disparity in capability between DoD’s industrial base and those of U.S. adversaries. For example, China’s defense industrial base seems to have a much faster and more streamlined path for developing and fielding new weapons (Tirpak, 2023). Additionally, many of the most sought-after contractors in the DoD are not interested in pursuing DoD contracts, but instead are “finding much more lucrative and favorable markets in places like China and Russia” (National Contract Management Association [NCMA], 2019, p. 5). Thus, the DoD recognizes the need to shorten the time from proposal to contract award and make the process “faster, more agile and responsive to the pace of change” (NCMA, 2019). Recent notable examples of this acquisition reform include the Middle Tier of Acquisition (MTA) Pathway for Rapid Prototyping and Rapid Fielding authorized by Section 804 of the National Defense Authorization Act (NDAA) for Fiscal Year 2016 (NDAA, 2015), awareness of Other Transaction (OT) Authority, and the adoption of industry standards in acquisition. Even with these reforms, the DoD acquisition process remains slow, expensive, and bureaucratic. In 2021, and in furtherance of rapid acquisition objectives, the U.S. Congress codified Public Law 117-81, the NDAA for FY 2022 (NDAA, 2021). Section 803 of the Act provides permanent authority for a new type of rapid acquisition, the Commercial Solutions Opening (CSO). The CSO is a solicitation technique that is designed as an innovative means to solve the problem of slow government procurement. At its core, the CSO seeks to take a broadly identified objective, stated in a manner that allows for diverse solutions, and award a contract to meet those objectives within a matter of weeks, as opposed to the methods that now take months or even years using traditional models. A CSO can result in both Federal Acquisition Regulation (FAR)-based and non-FAR-based contracts and is used to acquire an innovative technology or an innovative means or method to accomplish the objective. As an example of a CSO, Joint Base McGuire-Dix-Lakenhurst (MDL) accepted an innovation challenge from former Secretary of the Air Force Heather Wilson and Air Force Chief of Staff GEN David L. Goldfein to “Think Big, Start Small, and Scale Fast” with its first-ever Joint Base MDL Pitch Day event. At this event, 10 small businesses pitched their

Defense ARJ , Spring 2025, Vol. 32 No. 1

https://www.dau.edu

innovative technology to Joint Base MDL leaders, and five of them walked away with a one-page contract in hand and an initial payment in the bank (Golden, 2019). The primary purpose of this research is to identify the strengths, weaknesses, and best practices of the CSO as a solicitation technique leading to a contract award. Our work is a combination of intellectual and practical action research. Its purpose is to provide DoD organizations and their workforces with a consolidated report analyzing available data on the CSO solicitation technique and making recommendations based on the use of CSOs. This article is based on the graduate thesis by co-authors Eric W. Washburn and Mary Beth Colavito (Washburn & Colavito, 2023).

Assessment of different individuals’ varied interpretation and implementation of the flexible process to meet their specific program and agency goals will inform the categorization of strengths, weaknesses, and best practices.

We use an action research design, based on a qualitative approach, using an observation data collection method with document analysis of the CSO Cross Talks (George, 2024). Our research methodology will consist of extensive direct feedback captured from CSO Cross Talk meetings among Defense Agency points of contact who have previously conducted CSOs or are working to develop CSO policies/procedures at their individual agencies. These feedback meetings are led by the Deputy Assistant Secretary of the Air Force for Acquisition (Contracting; SAF/ AQC). This information will be reviewed for strengths and weaknesses regarding training and information sharing, internal agency processes, solicitation definition, and industry interaction. Assessment of different individuals’ varied interpretation and implementation of the flexible process to meet their specific program and agency goals will inform the categorization of strengths, weaknesses, and best practices. Similar direct user feedback will be discussed as compiled for and documented in other published briefings and reports. Our final results will be presented in the form of recommendations that DoD and its contracting offices can best leverage to implement CSOs.

Defense ARJ , Spring 2025, Vol. 32 No. 1

Commercial Solutions Opening

Literature Review

Innovation Theory The previous section established that CSOs present an opportunity for the DoD to make critical investments in technology and capability by leveraging the technological capabilities of the department’s industrial base. In fact, the adoption of CSOs as a permanent authority is, in itself, innovative. To understand how these innovative capabilities can shape the DoD, it is important to understand the theory supporting innovation in business. First, we considered the DoD as a type of knowledge management (KM) firm with “roles and processes to support decision- making” (Joint Chiefs of Staff, 2018, p. 1). The DoD as a KM firm is comprised of individuals with tacit, explicit, and implicit knowledge of the military’s operations, from munitions flight trajectories to the ideal length of a blade of grass along a flightline. Within this construct, the DoD is operating as a firm competing with other nations; this defines the marketplace within which innovation leads to competitive advantage and provides a framework against which innovation theory can be applied.

The DoD is operating as a firm competing with other nations; this defines the marketplace within which innovation leads to competitive advantage and provides a framework against which innovation theory can be applied.

Considering the DoD as a type of KM firm, the research team applied an assertion by Johannessen et al. (1999) that the “unending stream of knowledge keeping markets in perpetual motion, calls for companies to execute continuous improvements and continuous innovation, while simultaneously limiting imitation” (p. 122). Further, Johannessen et al. (1999) assert that “certain firms have more information than others, and turning this into knowledge gives them an advantage in ascertaining market inefficiencies, putting them in a better position to innovate” (p. 123). To truly capitalize on the benefits of CSOs, the DoD must consider itself as operating in a KM environment. KM is a key enabler in identifying problems and solutions and paving the way for innovation to occur. Conceptualizing and managing change through exploiting the learning

Defense ARJ , Spring 2025, Vol. 32 No. 1

https://www.dau.edu

capacity of knowledge workers is considered a competitive advantage according to Nonaka (2007). Individuals hold the ideas and knowledge necessary for the creation of new products and services as well as the ability to add value to old ideas and concepts (Seagal & Horne, 1997). Innovation theory states that a manager’s role in the knowledge-based industry is to “manage the environment or context in which work is done” (Johannessen et al., 1999, p. 132). For the DoD, this management comes from the program manager (for defense acquisition programs) and the functional services manager (for service acquisitions), with assistance from members of the acquisition team. The team leverages their expertise to achieve positive results in productivity, fostering innovation, and bolstering military capability while leading a team of government and contractor innovators. The winning manager provides “their people with the best weapons with which to compete, i.e., knowledge and service” (Johannessen et al., 1999, p. 132). The findings of this research will enable knowledge managers in the DoD to integrate the results into their own KM stream, fully capitalizing on the ability to achieve innovative solutions through the CSO process. Commercial Solutions Opening Legislative History, Policy, and Procedures With a basis of innovation theory and before delving into the FAR processes and other acquisition flexibilities that broadly led to the creation of the CSO, it is important to define its immediate history, policy, and procedures. On June 26, 2018, Class Deviation 2018-O0016, Defense Commercial Solutions Opening Pilot Program, was published allowing contracting officers to “acquire innovative commercial items, technologies, or services using a competitive procedure called a CSO” under the authority of Section 879 of the NDAA for FY 2017 (Assad, 2018). This authority was set to expire on September 30, 2022; however, less than 4 years later, on February 4, 2022, Class Deviation 2022-O0007, Defense Commercial Solutions Opening, rescinded and superseded the previous class deviation to give the CSO permanent authority by Section 803 of the NDAA FY 2022 (Tenaglia, 2022). Minimal procedures are required when a contracting officer chooses to utilize a CSO under this class deviation. However, key operational aspects germane to this research are provided in Figure 1.

Defense ARJ , Spring 2025, Vol. 32 No. 1

Commercial Solutions Opening

FIGURE 1. CSO DEVIATION KEY ASPECTS

y Treat items, technologies, and services as commercial y May use only — y To obtain solutions or potential capabilities that - fulfill requirements - close capability gaps, or

- provide potential technological advancements that are new as of the date of submission of a proposal or that are a new application as of the date of submission of a proposal of a technology, process, or method existing as of such date; y When meaningful proposals with varying technical or scientific approaches can be reasonably anticipated; and y When the contract entered into under the program will be fixed-price, including fixed-price incentive contracts. y May competitively select proposals received in response to a general solicitation, similar to a broad agency announcement. y CSO is considered to be a competitive procedure. y Primary evaluation factors shall be y technical, y importance to agency programs, y funds availability. y Price considered to the extent appropriate to determine fair and reasonable. y Written evaluation reports on individual proposals are required. y Proposals not evaluated against each other.

Restrictions for use

Competition

Evaluation for Award

Note. Adapted from “Class Deviation—Defense Commercial Solutions Opening” [Memorandum], by J. M. Tenaglia, 2022, Office of the Under Secretary of Defense for, Acquisition and Sustainment. https://www.acq.osd.mil/dpap/policy/policyvault/USA000138-22- DPC.pdf In January 2024, the Defense Federal Acquisition Regulation Supplement (DFARS) was revised to include subpart 212.70, Defense Commercial Solutions Opening. This statute implements 10 U.S.C. § 3458 for the acquisition of innovative commercial products or commercial services through the use of a general solicitation known as a CSO (DFARS, 2024). Beyond the relatively minimal guidance/instruction, the mechanics of utilizing a CSO are left up to the interpretation of the various DoD organizations and individual contracting officers. As such, organizations varied in their implementation of guidance and additional policies for CSOs. Identification of Strengths, Weaknesses, and Best Practices With strengths, weaknesses, and best practices at the core of this research and its primary questions, it is important to define those terms. A strength indicates an aspect of the CSO solicitation technique that has benefited the government, industry, or both. Examples could include an easier process to contract award than FAR-based procedures, reduced risk of protest, contracts for more innovative solutions than the government could have defined in a requirements statement, and so on.

Defense ARJ , Spring 2025, Vol. 32 No. 1

https://www.dau.edu

A weakness would indicate an aspect of the CSO that has hindered the government, industry, or both. Examples could include a more confusing process than FAR-based procedures, difficulty in securing a fair and reasonable price for the government, or uncertainty about how to award a follow-on contract to an initially innovative solution contract, etc. An observation may have attributes that result in both a strength and weakness. A best practice is defined by Merriam-Webster (n.d.) as “a procedure that has been shown by research and experience to produce optimal results and that is established or proposed as a standard suitable for widespread adoption.” Examples could include publishing an agency-specific CSO guidebook, using a gated/phased approach for CSO proposal submissions, or advertising CSOs through unconventional means. Not all observations may qualify as a strength, weakness, or best practice but still enhance or contribute to this research or areas for future research; those observations will be captured as “other observations” in the Findings section of this article. Research Methodology The DoD started holding CSO Cross Talk meetings quarterly in April 2022 as a forum for the contracting workforce to share “CSO policy changes, training, and success stories/best practices” (DoD, 2022). DoD agency points of contact who have previously conducted CSOs share a varied interpretation and implementation of the flexible solicitation technique to meet their specific program and agency goals. This is in an effort to benefit all those working to develop CSO policies/procedures at their individual agencies, whether they used them yet or not. Participants are encouraged to ask questions and suggest hot topics surrounding CSOs. SAF/AQC representatives organize and facilitate the meetings, and afterward they draft CSO Cross Talk Bulletins to summarize the meetings. These bulletins are disseminated with guidance for meeting attendees to share them among the acquisition workforce of each respective Defense Agency. For this research, we reviewed and analyzed the contents of these bulletins, which were primarily based on the feedback provided by Defense Agency points of contact who previously conducted CSOs, particularly regarding CSO strengths, weaknesses, and best practices.

Defense ARJ , Spring 2025, Vol. 32 No. 1

Commercial Solutions Opening

While a policy analyst or contracting officer may read these bulletins and simply capture mental notes for potential future use, our research will systematically break down all feedback data and categorize it by topic area. This research breakdown will readily lend itself to developing more strategic recommendations about actions that can be taken regarding CSOs. The four overarching categories are the following: 1. Training and Information Sharing —how the workforce is educated on this solicitation technique. 2. Internal Agency Processes —how individual DoD agencies structure their facilitation of evaluating and awarding CSOs. 3. Solicitation Definition —how various contracting officers draft individual CSOs. 4. Industry Interaction —how the government advertises to and receives information from potential offerors. These four categories are purposely broad to accommodate separation and subsequent categorization of a diverse range of feedback (the DoD agency points of contact were not required to structure their Cross Talk presentations in any particular way). We separated the feedback into these categories and then identified strengths, weaknesses, and best practices. Also, commonalities and focus areas for recommendations can be consolidated. Findings It may appear easy for one to predict potential strengths, weaknesses, and best practices of any new technique based on its developer’s intention or motivation, but having the firsthand experience to back up those findings and open oneself up to questions about them is another matter entirely. The CSO Cross Talks served as a forum for airing those findings starting in April 2022. As discussed in the Research Methodology section, we consolidated the feedback that follows from various agency representatives and then categorized it into broad categories to highlight possible focus areas on which to capitalize for recommendations. As previously discussed, we reviewed the CSO Cross Talks and developed categories under which to align our findings. The four categories, developed specifically for this research, follow.

Defense ARJ , Spring 2025, Vol. 32 No. 1

https://www.dau.edu

• Training and Information Sharing. This category covers observations related to how the workforce is educated on the CSO solicitation technique. The findings under this category are provided in Table 1. • Internal Agency Processes. The research team defined this category as how individual DoD agencies facilitate their evaluation and award of contracts because of the CSO solicitation technique. The findings under this category are provided in Table 2. • Solicitation Definition. This category lists findings about how various contracting officers draft individual CSO solicitations. The findings under this category are provided in Table 3. • Industry Interaction. The last category captures how the government advertises to and receives information from potential offerors under CSO solicitations. The findings under this category are provided in Table 4.

It may appear easy for one to predict potential strengths, weaknesses, and best practices of any new technique based on its developer’s intention or motivation, but having the firsthand experience to back up those findings and open oneself up to questions about them is another matter entirely.

Through this analysis, we captured the strengths, weaknesses, and best practices in Tables 1, 2, 3, and 4 using the identifiers “S”, “W”, and “BP”, respectively. The individual observations are not listed in priority order. Moreover, some findings indicate “BP” (followed by “S” or “W” in parentheses) to note that this is a best practice based on, or that resulted in, an observed strength or weakness.

Defense ARJ , Spring 2025, Vol. 32 No. 1

Commercial Solutions Opening

TABLE 1. CSO CROSS TALKS—TRAINING AND INFORMATION SHARING

Category

Findings

Contracting organizations should create a training team to do a deep dive into tactical processes for each CSO spiral, identify best practices, and target areas that have historically performed inconsistently (resulted in 3-month award time savings, helped mitigate protests, and expedited purchases). PCOs should understand that there are different challenges than a typical acquisition because solutions can vary widely (e.g., type of money needed, bona fide need, base spectrum approvals, Authorization to Operate requirement). DoD should stand up an Outreach Team to equip acquisition professionals with training, best practices, success stories, resources, and DoD-level and industry collaboration opportunities. DoD and contracting organizations should train on CSO policy/procedures to show how they differ from FAR-based acquisitions (e.g., know what processes/documents affect each contract from CSO level vs. individual contract level). DoD and contracting organizations should be educated on what authorities, regulations, and policies are available and how to differentiate among them.

BP (S)

BP (W)

Note. Adapted from DoD OT Quarterly Commercial Solutions Opening Cross Talk [Bulletin], Department of Defense (2022); PCOs = Procuring Contracting Officers; CSO = Commercial Solutions Opening.

TABLE 2. CSO CROSS TALKS—INTERNAL AGENCY PROCESSES

Category

Findings

Contracting organizations’ PCOs obligated awards competitively, within 60 days, and with substantial negotiated savings. Contracting organizations should assign a PCO to chair the execution team (a significant amount of confusion and rework reduced by establishing a PCO at the head of the evaluation and execution teams). PCOs did not observe awards from CSOs as a short process or end-of-year effort due to multiple workshops, time to develop problem statements, and acquisition process taking numerous months. Contracting organization observed a lack of accurate CSO data reporting for DoD as a whole. Contracting organization required a large team to evaluate over 500 submissions for different organizations in a reasonable amount of time. PCOs should ensure funding is ready to obligate from their program offices in order to quickly move to reduce or scale the requirement based on the available funding and then promptly award the contract (waited on funding for eight months in one instance). Contracting organizations and PCOs should establish a cloud-based document repository. The fast pace of the CSO process required an organized central repository for emails, documents, and spreadsheets that could be accessed by many and restricted as necessary. PCOs should decide how they will share documents with those that are not able to access the cloud-based document repository (e.g., contracted technical evaluators). PCOs should make sure all processes, procedures, and contractor responses under the CSO are uniform (also applies to Solicitation Definition section). PCOs should ensure acquisition/evaluation teams are filing electronic documents in a standardized manner. Continued on the next page

BP (S)

BP (W)

Defense ARJ , Spring 2025, Vol. 32 No. 1

https://www.dau.edu

TABLE 2. CSO CROSS TALKS—INTERNAL AGENCY PROCESSES (CONTINUED) Category Findings

PCOs should rely on program managers and technical specialists to frame Statements of Work and Contract Line Item Number (CLIN) structures. PCOs should work with Defense Contract Management Agency at the onset if they will be assisting with pre-award surveys or post-award administration. PCOs should work closely with legal support to avoid issues with a wide range of solutions.

PCOs should create/maintain relevant documentation.

PCOs should ensure subject matter experts (SMEs) perform robust technical evaluations. PCOs should negotiate price and terms and conditions bilaterally (after proposals are solicited competitively), in line with industry’s normal practices. Contracting organizations should plan appropriately to facilitate shorter procurement acquisition lead times (e.g., hiring/assigning additional personnel and deprioritizing the team’s other workload to award high-dollar requirements in a compressed timeline). Contracting organizations and PCOs should establish a central document repository for oversight and surveillance documents. Contracting organizations should establish a unified contracting division for CSO solicitations and awards. Contracting organizations and PCOs should integrate acquisition professionals and SMEs in acquisition planning and development.

PCOs should establish relationships among stakeholders.

PCOs should ensure consistent socialization and communication of timeline, plans, and processes.

Note. Adapted from DoD OT Quarterly Commercial Solutions Opening Cross Talk [Bulletin], Department of Defense (2022, 2023); PCOs = Procuring Contracting Officers; CSO = Commercial Solutions Opening.

TABLE 3. CSO CROSS TALKS—SOLICITATION DEFINITION

Category

Findings

Contracting organizations observed that every agency/subunit/etc., has its own unique requirements; even if another part of DoD has contracted for a certain product/service, it could be considered “new/innovative” to your specific part of DoD and warrant an award from a CSO. PCOs should draft CSOs with a phased approach (e.g., abstracts and oral presentations) to reduce the workload for both government and contractor. PCOs should provide enough background/contextual information for the problem statement or Area of Interest (AOI) in the CSO in order for offerors to construct a successful proposal. PCOs should standardize evaluation criteria across the AOI (each solution will still differ but can be assessed based on its unique ability to respond to the government requirement). PCOs should adjust scope and specificity of CSO language based on individual circumstances.

BP (S)

Note. Adapted from DoD OT Quarterly Commercial Solutions Opening Cross Talk [Bulletin], Department of Defense (2022, 2023); PCOs = Procuring Contracting Officers; CSO = Commercial Solutions Opening.

Defense ARJ , Spring 2025, Vol. 32 No. 1

Commercial Solutions Opening

TABLE 4. CSO CROSS TALKS—INDUSTRY INTERACTION

Category

Findings

Contracting organizations and PCOs should consider use of AFWERX Google Suite as a secure one-stop shop for correspondence, documentation, and meetings with vendors (proved to be an effective tool resulting in 6-month award time savings). PCOs should allow industry to have access to and communication with end users in a controlled way.

BP (S)

PCOs should survey industry for inputs into the CSO process.

Contracting organizations and PCOs should use social media and a wide variety of online options to reach potential offerors (some market segments, like cyber, will actively avoid resources that are too associated with the government). PCOs should encourage program managers to reach out through their contacts and colleagues for potential offerors.

BP Contracting organizations and PCOs should learn about the market segments they are trying to attract and how they typically find opportunities. Note. Adapted from DoD OT Quarterly Commercial Solutions Opening Cross Talk [Bulletin], Department of Defense (2022).

Implications of Findings Most of the listed CSO Cross Talk comments were categorized as best practices since the agency representatives primarily framed their feedback as subjective recommendations to other agencies. Objective strengths and weaknesses may have been few in number as a result of the noted lack of accurate CSO data reporting in Table 2. It is possible to infer that some of the best practices could be due to a strength being the flexibility of the CSO solicitation technique. Alternatively, a weakness denoting ambiguity or confusion could also be inferred when considering the extensive best practices, the majority of which were categorized under Internal Agency Processes. These practices were subsequently recommended to ensure efficiency and successful contracts that, otherwise, may not be achieved. The most comments, categorized under Internal Agency Processes, are also notable in the types of recommendations for which the acquisition community expressed a need. We anticipate that these recommendations will be well-received and utilized. Finally, note that a few of the observations are duplicative, but they were all retained to highlight how multiple agencies made similar comments that could influence prioritization of recommendations at the end of this article. Expanding beyond just the limited number of strengths and weaknesses identified in

Defense ARJ , Spring 2025, Vol. 32 No. 1

https://www.dau.edu

the CSO Cross Talks feedback, the other findings discussed in this article capture that, at this point and overall, CSOs have many more strengths than weaknesses. In total, we made 66 individual observations of strengths, weaknesses, and best practices. Within those observations, the research team identified 27 strengths, seven weaknesses, and 43 best practices in the documented findings of the CSO data. Although the total adds up to 77, some of these observations were assigned to multiple categories or were defined as both a best practice and a strength or a weakness; therefore, the total observations of 66 account for responses, while the 77 observations account for the researchers’ total categorization. These findings were also categorized across 10 categories according to their central theme(s), with some findings falling into multiple categories. The total quantities of strengths and weaknesses by category are provided in Table 5. The protest findings, especially, are a very telling representation of the significant advantage that CSOs may have over FAR-based solicitation techniques in that so few protests have been filed, and none have been sustained that were filed on the basis of the CSO process itself. Additionally, the process flexibility and limited scope of litigation that come from judicial deference are strengths that merit prudent planning and potential opportunities that contracting activities can embrace in their own solicitation planning process.

TABLE 5. QUANTITY OF STRENGTHS AND WEAKNESSES BY CATEGORY Category

Strengths Weaknesses

Training and Information Sharing

Internal Agency Processes

Solicitation Definition

Industry Interaction

Expanded Solution Horizons

Industry Participation and Competition

Cost/Price/Budgeting Schedule and Planning

Process Flexibility

Scope of Litigation

Given the research we conducted, it is believed that the CSO process should be embraced by agencies seeking to expand their technological horizons and capabilities. The strengths identified by the researchers greatly outweigh the weaknesses. Using the best practices and observations the researchers noted, agencies can equip themselves with

Defense ARJ , Spring 2025, Vol. 32 No. 1

Commercial Solutions Opening

the best means and processes to execute successful CSO solicitations. From the data, the researchers found that the CSO solicitation technique also has applications beyond the research and development arenas. The CSO technique can also be used to identify innovative means to accomplish operations, sustainment, and even maintenance tasks, potentially providing total life-cycle cost savings to the government as a result.

The CSO technique can also be used to identify innovative means to accomplish operations, sustainment, and even maintenance tasks, potentially providing total life-cycle cost savings to the government as a result.

As discussed throughout this research, we also note that the CSO process and procedures are relatively immature and rapidly evolving as compared to other solicitation methodologies. To ensure the continued success of the CSO as a solicitation technique to drive innovation, the researchers provide the targeted recommendations that follow in the areas of training and development, policy changes, and tracking and reporting. Recommendations This section presents focused recommendations based on the results of the analysis found in this research. In total, we provide eight recommendations, each with their anticipated benefits and methods by which they can be successfully implemented. The recommendations encompass three categories: training and development, policy changes, and tracking and reporting. Federal Procurement Data System Modification The first recommendation involves both a policy and reporting change. We perceive this recommendation to be the simplest to implement. We propose a two-part modification to the Federal Procurement Data System (FPDS) contract action report. The first modification is to include Solicitation Type and Procurement Method as reporting criteria. These fields would differentiate between the

Defense ARJ , Spring 2025, Vol. 32 No. 1

https://www.dau.edu

procurement method and solicitation type used to award the contract being reported and should include drop-down selections for procurement method and solicitation type. For example, the FPDS contract action report would include a drop-down selection for CSO field as well as selections for other solicitation types such as request for proposal, request for quote, Broad Agency Announcement, invitation for bid, and others. With the addition of the Solicitation Type reporting criterion, the government and future researchers will be able to analyze specifics about solicitation methodologies and the contract awards that follow in a manner similar to the analysis we conducted in this research. The inclusion of the Solicitation Type reporting criterion will also allow for the analysis of other areas that extend beyond the scope of our research, such as industry involvement across differing solicitation types, cost/ price history and modification metrics, small business participation across solicitation techniques, and targeted areas to bolster training in solicitation types. Absent a dedicated field to report solicitation type, we recommend the action description field be modified to enable reporting of the solicitation type, which would still present opportunities for future reporting, analysis, and informed decision making. The second modification we recommend to the FPDS contract action report is the inclusion of Initial Proposal Receipt Date as a reporting criterion. This new field should be a date field, which reports the date the initial proposal was received for all new awards being reported into the FPDS. The FPDS contract action report currently includes a field to report the solicitation date; however, this is not necessarily a useful data point for general solicitations, which can be open for long periods of time and which can invite multiple proposals during its open period(s). Absent this modification to the FPDS, there is no discernible means to distinguish the procurement lead times between a contract action where the proposal was received one day after the CSO was issued, and a contract action where the proposal was received one year after the CSO was issued. The addition of proposal receipt reporting will enable future analysis of procurement lead time for both contracts awarded from CSO solicitations and those awarded by other means.

Defense ARJ , Spring 2025, Vol. 32 No. 1

Page 1 Page 2 Page 3 Page 4 Page 5 Page 6 Page 7 Page 8 Page 9 Page 10 Page 11 Page 12 Page 13 Page 14 Page 15 Page 16 Page 17 Page 18 Page 19 Page 20 Page 21 Page 22 Page 23 Page 24 Page 25 Page 26 Page 27 Page 28 Page 29 Page 30 Page 31 Page 32 Page 33 Page 34 Page 35 Page 36 Page 37 Page 38 Page 39 Page 40 Page 41 Page 42 Page 43 Page 44 Page 45 Page 46 Page 47 Page 48 Page 49 Page 50 Page 51 Page 52 Page 53 Page 54 Page 55 Page 56 Page 57 Page 58 Page 59 Page 60 Page 61 Page 62 Page 63 Page 64 Page 65 Page 66 Page 67 Page 68 Page 69 Page 70 Page 71 Page 72 Page 73 Page 74 Page 75 Page 76 Page 77 Page 78 Page 79 Page 80 Page 81 Page 82 Page 83 Page 84 Page 85 Page 86 Page 87 Page 88 Page 89 Page 90 Page 91 Page 92 Page 93 Page 94 Page 95 Page 96 Page 97 Page 98 Page 99 Page 100 Page 101 Page 102 Page 103 Page 104 Page 105 Page 106 Page 107 Page 108 Page 109 Page 110 Page 111 Page 112 Page 113 Page 114 Page 115 Page 116 Page 117 Page 118 Page 119 Page 120

Made with FlippingBook - Online Brochure Maker