Commercial firms gained more knowledge about a product's technology, performance, and producibility much earlier in the product development process than DOD. Product development in commercial ventures was a clearly defined undertaking for which firms insisted on having the technology in hand to meet customer requirements before starting. Once underway, these firms demanded - and got - specific knowledge about a new product before production began. The process of discovery - the accumulation of knowledge and the elimination of unknown - was completed for the best commercial programs well ahead of production. [Best Practices: Successful Application to Weapon Acquisitions Requires Changes in DOD's Environment, GAO/NSIAD-98-56.]

The GAO study illustrated the "best practice" of fully developing and testing major new products before producing them with numerous case studies, such as the Boeing 777 commercial airliner.

Air Force Assistant Secretary for Acquisition Arthur Money is aware of the "best practice" rhetoric and made the strange claim that "commercial practices to the hilt are being used here." To Money, commercial best practice testing apparently means computer simulation, not actual flight testing, and testing parts separately, rather than after they have all been put together in an actual airplane. Given the F-22's unprecedented complexity, reliance on unfamiliar stealthy materials, and revolutionary level of integration of electronics and software, this is an extremely hazardous assumption to follow.

Actual experience belies the confidence the Air Force shows in computer simulations. All the computer and wind-tunnel testing performed on the E/F version of the Navy F/A-18 fighter-bomber was not enough to prevent the late discovery of a crippling wing-drop phenomenon - even though the E/F version was supposedly a minor modification of an existing F/A-18 aircraft. In contrast, the F-22 is a completely new design. The F/A-18E/F had potentially catastrophic problems despite completing 20% of its flight testing before production, yet the Air Force wants to proceed with just 4% of testing completed on the F-22.

Putting the most complex fighter ever made into production with only 4% of its flight testing performed patently contradicts common sense. The program has already experienced an increase in procurement cost per plane from $87 million to $127 million in just the past five years, and numerous program delays. Decades of experience in developing weapons shows that realistic testing brings major problems to light early. The B-1B bomber's seemingly-endless defensive avionics problems provide one recent example of extremely expensive post-production fixes that should have been addressed before. In the 1980s, late testing revealed such serious problems with the Sergeant York air defense gun that it was canceled.

No Longer In Such a Rush

During the Cold War, in an atmosphere of imminent Soviet attack, the claim was frequently made that the latest technology had to be rushed into production - with less-than-full testing - to get weapons fielded first. There is no longer a global arms race to justify such haste. The Air Force should live by its rhetoric, adopt commercial best practices and follow the obvious wisdom of "Fly before Buy." If the Air Force won't do this on its own, the Congress owes it to the troops in the field and to the taxpayer at home to require more testing before production funds are released.