|
This paper presents a new methodology for assessing the risks associated with the level of Verification,
Validation and Accreditation (VV&A) of a given Model and/or Simulation when used in support of major
decisions. As stated by DoD Instruction 5000.61 , "It is DoD policy that: Models and Simulations (M&S)
used to support major DoD decision-making organizations and processes shall be accredited for that
specific purpose by the DoD Component M&S Application Sponsor." This Instruction applies to "All
models and simulations developed, used, or managed by the DoD Components after the effective date of
this Instruction." The requirements cited above have set the need for VV&A of M&S at the forefront of
concerns for DoD and DoD-Components acquisition personnel. When an acquisition program involves a
large number of models, cost associated with VV&A can become enormous. There is a need therefore to
have a systematic approach for assessing and prioritizing the risks associated with the level to which
individual models have been verified, validated, and accredited. To provide decision makers with a
judicious way for determining the risks associated with using a given M&S, and the extent to which VV&A
work will be needed to meet these requirements, we have developed a methodology for assessing the risks
associated with the level of VV&A of a given M&S when used to support decision-making. This approach
parallels the formal DoD Risk Assessment procedure, but with application to the use of M&S, as it relates
to VV&A. Risks associated with the levels of VV&A are evaluated and assessed based on the following
criteria:
- Likelihood of the M&S being inaccurate and/or inappropriate for the application.
- Consequences of the M&S being inaccurate and/or inappropriate for the
application.
- Importance of the (acquisition) decision supported by the M&S.
- Level of Reliance of the (acquisition) decision on the M&S.
The assessment results are used to classify individual models into three risk categories
(Red for High Risk, Yellow for Medium Risk, and Green for Low Risk). All models
classified as High Risk, are subjected to further analysis and recommendations made as to
further work needed to reduce the risk to an acceptable level through formal VV&A.
Models assessed as Medium Risk are also further evaluated and recommendations made
with regards to the need for further risk mitigation. Models rated Low Risk are either not
of primary importance in supporting the (acquisition) decisions being made, or they may
have been assessed to have been satisfactorily verified, validated, and possibly accredited
to an acceptable level.
|
