The purpose of this one-day seminar is to present a comprehensive discussion of practical techniques for building valid and credible simulation models. All techniques will be illustrated by one or more examples based on actual simulation projects. At the end of the seminar, each attendee will be familiar with "the" twelve fundamental validation techniques and know how to apply them to their models and simulations. The development of this seminar has benefited from contracts from the Defense Modeling and Simulation Office (now the Modeling and Simulation Coordination Office), the Navy Modeling and Simulation Office, and the Office of Naval Research.
A particular highlight of this seminar is the discussion and illustration of an assumptions document, which is a detailed report delineating all model concepts, assumptions, algorithms, and data summaries. It serves as the main vehicle for communications among the project team, and it is a "blueprint" for creating the simulation computer program. It should not be confused with a conceptual model, which can be thought of as initial ideas on what a model will look like.
Dr. Averill M. Law, the course instructor, has been intimately involved in the problem formulation, validation, and analysis of approximately 40 simulation models. He has been a validation consultant to organizations such as Booz Allen and Hamilton, the Federal Highway Administration, the U.S. Army, and the U.S. Air Force.
Versions of this seminar have been presented to Boeing, Raytheon, the U.S. Army, the U.S. Navy,
the International Council on Systems Engineering (INCOSE), the International
Society for Optical Engineering (SPIE), the International Test and Evaluation Association
(ITEA), and the Military Operations
Research Society (MORS).
What You Will Learn:
1. Seven Important Steps in a Sound Study
2. Formulating the Problem Precisely
3.
Talking to a Wide Variety of Subject-Matter Experts (SMEs)4.
Interacting with the Decision-Maker on a Regular BasisHelps ensure that the correct problem is being addressed
Enhances the credibility of the simulation model
5.
Using
Quantitative Techniques to Validate Components of the Model
6.
Purpose, components, and format
Detailed example
Construction for a legacy simulation model that has limited documentation
7. Performing a Structured Walk-Through of
the
Assumptions Document
Format and benefits Required attendees
8. Using Sensitivity Analysis to Determine Important Model Factors
The
Introduction to the proper design of experiments
9. Results Validation
Comparison of model output data with the comparable output data from a similar
Use of a Turing test
Evaluation of model output data by SMEs
Comparison of model output data with the comparable output data from another model that is thought to be "valid"
Use of confidence intervals and hypothesis tests to make comparisons
10. Using Animation to Show that a Model is not Valid and to Enhance Credibility
11. Guidelines for Obtaining Good Model Data
Two basic principles
Common difficulties
Model calibration and how it differs from validation
Independent model validation
13. 6 Critical Pitfalls and How to Avoid Them
