Validation OF MODELS AND SIMULATIONS
The purpose of this live online course is twofold. First, we present a comprehensive discussion of practical techniques for validating existing models and simulations. We also describe a detailed seven-step approach for building models that are valid, credible, and appropriately detailed, so that they are actually used in the decision-making process. All techniques will be illustrated by one or more examples based on actual simulation projects. The development of this seminar has benefited from contracts with the Defense Modeling and Simulation Office and the Office of Naval Research.
A highlight of this seminar is the discussion and illustration of an assumptions document, which is a detailed report delineating all model concepts, assumptions, algorithms, input parameters, output performance measures, and data summaries. It serves as the main vehicle for communications among the project team, and it is a “blueprint” for creating the simulation computer program. It should not be confused with a conceptual model, which can be thought of as initial ideas on what a model will look like. Most models and simulations are simply not well documented, resulting in communication errors and costly, incorrect decisions.
Dr. Averill M. Law, the course instructor, has been intimately involved in the problem formulation, validation, and analysis of approximately 50 models and simulations. He has been a validation consultant to organizations such as Booz Allen and Hamilton, Ford, ITT, Stanley Black & Decker, U.S. Air Force, U.S. Army, U.S. Marine Corps, and U.S. Navy.
Live versions of this seminar have been presented to Boeing, Lockheed Martin, NSA, Raytheon, Sasol Technology (South Africa), U.S. Army (2 times), U.S. Navy (2 times), Georgia Tech Research Institute, Engineering Industries eXcellence, International Council on Systems Engineering (INCOSE), International Society for Optical Engineering (SPIE), and International Test and Evaluation Association (ITEA).
This course is also available in a Live Online format.
“We were very happy with the course on model validation that you conducted for Engineering Industries eXcellence.” Scott Cummings, Simulation Architect, Engineering Industries eXcellence, December 6, 2021
“I think that this course is a great introduction to the processes of modeling and validation in general. It is a course that I wish I had taken before I had spent any time modeling. I enjoyed the real-life examples.” Operations Research Analyst, U.S. Army, White Sands Missile Range, May 3, 2019
Outline
What You Will Learn (look to the right for more detail):
- Seven Important Steps in a Sound Study
- Formulating the Problem Precisely
- Talking to Appropriate Subject-Matter Experts
- Interacting with the Decision-Maker on a Regular Basis
- Using Quantitative Techniques to Validate Components of the Model
- Developing a Written Assumptions Document
- Performing a Structured Walk-through of the Assumptions Document
- Using Sensitivity Analysis to Determine Important Model Factors
- Results Validation
- Using Animation to Show that a Model is not Valid and to Enhance Credibility
- Guidelines for Obtaining Good Model Data
- Calibration versus Validation
- 6 Critical Pitfalls and How to Avoid Them