Validation OF MODELS AND SIMULATIONS

The purpose of this online course is twofold. First, we present a comprehensive discussion of practical techniques for validating existing models and simulations. We also describe a detailed seven-step approach for building models that are valid, credible, and appropriately detailed, so that they are actually used in the decision-making process. All techniques will be illustrated by one or more examples based on actual simulation projects. The development of this seminar has benefited from contracts with the Defense Modeling and Simulation Office and the Office of Naval Research.

A highlight of this seminar is the discussion and illustration of an assumptions document, which is a detailed report delineating all model concepts, assumptions, algorithms, and data summaries. It serves as the main vehicle for communications among the project team, and it is a “blueprint” for creating the simulation computer program. It should not be confused with a conceptual model, which can be thought of as initial ideas on what a model will look like.

Technical support is provided by e-mail. The course length corresponds to that of a 1.5-day live seminar.  Recent subscribers to this course include Johns Hopkins Applied Physics Lab, Manitoba Hydro, Penn State Applied Research Lab, and Ohio State University.

Dr. Averill M. Law, the course instructor, has been intimately involved in the problem formulation, validation, and analysis of approximately 40 models and simulations. He has been a validation consultant to organizations such as Booz Allen and Hamilton, Ford, ITT, Stanley Black & Decker, U.S. Air Force, U.S. Army, U.S. Marine Corps, and U.S. Navy.

Live versions of this seminar have been presented to Boeing, Lockheed Martin, NSA, Raytheon, Sasol Technology (South Africa), U.S. Army, U.S. Navy (2 times), International Council on Systems Engineering (INCOSE), International Society for Optical Engineering (SPIE), and International Test and Evaluation Association (ITEA).

Outline

What You Will Learn (look to the right for more detail):

  1. Seven Important Steps in a Sound Study
  2. Formulating the Problem Precisely
  3. Talking to Appropriate Subject-Matter Experts
  4. Interacting with the Decision Maker on a Regular Basis
  5. Using Quantitative Techniques to Validate Components of the Model
  6. Developing a Written Assumptions Document
  7. Performing a Structured Walk-through of the Assumptions Document
  8. Using Sensitivity Analysis to Determine Important Model Factors
  9. Results Validation
  10. Using Animation to Show that a Model is not Valid and to Enhance Credibility
  11. Guidelines for Obtaining Good Model Data
  12. Calibration versus Validation
  13. 6 Critical Pitfalls and How to Avoid Them