Design of Experiments for Simulation Modeling

Discrete-event and agent-based simulation models often have many input factors, and determining which ones have a significant impact on performance measures (responses) of interest can be a truly daunting task.  The common approach of changing one factor at a time is statistically inefficient and, more importantly, is very often just incorrect, because for many models factors interact to impact on the responses.  In this seminar, we present a comprehensive introduction to design of experiments (DOE), whose major goal in simulation modeling is to determine which factors have the greatest effect on the responses , and to do so with the least amount of simulating.  Another important use of DOE is to develop a metamodel (a model of a model) or response surface based on the important factors to predict the model responses for factor combinations that were not actually simulated, since the execution time for the simulation model might be large.

We discuss a simple and widely applicable approach to performing DOE in the context of simulation modeling, whereas methods based on classical statistics (i.e., ANOVA) make unrealistic assumptions such as constant variances and normally distributed residuals.  Furthermore, the common remedy of transforming the data often does not work either.  A version of this seminar has been presented to the Military Operations Research Society (MORS).

1. Factorial Designs

2. Fractional Factorial Designs

3. Metamodels and Response Surfaces

• Central composite designs for fitting second-order metamodels

• Predicting model responses for factor combinations that were not simulated 

• Finding the factor-level combination that optimizes a simulation response 

4. Space-Filling Designs