Lecture K1 (2022-11-15): Variance Reduction Techniques, Part 1 (CRNs and Control Variates)

In this lecture, we start by reviewing approaches for absolute and relative performance estimation in stochastic simulation. This begins with a reminder of the use of confidence intervals for estimation of performance for a single simulation model. We then move to different ways to use confidence intervals on mean DIFFERENCES to compare two different simulation models. We then move to the ranking and selection problem for three or more different simulation models, which allows us to talk about analysis of variance (ANOVA) and post hoc tests (like the Tukey HSD or Fisher's LSD). After that review, we move on to introducing variance reduction techniques (VRTs) which reduce the size of confidence intervals by experimentally controlling/accounting for alternative sources of variance (and thus reducing the observed variance in response variables). We discuss Common Random Numbers (CRNs), which use a paired/blocked design to reduce the variance caused by different random-number streams. We start to discuss control variates (CVs), but that discussion will be picked up at the start of the next lecture.