In this lecture, we review what we have learned about one-sample confidence intervals (i.e., how to use them as graphical versions of one-sample t-tests) for absolute performance estimation in order to motivate the problem of relative performance estimation. We introduce two-sample confidence intervals (i.e., confidence intervals on DIFFERENCES based on different two-sample t-tests) that are tested against a null hypothesis of 0. This means covering confidence interval half widths for the paired-difference t-test, the equal-variance (pooled) t-test, and Welch's unequal variance t-test. Each of these different experimental conditions sets up a different standard error of the mean formula and formula for degrees of freedom that are used to define the actual confidence interval half widths (centered on the difference in sample means in the pairwise comparison of systems). We then generalize to the case of more than 2 systems, particularly for "ranking and selection (R&S)." This lets us review the multiple-comparisons problem (and Bonferroni correction) and how post hoc tests (after an ANOVA) are more statistically powerful ways to do comparisons.
Archived lectures from undergraduate course on stochastic simulation given at Arizona State University by Ted Pavlic
Thursday, November 10, 2022
Lecture J4 (2022-11-10): Estimation of Relative Performance
Subscribe to:
Post Comments (Atom)
Popular Posts
-
In this lecture, we go over course policies for the Fall 2022 session of IEE 475.
-
In this lecture, we introduce the three different simulation methodologies (agent-based modeling, system dynamics modeling, and discrete eve...
-
In this lecture, we wrap up the course content in IEE 475. We first do a quick overview of the four variance reduction techniques (VRT's...
-
This lecture section is a cumulative review of material from the semester and is meant to serve as a study guide for students preparing for ...
-
Today's lecture covers the basics of probability (including introduction to measure spaces) and random variables. We also go over some r...
-
In this lecture, we continue to discuss hypothesis testing -- introducing parametric, non-parametric, exact, and non-exact tests and reviewi...
-
This lecture continues to discuss issues related to estimating absolute performance from transient and steady-state simulations (of terminat...
-
In this lecture, we (nearly) finish our coverage of Input Modeling, where the focus of this lecture is on parameter estimation and assessing...
-
In this lecture, we prepare for the final exam and give a brief review of all topics from the course.
-
In this lecture, we review four different Variance Reduction Techniques (VRT's). Namely, we discuss common random numbers (CRNs), contro...
No comments:
Post a Comment