In this lecture, we review statistical fundamentals – such as the origins of the t-test, the meaning of type-I and type-II error (and alternative terminology for both, such as false positive rate and false negative rate) and the connection to statistical power (sensitivity). We review the Receiver Operating Characteristic (ROC) curve and give a qualitative description of where it gets its shape in a hypothesis test. We close with a validation example (from Lecture H) where we use a power analysis on a one-sample t-test to help justify whether we have gathered enough data to trust that a simulation model is a good match for reality when it has a similar mean output performance to the real system. Peppered throughout the lecture are also comments about why normality is required for t-tests, why there is a minimum expected count for chi-squared tests, and how to avoid statistical inference issues when making multiple comparisons. The lecture also discusses how nonparametric tests have fewer assumptions than parametric tests but tend to pay a cost in terms of statistical power.
The following tools and concept explorers are referred to in this lecture:
- Concept Explorer: Statistical Power Analysis
- Concept Explorer: Output Analyzer (fully functional statistical analysis tool)
- Concept Explorer: Input Analyzer (fully functional Input Modeling tool)
No comments:
Post a Comment