In this lecture, we aim to finish our coverage of Input Modeling, where the focus of this lecture is on parameter estimation and assessing goodness of fit. We start by identifying different shape parameters (including location, scale, and rate) as quantities that need to be estimated. Before getting to their estimation, we also discuss summary statistics (sample mean and sample variance) and maximum likelihood estimation (MLE), with an example for a point estimate of the rate of an exponential. We then discuss how AIC and BIC provide modern ways of ranking models by MLE that also account for the number of parameters in that candidate model. After covering these model selection topics, we transition to a discussion of goodness of fit. That first requires a review of the fundamentals of hypothesis testing. We discuss type-I error, p-values, type-II error, effect sizes, and statistical power. We discuss the dangers of using p-values at very large sample sizes (where small p-values are not meaningful) and at very small sample sizes (where large p-values are not meaningful). We give some examples of this applied to best-of-7 sports tournaments and voting. Finally, we close by re-introducing the chi-squared goodness of fit test for larger sample sizes and the Kolmogorov–Smirnov (K–S, higher powered) goodness-of-fit test (and a relative, Anderson-Darling, and a distribution specific alternative for testing normality, Shapiro–Wilk) for smaller sample sizes. What we do not finish in the coverage of goodness-of-fit here will be covered in the next lecture.
Interactive widgets relevant to this lecture include:
- https://tpavlic.github.io/asu-simulating-stochastic-systems/input_modeling/input_analyzer.html
- https://tpavlic.github.io/asu-simulating-stochastic-systems/input_modeling/prob_models.html
- https://tpavlic.github.io/asu-simulating-stochastic-systems/power_analysis/power_explorer.html
- https://tpavlic.github.io/asu-simulating-stochastic-systems/output_analysis/output_analyzer/output_analyzer.html
No comments:
Post a Comment