But Who Should Have Won? Simulating Outcomes of Judging Protocols and Ranking Systems
Ann W. Clifton, and Allison L. Lewis
We all know the world loves an underdog. But who should have won? Simulation models can give us a way to reproduce contest scenarios using computer power, allowing us to simulate versions of reality numerous times and observe a wide range of possible outcomes. Using basic probability calculations and a wide range of programming tools, we can mimic real life and answer such questions as “how likely is Team A to win the tournament?”, “how biased would Judge B have to be in favor of an athlete for them to come out on top?”, or “is there a particular voting structure that would allow for the emergence of Candidate C as the winner?” In this chapter, we lay out the fundamental skills for pursuing answers to questions like these via simulation, and leave the interested reader with a set of potential research projects on the theme of evaluating fairness, bias, and other issues relevant to ranking schemes and judging protocols.
