Historically, there have been three different forms of arguments to show that we can choose between theories in a non-arbitrary, and truth-tracking fashion: non-empirical virtues of theories provide a truth-tracking criteria, the evidence itself allows us to choose between the theories, and the historical success of science provides evidence for one theory over its rivals. I argue that each of these arguments fails to defeat the claim that, when faced with empirically equivalent rivals, selecting one theory over the others is not arbitrary. However, a defense of the claim that theory choice is arbitrary does not amount to the claim that theory selection is arbitrary. Thus, I argue that there is a motivation for the claim that we should not select any given theory, and that it is in principle possible to defend such a claim. However, my defense can only justify the claim that theory choice is arbitrary in a narrow set of cases: when it is used against people who are undecided with respect to realism and anti-realism, and when it used against people who are realists for reasons other than the belief that there are non-empirical virtues of theories. (Abstract shortened by UMI.)