Falsifification Assessment Theory

Falsifiability increases empirical knowledge – an example in search assessment
Swets’ ROC analysis was developed and validated for signal detection given a known location. Clinically relevant tasks are often more complicated, e.g., search, multi-class classification, estimation, etc. Many extended ROC methods are proposed for each task, and there is a sincere lack of consensus concerning which extended ROC method is more appropriate. Before discussing how the lack of falsifiability contributes to this lack of consensus, in this section, I present a falsifiable search assessment method.
Just as Swets’ ROC, a falsifiable assessment theory is aimed at understanding “reality”. Thus, in addition to preserving at least some empirical content of Swets’ ROC, e.g., prevalence-independence, a search assessment
1. The generation of the core hypothesis is known to be an irrational process 2,14,28. The readers may refer to He et. al. for a set of evidence that leads to the following core hypothesis27. However, there is no logical path in reaching the core hypotheses. The ESS theory starts with the core
Because M* is constant across a range of scenarios, it should allow predictions. For example, given M* and LKE ROC measured from an observer, one should be able to predict the same observer’s absolute performance in different search tasks where LKE ROC and M* are claimed constant, including 1) single-response search tasks with one or multiple signals in the signal-present images 2) search tasks with a larger or smaller search size, and 3) single-response or free-response tasks,