Statistical test

Definition

We use the term statistical test for a test, $T$, with two outcomes (usually "yes"/1 or "no"/0). Let $P\eq 1$ denote that the unit under test actually has this property we're testing for, and $P\eq 0$ if it does not.

The statistical test is defined by the following four probabilities:

• $\mathbb{P}[T\eq 1\ \vert\ P\eq 1]\eq\alpha$ - "true positive"
• $\mathbb{P}[T\eq 1\ \vert\ P\eq 0]\eq\beta$ - "false positive" as the test incorrectly reports positive.
• $\mathbb{P}[T\eq 0\ \vert\ P\eq 1]\eq\gamma$ - "false negative" as the test incorrectly reports negative yet the property is present
• $\mathbb{P}[T\eq 0\ \vert\ P\eq 0]\eq\delta$ - "true negative"

Todo

Add an example where 1 in 10,000 actually has the property, but the test has a non-zero false-positive rate, actually meaning that if the test returns positive it's unlikely you actually have the property, relate to statistical power.