Proportionate reduction of error

Proportionate reduction of error (PRE) is the gain in precision of predicting dependent variable ${\displaystyle y}$ from knowing the independent variable ${\displaystyle x}$ (or a collection of multiple variables). It is a goodness of fit measure of statistical models, and forms the mathematical basis for several correlation coefficients.^[1] The summary statistics is particularly useful and popular when used to evaluate models where the dependent variable is binary, taking on values {0,1}.

If both ${\displaystyle x}$ and ${\displaystyle y}$ vectors have cardinal (interval or rational) scale, then without knowing ${\displaystyle x}$, the best predictor for an unknown ${\displaystyle y}$ would be ${\displaystyle {\bar {y}}}$, the arithmetic mean of the ${\displaystyle y}$-data. The total prediction error would be ${\displaystyle E_{1}=\sum _{i=1}^{n}{(y_{i}-{\bar {y}})^{2}}}$ .

If, however, ${\displaystyle x}$ and a function relating ${\displaystyle y}$ to ${\displaystyle x}$ are known, for example a straight line ${\displaystyle {\hat {y}}_{i}=a+bx_{i}}$, then the prediction error becomes ${\displaystyle E_{2}=\sum _{i=1}^{n}{(y_{i}-{\hat {y}})^{2}}}$. The coefficient of determination then becomes ${\displaystyle r^{2}={\frac {E_{1}-E_{2}}{E_{1}}}=1-{\frac {E_{2}}{E_{1}}}}$ and is the fraction of variance of ${\displaystyle y}$ that is explained by ${\displaystyle x}$. Its square root is Pearson's product-moment correlation ${\displaystyle r}$.

There are several other correlation coefficients that have PRE interpretation and are used for variables of different scales:

predict	from	coefficient	symmetric
nominal, binary	nominal, binary	Guttman's λ[2]	yes
ordinal	nominal	Freeman's θ[3]	yes
cardinal	nominal	η${\displaystyle ^{2}}$[4]	no
ordinal	binary, ordinal	Wilson's e [5]	yes
cardinal	binary	point biserial correlation	yes