Generalized Linear Models

Let the outcome belong to ⚠ $ [Y_1,Y_2] $. We say that Expert ⚠ $\theta$'s prediction at step ⚠ $t$ is denoted ⚠ $\xi^\theta_t$ and is equal to ⚠ $$\xi_t^\theta = Y_1+(Y_2-Y_1)\sigma(\theta'x_t).⚠ $$ Here ⚠ $\sigma: \mathbb{R}\to\mathbb{R}$ is a fixed \emph{activation function}. We have ⚠ $\sigma:\mathbb{R}\to [0,1]$ in all the cases except linear regression (see below). If the range of the function ⚠ $\sigma$ is ⚠ $ [0,1] $, the experts necessarily give predictions from ⚠ $ [Y_1,Y_2] $.

• Logistic activation function is ⚠ $\sigma(z) = \frac{1}{1+e^{-z}}$.
• Probit activation function is ⚠ $$\sigma(z) = \Phi(z) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^z e^{-v^2/2} dv,⚠ $$

where ⚠ $\Phi$ is the cumulative distribution function of the normal distribution with zero mean and unit variance.

• Complementary log-log (comlog) activation function ⚠ $\sigma(z) = 1-\exp (-\exp(z))$.

Generalized Linear Models are often used for classification purposes.

Bibliography

• Mc Cullagh, P., Nelder, J. Generalized Linear Models. Chapman & Hall/CRC, second edn. 1989.