Statistical And On-line Compression Modelling
In standard statistical modelling Reality is modelled as a family probability measures ⚠ $\{P_{\theta} \mid \theta\in\Theta\}$
. In on-line compression modelling Reality is modelled as a 5-tuple whose key elements are the forward functions and backward kernels. With each on-line compression model ⚠ $M$
we can associate the statistical model ⚠ $\phi(M)$
defined as the extreme points of the probability measures on ⚠ $\mathbf{Z}^{\infty}$
(where ⚠ $\mathbf{Z}$
is the example space) that agree with the on-line compression model. Natural questions (some rather vague) are:
- What are the statistical models that can be obtained in this way (are of the form
⚠ $\phi(M)$
for some⚠ $M$
)? - Characterize the on-line compression models
⚠ $M$
for which there is no "loss of information" in replacing⚠ $M$
by⚠ $\phi(M)$
. - Does
⚠ $\phi$
establish a bijection between some wide and natural classes of on-line compression and statistical models?
Some work in this direction has been done by Martin-Lof, Lauritzen, and other authors for repetitive structures (models very closely related to on-line compression models). See Vovk et al. (2005), Section 8.8.
Bibliography
- Vladimir Vovk, Alexander Gammerman and Glenn Shafer (2005). Algorithmic learning in a random world. Springer, New York.