Vocabulary:
lcsh
Type:
Topic
Note:
Work cat.: 2004055932: Advances in minimum description length, 2005: p. 3 ("The minimum description length (MDL) principle
is a relative recent method for inductive inference that provides a generic solution to the model selection problem. MDL is
based on the following insight: any regularity in the data can be used to compress that data, that is, to describe it using
fewer symbols than the number of symbols needed to describe the data literally. The more regularities there are, the more
the data can be compressed.") p. 66 ("MDL shares some ideas with the minimum message length (MML) principle which predates
MDL by ten years... Just as in MDL, MML chooses the hypothesis minimizing the code length of the data. But the codes that
are used are quite different from those in MDL. First of all, in MML one always uses two-part codes, so that MML automatically
selects both a model family and parameter values.")
Note:
Wikipedia, Aug. 16, 2005 (minimum description length: "a formalization of Occam's Razor in which the best hypothesis for a
given set of data is the one that leads to the largest compression of the data. MDL is important in information theory and
learning theory... MDL was not the first attempt to do hypothesis selection through minimizing description length; as early
as 1968 Wallace and Boulton pioneered a related concept called Minimum Message Length (MML). MDL was introduced by Jorma Rissanen
in 1978; it differs from MML in several ways, most notably (at least in most of J. Rissanen's early MLD papers) in its extensive
use of one-part rather than two-part codes.")
