Entropy estimates of small data sets

Bonachela, Juan A.; Hinrichsen, Haye; Munoz, Miguel A.

Entropy estimates of small data sets

Authors

Bonachela, Juan A.

Hinrichsen, Haye

Munoz, Miguel A.

Description

Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon, Rényi and Tsallis) specially devised to provide a compromise between low bias and small statistical errors, for short data series. This new estimator out-performs other currently available ones when the data sets are small and the probabilities of the possible outputs of the random variable are not close to zero. Otherwise, other well-known estimators remain a better choice. The potential range of applicability of this estimator is quite broad specially for biological and digital data series.
11 pages, 2 figures