ИСТИНА

Along with brief survey we develop the quite recent papers [1] and [2] to study statistical estimation of mutual information and other divergences. Such estimates are employed, for instance, in machine learning, textures inhomogeneities detection and feature selection (see, e.g., [5]). We investigate the asymptotic properties of proposed estimates constructed by means of i.i.d. (vector-valued) observations. For this purpose we apply the techniques involving the nearest neighbor statistics. Ensemble methods are discussed as well. Special attention is payed to results of computer simulations in the framework of mixed models (see, e.g. [3], [4]) comprising the widely used logistic regression. In contrast to previous works we do not suppose that one can measure the distances between the values of a discrete response variable. This is essential in many cases for analysis of medical and biological data. References [1] Bulinski, A., Dimitrov, D. Statistical estimation of the Shannon entropy. Acta Mathematica Sinica. English Series. Published online: September 7 (2018), p.1-28. DOI: https://doi.org/10.1007/s10114-018-7440-z. [2] Bulinski, A., Kozhevin, A. Statistical estimation of conditional Shannon entropy. ESAIM: Probability and Statistics. Published online: November 28 (2018), p.1-35. DOI: https://doi.org/10.1051/ps/2018026. [3] Coelho, F., Braga, A.P., Verleysen, M. A mutual information estimator for continuous and discrete variables applied to feature selection and classification problems, International Journal of Computational Intelligence Systems, (2016), 9, p. 726-733. DOI: https://doi.org/10.1080/18756891.2016.1204120. [4] Gao, W., Kannan, S., Oh, S., Viswanath, P. Estimating mutual information for discrete-continuous mixtures, arXiv:1709.06212v3, 9 Oct 2018, p. 1-26. [5] Vergara, J.R., Estévez, P.A. A review of feature selection methods based on mutual information. Neural Computing and Applications, (2014), 24, p. 175-186. DOI: https://doi.org/10.1007/s00521-013-1368-0.