Comptes Rendus
Probability Theory
On the minimum f-divergence for given total variation
Comptes Rendus. Mathématique, Volume 343 (2006) no. 11-12, pp. 763-766.

We want to find a lower bound for an f-divergence Df in terms of variational distance V which is best possible for any given V. In other words, we want to find LDf(v)=inf{Df(P,Q):V(P,Q)=v}. In this note we solve this problem for any convex f. Although the form of LDf(V) depends on inverting some expressions which may be difficult in general, simplifications can occur when f has some kind of symmetry. For instance, if Df is symmetric in the sense that Df(P,Q)=Df(Q,P), we show that LDf(v)=2v2f(2+v2v)f(1)v. For the Kullback–Leibler divergence K we obtain an expression of LK in terms of the two real branches of Lambert's W function.

Pour chaque distance variationnelle V donnée on veut trouver la meilleure borne inférieure possible pour une f-divergence Df. En d'autres termes, on veut trouver LDf(v)=inf{Df(P,Q):V(P,Q)=v}. Dans cette note on résout ce problème pour toute fonction f convexe. Bien que la forme de LDf(V) dépende de l'inversion de quelques expressions, ce qui peut être difficile en général, des simplifications peuvent se produire quand f a une certaine symétrie. Par exemple, si Df est symétrique dans le sens : Df(P,Q)=Df(Q,P), on prouve que LDf(v)=2v2f(2+v2v)f(1)v. Pour la divergence de Kullback–Leibler K nous obtenons une expression de LK à l'aide des deux branches réelles de la fonction W de Lambert.

Published online:
DOI: 10.1016/j.crma.2006.10.027

Gustavo L. Gilardoni 1

1 Departamento de Estatística, Universidade de Brasília, Brasília, DF 70910-900, Brazil
     author = {Gustavo L. Gilardoni},
     title = {On the minimum \protect\emph{f}-divergence for given total variation},
     journal = {Comptes Rendus. Math\'ematique},
     pages = {763--766},
     publisher = {Elsevier},
     volume = {343},
     number = {11-12},
     year = {2006},
     doi = {10.1016/j.crma.2006.10.027},
     language = {en},
AU  - Gustavo L. Gilardoni
TI  - On the minimum f-divergence for given total variation
JO  - Comptes Rendus. Mathématique
PY  - 2006
SP  - 763
EP  - 766
VL  - 343
IS  - 11-12
PB  - Elsevier
DO  - 10.1016/j.crma.2006.10.027
LA  - en
ID  - CRMATH_2006__343_11-12_763_0
ER  - 
%0 Journal Article
%A Gustavo L. Gilardoni
%T On the minimum f-divergence for given total variation
%J Comptes Rendus. Mathématique
%D 2006
%P 763-766
%V 343
%N 11-12
%I Elsevier
%R 10.1016/j.crma.2006.10.027
%G en
%F CRMATH_2006__343_11-12_763_0
Gustavo L. Gilardoni. On the minimum f-divergence for given total variation. Comptes Rendus. Mathématique, Volume 343 (2006) no. 11-12, pp. 763-766. doi : 10.1016/j.crma.2006.10.027.

[1] S.M. Ali; S.D. Silvey A general class of coefficients of divergence of one distribution from another, J. Roy. Statist. Soc. Ser. B, Volume 28 (1966), pp. 131-142

[2] R.M. Corless; G.H. Gonnet; D.E.G. Hare; D.J. Jeffrey; D.E. Knuth On the Lambert W function, Adv. Comput. Math., Volume 5 (1996) no. 4, pp. 329-359

[3] I. Csiszár Information-type measures of difference of probability distributions and indirect observations, Studia Sci. Math. Hungar., Volume 2 (1967), pp. 299-318

[4] M. Del Pino; J. Dolbeault Nonlinear diffusions and optimal constants in Sobolev type inequalities: asymptotic behaviour of equations involving the p-Laplacian, C. R. Acad. Sci. Paris, Ser. I, Volume 334 (2002) no. 5, pp. 365-370

[5] A. Fedotov; P. Harremoës; F. Topsøe Refinements of Pinsker's inequality, IEEE Trans. Inform. Theory, Volume 49 (2003), pp. 1491-1498

[6] I. Vajda Note on discrimination information and variation, IEEE Trans. Inform. Theory, Volume 16 (1970), pp. 771-773

Cited by Sources:

Comments - Policy