Comptes Rendus
Analyse numérique
A convergent Deep Learning algorithm for approximation of polynomials
Bruno Després1
1 Laboratoire Jacques-Louis Lions, Sorbonne Université, 4 place Jussieu, 75005 Paris, France
Comptes Rendus. Mathématique, Volume 361 (2023), pp. 1029-1040.

We start from the contractive functional equation proposed in [4], where it was shown that the polynomial solution of functional equation can be used to initialize a Neural Network structure, with a controlled accuracy. We propose a novel algorithm, where the functional equation is solved with a converging iterative algorithm which can be realized as a Machine Learning training method iteratively with respect to the number of layers. The proof of convergence is performed with respect to the L ∞ norm. Numerical tests illustrate the theory and show that stochastic gradient descent methods can be used with good accuracy for this problem.

Reçu le :
Accepté le :
Accepté après révision le :
Publié le :
DOI : 10.5802/crmath.462
Classification : 65Q20, 65Y99, 78M32
Bruno Després 1

1 Laboratoire Jacques-Louis Lions, Sorbonne Université, 4 place Jussieu, 75005 Paris, France
Licence : CC-BY 4.0
Droits d'auteur : Les auteurs conservent leurs droits 
@article{CRMATH_2023__361_G6_1029_0,
     author = {Bruno Despr\'es},
     title = {A convergent {Deep} {Learning} algorithm for approximation of polynomials},
     journal = {Comptes Rendus. Math\'ematique},
     pages = {1029--1040},
     publisher = {Acad\'emie des sciences, Paris},
     volume = {361},
     year = {2023},
     doi = {10.5802/crmath.462},
     language = {en},
}
TY  - JOUR
AU  - Bruno Després
TI  - A convergent Deep Learning algorithm for approximation of polynomials
JO  - Comptes Rendus. Mathématique
PY  - 2023
SP  - 1029
EP  - 1040
VL  - 361
PB  - Académie des sciences, Paris
DO  - 10.5802/crmath.462
LA  - en
ID  - CRMATH_2023__361_G6_1029_0
ER  - 
%0 Journal Article
%A Bruno Després
%T A convergent Deep Learning algorithm for approximation of polynomials
%J Comptes Rendus. Mathématique
%D 2023
%P 1029-1040
%V 361
%I Académie des sciences, Paris
%R 10.5802/crmath.462
%G en
%F CRMATH_2023__361_G6_1029_0
Bruno Després. A convergent Deep Learning algorithm for approximation of polynomials. Comptes Rendus. Mathématique, Volume 361 (2023), pp. 1029-1040. doi : 10.5802/crmath.462. https://comptes-rendus.academie-sciences.fr/mathematique/articles/10.5802/crmath.462/

[1] François Chollet Deep Learning with Python, Manning Shelter Island, 2018

[2] Ingrid Daubechies; Ronald DeVore; Simon Foucart; B. Hanin; Guergana Petrova Nonlinear Approximation and (Deep) ReLU Networks, Computing, Volume 55/1 (2022), pp. 127-172 | Zbl

[3] Bruno Després Neural Networks and Numerical Analysis, Walter de Gruyter, 2022 | DOI

[4] Bruno Després; Matthieu Ancellin A functional equation with polynomial solutions and application to Neural Networks, C. R. Math. Acad. Sci. Paris, Volume 358 (2020) no. 9-10, pp. 1059-1072 | Numdam | MR | Zbl

[5] Ronald DeVore Nonlinear Approximation by Deep ReLU Neural Networks (2019) (https://devore2019.sciencesconf.org/resource/page/id/1)

[6] Ronald DeVore; George G. Lorentz Constructive Approximation, Grundlehren der Mathematischen Wissenschaften, 303, Springer, 1993 | DOI

[7] Ian Goodfellow; Yoshua Bengio; Aaron Courville Deep Learning, MIT Press, 2016 (http://www.deeplearningbook.org)

[8] Masayoshi Hata; Masaya Yamaguti The Takagi Function and Its Generalization, Japan J. Appl. Math., Volume 1 (1984) no. 1, pp. 83-199 | MR | Zbl

[9] Juncai He; Lin Li; Jinchao Xu; Chunyue Zheng ReLU Deep Neural Networks and Linear Finite Elements, J. Comput. Math., Volume 38 (2020) no. 3, pp. 502-527 | MR | Zbl

[10] Jean-Baptiste Hiriart-Urruty; Claude Lemaréchal Convex analysis and minimization algorithms. I, 305, Springer, 1993

[11] Yann Le Cun Quand la machine apprend, Odile Jacob, 2019

[12] Bo Li; Shanshan Tang; Haijun Yu Better approximations of high dimensional smooth functions by deep neural networks with rectified power units, Commun. Comput. Phys., Volume 27 (2020) no. 2, pp. 379-411 | MR | Zbl

[13] Joost A. A. Opschoor; Philipp C. Petersen; Christoph Schwab Deep ReLU networks and high-order finite element methods, Anal. Appl., Singap., Volume 18 (2020) no. 5, pp. 715-770 | DOI | MR | Zbl

[14] Gabrel Turinici The convergence of the Stochastic Gradient Descent (SGD) : a self-contained proof (2021) | arXiv

[15] E. Weinan; Stephan Wojtowytsch Representation formulas and pointwise properties for Barron functions (2021) | arXiv

[16] Dmitry Yarotsky Error bounds for approximations with deep ReLU networks, Neural Netw., Volume 94 (2017), pp. 103-114 | DOI | Zbl

Cité par Sources :

Commentaires - Politique

Ces articles pourraient vous intéresser

A functional equation with polynomial solutions and application to Neural Networks

Bruno Després; Matthieu Ancellin

C. R. Math (2020)