PR402 - 1 : Neural Network Approximation of Refinable Functions

Review paper : Neural Network Approximation of Refinable Functions

(ICML 2021 Accepted Paper / Daubechies et al.)

Link - https://arxiv.org/abs/2107.13191

Summary

Set of functions that can be approximated with exponential accuracy (in terms of the number of parameters used) includes (1) smooth functions (polynomials and analytic functions) / (2) rough functions (Weierstrass function). 

This paper adds 'refinable functions' to the rough functions, which is used in the wavelets construction.

Main Flow

It aims to prove that refinable functions can be expressed by neural networks. 

It can be described as follows:

CPwL above means Continuous Piecewise Linear functions on R.

They show the above statements first for CPwL function (hat), and generalize the statement.

CPwL hat  is as above.

The first property, CPwL hat can be expressed as neural network's output, is shown in "Neural Network Approximation" (Acta Numerica, Devore et al.)

The rest of the paper is for generalization to the Matrix form, and to refinnable functions, wavelet approximation.

Conclusion

This paper mathematically shows that refinable functions is in the set of functions can be expressed by neural network's output. I think it is not quite useful in many  domains. However, for some part of temporal data, it may give some intuition for architecture constructions.