Solving initial value problems using multilayer perceptron artificial neural networks

Document Type : Research Paper

Authors

1 Department of Mathematics, Faculty of Science, Tabriz Branch, Islamic Azad University, Tabriz, Iran.

2 Department of Applied Mathematics, Faculty of Mathematics, Statistics and Computer Science, University of Tabriz, Tabriz, Iran.

Abstract

This research introduces a novel approach using artificial neural networks (ANNs) to tackle ordinary differential equations (ODEs) through an innovative technique called enhanced back-propagation (EBP). The ANNs adopted in this study, particularly multilayer perceptron neural networks (MLPNNs), are equipped with tunable parameters such as weights and biases. The utilization of MLPNNs with universal approximation capabilities proves to be advantageous for ODE problem solving. By leveraging the enhanced back-propagation algorithm, the network is fine-tuned to minimize errors during unsupervised learning sessions. To showcase the effectiveness of this method, a diverse set of initial value problems for ODEs are solved and the results are compared against analytical solutions and conventional techniques, demonstrating the superior performance of the proposed approach.

Keywords

Main Subjects

References

  • [1] M. Abadi, (2015), Tensoflow.org. [Online]. Available: https://www.tensoflow.org/.
  • [2] A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, Automatic differentiation in machine learning: a survey, Journal of Marchine Learning Research, 18 (2018), 1-43.
  • [3] S. Chakraverty and S. Mall, Artificial neural networks for engineers and scientists: solving ordinary differential equations, CRC Press, (2017).
  • [4] M. Di Giovanni, D. Sondak, P. Protopapas, and M. Brambilla, Finding multiple solutions of odes with neural networks, In Combining Artificial Intelligence and Machine Learning with Physical Sciences, 2587 (2020), 1-7.
  • [5] R. Fletcher, Practical methods of optimization, John Wiley and Sons, (2013).
  • [6] A. H. Hadian-Rasanan, D. Rahmati, S. Gorgin, and K. Parand A single layer fractional orthogonal neural network for solving various types of Lane-Emden equation New Astronomy, 75 (2020), 101307.1–101307.14.
  • [7] Z. Hajimohammadi, F. Baharifard, and K. Parand, A new numerical learning approach to solve general Falkner–Skan model, Engineering with Computers, 38 (2022), 121–137.
  • [8] J. Han and A. Jentzen, Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations, Communications in mathematics and statistics, 5(4) (2017), 349-380.
  • [9] T. I. Haweel and T. N. Abdelhameed, Power series neural network solution for ordinary differential equations with initial conditions, In 2015 International Conference on Communications, Signal Processing, and their Applications (ICCSPA’15) IEEE., (2015), 1-5.
  • [10] I. E. Lagaris, A. Likas, and D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE transactions on neural networks, 9(5) (1998), 987-1000.
  • [11] X. Li-ying, W. Hui, and Z. Zhe-zhao, The algorithm of neural networks on the initial value problems in ordinary differential equations, In 2007 2nd IEEE Conference on Industrial Electronics and Applications. IEEE., (2007), 813-816.
  • [12] L. Lu, X. Meng, Z. Mao, and G. E. Karniadakis, DeepXDE: A deep learning library for solving differential equations, SIAM review, 63(1) (2021), 208-228.
  • [13] A. Malek and R. S. Beidokhti, Numerical solution for high order differential equations using a hybrid neural network optimization method, Applied Mathematics and Computation, 183(1) (2006), 260-271.
  • [14] S. Mall and S. Chakraverty, Application of Legendre neural network for solving ordinary differential equations, Applied Soft Computing, 43 (2016), 347-356.
  • [15] K. Parand, A. A. Aghaei, S. Kiani, T. Ilkhas Zadeh, and Z. Khosravi A neural network approach for solving nonlinear differential equations of Lane–Emden type, Engineering with Computers, 40 (2024), 953-969.
  • [16] K. Parand, A. A. Aghaei, M. Jani, and A. Ghodsi, Parallel LS-SVM for the numerical simulation of fractional Volterra’s population model, Alexandria Engineering Journal, 60(6) (2021), 5637-5647.
  • [17] K. Parand, A. A. Aghaei, M. Jani, and A. Ghodsi, A new approach to the numerical solution of Fredholm integral equations using least squares-support vector regression, Math. Comput. Simul., 180 (2021), 114–128.
  • [18] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, and S. Chintala, Pytorch: An imperative style, high-performance deep learning library, Advances in neural information processing systems, 32 (2019).
  • [19] W. Peng, J. Zhang, W. Zhou, X. Zhao, W. Yao, and X. Chen, IDRLnet: A physics-informed neural network library, arXiv preprint arXiv:2107.04320, (2021).
  • [20] M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational physics, 378 (2019), 686-707.
  • [21] Z. Sabir et al., Novel design of Morlet wavelet neural network for solving second order Lane-Emden equation, Math. Comput. Simul., 172 (2020) 1–14.
  • [22] A. Sacchetti, B. Bachmann, K. L¨offel, U. M. Ku¨nzi, and B. Paoli, Neural networks to solve partial differential equations: a comparison with finite elements, IEEE Access, 10 (2022), 32271-32279.
  • [23] S. Susmita Mall and S. Chakraverty, A novel Chebyshev neural network approach for solving singular arbitrary order Lane-Emden equation arising in astrophysics, In: Network: Computation in Neural Systems, 31(1-4) (2020), 142-165.
  • [24] L. S. Tan, Z. Zainuddin, and P. Ong, Solving ordinary differential equations using neural networks, In AIP Conference Proceedings, AIP Publishing LLC., 1974(1) (2018), 020070.
Computational Methods for Differential Equations
Volume 13, Issue 1
January 2025
Pages 13-24

Files

History

  • Receive Date: 09 October 2023
  • Revise Date: 04 March 2024
  • Accept Date: 27 April 2024

Share

How to cite

Statistics