For citation:
Obukhov A. D. Method of automatic search for the structure and parameters of neural networks for solving information processing problems. Izvestiya of Saratov University. Mathematics. Mechanics. Informatics, 2023, vol. 23, iss. 1, pp. 113-125. DOI: 10.18500/1816-9791-2023-23-1-113-125, EDN: QQEIGU
Method of automatic search for the structure and parameters of neural networks for solving information processing problems
Neural networks are actively used in solving various applied problems of data analysis, processing and generation. When using them, one of the difficult stages is the selection of the structure and parameters of neural networks (the number and types of layers of neurons, activation functions, optimizers, and so on) that provide the greatest accuracy and, therefore, the success of solving the problem. Currently, this issue is being solved by analytical selection of the neural network architecture by a researcher or software developer. Existing automatic tools (AutoKeras, AutoGAN, AutoSklearn, DEvol and others) are not universal and functional enough. Therefore, within the framework of this work, a method of automatic search for the structure and parameters of neural networks of various types (multilayer dense, convolutional, generative-adversarial, autoencoders, and others) is considered for solving a wide class of problems. The formalization of the method and its main stages are presented. The approbation of the method is considered, which proves its effectiveness in relation to the analytical solution in the selection of the architecture of the neural network. A comparison of the method with existing analogues is carried out, its advantage is revealed in terms of the accuracy of the formed neural networks and the time to find a solution. The research results can be used to solve a large class of data processing problems for which it is required to automate the selection of the structure and parameters of a neural network.
- Beskrovny A. S., Bessonov L. V., Ivanov D. V., Kirillova I. V., Kossovich L. Yu. Using the mask-RCNN convolutional neural network to automate the construction of two-dimensional solid vertebral models. Izvestiya of Saratov University. Mathematics. Mechanics. Informatics, 2020, vol. 20, iss. 4, pp. 502–516 (in Russian). https://doi.org/10.18500/1816-9791-2020-20-4-502-516
- Li H., Yuan D.,Ma X., Cui D., Cao L. Genetic algorithm for the optimization of features and neural networks in ECG signals classification. Scientific Reports, 2017, vol. 7, iss. 1, pp. 1–12. https://doi.org/10.1038/srep41011
- He X., Zhao K., Chu X. AutoML: A survey of the state-of-the-art. Knowledge-Based Systems, 2021, vol. 212, pp. 106622. https://doi.org/10.1016/j.knosys.2020.106622
- Jin H., Song Q., Hu X. Auto-keras: An efficient neural architecture search system. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019, pp. 1946–1956. https://doi.org/10.1145/3292500.3330648
- Real E., Aggarwal A., Huang Y., Le Q. V. Regularized evolution for image classifier architecture search. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, vol. 33, pp. 4780–4789. https://doi.org/10.1609/aaai.v33i01.33014780
- Budjac R., Nikmon M., Schreiber P., Zahradnikova B., Janacova D. Automated machine learning overview. Research Papers Faculty of Materials Science and Technology Slovak University of Technology, 2019, vol. 27, iss. 45, pp. 107–112. https://doi.org/10.2478/rput-2019-0033
- Feurer M., Klein A., Eggensperger K., Springenberg J., Blum M., Hutter F. Efficient and robust automated machine learning. Automated Machine Learning, 2019, pp. 113–134. https://doi.org/10.1007/978-3-030-05318-5_6
- Le Q., Zoph B. Using machine learning to explore neural network architecture. Google Research Blog, 2017. Available at: https://research.googleblog.com/2017/05/using-machine-learning-to-explor... (accessed 5 April 2022).
- Gong X., Chang S., Jiang Y., Wang Z. Autogan: Neural architecture search for generative adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 3224–3234. https://doi.org/10.1109/ICCV.2019.00332
- Liu C., Zoph B., Neumann M., Shlens J., Hua W., Li L. J., Murphy K. Progressive Neural Architecture Search. In: Ferrari V., Hebert M., Sminchisescu C., Weiss Y. (eds.) Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, vol. 11205. Cham, Springer, 2018, pp. 19–34. https://doi.org/10.1007/978-3-030-01246-5_2
- Cai H., Gan C., Han S. Once for all: Train one network and specialize it for efficient deployment. arXiv preprint, arXiv:1908.09791. 2019.
- Truong A., Walters A., Goodsitt J., Hines K., Bruss C. B., Farivar R. Towards automated machine learning: Evaluation and comparison of AutoML approaches and tools. 2019 IEEE 31st International Conference on Tools with Artificial Intelligence, 2019, pp. 1471–1479. https://doi.org/10.1109/ICTAI.2019.00209
- Moen E., Bannon D., Kudo T., Graf W., Covert M., Van Valen D. Deep learning for cellular image analysis. Nature Methods, 2019, vol. 16, iss. 12, pp. 1233–1246. https://doi.org/10.1038/s41592-019-0403-1
- Obukhov A., Krasnyanskiy M. Quality assessment method for GAN based on modified metrics inception score and Frechet inception distance. Proceedings of the Computational Methods in Systems and Software, 2020, pp. 102–114. https://doi.org/10.1007/978-3-030-63322-6_8
- MNIST handwritten digit database, Yann LeCun, Corinna Cortes and Chris Burges. Available at: http://yann.lecun.com/exdb/mnist/ (accessed 5 April 2022).
- UCI Machine Learning Repository. Available at: https://archive.ics.uci.edu/ml/datasets.php (accessed 5 April 2022).
- Obukhov A., Siukhin A., Dedov D. The model of the automatic control system for a treadmill based on neural networks. 2020 International Multi-Conference on Industrial Engineering and Modern Technologies (FarEastCon), 2020, pp. 1–5. https://dx.doi.org/10.1109/FarEastCon50210.2020.9271589
- Obukhov A., Krasnyanskiy M. Neural network method of data processing and transmission in adaptive information systems. Vestnik Udmurtskogo Universiteta. Matematika. Mekhanika. Komp’yuternye Nauki, 2021, vol. 31, iss. 1, pp. 149–164 (in Russian). https://doi.org/10.35634/vm210111
- 1137 reads