Kolmogorov-Arnold Neural Network's optimization and architecture analysis

Abstract

This study aims to conduct an in-depth analysis of the Kolmogorov-Arnold neural network architecture and its functioning principles, comparing it with the Multi-Layer Perceptron and identifying possible optimization paths. The Kolmogorov-Arnold Network, introduced in May 2024, retains a fully connected structure but introduces trainable activation functions on the edges instead of fixed activation functions at the nodes. This allows for increased modeling accuracy and efficiency, with splines helping KAN networks learn and adapt in a controlled manner.