Robust modification of PSO algorithm based on M-means for solving linear regression

Linear regression is a fundamental data analysis tool, but it can be greatly affected by outliers and noise, which can lead to distorted coefficients and a reduction in forecast quality. Classical heuristic PSO (Particle Swarm Optimization) and Jaya algorithms demonstrate good performance on smooth loss functions, but they are sensitive to outliers when applying the standard mean squared error (MSE). This creates a need for simple but effective modifications that would maintain global search capabilities without complicating the basic algorithm'sstructure.

Aim. The study is to develop and subject to experimental evaluation modification of PSO algorithm (PSO-Robust) that ensures the robustness of linear regression to outliers without complicating the core algorithm or introducing additional hyperparameters.

Methods. Algorithmic idea: to keep the standard PSO equations of motion; interfering only with the fitness function. Instead of the root mean squire loss, an M-means function based on the Huber function is used, with adaptive weights that reduce the contribution of outliers. Experiments have been conducted on synthetic data with 15 % and 25 % outliers, with the same hyperparameters for all compared algorithms and 30 independent runs. Evaluation by mean and median test errors, as well as by dispersion estimation (variance, interquartile range). Visual analysis – boxplot of error distributions and regression lines. Evaluation by mean and median test errors, as well as dispersion (variance, interquartile range). Visual analysis – boxplot of error distributions and regression lines.

Results. PSO-Robust consistently outperforms classic PSO and Jaya in terms of mean and median test errors. The results show a smaller spread (variance). The visual analysis confirms reduced sensitivity to outliers (more compact box plots, more consistent regression lines).

Conclusion. The PSO-Robust modification demonstrates consistent superiority over the original algorithms in accuracy and robustness, producing more compact boxplots andless distorted regression lines. The proposed approach combines simplicity of implementation and robustness, increasing the reliability of regression in the case with heterogeneous data. Future developments include extending the method to multivariate and nonlinear models, as well as exploring alternative robust loss functions.