Perturbations and Phase Transitions in Swarm Optimization Algorithms

Abstract

Natural systems often exhibit chaotic behavior in their space-time evolution. Systems transiting between chaos and order manifest a potential to compute, as shown with cellular automata and artificial neural networks. We demonstrate that swarm optimization algorithms also exhibit transitions from chaos, analogous to a motion of gas molecules, when particles explore solution space disorderly, to order, when particles follow a leader, similar to molecules propagating along diffusion gradients in liquid solutions of reagents. We analyze these `phase-like' transitions in swarm optimization algorithms using recurrence quantification analysis and Lempel-Ziv complexity estimation. We demonstrate that converging iterations of the optimization algorithms are statistically different from non-converging ones in a view of applied chaos, complexity and predictability estimating indicators. An identification of a key factor responsible for the intensity of their phase transition is the main contribution of this paper. We examined an optimization as a process with three variable factors -- an algorithm, number generator and optimization function. More than 9.000 executions of the optimization algorithm revealed that the nature of an applied algorithm itself is the main source of the phase transitions. Some of the algorithms exhibit larger transition-shifting behavior while others perform rather transition-steady computing. These findings might be important for future extensions of these algorithms.