Lecture

Particle Swarm Optimization

Multi-disciplinary Design Optimization

by

Ramkumar

ramkumars.24@res.iist.ac.in

Department of Aerospace Engineering,

Indian Institute of Space science and Technology,

Thiruvananthapuram, Kerala - 695547

2026-03-24

Content

  1. Conceptual Example and introduction

  2. PSO algorithm and its mathematics

  3. Advantages, limitations and variants

  4. Assignment problem

  5. References

Conceptual Example

Shop Locations

PSO Example

Himmelblau function

\[ f(x,y) = \left(x^2+y-11\right)^2+\left(x+y^2-7\right)^2 \]

  • It has four minima points
    • \(x = 3\), \(y = 2\)
    • \(x = -2.8\), \(y = 3.13\)
    • \(x=-3.77\), \(y=-3.28\)
    • \(x=3.58\), \(y=-1.84\)

Key points on PSO algorithm

  • Swarm intelligence-based meta-heuristic evolutionary algorithm. It is inspired from the nature, like a flock of birds, a school of fish etc…

  • Introduced by Kennedy and Eberhart in 1995 [1]

  • Swarm intelligence: the particles are not only communitcate with one another, they also influence one another’s behavior and equipped with a memory-like feature to keep track of best position it encountered thus far

  • Meta-heuristic: A strategy that guides the search process, rather than directly solving the problem

  • Each particle will carry two aspiration points: the best point visited by the swarm thus far, and the best point previously encountered by the said particle.

  • This enables to reach optimum solution much quicker than non-swarm intelligence-based, population based algorithms like the genetic algorithm.

  • PSO’s computational algorithm is based on simple and primitive vector-based mathematics only.

  • It is a parallelizable, gradient-free algorithm with less memory requirements and is independent of problem type: linear/non-linear, continuous/discrete etc…

  • It can yield a fast, global and mostly suboptimal solution to the given problem

PSO algorithm and its mathematics

PSO algorithm

There are three stages

  1. Initialization stage

  2. Searching stage

  3. Termination stage

PSO - Initialization stage - 1

  • Let, \(N\) - number of dimensions in the design space

  • Let, \(X\) - denotes a particle array

    \[ \begin{aligned} X = \begin{bmatrix} x_1 & x_2 & \dots & x_N \end{bmatrix} \end{aligned} \]

  • Let, \(M\) - denotes population size i.e. number of particles. And let \(pop\) denotes the population, then

    \[ \begin{aligned} pop = \begin{bmatrix} X_1 \\ X_2 \\ \vdots \\ X_M \end{bmatrix} = \begin{bmatrix} x_{1,1} & x_{1,2} & \dots & x_{1,j} & \dots & x_{1,N} \\ x_{2,1} & x_{2,2} & \dots & x_{2,j} & \dots & x_{2,N} \\ & & & \vdots & \\ x_{i,1} & x_{i,2} & \dots & x_{i,j} & \dots & x_{i,N} \\ & & & \vdots & \\ x_{M,1} & x_{M,2} & \dots & x_{M,j} & \dots & x_{M,N} \\ \end{bmatrix}_{M\times N} \end{aligned} \]

  • All \(x_{i,j}\) are randomly initialized

PSO - Initialization stage - 2

  • Each particle has two aspiration points in the design space

    • \(P_{best}\) - best position found by the particle

      \[ \begin{aligned} P_{best,i} = \begin{bmatrix} p_{i,1} & p_{i,2} & \dots & p_{i,N} \end{bmatrix} \ \ \ \forall i \end{aligned} \]

    • \(G_{best}\) - best position found globally by the swarm

      \[ \begin{aligned} G_{best} = \begin{bmatrix} g_{1} & g_{2} & \dots & g_{N} \end{bmatrix} \end{aligned} \]

    • Initially, \[ \begin{aligned} \mathbf{P}_{best} = \begin{bmatrix} P_{best,1} \\ P_{best,2} \\ \vdots \\ P_{best,M} \end{bmatrix}_{M\times N} = pop \end{aligned} \]

PSO - Searching stage - 1

  • The idea behind the PSO algorithm is to coordinate the motions of the particles as they enumerate the search space

  • The motion of each particle is captured by its velocity vector \(V_i\)

    \[ \begin{aligned} V_i = \begin{bmatrix} v_{i,1} & v_{i,2} & \dots & v_{i,N} \end{bmatrix} \ \ \ \forall i \end{aligned} \]

  • The velocity of each particle along each dimension in design space has to be updated during every iteration using

    \[ v_{i,j}^{new} = \omega \times v_{i,j} + C_1 \times r_1 \times \left(p_{i,j} - x_{i,j}\right) + C_2 \times r_2 \times \left(g_j - x_{i,j}\right) \ \ \forall i,j \]

  • where

    • \(\omega\) - inertia of particle
    • \(C_1,C_2\) - Cognitive and Social parameters, respectively
    • \(r_1,r_2\) - random variables between 0 and 1

  • \(C_1, C_2\) are the model parameters of PSO algorithm

PSO - Searching stage - 2

  • \(\omega\) - inertia of each particle, a non-zero positive number is dynamically adjusted by the algorithm to reduce over iterations

  • \(C_1\) - Cognitive parameter, a non-zero positive number that determines how much weigtage to be given for personal best position found by each particle

  • \(C_2\) - Social parameter, a non-zero positive number that determines how much weightage to be give for the global best position found by the swarm by each particle

  • \(r_1,r_2\) - random numbers introducing stochastic nature into the process

  • After computing new velocitiesm, the position of each particle along each dimension in the design space is updated as

    \[ x_{i,j}' = x_{i,j} + v_{i,j}^{new} \ \ \ \forall i,j \]

    \[ \begin{aligned} X_{i,j}^{new} = \begin{bmatrix} x'_{i,1} & x_{i,2}' & \dots & x_{i,N}' \end{bmatrix} \ \ \ \forall i \end{aligned} \]

PSO - Termination stage

  • There is no explicitly defined, unique termination mechanism

  • Some mechanisms from literature are

    • Limiting number of iterations
    • Limiting run time
    • Terminating if no significant improvement made to \(G_{best}\) solution

  • Another mechanism to make the algorithm to stop is the Inertia reduction

  • Inertia \(\omega\) is popularly reduced in two ways

    • Linear method \[ \omega_t = \omega_0 - \left[\left(\omega_T - \omega_0\right)\times\frac{t}{T}\right] \ \ \ \forall t \]

    • where \(\omega_0\) and \(\omega_T\) are the initial and final user-defined inertia values

    • Geometric method \[ \omega_t = \omega_0 \times \gamma^t \ \ \ \forall t, \ \ \ \gamma \in (0,1) \]

PSO - overall steps in the algorithm

  1. Define the objective function \(f_{obj}(.)\)

  2. Define number of particles in the swarm \(M\)

  3. Define the termination criterion

  4. Define the initial inertia of particles \(\omega_0\), Cognitive parameter \(C_1\) and Social parameter \(C_2\)

  5. Initialize the particle positions randomly and uniformly in the design space

  6. Compute the \(P_{best}\) and \(G_{best}\) arrays by evaluating the \(f_{obj}(.)\) over all the particles’ positions

  7. Compute the velocity vector of each particle with updated \(P_{best}\) and \(G_{best}\)

  8. Update the position of each particle with computed velocity

  9. Repeat steps 6, 7 and 8 until the termination criterion is satisfied

  10. Return the \(G_{best}\) array as the final design variable values.

PSO - Algorithm / pseudo code

Advantages, limitations and variants

PSO - Advantages and limitations

  • Advantages:
    • Gradient-free
    • Global optimization algorithm
    • Simple mathematics with highly parallelizable steps
  • Limitations:
    • Mostly provides sub-optimal solution
    • Solution depends on number of particles. Larger the population, the better.
    • Requires parameter selection and tuning: \(\omega, C_1, C_2,\) which doesn’t have an explicit rule

PSO - Variants and applications

  • There are many variants of PSO algorithm from the literature, some of them are

    • Comprehensive learning PSO [2]
    • Adaptive PSO [3]
    • Chaotic PSO [4]

  • PSO is applied in almost all of the scientific fields, some examples are

    • Agricultural engineering [5]
    • Energy sector [5]
    • Computational science [6]
    • Mechanical engineering [7]

Main reference

The main reference for this algorithm is: [8]

Assignment problem

  • Take one of the functions from here

  • Develop a computer program, with any programming language, applying the PSO algorithm on to the chosen function

  • Create a brief report with results and computer program, and submit to ramkumars.24@res.iist.ac.in

References

[1]
Kennedy, J., and Eberhart, R., “Particle Swarm Optimization,” Vol. 4, 1995, pp. 1942–1948.
[2]
Liang, J. J., Qin, A. K., Suganthan, P. N., and Baskar, S., “Comprehensive Learning Particle Swarm Optimizer for Global Optimization of Multimodal Functions,” IEEE transactions on evolutionary computation, Vol. 10, No. 3, 2006, pp. 281–295.
[3]
Zhan, Z.-H., Zhang, J., Li, Y., and Chung, H. S.-H., “Adaptive Particle Swarm Optimization,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), Vol. 39, No. 6, 2009, pp. 1362–1381.
[4]
Alatas, B., Akin, E., and Ozer, A. B., “Chaos Embedded Particle Swarm Optimization Algorithms,” Chaos, Solitons & Fractals, Vol. 40, No. 4, 2009, pp. 1715–1734.
[5]
Noory, H., Liaghat, A. M., Parsinejad, M., and Haddad, O. B., “Optimizing Irrigation Water Allocation and Multicrop Planning Using Discrete PSO Algorithm,” Journal of Irrigation and Drainage Engineering, Vol. 138, No. 5, 2012, pp. 437–444.
[6]
De Falco, I., Della Cioppa, A., and Tarantino, E., “Facing Classification Problems with Particle Swarm Optimization,” Applied Soft Computing, Vol. 7, No. 3, 2007, pp. 652–658.
[7]
Chen, H., Fan, D. L., Fang, L., Huang, W., Huang, J., Cao, C., Yang, L., He, Y., and Zeng, L., “Particle Swarm Optimization Algorithm with Mutation Operator for Particle Filter Noise Reduction in Mechanical Fault Diagnosis,” International journal of pattern recognition and artificial intelligence, Vol. 34, No. 10, 2020, p. 2058012.
[8]
Zolghadr-Asli, B., “Computational Intelligence-Based Optimization Algorithms: From Theory to Practice,” CRC Press, 2023.