What is SA?

SA, or Simulated Annealing, is a probabilistic technique used for approximating the global optimum of a given function. It is often used when the search space is discrete and combinatorial, such as in optimization problems in computer science, operations research, and machine learning. The algorithm derives its name from the annealing process in metallurgy, where a material is heated and then slowly cooled to reduce defects and achieve a stable, low-energy state. In SA, the algorithm explores the search space by making random moves, accepting moves that improve the objective function (like reducing cost or increasing performance) and also probabilistically accepting moves that worsen the objective function. This acceptance of worse solutions helps the algorithm escape local optima and find a better global solution. The probability of accepting a worse solution decreases as the 'temperature' parameter decreases over time, gradually reducing the exploration to focus on exploitation around the best-found solutions.