What is gradient descent?

Gradient descent is an iterative optimization algorithm used to find the minimum of a function. In machine learning, it's commonly used to train models by minimizing the cost function, which represents the error between the model's predictions and the actual values. The algorithm iteratively adjusts the model's parameters (e.g., weights and biases in a neural network) in the direction of the steepest decrease of the cost function.