A Beginner's Guide to Gradient Descent - Understanding the Core of Machine Learning Optimization

Sun, 22 Dec 2024 00:00:00 +0000

In this blog, I will try to explain what exactly is gradient descent and it’s importance in optimizing machine learning models. Gradient descent is an algorithm (just like million other algorithms that exists in computer science) that is designed to minimize a cost function.

What is a cost function?

Well, it is essentially the error between predicted and actual outputs. Which means if we want to increase our model’s accuracy, we need to somehow reduce this cost function and for that we’ve got gradient descent.

What is a Gradient?

A gradient is essentially the derivative of a function, which tells us how the output is affected with little variations in input. This gradient, helps us in adjusting model parameters (like weights and biases) to reduce the cost function.

Why Gradient Descent?

It helps us in minimizing the error in predictions, bringing the model closer to the actual data. It iteratively keep adjusting the model’s parameters until minimum cost function state is achieved.
This iterative process ensures the model learns and improves its performance.

What are Weights and Biases?

Weights: Weights (W) are parameters that define influence of an input on the model’s output. Biases: Biases (b) is an additional parameter that shifts the model’s prediction.

Blogs on

A Beginner's Guide to Gradient Descent - Understanding the Core of Machine Learning Optimization

What is a cost function?

What is a Gradient?

Why Gradient Descent?

What are Weights and Biases?