What is Deep Learning?

In the previous tutorial, we learned about Neural Networks.

We saw that a neural network can take some input, process it through different layers, and produce an output.

Now we need to understand:

What makes a neural network “deep”?

This is where Deep Learning comes in.

🧱 What is Deep Learning?

Deep Learning is a type of Machine Learning that uses neural networks with multiple layers to learn complex patterns from large amounts of data.

The word deep simply refers to having multiple layers in the neural network.

A very simple view is:

Let’s understand why these layers are useful.

🤔 Why Do We Need Multiple Layers?

Let’s take an example of image recognition.

Suppose we want a computer to recognise a car in an image.

A car can have many different shapes, colours, sizes and designs.

It would be difficult for one simple layer to understand the entire image.

Instead, different layers can gradually learn different levels of patterns.

For example:

This is one of the important ideas behind Deep Learning.

💡 A Simple Example

Imagine you want to recognise a person’s face.

The first layer might learn simple things such as:

Lines
Edges
Curves

The next layers can combine these patterns:

Eyes
Nose
Mouth

Further layers can combine them:

Face

So the network can gradually move from simple patterns to complex patterns.

This is why having multiple layers can be powerful.

🕸️ What Does a Deep Neural Network Look Like?

A simple neural network might have only a few layers:

A deep neural network can have many layers:

Real-world deep-learning models can contain many more layers.

🎛️ What is a Weight?

Now we need to understand one of the most important concepts in neural networks:

Weights.

Don’t worry—the idea is simpler than the name sounds.

A neural network has many connections between its neurons.

Each connection has a numerical value called a weight.

A simplified example:

The weights control how important each input is to the calculation.

For example, imagine predicting whether a student will pass an exam.

We might have:

Hours studied
Attendance
Previous marks

The model may learn that some of these inputs are more useful than others.

The weights represent these learned relationships.

⚙️ How Does the Model Learn the Weights?

This is the heart of Deep Learning.

Initially, the model does not know the correct weights.

It starts with values that are not yet useful.

Then it makes a prediction.

We compare the prediction with the correct answer.

Suppose:

Correct answer: Cat
Model prediction: Dog

The model has made a mistake.

We need a way to measure how wrong the prediction was.

This brings us to the concept of Loss.

🧱 What is Loss?

Loss is a number that represents how far the model’s prediction is from the correct answer.

For example:

But:

The exact calculation depends on the problem.

For now, simply remember:

Loss tells the model how wrong its prediction was.

The goal during training is to reduce the loss.

⚙️ How Does the Model Reduce the Loss?

This is where backpropagation comes in.

The name may sound complicated, but the basic idea is straightforward.

After making a prediction, the model calculates the error.

Then it works backwards through the network to determine which weights contributed to that error.

This process is called backpropagation.

🧱 What is Gradient Descent?

Another important term is Gradient Descent.

Don’t worry about the mathematics yet.

Think of it like trying to find the lowest point in a valley.

The model tries to adjust its weights in a direction that reduces the loss.

This process is commonly called gradient descent.

So during training, two concepts work together:

The process repeats many times.

🧱 Training a Deep Learning Model

Let’s put everything together.

Suppose we want to train a model to recognise cats.

We provide training images:

The training process looks roughly like this:

The model performs this process again and again.

Over time, the model’s predictions can improve.

🧱 What is an Epoch?

You may come across another important word:

Epoch.

An epoch means the model has gone through the entire training dataset once.

For example, suppose we have:

10,000 training examples

If the model processes all 10,000 examples once:

1 Epoch

If it processes the entire dataset five times:

5 Epochs

A simplified training process might look like:

However, more epochs do not always mean a better model. Training for too long can cause another problem called overfitting, which we’ll discuss separately.

🎛️ What is a Parameter?

We have already discussed weights.

Weights are examples of parameters in a neural network.

A parameter is a numerical value that the model learns during training.

For example:

A modern deep-learning model can have millions, billions, or even more parameters.

This is especially important when we start discussing Large Language Models.

🤔 Why Do Large Models Need So Many Parameters?

Consider language.

There are complex relationships between:

• Words
• Sentences
• Context
• Grammar
• Meaning
• Different topics
• Different languages

A model needs a large number of learned parameters to represent complex patterns.

This is one reason modern AI models can be extremely large.

But remember:

More parameters does not automatically mean a model is better.

Model architecture, training data, training process and many other factors also matter.

🤔 Why Did Deep Learning Become So Important?

Deep Learning became much more practical when several things came together.

1. Large amounts of data

The internet and digital systems created huge amounts of data.

2. Powerful hardware

GPUs and specialised AI hardware made large-scale mathematical calculations much faster.

3. Better algorithms

Researchers developed better techniques for training neural networks.

4. Better architectures

New neural-network architectures made it possible to solve increasingly difficult problems.

Together:

This eventually led to today’s modern AI systems.

🧱 Deep Learning in Real Life

Deep Learning is used in many areas.

Computer Vision

Understanding images and videos.

Speech Recognition

Language

Recommendation Systems

Generative AI

⚖️ Deep Learning vs Traditional Machine Learning

You may now wonder:

“Is Deep Learning different from Machine Learning?”

Yes, but Deep Learning is a type of Machine Learning.

A simplified relationship is:

Traditional Machine Learning can use algorithms such as:

• Decision Trees
• Linear Regression
• Logistic Regression
• Support Vector Machines
• Random Forests

Deep Learning primarily uses neural networks with multiple layers.

🤔 Why Is Deep Learning Important for Generative AI?

This is where our tutorial journey becomes interesting.

Modern Generative AI depends heavily on Deep Learning.

The progression looks roughly like this:

So when you use ChatGPT to generate an answer, there is a huge amount of deep-learning technology underneath it.

📌 A Note About the Human Brain

You may hear:

“Deep Learning works like the human brain.”

This is an oversimplification.

Neural networks were loosely inspired by biological neurons, but modern neural networks are mathematical models running on computers.

They don’t work exactly like the human brain.

It is better to say:

Neural networks are mathematical models loosely inspired by biological neural networks.

🧩 Key Points

Let’s summarise what we learned.

• Deep Learning

  Deep Learning is a type of Machine Learning that uses neural networks with multiple layers to learn complex patterns.

• Neural Network

• Weight

  A numerical value associated with connections in the network that the model learns during training.

• Loss

  A number that represents how wrong the model’s prediction is.

• Backpropagation

  A method used to determine how the model’s parameters should be adjusted based on the error.

• Gradient Descent

  A method used to adjust parameters in a direction that generally reduces the loss.

• Epoch

  One complete pass through the training dataset.

• Parameter

  A value learned by the model during training, such as a weight.

🧩 Our AI Journey So Far

We have now built a proper foundation:

But there is still one major piece missing.

We’ve talked about neural networks in general, but how did neural networks become capable of understanding language so well?

That brings us to one of the most important breakthroughs in modern AI:

We’ll start with the problem older neural networks had when dealing with sentences, and then understand attention and why the Transformer architecture changed AI completely.