Python Generators

Table of Contents

In the last lesson, we learned about Python Iterators. Now let us create simpler, memory-efficient iterators with generators and yield.

What Is a Generator in Python?

A generator is a function that produces a sequence of values. It uses the yield keyword instead of return. When called, it returns a generator object. We can iterate over it, just like a list.

Generators are lazy. They produce values one at a time, only when asked. That makes them memory efficient.

Why Do We Need Generators in Python?

Generators are useful when we work with large sequences. If we create a list of a million numbers, it uses a lot of memory. But a generator produces one number at a time. It only uses memory for the current value.

Generators also make code simpler. We do not need to write a class with __iter__ and __next__. We just write a function.

How to Create a Generator in Python

We create a generator by using yield inside a function.

def count_up_to(limit):
    current = 1
    while current <= limit:
        yield current
        current += 1


for num in count_up_to(3):
    print(num)

Output 

1
2
3

The function count_up_to is a generator. When we call it, it returns a generator object. We can loop over it, just like a list.

How Generators Work in Python

When we call a generator function, it does not run the code immediately. It returns a generator object. The code runs when we iterate over it.

Each time we ask for a value, the generator runs until it hits yield. It returns that value and pauses. The next time we ask, it resumes from where it left off.

def my_generator():
    print("Start")
    yield 1
    print("After first yield")
    yield 2
    print("After second yield")


gen = my_generator()
print(next(gen))
print(next(gen))

Output 

Start
1
After first yield
2

Notice how the print statements run only when we ask for the next value.

Generator Expressions in Python

We can also create generators using expressions, similar to list comprehensions. We use parentheses instead of square brackets.

numbers = [1, 2, 3, 4, 5]
squares = (n * n for n in numbers)


for square in squares:
    print(square)

Output 

1
4
9
16
25

That is a generator expression. It is a quick way to create a generator without writing a function.

A Real-Life Example – Reading a Large File

Let us read a large file line by line using a generator.

def read_lines(filename):
    with open(filename, "r") as file:
        for line in file:
            yield line.strip()


for line in read_lines("data.txt"):
    print(line)

This reads one line at a time. It does not load the whole file into memory. That is efficient for large files.

Another Example – Fibonacci Generator

def fibonacci(limit):
    a, b = 0, 1
    count = 0
    while count < limit:
        yield a
        a, b = b, a + b
        count += 1


for num in fibonacci(5):
    print(num)

Output 

0
1
1
2
3

That is clean and simple.

Common Mistakes with Generators in Python

Mistake 1: Using return instead of yield

def count_up_to(limit):
    current = 1
    while current <= limit:
        return current
        current += 1

This returns only the first value. It does not generate a sequence.

Fix: Use yield.

yield current

Mistake 2: Trying to reuse a generator

gen = count_up_to(3)
for num in gen:
    print(num)
for num in gen:
    print(num)

The second loop prints nothing. Because the generator is exhausted.

Fix: Create a new generator if needed.

Mistake 3: Forgetting that generators are lazy

def my_generator():
    print("Start")
    yield 1
    print("End")


gen = my_generator()
print("Before next")
print(next(gen))

Output 

Before next
Start
1

The code inside the generator does not run until we call next(). That is the lazy behavior.

Next, we will learn about Python Decorators and extend function behavior without changing the original function.

Previous Python Iterators Next Python Decorators