Python Generators

In Python, a generator is a function that produces a sequence of values, one at a time, when iterated over. Generators are a more memory-efficient and faster way to create iterators, especially when working with large sequences or when the values in the sequence need to be created on the fly.

Why Use a Generator

There are several reasons to use a generator in Python:

Memory efficiency: Generators do not store all the values in memory at once. They generate the values one at a time, on demand. This makes them more memory-efficient than lists or other data structures that store all the values in memory at once.

Performance: Generators are faster than lists or other data structures because they do not need to store all the values in memory at once. They can generate the values one at a time, on demand, which makes them faster to iterate over.

Readability: Generators can make your code more readable by reducing the amount of state that needs to be tracked. For example, if you have a list that you are filtering and transforming, you might need to store the transformed values in a separate list. With a generator, you can do the filtering and transformation in a single step, which can make your code more readable.

Generator Syntax

Generators in Python are defined using the def keyword, just like normal functions. However, instead of using the return keyword to return a value, generators use the yield keyword to yield a value. Here is an example of a generator function that yields even numbers between 0 and n:

def even_numbers(n):
    for i in range(n):
        if i % 2 == 0:
            yield i

To iterate over the generator, you can use a for loop:

for value in even_numbers(10):
    # do something with the value

You can also use the next function to iterate over the generator one value at a time:

value = next(even_numbers(10))

Difference between a normal function and a generator

At first glance, a generator function may look similar to a normal function. However, there are a few key differences between the two:

A normal function executes all the code in the function and returns a single value (or a tuple of values). A generator function also executes all the code in the function, but instead of returning a single value, it yields multiple values, one at a time, when iterated over.

When a normal function is called, it executes the code in the function from the beginning to the end. When a generator function is called, it does not execute the code in the function. Instead, it returns a generator object that can be used to execute the code in the function.

A normal function uses the return statement to return a value. A generator function uses the yield statement to yield a value.

Here is an example of a normal function that returns a list of even numbers between 0 and n:

def even_numbers(n):
    evens = []
    for i in range(n):
        if i % 2 == 0:
            evens.append(i)
    return evens

evens = even_numbers(10)
print(evens)  # [0, 2, 4, 6, 8]

Here is the same function, written as a generator:

def even_numbers(n):
    for i in range(n):
        if i % 2 == 0:
            yield i

evens = even_numbers(10)
print(evens)  # generator object

for even in evens:
    print(even)  # 0, 2, 4, 6, 8

Real life examples

Here are some examples of how you might use generators in real life:

Reading large files: If you have a large file that you need to process line by line, you can use a generator to read the file one line at a time, instead of reading the entire file into memory at once. This can save a lot of memory, especially if the file is very large.

def read_file(filename):
    with open(filename, 'r') as f:
        for line in f:
            yield line

for line in read_file('large_file.txt'):
    print(line)
Filtering and transforming data: Generators can be used to filter and transform data in a single step, which can make your code more readable.
```python
def even_numbers(numbers):
    for number in numbers:
        if number % 2 == 0:
            yield number

def square(numbers):
    for number in numbers:
        yield number ** 2

numbers = [1, 2, 3, 4, 5, 6]
even_squares = square(even_numbers(numbers))
print(list(even_squares))  # [4, 16, 36]

Generating infinite sequences: Generators can be used to generate infinite sequences, such as the Fibonacci sequence or an endless stream of random numbers.

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

for i, fib in enumerate(fibonacci()):
    print(fib)
    if i > 10:
        break

Best practices

Here are a few best practices for using generators in Python:

Use generators when you need to process a large sequence of values, or when the values in the sequence need to be created on the fly.

Use the generator expression syntax to create simple generators. For example, you can use the following syntax to create a generator that yields the even numbers between 0 and n:

evens = (i for i in range(n) if i % 2 == 0)

Use the yield from syntax to delegate the work of iterating over a sequence to another generator. This can make your code more readable.

def even_numbers(numbers):
    yield from (number for number in numbers if number % 2 == 0)

numbers = [1, 2, 3, 4, 5, 6]
evens = even_numbers(numbers)
print(list(evens))  # [2, 4, 6]