Generator Expression

A generator expression is a concise, inline construct that yields a generator object. It utilizes lazy evaluation to produce items one at a time on demand, rather than computing and storing an entire sequence in memory simultaneously.

Syntax

The syntax is identical to a list comprehension, but it is enclosed in parentheses () instead of square brackets [].

(expression for item in iterable if condition)

When passed as the sole argument to a function, the enclosing parentheses can be omitted:

sum(x**2 for x in range(10))

Technical Mechanics

• Evaluation Semantics (Immediate vs. Lazy): The outermost iterable is evaluated eagerly at definition time to obtain its iterator. However, the iteration process, any nested for clauses, if conditions, and the final yielded expression are evaluated lazily (deferred until the generator is explicitly consumed).
• Iterator Protocol: The resulting object implements the Python iterator protocol. It possesses both an __iter__() method (returning itself) and a __next__() method (computing and returning the next value).
• State Suspension: Between calls to __next__(), the generator suspends its execution state. It retains the current bindings of local variables, the instruction pointer, and the internal state of the iterator it is consuming.
• Exhaustibility: Generator objects are single-pass iterators. Once the underlying iterable is fully consumed, the generator raises a StopIteration exception. It cannot be reset or iterated over a second time.

Code Visualization

The following example demonstrates the creation, evaluation semantics, and consumption of a generator expression:

def get_iterable():
    print("Evaluating outermost iterable...")
    return range(3)


# 1. Creation: The outermost iterable is evaluated eagerly.

# The iteration and the expression (x * 2) are deferred.
gen = (x * 2 for x in get_iterable())

# Console output immediately shows: "Evaluating outermost iterable..."


# 2. Type verification
print(type(gen))  # <class 'generator'>


# 3. Manual consumption via __next__()
print(next(gen))  # Outputs: 0
print(next(gen))  # Outputs: 2


# 4. Implicit consumption via iteration
for remaining in gen:
    print(remaining)  # Outputs: 4


# 5. Exhaustion: The generator is now empty

# next(gen) would now raise StopIteration

Memory Complexity

Because a generator expression yields items one by one, its memory footprint is typically $O(1)$ relative to the size of the iterable. This is in direct contrast to a list comprehension, which has an $O(N)$ memory footprint because it eagerly evaluates the expression and allocates memory for the entire resulting array before returning it to the caller.

Master Python with Deep Grasping Methodology!

Learn More