Python Generators vs. Closures

Generators and closures are two special function types that can come in handy under the right circumstances. To illustrate the basic usage and differences I will use some example code.


# python

def inc_by_10_generator():
    # returns infinite series of 10's starting at 10 - recalls position
    i = 10
    while True:
        yield i
        i += 10

def inc_by_10_closure():
    # returns a function that increments a value by 10
    def fxn(val):
        return val + 10
    return fxn

def inc_by_x_closure(x=1):
    # increment by a dynamic variable
    def fxn(val):
        return val + x
    return fxn

if __name__ == "__main__":
    print "\n*** Inc by 10 Generator ***\n"
    by_10 = inc_by_10_generator()
    for i in range(3):
        print next(by_10)

    print "\n*** Inc by 10 Generator (a different one)***\n"
    by_10_2 = inc_by_10_generator()
    for i in range(3):
         print next(by_10_2)

    print "\n*** Inc by 10 Closure (static inc)***\n"
    by_10_closure = inc_by_10_closure()
    val = 0
    for i in range(3):
        val = by_10_closure(val)
        print val

    print "\n*** Inc by 2 Closure (variable inc)***\n"
    val = 0
    by_2_closure = inc_by_x_closure(2)
    for i in range(3):
        val = by_2_closure(val)
        print val

    print "\n*** Inc by 3 Closure***\n"
    val = 0
    by_3_closure = inc_by_x_closure(3)
    for i in range(3):
        val = by_3_closure(val)
        print val

Before reading the explanation try running and investigating the code yourself. Ask yourself what are the similarities? What are the differences?


OK. That done. Let’s start with the generator function. What a generator does is create an iterator function for a defined, sequential series of output values. The example generator function, inc_by_10_generator, will produce an increasing, infinite, sequence of multiples of 10. As an iterator, the generator instance is called with next(<my_generator_instance>). It can also be used as if it was a list variable in a for loop – i.e. for i in by_10: 

A few important points:

  • The generator is instantiated from the generator function. It is not iterated directly (see the code).
  • The generator instance remembers its state.
  • Generators are faster and more efficient than creating long lists to iterate over, e.g. inc_by_10_generator is preferred over inc_by_10s = [ i*10 for i in range(100000)] because it need not remain resident in memory as large block, and is more reusable.
  • The generator instance remembers its state. An instance of a generator is independent of other instances.
  • A generator is usually built around a while or for loop.
  • A generator function uses yield to return its values instead of return. The yield simply means indicates the interrupt/continue point in the loop. When the generator instance is called again it will resume after the yield.

Now let’s get to closures.  Closures are generated functions, like generators, but they have a different purpose. A closure is a custom generated function that dynamically sets its defining state. Read the code for inc_by_10_closure and inc_by_x_closure to see what I mean by that. Unlike generators, closures use the return , just like a regular function. A closure is useful for doing things like configuring customized object instances, or creating a blind (i.e.  caller context does not need to understand its workings) handle for processing an object.

A few important points:

  • Like a generator, a closure is a generated function. You use the return value of the closure function as a function itself (see code).
  • Closures, unlike generators, are frozen after they are instantiated. A generator returns the next value in the sequence when it is called in its iteratory context. A closure, once produced, operates like any other regular function in python.
  • Closures can make debugging more difficult, and code hard to understand. Use them with purpose.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s