There's this thing in Python that always trips me up. It's not that tricky, once you know what you're looking for, but it's not intuitive for me, so I do forget. It's that shadowing a variable can sometimes give you an UnboundLocalError!

It happened to me last week while working on a workflow engine with a coworker. We were refactoring some of the code.

I can't share that code (yet?) so let's use a small example that illustrates the same problem. Let's start with some working code, which we had before our refactoring caused a problem. Here's some code that defines a decorator for a function, which will trigger some other functions after it runs.

def trigger(*fns):
  """After the decorated function runs, it will
  trigger the provided functions to run
  sequentially.

  You can provide multiple functions and they run
  in the provided order.

  This function *returns* a decorator, which is
  then applied to the function we want to use to
  trigger other functions.
  """

  def decorator(fn):
    """This is the decorator, which takes in a
    function and returns a new, wrapped, function
    """
    fn._next = fns

    def _wrapper():
      """This is the function we will now invoke
      when we call the wrapped function.
      """
      fn()
      for f in fn._next:
        f()

    return _wrapper

  return decorator

The outermost function has one job: it creates a closure for the decorator, capturing the passed in functions. Then the decorator itself will create another closure, which captures the original wrapped function.

Here's an example of how it would be used^[1].

def step_2():
  print("step 2")

def step_3():
  print("step 3")

@trigger(step_2, step_3)
def step_1():
  print("step 1")

step_1()

This prints out

step 1
step 2
step 3

Here's the code of the wrapper after I made a small change (omitting docstrings here for brevity, too). I changed the for loop to name the loop variable fn instead of f, to shadow it and reuse that name.

  def decorator(fn):
    fn._next = fns

    def _wrapper():
      fn()
      for fn in fn._next:
        fn()

And then when we ran it, we got an error!

UnboundLocalError: cannot access local variable 'fn' where it is not associated with a value

But why? You look at the code and it's defined. Right out there, it is bound. If you print out the locals, trying to chase that down, you'll see that there does not, in fact, exist fn yet.

The key lies in Python's scoping rules. Variables are defined for their entire scope, which is a module, class body, or function body. If you define a variable within a scope, anywhere inside a function, then that variable has that name as its own for the entire scope.

The docs make this quite clear:

If a name binding operation occurs anywhere within a code block, all uses of the name within the block are treated as references to the current block. This can lead to errors when a name is used within a block before it is bound. This rule is subtle. Python lacks declarations and allows name binding operations to occur anywhere within a code block. The local variables of a code block can be determined by scanning the entire text of the block for name binding operations. See the FAQ entry on UnboundLocalError for examples.

This comes up in a few other places, too. You can use a loop variable anywhere inside the enclosing scope, for example.

def my_func():
  for x in [1,2,3]:
    print(x)

  # this will still work!
  # x is still defined!
  print(x)

So once I saw an UnboundLocalError after I'd shadowed it, I knew what was going on. The name was used by the local for the entire function, not just after it was initialized! I'm used to shadowing being the idiomatic thing in Rust, then had to recalibrate for writing Python again. It made sense once I remembered what was going on, but I think it's one of Python's little rough edges.

1. This is not how you'd want to do it in production usage, probably. It's a somewhat contrived example for this blog post. ↩