A brief explanation of Python Decorators

Introduction

Decorators are simply callables for decorating a function. It helps add new functionalities to a function without changing its original structure. So, what are Python decorators? In this article, we are going to learn the hows, whats, and whys of decorators. But before delving into Decorators we must get familiarized with certain concepts like first-class citizens, nesting of functions, closures, nonlocal scopes, etc. These topics are essential for understanding Python  Decorators. We will go through each topic one by one to ensure complete clarity.

This article was published as a part of the Data Science Blogathon

First-class citizens in Python

In Python, any object that can be assigned to a variable, passed as an argument, returned from a function will be considered as a first-class citizen. in short, there are almost no restrictions on their uses. Some of the examples are data types like int, floats, strings, etc, data structures like lists, tuples, etc. Python functions also satisfy the requirements for being a first-class citizen. This is a fundamental concept to understand the creation of Decorators.

Assigning functions to variables

Like any other object like lists, tuples, or dictionaries Python functions can also be assigned to variables. For example:

def greet(msg):
  return f'hello! {msg}'
var = greet    #function greet is assigned to var
var('Peter')   # var is called

output: 'hello! Peter'

Observe when we assigned function greet to var, we didn’t use parenthesis alongside greet. That is because we are not calling the function instead we are assigning the function greet to the variable var. Here, var points to function object referenced by greet. This means var now will be able to do whatever greet can do.

Functions as parameters

Function objects can also be passed as arguments to other functions. See the below example.

def upper_text(msg):
  return msg.upper()
def greet(func):
  var = func('hello! Peter')
  return var
greet(upper_text) #func was sent as a parameter to my_func

Output: 'HELLO! PETER'

In the above example observe when greet() was called upper_text was sent as an argument, which is a function itself. Now, inside the greet function, we called the function func() which corresponds to the function upper_text(). So upper_text() was executed and returned upper-lettered text.

Functions returning other functions

Functions can also be returned from other functions like any other object like lists, strings, etc. We will show that using a simple example

 def outer():
  def inner():
    return 'Freedom in thought'
  return inner
 var = outer()
 print(var())

output: 'Freedom in thought'

Here, we have taken two functions outer() and inner(). The former encloses the latter. The outer was executed and returned a reference of inner to var. Now, var has the same functionality as that of inner(). So, when var() was executed it returned the same string as inner().

Nesting Of Functions

Nesting of functions is nothing but defining a function within another function. The inner functions can access the variables of the outer function.

def outer(x):
  print(f'Hey! {x} this is outer function')
  def inner():
    print(f'Hey! {X} this is inner function')
  inner() 
outer('Jose')

output: Hey! Jose this is outer function
        HEy! Jose this is inner function

In the above example, outer() is the enclosing function while inner() is the nested function. inner() is defined inside the outer(). While calling outer we sent an argument string ‘Jose’. First-line was printed when outer() was called initially and the second line was called when inner() was called inside the outer() function. Observe that the nested function inner() is also able to access the variable x which is inside the outer() functions’ scope.

Hence, we learned that the nested functions are also able to access the objects that are present in the enclosing scope(outer() in this case). But the opposite isn’t true, Objects in the inner() scope can not be accessed by outer().

Nonlocal Scope in Python

The nonlocal scope comes into the picture when we deal with nested functions. The scope of nested functions is called nonlocal scopes and variables defined inside of them are called nonlocal variables. These variables can neither be in local scope nor in global.  Let’s understand this by an example.

def outer(x):
  def inner():
    x = 'Tom'
    print(f"{x}'s spider-man is the best")
  inner()
  print(f"{x}'s spider-man is the best")
outer('Tobey')

output: Tom's spider-man is the best
        Tobey's spider-man is the best

In the above example, we sent an argument ‘Tobey’ to the outer() function but inside the inner() we assigned ‘Tom’ to the variable x. So, when outer() was executed it printed a string with ‘Tobey’ but inner() printed a string with ‘Tom’. The variable x inside inner() is limited to inner function only. We can use the ‘nonlocal’ keyword to create nonlocal variables.

def outer(x):
  def inner():
    nonlocal x
    x = 'Tom'
    print(f"{x}'s spider-man is the best")
  inner()
  print(f"{x}'s spider-man is the best")
outer('tobey')

output: Tom's spider-man is the best
        Tom's spider-man is the best

Note: Changing the value of a nonlocal variable will also be reflected in the local scope.

In the above example change in the value of a variable in nonlocal scope also changed the variable in the local scope.

Closures in Python

In simple terms, closures are function objects that remember values in enclosing scopes even if they are not present in memory. Python functions form closures. These closures remember all the local variables and the environment within which the function was declared. See the below example.

 def outer(text):
   "enclosing function"
   def inner():
     "nested function"
     print(text)
   return inner
var = outer('food for brain')
var()

output: food for brain

As observed from the above example, the inner() function is limited to the scope of the outer() function only. But because of the closures, we could invoke it beyond its regular scope i.e. outside the outer() function.

The technique by which some data is attached to some code even after the execution of the original function is finished is called closures. Even if we delete the original function the values in the enclosing scope are remembered.

del outer #outer is deleted
var()
output:'food for the brain'
outer('food for the brain')

output: NameError:name 'outer' is not defined

Decorators in Python

With all the pre-requisites donned we can now safely move on to the main event. As the name suggests Decorators are there to decorate other functions and give them new look and feel, exactly like a gift box.

A simple example for a decorator:

def outer(f):
  def inner():
    msg = f()
    return msg.upper()
  return inner
def func():
  return 'hello! Peter'
func = outer(func)
print(func())

output: HELLO! PETER

In the above case outer() is our enclosing function and inner() is our nested function. Observe when we called the outer() function, we passed in another function named func() as the argument.  This function is now available in the outer() function’s scope as f. And again we reassigned the func with the returned object of the outer() i.e. inner. As we have learnt earlier now func holds the reference to the function inner(). In the next line func() was called and it returned ‘HELLO! PETER’ as expected. Go through the code again to understand better.

However, Python has a better way to implement this, we will use @ symbol before the decorator function. This is nothing but syntactic sugar. Let’s see how it’s done

def outer(f):
  def inner():
    msg = f()
    return msg.upper()
  return inner
@outer
def func():
  return 'hello! Peter'
func()

output: HELLO! PETER

We stacked the outer() above the function that needs to be decorated, this is the same as that we used before but more Pythonic. Let’s take another example where we want to find out the greater number in each tuple from a list of tuples.

def outer(func):
  def inner(args):
    return [func(var[0],var[1]) for var in args]
  return inner
@outer
def func(a,b):
  return a if a>b else b   
print(func([(1,4),(5,3)]))

output: [4, 5]

Here, we could pass two variables to the func(). In this way, we were able to add new functionality to our function func(). func() originally could take only two arguments but now it can take a list of tuples.

Passing arguments to decorators in Python

We can also pass arguments to the decorators themselves, see the following example

def meta_decorator(x):
  def outer(func):
    def inner(args):
      return [func(var[0],var[1])**x for var in args]
    return inner
  return outer
@meta_decorator(2)
def func(a,b):
  return a if a>b else b
 print(func([(1,4),(5,3),(6,5)]))

output: [16, 25, 36]

in the above example, we passed an integer 2 to the decorator itself. This integer is used to square the greater term. meta_decorator in this case took 2 as an argument as we wanted to square the terms.

Generalizing decorators for multiple parameters in Python

Just as any other function we can take the help of *args and **kwargs to generalize decorators for multiple parameters intake. *args will be a tuple of positional arguments and **kwargs will be a dictionary for keyword arguments. Let’s see an example.

def outer(func):
    def inner(*args,**kwargs):
        func()
        print(f'poistional arguments {args}')
        print(f'keyword argumenrs are {kwargs}')
    return inner

@outer
def func():
    print('arguments passed are shown below')
func(6,8,name='sunil',age=21)

output: arguments passed are shown below
poistional arguments (6, 8)
keyword argumenrs are {'name': 'sunil', 'age': 21}

From the above example, we learnt how to pass multiple parameters to the decorator function. Here we passed both positional arguments as well as keyword arguments in a single line. Remember the convention is to use positional arguments before keyword arguments.

Original functions or those that are to be decorated can also take arguments, but those arguments need to be passed to the function from the wrapper or inner function.

def outer(func):
    def inner(*args,**kwargs):
        func(2) #arguments passed to the original function
        print(f'poistional arguments {args}')
        print(f'keyword argumenrs are {kwargs}')
    return inner
@outer
def func(a):
    print(f'arguments for {a} cases are shown below')
func(6,9,name='sunil',age=21)

output: arguments for 2 cases are shown below
poistional arguments (6, 9)
keyword argumenrs are {'name': 'sunil', 'age': 21}

Applying Multiple decorators in Python

We can also use more than one decorators to decorate a single function by stacking them above the function. Let’s see an example where we have two decorator functions one is for squaring and another is for finding prime numbers.

def square(func):
  def inner_one():
    prime_nums = func()
    return [i**2 for i in odd_nums]
  return inner_one
def find_prime(func):
  def inner_two():
    prime = []
    for i in func():
      count = 0
      for j in range(1,i):
        if i%j==0:
          count+=1
      if count<2:
        prime.append(i)
    return prime
  return inner_two
@square
@find_prime
def printer():
  return [5,8,4,3,11,13,12]
printer()

output: [25,9,121,169]

Observe the order in which the decorators are stacked and their order of execution. We saw that these stacked python decorators follow a bottom-up approach that means the decorator placed just above the function will be used first then it will move on to the second one.

Here, find_prime() was executed first and then square(). If we change the order the result will be an empty list(why?).

The functools.wraps()

So far so good but there is a problem that we have overlooked. See the below example

def outer(func):
  def inner():
    'inside inner function'
    msg = func()
    return msg.upper()
  return inner
@outer
def function():
  'inside original function'
  return 'hello! Peter'
#if we run this we gwt
print(function.__name__)
print(function.__doc__)

output:inner
inside inner function

In the above example, we saw function.__name__ showed inner while it should have been ‘function’ and same for docstrings too. The function() got replaced by inner(). It overrode the name and docstring of the original function, but we want to retain the information of our original function. So to do that Python provides a simple solution i.e. functools.wraps(). 

<div>
<pre>from functools import wraps
def outer(func):
  @wraps(func)
  def inner():
    'inside inner function'
    msg = func()
    return msg.upper()
  return inner
@outer
def function():
  'inside original function'
  return 'hello! Peter'
#if we run this we gwt
print(function.__name__)
print(function.__doc__)</pre>
</div>

output:function
inside original function

In the above example, we used the wraps() method of functions inside the outer(). Observe that the wraps() method here itself was used as a decorator with func() as the argument. This decorator stores the metadata(name, docstring, etc) of the function to be decorated. Not doing this will not be harmful but will make debugging tedious, So it is prudent to use functools.wraps() whenever decorators are used.

Conclusion

We might not need python decorators often in the day-to-day coding tasks but these come in handy in cases when we need some form of data hiding and it also makes our code more readable, more Pythonic. It provides an elegant way of adding new functionalities to our existing functions without altering the source code. Several other use-cases of decorators involve measuring execution time, logging, etc. These are also extensively used in web frameworks like Flask and Django for user authorization purposes.

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion

Sunil Kumar

Meet your author Sunil kumar Dash, a developer and a writer. Has diverse interests in tech, pop culture, wellness, philosophy and Anime. Exploring underrated music is his hobby. And loves to doom scroll Twitter when bored.