Benchmark and Profile Django APIs Graphically

Are API Calls taking too long? Need to debug where your code is taking a significant amount of time? Use these methods to profile your Django API.

Django was first released in November 2005, On Github, it has more than 28 thousand stars, with a community of more than 1200 developers. (Photo by Faisal on Unsplash)

I have been working on Django for the past year. It is an extraordinary tool for developing web applications and relational database management. Django APPs can be scaled up quickly using Dockers, Kubernetes, and auto-scaling on AWS with Load balancers in place. But this article covers an essential part of optimizing your Django APIs.

We can utterly manage the load using the above tools. But we still need to ensure that the code quality is good. The bulk queries running are the powerhouse of your APP and must be optimal. No one appreciates slow websites in this 5G world.

And to check the optimality of queries, we should benchmark the code. And, the code here means views and models for Django. I had a similar use case. My APIs were performing very slowly. Hence, I complied with the tools I used to investigate the code.

__init__: The Initial Setup

Let's quickly go through the initial setup. I have a Django app that contains only one view. This view will run on hitting /site URL.

django-view. (view.py)

In this view, two dummy internal functions are being called. One takes 2 seconds and the other one 4 seconds. The complete code can be found here.

Setup Django Profiler Middleware

Setting up the Django middleware profiler is very simple.

• Install django-cprofile-middleware using pip.
• Add the below middleware at the last of the middleware’s list in your Django app settings.

django_cprofile_middleware.middleware.ProfilerMiddleware

• Log in to the user in Django APP. (If you are not using authentication, please create a super user)
• Fetch the user object, and set is_staff to True. If you are the only user, you can use Django shell and run the following command.

from django.contrib.auth.models import User
user = User.objects.last()
user.is_staff = True
user.save()

Great, the setup is done!!!

How to View Stats

When you hit the API, you can see the following response:

API Response

For getting the benchmarking data, add prof a GET parameter.

API Response when profiling is done.

Let's try to interpret this table.

• As you can see on the top, 932 functions were called during execution. Of these, 931 were primitive calls. These primitive calls are nothing but internal calls made by python and Django rest framework before, after, and during running the view file.
• tottime : is the total time spent executing this function. (excluding the time made in calls to sub-functions)
• cumtime : is the cumulative time spent executing this function.
• ncall : total number of calls.

You can check more about profiling on their official website.

The data above does not make much sense. I want the time taken by each function in my code to run. However, the readability of this data is low. To enhance readability and use graphs to ease the investigations, let’s download this data.

Hit the following URL to download the profiled data. This will download a view.prof file for you.

http://127.0.0.1:8000/site/?prof&download

Visulaization of Stats

Snakeviz is a visualization tool used to analyze the output of Python’s cProfile Module. It will give you a nested graphical representation of the time taken by each function and its component.

First, install Snakeviz.

pip install snakeviz

Go to the directory where the view.prof file was downloaded. And, run the following command:

snakeviz view.prof

The Snakeviz will run a local server at port 8080. Once you open the URL, you can see the visualization as follows.

Snakeviz Profiler

Snakeviz Interpretation:

• When you hit an API, the request is first sent to csrf.py followed by the APIView (Django rest framework).
• It arrives in the view.py, which is my view file. The total time to process the get function in the view file is 6.01 seconds.
• Further, in view.py, two internal functions took 4 and 2 seconds respectively. It is obvious if you see view.py .
• Each function has a single component of time.sleep. Further, we can see there are time.sleep class functions in the internal functions.

This hierarchical-graphical representation helps in better analysis of each request in Django. Also, this visualization is highly interactive.

Wrap Up

I was amazed by the ease of benchmarking Django APIs. Usually, benchmarking takes a lot of manual work. It was a quick solution.
Also, we could get the query time along with the function execution time. While using databases, you also need to optimize the query time. cProfile takes care of this as it includes the query time too.