By Konrad Hałas and Szymon Teżewski

Introduction

Dependency injection (DI) is a design pattern for writing loosely coupled code. Using this approach can make your codebase easier to maintain and extend.

Because of the dynamic nature of Python and its “glue language” background, some of the design patterns and techniques are not as popular as in other programming languages. This is the case with dependency injection.

In this blog post you will learn that you can implement DI in your project at multiple levels of engagement – from manually passing dependencies in your functions to using a specialized dependency injection container. It doesn’t bite, and it’s “Pythonic.”

Our codebase (the Marketing Technology product within The Revenue Operating System^® from FLYR) has almost 100,000 lines of code. It’s not huge, but it’s definitely not a small project. We use DI (container) extensively, and it works for us. There is a high chance that it will work for you too.

As always, this type of technique introduces some sort of additional complexity. Nevertheless, the benefits outweigh the added difficulty, and we hope you will give it a try.

Let’s start with something really simple: a function.

Function

Imagine that we have a system that sells airline tickets (don’t forget that we are an airline-focused company).

We need a function that reminds our customers about their flight and also provides the weather forecast for their destination. This function is called the day before departure date and takes the ticket number as an input.

It looks like this:

# core.py

from database import database
from emails import emails_sender
from weather import weather_api


def send_flight_reminder(ticket_number):
    ticket = database.get_ticket(number=ticket_number)
    weather = weather_api.fetch_weather(airport=ticket.destination, date=ticket.arrival)
    emails_sender.send_email(
        subject=f"It's tomorrow – your flight to {ticket.destination}",
        body="Grab your umbrella!" if weather.will_it_rain else "Pack your sunglasses!",
        to=ticket.customer_email,
    )

The send_flight_reminder function has a single parameter – the ticket number.

Within our function, we perform three steps:

• Fetch the ticket details from our database;
• Gather a weather forecast from an external weather API; and
• Send an email to our customer.

We don’t want to focus on the technical details of a database, a weather API, or an email sender, so let’s assume that we have such modules within the scope of our project. The most important thing here is that those modules are doing a real job – for example, the database module queries DB to fetch ticket details, the weather API requests data via HTTP, and the email sender uses SMTP to send an email.

Our function is ready, so let’s see how we can use it:

# tasks.py

from core import send_flight_reminder

def everyday_task():
    tomorrow_ticket_numbers = ... # query database

    for ticket_number in tomorrow_ticket_numbers:
        send_flight_reminder(ticket_number=ticket_number)

We have almost everything we need, except one important part: tests.

Tests

Even smaller codebases should have tests. You need these tests to develop and move forward with your project without fear of breaking old stuff.

Unfortunately, testing our function is a bit problematic. It interacts with the outside world extensively. It has some side effects, but it doesn’t return anything.

We have two options:

• Implement a full-blown, end-to-end test that checks the behavior of our function but also the interactions and behaviors of external systems; or
• Implement a precise unit test to validate only our function.

Because the configuration of the local SMTP server is out of the scope of this blog post, let’s go with the second option.

Remember, we have multiple interactions with outside systems within our function. How can we test it without performing a database query, hitting a weather API, or sending an email? Python World has an easy answer to this question – monkey patching!

Our first test looks like this:

# tests.py

import datetime
from unittest import mock

from database import Ticket
from core import send_flight_reminder
from weather import Weather


def test_send_flight_reminder_with_a_rainy_weather():
    with mock.patch("core.database") as database_mock, \
            mock.patch("core.weather_api") as weather_api_mock, \
            mock.patch("core.emails_sender") as emails_sender:
        database_mock.get_ticket.return_value = Ticket(
            destination="ATL",
            arrival=datetime.datetime(2022, 1, 1, 12, 00),
            customer_email="email@customer.com",
        )
        weather_api_mock.fetch_weather.return_value = Weather(will_it_rain=True)

        send_flight_reminder(ticket_number="AZ123")

        emails_sender.send_email.assert_called_once_with(
            subject=f"It's tomorrow – your flight to ATL",
            body="Grab your umbrella!",
            to="email@customer.com",
        )

It does its job, but there are a few problems with our function and above test:

1. We need to know an internal implementation of our function to mock specific dependencies. We can’t see what type of dependencies it uses without looking into the code. It’s not a big deal if we are the author and a function is simple, but what if somebody else wants to use/test it?
2. We tighten our test with our function’s internal details. It’s even worse – we had to provide an exact import path to mock.patch, so even such a trivial refactor as change of import will require a change in tests.
3. Our code is strongly coupled with our dependencies. What if we want to change the provider of weather data? We will likely have to update our function and tests.

We clearly see that there is room for improvement. Let’s try dependency injection.

Dependency injection

It looks like our dependencies are the root of the problem. They are strongly glued to our function, but we can change that.

Instead of a direct import approach, we can pass all dependencies as arguments to our function, like this:

# core.py

def send_flight_reminder(ticket_number, database, weather_api, emails_sender):
    ticket = database.get_ticket(number=ticket_number)
    weather = weather_api.fetch_weather(airport=ticket.destination, date=ticket.arrival)
    emails_sender.send_email(
        subject=f"It's tomorrow – your flight to {ticket.destination}",
        body="Grab your umbrella!" if weather.will_it_rain else "Pack your sunglasses!",
        to=ticket.customer_email,
    )

This is dependency injection in a nutshell – we passed (injected) all dependencies via function parameters. With this simple refactor we solved all of our problems: dependencies are now explicit. You can tell what your function needs just from its header. You can now provide different implementations of your dependency (e.g. a different weather API) whenever you want. You can now easily provide fake implementation in your tests, and you even don’t have to use mock; you can write your own test-only implementation.

Now our test looks a little bit cleaner, and we don’t need to monkey-patch our function.

# tests.py

import datetime
from unittest.mock import Mock

from database import Ticket
from core import send_flight_reminder
from weather import Weather

def test_send_flight_reminder_with_a_rainy_weather():
    database_mock = Mock()
    database_mock.get_ticket.return_value = Ticket(
        destination="ATL",
        arrival=datetime.datetime(2022, 1, 1, 12, 00),
        customer_email="email@customer.com",
    )
    weather_api_mock = Mock()
    weather_api_mock.fetch_weather.return_value = Weather(will_it_rain=True)
    emails_sender_mock = Mock()

    send_flight_reminder(
        ticket_number="AZ123",
        database=database_mock,
        weather_api=weather_api_mock,
        emails_sender=emails_sender_mock,
    )

    emails_sender_mock.send_email.assert_called_once_with(
        subject="It's tomorrow – your flight to ATL",
        body="Grab your umbrella!",
        to="email@customer.com",
    )

As we’ve said, if we want we can even take it one step further and get rid of mock completely. In the previous test, we still expose internal details of our dependencies – e.g. we still have the fetch_weather method mentioned in our test, because we need to mock it. If somebody changes it, we will have to update our tests. Instead of that, we can implement a stub:

# tests.py

class WeatherAPIStub:

    def __init__(self, weather):
        self.weather = weather

    def fetch_weather(self, airport, date):
        return self.weather

def test_send_flight_reminder_with_a_rainy_weather():
    ...
    weather_api = WeatherAPIStub(weather=Weather(will_it_rain=True))
    
    send_flight_reminder(
        ...
        weather_api=weather_api,
    )

In the stub, we need to implement the fetch_weather method too, but this will be the only place where we’ve coupled tests with our dependency.

If you want to use the basic way of DI you can stop here. This approach is very simple and it has many benefits. On the other hand, it also has some drawbacks, but we know how to tackle them.

Drawbacks

We’ve introduced DI, and we’ve uncoupled our code from our dependencies, but not everything looks so beautiful:

1. Our module doesn’t have dependency imports, but now you don’t know what exactly you can pass as a dependency argument. You see its name, but you don’t know its type. Your IDE doesn’t know that either, so autocomplete doesn’t work.
2. Our function header looks ugly. We mixed up “regular” parameters with dependency parameters.
3. Whenever we want to use our function, we have to provide all the dependencies. It’s not convenient.

Some of you probably noticed that we haven’t used any type hints so far in our examples. This was on purpose; we didn’t want to introduce it all at once, and we didn’t want to scare you with too many details. Now we have to admit, at FLYR we use type hints a lot. We love them, and we believe that in a big codebase they are just as necessary as the high test coverage.

We can use type hints to solve the first problem from the list above:

# core.py

from database import Database
from emails import EmailsSender
from weather import WeatherAPI


def send_flight_reminder(
    ticket_number: str,
    database: Database,
    weather_api: WeatherAPI,
    emails_sender: EmailsSender,
) > WeatherAPI:
    ...

Now we (and our IDE) clearly see what our function needs, but unfortunately we did two steps forward and one step backward. Again, we have direct import of our external dependencies.

We have to mention it again: if you want, you can stop here and you don’t have to follow the next refactor. In small projects, this type of coupling is not a big deal. But as your codebase grows, it could become a huge pain. You don’t want to tighten your “business logic” with technical details.

There is a proverb:

“All problems in computer science can be solved by another level of indirection.”

In our case, we can achieve “indirection” with an abstraction. We need types, but they will only serve the role of interfaces.

Here you have an example of such an interface for the weather API client. The database and email sender interfaces look similar.

# core.py

import abc
import dataclass
import datetime


@dataclasses.dataclass
class Weather:
    will_it_rain: bool


class WeatherAPIInterface(abc.ABC):
    @abc.abstractmethod
    def fetch_weather(self, airport: str, date: datetime.datetime) > Weather:
        pass


def send_flight_reminder(
    ...
    weather_api: WeatherAPIInterface,
) -> WeatherAPI:
    ...

We like to use the abc module to emphasize that you cannot create instances of such a class.

We don’t want to focus on technical details of our dependencies, but here is an example of weather API implementation.

# weather.py

import datetime
import requests as requests
from core import WeatherAPIInterface, Weather


class WeatherAPI(WeatherAPIInterface):
    def fetch_weather(self, airport: str, date: datetime.datetime) -> Weather:
        response = requests.get(
            url="https://freeweatherapi.org/",
            params={
                "airport": airport,
                "time": date.isoformat(),
            },
        )
        will_it_rain = response.json()["rain_probability"] > 80
        return Weather(will_it_rain=will_it_rain)

You see its interaction with the outside world – HTTP request via requests library – and all internal details of API response. We don’t want to care about those details when we develop or test our “core” logic. Thanks to our interfaces, we have a strong boundary between “real” implementations and our function.

We solved the first “problem” from our list, so let’s check the next one: mixing of “regular” and dependency parameters. We don’t like how our function header looks right now, so we have to separate those two types of parameters.

Our approach is quite simple – we transform our function into a class. We use the __init__ method to receive all dependencies, and we create a method that looks almost the same as our old function, but it has only a single parameter, as at the beginning of our blog post.

# core.py

class FlightReminder:
    def __init__(
        self,
        database: DatabaseInteface,
        weather_api: WeatherAPIInteface,
        emails_sender: EmailsSenderInterface,
    ) -> None:
        self.database = database
        self.weather_api = weather_api
        self.emails_sender = emails_sender

    def send(self, ticket_number: str) -> None:
        ticket = self.database.get_ticket(number=ticket_number)
        weather = self.weather_api.fetch_weather(airport=ticket.destination, date=ticket.arrival)
        self.emails_sender.send_email(
            subject=f"It's tomorrow – your flight to {ticket.destination}",
            body="Grab your umbrella!"
            if weather.will_it_rain
            else "Pack your sunglasses!",
            to=ticket.customer_email,
        )

Now it looks much better. Let’s see how we can use our new class in tests:

# tests.py

def test_send_flight_reminder_with_a_rainy_weather():
    database_mock = Mock()
    database_mock.get_ticket.return_value = Ticket(
        destination="ATL",
        arrival=datetime.datetime(2022, 1, 1, 12, 00),
        customer_email="email@customer.com",
    )
    weather_api_mock = Mock()
    weather_api_mock.fetch_weather.return_value = Weather(will_it_rain=True)
    emails_sender_mock = Mock()
    flight_reminder = FlightReminder(
        database=database_mock,
        weather_api=weather_api_mock,
        emails_sender=emails_sender_mock,
    )

    flight_reminder.send(ticket_number="AZ123")

    emails_sender_mock.send_email.assert_called_once_with(
        subject=f"It's tomorrow – your flight to ATL",
        body="Grab your umbrella!",
        to="email@customer.com",
    )

All looks good, but we still have to provide our dependencies whenever we want to use our class. We have multiple places in our codebase where we call our FlightReminder.send method, and it would be very inconvenient to create a FlightReminder instance from scratch all the time.

As always, we have a few options. The simplest one is to create a module in our project that will gather all dependencies and inject them into our classes. This approach is known as a “composition root,” and it can look like this:

# root.py

from database import Database
from weather import WeatherAPI
from emails import EmailsSender
from core import FlightReminder

database = Database()
weather_api = WeatherAPI()
emails_sender = EmailsSender()

flight_reminder = FlightReminder(
    database=database,
    weather_api=weather_api,
    emails_sender=emails_sender,
)

Now whenever you want to use an instance of FlightReminder, you can just import it from this module:

# tasks.py

from root import flight_reminder

def everyday_task():
    tomorrow_ticket_numbers = ...  # query database

    for ticket_number in tomorrow_ticket_numbers:
        flight_reminder.send(ticket_number=ticket_number)

If you don’t have too many such classes this approach might be enough. But sometimes your “composition root” is so big that you need some more sophisticated method. You need a dependency injection container.

Dependency injection container

The recipe for creating an instance of our class is presented in the header of __init__ method. We see what instances of which types we need to satisfy our class. Sometimes our dependencies are abstract (as in our example). If we can somehow combine our interfaces with implementations, we will be able to get rid of manual composition and pass this task to the tool. We call this tool the dependency injection container.

There are a few DI containers available in PyPI. We’ve chosen injector because of its straightforward API.

Here is how you can use it with our example:

# root.py

import injector
from injector import Module, Binder

from core import WeatherAPIInterface

from weather import WeatherAPI


class TicketingModule(Module):

    def configure(self, binder: Binder) -> None:
        binder.bind(interface=WeatherAPIInterface, to=WeatherAPI)  # 1


container = injector.Injector(modules=[TicketingModule()])  # 2

# core.py

from injector import inject

class FlightReminder:
    @inject  # 3
    def __init__(
            self,
            database: DatabaseInterface,
            weather_api: WeatherAPIInteface,
            emails_sender: EmailsSenderInterface,
    ) -> None:
        self.database = database
        self.weather_api = weather_api
        self.emails_sender = emails_sender

    ...

# tasks.py

from core import FlightReminder
from root import container


def everyday_task():
    ...
    flight_reminder = container.get(FlightReminder)  # 4
    for ticket_number in tomorrow_ticket_numbers:
        flight_reminder.send(ticket_number=ticket_number)

Let’s go through this code together:

1. At the beginning, we have to tell injector which implementation class it should use when it comes across a particular interface. In our example, when somebody wants an instance with WeatherAPIInterface interface, our container should use WeatherAPI to create such an instance.
2. We create our container based on a previously prepared module, which we will use to build instances of our class.
3. We have to mark the __init__ method of our class with @inject decorator. Now injector knows that it is in charge of creating instances of our class.
4. We can ask injector to create an instance of our class.

With this DI container, we don’t have to manually build our objects. Whenever we need some more dependencies within our class, we can just add them to the __init__ method. We don’t have to go back and amend our “composition root.”

Finally, we should get back to our tests – do they change after we add injector to our project? That depends on what type of test we’re talking about.

In unit tests, you should stick with a mock/stub approach. Don’t worry about `@inject` decorator – you can still manually create your instances.

In end-to-end (or non-unit) tests, we have two options:

• You can amend the place where you create the container, and, based on your environment, you can provide a different configuration – e.g. you can distinguish between production, local, and test environments and provide implementations appropriate to the given use case.
• You can create the DI container from scratch, bind interfaces with stubs, and use this container to get your instances within your tests.

Summary

It was a long journey, but we hope you learned something new.

DI decreases the coupling between your code and its dependencies. It shows explicitly what your function or class needs to get the job done. Your implementation is more readable, and your tests are easier to write and maintain.

You can choose the best form of dependency injection in the context of your project. Maybe simply passing dependencies as function parameters is enough for you. Maybe you need something more sophisticated and want to play with dependency injection containers. It’s all up to you, but we believe that whatever you choose will make your code better.