Skip to content

Reproducible Dependencies

Objectives

  • Why dependency management matters
  • What a requirements file is
  • How to create a requirements file
  • How to install dependencies from a requirements file
  • How to recreate an environment

Requirements Files

Imagine: a colleague attempts to rerun a text-analysis pipeline six months after publication. Several packages have changed since the original work was completed. Without a record of the required dependencies, reproducing the original environment becomes difficult.

The virtual environment itself is not committed to Git. Instead, we commit information describing how to recreate that environment using the requirements.txt file. This file lists the Python packages required by a project, and can be used as a shortcut to install all the required packages in one go by running pip install -r requirements.txt

A simple requirements file might contain:

requests
pandas
matplotlib

But for optimal reusability, package versions need to be specified: 

requests==2.32.0
pandas==2.2.3
matplotlib==3.10.1

Version pinning

You will learn the simplest way to create a requirements.txt file in this module, but as you make your way to even larger-scale projects, you will probably end up switching to a more elaborate way of keeping track of your package versions, such as piptools.

Exercise: create a requirements file

Once packages have been installed into a virtual environment, pip can automatically generate a requirements file.

  • Activate the virtual environment.
  • Install a library (or several) using pip install
  • Run pip freeze > requirements.txt
  • Inspect the file. Do you see your library in it? What else do you see?
  • Commit the requirements file.

Installing from a Requirements File

A requirements file can be used to recreate an environment on another machine. The -r flag tells pip to read package names from a file. We will do that now on one machine, but imagine that you are an exploring researcher stumbling on this new repository that they want to install locally.

Exercise: recreate an environment

  • Deactivate and delete .env
  • Create a new environment and activate it.
  • Check which packages have already been installed using pip list
  • Run pip install -r requirements.txt
  • Verify that the required packages have been installed using pip list

Updating Dependencies

Projects evolve over time. New dependencies are added and existing dependencies may change. Whenever dependencies change, the requirements file should be updated as well.

Exercise: update requirements.txt

  • Run pip install tqdm
  • Run pip freeze > requirements.txt
  • Inspect changes using git diff requirements.txt
  • Commit the file.

Works cited:

https://packaging.python.org/ https://pip.pypa.io/en/stable/