Virtual Environments

In brief

Article summary

In this article, we learn how to create isolated Python environments using virtualenv.

Main takeaways

• virtualenv is a tool for creating isolated Python environments.

Article contents

1 — What is virtualenv?

virtualenv is a tool to create isolated Python environments. Since different projects may require different versions of Python packages, if you use the global site-packages directory for storing your packages, then upgrading a package for a project will also upgrade it for all other projects, risking breaking them. Hence, keeping packages separate using isolated environments is a good way to ensure software stability.

Virtualenv allows:

• installation of packages into a separate environment (not in the global site-packages directory), which is convenient for creating a minimal working environment. It also avoids bloating the global site-packages directory.
• exporting the list of dependencies as a requirements.txt file, with their corresponding package versions. This file can be shared and used to recreate a virtual environment.

In this way, you can ensure that an application continues working by isolating freezing its environment, meaning that the dependency packages

It is useful when different applications required different versions of packages.

You can install virtualenv using pip:

python -m pip install --user virtualenv
python -m virtualenv --help

For installation on other OS, see this page.

2 — How to use virtualenv [*]?

Before we start using virtual environments, let us see the packages installed in our global site-packages directory:

pip list

Let us now create a directory for storing our virtualenv environments:

# Create a directory for storing your virtualenv environments.
mkdir -p ~/virtualenv/environments
# Navigate to this directory
cd ~/virtualenv/environments

Let us now create a virtual environment called env_name (creates a folder in your current directory):

virtualenv env_name

The created directory is used by virtualenv to store dependency packages. You should not place nor develop anything else inside it. It is supposed to be a directory that you can delete without losing any of your code.

Activate a virtual environment:

source env_name/bin/activate

If you run pip list inside this virtual environment, you will notice that it only contains three pre-installed packages: pip, setuptools, and wheel. These are helpful for installing other packages.

Let us now install some packages (numpy, and scikit-learn), simply as an example:

pip install numpy scikit-learn

Suppose that you want to export the list of all these packages and their corresponding versions. You can do it using:

# Export the list of requirements to a file called "requirements.txt"
pip freeze --local > requirements.txt
# Read the content of "requirements.txt"
cat requirements.txt

To deactivate a virtual environment, simply run the deactivate command:

deactivate

To remove an environment, simply delete its corresponding directory:

rm -rf env_name

To create an environment with a specific Python version, pass the correspoding Python executable file as a parameter:

# See which Python version are available on your computer
ls /usr/bin | grep python
# Use a specific python version from the list above
virtualenv -p /usr/bin/python3.9 python39_env

We can install the required packages from the requirements.txt using the command:

pip install -r requirements.txt
# Verify that all the required packages were installed
pip list

If you are interested to see these explanations in a video format, see: Python Tutorial: virtualenv and why you should use virtual environments.

Warning [*]

“Created python virtual environments are usually not self-contained. A complete python packaging is usually made up of thousands of files, so it’s not efficient to install the entire python again into a new folder. Instead virtual environments are mere shells, that contain little within themselves, and borrow most from the system python (this is what you installed, when you installed python itself). This does mean that if you upgrade your system python, your virtual environments might break, so watch out. The upside of this, referring to the system python, is that creating virtual environments can be fast.”

If you want need a self-contained environment, see OS virtualization tools, like SingularityCE or Docker.