aci-ref.org

A short note on the Python Virtual Environment

In this post we will discuss the concept of a Virtual Environment in Python.
We will discuss the following topics:

  • What is a Virtual Environment (VE)
  • Setting up a VE
  • Activation of a VE
  • Installation of packages within the VE
  • Deactivation of a VE
  • Removal of a VE

What is a virtual environment?
A Python virtual environment is a child/offshoot of a parent python distribution which allows you to use the packages in the parent distribution as well as to install packages that are only visible to the child distribution (cfr. inheritance in OO).

Setting-up the virtual environment
In the first step, we load the parent python distribution in our environment.
The parent python distribution (in this example) is python 3.3.6 (built from source).
Within the parent python distribution an array of packages have been installed: numpy, scipy, matplotlib, etc.

[u0253283@dirac ~]$ module load python/3.3.6   # Loading python 3.3.6 using LMOD
[u0253283@dirac ~]$ which python3              # Check the executable
/uufs/chpc.utah.edu/sys/installdir/python/3.3.6/bin/python3

[u0253283@dirac ~]$ python3 --version           # Version of python3
Python 3.3.6

The parent python distribution contains the pyvenv executable which is used to create the virtual environment:
[u0253283@dirac ~]$ which pyvenv                # Check the version of pyvenv
/uufs/chpc.utah.edu/sys/installdir/python/3.3.6/bin/pyvenv

pyvenv --system-site-packages ~/MyVenv
The newly created virtual environment resides in the directory $HOME/MyVenv. The flag ‘–system-site-packages’ grants access to the parent site-packages dir.

Activation of the virtual environment
In order to use the virtual environment, we first need to unload the parent python distribution. In a subsequent step, we need to activate the new virtual environment.
module unload python/3.3.6   # Unload the parent distribution

cd ~/MyVenv
source bin/activate
[u0253283@dirac MyVenv]$ source bin/activate
(MyVenv) [u0253283@dirac MyVenv]$

(MyVenv) [u0253283@dirac MyVenv]$ which python3  # Check the distribution
~/MyVenv/bin/python3

(MyVenv) [u0253283@dirac MyVenv]$ python3 --version
Python 3.3.6

In the following snippet of code (vide infra) we check whether the child distribution can indeed use the numpy functionality from the parent distribution:
python3
>>>import numpy as np
>>>np.__path__
['/uufs/chpc.utah.edu/sys/installdir/python/3.3.6/lib/python3.3/site-packages/numpy']

>>>a = np.arange(10)
>>>a**3
array([  0,   1,   8,  27,  64, 125, 216, 343, 512, 729])

Installation of packages within the virtual environment

In this section we explain how to install packages in the newly created virtual environment. We will install 2 packages using ‘pip install‘ (Example 1) and ‘setup.py‘ (Example 2).

Note that the pip executable is NOT installed in the newly created ~/MyVenv/bin directory.
However, the pip executable can be easily installed in the Virtual Environment as follows:
(MyVenv) [u0253283@dirac ~]$ python3 -m pip install -U pip

We will now show how to do some simple installations in our virtual environment:

  • Example 1: Installation of qit using pip:
    (MyVenv) [u0253283@dirac ~]$ pip install qit
    (MyVenv) [u0253283@dirac ~]$ python3
    Python 3.3.6 (default, Jan 20 2015, 11:39:02)
    [GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import qit
    >>> qit.__path__
    ['/uufs/chpc.utah.edu/common/home/u0253283/MyVenv/lib/python3.3/site-packages/qit']
  • Example 2: Installation of sunpy using setup.py
    # Requirements:

    • * [Python](http://www.python.org)
    • * [Astropy](http://astropy.org) (1.0.0)
    • * [NumPy](http://numpy.scipy.org/)
    • * [SciPy](http://www.scipy.org/)
    • * [Matplotlib](http://matplotlib.sourceforge.net/) (1.1+)
    • * [suds-jurko](https://bitbucket.org/jurko/suds)
    • * [pandas](http://pandas.pydata.org/) (0.10.0+)
    • * [beautifulsoup4](http://www.crummy.com/software/BeautifulSoup/)
    • * [sqlalchemy](http://www.sqlalchemy.org/)

    module load git
    cd ~
    git clone https://github.com/sunpy/sunpy.git sunpy_src
    cd sunpy_src
    python3 setup.py build
    python3 setup.py install
    cd ~ ; rm -rf sunpy_src

    Note that we are not required to use the –prefix flag, because
    the user has by default access to the subdirectories within the virtual
    environment.

    # Test of the installation:
    python3
    >>> import sunpy
    >>> sunpy.__path__
    ['/uufs/chpc.utah.edu/common/home/u0253283/MyVenv/lib/python3.3/site-packages/sunpy-0.8.dev7032-py3.3.egg/sunpy']
    >>> import sunpy.data
    >>> sunpy.data.download_sample_data()
    >>> import sunpy.data.sample
    >>> import sunpy.map
    >>> aia = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
    >>> aia.peek()

Deactivation of the Virtual Environment
We can leave the Virtual Environment easily by invoking ‘deactivate‘:
(MyVenv) [u0253283@dirac ~]$ deactivate
[u0253283@dirac ~]$

Removal of the Virtual Environment
We can get rid of the Virtual Environment by deleting the directory:
[u0253283@dirac ~]$ rm -rf ~/MyVenv

wcardoen

, ,