#

Python

NOTE: This documentation describes the new lmod module system on the cluster, currently in opt-in phase. See this page for details. In particular you need to activate the lmod system with the following command: source new-modules.sh

Python on Odyssey

We like the Anaconda python distribution from Continuum Analytics. It includes hundreds of the most popular packages for large-scale data processing, predictive analytics, and scientific computing (numpy, scipy, ipython, pandas, scikit-learn, mpi4py, etc.; see below for the full list). It generally does a great job of including the latest versions of all the packages while maintaining compatibility between them all.

You can use Python and Anaconda on Odyssey by running:

$ module load python

Customizing the environment

Anaconda has a concept of environments; these are used to manage alternative package sets and versions. For example, if you want newer versions of some packages in the default environment, you can make a new environment with your customizations.

Load the base environment:

$ module load python/2.7.11-fasrc01

In this case, it's best to load a specific version (rather than simply python) so that your custom environment remains consistent even after a newer version of python is available.

Create a new environment; substitute ody with whatever name you wish:

$ conda create -n ody --clone="$PYTHON_HOME"

This will create a directory ~/.conda/envs/ody. Use this environment by running the command:

$ source activate ody

If you want to use this environment all the time, add the above line to your ~/.bashrc (or other appropriate shell config file) after the line that loads the module.

At this point you can upgrade or install a package named PACKAGE with the following command (it's the same whether installing or upgrading):

$ conda install PACKAGE

The commands conda list and conda show, which list installed and available packages, respectively. See the conda documentation for all the details. If the package is not available from conda, you can install it into your environment using pip:

$ pip install PACKAGE

Note: Anaconda generally has the lastest versions of all packages that are compatible. When you install an extra package, it'll often update core packages like numpy and scipy; other packages may then downgrade them back. This is why we recommend sticking to the default environment if possible.

If you have problems updating a package that is already installed in the Anaconda environment, you might need to remove the package first:

$ conda remove PACKAGE
$ conda install PACKAGE

This will often bypass update errors, especially with certain versions of matplotlib.

To stop using the custom environment, run:

$ source deactivate

CC BY-NC 4.0 This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. Permissions beyond the scope of this license may be available at Attribution.