Anaconda Python 3 with conda

Anaconda is a special distribution of Python that is purposed for scientific workloads. Anaconda claims to provide everything you need for data science development and experimentation, including it’s own specialized package manager. Some researchers prefer to use Anaconda for their Python workloads. The following steps describe the recommended method for creating an Anaconda (conda) virtual environment …

Step 1: Housekeeping

First, ensure that you are in your home directory and have a clean environment, with no other Python-specific modules loaded…

cd
module list

If you see any Python modules loaded, it might be a good idea to log out, then back in to reset your environment.

Step 2: Load Anaconda

Once you have confirmed that your environment is clean, you can load one of the Anaconda Python modules from which you can begin configuring your virtual environment. Here, we load the default (latest) version…

module av python

Console: /bin/bash

(env2) hpcuser@easley01:~ > module av python

———————————– Programming Languages & Environments —————————————-

python/3.8.6 python/anaconda/2.7.14 … python/anaconda/3.7.0 python/anaconda/3.8.6 (L,D)

python/3.9.2 python/anaconda/2.7.15 … python/anaconda/3.7.4 python/intel/3.7.9

module load python/anaconda

Step 3: Create a New Virtual Environment

With Anaconda Python loaded, your environment should now be set to use the Python binaries. Because the Python installation is purposed for a multi-user environment, the path for the default package location is not writable by normal users. In order to use our own location for packages, we need to set the CONDA_PKGS_DIRS environment variable so that conda does not attempt to write to the shared location…

export CONDA_PKGS_DIRS=~/.conda/pkgs

Now we can create a new virtual environment using conda create using the -n argument to give it a specific name. For this example, the virtual environment name will be env1. …

conda create -n env1

Console: /bin/bash

hpcuser@easley01:~ > conda create -n env1

Collecting package metadata (current_repodata.json): done

Solving environment: done

## Package Plan ##

environment location: /home/hpcuser/.conda/envs/env1

Proceed ([y]/n)?

Preparing transaction: done

Verifying transaction: done

Executing transaction: done

#

# To activate this environment, use

#

# $ conda activate env1

#

# To deactivate an active environment, use

#

# $ conda deactivate

We can see from the output that conda wants to create the environment in a location in our home directory ~/.conda/envs/env1. If conda create was successful, we should see some new files and directories in that location …

Console: /bin/bash

$ ls ~/.conda/

environments.txt envs pkgs

$ ls ~/.conda/envs/env1/

conda-meta

Step 4: Activate the Environment

Also from the conda create output, we can see that Anaconda Python has advised us to use conda activate env1 to use the environment. However, if you have not yet used Anaconda Python, you might see an error when running that command …

Console: /bin/bash

$ conda activate env1

CommandNotFoundError: Your shell has not been properly configured to use ‘conda activate’.

To initialize your shell, run

$ conda init <SHELL_NAME>

Currently supported shells are:

- bash

- fish

- tcsh

- xonsh

- zsh

- powershell

See ‘conda init –help’ for more information and options.

IMPORTANT: You may need to close and restart your shell after running ‘conda init’.

Warning

We could certainly run conda init, but doing this will alter our ~/.bashrc file with settings specific to the loaded version of Python. Because .bashrc is sourced every time we log on to the cluster, this could potentially cause a conflict if we ever want to change our Python environment.

Our recommended method for activating the environment is to use source activate …

source activate env1

If the activation is successful, your command prompt should change to indicate the environment in which you are running …

Console: /bin/bash

hpcuser@easley01:~ > source activate env1

(env1) hpcuser@easley01:~ >

Step 5: Install Packages

Now, we should be able to install packages into our virtual environment using conda install <package_name>. Let’s see if we can install the curl module …

conda install curl

Console: /bin/bash

(env1) hpcuser@easley01:~ > conda install curl
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

environment location: /home/hpcuser/.conda/envs/env1

The following packages will be downloaded:

package                      build
------------------------------------------------------------
_libgcc_mutex-0.1            main           3 KB
ca-certificates-2021.4.13    h06a4308_1     114 KB
curl-7.71.1                  hbc83047_1     140 KB
...
------------------------------------------------------------
                                      Total:        13.9 MB

The following NEW packages will be INSTALLED:

_libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main
ca-certificates    pkgs/main/linux-64::ca-certificates-2021.4.13-h06a4308_1
curl               pkgs/main/linux-64::curl-7.71.1-hbc83047_1
...

Proceed ([y]/n)? y

Downloading and Extracting Packages
curl-7.71.1           140 KB     ######################  100%
...

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Step 6: Exiting

To exit the execution context of the virtual environment and get back to our regular shell, conda deactivate …

Console: /bin/bash

(env1) hpcuser@easley01:morgaia > conda deactivate
hpcuser@easley01:morgaia >