GPU Workloads

Presentation Slides


Easley GPU nodes are equipped with NVIDA Tesla T4 devices. Two generally accessible GPU partitions exist, based on the number of GPU devices that are present per node.

1sinfo -p gpu2,gpu4 -O gres,partitionname,nodes

GPU Modules & Locations

CUDA Toolkit and a related programming framework module form the base for your GPU workload environment.

A number of code examples and scripts are also available in the /tools/gpu subdirectory …

module show cuda11.0/toolkit
ls /tools/gpu/cuda_simple

Basic CUDA C++ Example

Copy the sample source files to a location in your home directory…

cd ~
mkdir hpc_gpu
cd hpc_gpu
cp -R /tools/gpu/tutorials/* .
cd hello

Now let’s take a quick look at the source code, compile, and run it…

module load cuda11.0/toolkit
nvcc -o hello
srun -N1 -n1 --partition=gpu4 --gres=gpu:tesla:1 ./hello
nvcc -o threads
srun -N1 -n1 --partition=gpu4 --gres=gpu:tesla:1 ./threads

And to be sure, let’s use the CUDA profiler to see exactly how the programs are using the GPU …

srun -N1 -n1 -p gpu4 --gres=gpu:tesla:1 nvprof $HOME/hpc_gpu/hello/hello
srun -N1 -n1 -p gpu4 --gres=gpu:tesla:1 nvprof $HOME/hpc_gpu/hello/threads

Using Cuda with Pytorch

Pytorch is one of the many popular deep learning frameworks used among data scientist. In order to set up and run CUDA operations, Pytorch provides the torch.cuda package. This package adds support for CUDA tensor types, that impliment the same function as CPU tensors but utilizes GPUS for computation.

Installing Pytorch

A virtual environment will need to be created before installing Pytorch. For more information about virtual environments please visit the following:

cd ~/ hpc_gpu
module load python
python3 -m virtualenv pytorch_lab
source pytorch_lab/bin/activate

At this point, the virtual environment is created and activated using the source command. Now that the pytorch virtual environment is activated, install pytorch using the following

cd pytorch_lab
pip3 install torch torchvision torchaudio --extra-index-url
pip list installed

In this first exercise we will use the torch.cuda package to check the availability of the cuda device and to gather information.


srun -N1 -n1 --partition=gpu2 --gres=gpu:tesla:1 --pty /bin/bash
module load python
module load cuda11.0/toolkit
cd ~/hpc_gpu/pytorch_lab
source bin/activate
import torch

To check if your system supports cuda, use the following command. is_available() will return a bool value either true if your system supports cuda or false.


The current_device() command will provide information about the id of the cuda device


Taking the following id value provided above, you can also retrieve the name of the device using the following command

'Tesla T4'

To provide even further information using the id of the cuda device you can do the following

_CudaDeviceProperties(name='Tesla T4', major=7, minor=5, total_memory=15109MB, multi_processor_count=40)

Lets finish the interactive job by doing the following. Exit the python program and the interactive job as such


Copy the following exercise in the virtualenvs/pytorch directory

cp ../pytorch/ .

Once placed back onto the login node, deactivate the virtual environment


Batch Job Submission

Create a bash script named and place the following


#SBATCH --partition=gpu4
#SBATCH --time=5:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --gres=gpu:tesla:1
#SBATCH --job-name=pytorch_lab

module load python
module load cuda11.0/toolkit/11.0.3

source ~/hpc_gpu/pytorch_lab/bin/activate

python3  > results.out