Performance Tools

NVTOP

Like top but for the GPU. Great little tool for getting nice usage traces from your GPU jobs. It doesn’t do much but for questions like:

• How much memory am I using?
• Is memory usage churning?
• Could I fit more of these into a single job?

You can install this with spack or just:

# Download the AppImage from GitHub (update release as needed)
wget https://github.com/Syllo/nvtop/releases/download/3.1.0/nvtop-x86_64.AppImage
# Mark it as executable
chmod u+x nvtop-x86_64.AppImage

# Profit (ssh to your job's node first)
./nvtop-x86_64.AppImage