“Wimmy Wham Wham Wozzle!“
What SLURM?
SLURM is a workload manager that performs the complex job of coordinating efficient and fair access to computational nodes.
Where SLURM?
Carbonate, BigRed3, eventually Quartz
Why SLURM?
Job handler consistency among all IU’s supercomputers.
When SLURM?
April 12, 2021
Who SLURM?
Users of Carbonate, BigRed3, and Quartz supercomputers.
How SLURM?
KB offers a very useful, detailed guide to everything SLURM- related here. Below is the TLDR version.
Converting your Scripts
To assist you in converting your job scripts from Torque to Slurm, HPC Everywhere offers a handy script converter. Just be sure to give your converted script a once-over before testing it in the queue.
If you need additional help converting a job script, email the UITS Research Applications and Deep Learning team at radl@iu.edu or the NCGAS team at help@ncgas.org.
Research Technologies provides a Slurm debug partition on Carbonate for testing converted job scripts. Submit your test script to the debug partition using the ‘-p debug‘ argument to sbatch.
sbatch -p debug test.job
Common Commands Translation
| Command Function | Torque Command | Slurm Command |
| Submit a job |
qsub |
sbatch |
| Monitor job status information |
qstat |
squeue |
| Stop a queued or running job |
qdel |
scancel |
| Start an interactive job |
qsub |
srun |
You can get more detailed information about each of these commands by entering the command name preceded by ‘man‘ (e.g. man srun) on Carbonate.
SLURM Job Script
#!/bin/bash #read script using Bash command interpreter- don't leave this line out! #SBATCH -J job_name #job name #SBATCH -p general #submit to the general partition #SBATCH -o filename_%j.txt #output file name (%j is replaced by job ID) #SBATCH -e filename_%j.err #standard error file name (%j is replaced by job ID) #SBATCH --mail-type=ALL #send email for ALL, BEGIN, END, or FAIL #SBATCH --mail-user=username@iu.edu #the email address for job-related emails #SBATCH --nodes=1 #number of requested nodes #SBATCH --mem=128gb #memory per node #SBATCH --ntasks-per-node=1 #number of tasks per node #SBATCH --time=02:00:00 #requested time #load modules module load modulename #path to your working directory cd full/path/to/your/directory #commands to execute in following lines
Run Template
We recommend you keep a ‘Run Template’ in a readily accessible location, like home, so that you don’t have to re-create a job script from scratch. To do this, follow these simple steps.
Copy the job template above and personalize it as needed:
cd ~
nano RunTemplate
Then every time you want to create a new job script:
cp ~/RunTemplate new.job
Useful One-Liners
Submit a job script:
sbatch my_job.script
Start an interactive job with 6 cores for 2 hours of wall time in the general partition:
srun -p general -N 1 --ntasks-per-node=6 --time=02:00:00 --pty bash
View pending jobs:
squeue -u username -p general -t PENDING