LSF: Submit an LSF Job

The LSF queueing software manages the hundreds of computers in the Odyssey cluster. To run software on the cluster, you login to odyssey.fas.harvard.edu (the “head node”, which is really a nickname for three identical computers, iliadaccess01, 02, and 03). Then you use the bsub command to “submit” a job to an appropriate queue, where it waits for a computer to become free. At that point, LSF sends your job to that other computer, where it runs. When it finishes, LSF will email you.

Note that the iliadaccess machines are only for submitting jobs. If you try to run a program there without bsub, it will be stopped after a couple minutes.

Here is an LSF cheat sheet describing the basic LSF commands.

To run a job, first create a submission script. Make a file somewhere on Odyssey (in your home directory or a sub-directory of that) called hello.bsub that looks like this:


#!/bin/sh

#BSUB -u someguy@fas.harvard.edu

#BSUB -J my_blast

#BSUB -o blast.out

#BSUB -e blast.err

#BSUB -W 2:00

#BSUB -q normal_serial



blastall -p blastp -e 1e-5 -d nr myseqs.fasta

(Of course you’ll want to change the email address, program name, filenames, etc.) The lines beginning with #BSUB are instructions to LSF.

-u email says where to send the LSF reports
-J name names the job, which is useful if you’re running multiple jobs and run a bjobs command
-q queue names the queue you’re submitting the job to
-e filename and -o filename instruct LSF where to print output from the job. Without these definitions, the output will get emailed to you, or disappear if the output is many megabytes. (-o is stdout, the usual output of a program, while -e, stderr, is where errors usually go)
-W hh:mm defines how long the job will run for. When the job reaches this limit it will automatically be killed if it has not finished running. This is useful for telling the scheduler how long you think you job will run for so that it can fit you in to the queue easier.

After the instructions to LSF, you just type one or more commands that you would run on the command line. The above scripts runs a biology program called BLAST. Commands often have options, input filenames, etc. If you want to run a program you installed in your own directory, make sure to write ./programname

To submit the job, at the command line, type bsub, a less-than sign, and the name of the submission script:


[hptc@iliadaccess01 ~]$ bsub < hello.bsub.

Job <797679> is submitted to queue <short_serial>.

As shown above, LSF will return a job id, which you’ll see when you use the bjobs command. Also,if you want to stop the job, use “bkill jobid”.

It is possible to run interactive jobs on Odyssey. People often use this for Matlab or R sessions, to run one Matlab command, get a result, run the next comand.