Queuing and Scheduling

Useful HPC commands/links

Find out what queues are available:

qconf -sql

Find out information about a named queue

qconf -sq serial.q

Number of slots usually equals number of cores

Good intro
http://talby.rcs.manchester.ac.uk/~ri/_linux_and_hpc_lib/sge_intro.html

Translation between PBS and SGE
http://wiki.ibest.uidaho.edu/index.php/Tutorials:_SGE_PBS_Converting

Using environment vars/arguments
http://stackoverflow.com/questions/9719337/how-to-properly-pass-on-an-environment-variable-to-sun-grid-engine

Using python to poll the status of a job:

    cmd = 'qstat -f | grep %s' % job_id
    proc = subprocess.Popen([cmd], stdout=subprocess.PIPE, shell=True)
    output = proc.stdout.read()
    if output==None:
        return output
    
    else:
        status = output.split()[4].strip()
        return status


Explantion of queue error state E:

http://www.gridengine.info/2008/01/20/understanding-queue-error-state-e/

Compiling WPS

To compile WPS, had to modify configure.wps to indclude:

COMPRESSION_LIBS        =  -L/home/slha/usr/lib -ljasper \
                           -L/usr/lib -lpng12
COMPRESSION_INC         = -I/home/slha/usr/include

How many nodes?

Until now I didn't know how many cores I had to work on. The info in /proc/cpuinfo just didn't seem to tie in with what the vendor had supplied.  Then the penny dropped, /proc/cpuinfo only tells you about the current node, while I wanted to find out about the whole cluster. OK, seems obvious now, but here is some useful info I collected along the way.

One thing I haven't found is how to find out how many nodes (hosts) are in your local domain?

I'm adopting the following convention
1. A "chassis" houses one or more nodes.
2. A node contains one or more sockets.
3. A socket holds one processor.
4. A processor contains one or more (CPU) cores.
5. The cores perform FLOPS.

Command

Description

$cat /proc/cpuinfo

List processor information. Each physical chip has a unique ""physical id". Cores of the same chip have the same physical ID but different core ids. see http://www.richweb.com/cpu_info This will only tell you about the local node! You will need to SSH into other nodes to work out about them as well. Useful flags lm 64 bit support vmx Intel Virtualization Technology support ht indicates Hyperthreading support

Useful AWK script here:
http://superuser.com/questions/388115/interpreting-output-of-cat-proc-cpuinfo?rq=1

The head node on my cluster has two sockets, with quad core chips, but with hyperthreading enabled, so that it looks like 16 logical 'processors'.  The calculation nodes also have two sockets, with quad core CPUS, so 8 cores per node. In total I have 216 calculation cores to play with!