make printable		see also:
		Condor for QC

Condor^® Project University Wisconsin

Simple simulations for Condor

1. FORS1

The following simple sketch illustrates the potential of using Condor, both on a dual processor and a multi-processor platform. We assume a model cascade with three levels (dependencies) and up to 6 independent ABs ("FORS1"). To simplify further, we also assume similar execution times per recipe and processor. Then the execution time for a batch of jobs is the fundamental cycle time of the system, by which we measure the system performance. Grey is idle time, blue is active time:

The very same cascade on a multi-processor platform with more than 6 nodes cannot be made any faster because of the dependencies.

2. OCAM

The survey instruments with up to 32 detectors will require a higher processing multiplexity. The idea is to have the pipeline recipe access a particular extension (both in raw and product files) and hence have the same AB running 32 times, indexed by detector ID. Then there is an additional job, #33, to merge all 32 product files into a single final product. In the following we take a very simple-minded cascade for a 32 detector instrument (1 BIAS, 1 FLAT; "OCAM"), executed on a small and on an adapted cluster. Again, similar execution and performance times are assumed:

3. Hybrid: OCAM plus FORS1

In general, CONDOR will be used by QC to execute all pipeline jobs. Hence a hybrid scenario (with the two cascades from above submitted to the same cluster) could look like following:

Obviously the idle processor cycles could be used for further compute jobs. The important point is that by careful adaption of the resources to the compute needs the cost-to-benefit ratio can improve significantly (gain of 20% throughput by just adding 1 more CPU to existing 32).