Max concurrent RT engines is ignored
my use case is to execute initial load jobs for a large number of backend systems in an automated fashion without any resource contention.
I've configured an IdM 7.2 SP7 system with one dispatcher to start at most two concurrent runtime engines. This is configured both as a global setting and on the single dispatcher (see screenshots).
Then I execute a normal job with one toGeneric pass whose entry script uses uRunJobNow() to start the ~50 initial load jobs. My expectation was that these jobs would now be executed in sequence, with at most two jobs running in parallel.
Unfortunately, that's not the case. In fact, up to 16 runtimes are started in parallel, so resource contention occurs with the usual problems (OutOfMemory, timeout etc.). The dispatcher log emits warning messages:
Not allowed to start ANY runtimes (right now running runtimes = 16, allowed total runtimes = 2
Question: is there anything I need to change in the dispatcher configuration to achieve the desired behavior?
I think that in SP7 it simply counts running jobs (mc_jobs.state=2), so if a job takes a long time to start and the check frequency is low (1 or 2 seconds for instance) then the dispatcher has time to start more runtimes before the state of 2 jobs are changed to 2. I believe that later versions of the dispatcher checks the pids that were started to see how many are still active.
To test, change the max engines to start to 1, increase the dispatchers policy/check interval to 30 seconds. This should result in 1 runtime being started every 30 seconds and make sure that the jobs have had time to start properly and set their state to running. Setting the state to 2 should be almost instantaneous though so that 16 have time to start almost sounds like some setting is being ignored. You might also want to check that there isn't an excessive amount of javaclasses getting loaded that bloats the startup times.
Message was edited by: Per Krabsetsve