Skip to Content

Max concurrent RT engines is ignored

Dear experts,

my use case is to execute initial load jobs for a large number of backend systems in an automated fashion without any resource contention.

I've configured an IdM 7.2 SP7 system with one dispatcher to start at most two concurrent runtime engines. This is configured both as a global setting and on the single dispatcher (see screenshots).

Then I execute a normal job with one toGeneric pass whose entry script uses uRunJobNow() to start the ~50 initial load jobs. My expectation was that these jobs would now be executed in sequence, with at most two jobs running in parallel.

Unfortunately, that's not the case. In fact, up to 16 runtimes are started in parallel, so resource contention occurs with the usual problems (OutOfMemory, timeout etc.). The dispatcher log emits warning messages:

Not allowed to start ANY runtimes (right now running runtimes = 16, allowed total runtimes = 2

Question: is there anything I need to change in the dispatcher configuration to achieve the desired behavior?

Best regards,

Lambert

Add comment
10|10000 characters needed characters exceeded

  • Follow
  • Get RSS Feed

1 Answer

  • Best Answer
    avatar image
    Former Member
    Sep 19, 2014 at 10:35 AM

    I think that in SP7 it simply counts running jobs (mc_jobs.state=2), so if a job takes a long time to start and the check frequency is low (1 or 2 seconds for instance) then the dispatcher has time to start more runtimes before the state of 2 jobs are changed to 2. I believe that later versions of the dispatcher checks the pids that were started to see how many are still active.

    To test, change the max engines to start to 1, increase the dispatchers policy/check interval to 30 seconds. This should result in 1 runtime being started every 30 seconds and make sure that the jobs have had time to start properly and set their state to running. Setting the state to 2 should be almost instantaneous though so that 16 have time to start almost sounds like some setting is being ignored. You might also want to check that there isn't an excessive amount of javaclasses getting loaded that bloats the startup times.

    Br,

    Per Christian

    Message was edited by: Per Krabsetsve

    Add comment
    10|10000 characters needed characters exceeded

    • Thanks Chris,

      that's very helpful feedback. I changed "Max RT engines to start" to 1 as per your suggestion. Check interval was already 5 sec - the SAP recommendation for environments with one dispatcher - so I left that alone for now, assuming it is OK.

      Changing "Max RT engines to start" alone did not result in any improvement yet, however. What fixed the problem was to add an artificial wait time using uSleep(10000) after each invocation of uRunJobNow(). So it seems the key part of your tip was:

      make sure that the jobs have had time to start properly and set their state to running

      With these two changes in place, job execution proceeds with at most two jobs in parallel as expected.

      Best regards,

      Lambert