Discussion:
How to limit the total job count managed by the queue master ?
pmuller
2010-12-14 10:08:28 UTC
Permalink
Hi GridEngine users!

Is there a way to :
1 - Limit the total job count managed by the queue master
1 - Limit how many pending jobs an user can have

I'm looking for that because when users submit too many jobs like (>
10k), the qmaster become very slow (~10-15 seconds to answer a
request).
I know that moving the queue master to a bigger machine will certainly
improve the performances under huge load, but that kind of load is
unusual.

Thanks!

Philippe Muller

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=305381

To unsubscribe from this discussion, e-mail: [users-***@gridengine.sunsource.net].
reuti
2010-12-14 10:14:19 UTC
Permalink
Hi,
Post by pmuller
1 - Limit the total job count managed by the queue master
you can set "max_jobs" in SGE's configuration.
Post by pmuller
1 - Limit how many pending jobs an user can have
It's only possible in combination witht the running ones: "max_u_jobs" (`man sge_conf`).

-- Reuti
Post by pmuller
I'm looking for that because when users submit too many jobs like (>
10k), the qmaster become very slow (~10-15 seconds to answer a
request).
I know that moving the queue master to a bigger machine will certainly
improve the performances under huge load, but that kind of load is
unusual.
Thanks!
Philippe Muller
------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=305381
------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=305385

To unsubscribe from this discussion, e-mail: [users-***@gridengine.sunsource.net].
pmuller
2010-12-14 10:33:22 UTC
Permalink
Hi Reuti,

The manpage says "The number of active (not finished) jobs". I
interpreted that as "The number of running jobs". So I asked here
without testing... Sorry :-)
I'll try. Thanks!

Philippe Muller
Post by reuti
Hi,
Post by pmuller
1 - Limit the total job count managed by the queue master
you can set "max_jobs" in SGE's configuration.
Post by pmuller
1 - Limit how many pending jobs an user can have
It's only possible in combination witht the running ones: "max_u_jobs" (`man sge_conf`).
-- Reuti
Post by pmuller
I'm looking for that because when users submit too many jobs like (>
10k), the qmaster become very slow (~10-15 seconds to answer a
request).
I know that moving the queue master to a bigger machine will certainly
improve the performances under huge load, but that kind of load is
unusual.
Thanks!
Philippe Muller
------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=305381
------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=305385
------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=305392

To unsubscribe from this discussion, e-mail: [users-***@gridengine.sunsource.net].
reuti
2010-12-14 10:40:19 UTC
Permalink
Post by pmuller
Hi Reuti,
The manpage says "The number of active (not finished) jobs". I
interpreted that as "The number of running jobs".
For completeness: there is an extra setting in the scheduler configuration for the running jobs with a similar name "maxujobs" (unless you prefer to use an RQS for it).
Post by pmuller
So I asked here
without testing... Sorry :-)
No prob. - Reuti
Post by pmuller
I'll try. Thanks!
Philippe Muller
Post by reuti
Hi,
Post by pmuller
1 - Limit the total job count managed by the queue master
you can set "max_jobs" in SGE's configuration.
Post by pmuller
1 - Limit how many pending jobs an user can have
It's only possible in combination witht the running ones: "max_u_jobs" (`man sge_conf`).
-- Reuti
Post by pmuller
I'm looking for that because when users submit too many jobs like (>
10k), the qmaster become very slow (~10-15 seconds to answer a
request).
I know that moving the queue master to a bigger machine will certainly
improve the performances under huge load, but that kind of load is
unusual.
Thanks!
Philippe Muller
------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=305381
------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=305385
------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=305392
------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=305395

To unsubscribe from this discussion, e-mail: [users-***@gridengine.sunsource.net].
Loading...