Running Execution Hosts Concurrently

[adarsh]

Hi all,

I have established a Sge Cluster of 10 daemons ( one qmaster and 9 execd ) .

All execd nodes are also Hadoop Slave nodes.
I configured the cluster properly and run SGE jobs.

Hadoop executes its job by distributing into fragments to Tasktracker i.e why achieve performance on large datasets.

However when I ran any Hadoop parallel (hadoop pe) job ( simple wordount on 20GB file ) through SGE, it executes on single execution host

for e.g on ***@ws-test-lin

But I supposed it to run on atleast few execution hosts so that there is no network transfer of data.

I think it slows performance.

Is it my Queue Configuration problem or it always execute on one execution host.

Sun Grid Engine Documentation of Queue Configuration says it execute jobs on execution hosts concurrently, what I can't able to locate any useful document.

Please help


Thanks & Regards

Adarsh Sharma

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=302653

To unsubscribe from this discussion, e-mail: [users-***@gridengine.sunsource.net].