As soon as I get a moment to breathe, working on the SGE/Hadoop integration is next on my list of things to do. Hopefully I'll have more to report in the near future.
Daniel Rayson Ho wrote: > Wondering if anyone played with both projects?? > > http://opensolaris.org/os/project/hpc-stack/ > http://opensolaris.org/os/project/livehadoop/ > > Hadoop/MapReduce can solve some HPC problems quite well, and Grid > Engine can be used as the job scheduler. IMO it should be included in > the Hadoop Live CD! > > http://blogs.sun.com/ravee/entry/creating_hadoop_pe_under_sge > > Lastly, it's documented in Wikipedia that Grid Engine can't "do the > work in the same server or rack as the data", I am wondering if we can > add a soft request to qsub and bias the scheduling decisions a bit to > at least get around the problem?? > > http://en.wikipedia.org/wiki/Hadoop#Hadoop_with_Sun_Grid_Engine > > Rayson > _______________________________________________ > hpcdev-discuss mailing list > hpcdev-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/hpcdev-discuss >
