A Hadoop/CUDA example would be great, and probably of general interest to the list.
Russ Sent from my iPhone On Mar 13, 2011, at 4:14 AM, "p...@cse.unl.edu" <p...@cse.unl.edu> wrote: > Hello Adarsh, > > At the time I was doing this integration, I used Pipes because it's C++ > based, which is easier for porting CUDA C code I assume. > > It actually is quite straightforward I think. What I have done can be > summarized into: > > 1) a simple abstract class that implements the common protocol for > Mapper/Reducer such that each CUDA mapper/reducer does not have to deal > with the Pipes communication. They just launch CUDA kernel and collect > results. > > 2) a set of simple shell scripts for uploading the mapper/reducer/CUDA > kernel to the Hadoop DFS and kickstarts the runs. > > 3) modify the CUDA program into a mapper inheriting from the wrapper class > so that it takes input from Pipes and dump results also to Pipes. > > If you want, I can try to find the code for you but I think you get the > idea. I didn't do a whole lot because it's mainly a class project. We just > want to demonstrate its feasibility and power saving. > > Thank you for your interest in our work! Let me know if you need further > assistance. > > > Peng >> Thanks Chen for your guidance, >> >> To translate CUDA in Hadoop environment, following are the options : >> >> -- Hadoop Streaming: Standard I/O ( Recommended by you ) >> -- Hadoop Pipes: C++ library, Socket connection, ( I'm looking for ) >> -- JNI, JNI-based CUDA wrapper (JCUDA) ( Done and able to run a Jcuda >> Program in Hadoop Cluster ) >> >> We use Hadoop Pipes for our proposed technique >> . MapReduce applications/CUDA kernel ? written in C++ >> >> So i.e why I am trying a C Cuda code to run through Hadoop Pipes, is it >> difficult or not possible. >> I am looking for a demo Cuda program that is up and running in Hadoop >> Cluster that clarifies my basic concepts so that I program accordingly >> in future. >> >> >> Looking forward for some more guidance. >> >> Thanks once again for your wishes . >> >> >> With best Regards, >> >> Adarsh Sharma >> >> >> >> He Chen wrote: >>> Hi, Adarsh Sharma >>> >>> For C code >>> >>> My friend employ hadoop streaming to run CUDA C code. You can send email >>> to >>> him. p...@cse.unl.edu. >>> >>> Best wishes! >>> >>> Chen >>> >>> >>> On Thu, Mar 3, 2011 at 11:18 PM, Adarsh Sharma >>> <adarsh.sha...@orkash.com>wrote: >>> >>> >>>> Dear all, >>>> >>>> I followed a fantastic tutorial and able to run the Wordcont C++ >>>> Program in >>>> Hadoop Cluster. >>>> >>>> >>>> http://cs.smith.edu/dftwiki/index.php/Hadoop_Tutorial_2.2_--_Running_C%2B%2B_Programs_on_Hadoop >>>> >>>> But know I want to run a Cuda Program in the Hadoop Cluster but results >>>> in >>>> errors. >>>> Is anyone has done it before and guide me how to do this. >>>> >>>> I attached the both files. Please find the attachment. >>>> >>>> >>>> Thanks & best Regards, >>>> >>>> Adarsh Sharma >>>> >>>> >>> >>> >> >> > >
