Re: Not able to Run C++ code in Hadoop Cluster

2011-03-15 Thread Adarsh Sharma
Is it possible to run C++/GPU Code in Map-Reduce Framework through Hadoop Streaming, if there is simple example , Please let me know. Thanks & best Regards, Adarsh Sharma He Chen wrote: Agree with Keith Wiley, we use streaming also. On Mon, Mar 14, 2011 at 11:40 AM, Keith Wiley wrote:

Re: Load testing in hadoop

2011-03-15 Thread Steve Loughran
On 15/03/11 04:59, Kannu wrote: Please tell me how to use synthetic load generator in hadoop or suggest me any other way of load testing in hadoop cluster. thanks, kannu terasort is the one most people use, as it generates its own datasets. Otherwise you need a few TB of data and some custom

Installing Hadoop on Debian Squeeze

2011-03-15 Thread Dieter Plaetinck
Hi, I see there are various posts claiming hadoop is available through official debian mirrors (for debian squeeze, i.e. stable): * http://www.debian-news.net/2010/07/17/apache-hadoop-in-debian-squeeze/ * http://blog.isabel-drost.de/index.php/archives/213/apache-hadoop-in-debian-squeeze However,

c++ program

2011-03-15 Thread Manish Yadav
hi can any one tell me how to run a simple hello world program written in c ++ on hadoop. i know that hadoop is for map reduce ,hadoop uses pipes for c++. but just for experimentation purpose can anybody tell me how to do this . can any one tell me how to run simple Hello worl program in c or

Re: c++ program

2011-03-15 Thread Stanley Xu
http://wiki.apache.org/hadoop/C%2B%2BWordCount The famous word count example in cxx could be found on hadoop wiki. Best wishes, Stanley Xu On Tue, Mar 15, 2011 at 7:09 PM, Manish Yadav wrote: > hi > can any one tell me how to run a simple hello world program written in c > ++ on hadoop. i kn

Re: is a single thread allocated to a single output file ?

2011-03-15 Thread maha
By the way, how do I know if my map task is single threaded (ie. one thread executing for each record ) ? and how to change that into multi-threading ? Thank you, Maha On Mar 12, 2011, at 9:11 PM, Harsh J wrote: > Hello, > > On Sat, Mar 12, 2011 at 3:51 PM, Jun Young Kim wrote: >> hi, >> >>

Re: is a single thread allocated to a single output file ?

2011-03-15 Thread maha
I found it :) http://hadoop.apache.org/mapreduce/docs/current/api/org/apache/hadoop/mapreduce/lib/map/MultithreadedMapper.html Maha On Mar 15, 2011, at 2:18 PM, maha wrote: > By the way, how do I know if my map task is single threaded (ie. one thread > executing for each record ) ? and how to

Reducer error/hang on small single file Map input

2011-03-15 Thread Aaron Baff
I'm trying to figure out why, when I use a single input file that is relatively small (< 3 MB, ~95K records), my Reducer hangs at 33% and the attempt never completes, with the output below. It looks like there's a problem when it tries to write to the log file, for some reason it gets Interrupte

Re: Load testing in hadoop

2011-03-15 Thread Bharath Mundlapudi
Gridmix v3 and TeraSort are good benchmarks to test overall hadoop cluster. But synthetic load generator is specifically for Namenode. You can refer the documentation below for how to run this tool. http://hadoop.apache.org/common/docs/r0.20.2/SLG_user_guide.html Also, there are various other