Re: What is the preferred way to pass a small number of configuration parameters to a mapper or reducer

2012-12-30 Thread Jonathan Bishop
E. Store them in hbase... On Sun, Dec 30, 2012 at 12:24 AM, Hemanth Yamijala yhema...@thoughtworks.com wrote: If it is a small number, A seems the best way to me. On Friday, December 28, 2012, Kshiva Kps wrote: Which one is current .. What is the preferred way to pass a small number

Re: What is the preferred way to pass a small number of configuration parameters to a mapper or reducer

2012-12-30 Thread Jonathan Bishop
On Monday, December 31, 2012, Jonathan Bishop wrote: E. Store them in hbase... On Sun, Dec 30, 2012 at 12:24 AM, Hemanth Yamijala yhema...@thoughtworks.com wrote: If it is a small number, A seems the best way to me. On Friday, December 28, 2012, Kshiva Kps wrote: Which one is current

using log4j to suppress messages

2012-10-29 Thread Jonathan Bishop
Hi, I would like to suppress messages on the console like... 12/10/29 16:56:03 INFO mapreduce.LoadIncrementalHFiles: Trying to load hfile=hdfs... So I added the following line to log4j.properties in my hadoop/conf dir... .mapreduce.LoadIncrementalHFiles=WARN and rebooted hbase and hadoop. But

Re: strategies to share information between mapreduce tasks

2012-09-26 Thread Jonathan Bishop
Yes, Giraph seems like the best way to go - it is mainly a vertex evaluation with message passing between vertices. Synchronization is handled for you. On Wed, Sep 26, 2012 at 8:36 AM, Jane Wayne jane.wayne2...@gmail.comwrote: hi, i know that some algorithms cannot be parallelized and adapted

Re: hbase puts in map tasks don't seem to run in parallel

2012-06-03 Thread Jonathan Bishop
your mappers in such a way that each writes to the key ranges corresponding to the regions hosted locally on the corresponding region sever. Cheers, Joep Sent from my iPhone On Jun 2, 2012, at 6:25 PM, Jonathan Bishop jbishop@gmail.com wrote: Hi, I am new to hadoop and hbase

hbase puts in map tasks don't seem to run in parallel

2012-06-02 Thread Jonathan Bishop
Hi, I am new to hadoop and hbase, but have spent the last few weeks learning as much as I can... I am attempting to create an hbase table during a hadoop job by simply doing puts to a table from each map task. I am hoping that each map task will use the regionserver on its node so that all 10 of

suppressing INFO messages to console

2012-05-17 Thread Jonathan Bishop
Hi, Anyone know an easy change to suppress the INFO messages from various logging classes. I assume you need to edit conf/log4j.properties, but everything I tried seems to not work. For instance, to suppress messages like... 12/05/17 17:24:47 INFO mapred.JobClient: Map output records=0 I