can someone help me how to disable the log info in terminal when type command "bin/yarn node" in YARN

2013-03-03 Thread pengwenwu2008
Hi All, can someone help me how to disable the log info in terminal when type command "bin/yarn node" in YARN bin/yarn node 13/03/04 02:24:43 INFO service.AbstractService: Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. 13/03/04 02:24:43 INFO service.AbstractService: Service:o

Re: [Hadoop-Help]About Map-Reduce implementation

2013-03-03 Thread Mayur Patil
Hello, Now I am slowly understanding Hadoop working. As I want to collect the logs from three machines including Master itself . My small query is which mode should I implement for this?? - Standalone Operation - Pseudo-Distributed Operation - Fully-

Re: How to Create file in HDFS using java Client with Permission

2013-03-03 Thread anil gupta
As per the error below, the user trying to write/read the file does not have appropriate permission. File not found org.apache.hadoop.security. AccessControlException: Permission denied: user=hadoop, access=WRITE, inode="/user/dasmohap/samir_tmp":dasmohap:dasmohap:drwxr-xr-x HTH, Anil On Fri, Ma

Re: hadoop pipes or java with jni?

2013-03-03 Thread Charles Earl
Julian, It has been my experience that jni may be a cause for concern if you multiple complex native libraries. You might want to run with valgrind if possible to verify that there are no memory issues. Given that you are using the single library, this should not be an issue. Uniquely named files a

Re: hadoop pipes or java with jni?

2013-03-03 Thread Michael Segel
I'm partial to using Java and JNI and then use the distributed cache to push the native libraries out to each node if not already there. But that's just me... ;-) HTH -Mike On Mar 3, 2013, at 6:02 PM, Julian Bui wrote: > Hi hadoop users, > > Trying to figure out which interface would be b

hadoop pipes or java with jni?

2013-03-03 Thread Julian Bui
Hi hadoop users, Trying to figure out which interface would be best and easiest to implement my application : 1) hadoop pipes or 2) java with jni or 3) something else that I'm not aware of yet, as a hadoop newbie. I will use hadoop to take pictures as input and create output jpeg pictures as outp

Re: mapper combiner and partitioner for particular dataset

2013-03-03 Thread Harsh J
The MultipleInputs class only supports mapper configuration per dataset. It does not let you specify a partitioner and combiner as well. You will need a custom written "high level" partitioner and combiner that can create multiple instances of sub-partitioners/combiners and use the most likely one

Re: hadoop filesystem corrupt

2013-03-03 Thread Mohit Vadhera
Can anybody let me know how to get block size back into the running system ? I want data_brk files back into the directory data ? I want the running filesystem should start it reading . $ sudo ls -l /mnt/san1/hdfs/hdfs/dfs/ total 16 drwx--. 3 hdfs hdfs 4096 Mar 1 12:50 data drwx-- 3 hdf

Fwd: mapper combiner and partitioner for particular dataset

2013-03-03 Thread Vikas Jadhav
Hello 1) I have multiple types of datasets as input to my hadoop job i want write my own inputformat (Exa. MyTableInputformat) and how to specify mapper partitioner combiner per dataset manner I know MultiFileInputFormat class but if i want to asscoite combiner and partitioner class it wont h

Re: What is Block Pool ID namenode?

2013-03-03 Thread Azuryy Yu
it is used for HDFS federation, if you have more than one NN in your cluster, block pool id is different for each NN. On Mar 3, 2013 1:49 PM, "Dhanasekaran Anbalagan" wrote: > Hi Guys, > > In my node namenode page I seen > > Started:Wed Feb 27 12:41:28 EST 2013 Version:2.0.0-cdh4.0.1, > 4d98eb718