Re: HDFS upgrade problem of fsImage

cluster because I want hot upgrade on the prod cluster. On 2013-11-21 7:23 PM, Joshi, Rekha rekha_jo...@intuit.commailto:rekha_jo...@intuit.com wrote: Hi Azurry, This error occurs when FSImage finds previous fs state, and as log states you would need to either finalizeUpgrade or rollback

Re: HDFS upgrade problem of fsImage

Hi Azurry, This error occurs when FSImage finds previous fs state, and as log states you would need to either finalizeUpgrade or rollback to proceed.Below - bin/hadoop dfsadmin –finalizeUpgrade hadoop dfsadmin –rollback On side note for a small test cluster on which one might suspect you are

Re: how to store video,audio files into hdfs through java program

Hi, Think storing is not much of an issue, as much as some thought would be required in processing. Think you can, to the basic be able to use SequenceFileInputFormat, ByteArrayInputStream (and corresponding output) for the binary files. There are some experiments on audio, video here - Audio

Re: can someone help me how to disable the log info in terminal when type command bin/yarn node in YARN

Almost never silenced the logs on terminal, only tuned config for path/retention period on logs, so just top of mind, mostly –S/--silent for no logs, -V/--verbose for max logs possible works on executables, --help will confirm if it is possible. If it doesn't work, well it should :-) Thanks

Re: Hadoop Security and Kerberos

Hi Yongzhi, Well, I don't know if this will help, but I looked into source code, can see all token, authentication related features discussed in the design under- o.a.h.hdfs.security.*, o.a.h.mapreduce.security.*, o.a.h.security.* , o.a.h.security.authentication.* And HADOOP-4487 is marked fixed

Re: Hadoop HDFS and Mapreducer question

Refer hadoop put, get syntax for placing input files on hdfs (automate script) and pig dump, store after mapreduce to have your output directory - http://pig.apache.org/docs/r0.9.2/start.html#Pig+Tutorial+Files Thanks Rekha From: A Geek dw...@live.commailto:dw...@live.com Reply-To:

Re: How get information messages when a JobControl is used ?

problems. Regards Bertrand On Wed, Sep 12, 2012 at 12:09 PM, Joshi, Rekha rekha_jo...@intuit.commailto:rekha_jo...@intuit.com wrote: Hi Piter, JobControl just means there are multiple complex jobs, but you will see the information for each job on your hadoop web interface webhdfs still, wouldn't

Re: Non utf-8 chars in input

Hi Ajay, Try SequenceFileAsBinaryInputFormat ? Thanks Rekha On 11/09/12 11:24 AM, Ajay Srivastava ajay.srivast...@guavus.com wrote: Hi, I am using default inputFormat class for reading input from text files but the input file has some non utf-8 characters. I guess that TextInputFormat class

Re: Hadoop Streaming: Does not recognise PATH and CLASSPATH defined

Hi Andy, If you are referring to HADOOP_CLASSPATH, that is env variable on your cluster or effected via config xml.But if you need your own environment variables for streaming you may use -cmdenv PATH= on your streaming command.Or if you have specific jars for the streaming process -libjars

Re: error in shuffle in InMemoryMerger

Hi Abhay, Ideally the error line - Caused by: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local directory for output/map_128.out suggests you either do not have permissions for output folder or disk is full. Also 5 is not a big number on thread spawning,