Re: how to use ContentSumary

thanks a lot ,i can successful exec the follow code Configuration conf = new Configuration(); conf.set(fs.defaultFS, 192.1.1.1:8020); FileSystem fs = FileSystem.get(conf); ContentSummary cs = fs.getContentSummary(new Path(/sequence)); System.out.println(cs is what ? + cs.toString());

Problem with streaming exact binary chunks

Hello, I wrote a very simple InputFormat and RecordReader to send binary data to mappers. Binary data can contain anything (including \n, \t, \r), here is what next() may actually send: public class MyRecordReader implements RecordReaderBytesWritable, BytesWritable { ... public

RE: Problem with streaming exact binary chunks

Hi, The only way that I could find was to override the various InputWriter and OutputWriter classes. as defined by the configuration settings stream.map.input.writer.class stream.map.output.reader.class stream.reduce.input.writer.class stream.reduce. output.reader.class which was painful.

Oozie SSH Action Issue

#Oozie SSH Action Issue: *Issue:* We are trying to run few commands on a particular host machine of our cluster. We chose SSH Action for the same. We have been facing this SSH issue for some time now. What might be the real issue here? Please point me towards the solution. *logs:* AUTH_FAILED:

using environment variables in XML conf files

I have the following path hard-coded for hadoop.tmp.dir in conf/core-site.xml: configuration property namehadoop.tmp.dir/name value/home/aim/tmp/hadoop-${user.name}/value /property /configuration Is it possible to replace the /home/aim with a substitutable

Java version with Hadoop 2.0

I am preparing to deploy multiple cluster / distros of Hadoop for testing / benchmarking. In my research I have noticed discrepancies in the version of the JDK that various groups are using. Example: Hortonworks is suggesting JDK6u31, CDH recommends either 6 or 7 providing you stick to some

Re: Java version with Hadoop 2.0

maybe you've already seen this. http://wiki.apache.org/hadoop/HadoopJavaVersions On Oct 9, 2013, at 2:16 PM, SF Hadoop sfhad...@gmail.commailto:sfhad...@gmail.com wrote: I am preparing to deploy multiple cluster / distros of Hadoop for testing / benchmarking. In my research I have noticed

Re: Java version with Hadoop 2.0

I hadn't. Thank you!!! Very helpful. Andy On Wed, Oct 9, 2013 at 2:25 PM, Patai Sangbutsarakum patai.sangbutsara...@turn.com wrote: maybe you've already seen this. http://wiki.apache.org/hadoop/HadoopJavaVersions On Oct 9, 2013, at 2:16 PM, SF Hadoop sfhad...@gmail.com wrote:

Re: Java version with Hadoop 2.0

also keep in mind, that java 6 no longer gets public updates from Oracle: http://www.oracle.com/technetwork/java/eol-135779.html - André On Wed, Oct 9, 2013 at 11:48 PM, SF Hadoop sfhad...@gmail.com wrote: I hadn't. Thank you!!! Very helpful. Andy On Wed, Oct 9, 2013 at 2:25 PM, Patai

Re: Java version with Hadoop 2.0

Does that mean for the new cluster, we probably try to start to aim to test/use/deploy at java 7? On Oct 9, 2013, at 3:05 PM, Andre Kelpe ake...@concurrentinc.com wrote: also keep in mind, that java 6 no longer gets public updates from Oracle:

issue about combine map task output

hi,all: i read the doc ,and have a question about combine,if i set min.num.spills.for.combine = 3 ,so when the 3rd spill happened,the combine also happend but i do not know when combine happend if it take three spill file as input ,and merge into one big spill file as output?

Re: using environment variables in XML conf files

Java provides a system property for ${user.home} that is usually set to the same as $HOME of the session on Linux. You can therefore use ${user.home}/tmp/hadoop-${user.name}. On Wed, Oct 9, 2013 at 9:49 PM, Andrew McDermott andrew.mcderm...@linaro.org wrote: I have the following path hard-coded