Re: mysterious NumberFormatException

2011-12-19 Thread Ted Yu
Thanks for the analysis, Nicolas. Is it reasonable to change allProcessInfo to Map so that we don't encounter this problem by avoiding parsing large integer ? On Mon, Dec 19, 2011 at 9:59 PM, Tsz Wo Sze wrote: > Hi, > > It looks like that the ppid is a 64-bit positive integer but Java long is >

Re: mysterious NumberFormatException

2011-12-19 Thread Tsz Wo Sze
Hi, It looks like that the ppid is a 64-bit positive integer but Java long is signed and so only works with 63-bit positive integers.  In your case,   2^64 > 18446743988060683582 > 2^63. Therefore, there is a NFE.  I think it is a bug in ProcfsBasedProcessTree. Regards, Nicholas Sze _

mysterious NumberFormatException

2011-12-19 Thread Ted Yu
Hi, HBase PreCommit builds frequently gave us mysterious NumberFormatException From https://builds.apache.org/job/PreCommit-HBASE-Build/553//testReport/org.apache.hadoop.hbase.mapreduce/TestHFileOutputFormat/testMRIncrementalLoad/ : 2011-12-20 01:44:01,180 WARN [main] mapred.JobClient(784): No j

[jira] [Resolved] (MAPREDUCE-2165) Add support for skipping records in streaming for new api.

2011-12-19 Thread Harsh J (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved MAPREDUCE-2165. Resolution: Won't Fix Resolving as Won't Fix - See MAPREDUCE-1932 (parent) for the reasoning.

Re: Reduce output is strange

2011-12-19 Thread Robert Evans
Oh I forgot to say that part of the Random Characters are actually random characters. Sequence files store a set of random characters as synch points within the file. This allows for splitting the file easily without a high risk that the random sequence appears inside the data itself just by c

Re: Reduce output is strange

2011-12-19 Thread Robert Evans
It looks mostly correct to me. I am not an expert on sequence files, and I have not checked the text against the spec nor have I checked the binary numbers in it to be sure they add up to the correct lengths etc, but it looks good from a first glance. I can see the SEQ tag at the beginning to

Reduce output is strange

2011-12-19 Thread Pedro Costa
Hi, In the hadoop MapReduce, I've executed the webdatascan example, and the reduce output is in a SequeceFile. The result is shows here ( http://paste.lisp.org/display/126572). What's the trash (random characters), like "u 265 100 330 320 252 " \n # ; 374 5 211 V ' 340 376" in the output? Is t