Build failed in Jenkins: Hadoop-Common-0.23-Build #296

[Apache Jenkins Server]

See 

Changes:

[bobby] svn merge -c 1354181 FIXES: MAPREDUCE-4228. 
mapreduce.job.reduce.slowstart.completedmaps is not working properly (Jason 
Lowe via bobby)

[bobby] svn merge -c 1354093. FIXES: HADOOP-8129. ViewFileSystemTestSetup 
setupForViewFileSystem is erring (Ahmed Radwan and Ravi Prakash via bobby)

--
[...truncated 181867 lines...]
Running org.apache.hadoop.fs.viewfs.TestViewFsWithAuthorityLocalFs
Tests run: 42, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.183 sec
Running org.apache.hadoop.fs.viewfs.TestViewFsLocalFs
Tests run: 42, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.07 sec
Running org.apache.hadoop.fs.TestGlobPattern
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.162 sec
Running org.apache.hadoop.fs.TestS3_LocalFileContextURI
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.13 sec
Running org.apache.hadoop.fs.TestLocalFSFileContextCreateMkdir
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.732 sec
Running org.apache.hadoop.fs.TestHarFileSystem
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.335 sec
Running org.apache.hadoop.fs.TestFileSystemCaching
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.733 sec
Running org.apache.hadoop.fs.TestLocalFsFCStatistics
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.542 sec
Running org.apache.hadoop.fs.TestHardLink
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.325 sec
Running org.apache.hadoop.fs.TestCommandFormat
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.169 sec
Running org.apache.hadoop.fs.TestLocal_S3FileContextURI
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.071 sec
Running org.apache.hadoop.fs.TestLocalFileSystem
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.809 sec
Running org.apache.hadoop.fs.TestFcLocalFsPermission
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.651 sec
Running org.apache.hadoop.fs.TestListFiles
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.562 sec
Running org.apache.hadoop.fs.TestPath
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.817 sec
Running org.apache.hadoop.fs.kfs.TestKosmosFileSystem
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.595 sec
Running org.apache.hadoop.fs.TestGlobExpander
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.107 sec
Running org.apache.hadoop.fs.TestFilterFileSystem
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.617 sec
Running org.apache.hadoop.fs.TestFcLocalFsUtil
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.606 sec
Running org.apache.hadoop.fs.TestGetFileBlockLocations
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.796 sec
Running org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract
Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.858 sec
Running org.apache.hadoop.fs.s3.TestINode
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.117 sec
Running org.apache.hadoop.fs.s3.TestS3Credentials
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.205 sec
Running org.apache.hadoop.fs.s3.TestS3FileSystem
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.221 sec
Running org.apache.hadoop.fs.TestDU
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.289 sec
Running org.apache.hadoop.record.TestBuffer
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.149 sec
Running org.apache.hadoop.record.TestRecordVersioning
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.19 sec
Running org.apache.hadoop.record.TestRecordIO
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.215 sec
Running org.apache.hadoop.metrics2.source.TestJvmMetrics
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.43 sec
Running org.apache.hadoop.metrics2.util.TestSampleStat
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.139 sec
Running org.apache.hadoop.metrics2.util.TestMetricsCache
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.165 sec
Running org.apache.hadoop.metrics2.lib.TestInterns
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.277 sec
Running org.apache.hadoop.metrics2.lib.TestMetricsAnnotations
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.498 sec
Running org.apache.hadoop.metrics2.lib.TestMutableMetrics
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.454 sec
Running org.apache.hadoop.metrics2.lib.TestUniqNames
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.145 sec
Running org.apache.hadoop.metrics2.lib.TestMetricsRegistry
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.428 sec
Running org.apache.ha

Modification of wiki page "Distributions and Commercial Support"

[Olivier Lamy]

Hi,
What is the process to modify the page
http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
?
My company (Talend) will be happy to see some modifications.

Changing the Talend part to:

Talend - The Open Source Integration Company
* Talend Platform for Big Data(link:
http://www.talend.com/products-big-data/) includes support and
management tools for all the major Apache Hadoop distributions.
* Talend Open Studio for Big Data(link:
http://www.talend.com/products-big-data/open-studio-bd.php) is an
Apache License Eclipse IDE, that provides a set of graphical
components for HDFS, HBase, Pig, Sqoop and Hive. A configuration of a
big data connection is represented graphically and the underlying code
is automatically generated and then can be deployed as a service,
executable or stand-alone job.

Thanks in advance,
--
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy

[jira] [Created] (HADOOP-8539) RMContainerImpl State Machine Doesn't Handle RELEASED State from AM Properly

[Jack Dintruff (JIRA)]

Jack Dintruff created HADOOP-8539:
-

 Summary: RMContainerImpl State Machine Doesn't Handle RELEASED 
State from AM Properly
 Key: HADOOP-8539
 URL: https://issues.apache.org/jira/browse/HADOOP-8539
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 0.23.3
Reporter: Jack Dintruff


Encountered this error with an RMContainerImpl unable to go from the RUNNING 
state to the RELEASED state.  Upon further inspection, the state machine has no 
means of dealing with this and since the RELEASED state can be sent from the AM 
at any time, it should be able to handle this in both the RUNNING state and the 
ALLOCATED state.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Modification of wiki page "Distributions and Commercial Support"

[Roman Shaposhnik]

On Thu, Jun 28, 2012 at 3:27 AM, Olivier Lamy  wrote:
> Hi,
> What is the process to modify the page
> http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
> ?

On a related note, I would like to add Apache Bigtop (incubating)
Hadoop distribution
to that wiki page.

Can anybody help me with that? The wiki seems to be locked.

Thanks,
Roman.

Re: Modification of wiki page "Distributions and Commercial Support"

[Konstantin Boudnik]

I will, I have the access.

Cos

On Thu, Jun 28, 2012 at 09:52AM, Roman Shaposhnik wrote:
> On Thu, Jun 28, 2012 at 3:27 AM, Olivier Lamy  wrote:
> > Hi,
> > What is the process to modify the page
> > http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
> > ?
> 
> On a related note, I would like to add Apache Bigtop (incubating)
> Hadoop distribution
> to that wiki page.
> 
> Can anybody help me with that? The wiki seems to be locked.
> 
> Thanks,
> Roman.

Re: Modification of wiki page "Distributions and Commercial Support"

[Olivier Lamy]

Thanks!
Note: Harsh J  did it and give me karma to modify the page.

--
Olivier


2012/6/28 Konstantin Boudnik :
> I will, I have the access.
>
> Cos
>
> On Thu, Jun 28, 2012 at 09:52AM, Roman Shaposhnik wrote:
>> On Thu, Jun 28, 2012 at 3:27 AM, Olivier Lamy  wrote:
>> > Hi,
>> > What is the process to modify the page
>> > http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
>> > ?
>>
>> On a related note, I would like to add Apache Bigtop (incubating)
>> Hadoop distribution
>> to that wiki page.
>>
>> Can anybody help me with that? The wiki seems to be locked.
>>
>> Thanks,
>> Roman.



-- 
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy

[jira] [Created] (HADOOP-8540) Compression with non-default codec's fail on winhadoop

[Trupti Dhavle (JIRA)]

Trupti Dhavle created HADOOP-8540:
-

 Summary: Compression with non-default codec's fail on winhadoop
 Key: HADOOP-8540
 URL: https://issues.apache.org/jira/browse/HADOOP-8540
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Trupti Dhavle


I was trying to run the compression test case with GZipCodec. However the test 
fails with NPE. It is running fine with DefaultCodec.

With text input
c:\Workspace\winhadoopqe>c:\hdp\branch-1-win\bin\hadoop jar c:\hdp\branch-1-win\
build\hadoop-examples-1.1.0-SNAPSHOT.jar sort "-Dmapred.compress.map.output=true
" "-Dmapred.map.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec
" "-Dmapred.output.compress=true"  "-Dmapred.output.compression.type=NONE"  "-Dm
apred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec" -outKey
org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text  Compression/texti
nput Compression/textoutput-123


Running on 1 nodes to sort from hdfs://localhost:8020/user/Administrator/Compres
sion/textinput into hdfs://localhost:8020/user/Administrator/Compression/textout
put-123 with 1 reduces.
Job started: Thu Jun 28 10:02:18 PDT 2012
12/06/28 10:02:18 INFO mapred.FileInputFormat: Total input paths to process : 1
12/06/28 10:02:18 INFO mapred.JobClient: Running job: job_201206271409_0045
12/06/28 10:02:19 INFO mapred.JobClient:  map 0% reduce 0%
12/06/28 10:02:36 INFO mapred.JobClient: Task Id : attempt_201206271409_0045_m_0
0_0, Status : FAILED
java.lang.NullPointerException
at org.apache.hadoop.mapred.IFile$Writer.(IFile.java:102)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask
.java:1407)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1
298)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:437)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:271)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
tion.java:1103)
at org.apache.hadoop.mapred.Child.main(Child.java:265)



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Modification of wiki page "Distributions and Commercial Support"

[Konstantin Boudnik]

Sorry, did what? The page doesn't have any references to the BigTop as of now.

Cos

On Thu, Jun 28, 2012 at 07:11PM, Olivier Lamy wrote:
> Thanks!
> Note: Harsh J  did it and give me karma to modify the page.
> 
> --
> Olivier
> 
> 
> 2012/6/28 Konstantin Boudnik :
> > I will, I have the access.
> >
> > Cos
> >
> > On Thu, Jun 28, 2012 at 09:52AM, Roman Shaposhnik wrote:
> >> On Thu, Jun 28, 2012 at 3:27 AM, Olivier Lamy  wrote:
> >> > Hi,
> >> > What is the process to modify the page
> >> > http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
> >> > ?
> >>
> >> On a related note, I would like to add Apache Bigtop (incubating)
> >> Hadoop distribution
> >> to that wiki page.
> >>
> >> Can anybody help me with that? The wiki seems to be locked.
> >>
> >> Thanks,
> >> Roman.
> 
> 
> 
> -- 
> Olivier Lamy
> Talend: http://coders.talend.com
> http://twitter.com/olamy | http://linkedin.com/in/olamy

Re: Modification of wiki page "Distributions and Commercial Support"

[Olivier Lamy]

Ooops I misread :-)
Sorry for confusion

--
Olivier
Le 28 juin 2012 20:58, "Konstantin Boudnik"  a écrit :

> Sorry, did what? The page doesn't have any references to the BigTop as of
> now.
>
> Cos
>
> On Thu, Jun 28, 2012 at 07:11PM, Olivier Lamy wrote:
> > Thanks!
> > Note: Harsh J  did it and give me karma to modify the page.
> >
> > --
> > Olivier
> >
> >
> > 2012/6/28 Konstantin Boudnik :
> > > I will, I have the access.
> > >
> > > Cos
> > >
> > > On Thu, Jun 28, 2012 at 09:52AM, Roman Shaposhnik wrote:
> > >> On Thu, Jun 28, 2012 at 3:27 AM, Olivier Lamy 
> wrote:
> > >> > Hi,
> > >> > What is the process to modify the page
> > >> >
> http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
> > >> > ?
> > >>
> > >> On a related note, I would like to add Apache Bigtop (incubating)
> > >> Hadoop distribution
> > >> to that wiki page.
> > >>
> > >> Can anybody help me with that? The wiki seems to be locked.
> > >>
> > >> Thanks,
> > >> Roman.
> >
> >
> >
> > --
> > Olivier Lamy
> > Talend: http://coders.talend.com
> > http://twitter.com/olamy | http://linkedin.com/in/olamy
>

[jira] [Created] (HADOOP-8541) Better high-percentile latency metrics

[Andrew Wang (JIRA)]

Andrew Wang created HADOOP-8541:
---

 Summary: Better high-percentile latency metrics
 Key: HADOOP-8541
 URL: https://issues.apache.org/jira/browse/HADOOP-8541
 Project: Hadoop Common
  Issue Type: Improvement
  Components: metrics
Reporter: Andrew Wang


Based on discussion in HBASE-6261 and with some HDFS devs, I'd like to make 
better high-percentile latency metrics a part of hadoop-common.

I've already got a working implementation of [1], an efficient algorithm for 
estimating quantiles on a stream of values. It allows you to specify arbitrary 
quantiles to track (e.g. 50th, 75th, 90th, 95th, 99th), along with tight error 
bounds. This estimator can be snapshotted and reset periodically to get a feel 
for how these percentiles are changing over time.

I propose creating a new MutableQuantiles class that does this. [1] isn't 
completely without overhead (~1MB memory for reasonably sized windows), which 
is why I hesitate to add it to the existing MutableStat class.

[1] Cormode, Korn, Muthukrishnan, and Srivastava. "Effective Computation of 
Biased Quantiles over Data Streams" in ICDE 2005.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira