Build failed in Jenkins: Hadoop-Common-trunk #543

[Apache Jenkins Server]

See 

Changes:

[suresh] HADOOP-8815. RandomDatum needs to override hashCode(). Contributed by 
Brandon Li.

[suresh] HADOOP-8813. Add InterfaceAudience and InterfaceStability annotations 
to RPC Server and Client classes. Contributed by Brandon Li.

[shv] Move MAPREDUCE-4645 under 0.23.4 release section in CHANGES.txt

[shv] MAPREDUCE-4645. Provide a random seed to Slive to make the sequence of 
file names deterministic. Contributed by Ravi Prakash.

[suresh] Moving HDFS-234 from trunk to Release 2.0.0-alpha section

--
[...truncated 27771 lines...]
[DEBUG]   (s) debug = false
[DEBUG]   (s) effort = Default
[DEBUG]   (s) failOnError = true
[DEBUG]   (s) findbugsXmlOutput = false
[DEBUG]   (s) findbugsXmlOutputDirectory = 

[DEBUG]   (s) fork = true
[DEBUG]   (s) includeTests = false
[DEBUG]   (s) localRepository =id: local
  url: file:///home/jenkins/.m2/repository/
   layout: none

[DEBUG]   (s) maxHeap = 512
[DEBUG]   (s) nested = false
[DEBUG]   (s) outputDirectory = 

[DEBUG]   (s) outputEncoding = UTF-8
[DEBUG]   (s) pluginArtifacts = 
[org.codehaus.mojo:findbugs-maven-plugin:maven-plugin:2.3.2:, 
com.google.code.findbugs:bcel:jar:1.3.9:compile, 
org.codehaus.gmaven:gmaven-mojo:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-api:jar:1.3:compile, 
org.codehaus.gmaven.feature:gmaven-feature-api:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-1.5:jar:1.3:compile, 
org.codehaus.gmaven.feature:gmaven-feature-support:jar:1.3:compile, 
org.codehaus.groovy:groovy-all-minimal:jar:1.5.8:compile, 
org.apache.ant:ant:jar:1.7.1:compile, 
org.apache.ant:ant-launcher:jar:1.7.1:compile, jline:jline:jar:0.9.94:compile, 
org.codehaus.plexus:plexus-interpolation:jar:1.1:compile, 
org.codehaus.gmaven:gmaven-plugin:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-loader:jar:1.3:compile, 
org.codehaus.gmaven.runtime:gmaven-runtime-support:jar:1.3:compile, 
org.sonatype.gshell:gshell-io:jar:2.0:compile, 
com.thoughtworks.qdox:qdox:jar:1.10:compile, 
org.apache.maven.shared:file-management:jar:1.2.1:compile, 
org.apache.maven.shared:maven-shared-io:jar:1.1:compile, 
commons-lang:commons-lang:jar:2.4:compile, 
org.slf4j:slf4j-api:jar:1.5.10:compile, 
org.sonatype.gossip:gossip:jar:1.2:compile, 
org.apache.maven.reporting:maven-reporting-impl:jar:2.1:compile, 
commons-validator:commons-validator:jar:1.2.0:compile, 
commons-beanutils:commons-beanutils:jar:1.7.0:compile, 
commons-digester:commons-digester:jar:1.6:compile, 
commons-logging:commons-logging:jar:1.0.4:compile, oro:oro:jar:2.0.8:compile, 
xml-apis:xml-apis:jar:1.0.b2:compile, 
org.codehaus.groovy:groovy-all:jar:1.7.4:compile, 
org.apache.maven.reporting:maven-reporting-api:jar:3.0:compile, 
org.apache.maven.doxia:doxia-core:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-logging-api:jar:1.1.3:compile, 
xerces:xercesImpl:jar:2.9.1:compile, 
commons-httpclient:commons-httpclient:jar:3.1:compile, 
commons-codec:commons-codec:jar:1.2:compile, 
org.apache.maven.doxia:doxia-sink-api:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-decoration-model:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-site-renderer:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-module-xhtml:jar:1.1.3:compile, 
org.apache.maven.doxia:doxia-module-fml:jar:1.1.3:compile, 
org.codehaus.plexus:plexus-i18n:jar:1.0-beta-7:compile, 
org.codehaus.plexus:plexus-velocity:jar:1.1.7:compile, 
org.apache.velocity:velocity:jar:1.5:compile, 
commons-collections:commons-collections:jar:3.2:compile, 
org.apache.maven.shared:maven-doxia-tools:jar:1.2.1:compile, 
commons-io:commons-io:jar:1.4:compile, 
com.google.code.findbugs:findbugs-ant:jar:1.3.9:compile, 
com.google.code.findbugs:findbugs:jar:1.3.9:compile, 
com.google.code.findbugs:jsr305:jar:1.3.9:compile, 
com.google.code.findbugs:jFormatString:jar:1.3.9:compile, 
com.google.code.findbugs:annotations:jar:1.3.9:compile, 
dom4j:dom4j:jar:1.6.1:compile, jaxen:jaxen:jar:1.1.1:compile, 
jdom:jdom:jar:1.0:compile, xom:xom:jar:1.0:compile, 
xerces:xmlParserAPIs:jar:2.6.2:compile, xalan:xalan:jar:2.6.0:compile, 
com.ibm.icu:icu4j:jar:2.6.1:compile, asm:asm:jar:3.1:compile, 
asm:asm-analysis:jar:3.1:compile, asm:asm-commons:jar:3.1:compile, 
asm:asm-util:jar:3.1:compile, asm:asm-tree:jar:3.1:compile, 
asm:asm-xml:jar:3.1:compile, jgoodies:plastic:jar:1.2.0:compile, 
org.codehaus.plexus:plexus-resources:jar:1.0-alpha-4:compile, 
org.codehaus.plexus:plexus-utils:jar:1.5.1:compile]
[DEBUG]   (s) project = MavenProject: 
org.apache.hadoop:hadoop-common-project:3.0.0-SNAPSHOT @ 

[DEBUG]   (s) relaxed = false
[DEBUG]   (s) remoteArtifactRepositories 

[jira] [Created] (HADOOP-8838) Colorize the test-patch output sent to JIRA

[Harsh J (JIRA)]

Harsh J created HADOOP-8838:
---

 Summary: Colorize the test-patch output sent to JIRA
 Key: HADOOP-8838
 URL: https://issues.apache.org/jira/browse/HADOOP-8838
 Project: Hadoop Common
  Issue Type: Improvement
  Components: build
Reporter: Harsh J
Assignee: Harsh J
Priority: Trivial


It would be helpful to mark the -1s in red and +1s in green. Helps avoid 
missing stuff like findbugs warnings, etc., we've been bitten by. Also helps 
run through the results faster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8839) test-patch's -1 on @author tag presence doesn't cause a -1 to the overall result

[Harsh J (JIRA)]

Harsh J created HADOOP-8839:
---

 Summary: test-patch's -1 on @author tag presence doesn't cause a 
-1 to the overall result
 Key: HADOOP-8839
 URL: https://issues.apache.org/jira/browse/HADOOP-8839
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Reporter: Harsh J
Priority: Trivial


As observed on HADOOP-8838.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [REVIEW] Hadoop-1.1.0-RC4

[Arun C Murthy]

I'd also recommend MR-4558, MR-4328 & H-8748+friends. Thoughts?

thanks,
Arun

On Sep 24, 2012, at 9:58 PM, Matt Foley wrote:

> Okay.
> 
> On Mon, Sep 24, 2012 at 7:05 PM, Uma Maheswara Rao G
> wrote:
> 
>> Hi Matt,
>> 
>> Good to include HDFS-3701 also. I set the fixed versions as 1.1.0 as well.
>> 
>> Regards,
>> Uma
>> 
>> On Tue, Sep 25, 2012 at 7:29 AM, Matt Foley  wrote:
>>> Hi all,
>>> Since the Review RC went out, I've been asked to include the following
>>> fixes:
>>> 
>>> Three significant bugs:
>>> HDFS-3461 hftp uses http to the Namenode's https port, which doesn't work
>>> HDFS-3846 NN deadlock
>>> HDFS-3596 decrease the incidence of corrupted logs after disk full
>>> conditions
>>> and a related enhancement:
>>> HDFS-3521 deal with edit log corruption if it occurs
>>> 
>>> Four unit test fixes (obviously not critical, but good to have):
>>> HDFS-3698
>>> HDFS-3966
>>> MAPREDUCE-4673
>>> MAPREDUCE-4675
>>> 
>>> and four performance enhancements:
>>> HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
>>> HDFS-2751 Datanode drops OS cache behind reads even for short reads
>>> MAPREDUCE-1906 default minimum heartbeat
>>> MAPREDUCE-3289 use fadvise
>>> 
>>> I will spin a new RC with these fixes, and hope to offer it for vote by
>>> Wednesday.
>>> Thanks,
>>> --Matt
>>> 
>>> 
>>> On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley  wrote:
>>> 
 The RC has been uploaded to Nexus / maven.
 --Matt
 
 
 On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley  wrote:
 
> Hi,
> Please review this release candidate for Hadoop-1.1.0.  As suggested
> before, I am posting it for review, and will start a vote on it if it
> passes the next week with no serious issues being found.
> 
> Tarballs, rpms, and debs are available at
>http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
> The release notes are also at the top level of that directory.
> 
> Nexus seems a bit wedged currently.  I will try again to push to Nexus
> tomorrow morning.
> 
> Thanks,
> --Matt
> 
 
 
>> 

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

Re: [REVIEW] Hadoop-1.1.0-RC4

[Arun C Murthy]

On Sep 25, 2012, at 8:40 AM, Arun C Murthy wrote:

> I'd also recommend MR-4558, MR-4328 & H-8748+friends. Thoughts?

Also, MR-3837. 

Initially, I didn't recommend them since I thought 1.1 was too close, but since 
these (MR-3837 & MR-4328 etc.) are well-baked at this point (deployed at 
multiple places) I'd like to include them in.

thanks,
Arun

> 
> thanks,
> Arun
> 
> On Sep 24, 2012, at 9:58 PM, Matt Foley wrote:
> 
>> Okay.
>> 
>> On Mon, Sep 24, 2012 at 7:05 PM, Uma Maheswara Rao G
>> wrote:
>> 
>>> Hi Matt,
>>> 
>>> Good to include HDFS-3701 also. I set the fixed versions as 1.1.0 as well.
>>> 
>>> Regards,
>>> Uma
>>> 
>>> On Tue, Sep 25, 2012 at 7:29 AM, Matt Foley  wrote:
 Hi all,
 Since the Review RC went out, I've been asked to include the following
 fixes:
 
 Three significant bugs:
 HDFS-3461 hftp uses http to the Namenode's https port, which doesn't work
 HDFS-3846 NN deadlock
 HDFS-3596 decrease the incidence of corrupted logs after disk full
 conditions
 and a related enhancement:
 HDFS-3521 deal with edit log corruption if it occurs
 
 Four unit test fixes (obviously not critical, but good to have):
 HDFS-3698
 HDFS-3966
 MAPREDUCE-4673
 MAPREDUCE-4675
 
 and four performance enhancements:
 HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
 HDFS-2751 Datanode drops OS cache behind reads even for short reads
 MAPREDUCE-1906 default minimum heartbeat
 MAPREDUCE-3289 use fadvise
 
 I will spin a new RC with these fixes, and hope to offer it for vote by
 Wednesday.
 Thanks,
 --Matt
 
 
 On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley  wrote:
 
> The RC has been uploaded to Nexus / maven.
> --Matt
> 
> 
> On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley  wrote:
> 
>> Hi,
>> Please review this release candidate for Hadoop-1.1.0.  As suggested
>> before, I am posting it for review, and will start a vote on it if it
>> passes the next week with no serious issues being found.
>> 
>> Tarballs, rpms, and debs are available at
>>http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
>> The release notes are also at the top level of that directory.
>> 
>> Nexus seems a bit wedged currently.  I will try again to push to Nexus
>> tomorrow morning.
>> 
>> Thanks,
>> --Matt
>> 
> 
> 
>>> 
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
> 

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

[jira] [Created] (HADOOP-8840) Fix the test-patch colorizer to cover all sorts of +1 lines.

[Harsh J (JIRA)]

Harsh J created HADOOP-8840:
---

 Summary: Fix the test-patch colorizer to cover all sorts of +1 
lines.
 Key: HADOOP-8840
 URL: https://issues.apache.org/jira/browse/HADOOP-8840
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Reporter: Harsh J
Assignee: Harsh J


As noticed by Jason on HADOOP-8838, I missed some of the entries needed to be 
colorized.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [REVIEW] Hadoop-1.1.0-RC4

[Matt Foley]

Hi Arun,
can you get them committed to branch-1 and branch-1.1 today?
--Matt

On Tue, Sep 25, 2012 at 8:44 AM, Arun C Murthy  wrote:

>
> On Sep 25, 2012, at 8:40 AM, Arun C Murthy wrote:
>
> > I'd also recommend MR-4558, MR-4328 & H-8748+friends. Thoughts?
>
> Also, MR-3837.
>
> Initially, I didn't recommend them since I thought 1.1 was too close, but
> since these (MR-3837 & MR-4328 etc.) are well-baked at this point (deployed
> at multiple places) I'd like to include them in.
>
> thanks,
> Arun
>
> >
> > thanks,
> > Arun
> >
> > On Sep 24, 2012, at 9:58 PM, Matt Foley wrote:
> >
> >> Okay.
> >>
> >> On Mon, Sep 24, 2012 at 7:05 PM, Uma Maheswara Rao G
> >> wrote:
> >>
> >>> Hi Matt,
> >>>
> >>> Good to include HDFS-3701 also. I set the fixed versions as 1.1.0 as
> well.
> >>>
> >>> Regards,
> >>> Uma
> >>>
> >>> On Tue, Sep 25, 2012 at 7:29 AM, Matt Foley  wrote:
>  Hi all,
>  Since the Review RC went out, I've been asked to include the following
>  fixes:
> 
>  Three significant bugs:
>  HDFS-3461 hftp uses http to the Namenode's https port, which doesn't
> work
>  HDFS-3846 NN deadlock
>  HDFS-3596 decrease the incidence of corrupted logs after disk full
>  conditions
>  and a related enhancement:
>  HDFS-3521 deal with edit log corruption if it occurs
> 
>  Four unit test fixes (obviously not critical, but good to have):
>  HDFS-3698
>  HDFS-3966
>  MAPREDUCE-4673
>  MAPREDUCE-4675
> 
>  and four performance enhancements:
>  HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
>  HDFS-2751 Datanode drops OS cache behind reads even for short reads
>  MAPREDUCE-1906 default minimum heartbeat
>  MAPREDUCE-3289 use fadvise
> 
>  I will spin a new RC with these fixes, and hope to offer it for vote
> by
>  Wednesday.
>  Thanks,
>  --Matt
> 
> 
>  On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley  wrote:
> 
> > The RC has been uploaded to Nexus / maven.
> > --Matt
> >
> >
> > On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley 
> wrote:
> >
> >> Hi,
> >> Please review this release candidate for Hadoop-1.1.0.  As suggested
> >> before, I am posting it for review, and will start a vote on it if
> it
> >> passes the next week with no serious issues being found.
> >>
> >> Tarballs, rpms, and debs are available at
> >>http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
> >> The release notes are also at the top level of that directory.
> >>
> >> Nexus seems a bit wedged currently.  I will try again to push to
> Nexus
> >> tomorrow morning.
> >>
> >> Thanks,
> >> --Matt
> >>
> >
> >
> >>>
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>

[jira] [Created] (HADOOP-8841) In trunk for command rm, the flags -[rR] and -f are not documented

[Jing Zhao (JIRA)]

Jing Zhao created HADOOP-8841:
-

 Summary: In trunk for command rm, the flags -[rR] and -f are not 
documented
 Key: HADOOP-8841
 URL: https://issues.apache.org/jira/browse/HADOOP-8841
 Project: Hadoop Common
  Issue Type: Improvement
  Components: documentation
Affects Versions: 3.0.0
Reporter: Jing Zhao
Assignee: Jing Zhao
Priority: Minor


We need to add description about the -[rR] and -f flags in the document for 
trunk.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [REVIEW] Hadoop-1.1.0-RC4

[Arun C Murthy]

Yes, on it. Thanks Matt.

On Sep 25, 2012, at 9:46 AM, Matt Foley wrote:

> Hi Arun,
> can you get them committed to branch-1 and branch-1.1 today?
> --Matt
> 
> On Tue, Sep 25, 2012 at 8:44 AM, Arun C Murthy  wrote:
> 
>> 
>> On Sep 25, 2012, at 8:40 AM, Arun C Murthy wrote:
>> 
>>> I'd also recommend MR-4558, MR-4328 & H-8748+friends. Thoughts?
>> 
>> Also, MR-3837.
>> 
>> Initially, I didn't recommend them since I thought 1.1 was too close, but
>> since these (MR-3837 & MR-4328 etc.) are well-baked at this point (deployed
>> at multiple places) I'd like to include them in.
>> 
>> thanks,
>> Arun
>> 
>>> 
>>> thanks,
>>> Arun
>>> 
>>> On Sep 24, 2012, at 9:58 PM, Matt Foley wrote:
>>> 
 Okay.
 
 On Mon, Sep 24, 2012 at 7:05 PM, Uma Maheswara Rao G
 wrote:
 
> Hi Matt,
> 
> Good to include HDFS-3701 also. I set the fixed versions as 1.1.0 as
>> well.
> 
> Regards,
> Uma
> 
> On Tue, Sep 25, 2012 at 7:29 AM, Matt Foley  wrote:
>> Hi all,
>> Since the Review RC went out, I've been asked to include the following
>> fixes:
>> 
>> Three significant bugs:
>> HDFS-3461 hftp uses http to the Namenode's https port, which doesn't
>> work
>> HDFS-3846 NN deadlock
>> HDFS-3596 decrease the incidence of corrupted logs after disk full
>> conditions
>> and a related enhancement:
>> HDFS-3521 deal with edit log corruption if it occurs
>> 
>> Four unit test fixes (obviously not critical, but good to have):
>> HDFS-3698
>> HDFS-3966
>> MAPREDUCE-4673
>> MAPREDUCE-4675
>> 
>> and four performance enhancements:
>> HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
>> HDFS-2751 Datanode drops OS cache behind reads even for short reads
>> MAPREDUCE-1906 default minimum heartbeat
>> MAPREDUCE-3289 use fadvise
>> 
>> I will spin a new RC with these fixes, and hope to offer it for vote
>> by
>> Wednesday.
>> Thanks,
>> --Matt
>> 
>> 
>> On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley  wrote:
>> 
>>> The RC has been uploaded to Nexus / maven.
>>> --Matt
>>> 
>>> 
>>> On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley 
>> wrote:
>>> 
 Hi,
 Please review this release candidate for Hadoop-1.1.0.  As suggested
 before, I am posting it for review, and will start a vote on it if
>> it
 passes the next week with no serious issues being found.
 
 Tarballs, rpms, and debs are available at
   http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
 The release notes are also at the top level of that directory.
 
 Nexus seems a bit wedged currently.  I will try again to push to
>> Nexus
 tomorrow morning.
 
 Thanks,
 --Matt
 
>>> 
>>> 
> 
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>> 
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>> 
>> 
>> 

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

[jira] [Created] (HADOOP-8842) local file system behavior of mv into an empty directory is inconsistent with HDFS

[Julien Le Dem (JIRA)]

Julien Le Dem created HADOOP-8842:
-

 Summary: local file system behavior of mv into an empty directory 
is inconsistent with HDFS
 Key: HADOOP-8842
 URL: https://issues.apache.org/jira/browse/HADOOP-8842
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 0.20.2
Reporter: Julien Le Dem


moving into an empty directory replaces the directory instead.
See output of attached script to reproduce :

>>> HDFS: normal behavior
content of a: 1 part
Found 1 items
-rw-r--r--   3 julien g  0 2012-09-25 17:16 
/user/julien/local_fs_bug/a/part-
content of b/c: empty
mv a b/c
resulting content of b/c
Found 1 items
drwxr-xr-x   - julien g  0 2012-09-25 17:16 
/user/julien/local_fs_bug/b/c/a
a is moved inside of c

>>> local fs: bug
content of a: 1 part
12/09/25 17:16:34 WARN fs.FileSystem: "local" is a deprecated filesystem name. 
Use "file:///" instead.
Found 1 items
-rw-r--r--   1 julien g  0 2012-09-25 17:16 
/home/julien/local_fs_bug/a/part-
content of b/c: empty
12/09/25 17:16:34 WARN fs.FileSystem: "local" is a deprecated filesystem name. 
Use "file:///" instead.
mv a b/c
12/09/25 17:16:35 WARN fs.FileSystem: "local" is a deprecated filesystem name. 
Use "file:///" instead.
resulting content of b/c
12/09/25 17:16:35 WARN fs.FileSystem: "local" is a deprecated filesystem name. 
Use "file:///" instead.
Found 1 items
-rw-r--r--   1 julien g  0 2012-09-25 17:16 
/home/julien/local_fs_bug/b/c/part-
bug: a replaces c

>>> but it works if the destination is not empty
content of a2: 1 part
12/09/25 17:16:36 WARN fs.FileSystem: "local" is a deprecated filesystem name. 
Use "file:///" instead.
Found 1 items
-rw-r--r--   1 julien g  0 2012-09-25 17:16 
/home/julien/local_fs_bug/a2/part-
content of b2/c2: 1 dummy file
12/09/25 17:16:37 WARN fs.FileSystem: "local" is a deprecated filesystem name. 
Use "file:///" instead.
Found 1 items
-rw-r--r--   1 julien g  0 2012-09-25 17:16 
/home/julien/local_fs_bug/b2/c2/dummy
mv a2 b2/c2
12/09/25 17:16:37 WARN fs.FileSystem: "local" is a deprecated filesystem name. 
Use "file:///" instead.
resulting content of b/c
12/09/25 17:16:38 WARN fs.FileSystem: "local" is a deprecated filesystem name. 
Use "file:///" instead.
Found 2 items
drwxr-xr-x   - julien g   4096 2012-09-25 17:16 
/home/julien/local_fs_bug/b2/c2/a2
-rw-r--r--   1 julien g  0 2012-09-25 17:16 
/home/julien/local_fs_bug/b2/c2/dummy
a2 is moved inside of c2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8844) Add a plaintext fs -text test-case

[Harsh J (JIRA)]

Harsh J created HADOOP-8844:
---

 Summary: Add a plaintext fs -text test-case
 Key: HADOOP-8844
 URL: https://issues.apache.org/jira/browse/HADOOP-8844
 Project: Hadoop Common
  Issue Type: Test
  Components: fs
Affects Versions: 2.0.0-alpha
Reporter: Harsh J


The TestDFSShell's textTest(…) currently tests all sorts of binary and 
compressed files, but doesn't test plaintext files. We should add one test for 
plaintext as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8845) When looking for parent paths info, globStatus must filter out non-directory elements to avoid an AccessControlException

[Harsh J (JIRA)]

Harsh J created HADOOP-8845:
---

 Summary: When looking for parent paths info, globStatus must 
filter out non-directory elements to avoid an AccessControlException
 Key: HADOOP-8845
 URL: https://issues.apache.org/jira/browse/HADOOP-8845
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: Harsh J


A brief description from my colleague Stephen Fritz who helped discover it:

{quote}
[root@node1 ~]# su - hdfs
-bash-4.1$ echo "My Test String">testfile <-- just a text file, for testing 
below
-bash-4.1$ hadoop dfs -mkdir /tmp/testdir <-- create a directory
-bash-4.1$ hadoop dfs -mkdir /tmp/testdir/1 <-- create a subdirectory
-bash-4.1$ hadoop dfs -put testfile /tmp/testdir/1/testfile <-- put the test 
file in the subdirectory
-bash-4.1$ hadoop dfs -put testfile /tmp/testdir/testfile <-- put the test file 
in the directory
-bash-4.1$ hadoop dfs -lsr /tmp/testdir
drwxr-xr-x   - hdfs hadoop  0 2012-09-25 06:52 /tmp/testdir/1
-rw-r--r--   3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/1/testfile
-rw-r--r--   3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/testfile
All files are where we expect them...OK, let's try reading

-bash-4.1$ hadoop dfs -cat /tmp/testdir/testfile
My Test String <-- success!

-bash-4.1$ hadoop dfs -cat /tmp/testdir/1/testfile
My Test String <-- success!

-bash-4.1$ hadoop dfs -cat /tmp/testdir/*/testfile
My Test String <-- success!  
Note that we used an '*' in the cat command, and it correctly found the 
subdirectory '/tmp/testdir/1', and ignore the regular file 
'/tmp/testdir/testfile'

-bash-4.1$ exit
logout
[root@node1 ~]# su - testuser <-- lets try it as a different user:
[testuser@node1 ~]$ hadoop dfs -lsr /tmp/testdir
drwxr-xr-x   - hdfs hadoop  0 2012-09-25 06:52 /tmp/testdir/1
-rw-r--r--   3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/1/testfile
-rw-r--r--   3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/testfile
[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/testfile
My Test String <-- good

[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/1/testfile
My Test String <-- so far so good

[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/*/testfile
cat: org.apache.hadoop.security.AccessControlException: Permission denied: 
user=testuser, access=EXECUTE, 
inode="/tmp/testdir/testfile":hdfs:hadoop:-rw-r--r--
{code}

Essentially, we hit a ACE with access=EXECUTE on file /tmp/testdir/testfile 
cause we tried to access the /tmp/testdir/testfile/testfile as a path. This 
shouldn't happen, as the testfile is a file and not a path parent to be looked 
up upon.

Surprisingly the superuser avoids hitting into the error, as a result of 
bypassing permissions, but that can be looked up on another JIRA - if it is 
fine to let it be like that or not.

This JIRA targets a client-sided fix to not cause such /path/file/dir kinda 
lookups.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8846) Document backward incompatible changes introduced by HADOOP-8551

[Arpit Gupta (JIRA)]

Arpit Gupta created HADOOP-8846:
---

 Summary: Document backward incompatible changes introduced by 
HADOOP-8551
 Key: HADOOP-8846
 URL: https://issues.apache.org/jira/browse/HADOOP-8846
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 3.0.0, 2.0.2-alpha
Reporter: Arpit Gupta


The following incompatible changes were noticed between branch-1 and branch-2 
caused by HADOOP-8551

1. mkdir would create parent directories in branch-1 if they did not exist. In 
branch-2 users have to explicitly send mkdir -p

2. Create a multi level dir in branch 1 something like

mkdir /test/1

/test would get permissions 755 and /test/1 would get the permissions based on 
your umask settings

however if you run the command in branch-2
mkdir -p /test/1

both /test and /test/1 will get the permissions based on your umask. 

These are significant changes that we should document.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8847) Change untar to use Java API instead of spawning tar process

[Bikas Saha (JIRA)]

Bikas Saha created HADOOP-8847:
--

 Summary: Change untar to use Java API instead of spawning tar 
process
 Key: HADOOP-8847
 URL: https://issues.apache.org/jira/browse/HADOOP-8847
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Bikas Saha
Assignee: Bikas Saha


Currently FileUtil.unTar() spawns tar utility to do the work. Tar may not be 
present on all platforms by default eg. Windows. So changing this to use JAVA 
API's would help make it more cross-platform. FileUtil.unZip() uses the same 
approach.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Commits breaking compilation of MR 'classic' tests

[Arun C Murthy]

Committers,

 As most people are aware, the MapReduce 'classic' tests (in 
hadoop-mapreduce-project/src/test) still need to built using ant since they 
aren't mavenized yet.

 I've seen several commits (and 2 within the last hour i.e. MAPREDUCE-3681 and 
MAPREDUCE-3682) which lead me to believe developers/committers aren't checking 
for this.

 Henceforth, with all changes, before committing, please do run:
 $ mvn install
 $ cd hadoop-mapreduce-project
 $ ant veryclean all-jars -Dresolvers=internal

 These instructions were already in 
http://wiki.apache.org/hadoop/HowToReleasePostMavenization and I've just 
updated http://wiki.apache.org/hadoop/HowToContribute.

thanks,
Arun

Re: Commits breaking compilation of MR 'classic' tests

[Eli Collins]

How about adding this step to the MR PreCommit jenkins job so it's run
as part test-patch?

On Tue, Sep 25, 2012 at 7:48 PM, Arun C Murthy  wrote:
> Committers,
>
>  As most people are aware, the MapReduce 'classic' tests (in 
> hadoop-mapreduce-project/src/test) still need to built using ant since they 
> aren't mavenized yet.
>
>  I've seen several commits (and 2 within the last hour i.e. MAPREDUCE-3681 
> and MAPREDUCE-3682) which lead me to believe developers/committers aren't 
> checking for this.
>
>  Henceforth, with all changes, before committing, please do run:
>  $ mvn install
>  $ cd hadoop-mapreduce-project
>  $ ant veryclean all-jars -Dresolvers=internal
>
>  These instructions were already in 
> http://wiki.apache.org/hadoop/HowToReleasePostMavenization and I've just 
> updated http://wiki.apache.org/hadoop/HowToContribute.
>
> thanks,
> Arun
>