[jira] [Created] (HDFS-6502) a typo in distcp document

2014-06-07 Thread Yongjun Zhang (JIRA) 
Yongjun Zhang created HDFS-6502:
---

 Summary: a typo in distcp document
 Key: HDFS-6502
 URL: https://issues.apache.org/jira/browse/HDFS-6502
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.4.0
Reporter: Yongjun Zhang


In http://hadoop.apache.org/docs/r1.2.1/distcp2.html#UpdateAndOverwrite

The first statement of the "Update and Overwrite" section says:
{quote}
-update is used to copy files from source that don't exist at the target, or 
have different contents. -overwrite overwrites target-files even if they exist 
at the source, or have the same contents.
{quote}

The "Command Line Options" table says :
{quote}
  -overwrite: Overwrite destination
  -update: Overwrite if src size different from dst size
{quote}

Based on the implementation, they need to modified to:

The first statement of the "Update and Overwrite" section:
{code}
-update is used to copy files from source that don't exist at the target, or 
have different contents. -overwrite overwrites target-files if they exist at 
the target, and even if they have the same contents as the source-files.
{code}

The "Command Line Options" table:
{code}
  -overwrite: Overwrite destination
  -update: Overwrite if src and dst have different contents
{code}

Thanks.




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HDFS-6501) TestCrcCorruption#testCorruptionDuringWrt sometimes fails in trunk

2014-06-07 Thread Ted Yu (JIRA) 
Ted Yu created HDFS-6501:


 Summary: TestCrcCorruption#testCorruptionDuringWrt sometimes fails 
in trunk
 Key: HDFS-6501
 URL: https://issues.apache.org/jira/browse/HDFS-6501
 Project: Hadoop HDFS
  Issue Type: Test
Reporter: Ted Yu
Priority: Minor


>From https://builds.apache.org/job/Hadoop-Hdfs-trunk/1767/ :
{code}
REGRESSION:  org.apache.hadoop.hdfs.TestCrcCorruption.testCorruptionDuringWrt

Error Message:
test timed out after 5 milliseconds

Stack Trace:
java.lang.Exception: test timed out after 5 milliseconds
at java.lang.Object.wait(Native Method)
at 
org.apache.hadoop.hdfs.DFSOutputStream.waitForAckedSeqno(DFSOutputStream.java:2024)
at 
org.apache.hadoop.hdfs.DFSOutputStream.flushInternal(DFSOutputStream.java:2008)
at 
org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2107)
at 
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:70)
at 
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:98)
at 
org.apache.hadoop.hdfs.TestCrcCorruption.testCorruptionDuringWrt(TestCrcCorruption.java:133)
{code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Hadoop-Hdfs-trunk - Build # 1767 - Still Failing

2014-06-07 Thread Apache Jenkins Server 
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1767/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 15044 lines...]
[INFO] --- maven-source-plugin:2.1.2:test-jar-no-fork (hadoop-java-sources) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (dist-enforce) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ 
hadoop-hdfs-project ---
[INFO] Not executing Javadoc as the project is not a Java classpath-capable 
package
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (depcheck) @ hadoop-hdfs-project 
---
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.6:checkstyle (default-cli) @ 
hadoop-hdfs-project ---
[INFO] 
[INFO] --- findbugs-maven-plugin:2.3.2:findbugs (default-cli) @ 
hadoop-hdfs-project ---
[INFO] ** FindBugsMojo execute ***
[INFO] canGenerate is false
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop HDFS  FAILURE 
[2:19:26.527s]
[INFO] Apache Hadoop HttpFS .. SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal . SKIPPED
[INFO] Apache Hadoop HDFS-NFS  SKIPPED
[INFO] Apache Hadoop HDFS Project  SUCCESS [2.797s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 2:19:30.946s
[INFO] Finished at: Sat Jun 07 13:55:29 UTC 2014
[INFO] Final Memory: 37M/463M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on 
project hadoop-hdfs: There are test failures.
[ERROR] 
[ERROR] Please refer to 
/home/jenkins/jenkins-slave/workspace/Hadoop-Hdfs-trunk/trunk/hadoop-hdfs-project/hadoop-hdfs/target/surefire-reports
 for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Updating MAPREDUCE-5898
Updating HADOOP-10647
Updating HDFS-6487
Updating HDFS-6497
Updating YARN-2132
Updating YARN-2121
Updating HDFS-6297
Updating YARN-2122
Updating YARN-2128
Updating YARN-2117
Updating MAPREDUCE-5777
Updating YARN-2118
Sending e-mails to: hdfs-dev@hadoop.apache.org
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
1 tests failed.
REGRESSION:  org.apache.hadoop.hdfs.TestCrcCorruption.testCorruptionDuringWrt

Error Message:
test timed out after 5 milliseconds

Stack Trace:
java.lang.Exception: test timed out after 5 milliseconds
at java.lang.Object.wait(Native Method)
at 
org.apache.hadoop.hdfs.DFSOutputStream.waitForAckedSeqno(DFSOutputStream.java:2024)
at 
org.apache.hadoop.hdfs.DFSOutputStream.flushInternal(DFSOutputStream.java:2008)
at 
org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2107)
at 
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:70)
at 
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:98)
at 
org.apache.hadoop.hdfs.TestCrcCorruption.testCorruptionDuringWrt(TestCrcCorruption.java:133)

Build failed in Jenkins: Hadoop-Hdfs-trunk #1767

2014-06-07 Thread Apache Jenkins Server 
See 

Changes:

[kasha] YARN-2132. ZKRMStateStore.ZKAction#runWithRetries doesn't log the 
exception it encounters. (Vamsee Yarlagadda via kasha)

[cos] HDFS-6297. Add CLI testcases to reflect new features of dfs and dfsadmin 
(Contributed by Dasha Boudnik)

[kasha] YARN-2128. FairScheduler: Incorrect calculation of amResource usage. 
(Wei Yan via kasha)

[kasha] YARN-2122. In AllocationFileLoaderService, the reloadThread should be 
created in init() and started in start(). (Robert Kanter via kasha)

[cmccabe] HDFS-6497. Make TestAvailableSpaceVolumeChoosingPolicy deterministic 
(cmccabe)

[wang] HDFS-6487. TestStandbyCheckpoint#testSBNCheckpoints is racy. Contributed 
by Mit Desai.

[vinodkv] YARN-2121. Fixed NPE handling in Timeline Server's 
TimelineAuthenticator. Contributed by Zhijie Shen.

[zjshen] YARN-2117. Fixed the issue that secret file reader is potentially not 
closed in TimelineAuthenticationFilterInitializer. Contributed by Chen He.

[kasha] MAPREDUCE-5777. Support utf-8 text with Byte Order Marker. (Zhihai Xu 
via kasha)

[stevel] HADOOP-10647. String Format Exception in 
SwiftNativeFileSystemStore.java.

[zjshen] YARN-2118. Fixed the type mismatch in Map#containsKey check of 
TimelineWebServices#injectOwnerInfo. Contributed by Ted Yu.

[umamahesh] MAPREDUCE-5898. distcp to support preserving HDFS extended 
attributes(XAttrs). Contributed by Yi Liu.

--
[...truncated 14851 lines...]
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.406 sec - 
in org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
Running org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.975 sec - in 
org.apache.hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
Running org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.671 sec - in 
org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
Running org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.444 sec - in 
org.apache.hadoop.hdfs.server.namenode.TestNameNodeResourceChecker
Running org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl
Tests run: 61, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.729 sec - in 
org.apache.hadoop.hdfs.server.namenode.TestNameNodeAcl
Running org.apache.hadoop.hdfs.server.namenode.TestFSPermissionChecker
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.205 sec - in 
org.apache.hadoop.hdfs.server.namenode.TestFSPermissionChecker
Running org.apache.hadoop.hdfs.server.namenode.TestAddBlock
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.162 sec - in 
org.apache.hadoop.hdfs.server.namenode.TestAddBlock
Running org.apache.hadoop.hdfs.server.namenode.TestLeaseManager
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.609 sec - in 
org.apache.hadoop.hdfs.server.namenode.TestLeaseManager
Running org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
Tests run: 38, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 81.795 sec - 
in org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
Running org.apache.hadoop.hdfs.server.namenode.TestFSNamesystem
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.092 sec - in 
org.apache.hadoop.hdfs.server.namenode.TestFSNamesystem
Running org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionManager
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.155 sec - in 
org.apache.hadoop.hdfs.server.namenode.TestNNStorageRetentionManager
Running org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.607 sec - in 
org.apache.hadoop.hdfs.server.namenode.TestBlockUnderConstruction
Running org.apache.hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.828 sec - in 
org.apache.hadoop.hdfs.server.namenode.ha.TestFailureOfSharedDir
Running org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.016 sec - in 
org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
Running org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.152 sec - in 
org.apache.hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
Running org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 78.602 sec - 
in org.apache.hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
Running org.apache.hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.013 sec - in 
org.ap

[jira] [Created] (HDFS-6500) Snapshot shouldn't be removed silently after renaming to an existing snapshot

2014-06-07 Thread Junping Du (JIRA) 
Junping Du created HDFS-6500:


 Summary: Snapshot shouldn't be removed silently after renaming to 
an existing snapshot
 Key: HDFS-6500
 URL: https://issues.apache.org/jira/browse/HDFS-6500
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: snapshots
Affects Versions: 2.4.0
Reporter: Junping Du
Priority: Blocker


Assume you have two snapshots for /dir: s1, s2, when you do "hdfs dfs 
-renameSnapshot /dir s2 s1" which means you want to rename s2 to s1. It will 
cause s2 been dropped silently and s1 still keep the previous one. This is very 
confusing and easily cause wrong operation.
The right behavior should be providing a warning to remind user that the target 
name is already existed and do nothing. If user do want to overlap s1, he 
should add some option (i.e. "-f") or delete s1 first.



--
This message was sent by Atlassian JIRA
(v6.2#6252)