date:20120716


 [ 
https://issues.apache.org/jira/browse/HADOOP-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harsh J updated HADOOP-8531:


Hadoop Flags: Reviewed

 SequenceFile Writer can throw out a better error if a serializer or 
 deserializer isn't available
 

 Key: HADOOP-8531
 URL: https://issues.apache.org/jira/browse/HADOOP-8531
 Project: Hadoop Common
  Issue Type: Improvement
  Components: io
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: madhukara phatak
Priority: Trivial
  Labels: newbie
 Attachments: HADOOP-8531-1.patch, HADOOP-8531-2.patch, 
 HADOOP-8531-2.patch, HADOOP-8531.patch


 Currently, if the provided Key/Value class lacks a proper serializer in the 
 loaded config for the SequenceFile.Writer, we get an NPE as the null return 
 goes unchecked.
 Hence we get:
 {code}
 java.lang.NullPointerException
   at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1163)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1079)
   at 
 org.apache.hadoop.io.SequenceFile$RecordCompressWriter.init(SequenceFile.java:1331)
   at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:271)
 {code}
 We can provide a better message + exception in such cases. This is slightly 
 related to MAPREDUCE-2584.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8531) SequenceFile Writer can throw out a better error if a serializer or deserializer isn't available


 [ 
https://issues.apache.org/jira/browse/HADOOP-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harsh J updated HADOOP-8531:


  Component/s: io
 Target Version/s: 2.1.0-alpha
Affects Version/s: 2.0.0-alpha
  Summary: SequenceFile Writer can throw out a better error if a 
serializer or deserializer isn't available  (was: SequenceFile Writer can throw 
out a better error if a serializer isn't available)

 SequenceFile Writer can throw out a better error if a serializer or 
 deserializer isn't available
 

 Key: HADOOP-8531
 URL: https://issues.apache.org/jira/browse/HADOOP-8531
 Project: Hadoop Common
  Issue Type: Improvement
  Components: io
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: madhukara phatak
Priority: Trivial
  Labels: newbie
 Attachments: HADOOP-8531-1.patch, HADOOP-8531-2.patch, 
 HADOOP-8531-2.patch, HADOOP-8531.patch


 Currently, if the provided Key/Value class lacks a proper serializer in the 
 loaded config for the SequenceFile.Writer, we get an NPE as the null return 
 goes unchecked.
 Hence we get:
 {code}
 java.lang.NullPointerException
   at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1163)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1079)
   at 
 org.apache.hadoop.io.SequenceFile$RecordCompressWriter.init(SequenceFile.java:1331)
   at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:271)
 {code}
 We can provide a better message + exception in such cases. This is slightly 
 related to MAPREDUCE-2584.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8531) SequenceFile Writer can throw out a better error if a serializer or deserializer isn't available


[ 
https://issues.apache.org/jira/browse/HADOOP-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414931#comment-13414931
 ] 

Hudson commented on HADOOP-8531:


Integrated in Hadoop-Common-trunk-Commit #2477 (See 
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2477/])
HADOOP-8531. SequenceFile Writer can throw out a better error if a 
serializer or deserializer isn't available. Contributed by Madhukara Phatak. 
(harsh) (Revision 1361933)

 Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361933
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java


 SequenceFile Writer can throw out a better error if a serializer or 
 deserializer isn't available
 

 Key: HADOOP-8531
 URL: https://issues.apache.org/jira/browse/HADOOP-8531
 Project: Hadoop Common
  Issue Type: Improvement
  Components: io
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: madhukara phatak
Priority: Trivial
  Labels: newbie
 Attachments: HADOOP-8531-1.patch, HADOOP-8531-2.patch, 
 HADOOP-8531-2.patch, HADOOP-8531.patch


 Currently, if the provided Key/Value class lacks a proper serializer in the 
 loaded config for the SequenceFile.Writer, we get an NPE as the null return 
 goes unchecked.
 Hence we get:
 {code}
 java.lang.NullPointerException
   at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1163)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1079)
   at 
 org.apache.hadoop.io.SequenceFile$RecordCompressWriter.init(SequenceFile.java:1331)
   at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:271)
 {code}
 We can provide a better message + exception in such cases. This is slightly 
 related to MAPREDUCE-2584.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8531) SequenceFile Writer can throw out a better error if a serializer or deserializer isn't available


[ 
https://issues.apache.org/jira/browse/HADOOP-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414932#comment-13414932
 ] 

Hudson commented on HADOOP-8531:


Integrated in Hadoop-Hdfs-trunk-Commit #2542 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2542/])
HADOOP-8531. SequenceFile Writer can throw out a better error if a 
serializer or deserializer isn't available. Contributed by Madhukara Phatak. 
(harsh) (Revision 1361933)

 Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361933
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java


 SequenceFile Writer can throw out a better error if a serializer or 
 deserializer isn't available
 

 Key: HADOOP-8531
 URL: https://issues.apache.org/jira/browse/HADOOP-8531
 Project: Hadoop Common
  Issue Type: Improvement
  Components: io
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: madhukara phatak
Priority: Trivial
  Labels: newbie
 Attachments: HADOOP-8531-1.patch, HADOOP-8531-2.patch, 
 HADOOP-8531-2.patch, HADOOP-8531.patch


 Currently, if the provided Key/Value class lacks a proper serializer in the 
 loaded config for the SequenceFile.Writer, we get an NPE as the null return 
 goes unchecked.
 Hence we get:
 {code}
 java.lang.NullPointerException
   at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1163)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1079)
   at 
 org.apache.hadoop.io.SequenceFile$RecordCompressWriter.init(SequenceFile.java:1331)
   at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:271)
 {code}
 We can provide a better message + exception in such cases. This is slightly 
 related to MAPREDUCE-2584.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8531) SequenceFile Writer can throw out a better error if a serializer or deserializer isn't available


[ 
https://issues.apache.org/jira/browse/HADOOP-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414934#comment-13414934
 ] 

Hudson commented on HADOOP-8531:


Integrated in Hadoop-Mapreduce-trunk-Commit #2497 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2497/])
HADOOP-8531. SequenceFile Writer can throw out a better error if a 
serializer or deserializer isn't available. Contributed by Madhukara Phatak. 
(harsh) (Revision 1361933)

 Result = FAILURE
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361933
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java


 SequenceFile Writer can throw out a better error if a serializer or 
 deserializer isn't available
 

 Key: HADOOP-8531
 URL: https://issues.apache.org/jira/browse/HADOOP-8531
 Project: Hadoop Common
  Issue Type: Improvement
  Components: io
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: madhukara phatak
Priority: Trivial
  Labels: newbie
 Attachments: HADOOP-8531-1.patch, HADOOP-8531-2.patch, 
 HADOOP-8531-2.patch, HADOOP-8531.patch


 Currently, if the provided Key/Value class lacks a proper serializer in the 
 loaded config for the SequenceFile.Writer, we get an NPE as the null return 
 goes unchecked.
 Hence we get:
 {code}
 java.lang.NullPointerException
   at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1163)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1079)
   at 
 org.apache.hadoop.io.SequenceFile$RecordCompressWriter.init(SequenceFile.java:1331)
   at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:271)
 {code}
 We can provide a better message + exception in such cases. This is slightly 
 related to MAPREDUCE-2584.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8531) SequenceFile Writer can throw out a better error if a serializer or deserializer isn't available


 [ 
https://issues.apache.org/jira/browse/HADOOP-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harsh J updated HADOOP-8531:


   Resolution: Fixed
Fix Version/s: 2.1.0-alpha
   Status: Resolved  (was: Patch Available)

Committed to branch-2 and trunk. Thanks for the patch Madhukara! :)

 SequenceFile Writer can throw out a better error if a serializer or 
 deserializer isn't available
 

 Key: HADOOP-8531
 URL: https://issues.apache.org/jira/browse/HADOOP-8531
 Project: Hadoop Common
  Issue Type: Improvement
  Components: io
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: madhukara phatak
Priority: Trivial
  Labels: newbie
 Fix For: 2.1.0-alpha

 Attachments: HADOOP-8531-1.patch, HADOOP-8531-2.patch, 
 HADOOP-8531-2.patch, HADOOP-8531.patch


 Currently, if the provided Key/Value class lacks a proper serializer in the 
 loaded config for the SequenceFile.Writer, we get an NPE as the null return 
 goes unchecked.
 Hence we get:
 {code}
 java.lang.NullPointerException
   at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1163)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1079)
   at 
 org.apache.hadoop.io.SequenceFile$RecordCompressWriter.init(SequenceFile.java:1331)
   at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:271)
 {code}
 We can provide a better message + exception in such cases. This is slightly 
 related to MAPREDUCE-2584.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8531) SequenceFile Writer can throw out a better error if a serializer or deserializer isn't available


 [ 
https://issues.apache.org/jira/browse/HADOOP-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harsh J updated HADOOP-8531:


Target Version/s:   (was: 2.1.0-alpha)

 SequenceFile Writer can throw out a better error if a serializer or 
 deserializer isn't available
 

 Key: HADOOP-8531
 URL: https://issues.apache.org/jira/browse/HADOOP-8531
 Project: Hadoop Common
  Issue Type: Improvement
  Components: io
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: madhukara phatak
Priority: Trivial
  Labels: newbie
 Fix For: 2.1.0-alpha

 Attachments: HADOOP-8531-1.patch, HADOOP-8531-2.patch, 
 HADOOP-8531-2.patch, HADOOP-8531.patch


 Currently, if the provided Key/Value class lacks a proper serializer in the 
 loaded config for the SequenceFile.Writer, we get an NPE as the null return 
 goes unchecked.
 Hence we get:
 {code}
 java.lang.NullPointerException
   at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1163)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1079)
   at 
 org.apache.hadoop.io.SequenceFile$RecordCompressWriter.init(SequenceFile.java:1331)
   at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:271)
 {code}
 We can provide a better message + exception in such cases. This is slightly 
 related to MAPREDUCE-2584.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-7818) DiskChecker#checkDir should fail if the directory is not executable


[ 
https://issues.apache.org/jira/browse/HADOOP-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415024#comment-13415024
 ] 

Hudson commented on HADOOP-7818:


Integrated in Hadoop-Hdfs-trunk #1105 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1105/])
HADOOP-7818. DiskChecker#checkDir should fail if the directory is not 
executable. Contributed by Madhukara Phatak. (harsh) (Revision 1361717)

 Result = FAILURE
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361717
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DiskChecker.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestDiskChecker.java


 DiskChecker#checkDir should fail if the directory is not executable
 ---

 Key: HADOOP-7818
 URL: https://issues.apache.org/jira/browse/HADOOP-7818
 Project: Hadoop Common
  Issue Type: Bug
  Components: util
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Assignee: madhukara phatak
Priority: Minor
 Fix For: 2.1.0-alpha

 Attachments: HADOOP-7818-1.patch, HADOOP-7818-2.patch, 
 HADOOP-7818-3.patch, HADOOP-7818-4.patch, HADOOP-7818.patch


 DiskChecker#checkDir fails if a directory can't be created, read, or written 
 but does not fail if the directory exists and is not executable. This causes 
 subsequent code to think the directory is OK but later fail due to an 
 inability to access the directory (eg see MAPREDUCE-2921). I propose checkDir 
 fails if the directory is not executable. Looking at the uses, this should be 
 fine, I think it was ignored because checkDir is often used to create 
 directories and it creates executable directories.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8362) Improve exception message when Configuration.set() is called with a null key or value


[ 
https://issues.apache.org/jira/browse/HADOOP-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415027#comment-13415027
 ] 

Hudson commented on HADOOP-8362:


Integrated in Hadoop-Hdfs-trunk #1105 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1105/])
HADOOP-8362. Improve exception message when Configuration.set() is called 
with a null key or value. Contributed by Madhukara Phatak and Suresh Srinivas 
(harsh) (Revision 1361712)

 Result = FAILURE
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361712
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java


 Improve exception message when Configuration.set() is called with a null key 
 or value
 -

 Key: HADOOP-8362
 URL: https://issues.apache.org/jira/browse/HADOOP-8362
 Project: Hadoop Common
  Issue Type: Improvement
  Components: conf
Affects Versions: 2.0.0-alpha
Reporter: Todd Lipcon
Assignee: madhukara phatak
Priority: Trivial
  Labels: newbie
 Fix For: 2.1.0-alpha

 Attachments: HADOOP-8362-1.patch, HADOOP-8362-2.patch, 
 HADOOP-8362-3.patch, HADOOP-8362-4.patch, HADOOP-8362-5.patch, 
 HADOOP-8362-6.patch, HADOOP-8362-7.patch, HADOOP-8362-8.patch, 
 HADOOP-8362.10.patch, HADOOP-8362.9.patch, HADOOP-8362.patch


 Currently, calling Configuration.set(...) with a null value results in a 
 NullPointerException within Properties.setProperty. We should check for null 
 key/value and throw a better exception.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-7818) DiskChecker#checkDir should fail if the directory is not executable


[ 
https://issues.apache.org/jira/browse/HADOOP-7818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415095#comment-13415095
 ] 

Hudson commented on HADOOP-7818:


Integrated in Hadoop-Mapreduce-trunk #1138 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1138/])
HADOOP-7818. DiskChecker#checkDir should fail if the directory is not 
executable. Contributed by Madhukara Phatak. (harsh) (Revision 1361717)

 Result = FAILURE
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361717
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DiskChecker.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestDiskChecker.java


 DiskChecker#checkDir should fail if the directory is not executable
 ---

 Key: HADOOP-7818
 URL: https://issues.apache.org/jira/browse/HADOOP-7818
 Project: Hadoop Common
  Issue Type: Bug
  Components: util
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Assignee: madhukara phatak
Priority: Minor
 Fix For: 2.1.0-alpha

 Attachments: HADOOP-7818-1.patch, HADOOP-7818-2.patch, 
 HADOOP-7818-3.patch, HADOOP-7818-4.patch, HADOOP-7818.patch


 DiskChecker#checkDir fails if a directory can't be created, read, or written 
 but does not fail if the directory exists and is not executable. This causes 
 subsequent code to think the directory is OK but later fail due to an 
 inability to access the directory (eg see MAPREDUCE-2921). I propose checkDir 
 fails if the directory is not executable. Looking at the uses, this should be 
 fine, I think it was ignored because checkDir is often used to create 
 directories and it creates executable directories.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8531) SequenceFile Writer can throw out a better error if a serializer or deserializer isn't available


[ 
https://issues.apache.org/jira/browse/HADOOP-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415096#comment-13415096
 ] 

Hudson commented on HADOOP-8531:


Integrated in Hadoop-Mapreduce-trunk #1138 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1138/])
HADOOP-8531. SequenceFile Writer can throw out a better error if a 
serializer or deserializer isn't available. Contributed by Madhukara Phatak. 
(harsh) (Revision 1361933)

 Result = FAILURE
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361933
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestSequenceFile.java


 SequenceFile Writer can throw out a better error if a serializer or 
 deserializer isn't available
 

 Key: HADOOP-8531
 URL: https://issues.apache.org/jira/browse/HADOOP-8531
 Project: Hadoop Common
  Issue Type: Improvement
  Components: io
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: madhukara phatak
Priority: Trivial
  Labels: newbie
 Fix For: 2.1.0-alpha

 Attachments: HADOOP-8531-1.patch, HADOOP-8531-2.patch, 
 HADOOP-8531-2.patch, HADOOP-8531.patch


 Currently, if the provided Key/Value class lacks a proper serializer in the 
 loaded config for the SequenceFile.Writer, we get an NPE as the null return 
 goes unchecked.
 Hence we get:
 {code}
 java.lang.NullPointerException
   at org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1163)
   at 
 org.apache.hadoop.io.SequenceFile$Writer.init(SequenceFile.java:1079)
   at 
 org.apache.hadoop.io.SequenceFile$RecordCompressWriter.init(SequenceFile.java:1331)
   at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:271)
 {code}
 We can provide a better message + exception in such cases. This is slightly 
 related to MAPREDUCE-2584.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8362) Improve exception message when Configuration.set() is called with a null key or value


[ 
https://issues.apache.org/jira/browse/HADOOP-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415098#comment-13415098
 ] 

Hudson commented on HADOOP-8362:


Integrated in Hadoop-Mapreduce-trunk #1138 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1138/])
HADOOP-8362. Improve exception message when Configuration.set() is called 
with a null key or value. Contributed by Madhukara Phatak and Suresh Srinivas 
(harsh) (Revision 1361712)

 Result = FAILURE
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1361712
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java


 Improve exception message when Configuration.set() is called with a null key 
 or value
 -

 Key: HADOOP-8362
 URL: https://issues.apache.org/jira/browse/HADOOP-8362
 Project: Hadoop Common
  Issue Type: Improvement
  Components: conf
Affects Versions: 2.0.0-alpha
Reporter: Todd Lipcon
Assignee: madhukara phatak
Priority: Trivial
  Labels: newbie
 Fix For: 2.1.0-alpha

 Attachments: HADOOP-8362-1.patch, HADOOP-8362-2.patch, 
 HADOOP-8362-3.patch, HADOOP-8362-4.patch, HADOOP-8362-5.patch, 
 HADOOP-8362-6.patch, HADOOP-8362-7.patch, HADOOP-8362-8.patch, 
 HADOOP-8362.10.patch, HADOOP-8362.9.patch, HADOOP-8362.patch


 Currently, calling Configuration.set(...) with a null value results in a 
 NullPointerException within Properties.setProperty. We should check for null 
 key/value and throw a better exception.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8594) Upgrade to findbugs 2

2012-07-16 Thread Robert Joseph Evans (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415125#comment-13415125
 ] 

Robert Joseph Evans commented on HADOOP-8594:
-

I agree with Eli on this.  Switching to a new findbugs is a great thing, but I 
don't want to break the pre-commit builds in the process.  If we do that new 
findbugs issues will get past it simply because 54 looks a lot like 53 and it 
is simpler to say I didn't break it then to look at all 54 of the errors to see 
what the new ones might be. 

 Upgrade to findbugs 2
 -

 Key: HADOOP-8594
 URL: https://issues.apache.org/jira/browse/HADOOP-8594
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
 Attachments: HADOOP-8594.patch, findbugs-2-html-reports.tar.gz, 
 findbugs.html


 Harsh recently ran findbugs 2 (instead of 1.3.9 which is what jenkins runs) 
 and it showed thousands of warnings (they've made a lot of progress in 
 findbugs releases). We should upgrade to findbugs 2 and fix these. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HADOOP-7823) port HADOOP-4012 to branch-1 (splitting support for bzip2)

2012-07-16 Thread Andrew Purtell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell reassigned HADOOP-7823:
--

Assignee: (was: Andrew Purtell)

 port HADOOP-4012 to branch-1 (splitting support for bzip2)
 --

 Key: HADOOP-7823
 URL: https://issues.apache.org/jira/browse/HADOOP-7823
 Project: Hadoop Common
  Issue Type: New Feature
Affects Versions: 0.20.205.0
Reporter: Tim Broberg
 Attachments: HADOOP-7823-branch-1-v2.patch, 
 HADOOP-7823-branch-1-v3.patch, HADOOP-7823-branch-1-v3.patch, 
 HADOOP-7823-branch-1-v4.patch, HADOOP-7823-branch-1.patch


 Please see HADOOP-4012 - Providing splitting support for bzip2 compressed 
 files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8552) Conflict: Same security.log.file for multiple users.

2012-07-16 Thread Alejandro Abdelnur (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415387#comment-13415387
 ] 

Alejandro Abdelnur commented on HADOOP-8552:


Devaraj, are you OK with this patch after Karthik's clarifications?

 Conflict: Same security.log.file for multiple users. 
 -

 Key: HADOOP-8552
 URL: https://issues.apache.org/jira/browse/HADOOP-8552
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf, security
Affects Versions: 1.0.3, 2.0.0-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
 Attachments: HADOOP-8552_branch1.patch, HADOOP-8552_branch2.patch


 In log4j.properties, hadoop.security.log.file is set to SecurityAuth.audit. 
 In the presence of multiple users, this can lead to a potential conflict.
 Adding username to the log file would avoid this scenario.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8552) Conflict: Same security.log.file for multiple users.

2012-07-16 Thread Devaraj Das (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415412#comment-13415412
 ] 

Devaraj Das commented on HADOOP-8552:
-

Yes.

 Conflict: Same security.log.file for multiple users. 
 -

 Key: HADOOP-8552
 URL: https://issues.apache.org/jira/browse/HADOOP-8552
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf, security
Affects Versions: 1.0.3, 2.0.0-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
 Attachments: HADOOP-8552_branch1.patch, HADOOP-8552_branch2.patch


 In log4j.properties, hadoop.security.log.file is set to SecurityAuth.audit. 
 In the presence of multiple users, this can lead to a potential conflict.
 Adding username to the log file would avoid this scenario.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8552) Conflict: Same security.log.file for multiple users.

2012-07-16 Thread Alejandro Abdelnur (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alejandro Abdelnur updated HADOOP-8552:
---

   Resolution: Fixed
Fix Version/s: 2.0.1-alpha
   1.1.0
 Hadoop Flags: Incompatible change,Reviewed
   Status: Resolved  (was: Patch Available)

Thanks Karthik. Committed to trunk, branch-1 and branch-2.

 Conflict: Same security.log.file for multiple users. 
 -

 Key: HADOOP-8552
 URL: https://issues.apache.org/jira/browse/HADOOP-8552
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf, security
Affects Versions: 1.0.3, 2.0.0-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
 Fix For: 1.1.0, 2.0.1-alpha

 Attachments: HADOOP-8552_branch1.patch, HADOOP-8552_branch2.patch


 In log4j.properties, hadoop.security.log.file is set to SecurityAuth.audit. 
 In the presence of multiple users, this can lead to a potential conflict.
 Adding username to the log file would avoid this scenario.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8552) Conflict: Same security.log.file for multiple users.


[ 
https://issues.apache.org/jira/browse/HADOOP-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415423#comment-13415423
 ] 

Hudson commented on HADOOP-8552:


Integrated in Hadoop-Hdfs-trunk-Commit #2545 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2545/])
HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl 
via tucu) (Revision 1362151)

 Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1362151
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties


 Conflict: Same security.log.file for multiple users. 
 -

 Key: HADOOP-8552
 URL: https://issues.apache.org/jira/browse/HADOOP-8552
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf, security
Affects Versions: 1.0.3, 2.0.0-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
 Fix For: 1.1.0, 2.0.1-alpha

 Attachments: HADOOP-8552_branch1.patch, HADOOP-8552_branch2.patch


 In log4j.properties, hadoop.security.log.file is set to SecurityAuth.audit. 
 In the presence of multiple users, this can lead to a potential conflict.
 Adding username to the log file would avoid this scenario.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8552) Conflict: Same security.log.file for multiple users.


[ 
https://issues.apache.org/jira/browse/HADOOP-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415426#comment-13415426
 ] 

Hudson commented on HADOOP-8552:


Integrated in Hadoop-Common-trunk-Commit #2480 (See 
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2480/])
HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl 
via tucu) (Revision 1362151)

 Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1362151
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties


 Conflict: Same security.log.file for multiple users. 
 -

 Key: HADOOP-8552
 URL: https://issues.apache.org/jira/browse/HADOOP-8552
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf, security
Affects Versions: 1.0.3, 2.0.0-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
 Fix For: 1.1.0, 2.0.1-alpha

 Attachments: HADOOP-8552_branch1.patch, HADOOP-8552_branch2.patch


 In log4j.properties, hadoop.security.log.file is set to SecurityAuth.audit. 
 In the presence of multiple users, this can lead to a potential conflict.
 Adding username to the log file would avoid this scenario.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8552) Conflict: Same security.log.file for multiple users.


[ 
https://issues.apache.org/jira/browse/HADOOP-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415439#comment-13415439
 ] 

Hudson commented on HADOOP-8552:


Integrated in Hadoop-Mapreduce-trunk-Commit #2500 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2500/])
HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl 
via tucu) (Revision 1362151)

 Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1362151
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties


 Conflict: Same security.log.file for multiple users. 
 -

 Key: HADOOP-8552
 URL: https://issues.apache.org/jira/browse/HADOOP-8552
 Project: Hadoop Common
  Issue Type: Bug
  Components: conf, security
Affects Versions: 1.0.3, 2.0.0-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
 Fix For: 1.1.0, 2.0.1-alpha

 Attachments: HADOOP-8552_branch1.patch, HADOOP-8552_branch2.patch


 In log4j.properties, hadoop.security.log.file is set to SecurityAuth.audit. 
 In the presence of multiple users, this can lead to a potential conflict.
 Adding username to the log file would avoid this scenario.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8589) ViewFs tests fail when tests dir is under Jenkins home dir


[ 
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415460#comment-13415460
 ] 

Andrey Klochkov commented on HADOOP-8589:
-

The problem with the current code is that those 2 links can point to nested 
directories. In case of tests being run on a Jenkins master in it's default 
configuration one of the links the code will try to create will point to /var 
and another to /var/lib, and the creation of the second link will fail with 
Path /var already exists as dir; cannot create link. I'm proposing to use 
just the first component of the path for both links, i.e. cut off everything 
after /var in my example. It's already being done for the test link, and my 
patch is adding the same logic for the home link.

 ViewFs tests fail when tests dir is under Jenkins home dir
 --

 Key: HADOOP-8589
 URL: https://issues.apache.org/jira/browse/HADOOP-8589
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs, test
Affects Versions: 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Attachments: HADOOP-8859.patch


 TestFSMainOperationsLocalFileSystem fails in case when the test root 
 directory is under the user's home directory, and the user's home dir is 
 deeper than 2 levels from /. This happens with the default 1-node 
 installation of Jenkins. 
 This is the failure log:
 {code}
 org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as 
 dir; cannot create link here
   at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
   at org.apache.hadoop.fs.viewfs.InodeTree.init(InodeTree.java:334)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem$1.init(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
   at 
 org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
   at 
 org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
 ...
 Standard Output
 2012-07-11 22:07:20,239 INFO  mortbay.log (Slf4jLog.java:info(67)) - Home dir 
 base /var/lib
 {code}
 The reason for the failure is that the code tries to mount links for both 
 /var and /var/lib, and it fails for the 2nd one as the /var is mounted 
 already.
 The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8453) Add unit tests for winutils

2012-07-16 Thread Bikas Saha (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415492#comment-13415492
 ] 

Bikas Saha commented on HADOOP-8453:


Sure. Why dont you open a JIRA for that. We need to make sure these tests are 
executed on Windows. One option would be to check for that at startup of the 
test and return success for other OS's.

 Add unit tests for winutils
 ---

 Key: HADOOP-8453
 URL: https://issues.apache.org/jira/browse/HADOOP-8453
 Project: Hadoop Common
  Issue Type: Task
  Components: test
Affects Versions: 1.1.0, 0.24.0
Reporter: Chuan Liu
Assignee: Chuan Liu
Priority: Minor
 Attachments: HADOOP-8453-branch-1-win-2.patch, 
 HADOOP-8453-branch-1-win.patch


 In [Hadoop-8235|https://issues.apache.org/jira/browse/HADOOP-8235], we 
 created a Windows console program, named ‘winutils’, to emulate some Linux 
 command line utilities used by Hadoop. However no tests are provided in the 
 original patch. As this code is quite complicated, and the complexity may 
 even grow up in the future. We think unit tests are necessary to ensure code 
 quality, as well as smooth future development.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8589) ViewFs tests fail when tests dir is under Jenkins home dir

2012-07-16 Thread Ahmed Radwan (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415525#comment-13415525
 ] 

Ahmed Radwan commented on HADOOP-8589:
--

bq. I'm proposing to use just the first component of the path for both links, 
i.e. cut off everything after /var in my example. It's already being done for 
the test link, and my patch is adding the same logic for the home link.

* Can you clarify that in the ticket description since it is misleading?
* Would it be better to check if (homeDirFirstComponent == 
testDirFirstComponent), and just add a single link, instead of adding the same 
link twice?

 ViewFs tests fail when tests dir is under Jenkins home dir
 --

 Key: HADOOP-8589
 URL: https://issues.apache.org/jira/browse/HADOOP-8589
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs, test
Affects Versions: 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Attachments: HADOOP-8859.patch


 TestFSMainOperationsLocalFileSystem fails in case when the test root 
 directory is under the user's home directory, and the user's home dir is 
 deeper than 2 levels from /. This happens with the default 1-node 
 installation of Jenkins. 
 This is the failure log:
 {code}
 org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as 
 dir; cannot create link here
   at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
   at org.apache.hadoop.fs.viewfs.InodeTree.init(InodeTree.java:334)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem$1.init(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
   at 
 org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
   at 
 org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
 ...
 Standard Output
 2012-07-11 22:07:20,239 INFO  mortbay.log (Slf4jLog.java:info(67)) - Home dir 
 base /var/lib
 {code}
 The reason for the failure is that the code tries to mount links for both 
 /var and /var/lib, and it fails for the 2nd one as the /var is mounted 
 already.
 The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8589) ViewFs tests fail when tests dir is under Jenkins home dir

2012-07-16 Thread Daryn Sharp (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415545#comment-13415545
 ] 

Daryn Sharp commented on HADOOP-8589:
-

I'd like to see viewfs tests only accessing directories mounted under the 
build.test.dir.  There's been a number of recurring bugs centered around the 
mounting of the user's home dir.

Are you aware the trash test tries to blow away your home directory to verify 
it won't delete an ancestor of the trash dir?  Shining illustration of why 
everything should be under build.test.dir.



 ViewFs tests fail when tests dir is under Jenkins home dir
 --

 Key: HADOOP-8589
 URL: https://issues.apache.org/jira/browse/HADOOP-8589
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs, test
Affects Versions: 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Attachments: HADOOP-8859.patch


 TestFSMainOperationsLocalFileSystem fails in case when the test root 
 directory is under the user's home directory, and the user's home dir is 
 deeper than 2 levels from /. This happens with the default 1-node 
 installation of Jenkins. 
 This is the failure log:
 {code}
 org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as 
 dir; cannot create link here
   at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
   at org.apache.hadoop.fs.viewfs.InodeTree.init(InodeTree.java:334)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem$1.init(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
   at 
 org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
   at 
 org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
 ...
 Standard Output
 2012-07-11 22:07:20,239 INFO  mortbay.log (Slf4jLog.java:info(67)) - Home dir 
 base /var/lib
 {code}
 The reason for the failure is that the code tries to mount links for both 
 /var and /var/lib, and it fails for the 2nd one as the /var is mounted 
 already.
 The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested


 [ 
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Klochkov updated HADOOP-8589:


Summary: ViewFs tests fail when tests and home dirs are nested  (was: 
ViewFs tests fail when tests dir is under Jenkins home dir)

 ViewFs tests fail when tests and home dirs are nested
 -

 Key: HADOOP-8589
 URL: https://issues.apache.org/jira/browse/HADOOP-8589
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs, test
Affects Versions: 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Attachments: HADOOP-8859.patch


 TestFSMainOperationsLocalFileSystem fails in case when the test root 
 directory is under the user's home directory, and the user's home dir is 
 deeper than 2 levels from /. This happens with the default 1-node 
 installation of Jenkins. 
 This is the failure log:
 {code}
 org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as 
 dir; cannot create link here
   at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
   at org.apache.hadoop.fs.viewfs.InodeTree.init(InodeTree.java:334)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem$1.init(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
   at 
 org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
   at 
 org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
 ...
 Standard Output
 2012-07-11 22:07:20,239 INFO  mortbay.log (Slf4jLog.java:info(67)) - Home dir 
 base /var/lib
 {code}
 The reason for the failure is that the code tries to mount links for both 
 /var and /var/lib, and it fails for the 2nd one as the /var is mounted 
 already.
 The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8599) Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file

Andrey Klochkov created HADOOP-8599:
---

 Summary: Non empty response from FileSystem.getFileBlockLocations 
when asking for data beyond the end of file 
 Key: HADOOP-8599
 URL: https://issues.apache.org/jira/browse/HADOOP-8599
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs
Affects Versions: 2.0.0-alpha, 0.23.1, 1.0.3
Reporter: Andrey Klochkov


When FileSystem.getFileBlockLocations(file,start,len) is called with start 
argument equal to the file size, the response is not empty. There is a test 
TestGetFileBlockLocations.testGetFileBlockLocations2 which uses randomly 
generated start and len arguments when calling 
FileSystem.getFileBlockLocations and the test fails randomly (when the 
generated start value equals to the file size).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8599) Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file


[ 
https://issues.apache.org/jira/browse/HADOOP-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415576#comment-13415576
 ] 

Andrey Klochkov commented on HADOOP-8599:
-

The build https://builds.apache.org/job/Hadoop-Common-0.23-Build/312/ failed 
due to this bug, this is the failure log:

{code}
org.apache.hadoop.fs.TestGetFileBlockLocations.testGetFileBlockLocations2

Failing for the past 1 build (Since #312 )
Took 0.29 sec.

Stacktrace

junit.framework.AssertionFailedError: null
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at 
org.apache.hadoop.fs.TestGetFileBlockLocations.oneTest(TestGetFileBlockLocations.java:90)
at 
org.apache.hadoop.fs.TestGetFileBlockLocations.__CLR3_0_2h1r7rc12wg(TestGetFileBlockLocations.java:136)
at 
org.apache.hadoop.fs.TestGetFileBlockLocations.testGetFileBlockLocations2(TestGetFileBlockLocations.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at 
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at 
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
at 
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
at 
org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
at 
org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:81)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
Standard Output

2012-07-15 09:13:27,734 INFO  util.NativeCodeLoader 
(NativeCodeLoader.java:clinit(50)) - Loaded the native-hadoop library
{code}

 Non empty response from FileSystem.getFileBlockLocations when asking for data 
 beyond the end of file 
 -

 Key: HADOOP-8599
 URL: https://issues.apache.org/jira/browse/HADOOP-8599
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs
Affects Versions: 1.0.3, 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov

 When FileSystem.getFileBlockLocations(file,start,len) is called with start 
 argument equal to the file size, the response is not empty. There is a test 
 TestGetFileBlockLocations.testGetFileBlockLocations2 which uses randomly 
 generated start and len arguments when calling 
 FileSystem.getFileBlockLocations and the test fails randomly (when the 
 generated start value equals to the file size).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8598) Server-side Trash

2012-07-16 Thread Aaron T. Myers (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415580#comment-13415580
]

Aaron T. Myers commented on HADOOP-8598:

I think a quick and easy option for implementing server-side trash would be
to just not require any configuration client-side. This could be done by adding
an isTrashEnabled field to FsServerDefaults and changing the
TrashPolicyDefault#isEnabled method to call Filesystem#getServerDefaults and
return the value of isTrashEnabled. This won't result in a ton of extra RPCs to
the NN since the HDFS implementation of getServerDefaults is cached in the
DFSClient and rate-limited to one call per hour. Though it obviously wouldn't
cover the issue of APIs besides the FsShell, I think this would cover a very
large portion of the user problems we currently see with trash.

Server-side Trash
-

Key: HADOOP-8598
URL: https://issues.apache.org/jira/browse/HADOOP-8598
Project: Hadoop Common
Issue Type: New Feature
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Assignee: Eli Collins
Priority: Critical

There are a number of problems with Trash that continue to result in
permanent data loss for users. The primary reasons trash is not used:
- Trash is configured client-side and not enabled by default.
- Trash is shell-only. FileSystem, WebHDFS, HttpFs, etc never use trash.
- If trash fails, for example, because we can't create the trash directory or
the move itself fails, trash is bypassed and the data is deleted.
Trash was designed as a feature to help end users via the shell, however in
my experience the primary use of trash is to help administrators implement
data retention policies (this was also the motivation for HADOOP-7460). One
could argue that (periodic read-only) snapshots are a better solution to this
problem, however snapshots are not slated for Hadoop 2.x and trash is
complimentary to snapshots (and backup) - eg you may create and delete data
within your snapshot or backup window - so it makes sense to revisit trash's
design. I think it's worth bringing trash's functionality in line with what
users need.
I propose we enable trash on a per-filesystem basis and implement it
server-side. Ie trash becomes an HDFS feature enabled by administrators.
Because the trash emptier lives in HDFS and users already have a
per-filesystem trash directory we're mostly there already. The design
preference from HADOOP-2514 was for trash to be implemented in user code
however (a) in light of these problems, (b) we have a lot more user-facing
APIs than the shell and (c) clients increasingly span file systems (via
federation and symlinks) this design choice makes less sense. This is why we
already use a per-filesystem trash/home directory instead of the user's
client-configured one - otherwise trash would not work because renames can't
span file systems.
In short, HDFS trash would work similarly to how it does today, the
difference is that client delete APIs would result in a rename into trash
(ala TrashPolicyDefault#moveToTrash) if trash is enabled. Like today it would
be renamed to the trash directory on the file system where the file being
removed resides. The primary difference is that enablement and policy are
configured server-side by adminstrators and is used regardless of the API
used to access the filesytem. The one execption to this is that I think we
should continue to support the explict skipTrash shell option. The rationale
for skipTrash (HADOOP-6080) is that a move to trash may fail in cases where a
rm may not, if a user has a home directory quota and does a rmr /tonsOfData,
for example. Without a way to bypass this the user has no way (unless we
revisit quotas, permissions or trash paths) to remove a directory they have
permissions to remove without getting their quota adjusted by an admin. The
skip trash API can be implemented by adding an explicit FileSystem API that
bypasses trash and modifying the shell to use it when skipTrash is enabled.
Given that users must explicitly specify skipTrash the API is less error
prone. We could have the shell ask confirmation and annotate the API private
to FsShell to discourage programatic use. This is not ideal but can be done
compatibly (unlike redefining quotas, permissions or trash paths).
In terms of compatibility, while this proposal is technically an incompatible
change (client side configuration that disables trash and uses skipTrash with
a previous FsShell release will now both be ignored if server-side trash is
enabled, and non-HDFS file systems would need to make similar changes) I
think it's worth targeting for Hadoop 2.x given that the new semantics
preserve the current semantics.

[jira] [Commented] (HADOOP-8569) CMakeLists.txt: define _GNU_SOURCE and _LARGEFILE_SOURCE

2012-07-16 Thread Andy Isaacson (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-8569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415584#comment-13415584
 ] 

Andy Isaacson commented on HADOOP-8569:
---

The downside to defining {{_GNU_SOURCE}} everywhere -- in both code that is 
known to need it, and in code that is intended to be portable -- is that it 
makes it very easy to accidentally break compilation on non-GNU platforms by 
unintentionally changing code from portable to gnu-specific.  Suppose we 
have a project with {{foo-linux.c}} containing nonportable code, and 
{{generic.c}} containing POSIX portable code.  If I define {{_GNU_SOURCE}} in 
{{CFLAGS}} then unintentionally adding a call to {{sync_file_range}} to 
{{generic.c}} will silently work on Linux, and won't break the build until you 
try building on Darwin or Solaris or whatever.

If instead the project puts {{#define _GNU_SOURCE}} at the top of files 
intended to be platform-specific, then such portability breakage will be 
noticed immediately.

The argument doesn't extend to LFS support -- it's entirely reasonable to use 
64-bit-{{off_t}} everywhere including Linux-32.

{code}
+# note: can't enable -D_FILE_OFFSET_BITS=64: see MAPREDUCE-4258
+set(CMAKE_C_FLAGS ${CMAKE_C_FLAGS} -D_REENTRANT -D_GNU_SOURCE)
+set(CMAKE_C_FLAGS ${CMAKE_C_FLAGS} -D_LARGEFILE_SOURCE)
{code}
If we don't have FILE_OFFSET_BITS=64, then shouldn't we also leave out 
LARGEFILE_SOURCE?  (This is a serious question, I don't know how those two 
defines interact, I just vaguely remember that there's a complicated rule about 
how they should be used.)

In summary -- I'd slightly prefer to limit the GNU_SOURCE to a {{#define}} in 
the files that we intend to be Linux-specific.  The rest of this patch is good 
though, cleaning up the LFS defines and adding that comment about 
MAPREDUCE-4258.

 CMakeLists.txt: define _GNU_SOURCE and _LARGEFILE_SOURCE
 

 Key: HADOOP-8569
 URL: https://issues.apache.org/jira/browse/HADOOP-8569
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor
 Attachments: HADOOP-8569.001.patch


 In the native code, we should define _GNU_SOURCE and _LARGEFILE_SOURCE so 
 that all of the functions on Linux are available.
 _LARGEFILE enables fseeko and ftello; _GNU_SOURCE enables a variety of 
 Linux-specific functions from glibc, including sync_file_range.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested

2012-07-16 Thread Ahmed Radwan (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415587#comment-13415587
 ] 

Ahmed Radwan commented on HADOOP-8589:
--

I agree Daryn. Yes, TestViewFsTrash tries to delete the user's home directory!! 
Which raise the question of what if this testcase fails and it successfully 
deletes the user's home directory. I think this specific issue need to be 
prioritized and addressed in a separate ticket. 

 ViewFs tests fail when tests and home dirs are nested
 -

 Key: HADOOP-8589
 URL: https://issues.apache.org/jira/browse/HADOOP-8589
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs, test
Affects Versions: 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Attachments: HADOOP-8859.patch


 TestFSMainOperationsLocalFileSystem fails in case when the test root 
 directory is under the user's home directory, and the user's home dir is 
 deeper than 2 levels from /. This happens with the default 1-node 
 installation of Jenkins. 
 This is the failure log:
 {code}
 org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as 
 dir; cannot create link here
   at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
   at org.apache.hadoop.fs.viewfs.InodeTree.init(InodeTree.java:334)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem$1.init(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
   at 
 org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
   at 
 org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
 ...
 Standard Output
 2012-07-11 22:07:20,239 INFO  mortbay.log (Slf4jLog.java:info(67)) - Home dir 
 base /var/lib
 {code}
 The reason for the failure is that the code tries to mount links for both 
 /var and /var/lib, and it fails for the 2nd one as the /var is mounted 
 already.
 The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8598) Server-side Trash

2012-07-16 Thread Daryn Sharp (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415596#comment-13415596
]

Daryn Sharp commented on HADOOP-8598:
-

I haven't digested this jira, but be sure to keep viewfs in mind. Ie.
{{FileSystem#getServerDefaults(Path)}} would be required to get the info for
the right mount point.

Server-side Trash
-

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see:

[jira] [Commented] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested

2012-07-16 Thread Daryn Sharp (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415601#comment-13415601
 ] 

Daryn Sharp commented on HADOOP-8589:
-

What I'm trying to say is if the viewfs tests are confined to build.test.dir, 
then the mount path problems _and_ the trash problem are both solved.  They are 
only orthogonal if you believe that viewfs should mount your home dir, and 
trust it to not munge the contents of you home directory, and/or your ~/.Trash 
if you are on OS X.  This is at least the third jira to fix mounting home 
dirs.  It always fixes for some  breaks for others.

Are there drawbacks to confining the tests to mounts within build.test.dir?

 ViewFs tests fail when tests and home dirs are nested
 -

 Key: HADOOP-8589
 URL: https://issues.apache.org/jira/browse/HADOOP-8589
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs, test
Affects Versions: 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Attachments: HADOOP-8859.patch


 TestFSMainOperationsLocalFileSystem fails in case when the test root 
 directory is under the user's home directory, and the user's home dir is 
 deeper than 2 levels from /. This happens with the default 1-node 
 installation of Jenkins. 
 This is the failure log:
 {code}
 org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as 
 dir; cannot create link here
   at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
   at org.apache.hadoop.fs.viewfs.InodeTree.init(InodeTree.java:334)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem$1.init(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
   at 
 org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
   at 
 org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
   at 
 org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
   at 
 org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
 ...
 Standard Output
 2012-07-11 22:07:20,239 INFO  mortbay.log (Slf4jLog.java:info(67)) - Home dir 
 base /var/lib
 {code}
 The reason for the failure is that the code tries to mount links for both 
 /var and /var/lib, and it fails for the 2nd one as the /var is mounted 
 already.
 The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8569) CMakeLists.txt: define _GNU_SOURCE and _LARGEFILE_SOURCE

2012-07-16 Thread Colin Patrick McCabe (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-8569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415619#comment-13415619
]

Colin Patrick McCabe commented on HADOOP-8569:
--

_GNU_SOURCE was defined previously in most (if not all) of our native projects.
After I did the CMake conversion, the fact that it wasn't defined in the
CMakeLists.txt was a bug, not a feature. That's what I'm trying to fix here.

I realize that it's tempting to assume that code that you write without
_GNU_SOURCE defined will automatically be portable. However, this is *NOT
TRUE*. For example, even without _GNU_SOURCE defined, you still get the
non-POSIX definition of strerror_r out of glibc.

The only valid way to make sure your code is portable is to build and test it
on multiple platforms. Any other strategy is just a waste of time. Defining
GNU_SOURCE is similar to setting the correct DOCTYPE in your HTML file. It
tells the browser (or compiler in this case) to turn off quirks mode and give
you the real deal.

I don't think the CheckFunctionExists stuff in the CMakeLists.txt will work
consistently without _GNU_SOURCE defined.

There are better ways to improve our portability. For example, we should
probably have some OpenBSD jenkins build slaves. But let's not waste our time
messing with macros. It really adds nothing but inconvenience.

bq. If we don't have FILE_OFFSET_BITS=64, then shouldn't we also leave out
LARGEFILE_SOURCE?

_LARGEFILE_SOURCE exposes fseeko and ftello. _FILE_OFFSET_BITS changes the
default off_t type to be 64 bits. Basically _LARGEFILE_SOURCE is something you
need to define in addition to _FILE_OFFSET_BITS, but the two things do
different things.

CMakeLists.txt: define _GNU_SOURCE and _LARGEFILE_SOURCE

Key: HADOOP-8569
URL: https://issues.apache.org/jira/browse/HADOOP-8569
Project: Hadoop Common
Issue Type: Bug
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor
Attachments: HADOOP-8569.001.patch

In the native code, we should define _GNU_SOURCE and _LARGEFILE_SOURCE so
that all of the functions on Linux are available.
_LARGEFILE enables fseeko and ftello; _GNU_SOURCE enables a variety of
Linux-specific functions from glibc, including sync_file_range.

[jira] [Commented] (HADOOP-8598) Server-side Trash

2012-07-16 Thread Eli Collins (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-8598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415620#comment-13415620
]

Eli Collins commented on HADOOP-8598:
-

@ATM, like your idea, though it will only work with new clients (modified to
check the server config) and FsShell it seems like a reasonable approach for
branch-2.

Server-side Trash
-

[jira] [Created] (HADOOP-8600) libhdfs: fix some compiler warnings

2012-07-16 Thread Colin Patrick McCabe (JIRA)

Colin Patrick McCabe created HADOOP-8600:


 Summary: libhdfs: fix some compiler warnings
 Key: HADOOP-8600
 URL: https://issues.apache.org/jira/browse/HADOOP-8600
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 2.1.0-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor


Fix some compiler warnings.  Some of these are misuses of fprintf (forgetting 
the FILE* parameter), const mismatch warnings, format specifier warnings.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Work started] (HADOOP-8600) libhdfs: fix some compiler warnings

2012-07-16 Thread Colin Patrick McCabe (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-8600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HADOOP-8600 started by Colin Patrick McCabe.

 libhdfs: fix some compiler warnings
 ---

 Key: HADOOP-8600
 URL: https://issues.apache.org/jira/browse/HADOOP-8600
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 2.1.0-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor

 Fix some compiler warnings.  Some of these are misuses of fprintf (forgetting 
 the FILE* parameter), const mismatch warnings, format specifier warnings.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8599) Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file


 [ 
https://issues.apache.org/jira/browse/HADOOP-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Klochkov updated HADOOP-8599:


Attachment: HADOOP-8859-branch-0.23.patch

The patch fixes the condition of going beyond EOF in the 
FileSystem.getFileBlockLocations by changing 

{code}
if (file.getLen()  start) {
  return new BlockLocation[0];
{code}

to 

{code}
if (file.getLen() = start) {
  return new BlockLocation[0];
{code} 

An additional assert is added to the unit tests to verify the case. 

 Non empty response from FileSystem.getFileBlockLocations when asking for data 
 beyond the end of file 
 -

 Key: HADOOP-8599
 URL: https://issues.apache.org/jira/browse/HADOOP-8599
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs
Affects Versions: 1.0.3, 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
 Attachments: HADOOP-8859-branch-0.23.patch


 When FileSystem.getFileBlockLocations(file,start,len) is called with start 
 argument equal to the file size, the response is not empty. There is a test 
 TestGetFileBlockLocations.testGetFileBlockLocations2 which uses randomly 
 generated start and len arguments when calling 
 FileSystem.getFileBlockLocations and the test fails randomly (when the 
 generated start value equals to the file size).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8599) Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file


 [ 
https://issues.apache.org/jira/browse/HADOOP-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Klochkov updated HADOOP-8599:


Assignee: Andrey Klochkov
  Status: Patch Available  (was: Open)

 Non empty response from FileSystem.getFileBlockLocations when asking for data 
 beyond the end of file 
 -

 Key: HADOOP-8599
 URL: https://issues.apache.org/jira/browse/HADOOP-8599
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs
Affects Versions: 2.0.0-alpha, 0.23.1, 1.0.3
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Attachments: HADOOP-8859-branch-0.23.patch


 When FileSystem.getFileBlockLocations(file,start,len) is called with start 
 argument equal to the file size, the response is not empty. There is a test 
 TestGetFileBlockLocations.testGetFileBlockLocations2 which uses randomly 
 generated start and len arguments when calling 
 FileSystem.getFileBlockLocations and the test fails randomly (when the 
 generated start value equals to the file size).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8599) Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file

2012-07-16 Thread Hadoop QA (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415699#comment-13415699
]

Hadoop QA commented on HADOOP-8599:
---

+1 overall. Here are the results of testing the latest attachment

http://issues.apache.org/jira/secure/attachment/12536733/HADOOP-8859-branch-0.23.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test
files.

+1 javac. The applied patch does not increase the total number of javac
compiler warnings.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.

+1 release audit. The applied patch does not increase the total number of
release audit warnings.

+1 core tests. The patch passed unit tests in
hadoop-common-project/hadoop-common.

+1 contrib tests. The patch passed contrib unit tests.

Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1198//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1198//console

This message is automatically generated.

Non empty response from FileSystem.getFileBlockLocations when asking for data
beyond the end of file
-

Key: HADOOP-8599
URL: https://issues.apache.org/jira/browse/HADOOP-8599
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 1.0.3, 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
Attachments: HADOOP-8859-branch-0.23.patch

When FileSystem.getFileBlockLocations(file,start,len) is called with start
argument equal to the file size, the response is not empty. There is a test
TestGetFileBlockLocations.testGetFileBlockLocations2 which uses randomly
generated start and len arguments when calling
FileSystem.getFileBlockLocations and the test fails randomly (when the
generated start value equals to the file size).

[jira] [Commented] (HADOOP-8522) ResetableGzipOutputStream creates invalid gzip files when finish() and resetState() are used

2012-07-16 Thread Mike Percy (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415711#comment-13415711
]

Mike Percy commented on HADOOP-8522:

Apologies for the delay in following up on this.

Tom, do you expect GzipCodec to be able to read multi-member gzip files on
JDK1.6 without native libs installed?

ResetableGzipOutputStream creates invalid gzip files when finish() and
resetState() are used

Key: HADOOP-8522
URL: https://issues.apache.org/jira/browse/HADOOP-8522
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 1.0.3, 2.0.0-alpha
Reporter: Mike Percy
Assignee: Mike Percy
Attachments: HADOOP-8522-2a.patch

ResetableGzipOutputStream creates invalid gzip files when finish() and
resetState() are used. The issue is that finish() flushes the compressor
buffer and writes the gzip CRC32 + data length trailer. After that,
resetState() does not repeat the gzip header, but simply starts writing more
deflate-compressed data. The resultant files are not readable by the Linux
gunzip tool. ResetableGzipOutputStream should write valid multi-member gzip
files.
The gzip format is specified in [RFC
1952|https://tools.ietf.org/html/rfc1952].

[jira] [Resolved] (HADOOP-8423) MapFile.Reader.get() crashes jvm or throws EOFException on Snappy or LZO block-compressed data

2012-07-16 Thread Todd Lipcon (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-8423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon resolved HADOOP-8423.
-

   Resolution: Fixed
Fix Version/s: 1.2.0

Committed to branch-1 for 1.2. Thanks for backporting, Harsh.

 MapFile.Reader.get() crashes jvm or throws EOFException on Snappy or LZO 
 block-compressed data
 --

 Key: HADOOP-8423
 URL: https://issues.apache.org/jira/browse/HADOOP-8423
 Project: Hadoop Common
  Issue Type: Bug
  Components: io
Affects Versions: 0.20.2
 Environment: Linux 2.6.32.23-0.3-default #1 SMP 2010-10-07 14:57:45 
 +0200 x86_64 x86_64 x86_64 GNU/Linux
Reporter: Jason B
Assignee: Todd Lipcon
 Fix For: 1.2.0, 2.1.0-alpha

 Attachments: HADOOP-8423-branch-1.patch, HADOOP-8423-branch-1.patch, 
 MapFileCodecTest.java, hadoop-8423.txt


 I am using Cloudera distribution cdh3u1.
 When trying to check native codecs for better decompression
 performance such as Snappy or LZO, I ran into issues with random
 access using MapFile.Reader.get(key, value) method.
 First call of MapFile.Reader.get() works but a second call fails.
 Also  I am getting different exceptions depending on number of entries
 in a map file.
 With LzoCodec and 10 record file, jvm gets aborted.
 At the same time the DefaultCodec works fine for all cases, as well as
 record compression for the native codecs.
 I created a simple test program (attached) that creates map files
 locally with sizes of 10 and 100 records for three codecs: Default,
 Snappy, and LZO.
 (The test requires corresponding native library available)
 The summary of problems are given below:
 Map Size: 100
 Compression: RECORD
 ==
 DefaultCodec:  OK
 SnappyCodec: OK
 LzoCodec: OK
 Map Size: 10
 Compression: RECORD
 ==
 DefaultCodec:  OK
 SnappyCodec: OK
 LzoCodec: OK
 Map Size: 100
 Compression: BLOCK
 
 DefaultCodec:  OK
 SnappyCodec: java.io.EOFException  at
 org.apache.hadoop.io.compress.BlockDecompressorStream.getCompressedData(BlockDecompressorStream.java:114)
 LzoCodec: java.io.EOFException at
 org.apache.hadoop.io.compress.BlockDecompressorStream.getCompressedData(BlockDecompressorStream.java:114)
 Map Size: 10
 Compression: BLOCK
 ==
 DefaultCodec:  OK
 SnappyCodec: java.lang.NoClassDefFoundError: Ljava/lang/InternalError
 at 
 org.apache.hadoop.io.compress.snappy.SnappyDecompressor.decompressBytesDirect(Native
 Method)
 LzoCodec:
 #
 # A fatal error has been detected by the Java Runtime Environment:
 #
 #  SIGSEGV (0xb) at pc=0x2b068ffcbc00, pid=6385, tid=47304763508496
 #
 # JRE version: 6.0_21-b07
 # Java VM: Java HotSpot(TM) 64-Bit Server VM (17.0-b17 mixed mode linux-amd64 
 )
 # Problematic frame:
 # C  [liblzo2.so.2+0x13c00]  lzo1x_decompress+0x1a0
 #

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8593) add the missed @Override to methods in Metric/Metric2 package

2012-07-16 Thread Suresh Srinivas (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suresh Srinivas updated HADOOP-8593:


Affects Version/s: 3.0.0
   1.0.0

 add  the missed @Override to methods in Metric/Metric2 package
 --

 Key: HADOOP-8593
 URL: https://issues.apache.org/jira/browse/HADOOP-8593
 Project: Hadoop Common
  Issue Type: Improvement
  Components: metrics
Affects Versions: 1.0.0, 3.0.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Minor
 Fix For: 3.0.0

 Attachments: HADOOP-8593.patch


 Adding @Override to the proper methods to take advantage of the compiler 
 checking and make the code more readable. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8593) add the missed @Override to methods in Metric/Metric2 package

2012-07-16 Thread Suresh Srinivas (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suresh Srinivas updated HADOOP-8593:


  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

I committed the patch. Thank you Brandon.

 add  the missed @Override to methods in Metric/Metric2 package
 --

 Key: HADOOP-8593
 URL: https://issues.apache.org/jira/browse/HADOOP-8593
 Project: Hadoop Common
  Issue Type: Improvement
  Components: metrics
Affects Versions: 1.0.0, 3.0.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Minor
 Fix For: 3.0.0

 Attachments: HADOOP-8593.patch


 Adding @Override to the proper methods to take advantage of the compiler 
 checking and make the code more readable. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8599) Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file

2012-07-16 Thread Todd Lipcon (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415739#comment-13415739
 ] 

Todd Lipcon commented on HADOOP-8599:
-

+1, looks good to me. Thanks Andrey.

 Non empty response from FileSystem.getFileBlockLocations when asking for data 
 beyond the end of file 
 -

 Key: HADOOP-8599
 URL: https://issues.apache.org/jira/browse/HADOOP-8599
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs
Affects Versions: 1.0.3, 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Attachments: HADOOP-8859-branch-0.23.patch


 When FileSystem.getFileBlockLocations(file,start,len) is called with start 
 argument equal to the file size, the response is not empty. There is a test 
 TestGetFileBlockLocations.testGetFileBlockLocations2 which uses randomly 
 generated start and len arguments when calling 
 FileSystem.getFileBlockLocations and the test fails randomly (when the 
 generated start value equals to the file size).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HADOOP-8599) Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file

2012-07-16 Thread Todd Lipcon (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Todd Lipcon updated HADOOP-8599:


   Resolution: Fixed
Fix Version/s: 2.2.0-alpha
   3.0.0
   0.23.3
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

 Non empty response from FileSystem.getFileBlockLocations when asking for data 
 beyond the end of file 
 -

 Key: HADOOP-8599
 URL: https://issues.apache.org/jira/browse/HADOOP-8599
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs
Affects Versions: 1.0.3, 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Fix For: 0.23.3, 3.0.0, 2.2.0-alpha

 Attachments: HADOOP-8859-branch-0.23.patch


 When FileSystem.getFileBlockLocations(file,start,len) is called with start 
 argument equal to the file size, the response is not empty. There is a test 
 TestGetFileBlockLocations.testGetFileBlockLocations2 which uses randomly 
 generated start and len arguments when calling 
 FileSystem.getFileBlockLocations and the test fails randomly (when the 
 generated start value equals to the file size).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8599) Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file


[ 
https://issues.apache.org/jira/browse/HADOOP-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415745#comment-13415745
 ] 

Hudson commented on HADOOP-8599:


Integrated in Hadoop-Hdfs-trunk-Commit #2551 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2551/])
HADOOP-8599. Non empty response from FileSystem.getFileBlockLocations when 
asking for data beyond the end of file. Contributed by Andrey Klochkov. 
(Revision 1362295)

 Result = SUCCESS
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1362295
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestGetFileBlockLocations.java


 Non empty response from FileSystem.getFileBlockLocations when asking for data 
 beyond the end of file 
 -

 Key: HADOOP-8599
 URL: https://issues.apache.org/jira/browse/HADOOP-8599
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs
Affects Versions: 1.0.3, 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Fix For: 0.23.3, 3.0.0, 2.2.0-alpha

 Attachments: HADOOP-8859-branch-0.23.patch


 When FileSystem.getFileBlockLocations(file,start,len) is called with start 
 argument equal to the file size, the response is not empty. There is a test 
 TestGetFileBlockLocations.testGetFileBlockLocations2 which uses randomly 
 generated start and len arguments when calling 
 FileSystem.getFileBlockLocations and the test fails randomly (when the 
 generated start value equals to the file size).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8593) add the missed @Override to methods in Metric/Metric2 package


[ 
https://issues.apache.org/jira/browse/HADOOP-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415746#comment-13415746
 ] 

Hudson commented on HADOOP-8593:


Integrated in Hadoop-Hdfs-trunk-Commit #2551 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2551/])
HADOOP-8593. Add missed @Override annotations in Metric/Metrics2 package. 
Contributed by Brandon Li. (Revision 1362294)

 Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1362294
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/MetricsServlet.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/MetricsUtil.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/file/FileContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/ganglia/GangliaContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/ganglia/GangliaContext31.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/jvm/JvmMetrics.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/AbstractMetricsContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/CompositeContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/MetricsRecordImpl.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/NoEmitMetricsContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/NullContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/NullContextWithUpdateThread.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsIntValue.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsLongValue.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsTimeVaryingInt.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsTimeVaryingLong.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsTimeVaryingRate.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsConfig.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSinkAdapter.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MethodMetric.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MetricsSourceBuilder.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableGaugeLong.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableStat.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/FileSink.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/AbstractGangliaSink.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/GangliaSink31.java


 add  the missed @Override to methods in Metric/Metric2 package
 --

 Key: HADOOP-8593
 URL: https://issues.apache.org/jira/browse/HADOOP-8593
 Project: Hadoop Common
  Issue Type: Improvement
  Components: metrics
Affects Versions: 1.0.0, 3.0.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Minor
 Fix For: 3.0.0

 Attachments: HADOOP-8593.patch


 Adding @Override to the proper methods to take advantage of the compiler 
 checking and make the code more readable. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly,

[jira] [Commented] (HADOOP-8599) Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file


[ 
https://issues.apache.org/jira/browse/HADOOP-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415750#comment-13415750
 ] 

Hudson commented on HADOOP-8599:


Integrated in Hadoop-Common-trunk-Commit #2486 (See 
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2486/])
HADOOP-8599. Non empty response from FileSystem.getFileBlockLocations when 
asking for data beyond the end of file. Contributed by Andrey Klochkov. 
(Revision 1362295)

 Result = SUCCESS
todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1362295
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestGetFileBlockLocations.java


 Non empty response from FileSystem.getFileBlockLocations when asking for data 
 beyond the end of file 
 -

 Key: HADOOP-8599
 URL: https://issues.apache.org/jira/browse/HADOOP-8599
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs
Affects Versions: 1.0.3, 0.23.1, 2.0.0-alpha
Reporter: Andrey Klochkov
Assignee: Andrey Klochkov
 Fix For: 0.23.3, 3.0.0, 2.2.0-alpha

 Attachments: HADOOP-8859-branch-0.23.patch


 When FileSystem.getFileBlockLocations(file,start,len) is called with start 
 argument equal to the file size, the response is not empty. There is a test 
 TestGetFileBlockLocations.testGetFileBlockLocations2 which uses randomly 
 generated start and len arguments when calling 
 FileSystem.getFileBlockLocations and the test fails randomly (when the 
 generated start value equals to the file size).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-8593) add the missed @Override to methods in Metric/Metric2 package


[ 
https://issues.apache.org/jira/browse/HADOOP-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415751#comment-13415751
 ] 

Hudson commented on HADOOP-8593:


Integrated in Hadoop-Common-trunk-Commit #2486 (See 
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2486/])
HADOOP-8593. Add missed @Override annotations in Metric/Metrics2 package. 
Contributed by Brandon Li. (Revision 1362294)

 Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1362294
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/MetricsServlet.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/MetricsUtil.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/file/FileContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/ganglia/GangliaContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/ganglia/GangliaContext31.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/jvm/JvmMetrics.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/AbstractMetricsContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/CompositeContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/MetricsRecordImpl.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/NoEmitMetricsContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/NullContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/NullContextWithUpdateThread.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsIntValue.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsLongValue.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsTimeVaryingInt.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsTimeVaryingLong.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsTimeVaryingRate.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsConfig.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSinkAdapter.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MethodMetric.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MetricsSourceBuilder.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableGaugeLong.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableStat.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/FileSink.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/AbstractGangliaSink.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/GangliaSink31.java


 add  the missed @Override to methods in Metric/Metric2 package
 --

 Key: HADOOP-8593
 URL: https://issues.apache.org/jira/browse/HADOOP-8593
 Project: Hadoop Common
  Issue Type: Improvement
  Components: metrics
Affects Versions: 1.0.0, 3.0.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Minor
 Fix For: 3.0.0

 Attachments: HADOOP-8593.patch


 Adding @Override to the proper methods to take advantage of the compiler 
 checking and make the code more readable. 

--
This message is automatically generated by JIRA.
If you think it was sent

[jira] [Commented] (HADOOP-8593) add the missed @Override to methods in Metric/Metric2 package


[ 
https://issues.apache.org/jira/browse/HADOOP-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415752#comment-13415752
 ] 

Hudson commented on HADOOP-8593:


Integrated in Hadoop-Mapreduce-trunk-Commit #2506 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2506/])
HADOOP-8593. Add missed @Override annotations in Metric/Metrics2 package. 
Contributed by Brandon Li. (Revision 1362294)

 Result = FAILURE
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1362294
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/MetricsServlet.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/MetricsUtil.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/file/FileContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/ganglia/GangliaContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/ganglia/GangliaContext31.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/jvm/JvmMetrics.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/AbstractMetricsContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/CompositeContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/MetricsRecordImpl.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/NoEmitMetricsContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/NullContext.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/spi/NullContextWithUpdateThread.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsIntValue.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsLongValue.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsTimeVaryingInt.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsTimeVaryingLong.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics/util/MetricsTimeVaryingRate.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsConfig.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSinkAdapter.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSourceAdapter.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/impl/MetricsSystemImpl.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MethodMetric.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MetricsSourceBuilder.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableGaugeLong.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/MutableStat.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/FileSink.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/AbstractGangliaSink.java
* 
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/GangliaSink31.java


 add  the missed @Override to methods in Metric/Metric2 package
 --

 Key: HADOOP-8593
 URL: https://issues.apache.org/jira/browse/HADOOP-8593
 Project: Hadoop Common
  Issue Type: Improvement
  Components: metrics
Affects Versions: 1.0.0, 3.0.0
Reporter: Brandon Li
Assignee: Brandon Li
Priority: Minor
 Fix For: 3.0.0

 Attachments: HADOOP-8593.patch


 Adding @Override to the proper methods to take advantage of the compiler 
 checking and make the code more readable. 

--
This message is automatically generated by JIRA.
If you think it was sent

[jira] [Commented] (HADOOP-8599) Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file