[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493024#comment-13493024 ] Yu Li commented on HADOOP-8419: --- Attached the patch for branch-1 > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Attachments: HADOOP-8419-branch-1.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Li updated HADOOP-8419: -- Attachment: HADOOP-8419-branch-1.patch > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Attachments: HADOOP-8419-branch-1.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-8419 started by Yu Li. > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493011#comment-13493011 ] Yu Li commented on HADOOP-8419: --- More details about this issue: In IBM JDK, GZIPOutputStream class calls the deflater's end method as part of GZIPOutputStream.finish(), so deflater's reset can't be called after, while in SUN or Open JDK implementation this end method won't be called. To work-around this issue, we need to override the finish method of coresponding classes which extends GZIPOutputStream, thus won't depend on implementation of differenct JDK. And since the needed writeTrailer, writeInt and writeShort all become private method in JDK6(SUN/IBM/OPEN JDK), we also need to add these 3 methods in the patch. > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested
[
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493009#comment-13493009
]
Hudson commented on HADOOP-8589:
Integrated in Hadoop-trunk-Commit #2977 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/2977/])
HADOOP-8589 ViewFs tests fail when tests and home dirs are nested (sanjay
Radia) (Revision 1406939)
Result = SUCCESS
sradia : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406939
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DelegateToFileSystem.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFileSystem.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemTestHelper.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestChRootedFileSystem.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestChRootedFs.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestFcMainOperationsLocalFs.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemTestSetup.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFsTestSetup.java
> ViewFs tests fail when tests and home dirs are nested
> -
>
> Key: HADOOP-8589
> URL: https://issues.apache.org/jira/browse/HADOOP-8589
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 0.23.1, 2.0.0-alpha
>Reporter: Andrey Klochkov
>Assignee: Sanjay Radia
> Attachments: Hadoop-8589.patch, HADOOP-8589.patch, HADOOP-8589.patch,
> hadoop-8589-sanjay.patch, Hadoop-8589-v2.patch, HADOOP-8859.patch
>
>
> TestFSMainOperationsLocalFileSystem fails in case when the test root
> directory is under the user's home directory, and the user's home dir is
> deeper than 2 levels from /. This happens with the default 1-node
> installation of Jenkins.
> This is the failure log:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as
> dir; cannot create link here
> at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
> at org.apache.hadoop.fs.viewfs.InodeTree.(InodeTree.java:334)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem$1.(ViewFileSystem.java:167)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
> at
> org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
> ...
> Standard Output
> 2012-07-11 22:07:20,239 INFO mortbay.log (Slf4jLog.java:info(67)) - Home dir
> base /var/lib
> {code}
> The reason for the failure is that the code tries to mount links for both
> "/var" and "/var/lib", and it fails for the 2nd one as the "/var" is mounted
> already.
> The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested
[
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sanjay Radia updated HADOOP-8589:
-
Resolution: Fixed
Target Version/s: 0.23.4, 3.0.0, 2.0.3-alpha (was: 3.0.0, 0.23.4,
2.0.3-alpha)
Status: Resolved (was: Patch Available)
> ViewFs tests fail when tests and home dirs are nested
> -
>
> Key: HADOOP-8589
> URL: https://issues.apache.org/jira/browse/HADOOP-8589
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 0.23.1, 2.0.0-alpha
>Reporter: Andrey Klochkov
>Assignee: Sanjay Radia
> Attachments: Hadoop-8589.patch, HADOOP-8589.patch, HADOOP-8589.patch,
> hadoop-8589-sanjay.patch, Hadoop-8589-v2.patch, HADOOP-8859.patch
>
>
> TestFSMainOperationsLocalFileSystem fails in case when the test root
> directory is under the user's home directory, and the user's home dir is
> deeper than 2 levels from /. This happens with the default 1-node
> installation of Jenkins.
> This is the failure log:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as
> dir; cannot create link here
> at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
> at org.apache.hadoop.fs.viewfs.InodeTree.(InodeTree.java:334)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem$1.(ViewFileSystem.java:167)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
> at
> org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
> ...
> Standard Output
> 2012-07-11 22:07:20,239 INFO mortbay.log (Slf4jLog.java:info(67)) - Home dir
> base /var/lib
> {code}
> The reason for the failure is that the code tries to mount links for both
> "/var" and "/var/lib", and it fails for the 2nd one as the "/var" is mounted
> already.
> The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8953) Shell PathData parsing failures on Windows
[ https://issues.apache.org/jira/browse/HADOOP-8953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HADOOP-8953. - Resolution: Fixed Fix Version/s: trunk-win Hadoop Flags: Reviewed I committed the patch to branch-trunk-win. Thank you Arpit. Please continue additional work needed in HADOOP-9018. > Shell PathData parsing failures on Windows > -- > > Key: HADOOP-8953 > URL: https://issues.apache.org/jira/browse/HADOOP-8953 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: trunk-win >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Fix For: trunk-win > > Attachments: HADOOP-8953-branch-trunk-win-3.patch, > HADOOP-8953-branch-trunk-win-4.patch, HADOOP-8953-branch-trunk-win-6.patch, > HADOOP-8953-branch-trunk-win.patch > > > Several test suites fail on Windows, apparently due to Windows-specific path > parsing bugs in PathData. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8860) Split MapReduce and YARN sections in documentation navigation
[ https://issues.apache.org/jira/browse/HADOOP-8860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492896#comment-13492896 ] Andy Isaacson commented on HADOOP-8860: --- The patch seems reasonable to me. +1. > Split MapReduce and YARN sections in documentation navigation > - > > Key: HADOOP-8860 > URL: https://issues.apache.org/jira/browse/HADOOP-8860 > Project: Hadoop Common > Issue Type: Task > Components: documentation >Affects Versions: 2.0.1-alpha >Reporter: Tom White >Assignee: Tom White > Attachments: HADOOP-8860.patch, HADOOP-8860.sh > > > This JIRA is to change the navigation on > http://hadoop.apache.org/docs/r2.0.1-alpha/ to reflect the fact that > MapReduce and YARN are separate modules/sub-projects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8820) Backport HADOOP-8469 and HADOOP-8470: add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup)
[ https://issues.apache.org/jira/browse/HADOOP-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492890#comment-13492890 ] Junping Du commented on HADOOP-8820: Thanks Nicholas for reviewing. I will address your comments in HADOOP-8817 where I put all 4 patches together. Hopefully, it is easy to review and maintain. > Backport HADOOP-8469 and HADOOP-8470: add "NodeGroup" layer in new > NetworkTopology (also known as NetworkTopologyWithNodeGroup) > --- > > Key: HADOOP-8820 > URL: https://issues.apache.org/jira/browse/HADOOP-8820 > Project: Hadoop Common > Issue Type: New Feature > Components: net >Affects Versions: 1.0.0 >Reporter: Junping Du >Assignee: Junping Du > Attachments: HADOOP-8820.patch > > > This patch backport HADOOP-8469 and HADOOP-8470 to branch-1 and includes: > 1. Make NetworkTopology class pluggable for extension. > 2. Implement a 4-layer NetworkTopology class (named as > NetworkTopologyWithNodeGroup) to use in virtualized environment (or other > situation with additional layer between host and rack). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8977) multiple FsShell test failures on Windows
[ https://issues.apache.org/jira/browse/HADOOP-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492852#comment-13492852 ] Chris Nauroth commented on HADOOP-8977: --- Hi, Daryn. Do you have any additional feedback on the new version of the patch that I uploaded on 10/31? Thank you. > multiple FsShell test failures on Windows > - > > Key: HADOOP-8977 > URL: https://issues.apache.org/jira/browse/HADOOP-8977 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: trunk-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Attachments: HADOOP-8977-branch-trunk-win.patch, > HADOOP-8977-branch-trunk-win.patch > > > Multiple FsShell-related tests fail on Windows. Commands are returning > non-zero exit status. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9008) Building hadoop tarball fails on Windows
[
https://issues.apache.org/jira/browse/HADOOP-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-9008:
--
Attachment: HADOOP-9008-branch-trunk-win.patch
The attached patch ports the sh scripting in the distribution build to Python.
It wasn't possible to use only Maven plugins (like maven-antrun-plugin with a
task), because they mishandled permissions and symlinks in the built
tarballs.
I tested all of the following build variations:
Windows: mvn -Pnative-win -Pdist -Dtar -DskipTests clean package
Mac: mvn -Pdist -Dtar -DskipTests clean package
Ubuntu: mvn -Pnative -Pdist -Dtar -DskipTests clean package
Ubuntu: mvn -Pnative -Pdist -Dtar -Drequire.snappy -Dbundle.snappy
-Dsnappy.lib=/usr/local/lib -DskipTests clean package
This works on Windows. Additionally, on Mac and Ubuntu, I compared the built
tarballs from before and after my changes. I confirmed that the resulting
tarballs have exactly the same contents, including permissions and symlinks.
> Building hadoop tarball fails on Windows
>
>
> Key: HADOOP-9008
> URL: https://issues.apache.org/jira/browse/HADOOP-9008
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: trunk-win
>Reporter: Ivan Mitic
>Assignee: Chris Nauroth
> Attachments: HADOOP-9008-branch-trunk-win.patch
>
>
> Trying to build Hadoop trunk tarball via {{mvn package -Pdist -DskipTests
> -Dtar}} fails on Windows.
> Build system generates sh scripts that execute build tasks what does not work
> on Windows without Cygwin. It might make sense to apply the same pattern as
> in HADOOP-8924, and use python instead of sh.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8560) Change TestFTPFileSystem to use non-SNAPSHOT dependencies
[ https://issues.apache.org/jira/browse/HADOOP-8560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492839#comment-13492839 ] Yu Gao commented on HADOOP-8560: The SNAPSHOT ftpserver and mina jars should be removed from src/test/lib as well. > Change TestFTPFileSystem to use non-SNAPSHOT dependencies > - > > Key: HADOOP-8560 > URL: https://issues.apache.org/jira/browse/HADOOP-8560 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Gao >Priority: Minor > Attachments: hadoop-8560-branch-1.patch, test-patch.result, > test-TestFTPFileSystem.result > > > It would good if the stable hadoop release don't depend on SNAPSHOT ftpserver > artifacts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8560) Change TestFTPFileSystem to use non-SNAPSHOT dependencies
[ https://issues.apache.org/jira/browse/HADOOP-8560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492838#comment-13492838 ] Yu Gao commented on HADOOP-8560: Attached ant test and test-patch results. Also put the overall result here: ant test-patch: [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.1) warnings. ant test -Dtestcase=TestFTPFileSystem Testsuite: org.apache.hadoop.fs.ftp.TestFTPFileSystem Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 20.243 sec > Change TestFTPFileSystem to use non-SNAPSHOT dependencies > - > > Key: HADOOP-8560 > URL: https://issues.apache.org/jira/browse/HADOOP-8560 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Gao >Priority: Minor > Attachments: hadoop-8560-branch-1.patch, test-patch.result, > test-TestFTPFileSystem.result > > > It would good if the stable hadoop release don't depend on SNAPSHOT ftpserver > artifacts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8560) Change TestFTPFileSystem to use non-SNAPSHOT dependencies
[ https://issues.apache.org/jira/browse/HADOOP-8560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Gao updated HADOOP-8560: --- Attachment: test-TestFTPFileSystem.result test-patch.result > Change TestFTPFileSystem to use non-SNAPSHOT dependencies > - > > Key: HADOOP-8560 > URL: https://issues.apache.org/jira/browse/HADOOP-8560 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Gao >Priority: Minor > Attachments: hadoop-8560-branch-1.patch, test-patch.result, > test-TestFTPFileSystem.result > > > It would good if the stable hadoop release don't depend on SNAPSHOT ftpserver > artifacts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8953) Shell PathData parsing failures on Windows
[ https://issues.apache.org/jira/browse/HADOOP-8953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492793#comment-13492793 ] Arpit Agarwal commented on HADOOP-8953: --- Daryn suggested a few further improvements to make Windows URI handling more rigorous and I have filed HADOOP-9018 to track them. I would like to avoid inflating the scope of HADOOP-8953 and allow us to make forward progress with getting Windows support working in trunk-win. > Shell PathData parsing failures on Windows > -- > > Key: HADOOP-8953 > URL: https://issues.apache.org/jira/browse/HADOOP-8953 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: trunk-win >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-8953-branch-trunk-win-3.patch, > HADOOP-8953-branch-trunk-win-4.patch, HADOOP-8953-branch-trunk-win-6.patch, > HADOOP-8953-branch-trunk-win.patch > > > Several test suites fail on Windows, apparently due to Windows-specific path > parsing bugs in PathData. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9015) Standardize creation of SaslRpcServers
[ https://issues.apache.org/jira/browse/HADOOP-9015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated HADOOP-9015: Resolution: Fixed Fix Version/s: 2.0.3-alpha 3.0.0 Status: Resolved (was: Patch Available) Thanks Daryn, I put this into trunk and branch-2 > Standardize creation of SaslRpcServers > -- > > Key: HADOOP-9015 > URL: https://issues.apache.org/jira/browse/HADOOP-9015 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Fix For: 3.0.0, 2.0.3-alpha > > Attachments: HADOOP-9015.patch > > > To ease adding additional SASL support, need to merge the multiple switches > for mechanism type and server creation into a single switch with a single > call to createSaslServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9015) Standardize creation of SaslRpcServers
[ https://issues.apache.org/jira/browse/HADOOP-9015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492787#comment-13492787 ] Hudson commented on HADOOP-9015: Integrated in Hadoop-trunk-Commit #2976 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/2976/]) HADOOP-9015. Standardize creation of SaslRpcServers (daryn via bobby) (Revision 1406851) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406851 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java > Standardize creation of SaslRpcServers > -- > > Key: HADOOP-9015 > URL: https://issues.apache.org/jira/browse/HADOOP-9015 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Fix For: 3.0.0, 2.0.3-alpha > > Attachments: HADOOP-9015.patch > > > To ease adding additional SASL support, need to merge the multiple switches > for mechanism type and server creation into a single switch with a single > call to createSaslServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9015) Standardize creation of SaslRpcServers
[ https://issues.apache.org/jira/browse/HADOOP-9015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492783#comment-13492783 ] Robert Joseph Evans commented on HADOOP-9015: - The changes look OK to me. I am +1. I'll check them in. > Standardize creation of SaslRpcServers > -- > > Key: HADOOP-9015 > URL: https://issues.apache.org/jira/browse/HADOOP-9015 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Attachments: HADOOP-9015.patch > > > To ease adding additional SASL support, need to merge the multiple switches > for mechanism type and server creation into a single switch with a single > call to createSaslServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9018) Reject invalid Windows URIs
Arpit Agarwal created HADOOP-9018: - Summary: Reject invalid Windows URIs Key: HADOOP-9018 URL: https://issues.apache.org/jira/browse/HADOOP-9018 Project: Hadoop Common Issue Type: Improvement Components: fs Affects Versions: trunk-win Environment: Windows Reporter: Arpit Agarwal Assignee: Arpit Agarwal This JIRA is to make handling of improperly constructed file URIs for Windows local paths more rigorous. e.g. reject "file:///c:\\Windows" Valid file URI syntax explained at http://blogs.msdn.com/b/ie/archive/2006/12/06/file-uris-in-windows.aspx. Also see https://issues.apache.org/jira/browse/HADOOP-8953 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8953) Shell PathData parsing failures on Windows
[ https://issues.apache.org/jira/browse/HADOOP-8953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492772#comment-13492772 ] Suresh Srinivas commented on HADOOP-8953: - Given that this change is needed for making progress on gettign 100% unit tests passing on Windows, I recommend creating a follow on jira for completing the discussion and finishing the remaining discussion. Arpit, can you create another jira? > Shell PathData parsing failures on Windows > -- > > Key: HADOOP-8953 > URL: https://issues.apache.org/jira/browse/HADOOP-8953 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: trunk-win >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-8953-branch-trunk-win-3.patch, > HADOOP-8953-branch-trunk-win-4.patch, HADOOP-8953-branch-trunk-win-6.patch, > HADOOP-8953-branch-trunk-win.patch > > > Several test suites fail on Windows, apparently due to Windows-specific path > parsing bugs in PathData. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and getpwgid_r calls
[
https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492770#comment-13492770
]
Hadoop QA commented on HADOOP-7115:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12552549/HADOOP-7115.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1721//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1721//console
This message is automatically generated.
> Add a cache for getpwuid_r and getpwgid_r calls
> ---
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
>Reporter: Arun C Murthy
>Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch,
> hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and getpwgid_r calls
[
https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492754#comment-13492754
]
Todd Lipcon commented on HADOOP-7115:
-
{code}
+cacheTimeout = new Configuration().getLong(
+ CommonConfigurationKeys.HADOOP_SECURITY_UID_NAME_CACHE_TIMEOUT_KEY,
+ CommonConfigurationKeys.HADOOP_SECURITY_UID_NAME_CACHE_TIMEOUT_DEFAULT) *
+ 1000;
+LOG.debug("Initialized cache for IDs to User/Group mapping with a" +
+ " cache timeout of " + cacheTimeout/1000 + " seconds.");
{code}
This should move up into the try block above -- if the native library fails to
load, then it shouldn't log this debug message (since the cache won't ever be
used). Additionally you can reuse the Configuration instance that's already
constructed above.
- Just noticed an addition of {{syslog.h}} to the native code - doesn't seem to
be used.
Otherwise looks good.
> Add a cache for getpwuid_r and getpwgid_r calls
> ---
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
>Reporter: Arun C Murthy
>Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch,
> hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9015) Standardize creation of SaslRpcServers
[
https://issues.apache.org/jira/browse/HADOOP-9015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492750#comment-13492750
]
Hadoop QA commented on HADOOP-9015:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12552543/HADOOP-9015.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1720//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1720//console
This message is automatically generated.
> Standardize creation of SaslRpcServers
> --
>
> Key: HADOOP-9015
> URL: https://issues.apache.org/jira/browse/HADOOP-9015
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: ipc
>Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
> Attachments: HADOOP-9015.patch
>
>
> To ease adding additional SASL support, need to merge the multiple switches
> for mechanism type and server creation into a single switch with a single
> call to createSaslServer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and getpwgid_r calls
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492747#comment-13492747 ] Alejandro Abdelnur commented on HADOOP-7115: Forgot the mention before, the native part of the patch it was done by Sandy Ryza (thx Sandy). > Add a cache for getpwuid_r and getpwgid_r calls > --- > > Key: HADOOP-7115 > URL: https://issues.apache.org/jira/browse/HADOOP-7115 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4 >Reporter: Arun C Murthy >Assignee: Alejandro Abdelnur > Fix For: 0.22.1, 2.0.3-alpha > > Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, > hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch > > > As discussed in HADOOP-6978, a cache helps a lot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and getpwgid_r calls
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated HADOOP-7115: --- Attachment: HADOOP-7115.patch Thanks for the review Todd, new patch addressing you comments. Minor tweak, converted the INT constants to an ENUM but did not move the cache into the ENUM, left as before (not very keen of mutable ENUMs). > Add a cache for getpwuid_r and getpwgid_r calls > --- > > Key: HADOOP-7115 > URL: https://issues.apache.org/jira/browse/HADOOP-7115 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4 >Reporter: Arun C Murthy >Assignee: Alejandro Abdelnur > Fix For: 0.22.1, 2.0.3-alpha > > Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, > hadoop-7115-0.22.patch, HADOOP-7115.patch, HADOOP-7115.patch > > > As discussed in HADOOP-6978, a cache helps a lot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and getpwgid_r calls
[
https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492733#comment-13492733
]
Todd Lipcon commented on HADOOP-7115:
-
- why is cacheTimeout volatile? You're already synchronizing on the class
inside {{ensureInitialized}}.
- instead of {{ensureInitialized}}, why not initialize the configuration in the
existing static {...} block?
- instead of using ints for {{USER}} and {{GROUP}}, how about an enum like this:
{code}
enum IdCache {
USERNAME,
GROUP;
ConcurrentHashMap cache =
new ConcurrentHashMap();
}
...
private static String getName(IdCache cache, int id) {
CachedName cachedName = cache.cache.get(id);
...
}
{code}
this way you get typesafety, and you can just stringify the {{IdCache}}
instance to get a printable name
Otherwise looks good to me
> Add a cache for getpwuid_r and getpwgid_r calls
> ---
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
>Reporter: Arun C Murthy
>Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch,
> hadoop-7115-0.22.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9015) Standardize creation of SaslRpcServers
[ https://issues.apache.org/jira/browse/HADOOP-9015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-9015: Status: Patch Available (was: Open) > Standardize creation of SaslRpcServers > -- > > Key: HADOOP-9015 > URL: https://issues.apache.org/jira/browse/HADOOP-9015 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 2.0.0-alpha, 0.23.0, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Attachments: HADOOP-9015.patch > > > To ease adding additional SASL support, need to merge the multiple switches > for mechanism type and server creation into a single switch with a single > call to createSaslServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9015) Standardize creation of SaslRpcServers
[ https://issues.apache.org/jira/browse/HADOOP-9015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HADOOP-9015: Attachment: HADOOP-9015.patch Instead of having two switches and deferred instantiation of the sasl server, merge the two and immediately create the server once it's known to be needed. > Standardize creation of SaslRpcServers > -- > > Key: HADOOP-9015 > URL: https://issues.apache.org/jira/browse/HADOOP-9015 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Attachments: HADOOP-9015.patch > > > To ease adding additional SASL support, need to merge the multiple switches > for mechanism type and server creation into a single switch with a single > call to createSaslServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7115) Add a cache for getpwuid_r and getpwgid_r calls
[
https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492717#comment-13492717
]
Hadoop QA commented on HADOOP-7115:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12552538/HADOOP-7115.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1719//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1719//console
This message is automatically generated.
> Add a cache for getpwuid_r and getpwgid_r calls
> ---
>
> Key: HADOOP-7115
> URL: https://issues.apache.org/jira/browse/HADOOP-7115
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4
>Reporter: Arun C Murthy
>Assignee: Alejandro Abdelnur
> Fix For: 0.22.1, 2.0.3-alpha
>
> Attachments: h-7115.1.patch, hadoop-7115-0.22.patch,
> hadoop-7115-0.22.patch, HADOOP-7115.patch
>
>
> As discussed in HADOOP-6978, a cache helps a lot.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and getpwgid_r calls
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated HADOOP-7115: --- Attachment: HADOOP-7115.patch Patch for trunk. It revives the cache for ID->NAME. It has been refactored a bit to be used for both UIDs and GIDs. It has been tested in a secure cluster configured to use and external directory for user provisioning running terasort jobs. Without this patch the same cluster cannot run a single terasort job for non-local users. > Add a cache for getpwuid_r and getpwgid_r calls > --- > > Key: HADOOP-7115 > URL: https://issues.apache.org/jira/browse/HADOOP-7115 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 0.22.0, 2.0.2-alpha, 0.23.4 >Reporter: Arun C Murthy >Assignee: Alejandro Abdelnur > Fix For: 0.22.1, 2.0.3-alpha > > Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, > hadoop-7115-0.22.patch, HADOOP-7115.patch > > > As discussed in HADOOP-6978, a cache helps a lot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7115) Add a cache for getpwuid_r and getpwgid_r calls
[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated HADOOP-7115: --- Fix Version/s: 2.0.3-alpha Status: Patch Available (was: Open) > Add a cache for getpwuid_r and getpwgid_r calls > --- > > Key: HADOOP-7115 > URL: https://issues.apache.org/jira/browse/HADOOP-7115 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 0.23.4, 2.0.2-alpha, 0.22.0 >Reporter: Arun C Murthy >Assignee: Alejandro Abdelnur > Fix For: 0.22.1, 2.0.3-alpha > > Attachments: h-7115.1.patch, hadoop-7115-0.22.patch, > hadoop-7115-0.22.patch, HADOOP-7115.patch > > > As discussed in HADOOP-6978, a cache helps a lot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9017) fix hadoop-client-pom-template.xml and hadoop-client-pom-template.xml for version
[ https://issues.apache.org/jira/browse/HADOOP-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated HADOOP-9017: --- Attachment: HADOOP-9017.patch > fix hadoop-client-pom-template.xml and hadoop-client-pom-template.xml for > version > -- > > Key: HADOOP-9017 > URL: https://issues.apache.org/jira/browse/HADOOP-9017 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 1.0.4, 1.1.0 >Reporter: Giridharan Kesavan >Assignee: Giridharan Kesavan > Attachments: HADOOP-9017.patch > > > hadoop-client-pom-template.xml and hadoop-client-pom-template.xml references > to project.version variable, instead they should refer to @version token. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9017) fix hadoop-client-pom-template.xml and hadoop-client-pom-template.xml for version
Giridharan Kesavan created HADOOP-9017: -- Summary: fix hadoop-client-pom-template.xml and hadoop-client-pom-template.xml for version Key: HADOOP-9017 URL: https://issues.apache.org/jira/browse/HADOOP-9017 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 1.1.0, 1.0.4 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan hadoop-client-pom-template.xml and hadoop-client-pom-template.xml references to project.version variable, instead they should refer to @version token. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested
[
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492631#comment-13492631
]
Hadoop QA commented on HADOOP-8589:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12552511/Hadoop-8589-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1718//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1718//console
This message is automatically generated.
> ViewFs tests fail when tests and home dirs are nested
> -
>
> Key: HADOOP-8589
> URL: https://issues.apache.org/jira/browse/HADOOP-8589
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 0.23.1, 2.0.0-alpha
>Reporter: Andrey Klochkov
>Assignee: Sanjay Radia
> Attachments: Hadoop-8589.patch, HADOOP-8589.patch, HADOOP-8589.patch,
> hadoop-8589-sanjay.patch, Hadoop-8589-v2.patch, HADOOP-8859.patch
>
>
> TestFSMainOperationsLocalFileSystem fails in case when the test root
> directory is under the user's home directory, and the user's home dir is
> deeper than 2 levels from /. This happens with the default 1-node
> installation of Jenkins.
> This is the failure log:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as
> dir; cannot create link here
> at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
> at org.apache.hadoop.fs.viewfs.InodeTree.(InodeTree.java:334)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem$1.(ViewFileSystem.java:167)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
> at
> org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
> ...
> Standard Output
> 2012-07-11 22:07:20,239 INFO mortbay.log (Slf4jLog.java:info(67)) - Home dir
> base /var/lib
> {code}
> The reason for the failure is that the code tries to mount links for both
> "/var" and "/var/lib", and it fails for the 2nd one as the "/var" is mounted
> already.
> The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested
[
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HADOOP-8589:
---
Target Version/s: 0.23.4, 3.0.0, 2.0.3-alpha (was: 3.0.0, 0.23.4,
2.0.3-alpha)
Hadoop Flags: Reviewed
+1 the new patch looks good.
> ViewFs tests fail when tests and home dirs are nested
> -
>
> Key: HADOOP-8589
> URL: https://issues.apache.org/jira/browse/HADOOP-8589
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 0.23.1, 2.0.0-alpha
>Reporter: Andrey Klochkov
>Assignee: Sanjay Radia
> Attachments: Hadoop-8589.patch, HADOOP-8589.patch, HADOOP-8589.patch,
> hadoop-8589-sanjay.patch, Hadoop-8589-v2.patch, HADOOP-8859.patch
>
>
> TestFSMainOperationsLocalFileSystem fails in case when the test root
> directory is under the user's home directory, and the user's home dir is
> deeper than 2 levels from /. This happens with the default 1-node
> installation of Jenkins.
> This is the failure log:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as
> dir; cannot create link here
> at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
> at org.apache.hadoop.fs.viewfs.InodeTree.(InodeTree.java:334)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem$1.(ViewFileSystem.java:167)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
> at
> org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
> ...
> Standard Output
> 2012-07-11 22:07:20,239 INFO mortbay.log (Slf4jLog.java:info(67)) - Home dir
> base /var/lib
> {code}
> The reason for the failure is that the code tries to mount links for both
> "/var" and "/var/lib", and it fails for the 2nd one as the "/var" is mounted
> already.
> The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested
[
https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sanjay Radia updated HADOOP-8589:
-
Attachment: Hadoop-8589-v2.patch
Update patch - incorporates Nicolas's feedaback and run successfully against
homedir of /joe, /x/joe, /x/y/joe
> ViewFs tests fail when tests and home dirs are nested
> -
>
> Key: HADOOP-8589
> URL: https://issues.apache.org/jira/browse/HADOOP-8589
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, test
>Affects Versions: 0.23.1, 2.0.0-alpha
>Reporter: Andrey Klochkov
>Assignee: Sanjay Radia
> Attachments: Hadoop-8589.patch, HADOOP-8589.patch, HADOOP-8589.patch,
> hadoop-8589-sanjay.patch, Hadoop-8589-v2.patch, HADOOP-8859.patch
>
>
> TestFSMainOperationsLocalFileSystem fails in case when the test root
> directory is under the user's home directory, and the user's home dir is
> deeper than 2 levels from /. This happens with the default 1-node
> installation of Jenkins.
> This is the failure log:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as
> dir; cannot create link here
> at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
> at org.apache.hadoop.fs.viewfs.InodeTree.(InodeTree.java:334)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem$1.(ViewFileSystem.java:167)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
> at
> org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
> at
> org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
> ...
> Standard Output
> 2012-07-11 22:07:20,239 INFO mortbay.log (Slf4jLog.java:info(67)) - Home dir
> base /var/lib
> {code}
> The reason for the failure is that the code tries to mount links for both
> "/var" and "/var/lib", and it fails for the 2nd one as the "/var" is mounted
> already.
> The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9014) Standardize creation of SaslRpcClients
[ https://issues.apache.org/jira/browse/HADOOP-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492461#comment-13492461 ] Hudson commented on HADOOP-9014: Integrated in Hadoop-trunk-Commit #2972 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/2972/]) HADOOP-9014. Standardize creation of SaslRpcClients (daryn via bobby) (Revision 1406689) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406689 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java > Standardize creation of SaslRpcClients > -- > > Key: HADOOP-9014 > URL: https://issues.apache.org/jira/browse/HADOOP-9014 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Fix For: 3.0.0, 2.0.3-alpha > > Attachments: HADOOP-9014.patch > > > To ease adding additional SASL support, need to change the chained > conditionals into a switch and make one standard call to createSaslClient. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9014) Standardize creation of SaslRpcClients
[ https://issues.apache.org/jira/browse/HADOOP-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated HADOOP-9014: Resolution: Fixed Fix Version/s: 2.0.3-alpha 3.0.0 Status: Resolved (was: Patch Available) Thanks Daryn, I put this in trunk and branch-2 > Standardize creation of SaslRpcClients > -- > > Key: HADOOP-9014 > URL: https://issues.apache.org/jira/browse/HADOOP-9014 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Fix For: 3.0.0, 2.0.3-alpha > > Attachments: HADOOP-9014.patch > > > To ease adding additional SASL support, need to change the chained > conditionals into a switch and make one standard call to createSaslClient. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9013) UGI should not hardcode loginUser's authenticationType
[
https://issues.apache.org/jira/browse/HADOOP-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492453#comment-13492453
]
Hudson commented on HADOOP-9013:
Integrated in Hadoop-trunk-Commit #2971 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/2971/])
HADOOP-9013. UGI should not hardcode loginUser's authenticationType (daryn
via bobby) (Revision 1406684)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406684
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java
> UGI should not hardcode loginUser's authenticationType
> --
>
> Key: HADOOP-9013
> URL: https://issues.apache.org/jira/browse/HADOOP-9013
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, security
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: HADOOP-9013.patch
>
>
> {{UGI.loginUser}} assumes that the user's auth type for security on =
> kerberos, security off = simple. It should instead use the configured auth
> type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9014) Standardize creation of SaslRpcClients
[ https://issues.apache.org/jira/browse/HADOOP-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492452#comment-13492452 ] Robert Joseph Evans commented on HADOOP-9014: - This change looks good +1. > Standardize creation of SaslRpcClients > -- > > Key: HADOOP-9014 > URL: https://issues.apache.org/jira/browse/HADOOP-9014 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 0.23.0, 3.0.0, 2.0.3-alpha >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Attachments: HADOOP-9014.patch > > > To ease adding additional SASL support, need to change the chained > conditionals into a switch and make one standard call to createSaslClient. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9013) UGI should not hardcode loginUser's authenticationType
[
https://issues.apache.org/jira/browse/HADOOP-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Joseph Evans updated HADOOP-9013:
Resolution: Fixed
Fix Version/s: 2.0.3-alpha
3.0.0
Status: Resolved (was: Patch Available)
Thanks Daryn,
I put this into trunk and branch-2
> UGI should not hardcode loginUser's authenticationType
> --
>
> Key: HADOOP-9013
> URL: https://issues.apache.org/jira/browse/HADOOP-9013
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, security
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
> Fix For: 3.0.0, 2.0.3-alpha
>
> Attachments: HADOOP-9013.patch
>
>
> {{UGI.loginUser}} assumes that the user's auth type for security on =
> kerberos, security off = simple. It should instead use the configured auth
> type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9013) UGI should not hardcode loginUser's authenticationType
[
https://issues.apache.org/jira/browse/HADOOP-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492443#comment-13492443
]
Robert Joseph Evans commented on HADOOP-9013:
-
The change looks good to me too. +1
> UGI should not hardcode loginUser's authenticationType
> --
>
> Key: HADOOP-9013
> URL: https://issues.apache.org/jira/browse/HADOOP-9013
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, security
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
> Attachments: HADOOP-9013.patch
>
>
> {{UGI.loginUser}} assumes that the user's auth type for security on =
> kerberos, security off = simple. It should instead use the configured auth
> type.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8963) CopyFromLocal doesn't always create user directory
[ https://issues.apache.org/jira/browse/HADOOP-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492403#comment-13492403 ] Daryn Sharp commented on HADOOP-8963: - +1 > CopyFromLocal doesn't always create user directory > -- > > Key: HADOOP-8963 > URL: https://issues.apache.org/jira/browse/HADOOP-8963 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.0.3 >Reporter: Billie Rinaldi >Assignee: Arpit Gupta >Priority: Trivial > Attachments: HADOOP-8963.branch-1.patch, HADOOP-8963.branch-1.patch, > HADOOP-8963.branch-1.patch, HADOOP-8963.branch-1.patch > > > When you use the command "hadoop fs -copyFromLocal filename ." before the > /user/username directory has been created, the file is created with name > /user/username instead of a directory being created with file > /user/username/filename. The command "hadoop fs -copyFromLocal filename > filename" works as expected, creating /user/username and > /user/username/filename, and "hadoop fs -copyFromLocal filename ." works as > expected if the /user/username directory already exists. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9012) IPC Client sends wrong connection context
[ https://issues.apache.org/jira/browse/HADOOP-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492360#comment-13492360 ] Hudson commented on HADOOP-9012: Integrated in Hadoop-Mapreduce-trunk #1249 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1249/]) HADOOP-9012. IPC Client sends wrong connection context (daryn via bobby) (Revision 1406184) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406184 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java > IPC Client sends wrong connection context > - > > Key: HADOOP-9012 > URL: https://issues.apache.org/jira/browse/HADOOP-9012 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 3.0.0, 2.0.3-alpha >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Fix For: 3.0.0, 2.0.3-alpha > > Attachments: HADOOP-9012.patch > > > The IPC client will send the wrong connection context when asked to switch to > simple auth. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9004) Allow security unit tests to use external KDC
[ https://issues.apache.org/jira/browse/HADOOP-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492359#comment-13492359 ] Hudson commented on HADOOP-9004: Integrated in Hadoop-Mapreduce-trunk #1249 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1249/]) HADOOP-9004. Allow security unit tests to use external KDC. Contributed by Stephen Chu. (Revision 1406413) HADOOP-9004. Reverting the commit r1406202 to address patch issue (Revision 1406379) HADOOP-9004. Allow security unit tests to use external KDC. Contributed by Stephen Chu. (Revision 1406202) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406413 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/SecurityUtilTestHelper.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithExternalKdc.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStartSecureDataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNodeWithExternalKdc.java suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406379 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/SecurityUtilTestHelper.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithExternalKdc.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStartSecureDataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNodeWithExternalKdc.java suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406202 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/SecurityUtilTestHelper.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithExternalKdc.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStartSecureDataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNodeWithExternalKdc.java > Allow security unit tests to use external KDC > - > > Key: HADOOP-9004 > URL: https://issues.apache.org/jira/browse/HADOOP-9004 > Project: Hadoop Common > Issue Type: Improvement > Components: security, test >Affects Versions: 2.0.0-alpha >Reporter: Stephen Chu >Assignee: Stephen Chu > Fix For: 3.0.0 > > Attachments: HADOOP-9004.patch, HADOOP-9004.patch, > HADOOP-9004.patch.007, HADOOP-9004.patch.008 > > > I want to add the option of allowing security-related unit tests to use an > external KDC. > In HADOOP-8078, we add the ability to start and use an ApacheDS KDC for > security-related unit tests. It would be good to allow users to validate the > use of their own KDC, keytabs, and principals and to test different KDCs and > not rely on the ApacheDS KDC. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9012) IPC Client sends wrong connection context
[ https://issues.apache.org/jira/browse/HADOOP-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492318#comment-13492318 ] Hudson commented on HADOOP-9012: Integrated in Hadoop-Hdfs-trunk #1219 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1219/]) HADOOP-9012. IPC Client sends wrong connection context (daryn via bobby) (Revision 1406184) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406184 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java > IPC Client sends wrong connection context > - > > Key: HADOOP-9012 > URL: https://issues.apache.org/jira/browse/HADOOP-9012 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 3.0.0, 2.0.3-alpha >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Fix For: 3.0.0, 2.0.3-alpha > > Attachments: HADOOP-9012.patch > > > The IPC client will send the wrong connection context when asked to switch to > simple auth. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9004) Allow security unit tests to use external KDC
[ https://issues.apache.org/jira/browse/HADOOP-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492317#comment-13492317 ] Hudson commented on HADOOP-9004: Integrated in Hadoop-Hdfs-trunk #1219 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1219/]) HADOOP-9004. Allow security unit tests to use external KDC. Contributed by Stephen Chu. (Revision 1406413) HADOOP-9004. Reverting the commit r1406202 to address patch issue (Revision 1406379) HADOOP-9004. Allow security unit tests to use external KDC. Contributed by Stephen Chu. (Revision 1406202) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406413 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/SecurityUtilTestHelper.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithExternalKdc.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStartSecureDataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNodeWithExternalKdc.java suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406379 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/SecurityUtilTestHelper.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithExternalKdc.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStartSecureDataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNodeWithExternalKdc.java suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406202 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/SecurityUtilTestHelper.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithExternalKdc.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStartSecureDataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNodeWithExternalKdc.java > Allow security unit tests to use external KDC > - > > Key: HADOOP-9004 > URL: https://issues.apache.org/jira/browse/HADOOP-9004 > Project: Hadoop Common > Issue Type: Improvement > Components: security, test >Affects Versions: 2.0.0-alpha >Reporter: Stephen Chu >Assignee: Stephen Chu > Fix For: 3.0.0 > > Attachments: HADOOP-9004.patch, HADOOP-9004.patch, > HADOOP-9004.patch.007, HADOOP-9004.patch.008 > > > I want to add the option of allowing security-related unit tests to use an > external KDC. > In HADOOP-8078, we add the ability to start and use an ApacheDS KDC for > security-related unit tests. It would be good to allow users to validate the > use of their own KDC, keytabs, and principals and to test different KDCs and > not rely on the ApacheDS KDC. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8427) Convert Forrest docs to APT
[
https://issues.apache.org/jira/browse/HADOOP-8427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492311#comment-13492311
]
Tom White commented on HADOOP-8427:
---
Andy, this looks good so far. I generated the site with {{mvn site; mvn
site:stage -DstagingDirectory=/tmp/hadoop-site}} and the converted files looked
OK. The navigation is not wired up yet though - my patch in HADOOP-8860 sets up
the nav correctly so it would help if committed that one first. Does that patch
look OK to you?
Going through the other files in Common that have not been migrated,
cluster_setup.xml and single_node_setup.xml are already in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt so can be left,
but deployment_layout.xml, index.xml, native_libraries.xml,
service_level_auth.xml, and Superusers.xml should all be migrated. They will
need to be reviewed and updated, but this can be done separately as Nicholas
suggested.
> Convert Forrest docs to APT
> ---
>
> Key: HADOOP-8427
> URL: https://issues.apache.org/jira/browse/HADOOP-8427
> Project: Hadoop Common
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.0.0-alpha
>Reporter: Eli Collins
>Assignee: Andy Isaacson
> Labels: newbie
> Attachments: hadoop8427-1.txt, hadoop8427-3.txt, hadoop8427.txt
>
>
> Some of the forrest docs content in src/docs/src/documentation/content/xdocs
> has not yet been converted to APT and moved to src/site/apt. Let's convert
> the forrest docs that haven't been converted yet to new APT content in
> hadoop-common/src/site/apt (and link the new content into
> hadoop-project/src/site/apt/index.apt.vm) and remove all forrest dependencies.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9004) Allow security unit tests to use external KDC
[ https://issues.apache.org/jira/browse/HADOOP-9004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492256#comment-13492256 ] Hudson commented on HADOOP-9004: Integrated in Hadoop-Yarn-trunk #29 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/29/]) HADOOP-9004. Allow security unit tests to use external KDC. Contributed by Stephen Chu. (Revision 1406413) HADOOP-9004. Reverting the commit r1406202 to address patch issue (Revision 1406379) HADOOP-9004. Allow security unit tests to use external KDC. Contributed by Stephen Chu. (Revision 1406202) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406413 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/SecurityUtilTestHelper.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithExternalKdc.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStartSecureDataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNodeWithExternalKdc.java suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406379 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/SecurityUtilTestHelper.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithExternalKdc.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStartSecureDataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNodeWithExternalKdc.java suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406202 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/SecurityUtilTestHelper.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUGIWithExternalKdc.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStartSecureDataNode.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNodeWithExternalKdc.java > Allow security unit tests to use external KDC > - > > Key: HADOOP-9004 > URL: https://issues.apache.org/jira/browse/HADOOP-9004 > Project: Hadoop Common > Issue Type: Improvement > Components: security, test >Affects Versions: 2.0.0-alpha >Reporter: Stephen Chu >Assignee: Stephen Chu > Fix For: 3.0.0 > > Attachments: HADOOP-9004.patch, HADOOP-9004.patch, > HADOOP-9004.patch.007, HADOOP-9004.patch.008 > > > I want to add the option of allowing security-related unit tests to use an > external KDC. > In HADOOP-8078, we add the ability to start and use an ApacheDS KDC for > security-related unit tests. It would be good to allow users to validate the > use of their own KDC, keytabs, and principals and to test different KDCs and > not rely on the ApacheDS KDC. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9012) IPC Client sends wrong connection context
[ https://issues.apache.org/jira/browse/HADOOP-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492257#comment-13492257 ] Hudson commented on HADOOP-9012: Integrated in Hadoop-Yarn-trunk #29 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/29/]) HADOOP-9012. IPC Client sends wrong connection context (daryn via bobby) (Revision 1406184) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1406184 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestSaslRPC.java > IPC Client sends wrong connection context > - > > Key: HADOOP-9012 > URL: https://issues.apache.org/jira/browse/HADOOP-9012 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc >Affects Versions: 3.0.0, 2.0.3-alpha >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Fix For: 3.0.0, 2.0.3-alpha > > Attachments: HADOOP-9012.patch > > > The IPC client will send the wrong connection context when asked to switch to > simple auth. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9016) Provide unit tests for class org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream
Ivan A. Veselovsky created HADOOP-9016: -- Summary: Provide unit tests for class org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream Key: HADOOP-9016 URL: https://issues.apache.org/jira/browse/HADOOP-9016 Project: Hadoop Common Issue Type: Improvement Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Priority: Minor unit-test coverage of classes org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream, org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream.HarFsInputStream is zero. Suggested to provide unit-tests covering these classes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
