[jira] [Updated] (HADOOP-9523) Provide a generic IBM java vendor flag in PlatformName.java to support non-Sun JREs
[ https://issues.apache.org/jira/browse/HADOOP-9523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-9523: Resolution: Fixed Fix Version/s: 2.0.5-beta Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch to trunk and branch-2. Thank you Tian. Thanks to Aaron for the reviews. > Provide a generic IBM java vendor flag in PlatformName.java to support > non-Sun JREs > --- > > Key: HADOOP-9523 > URL: https://issues.apache.org/jira/browse/HADOOP-9523 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.0.4-alpha >Reporter: Tian Hong Wang >Assignee: Tian Hong Wang > Labels: patch > Fix For: 2.0.5-beta > > Attachments: HADOOP-9523.patch, HADOOP-9523-v1.patch, > HADOOP-9523-v2.patch > > > There are several different points between Sun jdk & IBM jdk, so there is a > need to provide a generic IBM java vendor flag. So enhance PlatformName.java > to add Java vendor information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9523) Provide a generic IBM java vendor flag in PlatformName.java to support non-Sun JREs
[ https://issues.apache.org/jira/browse/HADOOP-9523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648157#comment-13648157 ] Tian Hong Wang commented on HADOOP-9523: Thanks Suresh for your review. > Provide a generic IBM java vendor flag in PlatformName.java to support > non-Sun JREs > --- > > Key: HADOOP-9523 > URL: https://issues.apache.org/jira/browse/HADOOP-9523 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.0.4-alpha >Reporter: Tian Hong Wang >Assignee: Tian Hong Wang > Labels: patch > Attachments: HADOOP-9523.patch, HADOOP-9523-v1.patch, > HADOOP-9523-v2.patch > > > There are several different points between Sun jdk & IBM jdk, so there is a > need to provide a generic IBM java vendor flag. So enhance PlatformName.java > to add Java vendor information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9523) Provide a generic IBM java vendor flag in PlatformName.java to support non-Sun JREs
[ https://issues.apache.org/jira/browse/HADOOP-9523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648151#comment-13648151 ] Suresh Srinivas commented on HADOOP-9523: - Tian, sorry I was not following this jira closely. +1 for the patch. > Provide a generic IBM java vendor flag in PlatformName.java to support > non-Sun JREs > --- > > Key: HADOOP-9523 > URL: https://issues.apache.org/jira/browse/HADOOP-9523 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.0.4-alpha >Reporter: Tian Hong Wang >Assignee: Tian Hong Wang > Labels: patch > Attachments: HADOOP-9523.patch, HADOOP-9523-v1.patch, > HADOOP-9523-v2.patch > > > There are several different points between Sun jdk & IBM jdk, so there is a > need to provide a generic IBM java vendor flag. So enhance PlatformName.java > to add Java vendor information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9523) Provide a generic IBM java vendor flag in PlatformName.java to support non-Sun JREs
[ https://issues.apache.org/jira/browse/HADOOP-9523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648136#comment-13648136 ] Tian Hong Wang commented on HADOOP-9523: Suresh, Aaron, when will community include this patch? Subsequent patches based on this patch will be given to well support IBM JRE. > Provide a generic IBM java vendor flag in PlatformName.java to support > non-Sun JREs > --- > > Key: HADOOP-9523 > URL: https://issues.apache.org/jira/browse/HADOOP-9523 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 2.0.4-alpha >Reporter: Tian Hong Wang >Assignee: Tian Hong Wang > Labels: patch > Attachments: HADOOP-9523.patch, HADOOP-9523-v1.patch, > HADOOP-9523-v2.patch > > > There are several different points between Sun jdk & IBM jdk, so there is a > need to provide a generic IBM java vendor flag. So enhance PlatformName.java > to add Java vendor information. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9527) TestLocalFSFileContextSymlink is broken on Windows
[ https://issues.apache.org/jira/browse/HADOOP-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648131#comment-13648131 ] Arpit Agarwal commented on HADOOP-9527: --- Thanks for the pointer Ivan. I'll take a look. > TestLocalFSFileContextSymlink is broken on Windows > -- > > Key: HADOOP-9527 > URL: https://issues.apache.org/jira/browse/HADOOP-9527 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 3.0.0 >Reporter: Arpit Agarwal > Fix For: 3.0.0 > > Attachments: RenameLink.java > > > Multiple test cases are broken. I didn't look at each failure in detail. > The main cause of the failures appears to be that RawLocalFS.readLink() does > not work on Windows. We need "winutils readlink" to fix the test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9527) TestLocalFSFileContextSymlink is broken on Windows
[ https://issues.apache.org/jira/browse/HADOOP-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648129#comment-13648129 ] Ivan Mitic commented on HADOOP-9527: Hi Arpit, the following Jira should help with the above JDK6 VS 7 problem you described: HADOOP-9061 > TestLocalFSFileContextSymlink is broken on Windows > -- > > Key: HADOOP-9527 > URL: https://issues.apache.org/jira/browse/HADOOP-9527 > Project: Hadoop Common > Issue Type: Bug > Components: test >Affects Versions: 3.0.0 >Reporter: Arpit Agarwal > Fix For: 3.0.0 > > Attachments: RenameLink.java > > > Multiple test cases are broken. I didn't look at each failure in detail. > The main cause of the failures appears to be that RawLocalFS.readLink() does > not work on Windows. We need "winutils readlink" to fix the test. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9392) Token based authentication and Single Sign On
[ https://issues.apache.org/jira/browse/HADOOP-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648125#comment-13648125 ] Larry McCay commented on HADOOP-9392: - Thank you for the design doc, Kai. Interesting read. I will be organizing my thoughts and questions over the next couple days and will hopefully have some thoughts for you in a day or two. I think it may make sense to try and concentrate on an area at a time with my comments and discussion. Thanks again. > Token based authentication and Single Sign On > - > > Key: HADOOP-9392 > URL: https://issues.apache.org/jira/browse/HADOOP-9392 > Project: Hadoop Common > Issue Type: New Feature > Components: security >Reporter: Kai Zheng > Fix For: 3.0.0 > > Attachments: token-based-authn-plus-sso.pdf > > > This is an umbrella entry for one of project Rhino’s topic, for details of > project Rhino, please refer to > https://github.com/intel-hadoop/project-rhino/. The major goal for this entry > as described in project Rhino was > > “Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication > at the RPC layer, via SASL. However this does not provide valuable attributes > such as group membership, classification level, organizational identity, or > support for user defined attributes. Hadoop components must interrogate > external resources for discovering these attributes and at scale this is > problematic. There is also no consistent delegation model. HDFS has a simple > delegation capability, and only Oozie can take limited advantage of it. We > will implement a common token based authentication framework to decouple > internal user and service authentication from external mechanisms used to > support it (like Kerberos)” > > We’d like to start our work from Hadoop-Common and try to provide common > facilities by extending existing authentication framework which support: > 1.Pluggable token provider interface > 2.Pluggable token verification protocol and interface > 3.Security mechanism to distribute secrets in cluster nodes > 4.Delegation model of user authentication -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9535) HSSO Server - IDP/SP Authentication Endpoint Deployment
[ https://issues.apache.org/jira/browse/HADOOP-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648122#comment-13648122 ] Larry McCay commented on HADOOP-9535: - Hi Daryn - This subtask is primarily about being able to compose and expose token endpoints for authentication or federation of users' credentials or tokens from IDPs. The resulting hadoop common token could then be used as an AuthenticationToken in the hadoop.auth cookie. If that were to be the case then - yes - it would eliminate the need for a shared secret there. The BASIC->LDAP Authentication Provider is just an example of one such endpoint. I hope that answers your question. > HSSO Server - IDP/SP Authentication Endpoint Deployment > --- > > Key: HADOOP-9535 > URL: https://issues.apache.org/jira/browse/HADOOP-9535 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Reporter: Larry McCay > Original Estimate: 504h > Remaining Estimate: 504h > > This effort will result in an embedded Jetty based server that deploys > endpoints for the authentication or federation of entities. > It will leverage common facilities for authentication and federation > providers within servlet filters which allow composability of various > provider types to satisfy various token and authentication processing > requirements. > For instance: > In order for a client application to authenticate using BASIC credentials and > receive a token that will allow access to HDFS we may need to provide a > couple an endpoint comprised of an authentication provider and a token > generation provider: > 1. BASIC->LDAP Authentication Provider > 2. Access Token Generation Provider > This allows a user to authenticate to the HSSO service endpoint with simple > username/password using HTTP BASIC and leverage a simple username bind to an > LDAP server for authentication. It then uses the resulting normalized > java.security.Subject to generate an access token for the user to pass along > with subsequent requests to other Hadoop services. > Hadoop services only need to verify the token validity and trust of the > issuer - HSSO service - in order to authenticate access to its protected > resources. > This task is dependent on common authentication provider frameworks and will > need to insure compatibility and the composability described in this task. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9527) TestLocalFSFileContextSymlink is broken on Windows
[
https://issues.apache.org/jira/browse/HADOOP-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HADOOP-9527:
--
Attachment: RenameLink.java
This test is exposing a Java 1.6 bug on Windows when trying to rename a
symlink. Attached a minimal repro to demonstrate the JVM bug.
Here is the output on Windows using Java 1.7 and 1.6. 1.7 renames the symlink
as expected, while 1.6 renames the target of the symlink.
{code}
C:\tmp\test>dir
Directory of C:\tmp\test
05/02/2013 06:51 PM 7 file.txt
05/02/2013 06:51 PM link1 [file.txt]
C:\tmp\test>c:\progra~1\java\jdk1.7.0_09\bin\java RenameLink link1 link2
C:\tmp\test>dir
Directory of C:\tmp\test
05/02/2013 06:51 PM 7 file.txt
05/02/2013 06:51 PM link2 [file.txt]
C:\tmp\test>c:\progra~1\java\jdk1.6.0_35\bin\java RenameLink link2 link3
C:\tmp\test>dir
Directory of C:\tmp\test
05/02/2013 06:51 PM link2 [file.txt]
05/02/2013 06:51 PM 7 link3
{code}
> TestLocalFSFileContextSymlink is broken on Windows
> --
>
> Key: HADOOP-9527
> URL: https://issues.apache.org/jira/browse/HADOOP-9527
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
>Affects Versions: 3.0.0
>Reporter: Arpit Agarwal
> Fix For: 3.0.0
>
> Attachments: RenameLink.java
>
>
> Multiple test cases are broken. I didn't look at each failure in detail.
> The main cause of the failures appears to be that RawLocalFS.readLink() does
> not work on Windows. We need "winutils readlink" to fix the test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9425) Add error codes to rpc-response
[ https://issues.apache.org/jira/browse/HADOOP-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HADOOP-9425: --- Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change) +1 patch looks good. > Add error codes to rpc-response > --- > > Key: HADOOP-9425 > URL: https://issues.apache.org/jira/browse/HADOOP-9425 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sanjay Radia >Assignee: Sanjay Radia > Attachments: HADOOP-9425-1.patch, HADOOP-9425-2.patch, > HADOOP-9425-3.patch, HADOOP-9425-4.patch, HADOOP-9425-5.patch, > HADOOP-9425-6.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648066#comment-13648066 ] Arpit Agarwal commented on HADOOP-9483: --- Thanks Suresh. > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Fix For: 3.0.0 > > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.007.patch, HADOOP-9483.patch, > HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648009#comment-13648009 ] Suresh Srinivas commented on HADOOP-9483: - I committed the patch to trunk. > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Fix For: 3.0.0 > > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.007.patch, HADOOP-9483.patch, > HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9485) inconsistent defaults for hadoop.rpc.socket.factory.class.default
[
https://issues.apache.org/jira/browse/HADOOP-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HADOOP-9485:
-
Attachment: HADOOP-9485.006.patch
> inconsistent defaults for hadoop.rpc.socket.factory.class.default
> -
>
> Key: HADOOP-9485
> URL: https://issues.apache.org/jira/browse/HADOOP-9485
> Project: Hadoop Common
> Issue Type: Bug
> Components: net
>Affects Versions: 2.0.5-beta
>Reporter: Colin Patrick McCabe
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HADOOP-9485.001.patch, HADOOP-9485.002.patch,
> HADOOP-9485.003.patch, HADOOP-9485.004.patch, HADOOP-9485.005.patch,
> HADOOP-9485.006.patch
>
>
> In {{core-default.xml}}, {{hadoop.rpc.socket.factory.class.default}} defaults
> to {{org.apache.hadoop.net.StandardSocketFactory}}. However, in
> {{CommonConfigurationKeysPublic.java}}, there is no default for this key.
> This is inconsistent (defaults in the code versus defaults in the XML files
> should match.) It also leads to problems with {{RemoteBlockReader2}}, since
> the default {{SocketFactory}} creates a {{Socket}} without an associated
> channel. {{RemoteBlockReader2}} cannot use such a {{Socket}}.
> This bug only really becomes apparent when you create a {{Configuration}}
> using the {{Configuration(loadDefaults=true)}} constructor. Thanks to AB
> Srinivasan for his help in discovering this bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-9483: Resolution: Fixed Fix Version/s: 3.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1 for the patch. Thank you Arpit. Thanks to Chris and Ivan for the reviews. > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Fix For: 3.0.0 > > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.007.patch, HADOOP-9483.patch, > HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9043) disallow in winutils creating symlinks with forwards slashes
[ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647989#comment-13647989 ] Chris Nauroth commented on HADOOP-9043: --- Thank you for the commit, Suresh. Thanks also to Arpit for writing some of the code, and to all of the reviewers. > disallow in winutils creating symlinks with forwards slashes > > > Key: HADOOP-9043 > URL: https://issues.apache.org/jira/browse/HADOOP-9043 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0, 1-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 1-win > > Attachments: HADOOP-9043.branch-1.2.patch, > HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.6.patch, > HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch, > HADOOP-9043.trunk.3.patch, HADOOP-9043.trunk.4.patch, > HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.6.patch, HADOOP-9043.trunk.patch > > > In general, the winutils symlink command rejects attempts to create symlinks > targeting a destination file that does not exist. However, if given a > symlink destination with forward slashes pointing at a file that does exist, > then it creates the symlink with the forward slashes, and then attempts to > open the file through the symlink will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9511) Adding support for additional input streams (FSDataInputStream and RandomAccessFile) in SecureIOUtils.
[ https://issues.apache.org/jira/browse/HADOOP-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated HADOOP-9511: Status: Open (was: Patch Available) Apologies for going back and forth on this one. - May be we should just pass in a File to openFSDataInputStream? - The isExists() check is unnecesary? As the subsequent stat will anyways validate that, right? - Similarly for forceSecureOpenFSDataInputStream. > Adding support for additional input streams (FSDataInputStream and > RandomAccessFile) in SecureIOUtils. > -- > > Key: HADOOP-9511 > URL: https://issues.apache.org/jira/browse/HADOOP-9511 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Omkar Vinit Joshi >Assignee: Omkar Vinit Joshi > Attachments: HADOOP-9511.20130426.patch, HADOOP-9511.20130429.patch, > HADOOP-9511.20130430.1.patch, HADOOP-9511.20130430.patch > > > At present we can only get secured InputStream. However we want other types > of input streams to avoid possible security attacks. > YARN-578 needs this fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647978#comment-13647978 ] Chris Nauroth commented on HADOOP-9483: --- +1 for the rebased patch too. Thanks again! > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.007.patch, HADOOP-9483.patch, > HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Attachment: HADOOP-9483.007.patch Rebased patch. > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.007.patch, HADOOP-9483.patch, > HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9043) disallow in winutils creating symlinks with forwards slashes
[ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-9043: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patches to trunk and branch-1-win. Thank you Chris. > disallow in winutils creating symlinks with forwards slashes > > > Key: HADOOP-9043 > URL: https://issues.apache.org/jira/browse/HADOOP-9043 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0, 1-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 1-win > > Attachments: HADOOP-9043.branch-1.2.patch, > HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.6.patch, > HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch, > HADOOP-9043.trunk.3.patch, HADOOP-9043.trunk.4.patch, > HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.6.patch, HADOOP-9043.trunk.patch > > > In general, the winutils symlink command rejects attempts to create symlinks > targeting a destination file that does not exist. However, if given a > symlink destination with forward slashes pointing at a file that does exist, > then it creates the symlink with the forward slashes, and then attempts to > open the file through the symlink will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9043) disallow in winutils creating symlinks with forwards slashes
[ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-9043: Summary: disallow in winutils creating symlinks with forwards slashes (was: disallow winutils creating symlinks with forwards slashes) > disallow in winutils creating symlinks with forwards slashes > > > Key: HADOOP-9043 > URL: https://issues.apache.org/jira/browse/HADOOP-9043 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0, 1-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 1-win > > Attachments: HADOOP-9043.branch-1.2.patch, > HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.6.patch, > HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch, > HADOOP-9043.trunk.3.patch, HADOOP-9043.trunk.4.patch, > HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.6.patch, HADOOP-9043.trunk.patch > > > In general, the winutils symlink command rejects attempts to create symlinks > targeting a destination file that does not exist. However, if given a > symlink destination with forward slashes pointing at a file that does exist, > then it creates the symlink with the forward slashes, and then attempts to > open the file through the symlink will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9043) disallow winutils creating symlinks with forwards slashes
[ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-9043: -- Summary: disallow winutils creating symlinks with forwards slashes (was: winutils can create unusable symlinks) > disallow winutils creating symlinks with forwards slashes > - > > Key: HADOOP-9043 > URL: https://issues.apache.org/jira/browse/HADOOP-9043 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0, 1-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 1-win > > Attachments: HADOOP-9043.branch-1.2.patch, > HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.6.patch, > HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch, > HADOOP-9043.trunk.3.patch, HADOOP-9043.trunk.4.patch, > HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.6.patch, HADOOP-9043.trunk.patch > > > In general, the winutils symlink command rejects attempts to create symlinks > targeting a destination file that does not exist. However, if given a > symlink destination with forward slashes pointing at a file that does exist, > then it creates the symlink with the forward slashes, and then attempts to > open the file through the symlink will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9438) LocalFileContext does not throw an exception on mkdir for already existing directory
[
https://issues.apache.org/jira/browse/HADOOP-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647919#comment-13647919
]
Ivan Mitic commented on HADOOP-9438:
I run into this Jira and thought it might be worth to comment.
Per the FileSystem contract, mkdirs should succeed if the directory already
exist, right? Existing FileSystemContract tests validate this functionality.
Is this something that should be handled by the FileContext, or possibly a
documentation bug?
> LocalFileContext does not throw an exception on mkdir for already existing
> directory
>
>
> Key: HADOOP-9438
> URL: https://issues.apache.org/jira/browse/HADOOP-9438
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 2.0.3-alpha
>Reporter: Robert Joseph Evans
>Priority: Critical
> Attachments: HADOOP-9438.20130501.1.patch, HADOOP-9438.patch,
> HADOOP-9438.patch
>
>
> according to
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileContext.html#mkdir%28org.apache.hadoop.fs.Path,%20org.apache.hadoop.fs.permission.FsPermission,%20boolean%29
> should throw a FileAlreadyExistsException if the directory already exists.
> I tested this and
> {code}
> FileContext lfc = FileContext.getLocalFSFileContext(new Configuration());
> Path p = new Path("/tmp/bobby.12345");
> FsPermission cachePerms = new FsPermission((short) 0755);
> lfc.mkdir(p, cachePerms, false);
> lfc.mkdir(p, cachePerms, false);
> {code}
> never throws an exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647920#comment-13647920 ] Arpit Agarwal commented on HADOOP-9483: --- Thanks for the reviewing, Ivan and Chris. Chris - I noticed that and I tried to restructure the patch but the conflict seems unavoidable so I'll leave it as it is and rebase later. Committers, please consider committing HADOOP-9043 first. > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9043) winutils can create unusable symlinks
[ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647918#comment-13647918 ] Suresh Srinivas commented on HADOOP-9043: - +1 for the patch. > winutils can create unusable symlinks > - > > Key: HADOOP-9043 > URL: https://issues.apache.org/jira/browse/HADOOP-9043 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0, 1-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 1-win > > Attachments: HADOOP-9043.branch-1.2.patch, > HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.6.patch, > HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch, > HADOOP-9043.trunk.3.patch, HADOOP-9043.trunk.4.patch, > HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.6.patch, HADOOP-9043.trunk.patch > > > In general, the winutils symlink command rejects attempts to create symlinks > targeting a destination file that does not exist. However, if given a > symlink destination with forward slashes pointing at a file that does exist, > then it creates the symlink with the forward slashes, and then attempts to > open the file through the symlink will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9483) winutils support for readlink command
[
https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647917#comment-13647917
]
Chris Nauroth commented on HADOOP-9483:
---
+1 for the patch. Looks good!
One minor thing: my latest patch on HADOOP-9043 puts the {{assumeTrue}} call
into a {{@Before}} method so that the individual tests don't need to check it.
Whichever of these patches gets committed first, the other one can rebase
easily.
> winutils support for readlink command
> -
>
> Key: HADOOP-9483
> URL: https://issues.apache.org/jira/browse/HADOOP-9483
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Arpit Agarwal
> Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch,
> HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch
>
>
> The current codebase relies on the Unix readlink command to determine the
> target of a symlink on the local file system. winutils currently does not
> support this functionality on Windows. Adding the command to winutils will
> prevent the need to use GnuWin32 or Cygwin for readlink support.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9425) Add error codes to rpc-response
[ https://issues.apache.org/jira/browse/HADOOP-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay Radia updated HADOOP-9425: - Attachment: HADOOP-9425-6.patch > Add error codes to rpc-response > --- > > Key: HADOOP-9425 > URL: https://issues.apache.org/jira/browse/HADOOP-9425 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Sanjay Radia >Assignee: Sanjay Radia > Attachments: HADOOP-9425-1.patch, HADOOP-9425-2.patch, > HADOOP-9425-3.patch, HADOOP-9425-4.patch, HADOOP-9425-5.patch, > HADOOP-9425-6.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647900#comment-13647900 ] Ivan Mitic commented on HADOOP-9483: Latest patch looks good, +1 I verified that TestWinUtils passes on Windows. Thanks Arpit! > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9543) TestFsShellReturnCode may fail in branch-1
[ https://issues.apache.org/jira/browse/HADOOP-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647891#comment-13647891 ] Suresh Srinivas commented on HADOOP-9543: - +1 for the patch. > TestFsShellReturnCode may fail in branch-1 > -- > > Key: HADOOP-9543 > URL: https://issues.apache.org/jira/browse/HADOOP-9543 > Project: Hadoop Common > Issue Type: Bug > Components: test >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE >Priority: Minor > Fix For: 1.2.0 > > Attachments: c9543_20130502.patch > > > There is a hardcoded username "admin" in TestFsShellReturnCode. If "admin" > does not exist in the local fs, the test may fail. Before HADOOP-9502, the > failure of the command is ignored silently, i.e. the command returns success > even if it indeed failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9543) TestFsShellReturnCode may fail in branch-1
[ https://issues.apache.org/jira/browse/HADOOP-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE resolved HADOOP-9543. Resolution: Fixed Fix Version/s: 1.2.0 Hadoop Flags: Reviewed Thanks Arpit for reviewing the patch. I have committed this. > TestFsShellReturnCode may fail in branch-1 > -- > > Key: HADOOP-9543 > URL: https://issues.apache.org/jira/browse/HADOOP-9543 > Project: Hadoop Common > Issue Type: Bug > Components: test >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE >Priority: Minor > Fix For: 1.2.0 > > Attachments: c9543_20130502.patch > > > There is a hardcoded username "admin" in TestFsShellReturnCode. If "admin" > does not exist in the local fs, the test may fail. Before HADOOP-9502, the > failure of the command is ignored silently, i.e. the command returns success > even if it indeed failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9543) TestFsShellReturnCode may fail in branch-1
[ https://issues.apache.org/jira/browse/HADOOP-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647879#comment-13647879 ] Tsz Wo (Nicholas), SZE commented on HADOOP-9543: Manually tested TestFsShellReturnCode. It passed in my machine after the patch. > TestFsShellReturnCode may fail in branch-1 > -- > > Key: HADOOP-9543 > URL: https://issues.apache.org/jira/browse/HADOOP-9543 > Project: Hadoop Common > Issue Type: Bug > Components: test >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE >Priority: Minor > Attachments: c9543_20130502.patch > > > There is a hardcoded username "admin" in TestFsShellReturnCode. If "admin" > does not exist in the local fs, the test may fail. Before HADOOP-9502, the > failure of the command is ignored silently, i.e. the command returns success > even if it indeed failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Status: Open (was: Patch Available) > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Attachment: (was: HADOOP-9483.006.patch) > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Status: Patch Available (was: Open) > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9543) TestFsShellReturnCode may fail in branch-1
[ https://issues.apache.org/jira/browse/HADOOP-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647833#comment-13647833 ] Arpit Agarwal commented on HADOOP-9543: --- +1 > TestFsShellReturnCode may fail in branch-1 > -- > > Key: HADOOP-9543 > URL: https://issues.apache.org/jira/browse/HADOOP-9543 > Project: Hadoop Common > Issue Type: Bug > Components: test >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE >Priority: Minor > Attachments: c9543_20130502.patch > > > There is a hardcoded username "admin" in TestFsShellReturnCode. If "admin" > does not exist in the local fs, the test may fail. Before HADOOP-9502, the > failure of the command is ignored silently, i.e. the command returns success > even if it indeed failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9543) TestFsShellReturnCode may fail in branch-1
[ https://issues.apache.org/jira/browse/HADOOP-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HADOOP-9543: --- Attachment: c9543_20130502.patch c9543_20130502.patch: uses current username > TestFsShellReturnCode may fail in branch-1 > -- > > Key: HADOOP-9543 > URL: https://issues.apache.org/jira/browse/HADOOP-9543 > Project: Hadoop Common > Issue Type: Bug > Components: test >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE >Priority: Minor > Attachments: c9543_20130502.patch > > > There is a hardcoded username "admin" in TestFsShellReturnCode. If "admin" > does not exist in the local fs, the test may fail. Before HADOOP-9502, the > failure of the command is ignored silently, i.e. the command returns success > even if it indeed failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Moved] (HADOOP-9543) TestFsShellReturnCode may fail in branch-1
[ https://issues.apache.org/jira/browse/HADOOP-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE moved HDFS-4790 to HADOOP-9543: -- Component/s: (was: test) test Key: HADOOP-9543 (was: HDFS-4790) Project: Hadoop Common (was: Hadoop HDFS) > TestFsShellReturnCode may fail in branch-1 > -- > > Key: HADOOP-9543 > URL: https://issues.apache.org/jira/browse/HADOOP-9543 > Project: Hadoop Common > Issue Type: Bug > Components: test >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE >Priority: Minor > > There is a hardcoded username "admin" in TestFsShellReturnCode. If "admin" > does not exist in the local fs, the test may fail. Before HADOOP-9502, the > failure of the command is ignored silently, i.e. the command returns success > even if it indeed failed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Attachment: HADOOP-9483.006.patch Make TestWinUtils changes consistent with HADOOP-9043 to keep the rebase trivial. > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.006.patch, HADOOP-9483.patch, > HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9043) winutils can create unusable symlinks
[ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647786#comment-13647786 ] Arpit Agarwal commented on HADOOP-9043: --- Thanks for addressing that. +1 Verified tests pass on Windows and are skipped on OS X. > winutils can create unusable symlinks > - > > Key: HADOOP-9043 > URL: https://issues.apache.org/jira/browse/HADOOP-9043 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0, 1-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 1-win > > Attachments: HADOOP-9043.branch-1.2.patch, > HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.6.patch, > HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch, > HADOOP-9043.trunk.3.patch, HADOOP-9043.trunk.4.patch, > HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.6.patch, HADOOP-9043.trunk.patch > > > In general, the winutils symlink command rejects attempts to create symlinks > targeting a destination file that does not exist. However, if given a > symlink destination with forward slashes pointing at a file that does exist, > then it creates the symlink with the forward slashes, and then attempts to > open the file through the symlink will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9043) winutils can create unusable symlinks
[ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-9043: -- Status: Patch Available (was: Open) > winutils can create unusable symlinks > - > > Key: HADOOP-9043 > URL: https://issues.apache.org/jira/browse/HADOOP-9043 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0, 1-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 1-win > > Attachments: HADOOP-9043.branch-1.2.patch, > HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.6.patch, > HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch, > HADOOP-9043.trunk.3.patch, HADOOP-9043.trunk.4.patch, > HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.6.patch, HADOOP-9043.trunk.patch > > > In general, the winutils symlink command rejects attempts to create symlinks > targeting a destination file that does not exist. However, if given a > symlink destination with forward slashes pointing at a file that does exist, > then it creates the symlink with the forward slashes, and then attempts to > open the file through the symlink will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9043) winutils can create unusable symlinks
[
https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-9043:
--
Attachment: HADOOP-9043.trunk.6.patch
Good catch, thanks Arpit! I've attached version 6 of the patch for both trunk
and branch-1-win. This switches all tests in {{TestWinUtils}} to use
{{assumeTrue}} for the Windows check.
> winutils can create unusable symlinks
> -
>
> Key: HADOOP-9043
> URL: https://issues.apache.org/jira/browse/HADOOP-9043
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 3.0.0, 1-win
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 3.0.0, 1-win
>
> Attachments: HADOOP-9043.branch-1.2.patch,
> HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.6.patch,
> HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch,
> HADOOP-9043.trunk.3.patch, HADOOP-9043.trunk.4.patch,
> HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.6.patch, HADOOP-9043.trunk.patch
>
>
> In general, the winutils symlink command rejects attempts to create symlinks
> targeting a destination file that does not exist. However, if given a
> symlink destination with forward slashes pointing at a file that does exist,
> then it creates the symlink with the forward slashes, and then attempts to
> open the file through the symlink will fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9043) winutils can create unusable symlinks
[ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-9043: -- Attachment: HADOOP-9043.branch-1-win.6.patch > winutils can create unusable symlinks > - > > Key: HADOOP-9043 > URL: https://issues.apache.org/jira/browse/HADOOP-9043 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0, 1-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 1-win > > Attachments: HADOOP-9043.branch-1.2.patch, > HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.6.patch, > HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch, > HADOOP-9043.trunk.3.patch, HADOOP-9043.trunk.4.patch, > HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.patch > > > In general, the winutils symlink command rejects attempts to create symlinks > targeting a destination file that does not exist. However, if given a > symlink destination with forward slashes pointing at a file that does exist, > then it creates the symlink with the forward slashes, and then attempts to > open the file through the symlink will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9043) winutils can create unusable symlinks
[ https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HADOOP-9043: -- Status: Open (was: Patch Available) > winutils can create unusable symlinks > - > > Key: HADOOP-9043 > URL: https://issues.apache.org/jira/browse/HADOOP-9043 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 3.0.0, 1-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 3.0.0, 1-win > > Attachments: HADOOP-9043.branch-1.2.patch, > HADOOP-9043.branch-1-win.5.patch, HADOOP-9043.branch-1-win.6.patch, > HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch, > HADOOP-9043.trunk.3.patch, HADOOP-9043.trunk.4.patch, > HADOOP-9043.trunk.5.patch, HADOOP-9043.trunk.patch > > > In general, the winutils symlink command rejects attempts to create symlinks > targeting a destination file that does not exist. However, if given a > symlink destination with forward slashes pointing at a file that does exist, > then it creates the symlink with the forward slashes, and then attempts to > open the file through the symlink will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9485) inconsistent defaults for hadoop.rpc.socket.factory.class.default
[
https://issues.apache.org/jira/browse/HADOOP-9485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647762#comment-13647762
]
Colin Patrick McCabe commented on HADOOP-9485:
--
I think the issue here is that running without XML defaults works in branch-1,
and when upgrading to branch-2 or a distribution based on that, people view
this kind of regression as a stumbling block.
bq. Why is it risky to change the default in the code to match the default from
the XML file?
Currently, you can set {{hadoop.rpc.socket.factory.class.default = null}} and
we'll use {{SocketFactory.getDefault()}}. If we change this behavior we may
break users who are relying on it. It certainly breaks {{TestSaslRPC}}. It
would be an incompatible change that would probably cause problems for users.
I suppose we could make the code default to the same thing as the XML file, but
not change the meaning of setting the key to null. That might work.
> inconsistent defaults for hadoop.rpc.socket.factory.class.default
> -
>
> Key: HADOOP-9485
> URL: https://issues.apache.org/jira/browse/HADOOP-9485
> Project: Hadoop Common
> Issue Type: Bug
> Components: net
>Affects Versions: 2.0.5-beta
>Reporter: Colin Patrick McCabe
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HADOOP-9485.001.patch, HADOOP-9485.002.patch,
> HADOOP-9485.003.patch, HADOOP-9485.004.patch, HADOOP-9485.005.patch
>
>
> In {{core-default.xml}}, {{hadoop.rpc.socket.factory.class.default}} defaults
> to {{org.apache.hadoop.net.StandardSocketFactory}}. However, in
> {{CommonConfigurationKeysPublic.java}}, there is no default for this key.
> This is inconsistent (defaults in the code versus defaults in the XML files
> should match.) It also leads to problems with {{RemoteBlockReader2}}, since
> the default {{SocketFactory}} creates a {{Socket}} without an associated
> channel. {{RemoteBlockReader2}} cannot use such a {{Socket}}.
> This bug only really becomes apparent when you create a {{Configuration}}
> using the {{Configuration(loadDefaults=true)}} constructor. Thanks to AB
> Srinivasan for his help in discovering this bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9542) Define REST specification for JMX servelets
Karthik Kambatla created HADOOP-9542: Summary: Define REST specification for JMX servelets Key: HADOOP-9542 URL: https://issues.apache.org/jira/browse/HADOOP-9542 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.4-alpha Reporter: Karthik Kambatla In the process of defining compatibility on HADOOP-9517, we realized conf servlets don't have a specification. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9273) optimize ivy calls using macrodefs
[ https://issues.apache.org/jira/browse/HADOOP-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated HADOOP-9273: --- Status: Patch Available (was: Open) > optimize ivy calls using macrodefs > -- > > Key: HADOOP-9273 > URL: https://issues.apache.org/jira/browse/HADOOP-9273 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Giridharan Kesavan >Assignee: Giridharan Kesavan > Attachments: hadoop-9273.patch > > > use ant macrodefs for mvn-install mvn-deploy and ivy-retrieve, ivy-resolve -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9541) Define REST specification for conf servlets
Karthik Kambatla created HADOOP-9541: Summary: Define REST specification for conf servlets Key: HADOOP-9541 URL: https://issues.apache.org/jira/browse/HADOOP-9541 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.4-alpha Reporter: Karthik Kambatla In the process of defining compatibility on HADOOP-9517, we realized conf servlets don't have a specification. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9517) Define Hadoop Compatibility
[ https://issues.apache.org/jira/browse/HADOOP-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated HADOOP-9517: - Attachment: hadoop-9517.patch Thanks Steve and Nicholas. Uploading a new patch to incorporate your comments: # Added section on semantics compatibility reflects Steve's comments on the mailing list and this JIRA. # Updated REST APIs section: (1) separate out WebHDFS and HttpFS, (2) add links to all REST APIs as in the site, (3) remove JMX and conf from the list as they don't have a specification. I guess we should create JIRAs to define specification of conf and JMX servlets. > Define Hadoop Compatibility > --- > > Key: HADOOP-9517 > URL: https://issues.apache.org/jira/browse/HADOOP-9517 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Arun C Murthy >Assignee: Karthik Kambatla > Attachments: hadoop-9517.patch, hadoop-9517.patch > > > As we get ready to call hadoop-2 stable we need to better define 'Hadoop > Compatibility'. > http://wiki.apache.org/hadoop/Compatibility is a start, let's document > requirements clearly and completely. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9392) Token based authentication and Single Sign On
[ https://issues.apache.org/jira/browse/HADOOP-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HADOOP-9392: -- Attachment: token-based-authn-plus-sso.pdf Here is the first version of our architecture and design doc for review. Comments are welcome. > Token based authentication and Single Sign On > - > > Key: HADOOP-9392 > URL: https://issues.apache.org/jira/browse/HADOOP-9392 > Project: Hadoop Common > Issue Type: New Feature > Components: security >Reporter: Kai Zheng > Fix For: 3.0.0 > > Attachments: token-based-authn-plus-sso.pdf > > > This is an umbrella entry for one of project Rhino’s topic, for details of > project Rhino, please refer to > https://github.com/intel-hadoop/project-rhino/. The major goal for this entry > as described in project Rhino was > > “Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication > at the RPC layer, via SASL. However this does not provide valuable attributes > such as group membership, classification level, organizational identity, or > support for user defined attributes. Hadoop components must interrogate > external resources for discovering these attributes and at scale this is > problematic. There is also no consistent delegation model. HDFS has a simple > delegation capability, and only Oozie can take limited advantage of it. We > will implement a common token based authentication framework to decouple > internal user and service authentication from external mechanisms used to > support it (like Kerberos)” > > We’d like to start our work from Hadoop-Common and try to provide common > facilities by extending existing authentication framework which support: > 1.Pluggable token provider interface > 2.Pluggable token verification protocol and interface > 3.Security mechanism to distribute secrets in cluster nodes > 4.Delegation model of user authentication -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9517) Define Hadoop Compatibility
[ https://issues.apache.org/jira/browse/HADOOP-9517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647683#comment-13647683 ] Steve Loughran commented on HADOOP-9517: The closest thing we do have to strict specifications are the tests, perhaps that could be used to declare stability "stable with respect to the current test suite" That makes clear that changing the test suite is a way of changing the specification: more tests = tighter spec. But for all those bits that aren't tightly specified, that's all we can say "works-for-our-tests" > Define Hadoop Compatibility > --- > > Key: HADOOP-9517 > URL: https://issues.apache.org/jira/browse/HADOOP-9517 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Arun C Murthy >Assignee: Karthik Kambatla > Attachments: hadoop-9517.patch > > > As we get ready to call hadoop-2 stable we need to better define 'Hadoop > Compatibility'. > http://wiki.apache.org/hadoop/Compatibility is a start, let's document > requirements clearly and completely. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8711) provide an option for IPC server users to avoid printing stack information for certain exceptions
[ https://issues.apache.org/jira/browse/HADOOP-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-8711: Fix Version/s: (was: 3.0.0) 1.2.0 I committed the patch to branch-1 and branch-1.2. > provide an option for IPC server users to avoid printing stack information > for certain exceptions > - > > Key: HADOOP-8711 > URL: https://issues.apache.org/jira/browse/HADOOP-8711 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Affects Versions: 3.0.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 1.2.0, 0.23.7, 2.0.5-beta > > Attachments: HADOOP-8711.branch-1.patch.2, HADOOP-8711.patch, > HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, > HADOOP-8871.branch-1.patch > > > Currently it's hard coded in the server that it doesn't print the exception > stack for StandbyException. > Similarly, other components may have their own exceptions which don't need to > save the stack trace in log. One example is HDFS-3817. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9521) krb5 replay error triggers log file DoS with Safari
[ https://issues.apache.org/jira/browse/HADOOP-9521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647675#comment-13647675 ] Allen Wittenauer commented on HADOOP-9521: -- You are focusing too much on the replay error. If you look at YARN-621 (which was already referenced prior to your Kerberos 101 questioning in the links section), I can make the replay error go away by launching a job. This is purely a "web filter does bad things with a particular browser" bug. If you'd like to help debug the actual replay error, this is not the JIRA for that. > krb5 replay error triggers log file DoS with Safari > --- > > Key: HADOOP-9521 > URL: https://issues.apache.org/jira/browse/HADOOP-9521 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.0.4-alpha > Environment: Mac OS X 10.8.3, Safari 6.0.3 (8536.28.10) > Mac OS X 10.6.8, Safari 6.0.3 (8536.28.10) >Reporter: Allen Wittenauer >Priority: Blocker > > While investigating YARN-621, looking at the web interface with Safari > triggered a loop which both filled the log with stack traces as well as left > the browser in a continual loading situation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9540) Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit testing.
[ https://issues.apache.org/jira/browse/HADOOP-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647674#comment-13647674 ] Steve Loughran commented on HADOOP-9540: the change seems reasonable, but as these are for testing only, they should go into the src/test tree. People can still pick them up with an import of hadoop-test JAR, but it removes all confusion as to whether these are things you'd use in production. I'm not sure what tests are needed here to round things out. Anything? > Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit > testing. > > > Key: HADOOP-9540 > URL: https://issues.apache.org/jira/browse/HADOOP-9540 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3, test >Affects Versions: trunk-win >Reporter: Hariprasad Kuppuswamy >Priority: Minor > Attachments: HADOOP-9540.patch > > > The stub implementations available for InMemoryFileSytemStores for S3 and S3N > are currently restricted in package scope. These are quiet handy utilities > for unit testing and nice to be exposed in a public scope. > Or even conveniently I have added simple wrapper InMemoryFileSystem > implementations for these stores so that it can be easily leveraged by any > interested developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8711) provide an option for IPC server users to avoid printing stack information for certain exceptions
[ https://issues.apache.org/jira/browse/HADOOP-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647661#comment-13647661 ] Suresh Srinivas commented on HADOOP-8711: - +1 for the branch-1 patch. Thanks Brandon. > provide an option for IPC server users to avoid printing stack information > for certain exceptions > - > > Key: HADOOP-8711 > URL: https://issues.apache.org/jira/browse/HADOOP-8711 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Affects Versions: 3.0.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 3.0.0, 0.23.7, 2.0.5-beta > > Attachments: HADOOP-8711.branch-1.patch.2, HADOOP-8711.patch, > HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, > HADOOP-8871.branch-1.patch > > > Currently it's hard coded in the server that it doesn't print the exception > stack for StandbyException. > Similarly, other components may have their own exceptions which don't need to > save the stack trace in log. One example is HDFS-3817. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9483) winutils support for readlink command
[ https://issues.apache.org/jira/browse/HADOOP-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HADOOP-9483: -- Attachment: HADOOP-9483.005.patch Good catch, fixed! > winutils support for readlink command > - > > Key: HADOOP-9483 > URL: https://issues.apache.org/jira/browse/HADOOP-9483 > Project: Hadoop Common > Issue Type: Improvement > Components: util >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Arpit Agarwal > Attachments: HADOOP-9483.003.patch, HADOOP-9483.004.patch, > HADOOP-9483.005.patch, HADOOP-9483.patch, HADOOP-9483.patch > > > The current codebase relies on the Unix readlink command to determine the > target of a symlink on the local file system. winutils currently does not > support this functionality on Windows. Adding the command to winutils will > prevent the need to use GnuWin32 or Cygwin for readlink support. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9535) HSSO Server - IDP/SP Authentication Endpoint Deployment
[ https://issues.apache.org/jira/browse/HADOOP-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Larry McCay updated HADOOP-9535: Description: This effort will result in an embedded Jetty based server that deploys endpoints for the authentication or federation of entities. It will leverage common facilities for authentication and federation providers within servlet filters which allow composability of various provider types to satisfy various token and authentication processing requirements. For instance: In order for a client application to authenticate using BASIC credentials and receive a token that will allow access to HDFS we may need to provide a couple an endpoint comprised of an authentication provider and a token generation provider: 1. BASIC->LDAP Authentication Provider 2. Access Token Generation Provider This allows a user to authenticate to the HSSO service endpoint with simple username/password using HTTP BASIC and leverage a simple username bind to an LDAP server for authentication. It then uses the resulting normalized java.security.Subject to generate an access token for the user to pass along with subsequent requests to other Hadoop services. Hadoop services only need to verify the token validity and trust of the issuer - HSSO service - in order to authenticate access to its protected resources. This task is dependent on common authentication provider frameworks and will need to insure compatibility and the composability described in this task. was: This effort will result in an embedded Jetty based server that deploys endpoints for the authentication or federation of entities. It will leverage common facilities for authentication and federation providers within servlet filters which allow composability of various provider types to satisfy various token and authentication processing requirements. For instance: In order for a client application to authenticate using BASIC credentials and receive a token that will allow access to HDFS we may need to provide a couple an endpoint comprised of an authentication provider and a token generation provider: 1. BASIC->LDAP Authentication Provider 2. Access Token Generation Provider This allows a user to authentication to the HSSO service endpoint with simple username/password using HTTP BASIC and leverage a simple username bind to an LDAP server for authentication. It then uses the resulting normalized java.security.Subject to generate an access token for the user to pass along with subsequent requests to other Hadoop services. Hadoop services only need to verify the token validity and trust of the issuer - HSSO service - in order to authenticate access to its protected resources. This task is dependent on common authentication provider frameworks and will need to insure compatibility and the composability described in this task. > HSSO Server - IDP/SP Authentication Endpoint Deployment > --- > > Key: HADOOP-9535 > URL: https://issues.apache.org/jira/browse/HADOOP-9535 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Reporter: Larry McCay > Original Estimate: 504h > Remaining Estimate: 504h > > This effort will result in an embedded Jetty based server that deploys > endpoints for the authentication or federation of entities. > It will leverage common facilities for authentication and federation > providers within servlet filters which allow composability of various > provider types to satisfy various token and authentication processing > requirements. > For instance: > In order for a client application to authenticate using BASIC credentials and > receive a token that will allow access to HDFS we may need to provide a > couple an endpoint comprised of an authentication provider and a token > generation provider: > 1. BASIC->LDAP Authentication Provider > 2. Access Token Generation Provider > This allows a user to authenticate to the HSSO service endpoint with simple > username/password using HTTP BASIC and leverage a simple username bind to an > LDAP server for authentication. It then uses the resulting normalized > java.security.Subject to generate an access token for the user to pass along > with subsequent requests to other Hadoop services. > Hadoop services only need to verify the token validity and trust of the > issuer - HSSO service - in order to authenticate access to its protected > resources. > This task is dependent on common authentication provider frameworks and will > need to insure compatibility and the composability described in this task. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8711) provide an option for IPC server users to avoid printing stack information for certain exceptions
[ https://issues.apache.org/jira/browse/HADOOP-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647625#comment-13647625 ] Brandon Li commented on HADOOP-8711: Thanks for the reminder. The patch is split into two. A branch-1 patch is uploaded in HDFS-3817. One is uploaded for this JIRA. This JIRA's new branch-1 patch includes a unit test now. > provide an option for IPC server users to avoid printing stack information > for certain exceptions > - > > Key: HADOOP-8711 > URL: https://issues.apache.org/jira/browse/HADOOP-8711 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Affects Versions: 3.0.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 3.0.0, 0.23.7, 2.0.5-beta > > Attachments: HADOOP-8711.branch-1.patch.2, HADOOP-8711.patch, > HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, > HADOOP-8871.branch-1.patch > > > Currently it's hard coded in the server that it doesn't print the exception > stack for StandbyException. > Similarly, other components may have their own exceptions which don't need to > save the stack trace in log. One example is HDFS-3817. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8711) provide an option for IPC server users to avoid printing stack information for certain exceptions
[ https://issues.apache.org/jira/browse/HADOOP-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HADOOP-8711: --- Attachment: HADOOP-8711.branch-1.patch.2 > provide an option for IPC server users to avoid printing stack information > for certain exceptions > - > > Key: HADOOP-8711 > URL: https://issues.apache.org/jira/browse/HADOOP-8711 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Affects Versions: 3.0.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 3.0.0, 0.23.7, 2.0.5-beta > > Attachments: HADOOP-8711.branch-1.patch.2, HADOOP-8711.patch, > HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, > HADOOP-8871.branch-1.patch > > > Currently it's hard coded in the server that it doesn't print the exception > stack for StandbyException. > Similarly, other components may have their own exceptions which don't need to > save the stack trace in log. One example is HDFS-3817. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9532) HADOOP_CLIENT_OPTS is appended twice by Windows cmd scripts
[ https://issues.apache.org/jira/browse/HADOOP-9532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647549#comment-13647549 ] Hudson commented on HADOOP-9532: Integrated in Hadoop-Mapreduce-trunk #1417 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1417/]) HADOOP-9532. HADOOP_CLIENT_OPTS is appended twice by Windows cmd scripts. Contributed by Chris Nauroth. (Revision 1478072) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1478072 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.cmd > HADOOP_CLIENT_OPTS is appended twice by Windows cmd scripts > --- > > Key: HADOOP-9532 > URL: https://issues.apache.org/jira/browse/HADOOP-9532 > Project: Hadoop Common > Issue Type: Bug > Components: bin >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Fix For: 3.0.0 > > Attachments: HADOOP-9532.1.patch > > > This problem was reported initially for the shell scripts in HADOOP-9455. > This issue tracks the same problem for the Windows cmd scripts. Appending > HADOOP_CIENT_OPTS twice can cause an incorrect JVM launch, particularly if > trying to set remote debugging flags. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9455) HADOOP_CLIENT_OPTS appended twice causes JVM failures
[
https://issues.apache.org/jira/browse/HADOOP-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647547#comment-13647547
]
Hudson commented on HADOOP-9455:
Integrated in Hadoop-Mapreduce-trunk #1417 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1417/])
HADOOP-9455. HADOOP_CLIENT_OPTS appended twice causes JVM failures.
Contributed by Chris Nauroth. (Revision 1478067)
Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1478067
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
> HADOOP_CLIENT_OPTS appended twice causes JVM failures
> -
>
> Key: HADOOP-9455
> URL: https://issues.apache.org/jira/browse/HADOOP-9455
> Project: Hadoop Common
> Issue Type: Bug
> Components: bin
>Affects Versions: 3.0.0, 2.0.3-alpha
>Reporter: Sangjin Lee
>Assignee: Chris Nauroth
>Priority: Minor
> Fix For: 2.0.5-beta
>
> Attachments: HADOOP-9455.1.patch, HADOOP-9455.2.patch,
> HADOOP-9455.3.patch, HADOOP-9455-branch-2.patch
>
>
> If you set HADOOP_CLIENT_OPTS and run hadoop, you'll find that the
> HADOOP_CLIENT_OPTS value gets appended twice, and leads to JVM start failures
> for cases like adding debug flags.
> For example,
> {noformat}
> HADOOP_CLIENT_OPTS='-agentlib:jdwp=transport=dt_socket,address=localhost:9009,server=y,suspend=y'
> hadoop jar anything
> ERROR: Cannot load this JVM TI agent twice, check your java command line for
> duplicate jdwp options.
> Error occurred during initialization of VM
> agent library failed to init: jdwp
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9535) HSSO Server - IDP/SP Authentication Endpoint Deployment
[ https://issues.apache.org/jira/browse/HADOOP-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647538#comment-13647538 ] Daryn Sharp commented on HADOOP-9535: - Is this more or less a replacement for obtaining the shared secret for webhdfs auth cookies? Pluggable authentication in UGI for ldap and the existing SSO functionality seem to generally provide the required functionality? > HSSO Server - IDP/SP Authentication Endpoint Deployment > --- > > Key: HADOOP-9535 > URL: https://issues.apache.org/jira/browse/HADOOP-9535 > Project: Hadoop Common > Issue Type: Sub-task > Components: security >Reporter: Larry McCay > Original Estimate: 504h > Remaining Estimate: 504h > > This effort will result in an embedded Jetty based server that deploys > endpoints for the authentication or federation of entities. > It will leverage common facilities for authentication and federation > providers within servlet filters which allow composability of various > provider types to satisfy various token and authentication processing > requirements. > For instance: > In order for a client application to authenticate using BASIC credentials and > receive a token that will allow access to HDFS we may need to provide a > couple an endpoint comprised of an authentication provider and a token > generation provider: > 1. BASIC->LDAP Authentication Provider > 2. Access Token Generation Provider > This allows a user to authentication to the HSSO service endpoint with simple > username/password using HTTP BASIC and leverage a simple username bind to an > LDAP server for authentication. It then uses the resulting normalized > java.security.Subject to generate an access token for the user to pass along > with subsequent requests to other Hadoop services. > Hadoop services only need to verify the token validity and trust of the > issuer - HSSO service - in order to authenticate access to its protected > resources. > This task is dependent on common authentication provider frameworks and will > need to insure compatibility and the composability described in this task. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9521) krb5 replay error triggers log file DoS with Safari
[ https://issues.apache.org/jira/browse/HADOOP-9521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647530#comment-13647530 ] Daryn Sharp commented on HADOOP-9521: - Are all the hosts involved ntp synced? Kerberos can be sensitive to clock skew. Presumably linux and/or windows doesn't have the replay issue in your environment? > krb5 replay error triggers log file DoS with Safari > --- > > Key: HADOOP-9521 > URL: https://issues.apache.org/jira/browse/HADOOP-9521 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.0.4-alpha > Environment: Mac OS X 10.8.3, Safari 6.0.3 (8536.28.10) > Mac OS X 10.6.8, Safari 6.0.3 (8536.28.10) >Reporter: Allen Wittenauer >Priority: Blocker > > While investigating YARN-621, looking at the web interface with Safari > triggered a loop which both filled the log with stack traces as well as left > the browser in a continual loading situation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9540) Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit testing.
[
https://issues.apache.org/jira/browse/HADOOP-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647510#comment-13647510
]
Hadoop QA commented on HADOOP-9540:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12581517/HADOOP-9540.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2508//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2508//console
This message is automatically generated.
> Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit
> testing.
>
>
> Key: HADOOP-9540
> URL: https://issues.apache.org/jira/browse/HADOOP-9540
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3, test
>Affects Versions: trunk-win
>Reporter: Hariprasad Kuppuswamy
>Priority: Minor
> Attachments: HADOOP-9540.patch
>
>
> The stub implementations available for InMemoryFileSytemStores for S3 and S3N
> are currently restricted in package scope. These are quiet handy utilities
> for unit testing and nice to be exposed in a public scope.
> Or even conveniently I have added simple wrapper InMemoryFileSystem
> implementations for these stores so that it can be easily leveraged by any
> interested developers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9540) Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit testing.
[ https://issues.apache.org/jira/browse/HADOOP-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hariprasad Kuppuswamy updated HADOOP-9540: -- Attachment: HADOOP-9540.patch > Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit > testing. > > > Key: HADOOP-9540 > URL: https://issues.apache.org/jira/browse/HADOOP-9540 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3, test >Affects Versions: trunk-win >Reporter: Hariprasad Kuppuswamy >Priority: Minor > Attachments: HADOOP-9540.patch > > > The stub implementations available for InMemoryFileSytemStores for S3 and S3N > are currently restricted in package scope. These are quiet handy utilities > for unit testing and nice to be exposed in a public scope. > Or even conveniently I have added simple wrapper InMemoryFileSystem > implementations for these stores so that it can be easily leveraged by any > interested developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9540) Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit testing.
[ https://issues.apache.org/jira/browse/HADOOP-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hariprasad Kuppuswamy updated HADOOP-9540: -- Status: Patch Available (was: Open) > Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit > testing. > > > Key: HADOOP-9540 > URL: https://issues.apache.org/jira/browse/HADOOP-9540 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3, test >Affects Versions: trunk-win >Reporter: Hariprasad Kuppuswamy >Priority: Minor > > The stub implementations available for InMemoryFileSytemStores for S3 and S3N > are currently restricted in package scope. These are quiet handy utilities > for unit testing and nice to be exposed in a public scope. > Or even conveniently I have added simple wrapper InMemoryFileSystem > implementations for these stores so that it can be easily leveraged by any > interested developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9540) Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit testing.
Hariprasad Kuppuswamy created HADOOP-9540: - Summary: Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit testing. Key: HADOOP-9540 URL: https://issues.apache.org/jira/browse/HADOOP-9540 Project: Hadoop Common Issue Type: Improvement Components: fs/s3, test Affects Versions: trunk-win Reporter: Hariprasad Kuppuswamy Priority: Minor The stub implementations available for InMemoryFileSytemStores for S3 and S3N are currently restricted in package scope. These are quiet handy utilities for unit testing and nice to be exposed in a public scope. Or even conveniently I have added simple wrapper InMemoryFileSystem implementations for these stores so that it can be easily leveraged by any interested developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9532) HADOOP_CLIENT_OPTS is appended twice by Windows cmd scripts
[ https://issues.apache.org/jira/browse/HADOOP-9532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647494#comment-13647494 ] Hudson commented on HADOOP-9532: Integrated in Hadoop-Hdfs-trunk #1390 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1390/]) HADOOP-9532. HADOOP_CLIENT_OPTS is appended twice by Windows cmd scripts. Contributed by Chris Nauroth. (Revision 1478072) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1478072 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.cmd > HADOOP_CLIENT_OPTS is appended twice by Windows cmd scripts > --- > > Key: HADOOP-9532 > URL: https://issues.apache.org/jira/browse/HADOOP-9532 > Project: Hadoop Common > Issue Type: Bug > Components: bin >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Fix For: 3.0.0 > > Attachments: HADOOP-9532.1.patch > > > This problem was reported initially for the shell scripts in HADOOP-9455. > This issue tracks the same problem for the Windows cmd scripts. Appending > HADOOP_CIENT_OPTS twice can cause an incorrect JVM launch, particularly if > trying to set remote debugging flags. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9455) HADOOP_CLIENT_OPTS appended twice causes JVM failures
[
https://issues.apache.org/jira/browse/HADOOP-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647492#comment-13647492
]
Hudson commented on HADOOP-9455:
Integrated in Hadoop-Hdfs-trunk #1390 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1390/])
HADOOP-9455. HADOOP_CLIENT_OPTS appended twice causes JVM failures.
Contributed by Chris Nauroth. (Revision 1478067)
Result = FAILURE
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1478067
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
> HADOOP_CLIENT_OPTS appended twice causes JVM failures
> -
>
> Key: HADOOP-9455
> URL: https://issues.apache.org/jira/browse/HADOOP-9455
> Project: Hadoop Common
> Issue Type: Bug
> Components: bin
>Affects Versions: 3.0.0, 2.0.3-alpha
>Reporter: Sangjin Lee
>Assignee: Chris Nauroth
>Priority: Minor
> Fix For: 2.0.5-beta
>
> Attachments: HADOOP-9455.1.patch, HADOOP-9455.2.patch,
> HADOOP-9455.3.patch, HADOOP-9455-branch-2.patch
>
>
> If you set HADOOP_CLIENT_OPTS and run hadoop, you'll find that the
> HADOOP_CLIENT_OPTS value gets appended twice, and leads to JVM start failures
> for cases like adding debug flags.
> For example,
> {noformat}
> HADOOP_CLIENT_OPTS='-agentlib:jdwp=transport=dt_socket,address=localhost:9009,server=y,suspend=y'
> hadoop jar anything
> ERROR: Cannot load this JVM TI agent twice, check your java command line for
> duplicate jdwp options.
> Error occurred during initialization of VM
> agent library failed to init: jdwp
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9455) HADOOP_CLIENT_OPTS appended twice causes JVM failures
[
https://issues.apache.org/jira/browse/HADOOP-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647428#comment-13647428
]
Hudson commented on HADOOP-9455:
Integrated in Hadoop-Yarn-trunk #201 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/201/])
HADOOP-9455. HADOOP_CLIENT_OPTS appended twice causes JVM failures.
Contributed by Chris Nauroth. (Revision 1478067)
Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1478067
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
> HADOOP_CLIENT_OPTS appended twice causes JVM failures
> -
>
> Key: HADOOP-9455
> URL: https://issues.apache.org/jira/browse/HADOOP-9455
> Project: Hadoop Common
> Issue Type: Bug
> Components: bin
>Affects Versions: 3.0.0, 2.0.3-alpha
>Reporter: Sangjin Lee
>Assignee: Chris Nauroth
>Priority: Minor
> Fix For: 2.0.5-beta
>
> Attachments: HADOOP-9455.1.patch, HADOOP-9455.2.patch,
> HADOOP-9455.3.patch, HADOOP-9455-branch-2.patch
>
>
> If you set HADOOP_CLIENT_OPTS and run hadoop, you'll find that the
> HADOOP_CLIENT_OPTS value gets appended twice, and leads to JVM start failures
> for cases like adding debug flags.
> For example,
> {noformat}
> HADOOP_CLIENT_OPTS='-agentlib:jdwp=transport=dt_socket,address=localhost:9009,server=y,suspend=y'
> hadoop jar anything
> ERROR: Cannot load this JVM TI agent twice, check your java command line for
> duplicate jdwp options.
> Error occurred during initialization of VM
> agent library failed to init: jdwp
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9532) HADOOP_CLIENT_OPTS is appended twice by Windows cmd scripts
[ https://issues.apache.org/jira/browse/HADOOP-9532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647430#comment-13647430 ] Hudson commented on HADOOP-9532: Integrated in Hadoop-Yarn-trunk #201 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/201/]) HADOOP-9532. HADOOP_CLIENT_OPTS is appended twice by Windows cmd scripts. Contributed by Chris Nauroth. (Revision 1478072) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1478072 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.cmd > HADOOP_CLIENT_OPTS is appended twice by Windows cmd scripts > --- > > Key: HADOOP-9532 > URL: https://issues.apache.org/jira/browse/HADOOP-9532 > Project: Hadoop Common > Issue Type: Bug > Components: bin >Affects Versions: 3.0.0 >Reporter: Chris Nauroth >Assignee: Chris Nauroth >Priority: Minor > Fix For: 3.0.0 > > Attachments: HADOOP-9532.1.patch > > > This problem was reported initially for the shell scripts in HADOOP-9455. > This issue tracks the same problem for the Windows cmd scripts. Appending > HADOOP_CIENT_OPTS twice can cause an incorrect JVM launch, particularly if > trying to set remote debugging flags. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8711) provide an option for IPC server users to avoid printing stack information for certain exceptions
[ https://issues.apache.org/jira/browse/HADOOP-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647359#comment-13647359 ] Suresh Srinivas commented on HADOOP-8711: - Brandon can you please split this into two separate patches? > provide an option for IPC server users to avoid printing stack information > for certain exceptions > - > > Key: HADOOP-8711 > URL: https://issues.apache.org/jira/browse/HADOOP-8711 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc >Affects Versions: 3.0.0 >Reporter: Brandon Li >Assignee: Brandon Li > Fix For: 3.0.0, 0.23.7, 2.0.5-beta > > Attachments: HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8711.patch, > HADOOP-8711.patch, HADOOP-8711.patch, HADOOP-8871.branch-1.patch > > > Currently it's hard coded in the server that it doesn't print the exception > stack for StandbyException. > Similarly, other components may have their own exceptions which don't need to > save the stack trace in log. One example is HDFS-3817. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
