[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[
https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669286#comment-13669286
]
Daryn Sharp commented on HADOOP-9421:
-
bq. The main problem with mixing SASL exchange with RPC engine is that's
architecturally and semantically unsound
I differ in opinion. This is our last shot for a long time to make RPC
changes, so we should make it as flexible as possible.
* SASL should ideally be a detail of RPC calls instead of only an initial
connection detail (important for following points)
* Perhaps certain RPC calls shouldn't require SASL, like simple status
requests, but others will
* Server should be able to issue SASL challenges mid-stream, perhaps after a
connection has been open for an extended time, or to allow restricted
super-user operations like chown
* Likewise, server should be able to issue multiple SASL challenges after one
mechanism completes. Perhaps using a certain token requires a subsequent SASL
mechanism.
* Out of band SASL auth (not distinguishable from actual RPC calls) precludes
multiplexing multiple UGIs over a single connection - a concept y! has
internally contemplated.
bq. RPC engine should be constructed based on the IpcSerializationType, which
is now render completely useless
Agreed, but it's already useless. The the RPC request/response header must be
protobuf. The serialization of call/response payload appears to be
configurable, but the server doesn't allow non-protobuf. If different
serializations are ever supported, we can change SASL at that time.
bq. Jamming a session concept like SASL into a RPC call (with a fake call id
(-33) to boot) is an ugly hack that puts a square peg in a round hole.
Agreed about -33, but it's already there on the server-side so I used it to
minimize the client-side change. I originally started by adding a sasl auth to
RpcStatusProto, so I can change it.
bq. TestSaslRpc{Client,Server} should be unit testable
Maybe I missed them, but I didn't see tests for the actual wire data? The
existing SASL tests cover the gamut of the negotiation matrix.
Convert SASL to use ProtoBuf and add lengths for non-blocking processing
Key: HADOOP-9421
URL: https://issues.apache.org/jira/browse/HADOOP-9421
Project: Hadoop Common
Issue Type: Sub-task
Affects Versions: 2.0.3-alpha
Reporter: Sanjay Radia
Assignee: Daryn Sharp
Attachments: HADOOP-9421.patch, HADOOP-9421.patch,
HADOOP-9421-v2-demo.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9594) Update apache commons math dependency
[ https://issues.apache.org/jira/browse/HADOOP-9594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669373#comment-13669373 ] Timothy St. Clair commented on HADOOP-9594: --- Ready for review imho, as it is primarily a simple POM modification. Update apache commons math dependency - Key: HADOOP-9594 URL: https://issues.apache.org/jira/browse/HADOOP-9594 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0, 2.0.5-beta Reporter: Timothy St. Clair Labels: newbie Fix For: 2.0.5-beta Attachments: HADOOP-9594.patch Current dependency is against 2.1 which has been deprecated, and numerous bugz have been fixed since then. See http://commons.apache.org/proper/commons-math/changes-report.html for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669409#comment-13669409 ] Daryn Sharp commented on HADOOP-9421: - Of note, none of the benefits I mentioned is implemented. The negotiation isn't per-call. The current patch immediately initiates negotiation after connection header, and does it through the server's existing fake -33 call. Convert SASL to use ProtoBuf and add lengths for non-blocking processing Key: HADOOP-9421 URL: https://issues.apache.org/jira/browse/HADOOP-9421 Project: Hadoop Common Issue Type: Sub-task Affects Versions: 2.0.3-alpha Reporter: Sanjay Radia Assignee: Daryn Sharp Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7391) Document Interface Classification from HADOOP-5073
[ https://issues.apache.org/jira/browse/HADOOP-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated HADOOP-7391: - Summary: Document Interface Classification from HADOOP-5073 (was: Copy the interface classification documentation of HADOOP-5073 into javadoc) Document Interface Classification from HADOOP-5073 -- Key: HADOOP-7391 URL: https://issues.apache.org/jira/browse/HADOOP-7391 Project: Hadoop Common Issue Type: Bug Reporter: Sanjay Radia Assignee: Sanjay Radia Attachments: hadoop-7391-copy.patch, hadoop-7391-v1.patch, hadoop-7391-v2.patch, Hadoop Interface Taxonomy-3.html, Hadoop Interface Taxonomy.html The documentation for interface classification in Jira Hadoop-5073 was not copied to the Javadoc of the classification. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He updated HADOOP-9574:
Attachment: HADOOP-9574.2.patch
Add tests to ensure new APIs are called, and test for DelegationKey
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9287) Parallel testing hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated HADOOP-9287: Attachment: HADOOP-9287--N7.patch [~cnauroth], thanks for catching the issues with TestCopyPreserveFlag/TestLocalFileSystem and TestTFileNoneCodecsByteArrays. I fixed the 1st case according to your suggestion, increasing timeouts up to 10s. In the 2nd case, it turned out that TestTFileNoneCodecsByteArrays and TestTFileNoneCodecsStreams used the same file for temporary output, so I fixed all descendants of TestTFileStream and TestTFileByteArrays to use class names instead of hard-coded values, for tmp file names. Parallel testing hadoop-common -- Key: HADOOP-9287 URL: https://issues.apache.org/jira/browse/HADOOP-9287 Project: Hadoop Common Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: HADOOP-9287.1.patch, HADOOP-9287--N3.patch, HADOOP-9287--N3.patch, HADOOP-9287--N4.patch, HADOOP-9287--N5.patch, HADOOP-9287--N6.patch, HADOOP-9287--N7.patch, HADOOP-9287.patch, HADOOP-9287.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He updated HADOOP-9574:
Status: Patch Available (was: Open)
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669511#comment-13669511 ] Luke Lu commented on HADOOP-9421: - bq. This is our last shot for a long time to make RPC changes, so we should make it as flexible as possible. Agreed. But your proposal/patch actually makes it less flexible by making SASL depending on RPC engine implementation details. bq. SASL should ideally be a detail of RPC calls instead of only an initial connection detail (important for following points) My latest proposal decoupled the session concept from the underlying connection. By making the session explicit, you can have a more powerful and less leaky abstraction to handle all your use cases in a cleaner fashion. bq. Server should be able to issue SASL challenges mid-stream, perhaps after a connection has been open for an extended time, or to allow restricted super-user operations like chown This can be easily supported by simply close the session (with the connection remain open), and restart the session. bq. Out of band SASL auth (not distinguishable from actual RPC calls) precludes multiplexing multiple UGIs over a single connection - a concept y! has internally contemplated. SASL exchange could be much more expensive than typical RPC calls. Multiplexing multiple UGI over a single connection doesn't require per call SASL, which is an incredibly inefficient way of achieving the goal. A reasonable way to do it would be opening a proxy user session and add an optional user field per call to allow efficient multiple secure impersonation per call over the same session. Again the session concept is general enough to support per call SASL as well by optionally close the session per call. The session concept gives you the flexibility of amortizing the cost of SASL over multiple RPCs. bq. If different serializations are ever supported, we can change SASL at that time. This one of the major flaws of your proposal. We want to be able to support multiple RPC engines at the same time in the future to allow parallel development/deployment of RPC engines. We don't want to implement SASL again for each RPC engine. Convert SASL to use ProtoBuf and add lengths for non-blocking processing Key: HADOOP-9421 URL: https://issues.apache.org/jira/browse/HADOOP-9421 Project: Hadoop Common Issue Type: Sub-task Affects Versions: 2.0.3-alpha Reporter: Sanjay Radia Assignee: Daryn Sharp Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7391) Document Interface Classification from HADOOP-5073
[ https://issues.apache.org/jira/browse/HADOOP-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated HADOOP-7391: - Status: Patch Available (was: Open) Document Interface Classification from HADOOP-5073 -- Key: HADOOP-7391 URL: https://issues.apache.org/jira/browse/HADOOP-7391 Project: Hadoop Common Issue Type: Bug Reporter: Sanjay Radia Assignee: Sanjay Radia Attachments: hadoop-7391-copy.patch, hadoop-7391-v1.patch, hadoop-7391-v2.patch, hadoop-7391-v3.patch, Hadoop Interface Taxonomy-3.html, Hadoop Interface Taxonomy.html The documentation for interface classification in Jira Hadoop-5073 was not copied to the Javadoc of the classification. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7391) Document Interface Classification from HADOOP-5073
[ https://issues.apache.org/jira/browse/HADOOP-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated HADOOP-7391: - Attachment: hadoop-7391-v3.patch Updated patch addresses javadoc. I believe this is ready for review. Document Interface Classification from HADOOP-5073 -- Key: HADOOP-7391 URL: https://issues.apache.org/jira/browse/HADOOP-7391 Project: Hadoop Common Issue Type: Bug Reporter: Sanjay Radia Assignee: Sanjay Radia Attachments: hadoop-7391-copy.patch, hadoop-7391-v1.patch, hadoop-7391-v2.patch, hadoop-7391-v3.patch, Hadoop Interface Taxonomy-3.html, Hadoop Interface Taxonomy.html The documentation for interface classification in Jira Hadoop-5073 was not copied to the Javadoc of the classification. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669543#comment-13669543
]
Hadoop QA commented on HADOOP-9574:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585265/HADOOP-9574.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2577//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2577//console
This message is automatically generated.
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9287) Parallel testing hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669562#comment-13669562 ] Andrey Klochkov commented on HADOOP-9287: - [~jlowe], making YARN tests to run in parallel doesn't seem important today, and I'd say it's better to introduce changes for that when it happens to be needed. Just look at the amount of updates I'm putting here just to keep the patch up to date :-) Parallel testing hadoop-common -- Key: HADOOP-9287 URL: https://issues.apache.org/jira/browse/HADOOP-9287 Project: Hadoop Common Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: HADOOP-9287.1.patch, HADOOP-9287--N3.patch, HADOOP-9287--N3.patch, HADOOP-9287--N4.patch, HADOOP-9287--N5.patch, HADOOP-9287--N6.patch, HADOOP-9287--N7.patch, HADOOP-9287.patch, HADOOP-9287.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7391) Document Interface Classification from HADOOP-5073
[
https://issues.apache.org/jira/browse/HADOOP-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669575#comment-13669575
]
Hadoop QA commented on HADOOP-7391:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585267/hadoop-7391-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2579//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2579//console
This message is automatically generated.
Document Interface Classification from HADOOP-5073
--
Key: HADOOP-7391
URL: https://issues.apache.org/jira/browse/HADOOP-7391
Project: Hadoop Common
Issue Type: Bug
Reporter: Sanjay Radia
Assignee: Sanjay Radia
Attachments: hadoop-7391-copy.patch, hadoop-7391-v1.patch,
hadoop-7391-v2.patch, hadoop-7391-v3.patch, Hadoop Interface Taxonomy-3.html,
Hadoop Interface Taxonomy.html
The documentation for interface classification in Jira Hadoop-5073 was not
copied to the Javadoc
of the classification.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9287) Parallel testing hadoop-common
[
https://issues.apache.org/jira/browse/HADOOP-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669654#comment-13669654
]
Hadoop QA commented on HADOOP-9287:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585264/HADOOP-9287--N7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 58 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs
hadoop-tools/hadoop-distcp.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2578//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2578//console
This message is automatically generated.
Parallel testing hadoop-common
--
Key: HADOOP-9287
URL: https://issues.apache.org/jira/browse/HADOOP-9287
Project: Hadoop Common
Issue Type: Test
Components: test
Affects Versions: 3.0.0
Reporter: Tsuyoshi OZAWA
Assignee: Andrey Klochkov
Attachments: HADOOP-9287.1.patch, HADOOP-9287--N3.patch,
HADOOP-9287--N3.patch, HADOOP-9287--N4.patch, HADOOP-9287--N5.patch,
HADOOP-9287--N6.patch, HADOOP-9287--N7.patch, HADOOP-9287.patch,
HADOOP-9287.patch
The maven surefire plugin supports parallel testing feature. By using it, the
tests can be run more faster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669681#comment-13669681 ] Daryn Sharp commented on HADOOP-9421: - I still think there's confusion here. I'm not proposing per-call SASL. The SASL negotiation sequence is wrapped in a fake callId in response to the connection header. It will simplify the client server and open future possibilities. The session idea is interesting, but it closes the door on the server requiring multiple mechanism - ex. with all the other auth suggestions, maybe if the server auths a token, it will want to demand auth of an identity token or similar. I'm not sure using proxy/super-user authentication is desirable for multiplexing. If I have a process servicing multiple UGIs, I doesn't necessarily want that process to have hdfs root privileges. I want each multiplexed connection to use its specific token. For multiplexing to be feasible and remain async, the server must be able to determine what each received packet is - rpc call or sasl auth call. Convert SASL to use ProtoBuf and add lengths for non-blocking processing Key: HADOOP-9421 URL: https://issues.apache.org/jira/browse/HADOOP-9421 Project: Hadoop Common Issue Type: Sub-task Affects Versions: 2.0.3-alpha Reporter: Sanjay Radia Assignee: Daryn Sharp Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9509) Implement ONCRPC and XDR
[ https://issues.apache.org/jira/browse/HADOOP-9509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HADOOP-9509: --- Attachment: HADOOP-9509.4.patch Uploaded the updated patch with a SimpleUdpClient and some minor error handling added. Implement ONCRPC and XDR Key: HADOOP-9509 URL: https://issues.apache.org/jira/browse/HADOOP-9509 Project: Hadoop Common Issue Type: New Feature Reporter: Brandon Li Assignee: Brandon Li Attachments: HADOOP-9509.2patch, HADOOP-9509.3.patch, HADOOP-9509.4.patch, HADOOP-9509.patch This is to track the implementation of ONCRPC(rfc5531) and XDR(rfc4506). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9509) Implement ONCRPC and XDR
[
https://issues.apache.org/jira/browse/HADOOP-9509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669722#comment-13669722
]
Hadoop QA commented on HADOOP-9509:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585289/HADOOP-9509.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 10 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-nfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2580//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2580//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-nfs.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2580//console
This message is automatically generated.
Implement ONCRPC and XDR
Key: HADOOP-9509
URL: https://issues.apache.org/jira/browse/HADOOP-9509
Project: Hadoop Common
Issue Type: New Feature
Reporter: Brandon Li
Assignee: Brandon Li
Attachments: HADOOP-9509.2patch, HADOOP-9509.3.patch,
HADOOP-9509.4.patch, HADOOP-9509.patch
This is to track the implementation of ONCRPC(rfc5531) and XDR(rfc4506).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669723#comment-13669723 ] Luke Lu commented on HADOOP-9421: - bq. I still think there's confusion here. I'm not proposing per-call SASL I know what you're trying to do. You're overloading the protobuf rpc headers (a specific rpc impl detail) for sasl exchange, which I think is a confusing idea that prohibit clean separation of session/auth and rpc, which is required for multiple rpc engine support. bq. The session idea is interesting, but it closes the door on the server requiring multiple mechanisms. No it doesn't close the door. You can have arbitrary sasl mechanisms to establish an session. bq. If I have a process servicing multiple UGIs, I doesn't necessarily want that process to have hdfs root privileges I think you're confusing proxy user with hdfs root. Proxy user has well defined mechanisms (ACLs) to restrict what a particular user can impersonate, which is already audit logged. Convert SASL to use ProtoBuf and add lengths for non-blocking processing Key: HADOOP-9421 URL: https://issues.apache.org/jira/browse/HADOOP-9421 Project: Hadoop Common Issue Type: Sub-task Affects Versions: 2.0.3-alpha Reporter: Sanjay Radia Assignee: Daryn Sharp Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9605) Update junit dependency
Timothy St. Clair created HADOOP-9605: - Summary: Update junit dependency Key: HADOOP-9605 URL: https://issues.apache.org/jira/browse/HADOOP-9605 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0, 2.0.5-beta Reporter: Timothy St. Clair -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9605) Update junit dependency
[ https://issues.apache.org/jira/browse/HADOOP-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy St. Clair updated HADOOP-9605: -- Description: Simple update of the junit dependency to use newer version. E.g. when running maven-rpmbuild on Fedora 18. Update junit dependency --- Key: HADOOP-9605 URL: https://issues.apache.org/jira/browse/HADOOP-9605 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0, 2.0.5-beta Reporter: Timothy St. Clair Labels: maven Simple update of the junit dependency to use newer version. E.g. when running maven-rpmbuild on Fedora 18. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9605) Update junit dependency
[ https://issues.apache.org/jira/browse/HADOOP-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy St. Clair updated HADOOP-9605: -- Attachment: HADOOP-9605.patch Update JUnit Dep Update junit dependency --- Key: HADOOP-9605 URL: https://issues.apache.org/jira/browse/HADOOP-9605 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0, 2.0.5-beta Reporter: Timothy St. Clair Labels: maven Attachments: HADOOP-9605.patch Simple update of the junit dependency to use newer version. E.g. when running maven-rpmbuild on Fedora 18. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9605) Update junit dependency
[ https://issues.apache.org/jira/browse/HADOOP-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy St. Clair updated HADOOP-9605: -- Status: Patch Available (was: Open) Update junit dependency --- Key: HADOOP-9605 URL: https://issues.apache.org/jira/browse/HADOOP-9605 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0, 2.0.5-beta Reporter: Timothy St. Clair Labels: maven Attachments: HADOOP-9605.patch Simple update of the junit dependency to use newer version. E.g. when running maven-rpmbuild on Fedora 18. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9605) Update junit dependency
[ https://issues.apache.org/jira/browse/HADOOP-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669750#comment-13669750 ] Konstantin Boudnik commented on HADOOP-9605: While pretty tiny, I'd suggest to split the patch in two: - dependency's version bump - and update for the rule to be separated Update junit dependency --- Key: HADOOP-9605 URL: https://issues.apache.org/jira/browse/HADOOP-9605 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0, 2.0.5-beta Reporter: Timothy St. Clair Labels: maven Attachments: HADOOP-9605.patch Simple update of the junit dependency to use newer version. E.g. when running maven-rpmbuild on Fedora 18. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9605) Update junit dependency
[
https://issues.apache.org/jira/browse/HADOOP-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669766#comment-13669766
]
Hadoop QA commented on HADOOP-9605:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585298/HADOOP-9605.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1381 javac
compiler warnings (more than the trunk's current 1367 warnings).
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2581//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2581//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2581//console
This message is automatically generated.
Update junit dependency
---
Key: HADOOP-9605
URL: https://issues.apache.org/jira/browse/HADOOP-9605
Project: Hadoop Common
Issue Type: Improvement
Components: build
Affects Versions: 3.0.0, 2.0.5-beta
Reporter: Timothy St. Clair
Labels: maven
Attachments: HADOOP-9605.patch
Simple update of the junit dependency to use newer version. E.g. when
running maven-rpmbuild on Fedora 18.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9605) Update junit dependency
[ https://issues.apache.org/jira/browse/HADOOP-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy St. Clair updated HADOOP-9605: -- Attachment: HADOOP-9605.patch2 HADOOP-9605.patch1 Patch breakout per Cos's review Update junit dependency --- Key: HADOOP-9605 URL: https://issues.apache.org/jira/browse/HADOOP-9605 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0, 2.0.5-beta Reporter: Timothy St. Clair Labels: maven Attachments: HADOOP-9605.patch1, HADOOP-9605.patch2 Simple update of the junit dependency to use newer version. E.g. when running maven-rpmbuild on Fedora 18. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9605) Update junit dependency
[ https://issues.apache.org/jira/browse/HADOOP-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy St. Clair updated HADOOP-9605: -- Attachment: (was: HADOOP-9605.patch) Update junit dependency --- Key: HADOOP-9605 URL: https://issues.apache.org/jira/browse/HADOOP-9605 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0, 2.0.5-beta Reporter: Timothy St. Clair Labels: maven Attachments: HADOOP-9605.patch1, HADOOP-9605.patch2 Simple update of the junit dependency to use newer version. E.g. when running maven-rpmbuild on Fedora 18. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9606) Protocol buffer support cannot compile under C
Ralph Castain created HADOOP-9606: - Summary: Protocol buffer support cannot compile under C Key: HADOOP-9606 URL: https://issues.apache.org/jira/browse/HADOOP-9606 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.4-alpha Reporter: Ralph Castain When compiling Hadoop's .proto descriptions for use in C, an error occurs because one of the RPC's in NamenodeProtocol.proto is named register. This name is a reserved word in languages such as C. When using the Java and C++ languages, the name is hidden inside a class and therefore doesn't cause an error. Unfortunately, that is not the case in non-class languages such as C. Note: generating the C translation of the .proto files requires installation of the protobuf-c package from google: http://code.google.com/p/protobuf-c/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9605) Update junit dependency
[ https://issues.apache.org/jira/browse/HADOOP-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669775#comment-13669775 ] Konstantin Boudnik commented on HADOOP-9605: I should've be more explicit - I am sorry - I meant that test update also needs to go to a separate JIRA. +1 on the version bump, as it seems that javac warnings are legitimate. Update junit dependency --- Key: HADOOP-9605 URL: https://issues.apache.org/jira/browse/HADOOP-9605 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 3.0.0, 2.0.5-beta Reporter: Timothy St. Clair Labels: maven Attachments: HADOOP-9605.patch1, HADOOP-9605.patch2 Simple update of the junit dependency to use newer version. E.g. when running maven-rpmbuild on Fedora 18. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9605) Update junit dependency
[
https://issues.apache.org/jira/browse/HADOOP-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669783#comment-13669783
]
Hadoop QA commented on HADOOP-9605:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585304/HADOOP-9605.patch2
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2582//console
This message is automatically generated.
Update junit dependency
---
Key: HADOOP-9605
URL: https://issues.apache.org/jira/browse/HADOOP-9605
Project: Hadoop Common
Issue Type: Improvement
Components: build
Affects Versions: 3.0.0, 2.0.5-beta
Reporter: Timothy St. Clair
Labels: maven
Attachments: HADOOP-9605.patch1, HADOOP-9605.patch2
Simple update of the junit dependency to use newer version. E.g. when
running maven-rpmbuild on Fedora 18.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9509) Implement ONCRPC and XDR
[ https://issues.apache.org/jira/browse/HADOOP-9509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HADOOP-9509: --- Attachment: HADOOP-9509.5.patch Uploaded new patch to fix the findbugs warnings. Implement ONCRPC and XDR Key: HADOOP-9509 URL: https://issues.apache.org/jira/browse/HADOOP-9509 Project: Hadoop Common Issue Type: New Feature Reporter: Brandon Li Assignee: Brandon Li Attachments: HADOOP-9509.2patch, HADOOP-9509.3.patch, HADOOP-9509.4.patch, HADOOP-9509.5.patch, HADOOP-9509.patch This is to track the implementation of ONCRPC(rfc5531) and XDR(rfc4506). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9606) Protocol buffer support cannot compile under C
[
https://issues.apache.org/jira/browse/HADOOP-9606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669877#comment-13669877
]
Ralph Castain commented on HADOOP-9606:
---
Details of the error when executing the protobuf-c parser on
NamenodeProtocol.proto:
protoc-c --c_out=. -I . -Ipath-to-other-imports NamenodeProtocol.proto
The register function appears on line 714 of the resulting
NamenodeProtocol.pb-c.h file:
{code}
void (*register)(Hadoop__Hdfs__NamenodeProtocolService_Service *service,
const Hadoop__Hdfs__RegisterRequestProto *input,
Hadoop__Hdfs__RegisterResponseProto_Closure closure,
void *closure_data);
{code}
It's a direct consequence of line 228 in the .proto file:
{code}
/*
* Request to register a sub-ordinate namenode
*/
rpc register(RegisterRequestProto) returns(RegisterResponseProto);
{code}
The rpc needs to be renamed to something non-reserved.
Protocol buffer support cannot compile under C
--
Key: HADOOP-9606
URL: https://issues.apache.org/jira/browse/HADOOP-9606
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.4-alpha
Reporter: Ralph Castain
When compiling Hadoop's .proto descriptions for use in C, an error occurs
because one of the RPC's in NamenodeProtocol.proto is named register. This
name is a reserved word in languages such as C. When using the Java and C++
languages, the name is hidden inside a class and therefore doesn't cause an
error. Unfortunately, that is not the case in non-class languages such as C.
Note: generating the C translation of the .proto files requires installation
of the protobuf-c package from google:
http://code.google.com/p/protobuf-c/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9606) Protocol buffer support cannot compile under C
[ https://issues.apache.org/jira/browse/HADOOP-9606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ralph Castain updated HADOOP-9606: -- Attachment: NamenodeProtocol.pb-c.h NamenodeProtocol.pb-c.c Files output from protoc-c parser Protocol buffer support cannot compile under C -- Key: HADOOP-9606 URL: https://issues.apache.org/jira/browse/HADOOP-9606 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.4-alpha Reporter: Ralph Castain Attachments: NamenodeProtocol.pb-c.c, NamenodeProtocol.pb-c.h When compiling Hadoop's .proto descriptions for use in C, an error occurs because one of the RPC's in NamenodeProtocol.proto is named register. This name is a reserved word in languages such as C. When using the Java and C++ languages, the name is hidden inside a class and therefore doesn't cause an error. Unfortunately, that is not the case in non-class languages such as C. Note: generating the C translation of the .proto files requires installation of the protobuf-c package from google: http://code.google.com/p/protobuf-c/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9509) Implement ONCRPC and XDR
[
https://issues.apache.org/jira/browse/HADOOP-9509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669890#comment-13669890
]
Hadoop QA commented on HADOOP-9509:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585324/HADOOP-9509.5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 10 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-nfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2583//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2583//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-nfs.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2583//console
This message is automatically generated.
Implement ONCRPC and XDR
Key: HADOOP-9509
URL: https://issues.apache.org/jira/browse/HADOOP-9509
Project: Hadoop Common
Issue Type: New Feature
Reporter: Brandon Li
Assignee: Brandon Li
Attachments: HADOOP-9509.2patch, HADOOP-9509.3.patch,
HADOOP-9509.4.patch, HADOOP-9509.5.patch, HADOOP-9509.patch
This is to track the implementation of ONCRPC(rfc5531) and XDR(rfc4506).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9509) Implement ONCRPC and XDR
[ https://issues.apache.org/jira/browse/HADOOP-9509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HADOOP-9509: --- Attachment: HADOOP-9509.6.patch Implement ONCRPC and XDR Key: HADOOP-9509 URL: https://issues.apache.org/jira/browse/HADOOP-9509 Project: Hadoop Common Issue Type: New Feature Reporter: Brandon Li Assignee: Brandon Li Attachments: HADOOP-9509.2patch, HADOOP-9509.3.patch, HADOOP-9509.4.patch, HADOOP-9509.5.patch, HADOOP-9509.6.patch, HADOOP-9509.patch This is to track the implementation of ONCRPC(rfc5531) and XDR(rfc4506). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9509) Implement ONCRPC and XDR
[
https://issues.apache.org/jira/browse/HADOOP-9509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669905#comment-13669905
]
Hadoop QA commented on HADOOP-9509:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585335/HADOOP-9509.6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 10 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-nfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2584//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2584//console
This message is automatically generated.
Implement ONCRPC and XDR
Key: HADOOP-9509
URL: https://issues.apache.org/jira/browse/HADOOP-9509
Project: Hadoop Common
Issue Type: New Feature
Reporter: Brandon Li
Assignee: Brandon Li
Attachments: HADOOP-9509.2patch, HADOOP-9509.3.patch,
HADOOP-9509.4.patch, HADOOP-9509.5.patch, HADOOP-9509.6.patch,
HADOOP-9509.patch
This is to track the implementation of ONCRPC(rfc5531) and XDR(rfc4506).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669930#comment-13669930
]
Vinod Kumar Vavilapalli commented on HADOOP-9574:
-
Looks good, +1. Checking this in.
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated HADOOP-9574:
Resolution: Fixed
Fix Version/s: 2.0.5-beta
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed this to trunk and branch-2. Thanks Jian!
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Fix For: 2.0.5-beta
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13669940#comment-13669940
]
Hudson commented on HADOOP-9574:
Integrated in Hadoop-trunk-Commit #3804 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/3804/])
HADOOP-9574. Added new methods in AbstractDelegationTokenSecretManager for
helping YARN ResourceManager to reuse code for RM restart. Contributed by Jian
He. (Revision 1487692)
Result = SUCCESS
vinodkv :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1487692
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/DelegationKey.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/token/delegation/TestDelegationToken.java
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Fix For: 2.0.5-beta
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He reopened HADOOP-9574:
-
TestDelegationToken.testRollMasterKey is possibly failing because of timing
issue
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Fix For: 2.0.5-beta
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He updated HADOOP-9574:
Status: Patch Available (was: Reopened)
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Fix For: 2.0.5-beta
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch,
HADOOP-9574.fixtest.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He updated HADOOP-9574:
Attachment: HADOOP-9574.fixtest.patch
change the hard sleep to a while loop and attach a timeout
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Fix For: 2.0.5-beta
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch,
HADOOP-9574.fixtest.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1366#comment-1366
]
Jian He commented on HADOOP-9574:
-
The jeckin link of the possible test failure
https://issues.apache.org/jira/browse/MAPREDUCE-5261?focusedCommentId=13669981page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13669981
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Fix For: 2.0.5-beta
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch,
HADOOP-9574.fixtest.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9574) Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart
[
https://issues.apache.org/jira/browse/HADOOP-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670017#comment-13670017
]
Hadoop QA commented on HADOOP-9574:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585356/HADOOP-9574.fixtest.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2585//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2585//console
This message is automatically generated.
Add new methods in AbstractDelegationTokenSecretManager for restoring
RMDelegationTokens on RMRestart
-
Key: HADOOP-9574
URL: https://issues.apache.org/jira/browse/HADOOP-9574
Project: Hadoop Common
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Fix For: 2.0.5-beta
Attachments: HADOOP-9574.1.patch, HADOOP-9574.2.patch,
HADOOP-9574.fixtest.patch
we're considering to add the following methods in
AbstractDelegationTokenSecretManager for restoring RMDelegationTokens, these
methods can also possibly be reused by hdfsDelegationTokenSecretManager, see
YARN-638
protected void storeNewMasterKey(DelegationKey key) throws IOException {
return;
}
protected void removeStoredMasterKey(DelegationKey key) {
return;
}
protected void storeNewToken(TokenIdent ident, long renewDate) {
return;
}
protected void removeStoredToken(TokenIdent ident) throws IOException {
}
protected void updateStoredToken(TokenIdent ident, long renewDate) {
return;
}
Also move addPersistedDelegationToken in hdfs.DelegationTokenSecretManager,
to AbstractDelegationTokenSecretManager
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7391) Document Interface Classification from HADOOP-5073
[ https://issues.apache.org/jira/browse/HADOOP-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13670032#comment-13670032 ] Karthik Kambatla commented on HADOOP-7391: -- +1 to both the HTML and apt format documentation from my side. Document Interface Classification from HADOOP-5073 -- Key: HADOOP-7391 URL: https://issues.apache.org/jira/browse/HADOOP-7391 Project: Hadoop Common Issue Type: Bug Reporter: Sanjay Radia Assignee: Sanjay Radia Attachments: hadoop-7391-copy.patch, hadoop-7391-v1.patch, hadoop-7391-v2.patch, hadoop-7391-v3.patch, Hadoop Interface Taxonomy-3.html, Hadoop Interface Taxonomy.html The documentation for interface classification in Jira Hadoop-5073 was not copied to the Javadoc of the classification. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
