[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[
https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832595#comment-16832595
]
Ayush Saxena commented on HADOOP-16059:
---
Hi [~daryn]
{{new FastSaslClientFactory(null)}} This doesn't throw any checked exception,
Does this still bother?
> Use SASL Factories Cache to Improve Performance
> ---
>
> Key: HADOOP-16059
> URL: https://issues.apache.org/jira/browse/HADOOP-16059
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Critical
> Fix For: 3.3.0
>
> Attachments: After-Dn.png, After-Read.png, After-Server.png,
> After-write.png, Before-DN.png, Before-Read.png, Before-Server.png,
> Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch,
> HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch
>
>
> SASL Client factories can be cached and SASL Server Factories and SASL Client
> Factories can be together extended at SaslParticipant to improve performance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[
https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832509#comment-16832509
]
Daryn Sharp commented on HADOOP-16059:
--
A bit late, but static blocks that throw exceptions can cause very bizarre and
misleading errors. Is there any particular reason this:
{code:java}
+ private static SaslClientFactory saslFactory;
+ static {
+saslFactory = new FastSaslClientFactory(null);
+ }{code}
Isn't this:
{code:java}
+ private static final SaslClientFactory saslFactory = new
FastSaslClientFactory(null);{code}
> Use SASL Factories Cache to Improve Performance
> ---
>
> Key: HADOOP-16059
> URL: https://issues.apache.org/jira/browse/HADOOP-16059
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Critical
> Fix For: 3.3.0
>
> Attachments: After-Dn.png, After-Read.png, After-Server.png,
> After-write.png, Before-DN.png, Before-Read.png, Before-Server.png,
> Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch,
> HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch
>
>
> SASL Client factories can be cached and SASL Server Factories and SASL Client
> Factories can be together extended at SaslParticipant to improve performance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832280#comment-16832280 ] Hudson commented on HADOOP-16059: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16499 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/16499/]) HADOOP-16059. Use SASL Factories Cache to Improve Performance. (vinayakumarb: rev f1875b205e492ef071e7ef78b147efee0e51263d) * (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/FastSaslClientFactory.java * (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/FastSaslServerFactory.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcClient.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslParticipant.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SaslRpcServer.java > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Fix For: 3.3.0 > > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826215#comment-16826215 ] Íñigo Goiri commented on HADOOP-16059: -- +1 from my side too. > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16825718#comment-16825718 ] Vinayakumar B commented on HADOOP-16059: Changes are pretty straight forward. Similar changes already exist for SaslRpcServer, and same has been re-used. So no impact to any functionality IMO. +1, patch v4 LGTM. Will wait for few more days before commit. [~jojochuang], Please take a look at profiler screenshots. > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16820279#comment-16820279 ] Íñigo Goiri commented on HADOOP-16059: -- [~jojochuang] was the one with concerns regarding the approach. [~jojochuang], does the explanation suffice to you? > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819346#comment-16819346 ] Ayush Saxena commented on HADOOP-16059: --- [~vinayrpet] can you help give a check to the code? > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16811043#comment-16811043 ] Íñigo Goiri commented on HADOOP-16059: -- Thanks [~vinayrpet] for detailing the benefits. I think is worth having this. The code looks reasonable and matches the rest. However, I don't have much experience with this part. Someone else should take a look at this. > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16810519#comment-16810519 ] Vinayakumar B commented on HADOOP-16059: Thanks [~ayushtkn] for the contribution. Above screenshots of profiling shows the clear difference in time consumed while loading the SaslFactory. As [~jojochuang] mentioned, it may not add much of value in case of RPCs interacting with same RPC server continuosly as same RPC connection will be maintained. Only in case of client is idle for 10 seconds (default) connection needs to be recreated. Also, there are other cases in which this patch will help. # Same clients interacting with multiple RPC servers in not-so-frequent intervals. ** In this case, RPC connection to second server will be faster, as time to load the SASL factory will be zero. # Clients connecting to DataNodes to read/write data without using cached connection. ** HDFS Client's will write data to DataNodes using TCP connection using new connection everytime. There is NO connection cache for writeBlock() Op. ** For ReadBlock() op connection can be cached only after complete read of intended bytes. Ex: In case of sequential read, client should consume entire block data. ** Socket cache capacity is limited ( 16 ) and expires quickly (4 sec) by default. ** HDFS Client is Non-data-local, then it might be getting different datanode's location for each block, in this case, cache-hits will be less. [~elgoiri] , I believe this change will help above case #2 more as that is more common. Its evident in the above screenshot of *SaslParticipant.createClientSaslParticipant() and S**aslParticipant.createServerSaslParticipant()* **takes far less time for same number of connections. Hope its clear. > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808005#comment-16808005 ] Íñigo Goiri commented on HADOOP-16059: -- It would be good to have an idea on how much we are saving and in which cases. [~jojochuang] thoughts here? > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16806844#comment-16806844 ] Ayush Saxena commented on HADOOP-16059: --- [~elgoiri] Client to datanode connection will get fast for any operation that shall require the same > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16802631#comment-16802631 ] Íñigo Goiri commented on HADOOP-16059: -- Thanks [~ayushtkn] for the numbers. This obviously reduces the time to load the client. However, as [~jojochuang] mentioned, what is the benefit in the common case? Do you guys see this having a significant impact? > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16800809#comment-16800809 ] Ayush Saxena commented on HADOOP-16059: --- [~vinayrpet] [~elgoiri] [~jojochuang] Anyone available for a review > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793724#comment-16793724 ] Ayush Saxena commented on HADOOP-16059: --- Hi [~jojochuang] Got the profiler results, Sorry for coming up late. Can you give a check!!! *At Server-->* Before Patch !Before-Server.png! After Patch !After-Server.png! *For Read-->* Before Patch !Before-Read.png! After Patch !After-Read.png! *For Write-->* Before Patch !Before-Write.png! After Patch !After-write.png! *DN to DN -->* Before Patch !Before-DN.png! After Patch !After-Dn.png! Scenerio :: 3 Datanodes , wrote 100 files and read those 100 files. Let me know, if something more is required. Pls Review :) > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: After-Dn.png, After-Read.png, After-Server.png, > After-write.png, Before-DN.png, Before-Read.png, Before-Server.png, > Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[
https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793726#comment-16793726
]
Hadoop QA commented on HADOOP-16059:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} HADOOP-16059 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-16059 |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16057/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Use SASL Factories Cache to Improve Performance
> ---
>
> Key: HADOOP-16059
> URL: https://issues.apache.org/jira/browse/HADOOP-16059
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Critical
> Attachments: After-Dn.png, After-Read.png, After-Server.png,
> After-write.png, Before-DN.png, Before-Read.png, Before-Server.png,
> Before-Write.png, HADOOP-16059-01.patch, HADOOP-16059-02.patch,
> HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch
>
>
> SASL Client factories can be cached and SASL Server Factories and SASL Client
> Factories can be together extended at SaslParticipant to improve performance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[
https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793708#comment-16793708
]
Hadoop QA commented on HADOOP-16059:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color}
| {color:red} HADOOP-16059 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-16059 |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16056/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Use SASL Factories Cache to Improve Performance
> ---
>
> Key: HADOOP-16059
> URL: https://issues.apache.org/jira/browse/HADOOP-16059
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Critical
> Attachments: After-Server.png, Before-Server.png,
> HADOOP-16059-01.patch, HADOOP-16059-02.patch, HADOOP-16059-02.patch,
> HADOOP-16059-03.patch, HADOOP-16059-04.patch
>
>
> SASL Client factories can be cached and SASL Server Factories and SASL Client
> Factories can be together extended at SaslParticipant to improve performance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16764973#comment-16764973 ] Ayush Saxena commented on HADOOP-16059: --- Thanx [~jojochuang] for the review!!! Regarding the performance. Actually at the datanode side. (SASLParticipant) neither server nor client cache is there. So whenever a client connects to a datanode the SASLClient shall be created for any operation say read, write whatever the operation that a client needs and SASLServer at the dn side. The number of dn's usually tends to be huge. This is where this cache seems to be useful. > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761124#comment-16761124 ] Wei-Chiu Chuang commented on HADOOP-16059: -- Thanks for clarification, [~ayushtkn] I understand the idea of caching, and from a quick peek the patch looks reasonable. It is especially useful for SASL server since it gets lots of client connections. On the other hand, clients are usually long lived connections and you typically don't connect to more a handful of name services at a time, so I suspected the overhead of setting up the SASL client isn't so noticeable. "premature optimization is the root of all evils" as it is claimed, so wanted to know how much overhead does this patch eliminates, if there's a profiling already done. > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760774#comment-16760774 ] Ayush Saxena commented on HADOOP-16059: --- Thanx [~jojochuang] for coming up and giving it a look. :) This is strictly a performance improvement only. Doesn't fixes a broken stuff , neither does it intends to fix something broken. You can check HADOOP-10172 it finds some relation with the present change. Regarding the scenario here , would give a brief regarding the scenario , for a call always the SASLClient is created which goes and fetches up info and perform several actions which are time consuming . The idea here is to cache the results and save efforts for the subsequent processing and calls. The HADOOP-10172 already had it done at the server layer , here I have extended the scenario at the client layer And even extended the already available server cache and the newly formed client cache at SASLParticipant level to save the efforts up there too. Let me know , if something isn't clear . I shall try my best to clarify. Pls Review!!! > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16059) Use SASL Factories Cache to Improve Performance
[ https://issues.apache.org/jira/browse/HADOOP-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16759974#comment-16759974 ] Wei-Chiu Chuang commented on HADOOP-16059: -- Hi [~ayushtkn] thanks for reporting the issue & providing the patch. In order to understand this issue better, would you please help me answer following questions: # is it a regression? # How to reproduce? # what was the symptom prior to the patch? Are there any profiler output or error messages? # Any tests? Or is it strictly a performance improvement? By answering these questions, you also help the community to troubleshoot the same issues better in the future. > Use SASL Factories Cache to Improve Performance > --- > > Key: HADOOP-16059 > URL: https://issues.apache.org/jira/browse/HADOOP-16059 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Critical > Attachments: HADOOP-16059-01.patch, HADOOP-16059-02.patch, > HADOOP-16059-02.patch, HADOOP-16059-03.patch, HADOOP-16059-04.patch > > > SASL Client factories can be cached and SASL Server Factories and SASL Client > Factories can be together extended at SaslParticipant to improve performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
