[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[ https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936220#comment-16936220 ] Masatake Iwasaki commented on HADOOP-16581: --- Thanks, [~xkrogen]. > ValueQueue does not trigger an async refill when number of values falls below > watermark > --- > > Key: HADOOP-16581 > URL: https://issues.apache.org/jira/browse/HADOOP-16581 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Affects Versions: 2.7.4, 3.2.0 >Reporter: Yuval Degani >Assignee: Yuval Degani >Priority: Major > Fix For: 2.10.0, 3.3.0, 3.1.4, 3.2.2 > > Attachments: HADOOP-16581-branch-2.001.addendum.patch > > > The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so > that EDEKs could be served quickly, while the cache is replenished in a > background thread. > The existing code for triggering an asynchronous refill is only triggered > when a key queue becomes empty, rather than when it falls below the > configured watermark. > This is a relatively minor fix in the main code, however, most of the tests > require some changes as they verify the previous unintended behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[ https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16935957#comment-16935957 ] Erik Krogen commented on HADOOP-16581: -- Thanks [~iwasakims]! I had prepared the same patch and thought I included it in the commit I pushed, but it seems it was sitting in my local directory only 😓 I just pushed up the fix. > ValueQueue does not trigger an async refill when number of values falls below > watermark > --- > > Key: HADOOP-16581 > URL: https://issues.apache.org/jira/browse/HADOOP-16581 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Affects Versions: 2.7.4, 3.2.0 >Reporter: Yuval Degani >Assignee: Yuval Degani >Priority: Major > Fix For: 2.10.0, 3.3.0, 3.1.4, 3.2.2 > > Attachments: HADOOP-16581-branch-2.001.addendum.patch > > > The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so > that EDEKs could be served quickly, while the cache is replenished in a > background thread. > The existing code for triggering an asynchronous refill is only triggered > when a key queue becomes empty, rather than when it falls below the > configured watermark. > This is a relatively minor fix in the main code, however, most of the tests > require some changes as they verify the previous unintended behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[ https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16935220#comment-16935220 ] Masatake Iwasaki commented on HADOOP-16581: --- I attached addendum patch to fix the build failure in branch-2. > ValueQueue does not trigger an async refill when number of values falls below > watermark > --- > > Key: HADOOP-16581 > URL: https://issues.apache.org/jira/browse/HADOOP-16581 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Affects Versions: 2.7.4, 3.2.0 >Reporter: Yuval Degani >Assignee: Yuval Degani >Priority: Major > Fix For: 2.10.0, 3.3.0, 3.1.4, 3.2.2 > > Attachments: HADOOP-16581-branch-2.001.addendum.patch > > > The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so > that EDEKs could be served quickly, while the cache is replenished in a > background thread. > The existing code for triggering an asynchronous refill is only triggered > when a key queue becomes empty, rather than when it falls below the > configured watermark. > This is a relatively minor fix in the main code, however, most of the tests > require some changes as they verify the previous unintended behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[
https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16935214#comment-16935214
]
Masatake Iwasaki commented on HADOOP-16581:
---
[~xkrogen] branch-2 build failed due to lambda left in the backported fix.
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile
(default-testCompile) on project hadoop-common: Compilation failure
[ERROR]
/ext/srcs/hadoop-branch-2/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestValueQueue.java:[67,33]
lambda expressions are not supported in -source 1.7
[ERROR] (use -source 8 or higher to enable lambda expressions)
{noformat}
> ValueQueue does not trigger an async refill when number of values falls below
> watermark
> ---
>
> Key: HADOOP-16581
> URL: https://issues.apache.org/jira/browse/HADOOP-16581
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, kms
>Affects Versions: 2.7.4, 3.2.0
>Reporter: Yuval Degani
>Assignee: Yuval Degani
>Priority: Major
> Fix For: 2.10.0, 3.3.0, 3.1.4, 3.2.2
>
>
> The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so
> that EDEKs could be served quickly, while the cache is replenished in a
> background thread.
> The existing code for triggering an asynchronous refill is only triggered
> when a key queue becomes empty, rather than when it falls below the
> configured watermark.
> This is a relatively minor fix in the main code, however, most of the tests
> require some changes as they verify the previous unintended behavior.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[ https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934615#comment-16934615 ] Yuval Degani commented on HADOOP-16581: --- Thanks for reviewing and merging, folks! > ValueQueue does not trigger an async refill when number of values falls below > watermark > --- > > Key: HADOOP-16581 > URL: https://issues.apache.org/jira/browse/HADOOP-16581 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Affects Versions: 2.7.4, 3.2.0 >Reporter: Yuval Degani >Assignee: Yuval Degani >Priority: Major > Fix For: 2.10.0, 3.3.0, 3.1.4, 3.2.2 > > > The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so > that EDEKs could be served quickly, while the cache is replenished in a > background thread. > The existing code for triggering an asynchronous refill is only triggered > when a key queue becomes empty, rather than when it falls below the > configured watermark. > This is a relatively minor fix in the main code, however, most of the tests > require some changes as they verify the previous unintended behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[ https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934591#comment-16934591 ] Erik Krogen commented on HADOOP-16581: -- Also backported to branch-3.2, branch-3.1, and branch-2. There was a minor change I needed to make to the test case in branch-2 due to the lack of Java 8 support (removed the lambda, made some parameters final). Thanks [~yuvaldeg]! > ValueQueue does not trigger an async refill when number of values falls below > watermark > --- > > Key: HADOOP-16581 > URL: https://issues.apache.org/jira/browse/HADOOP-16581 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Affects Versions: 2.7.4, 3.2.0 >Reporter: Yuval Degani >Assignee: Yuval Degani >Priority: Major > Fix For: 2.10.0, 3.3.0, 3.1.4, 3.2.2 > > > The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so > that EDEKs could be served quickly, while the cache is replenished in a > background thread. > The existing code for triggering an asynchronous refill is only triggered > when a key queue becomes empty, rather than when it falls below the > configured watermark. > This is a relatively minor fix in the main code, however, most of the tests > require some changes as they verify the previous unintended behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[ https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934582#comment-16934582 ] Hudson commented on HADOOP-16581: - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #17343 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17343/]) HADOOP-16581. Revise ValueQueue to correctly replenish queues that go (xkrogen: rev dd0834696a694564af65a1355c9d13275f44df51) * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestValueQueue.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/ValueQueue.java > ValueQueue does not trigger an async refill when number of values falls below > watermark > --- > > Key: HADOOP-16581 > URL: https://issues.apache.org/jira/browse/HADOOP-16581 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Affects Versions: 2.7.4, 3.2.0 >Reporter: Yuval Degani >Assignee: Yuval Degani >Priority: Major > Fix For: 3.2.1 > > > The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so > that EDEKs could be served quickly, while the cache is replenished in a > background thread. > The existing code for triggering an asynchronous refill is only triggered > when a key queue becomes empty, rather than when it falls below the > configured watermark. > This is a relatively minor fix in the main code, however, most of the tests > require some changes as they verify the previous unintended behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[ https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934568#comment-16934568 ] Erik Krogen commented on HADOOP-16581: -- Just merged this to trunk based on +1 from Anu and myself. Thanks for the contribution [~yuvaldeg]! > ValueQueue does not trigger an async refill when number of values falls below > watermark > --- > > Key: HADOOP-16581 > URL: https://issues.apache.org/jira/browse/HADOOP-16581 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Affects Versions: 2.7.4, 3.2.0 >Reporter: Yuval Degani >Assignee: Yuval Degani >Priority: Major > Fix For: 3.2.1 > > > The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so > that EDEKs could be served quickly, while the cache is replenished in a > background thread. > The existing code for triggering an asynchronous refill is only triggered > when a key queue becomes empty, rather than when it falls below the > configured watermark. > This is a relatively minor fix in the main code, however, most of the tests > require some changes as they verify the previous unintended behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[ https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16933525#comment-16933525 ] Erik Krogen commented on HADOOP-16581: -- Got a clean Jenkins run. +1 from me, will give one more day before committing for anyone else to chime in. Ping [~aengineer] since it looks like I messed up the tag in my last comment. > ValueQueue does not trigger an async refill when number of values falls below > watermark > --- > > Key: HADOOP-16581 > URL: https://issues.apache.org/jira/browse/HADOOP-16581 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Affects Versions: 2.7.4, 3.2.0 >Reporter: Yuval Degani >Assignee: Yuval Degani >Priority: Major > Fix For: 3.2.1 > > > The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so > that EDEKs could be served quickly, while the cache is replenished in a > background thread. > The existing code for triggering an asynchronous refill is only triggered > when a key queue becomes empty, rather than when it falls below the > configured watermark. > This is a relatively minor fix in the main code, however, most of the tests > require some changes as they verify the previous unintended behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[ https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932659#comment-16932659 ] Erik Krogen commented on HADOOP-16581: -- The change LGTM, ping [~xiaochen] who I see has some previous commits in this file, and [~anu] / [~daryn] / [~kihwal] who I believe have some experience with the KMS. > ValueQueue does not trigger an async refill when number of values falls below > watermark > --- > > Key: HADOOP-16581 > URL: https://issues.apache.org/jira/browse/HADOOP-16581 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Affects Versions: 2.7.4, 3.2.0 >Reporter: Yuval Degani >Assignee: Yuval Degani >Priority: Major > Fix For: 3.2.1 > > > The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so > that EDEKs could be served quickly, while the cache is replenished in a > background thread. > The existing code for triggering an asynchronous refill is only triggered > when a key queue becomes empty, rather than when it falls below the > configured watermark. > This is a relatively minor fix in the main code, however, most of the tests > require some changes as they verify the previous unintended behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
