[jira] [Work logged] (HIVE-26425) Skip SSL cert verification for downloading JWKS in HS2
[ https://issues.apache.org/jira/browse/HIVE-26425?focusedWorklogId=797068&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-797068 ] ASF GitHub Bot logged work on HIVE-26425: - Author: ASF GitHub Bot Created on: 02/Aug/22 00:41 Start Date: 02/Aug/22 00:41 Worklog Time Spent: 10m Work Description: dengzhhu653 merged PR #3473: URL: https://github.com/apache/hive/pull/3473 Issue Time Tracking --- Worklog Id: (was: 797068) Time Spent: 1.5h (was: 1h 20m) > Skip SSL cert verification for downloading JWKS in HS2 > -- > > Key: HIVE-26425 > URL: https://issues.apache.org/jira/browse/HIVE-26425 > Project: Hive > Issue Type: New Feature >Reporter: Yu-Wen Lai >Assignee: Yu-Wen Lai >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > In a dev/test/staging environment, we would probably use letsencrypt staging > certificate for a token generation service. However, its certificate is not > accepted by JVM by default. To ease JWT testing in those kind of > environments, we can introduce a property to disable the certificate > verification just for JWKS downloads. > Ref: https://letsencrypt.org/docs/staging-environment/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26425) Skip SSL cert verification for downloading JWKS in HS2
[ https://issues.apache.org/jira/browse/HIVE-26425?focusedWorklogId=795658&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795658 ] ASF GitHub Bot logged work on HIVE-26425: - Author: ASF GitHub Bot Created on: 27/Jul/22 13:36 Start Date: 27/Jul/22 13:36 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on PR #3473: URL: https://github.com/apache/hive/pull/3473#issuecomment-1196773584 Hi @hsnusonic, cloud you please elaborate a litte bit why we need this in test while others don't? Issue Time Tracking --- Worklog Id: (was: 795658) Time Spent: 1h 20m (was: 1h 10m) > Skip SSL cert verification for downloading JWKS in HS2 > -- > > Key: HIVE-26425 > URL: https://issues.apache.org/jira/browse/HIVE-26425 > Project: Hive > Issue Type: New Feature >Reporter: Yu-Wen Lai >Assignee: Yu-Wen Lai >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > In a dev/test/staging environment, we would probably use letsencrypt staging > certificate for a token generation service. However, its certificate is not > accepted by JVM by default. To ease JWT testing in those kind of > environments, we can introduce a property to disable the certificate > verification just for JWKS downloads. > Ref: https://letsencrypt.org/docs/staging-environment/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26425) Skip SSL cert verification for downloading JWKS in HS2
[
https://issues.apache.org/jira/browse/HIVE-26425?focusedWorklogId=795237&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795237
]
ASF GitHub Bot logged work on HIVE-26425:
-
Author: ASF GitHub Bot
Created on: 26/Jul/22 12:29
Start Date: 26/Jul/22 12:29
Worklog Time Spent: 10m
Work Description: hsnusonic commented on code in PR #3473:
URL: https://github.com/apache/hive/pull/3473#discussion_r929902912
##
service/src/java/org/apache/hive/service/auth/jwt/URLBasedJWKSProvider.java:
##
@@ -52,12 +62,42 @@ public URLBasedJWKSProvider(HiveConf conf) throws
IOException, ParseException {
* Fetches the JWKS and stores into memory. The JWKS are expected to be in
the standard form as defined here -
* https://datatracker.ietf.org/doc/html/rfc7517#appendix-A.
*/
- private void loadJWKSets() throws IOException, ParseException {
+ private void loadJWKSets() throws IOException, ParseException,
GeneralSecurityException {
String jwksURL = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_URL);
+if (jwksURL == null || jwksURL.isEmpty()) {
+ throw new IOException("Invalid value of property: " +
+ HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_URL.varname);
+}
String[] jwksURLs = jwksURL.split(",");
for (String urlString : jwksURLs) {
- URL url = new URL(urlString);
- jwkSets.add(JWKSet.load(url));
+ SSLContext context = null;
+ if (HiveConf.getBoolVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_SKIP_SSL_CERT, false)) {
+context = SSLContext.getInstance("TLS");
+X509TrustManager trustAllManager = new X509TrustManager() {
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String
authType)
+ throws CertificateException {
+ }
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String
authType)
+ throws CertificateException {
+ }
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+return new X509Certificate[0];
+ }
+};
+context.init(null, new X509TrustManager[]{trustAllManager}, new
SecureRandom());
+ }
+ HttpGet get = new HttpGet(urlString);
+ try (CloseableHttpClient httpClient = (context == null) ?
Review Comment:
Thanks for the suggestion!
Issue Time Tracking
---
Worklog Id: (was: 795237)
Time Spent: 1h 10m (was: 1h)
> Skip SSL cert verification for downloading JWKS in HS2
> --
>
> Key: HIVE-26425
> URL: https://issues.apache.org/jira/browse/HIVE-26425
> Project: Hive
> Issue Type: New Feature
>Reporter: Yu-Wen Lai
>Assignee: Yu-Wen Lai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In a dev/test/staging environment, we would probably use letsencrypt staging
> certificate for a token generation service. However, its certificate is not
> accepted by JVM by default. To ease JWT testing in those kind of
> environments, we can introduce a property to disable the certificate
> verification just for JWKS downloads.
> Ref: https://letsencrypt.org/docs/staging-environment/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26425) Skip SSL cert verification for downloading JWKS in HS2
[
https://issues.apache.org/jira/browse/HIVE-26425?focusedWorklogId=795168&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795168
]
ASF GitHub Bot logged work on HIVE-26425:
-
Author: ASF GitHub Bot
Created on: 26/Jul/22 07:58
Start Date: 26/Jul/22 07:58
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on code in PR #3473:
URL: https://github.com/apache/hive/pull/3473#discussion_r929646063
##
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##
@@ -4250,6 +4250,9 @@ public static enum ConfVars {
HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_URL("hive.server2.authentication.jwt.jwks.url",
"",
"URL of the file from where URLBasedJWKSProvider will try to load JWKS
if JWT is enabled for the\n" +
"authentication mode."),
+
HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_SKIP_SSL_CERT("hive.server2.authentication.jwt.jwks.skip.ssl.cert",
false,
Review Comment:
I see, thanks for the explanation
Issue Time Tracking
---
Worklog Id: (was: 795168)
Time Spent: 1h (was: 50m)
> Skip SSL cert verification for downloading JWKS in HS2
> --
>
> Key: HIVE-26425
> URL: https://issues.apache.org/jira/browse/HIVE-26425
> Project: Hive
> Issue Type: New Feature
>Reporter: Yu-Wen Lai
>Assignee: Yu-Wen Lai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In a dev/test/staging environment, we would probably use letsencrypt staging
> certificate for a token generation service. However, its certificate is not
> accepted by JVM by default. To ease JWT testing in those kind of
> environments, we can introduce a property to disable the certificate
> verification just for JWKS downloads.
> Ref: https://letsencrypt.org/docs/staging-environment/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26425) Skip SSL cert verification for downloading JWKS in HS2
[
https://issues.apache.org/jira/browse/HIVE-26425?focusedWorklogId=795153&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795153
]
ASF GitHub Bot logged work on HIVE-26425:
-
Author: ASF GitHub Bot
Created on: 26/Jul/22 07:22
Start Date: 26/Jul/22 07:22
Worklog Time Spent: 10m
Work Description: hsnusonic commented on code in PR #3473:
URL: https://github.com/apache/hive/pull/3473#discussion_r929612792
##
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##
@@ -4250,6 +4250,9 @@ public static enum ConfVars {
HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_URL("hive.server2.authentication.jwt.jwks.url",
"",
"URL of the file from where URLBasedJWKSProvider will try to load JWKS
if JWT is enabled for the\n" +
"authentication mode."),
+
HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_SKIP_SSL_CERT("hive.server2.authentication.jwt.jwks.skip.ssl.cert",
false,
Review Comment:
I feel `hive.in.test` is used in unit tests only and some server behaviors
are changed. Won't `hive.in.test` interfere other functionalities when we spin
up a cluster?
Issue Time Tracking
---
Worklog Id: (was: 795153)
Time Spent: 50m (was: 40m)
> Skip SSL cert verification for downloading JWKS in HS2
> --
>
> Key: HIVE-26425
> URL: https://issues.apache.org/jira/browse/HIVE-26425
> Project: Hive
> Issue Type: New Feature
>Reporter: Yu-Wen Lai
>Assignee: Yu-Wen Lai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In a dev/test/staging environment, we would probably use letsencrypt staging
> certificate for a token generation service. However, its certificate is not
> accepted by JVM by default. To ease JWT testing in those kind of
> environments, we can introduce a property to disable the certificate
> verification just for JWKS downloads.
> Ref: https://letsencrypt.org/docs/staging-environment/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26425) Skip SSL cert verification for downloading JWKS in HS2
[
https://issues.apache.org/jira/browse/HIVE-26425?focusedWorklogId=795121&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795121
]
ASF GitHub Bot logged work on HIVE-26425:
-
Author: ASF GitHub Bot
Created on: 26/Jul/22 04:18
Start Date: 26/Jul/22 04:18
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on code in PR #3473:
URL: https://github.com/apache/hive/pull/3473#discussion_r929509259
##
service/src/java/org/apache/hive/service/auth/jwt/URLBasedJWKSProvider.java:
##
@@ -52,12 +62,42 @@ public URLBasedJWKSProvider(HiveConf conf) throws
IOException, ParseException {
* Fetches the JWKS and stores into memory. The JWKS are expected to be in
the standard form as defined here -
* https://datatracker.ietf.org/doc/html/rfc7517#appendix-A.
*/
- private void loadJWKSets() throws IOException, ParseException {
+ private void loadJWKSets() throws IOException, ParseException,
GeneralSecurityException {
String jwksURL = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_URL);
+if (jwksURL == null || jwksURL.isEmpty()) {
+ throw new IOException("Invalid value of property: " +
+ HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_URL.varname);
+}
String[] jwksURLs = jwksURL.split(",");
for (String urlString : jwksURLs) {
- URL url = new URL(urlString);
- jwkSets.add(JWKSet.load(url));
+ SSLContext context = null;
+ if (HiveConf.getBoolVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_SKIP_SSL_CERT, false)) {
+context = SSLContext.getInstance("TLS");
+X509TrustManager trustAllManager = new X509TrustManager() {
+ @Override
+ public void checkClientTrusted(X509Certificate[] chain, String
authType)
+ throws CertificateException {
+ }
+ @Override
+ public void checkServerTrusted(X509Certificate[] chain, String
authType)
+ throws CertificateException {
+ }
+ @Override
+ public X509Certificate[] getAcceptedIssuers() {
+return new X509Certificate[0];
+ }
+};
+context.init(null, new X509TrustManager[]{trustAllManager}, new
SecureRandom());
+ }
+ HttpGet get = new HttpGet(urlString);
+ try (CloseableHttpClient httpClient = (context == null) ?
Review Comment:
So the `context` is only used for downloading JWT(?) in test/staging, can we
create `httpClient` by `HttpClients.custom().setSSLContext(context).build()`
regardless of the nullable `context`?
Issue Time Tracking
---
Worklog Id: (was: 795121)
Time Spent: 40m (was: 0.5h)
> Skip SSL cert verification for downloading JWKS in HS2
> --
>
> Key: HIVE-26425
> URL: https://issues.apache.org/jira/browse/HIVE-26425
> Project: Hive
> Issue Type: New Feature
>Reporter: Yu-Wen Lai
>Assignee: Yu-Wen Lai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In a dev/test/staging environment, we would probably use letsencrypt staging
> certificate for a token generation service. However, its certificate is not
> accepted by JVM by default. To ease JWT testing in those kind of
> environments, we can introduce a property to disable the certificate
> verification just for JWKS downloads.
> Ref: https://letsencrypt.org/docs/staging-environment/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26425) Skip SSL cert verification for downloading JWKS in HS2
[
https://issues.apache.org/jira/browse/HIVE-26425?focusedWorklogId=795117&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795117
]
ASF GitHub Bot logged work on HIVE-26425:
-
Author: ASF GitHub Bot
Created on: 26/Jul/22 03:44
Start Date: 26/Jul/22 03:44
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on code in PR #3473:
URL: https://github.com/apache/hive/pull/3473#discussion_r929496509
##
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##
@@ -4250,6 +4250,9 @@ public static enum ConfVars {
HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_URL("hive.server2.authentication.jwt.jwks.url",
"",
"URL of the file from where URLBasedJWKSProvider will try to load JWKS
if JWT is enabled for the\n" +
"authentication mode."),
+
HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_SKIP_SSL_CERT("hive.server2.authentication.jwt.jwks.skip.ssl.cert",
false,
Review Comment:
Since this property only is enabled in test, how about using `hive.in.test`
instead?
Issue Time Tracking
---
Worklog Id: (was: 795117)
Time Spent: 0.5h (was: 20m)
> Skip SSL cert verification for downloading JWKS in HS2
> --
>
> Key: HIVE-26425
> URL: https://issues.apache.org/jira/browse/HIVE-26425
> Project: Hive
> Issue Type: New Feature
>Reporter: Yu-Wen Lai
>Assignee: Yu-Wen Lai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In a dev/test/staging environment, we would probably use letsencrypt staging
> certificate for a token generation service. However, its certificate is not
> accepted by JVM by default. To ease JWT testing in those kind of
> environments, we can introduce a property to disable the certificate
> verification just for JWKS downloads.
> Ref: https://letsencrypt.org/docs/staging-environment/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26425) Skip SSL cert verification for downloading JWKS in HS2
[
https://issues.apache.org/jira/browse/HIVE-26425?focusedWorklogId=795115&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795115
]
ASF GitHub Bot logged work on HIVE-26425:
-
Author: ASF GitHub Bot
Created on: 26/Jul/22 03:42
Start Date: 26/Jul/22 03:42
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on code in PR #3473:
URL: https://github.com/apache/hive/pull/3473#discussion_r929496509
##
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##
@@ -4250,6 +4250,9 @@ public static enum ConfVars {
HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_URL("hive.server2.authentication.jwt.jwks.url",
"",
"URL of the file from where URLBasedJWKSProvider will try to load JWKS
if JWT is enabled for the\n" +
"authentication mode."),
+
HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_SKIP_SSL_CERT("hive.server2.authentication.jwt.jwks.skip.ssl.cert",
false,
Review Comment:
Since this property only be enabled in test, how about using `hive.in.test`
instead?
Issue Time Tracking
---
Worklog Id: (was: 795115)
Time Spent: 20m (was: 10m)
> Skip SSL cert verification for downloading JWKS in HS2
> --
>
> Key: HIVE-26425
> URL: https://issues.apache.org/jira/browse/HIVE-26425
> Project: Hive
> Issue Type: New Feature
>Reporter: Yu-Wen Lai
>Assignee: Yu-Wen Lai
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In a dev/test/staging environment, we would probably use letsencrypt staging
> certificate for a token generation service. However, its certificate is not
> accepted by JVM by default. To ease JWT testing in those kind of
> environments, we can introduce a property to disable the certificate
> verification just for JWKS downloads.
> Ref: https://letsencrypt.org/docs/staging-environment/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26425) Skip SSL cert verification for downloading JWKS in HS2
[ https://issues.apache.org/jira/browse/HIVE-26425?focusedWorklogId=794396&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794396 ] ASF GitHub Bot logged work on HIVE-26425: - Author: ASF GitHub Bot Created on: 22/Jul/22 21:54 Start Date: 22/Jul/22 21:54 Worklog Time Spent: 10m Work Description: hsnusonic opened a new pull request, #3473: URL: https://github.com/apache/hive/pull/3473 ### What changes were proposed in this pull request? Introduce a property to skip SSL cert verification for downloading JWKS in HS2 ### Why are the changes needed? We usually used a self-signed certificate for a token generation service in testing/staging environment and this kind of certificates is not accepted in JVM. To ease the need of testing, we can use this property to turn off certificate verification for testing purpose. ### Does this PR introduce _any_ user-facing change? Yes, users can turn off certificate verification for testing JWT authentication. ### How was this patch tested? Manually tested Issue Time Tracking --- Worklog Id: (was: 794396) Remaining Estimate: 0h Time Spent: 10m > Skip SSL cert verification for downloading JWKS in HS2 > -- > > Key: HIVE-26425 > URL: https://issues.apache.org/jira/browse/HIVE-26425 > Project: Hive > Issue Type: New Feature >Reporter: Yu-Wen Lai >Assignee: Yu-Wen Lai >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > In a dev/test/staging environment, we would probably use letsencrypt staging > certificate for a token generation service. However, its certificate is not > accepted by JVM by default. To ease JWT testing in those kind of > environments, we can introduce a property to disable the certificate > verification just for JWKS downloads. > Ref: https://letsencrypt.org/docs/staging-environment/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
