[
https://issues.apache.org/jira/browse/HIVE-25575?focusedWorklogId=744027&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744027
]
ASF GitHub Bot logged work on HIVE-25575:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Mar/22 14:58
Start Date: 18/Mar/22 14:58
Worklog Time Spent: 10m
Work Description: sourabh912 commented on a change in pull request #3006:
URL: https://github.com/apache/hive/pull/3006#discussion_r829414588
##########
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
##########
@@ -4127,7 +4127,9 @@ private static void populateLlapDaemonVarsSet(Set<String>
llapDaemonVarsSetLocal
" (Use with property
hive.server2.custom.authentication.class)\n" +
" PAM: Pluggable authentication module\n" +
" NOSASL: Raw transport\n" +
- " SAML: SAML 2.0 compliant authentication. This is only supported in
http transport mode."),
+ " SAML: SAML 2.0 compliant authentication. This is only supported in
http transport mode.\n" +
+ " JWT: JWT based authentication, JWT needs to contain the user name
as subject. This is only supported in\n" +
Review comment:
nit: Here we should also document that HS2 expects Asymmetric key for
JWT signature verification.
##########
File path:
service/src/java/org/apache/hive/service/auth/jwt/URLBasedJWKSProvider.java
##########
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.jwk.JWK;
+import com.nimbusds.jose.jwk.JWKMatcher;
+import com.nimbusds.jose.jwk.JWKSelector;
+import com.nimbusds.jose.jwk.JWKSet;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URL;
+import java.text.ParseException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Provides a way to get JWKS json. Hive will use this to verify the incoming
JWTs.
+ */
+public class URLBasedJWKSProvider {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(URLBasedJWKSProvider.class.getName());
+ private final HiveConf conf;
+ private List<JWKSet> jwkSets = new ArrayList<>();
+
+ public URLBasedJWKSProvider(HiveConf conf) throws IOException,
ParseException {
+ this.conf = conf;
+ loadJWKSets();
+ }
+
+ /**
+ * Fetches the JWKS and stores into memory. The JWKS are expected to be in
the standard form as defined here -
+ * https://datatracker.ietf.org/doc/html/rfc7517#appendix-A.
+ */
+ private void loadJWKSets() throws IOException, ParseException {
+ String jwksURL = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_URL);
+ String[] jwksURLs = jwksURL.split(",");
+ for (String urlString : jwksURLs) {
+ URL url = new URL(urlString);
+ jwkSets.add(JWKSet.load(url));
+ LOG.info("Loaded JWKS from " + urlString);
+ }
+ }
+
+ /**
+ * Returns filtered JWKS by one or more criteria, such as kid, typ, alg.
+ */
+ public List<JWK> getJWKs(JWSHeader header) {
+ List<JWK> jwks = new ArrayList<>();
+ JWKSelector selector = new JWKSelector(JWKMatcher.forJWSHeader(header));
+ for (JWKSet jwkSet : jwkSets) {
+ List<JWK> selectedJwks = selector.select(jwkSet);
+ if (selectedJwks != null) {
Review comment:
`select` api in selector never returns null. Therefore we don't need a
null check
##########
File path: service/src/java/org/apache/hive/service/auth/jwt/JWTValidator.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.google.common.base.Preconditions;
+import com.nimbusds.jose.JOSEException;
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.JWSObject;
+import com.nimbusds.jose.JWSVerifier;
+import com.nimbusds.jose.crypto.factories.DefaultJWSVerifierFactory;
+import com.nimbusds.jose.jwk.AsymmetricJWK;
+import com.nimbusds.jose.jwk.JWK;
+import com.nimbusds.jwt.JWTClaimsSet;
+import com.nimbusds.jwt.SignedJWT;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.security.sasl.AuthenticationException;
+import java.io.IOException;
+import java.security.Key;
+import java.text.ParseException;
+import java.util.Date;
+import java.util.List;
+
+/**
+ * This class is used to validate JWT. JWKS is fetched during instantiation
and kept in the memory.
+ * We disallow JWT signature verification with symmetric key, because that
means anyone can get the same key
+ * and use it to sign a JWT.
+ */
+public class JWTValidator {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(JWTValidator.class.getName());
+ private final URLBasedJWKSProvider jwksProvider;
+ private static final DefaultJWSVerifierFactory verifierFactory = new
DefaultJWSVerifierFactory();
+
+ public JWTValidator(HiveConf conf) throws IOException, ParseException {
+ this.jwksProvider = new URLBasedJWKSProvider(conf);
+ }
+
+ public String validateJWTAndExtractUser(String signedJwt) throws
ParseException, AuthenticationException {
+ Preconditions.checkNotNull(jwksProvider);
+ final SignedJWT parsedJwt = SignedJWT.parse(signedJwt);
+ List<JWK> matchedJWKS = jwksProvider.getJWKs(parsedJwt.getHeader());
+
Review comment:
nit: If `matchedJWKS` list is empty, we can throw the exception,
something like:
```
if (matchedJWKS.isEmpty()) {
throw new AuthenticationException("Failed to verify JWT signature with key
id: " + parsedJwt.getHeader().getKeyID() + " because it did not match with any
of the already JWKs");
}
```
##########
File path: service/src/java/org/apache/hive/service/auth/jwt/JWTValidator.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.google.common.base.Preconditions;
+import com.nimbusds.jose.JOSEException;
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.JWSObject;
+import com.nimbusds.jose.JWSVerifier;
+import com.nimbusds.jose.crypto.factories.DefaultJWSVerifierFactory;
+import com.nimbusds.jose.jwk.AsymmetricJWK;
+import com.nimbusds.jose.jwk.JWK;
+import com.nimbusds.jwt.JWTClaimsSet;
+import com.nimbusds.jwt.SignedJWT;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.security.sasl.AuthenticationException;
+import java.io.IOException;
+import java.security.Key;
+import java.text.ParseException;
+import java.util.Date;
+import java.util.List;
+
+/**
+ * This class is used to validate JWT. JWKS is fetched during instantiation
and kept in the memory.
+ * We disallow JWT signature verification with symmetric key, because that
means anyone can get the same key
+ * and use it to sign a JWT.
+ */
+public class JWTValidator {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(JWTValidator.class.getName());
+ private final URLBasedJWKSProvider jwksProvider;
+ private static final DefaultJWSVerifierFactory verifierFactory = new
DefaultJWSVerifierFactory();
+
+ public JWTValidator(HiveConf conf) throws IOException, ParseException {
+ this.jwksProvider = new URLBasedJWKSProvider(conf);
+ }
+
+ public String validateJWTAndExtractUser(String signedJwt) throws
ParseException, AuthenticationException {
+ Preconditions.checkNotNull(jwksProvider);
+ final SignedJWT parsedJwt = SignedJWT.parse(signedJwt);
+ List<JWK> matchedJWKS = jwksProvider.getJWKs(parsedJwt.getHeader());
+
+ // verify signature
+ Exception lastException = null;
+ for (JWK matchedJWK : matchedJWKS) {
+ try {
+ JWSVerifier verifier = getVerifier(parsedJwt.getHeader(), matchedJWK);
+ if (parsedJwt.verify(verifier)) {
+ break;
+ }
+ } catch (Exception e) {
+ lastException = e;
+ LOG.warn("Failed to verify JWT {} by JWK {}", parsedJwt.getPayload(),
matchedJWK, e);
+ }
+ }
+ if (parsedJwt.getState() != JWSObject.State.VERIFIED) {
+ throw new AuthenticationException("Failed to verify JWT signature",
lastException);
Review comment:
nit: can add more details for the signedJWT like its keyID so that a
user can differentiate the exception for two different signed tokens.
##########
File path:
itests/hive-unit/src/test/java/org/apache/hive/service/auth/jwt/TestHttpJwtAuthentication.java
##########
@@ -0,0 +1,219 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.github.tomakehurst.wiremock.junit.WireMockRule;
+import com.nimbusds.jose.JWSAlgorithm;
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.JWSSigner;
+import com.nimbusds.jose.crypto.RSASSASigner;
+import com.nimbusds.jose.jwk.RSAKey;
+import com.nimbusds.jwt.JWTClaimsSet;
+import com.nimbusds.jwt.SignedJWT;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.conf.HiveConf.ConfVars;
+import org.apache.hive.jdbc.HiveConnection;
+import org.apache.hive.jdbc.Utils;
+import org.apache.hive.jdbc.miniHS2.MiniHS2;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+
+import java.io.File;
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+import static com.github.tomakehurst.wiremock.client.WireMock.get;
+import static com.github.tomakehurst.wiremock.client.WireMock.ok;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+
+public class TestHttpJwtAuthentication {
+ private static final Map<String, String> DEFAULTS = new
HashMap<>(System.getenv());
+ private static Map<String, String> envMap;
+
+ private static final File jwtAuthorizedKeyFile =
+ new File("src/test/resources/auth.jwt/jwt-authorized-key.json");
+ private static final File jwtUnauthorizedKeyFile =
+ new File("src/test/resources/auth.jwt/jwt-unauthorized-key.json");
+ private static final File jwtVerificationJWKSFile =
+ new File("src/test/resources/auth.jwt/jwt-verification-jwks.json");
+
+ public static final String USER_1 = "USER_1";
+
+ private static MiniHS2 miniHS2;
+
+ private static final int MOCK_JWKS_SERVER_PORT = 8089;
+ @ClassRule
+ public static final WireMockRule MOCK_JWKS_SERVER = new
WireMockRule(MOCK_JWKS_SERVER_PORT);
+
+ @BeforeClass
+ public static void makeEnvModifiable() throws Exception {
Review comment:
Thanks for sharing. Please add this link in a comment for future readers.
##########
File path: service/src/java/org/apache/hive/service/auth/jwt/JWTValidator.java
##########
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.google.common.base.Preconditions;
+import com.nimbusds.jose.JOSEException;
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.JWSObject;
+import com.nimbusds.jose.JWSVerifier;
+import com.nimbusds.jose.crypto.factories.DefaultJWSVerifierFactory;
+import com.nimbusds.jose.jwk.AsymmetricJWK;
+import com.nimbusds.jose.jwk.JWK;
+import com.nimbusds.jwt.JWTClaimsSet;
+import com.nimbusds.jwt.SignedJWT;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.security.sasl.AuthenticationException;
+import java.io.IOException;
+import java.security.Key;
+import java.text.ParseException;
+import java.util.Date;
+import java.util.List;
+
+/**
+ * This class is used to validate JWT. JWKS is fetched during instantiation
and kept in the memory.
+ * We disallow JWT signature verification with symmetric key, because that
means anyone can get the same key
+ * and use it to sign a JWT.
+ */
+public class JWTValidator {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(JWTValidator.class.getName());
+ private final URLBasedJWKSProvider jwksProvider;
+ private static final DefaultJWSVerifierFactory verifierFactory = new
DefaultJWSVerifierFactory();
+
+ public JWTValidator(HiveConf conf) throws IOException, ParseException {
+ this.jwksProvider = new URLBasedJWKSProvider(conf);
+ }
+
+ public String validateJWTAndExtractUser(String signedJwt) throws
ParseException, AuthenticationException {
+ Preconditions.checkNotNull(jwksProvider);
+ final SignedJWT parsedJwt = SignedJWT.parse(signedJwt);
+ List<JWK> matchedJWKS = jwksProvider.getJWKs(parsedJwt.getHeader());
+
+ // verify signature
+ Exception lastException = null;
+ for (JWK matchedJWK : matchedJWKS) {
+ try {
+ JWSVerifier verifier = getVerifier(parsedJwt.getHeader(), matchedJWK);
+ if (parsedJwt.verify(verifier)) {
+ break;
+ }
+ } catch (Exception e) {
+ lastException = e;
+ LOG.warn("Failed to verify JWT {} by JWK {}", parsedJwt.getPayload(),
matchedJWK, e);
+ }
+ }
+ if (parsedJwt.getState() != JWSObject.State.VERIFIED) {
+ throw new AuthenticationException("Failed to verify JWT signature",
lastException);
+ }
+
+ // verify claims
+ JWTClaimsSet claimsSet = parsedJwt.getJWTClaimsSet();
+ Date expirationTime = claimsSet.getExpirationTime();
+ if (expirationTime != null) {
+ Date now = new Date();
+ if (now.after(expirationTime)) {
+ throw new AuthenticationException("JWT has been expired");
Review comment:
nit: same as above comment, add more details to identify it.
##########
File path:
service/src/java/org/apache/hive/service/auth/jwt/URLBasedJWKSProvider.java
##########
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.jwk.JWK;
+import com.nimbusds.jose.jwk.JWKMatcher;
+import com.nimbusds.jose.jwk.JWKSelector;
+import com.nimbusds.jose.jwk.JWKSet;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URL;
+import java.text.ParseException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link JWKSProvider} which reads JWKS from URL.
+ */
+public class URLBasedJWKSProvider implements JWKSProvider {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(URLBasedJWKSProvider.class.getName());
+ private final HiveConf conf;
+ private List<JWKSet> jwkSets = new ArrayList<>();
+
+ public URLBasedJWKSProvider(HiveConf conf) {
+ this.conf = conf;
+ loadJWKSets();
+ }
+
+ private void loadJWKSets() {
+ String jwksURL = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_THRIFT_HTTP_JWT_JWKS_URL);
+ List<String> jwksURLs =
Arrays.stream(jwksURL.split(",")).collect(Collectors.toList());
+ for (String urlString : jwksURLs) {
+ try {
+ URL url = new URL(urlString);
+ jwkSets.add(JWKSet.load(url));
+ LOG.info("Loaded JWKS from " + urlString);
+ } catch (IOException | ParseException e) {
+ LOG.info("Failed to retrieve JWKS from {}: {}", urlString,
e.getMessage());
+ }
+ }
+ }
+
+ @Override
+ public List<JWK> getJWKs(JWSHeader header) {
+ List<JWK> jwks = new ArrayList<>();
+ JWKSelector selector = new JWKSelector(JWKMatcher.forJWSHeader(header));
Review comment:
I was checking the implementation of JWKMatcher and JWKSelector. It
looks like `JWKMatcher.forJWSHeader(header)` can return null for certain
scenarios and so JWKSelector would throw an illegal argument exception.
One way to approach this is return early if
`JWKMatcher.forJWSHeader(header)` is null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 744027)
Time Spent: 4.5h (was: 4h 20m)
> Add support for JWT authentication in HTTP mode
> -----------------------------------------------
>
> Key: HIVE-25575
> URL: https://issues.apache.org/jira/browse/HIVE-25575
> Project: Hive
> Issue Type: New Feature
> Components: HiveServer2, JDBC
> Affects Versions: 4.0.0
> Reporter: Shubham Chaurasia
> Assignee: Yu-Wen Lai
> Priority: Major
> Labels: pull-request-available
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> It would be good to support JWT auth mechanism in hive. In order to implement
> it, we would need the following -
> On HS2 side -
> 1. Accept JWT in Authorization: Bearer header.
> 2. Fetch JWKS from a public endpoint to verify JWT signature, to start with
> we can fetch on HS2 start up.
> 3. Verify JWT Signature.
> On JDBC Client side -
> 1. Hive jdbc client should be able to accept jwt in JDBC url. (will add more
> details)
> 2. Client should also be able to pick up JWT from an env var if it's defined.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
