[ https://issues.apache.org/jira/browse/FLINK-20714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvid Heise updated FLINK-20714: -------------------------------- Fix Version/s: (was: 1.12.3) 1.12.4 > Hive delegation token is not obtained when using `kinit` to submit Yarn > per-job > -------------------------------------------------------------------------------- > > Key: FLINK-20714 > URL: https://issues.apache.org/jira/browse/FLINK-20714 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN > Affects Versions: 1.11.2, 1.11.3, 1.12.0 > Environment: Flink 1.11.2 on Yarn > Reporter: jackwangcs > Priority: Major > Labels: keberos > Fix For: 1.11.4, 1.13.0, 1.12.4 > > > Hive delegation token is not obtained when using `kinit` to submit Yarn > per-job. > In YarnClusterDescriptor, it calls org.apache.flink.yarn.Utils#setTokensFor > to obtain tokens for the job. But setTokensFor only obtains HDFS and HBase > tokens currently, since the Hive integration is supported, the Hive > delegation should be obtained also. > Otherwise, it will throw the following error when it tries to connect to > Hive metastore: > {code:java} > Caused by: MetaException(message:Could not connect to meta store using any of > the URIs provided. Most recent failure: > org.apache.thrift.transport.TTransportException: GSS initiate failedCaused > by: MetaException(message:Could not connect to meta store using any of the > URIs provided. Most recent failure: > org.apache.thrift.transport.TTransportException: GSS initiate failed at > org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232) > at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316) > at > org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) > at java.security.AccessController.doPrivileged(Native Method) at > javax.security.auth.Subject.doAs(Subject.java:422) at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924) > at > org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:464) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:244) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:187) > at > org.apache.flink.table.catalog.hive.client.HiveShimV100.getHiveMetastoreClient(HiveShimV100.java:97) > at > org.apache.flink.table.catalog.hive.client.HiveMetastoreClientWrapper.createMetastoreClient(HiveMetastoreClientWrapper.java:240) > at > org.apache.flink.table.catalog.hive.client.HiveMetastoreClientWrapper.<init>(HiveMetastoreClientWrapper.java:71) > at > org.apache.flink.table.catalog.hive.client.HiveMetastoreClientFactory.create(HiveMetastoreClientFactory.java:35) > at > org.apache.flink.connectors.hive.HiveTableMetaStoreFactory$HiveTableMetaStore.<init>(HiveTableMetaStoreFactory.java:74) > at > org.apache.flink.connectors.hive.HiveTableMetaStoreFactory$HiveTableMetaStore.<init>(HiveTableMetaStoreFactory.java:68) > at > org.apache.flink.connectors.hive.HiveTableMetaStoreFactory.createTableMetaStore(HiveTableMetaStoreFactory.java:65) > at > org.apache.flink.connectors.hive.HiveTableMetaStoreFactory.createTableMetaStore(HiveTableMetaStoreFactory.java:43) > at > org.apache.flink.table.filesystem.PartitionLoader.<init>(PartitionLoader.java:61) > at > org.apache.flink.table.filesystem.FileSystemCommitter.commitUpToCheckpoint(FileSystemCommitter.java:97) > at > org.apache.flink.table.filesystem.FileSystemOutputFormat.finalizeGlobal(FileSystemOutputFormat.java:95) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)