Alessandro Di Diego created HIVE-22041:
------------------------------------------
Summary: HiveServer2 creates Delegation Token crc files and never
deletes them
Key: HIVE-22041
URL: https://issues.apache.org/jira/browse/HIVE-22041
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 1.1.0
Reporter: Alessandro Di Diego
It seems that in secure clusters, HiveServer2 creates an unbounded amount of
crc files related to the Delegation Token, e.g.:
{quote}{{# ls -latr .hive_hadoop_delegation_token*.tmp.crc | more}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:41
.hive_hadoop_delegation_token7336761305415229297.tmp.crc}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:41
.hive_hadoop_delegation_token2098027444797756507.tmp.crc}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:41
.hive_hadoop_delegation_token570615377517838929.tmp.crc}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:41
.hive_hadoop_delegation_token2806157469169711507.tmp.crc}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:41
.hive_hadoop_delegation_token4743780849236152782.tmp.crc}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:41
.hive_hadoop_delegation_token4779276962989484605.tmp.crc}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:41
.hive_hadoop_delegation_token614278562135964419.tmp.crc}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:41
.hive_hadoop_delegation_token5163699580858054526.tmp.crc}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:41
.hive_hadoop_delegation_token6764054090932425303.tmp.crc}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:42
.hive_hadoop_delegation_token6389909203128084522.tmp.crc}}
{{-rw-r--r-- 1 hive hive 12 Jul 22 18:42
.hive_hadoop_delegation_token7810958545651446366.tmp.crc}}
{{[...]}}{quote}
this could quickly fills the inode table of the filesystem (usually the one
mounted on /tmp).
I've experienced it in Hive 1.1.0 (CDH 5.16.2), but I think the same issue is
present in the current master branch, since here:
[https://github.com/apache/hive/blob/d1343a69e6be7a312b7c0bb2aeebefaa40535b65/ql/src/java/org/apache/hadoop/hive/ql/exec/SecureCmdDoAs.java#L71]
the crc file gets created, but only the token file gets deleted:
[https://github.com/apache/hive/blob/d1343a69e6be7a312b7c0bb2aeebefaa40535b65/ql/src/java/org/apache/hadoop/hive/ql/exec/SecureCmdDoAs.java#L80]
HIVE-13883 is greatly related to the issue, I think that the same logic could
be used to fix this.
Although, I wonder if the Credentials class should expose some delete method
that also deletes the crc file...
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
