mahesh kumar behera created HIVE-20511:
------------------------------------------
Summary: REPL DUMP is leaking metastore connections
Key: HIVE-20511
URL: https://issues.apache.org/jira/browse/HIVE-20511
Project: Hive
Issue Type: Bug
Components: repl
Affects Versions: 4.0.0
Reporter: mahesh kumar behera
Assignee: mahesh kumar behera
Fix For: 4.0.0
With remote metastore, REPL DUMP leaking connections. Each repl dump task is
leaking one connection due to the usage of stale hive object.
{code}
18/09/04 16:01:46 INFO ReplState: REPL::EVENT_DUMP:
{"dbName":"*","eventId":"566","eventType":"EVENT_COMMIT_TXN","eventsDumpProgress":"1/0","dumpTime":1536076906}
18/09/04 16:01:46 INFO events.AbstractEventHandler: Processing#567 OPEN_TXN
message :
{"txnIds":null,"timestamp":1536076905,"fromTxnId":269,"toTxnId":269,"server":"thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083","servicePrincipal":""}
18/09/04 16:01:46 INFO ReplState: REPL::EVENT_DUMP:
{"dbName":"*","eventId":"567","eventType":"EVENT_OPEN_TXN","eventsDumpProgress":"2/0","dumpTime":1536076906}
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Trying to connect to
metastore with URI
thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Opened a connection to
metastore, current connections: 471
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Connected to metastore.
18/09/04 16:01:46 INFO metastore.RetryingMetaStoreClient:
RetryingMetaStoreClient proxy=class
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=hive
(auth:SIMPLE) retries=24 delay=5 lifetime=0
18/09/04 16:01:46 INFO ReplState: REPL::END:
{"dbName":"*","dumpType":"INCREMENTAL","actualNumEvents":2,"dumpEndTime":1536076906,"dumpDir":"/user/hive/repl/e45bde27-74dc-45cd-9823-400a8fc1aea3","lastReplId":"567"}
18/09/04 16:01:46 INFO repl.ReplDumpTask: Done dumping events, preparing to
return /user/hive/repl/e45bde27-74dc-45cd-9823-400a8fc1aea3,567
18/09/04 16:01:46 INFO ql.Driver: Completed executing
command(queryId=hive_20180904160145_30f9570a-44e0-4f3b-b961-1906d3972fc4); Time
taken: 0.585 seconds
OK
18/09/04 16:01:46 INFO ql.Driver: OK
18/09/04 16:01:46 INFO lockmgr.DbTxnManager: Stopped heartbeat for query:
hive_20180904160145_30f9570a-44e0-4f3b-b961-1906d3972fc4
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Trying to connect to
metastore with URI
thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Opened a connection to
metastore, current connections: 472
18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Connected to metastore.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
