[jira] [Work logged] (HIVE-25080) Create metric about oldest entry in "ready for cleaning" state
[ https://issues.apache.org/jira/browse/HIVE-25080?focusedWorklogId=601583=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-601583 ] ASF GitHub Bot logged work on HIVE-25080: - Author: ASF GitHub Bot Created on: 25/May/21 08:51 Start Date: 25/May/21 08:51 Worklog Time Spent: 10m Work Description: klcopp merged pull request #2297: URL: https://github.com/apache/hive/pull/2297 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 601583) Time Spent: 0.5h (was: 20m) > Create metric about oldest entry in "ready for cleaning" state > -- > > Key: HIVE-25080 > URL: https://issues.apache.org/jira/browse/HIVE-25080 > Project: Hive > Issue Type: Bug >Reporter: Antal Sinkovits >Assignee: Antal Sinkovits >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > When a compaction txn commits, COMPACTION_QUEUE.CQ_COMMIT_TIME is updated > with the current time. Then the compaction state is set to "ready for > cleaning". (... and then the Cleaner runs and the state is set to "succeeded" > hopefully) > Based on this we know (roughly) how long a compaction has been in state > "ready for cleaning". > We should create a metric similar to compaction_oldest_enqueue_age_in_sec > that would show that the cleaner is blocked by something i.e. find the > compaction in "ready for cleaning" that has the oldest commit time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25080) Create metric about oldest entry in "ready for cleaning" state
[
https://issues.apache.org/jira/browse/HIVE-25080?focusedWorklogId=599211=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-599211
]
ASF GitHub Bot logged work on HIVE-25080:
-
Author: ASF GitHub Bot
Created on: 19/May/21 13:08
Start Date: 19/May/21 13:08
Worklog Time Spent: 10m
Work Description: klcopp commented on a change in pull request #2297:
URL: https://github.com/apache/hive/pull/2297#discussion_r635220587
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
##
@@ -292,7 +292,10 @@
TxnStatus.OPEN + "' AND \"TXN_TYPE\" != "+
TxnType.REPL_CREATED.getValue() +") \"T\" CROSS JOIN (" +
"SELECT COUNT(*), MIN(\"TXN_ID\"), ({0} - MIN(\"TXN_STARTED\"))/1000
FROM \"TXNS\" WHERE \"TXN_STATE\"='" +
TxnStatus.ABORTED + "') \"A\" CROSS JOIN (" +
- "SELECT COUNT(*), ({0} - MIN(\"HL_ACQUIRED_AT\"))/1000 FROM
\"HIVE_LOCKS\") \"HL\"";
+ "SELECT COUNT(*), ({0} - MIN(\"HL_ACQUIRED_AT\"))/1000 FROM
\"HIVE_LOCKS\") \"HL\" CROSS JOIN (" +
+ "SELECT ({0} - MIN(\"CQ_ENQUEUE_TIME\"))/1000 from \"COMPACTION_QUEUE\"
WHERE " +
Review comment:
I think CQ_ENQUEUE_TIME is the time that the compaction was put in
"initiated" state. Either CQ_ENQUEUE_TIME value should be updated when the
compaction is put in "ready for cleaning" or we need a new column in the
compaction queue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 599211)
Time Spent: 20m (was: 10m)
> Create metric about oldest entry in "ready for cleaning" state
> --
>
> Key: HIVE-25080
> URL: https://issues.apache.org/jira/browse/HIVE-25080
> Project: Hive
> Issue Type: Bug
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When a compaction txn commits, COMPACTION_QUEUE.CQ_COMMIT_TIME is updated
> with the current time. Then the compaction state is set to "ready for
> cleaning". (... and then the Cleaner runs and the state is set to "succeeded"
> hopefully)
> Based on this we know (roughly) how long a compaction has been in state
> "ready for cleaning".
> We should create a metric similar to compaction_oldest_enqueue_age_in_sec
> that would show that the cleaner is blocked by something i.e. find the
> compaction in "ready for cleaning" that has the oldest commit time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25080) Create metric about oldest entry in "ready for cleaning" state
[ https://issues.apache.org/jira/browse/HIVE-25080?focusedWorklogId=598933=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-598933 ] ASF GitHub Bot logged work on HIVE-25080: - Author: ASF GitHub Bot Created on: 18/May/21 23:48 Start Date: 18/May/21 23:48 Worklog Time Spent: 10m Work Description: asinkovits opened a new pull request #2297: URL: https://github.com/apache/hive/pull/2297 …state ### What changes were proposed in this pull request? We should create a metric similar to compaction_oldest_enqueue_age_in_sec that would show that the cleaner is blocked by something i.e. find the compaction in "ready for cleaning" that has the oldest commit time. ### Why are the changes needed? Subtask is part of the compaction observability initiative. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Unit test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 598933) Remaining Estimate: 0h Time Spent: 10m > Create metric about oldest entry in "ready for cleaning" state > -- > > Key: HIVE-25080 > URL: https://issues.apache.org/jira/browse/HIVE-25080 > Project: Hive > Issue Type: Bug >Reporter: Antal Sinkovits >Assignee: Antal Sinkovits >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When a compaction txn commits, COMPACTION_QUEUE.CQ_COMMIT_TIME is updated > with the current time. Then the compaction state is set to "ready for > cleaning". (... and then the Cleaner runs and the state is set to "succeeded" > hopefully) > Based on this we know (roughly) how long a compaction has been in state > "ready for cleaning". > We should create a metric similar to compaction_oldest_enqueue_age_in_sec > that would show that the cleaner is blocked by something i.e. find the > compaction in "ready for cleaning" that has the oldest commit time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
