[jira] [Work logged] (HIVE-26704) Cleaner shouldn't be blocked by global min open txnId

Mon, 27 Feb 2023 16:33:04 -0800 


     [ 
https://issues.apache.org/jira/browse/HIVE-26704?focusedWorklogId=847904&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-847904
 ]

ASF GitHub Bot logged work on HIVE-26704:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 28/Feb/23 00:32
            Start Date: 28/Feb/23 00:32
    Worklog Time Spent: 10m 
      Work Description: sonarcloud[bot] commented on PR #3576:
URL: https://github.com/apache/hive/pull/3576#issuecomment-1447340160

   Kudos, SonarCloud Quality Gate passed!    [![Quality Gate 
passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png
 'Quality Gate 
passed')](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=3576)
   
   
[![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png
 
'Bug')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3576&resolved=false&types=BUG)
 
[![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png
 
'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3576&resolved=false&types=BUG)
 [0 
Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3576&resolved=false&types=BUG)
  
   
[![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png
 
'Vulnerability')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3576&resolved=false&types=VULNERABILITY)
 
[![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png
 
'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3576&resolved=false&types=VULNERABILITY)
 [0 
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3576&resolved=false&types=VULNERABILITY)
  
   [![Security 
Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png
 'Security 
Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3576&resolved=false&types=SECURITY_HOTSPOT)
 
[![E](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/E-16px.png
 
'E')](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3576&resolved=false&types=SECURITY_HOTSPOT)
 [2 Security 
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=3576&resolved=false&types=SECURITY_HOTSPOT)
  
   [![Code 
Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png
 'Code 
Smell')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3576&resolved=false&types=CODE_SMELL)
 
[![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png
 
'A')](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3576&resolved=false&types=CODE_SMELL)
 [18 Code 
Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=3576&resolved=false&types=CODE_SMELL)
   
   [![No Coverage 
information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png
 'No Coverage 
information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3576&metric=coverage&view=list)
 No Coverage information  
   [![No Duplication 
information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/NoDuplicationInfo-16px.png
 'No Duplication 
information')](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=3576&metric=duplicated_lines_density&view=list)
 No Duplication information
   
   




Issue Time Tracking
-------------------

    Worklog Id:     (was: 847904)
    Time Spent: 5h 10m  (was: 5h)

> Cleaner shouldn't be blocked by global min open txnId
> -----------------------------------------------------
>
>                 Key: HIVE-26704
>                 URL: https://issues.apache.org/jira/browse/HIVE-26704
>             Project: Hive
>          Issue Type: Task
>            Reporter: Denys Kuzmenko
>            Assignee: Denys Kuzmenko
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 5h 10m
>  Remaining Estimate: 0h
>
> *Single transaction blocks cluster-wide Cleaner operations*
> Currently, if there is a single long-running transaction that can prevent the 
> Cleaner to clean up any tables. This causes file buildup in tables, which can 
> cause performance penalties when listing the directories (note that the 
> compaction is not blocked by this, so unnecessary data is not read, but the 
> files remain there which causes performance penalty). 
> We can reduce the protected files from the open transaction if we have 
> query-table correlation data stored in the backend DB, but this change will 
> need the current method of recording that detail to be revisited. 
> The naive and somewhat backward-compatible approach is to capture the 
> minOpenWriteIds per table. It involves a non-mutation operation (as in, there 
> is no need for the HMS DB to wait for another user’s operation to record it). 
> This does spew data writes into the HMS backend DB, but this is a blind 
> insert operation that can be group-committed across many users. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to