Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1926457261 Thank you, @deniskuzZ , checking `HMSBenchmarks` and related stuff. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

SourabhBadhya commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1923813701 The commit is reverted via #5058 . cc @InvisibleProgrammer @deniskuzZ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1923767559 Hi @deniskuzZ , thank you for the review. Can I ask you to roll back the change? I'm highly against keeping difference between upstream and downstream. Unfortunately, I ha

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

deniskuzZ commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1923654410 @InvisibleProgrammer , @SourabhBadhya, any measurements on a query performance change after the "inner select"? what is the effect of the optimization? Are we sure it won't cause this quer

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

SourabhBadhya merged PR #4740: URL: https://github.com/apache/hive/pull/4740 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apa

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1918780776 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on code in PR #4740: URL: https://github.com/apache/hive/pull/4740#discussion_r1471542557 ## ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/handler/TestAbortedTxnCleaner.java: ## @@ -315,7 +327,6 @@ public void testAbortedCleaningWithThreeTxns

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on code in PR #4740: URL: https://github.com/apache/hive/pull/4740#discussion_r1471540262 ## ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/handler/TestAbortedTxnCleaner.java: ## @@ -282,9 +284,19 @@ public void testCleaningOfAbortedDirectories

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1916917216 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

SourabhBadhya commented on code in PR #4740: URL: https://github.com/apache/hive/pull/4740#discussion_r1469672819 ## ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/handler/TestAbortedTxnCleaner.java: ## @@ -282,9 +284,19 @@ public void testCleaningOfAbortedDirectoriesBelowB

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

SourabhBadhya commented on code in PR #4740: URL: https://github.com/apache/hive/pull/4740#discussion_r1469669850 ## ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/handler/TestAbortedTxnCleaner.java: ## @@ -282,9 +284,19 @@ public void testCleaningOfAbortedDirectoriesBelowB

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on code in PR #4740: URL: https://github.com/apache/hive/pull/4740#discussion_r1463281761 ## ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/handler/TestAbortedTxnCleaner.java: ## @@ -282,9 +284,19 @@ public void testCleaningOfAbortedDirectories

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

SourabhBadhya commented on code in PR #4740: URL: https://github.com/apache/hive/pull/4740#discussion_r1463084357 ## ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/handler/TestAbortedTxnCleaner.java: ## @@ -282,9 +284,19 @@ public void testCleaningOfAbortedDirectoriesBelowB

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1904173863 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1897246567 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1896336789 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1884764821 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1881790591 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1880867812 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1879827145 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1878797904 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1877439830 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1873784502 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1866799901 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1852225849 ## [](https://sonarcloud.

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1828706988 Kudos, SonarCloud Quality Gate passed!    [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1828322560 Kudos, SonarCloud Quality Gate passed!    [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1817666300 Kudos, SonarCloud Quality Gate passed!    [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1816341957 Kudos, SonarCloud Quality Gate passed!    [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

SourabhBadhya commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1782445283 I think we should consider all records which are in initiated state. This would greatly reduce the number of abort cleanup requests. -- This is an automated message from the Apache G

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1782414344 Hi, @SourabhBadhya ! Which compaction queue states should I consider? My educated guess is only the working state but I'm not sure about Initiated and Ready for cleaning

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1767218046 Kudos, SonarCloud Quality Gate passed!    [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-176672 Kudos, SonarCloud Quality Gate passed!    [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on code in PR #4740: URL: https://github.com/apache/hive/pull/4740#discussion_r1362347505 ## ql/src/test/org/apache/hadoop/hive/metastore/txn/TestCompactionTxnHandler.java: ## @@ -1055,6 +1055,38 @@ public void testFindReadyToCleanAborts_limitFetchS

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on code in PR #4740: URL: https://github.com/apache/hive/pull/4740#discussion_r1362344051 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/impl/ReadyToCleanAbortHandler.java: ## @@ -76,7 +76,13 @@ public cl

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on code in PR #4740: URL: https://github.com/apache/hive/pull/4740#discussion_r1362341302 ## ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/handler/TestAbortedTxnCleaner.java: ## @@ -290,49 +341,6 @@ public void testCleaningOfAbortedDirectories

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

SourabhBadhya commented on code in PR #4740: URL: https://github.com/apache/hive/pull/4740#discussion_r1361606801 ## standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/impl/ReadyToCleanAbortHandler.java: ## @@ -76,7 +76,13 @@ public class Re

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1757650756 @SourabhBadhya , can I ask you to review it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1757474287 Kudos, SonarCloud Quality Gate passed!    [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1757034146 > +1 I think it is fine. Also the test is also kinda ok. Do you know need anything that remain? It seems I have one test to fix. -- This is an automated message from the

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1755522485 Kudos, SonarCloud Quality Gate passed!    [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

InvisibleProgrammer closed pull request #4740: HIVE-27637: Compare highest write ID of compaction records when tryin… URL: https://github.com/apache/hive/pull/4740 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] HIVE-27637: Compare highest write ID of compaction records when tryin… [hive]

sonarcloud[bot] commented on PR #4740: URL: https://github.com/apache/hive/pull/4740#issuecomment-1745874781 Kudos, SonarCloud Quality Gate passed!    [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality