[hive-site] branch main updated: dummy commit
This is an automated email from the ASF dual-hosted git repository. sunchao pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/hive-site.git The following commit(s) were added to refs/heads/main by this push: new f2bc1dd dummy commit f2bc1dd is described below commit f2bc1dd87c46f905c7d876c1655675175e4140e6 Author: Chao Sun AuthorDate: Wed Mar 9 12:01:54 2022 -0800 dummy commit --- downloads.md | 1 - 1 file changed, 1 deletion(-) diff --git a/downloads.md b/downloads.md index e9a421a..e368a33 100644 --- a/downloads.md +++ b/downloads.md @@ -30,7 +30,6 @@ On the mirror, all recent releases are available, but are not guaranteed to be stable. For stable releases, look in the stable directory. - ## News ### 9 June 2021: release 2.3.9 available This release works with Hadoop 2.x.y
[hive-site] branch gh-pages updated: dummy commit
This is an automated email from the ASF dual-hosted git repository. sunchao pushed a commit to branch gh-pages in repository https://gitbox.apache.org/repos/asf/hive-site.git The following commit(s) were added to refs/heads/gh-pages by this push: new ba13381 dummy commit ba13381 is described below commit ba13381f367f39e0d5e319e9aa51c850487c06bf Author: Chao Sun AuthorDate: Wed Mar 9 12:01:54 2022 -0800 dummy commit --- downloads.md | 1 - 1 file changed, 1 deletion(-) diff --git a/downloads.md b/downloads.md index e9a421a..e368a33 100644 --- a/downloads.md +++ b/downloads.md @@ -30,7 +30,6 @@ On the mirror, all recent releases are available, but are not guaranteed to be stable. For stable releases, look in the stable directory. - ## News ### 9 June 2021: release 2.3.9 available This release works with Hadoop 2.x.y
[hive-site] branch gh-pages updated: empty commit
This is an automated email from the ASF dual-hosted git repository. sunchao pushed a commit to branch gh-pages in repository https://gitbox.apache.org/repos/asf/hive-site.git The following commit(s) were added to refs/heads/gh-pages by this push: new 408fadd empty commit 408fadd is described below commit 408fadd28755fe84d77857fb431f350142b99f36 Author: Chao Sun AuthorDate: Wed Mar 9 12:00:12 2022 -0800 empty commit
[hive-site] branch main updated (55711e6 -> 2b54cb5)
This is an automated email from the ASF dual-hosted git repository. sunchao pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/hive-site.git. from 55711e6 initialise repo with a basic readme add 360d200 Add all files from hive.git/docs to gh-pages branch of hive.git add b0b9961 Add .asf.yaml to publish the website whenever a push goes to main add d3cda40 Add Hive 2.3.8 release information add 2b54cb5 Add Hive 2.3.9 release information No new revisions were added by this update. Summary of changes: .asf.yaml | 2 + Dockerfile | 51 +++ Gemfile | 3 + README.md | 25 ++- _config.yml | 17 +++ _includes/footer.html | 14 ++ _includes/header.html | 5 + _includes/sidenav.html | 50 ++ _includes/top.html | 21 +++ _layouts/default.html | 19 +++ css/hive.css| 365 doap_Hive.rdf | 58 +++ downloads.md| 221 +++ favicon.ico | Bin 0 -> 1150 bytes hcatalog_downloads.md | 43 ++ images/feather_small.gif| Bin 0 -> 7500 bytes images/hive-logo.jpg| Bin 0 -> 2498 bytes images/hive_logo_medium.jpg | Bin 0 -> 4372 bytes index.md| 62 issue_tracking.md | 31 javadoc.md | 32 mailing_lists.md| 78 ++ people.md | 156 +++ privacy_policy.md | 48 ++ version_control.md | 27 25 files changed, 1327 insertions(+), 1 deletion(-) create mode 100644 .asf.yaml create mode 100644 Dockerfile create mode 100644 Gemfile create mode 100644 _config.yml create mode 100644 _includes/footer.html create mode 100644 _includes/header.html create mode 100644 _includes/sidenav.html create mode 100644 _includes/top.html create mode 100644 _layouts/default.html create mode 100644 css/hive.css create mode 100644 doap_Hive.rdf create mode 100644 downloads.md create mode 100644 favicon.ico create mode 100644 hcatalog_downloads.md create mode 100644 images/feather_small.gif create mode 100644 images/hive-logo.jpg create mode 100644 images/hive_logo_medium.jpg create mode 100644 index.md create mode 100644 issue_tracking.md create mode 100644 javadoc.md create mode 100644 mailing_lists.md create mode 100644 people.md create mode 100644 privacy_policy.md create mode 100644 version_control.md
[hive-site] branch gh-pages updated: Add Hive 2.3.9 release information
This is an automated email from the ASF dual-hosted git repository.
sunchao pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/hive-site.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new 2b54cb5 Add Hive 2.3.9 release information
2b54cb5 is described below
commit 2b54cb5d57089f50b8a38607d089849b2fe0e350
Author: Chao Sun
AuthorDate: Wed Mar 9 09:54:00 2022 -0800
Add Hive 2.3.9 release information
---
downloads.md | 5 +
javadoc.md | 2 +-
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/downloads.md b/downloads.md
index ceba18e..e9a421a 100644
--- a/downloads.md
+++ b/downloads.md
@@ -32,6 +32,10 @@ directory.
## News
+### 9 June 2021: release 2.3.9 available
+This release works with Hadoop 2.x.y
+You can look at the complete [JIRA change log for this release][HIVE_2_3_9_CL].
+
### 17 January 2021: release 2.3.8 available
This release works with Hadoop 2.x.y
You can look at the complete [JIRA change log for this release][HIVE_2_3_8_CL].
@@ -183,6 +187,7 @@ You can look at the complete [JIRA change log for this
release][HIVE_10_CL].
[HIVE_DL]: http://www.apache.org/dyn/closer.cgi/hive/
[HIVE_3_1_2_CL]:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12344397=Html=12310843
+[HIVE_2_3_9_CL]:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12350009=Text=12310843
[HIVE_2_3_8_CL]:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12349428=Text=12310843
[HIVE_2_3_7_CL]:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12346056=Text=12310843
[HIVE_2_3_6_CL]:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345603=Text=12310843
diff --git a/javadoc.md b/javadoc.md
index 1348128..16ab3a4 100644
--- a/javadoc.md
+++ b/javadoc.md
@@ -24,7 +24,7 @@ layout: default
* [Hive 3.1.2 Javadocs]({{ site.old_javadoc }}/r3.1.2/api/index.html)
* [Hive 3.0.0 Javadocs]({{ site.old_javadoc }}/r3.0.0/api/index.html)
- * [Hive 2.3.8 Javadocs]({{ site.old_javadoc }}/r2.3.8/api/index.html)
+ * [Hive 2.3.9 Javadocs]({{ site.old_javadoc }}/r2.3.9/api/index.html)
* [Hive 2.2.0 Javadocs]({{ site.old_javadoc }}/r2.2.0/api/index.html)
* [Hive 2.1.1 Javadocs]({{ site.old_javadoc }}/r2.1.1/api/index.html)
* [Hive 1.2.2 Javadocs]({{ site.old_javadoc }}/r1.2.2/api/index.html)
[hive-site] branch gh-pages updated: Add Hive 2.3.8 release information
This is an automated email from the ASF dual-hosted git repository.
sunchao pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/hive-site.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new d3cda40 Add Hive 2.3.8 release information
d3cda40 is described below
commit d3cda400fc0b149c4a3251120dd9f7f177b44f6d
Author: Chao Sun
AuthorDate: Wed Mar 9 09:50:36 2022 -0800
Add Hive 2.3.8 release information
---
downloads.md | 5 +
javadoc.md | 2 +-
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/downloads.md b/downloads.md
index 891dbad..ceba18e 100644
--- a/downloads.md
+++ b/downloads.md
@@ -32,6 +32,10 @@ directory.
## News
+### 17 January 2021: release 2.3.8 available
+This release works with Hadoop 2.x.y
+You can look at the complete [JIRA change log for this release][HIVE_2_3_8_CL].
+
### 18 April 2020: release 2.3.7 available
This release works with Hadoop 2.x.y
You can look at the complete [JIRA change log for this release][HIVE_2_3_7_CL].
@@ -179,6 +183,7 @@ You can look at the complete [JIRA change log for this
release][HIVE_10_CL].
[HIVE_DL]: http://www.apache.org/dyn/closer.cgi/hive/
[HIVE_3_1_2_CL]:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12344397=Html=12310843
+[HIVE_2_3_8_CL]:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12349428=Text=12310843
[HIVE_2_3_7_CL]:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12346056=Text=12310843
[HIVE_2_3_6_CL]:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345603=Text=12310843
[HIVE_2_3_5_CL]:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345394=Text=12310843
diff --git a/javadoc.md b/javadoc.md
index 341c529..1348128 100644
--- a/javadoc.md
+++ b/javadoc.md
@@ -24,7 +24,7 @@ layout: default
* [Hive 3.1.2 Javadocs]({{ site.old_javadoc }}/r3.1.2/api/index.html)
* [Hive 3.0.0 Javadocs]({{ site.old_javadoc }}/r3.0.0/api/index.html)
- * [Hive 2.3.7 Javadocs]({{ site.old_javadoc }}/r2.3.7/api/index.html)
+ * [Hive 2.3.8 Javadocs]({{ site.old_javadoc }}/r2.3.8/api/index.html)
* [Hive 2.2.0 Javadocs]({{ site.old_javadoc }}/r2.2.0/api/index.html)
* [Hive 2.1.1 Javadocs]({{ site.old_javadoc }}/r2.1.1/api/index.html)
* [Hive 1.2.2 Javadocs]({{ site.old_javadoc }}/r1.2.2/api/index.html)
[hive] branch master updated (d3542e1 -> 1139c4b)
This is an automated email from the ASF dual-hosted git repository. dkuzmenko pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hive.git. from d3542e1 HIVE-25977: Enhance Compaction Cleaner to skip when there is nothing to do #2 (#2971) (Zoltan Haindrich reviewed by Karen Coppage and Denys Kuzmenko) add 1139c4b HIVE-25688: Addendum (Denys Kuzmenko, reviewed by Karen Coppage) No new revisions were added by this update. Summary of changes: .../org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java | 12 +++- 1 file changed, 7 insertions(+), 5 deletions(-)
[hive] branch master updated: HIVE-25977: Enhance Compaction Cleaner to skip when there is nothing to do #2 (#2971) (Zoltan Haindrich reviewed by Karen Coppage and Denys Kuzmenko)
This is an automated email from the ASF dual-hosted git repository.
kgyrtkirk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new d3542e1 HIVE-25977: Enhance Compaction Cleaner to skip when there is
nothing to do #2 (#2971) (Zoltan Haindrich reviewed by Karen Coppage and Denys
Kuzmenko)
d3542e1 is described below
commit d3542e1b35bbdbaafb52ad742a5168bd29549cee
Author: Zoltan Haindrich
AuthorDate: Wed Mar 9 13:04:11 2022 +0100
HIVE-25977: Enhance Compaction Cleaner to skip when there is nothing to do
#2 (#2971) (Zoltan Haindrich reviewed by Karen Coppage and Denys Kuzmenko)
---
.../txn/compactor/TestCleanerWithReplication.java | 4 +-
.../hadoop/hive/ql/txn/compactor/Cleaner.java | 38 ++---
.../hive/ql/txn/compactor/CompactorTest.java | 33 ++--
.../hadoop/hive/ql/txn/compactor/TestCleaner.java | 95 ++
4 files changed, 148 insertions(+), 22 deletions(-)
diff --git
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCleanerWithReplication.java
b/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCleanerWithReplication.java
index 429d55c..6353b37 100644
---
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCleanerWithReplication.java
+++
b/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCleanerWithReplication.java
@@ -143,7 +143,7 @@ public class TestCleanerWithReplication extends
CompactorTest {
addDeltaFile(t, null, 23L, 24L, 2);
addDeltaFile(t, null, 21L, 24L, 4);
-burnThroughTransactions(dbName, "camitc", 25);
+burnThroughTransactions(dbName, "camitc", 24);
CompactionRequest rqst = new CompactionRequest(dbName, "camitc",
CompactionType.MINOR);
compactInTxn(rqst);
@@ -161,7 +161,7 @@ public class TestCleanerWithReplication extends
CompactorTest {
addDeltaFile(t, p, 23L, 24L, 2);
addDeltaFile(t, p, 21L, 24L, 4);
-burnThroughTransactions(dbName, "camipc", 25);
+burnThroughTransactions(dbName, "camipc", 24);
CompactionRequest rqst = new CompactionRequest(dbName, "camipc",
CompactionType.MINOR);
rqst.setPartitionname("ds=today");
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java
b/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java
index 1e0dbf8..55a7802 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java
@@ -17,7 +17,6 @@
*/
package org.apache.hadoop.hive.ql.txn.compactor;
-import com.google.common.annotations.VisibleForTesting;
import org.apache.hadoop.hive.common.ValidTxnList;
import org.apache.hadoop.hive.metastore.ReplChangeManager;
import org.apache.hadoop.hive.metastore.api.DataOperationType;
@@ -62,6 +61,7 @@ import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.metastore.txn.CompactionInfo;
import org.apache.hadoop.hive.ql.io.AcidUtils;
import org.apache.hadoop.hive.ql.io.AcidUtils.ParsedBaseLight;
+import org.apache.hadoop.hive.ql.io.AcidUtils.ParsedDelta;
import org.apache.hadoop.hive.ql.io.AcidUtils.ParsedDeltaLight;
import org.apache.hadoop.security.UserGroupInformation;
import org.apache.hadoop.util.StringUtils;
@@ -71,14 +71,15 @@ import java.io.IOException;
import java.security.PrivilegedExceptionAction;
import java.util.ArrayList;
import java.util.Collections;
+import java.util.HashSet;
import java.util.List;
import java.util.Optional;
import java.util.concurrent.Callable;
+import java.util.Set;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
-import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;
import static
org.apache.hadoop.hive.conf.Constants.COMPACTOR_CLEANER_THREAD_NAME_FORMAT;
@@ -87,8 +88,6 @@ import static
org.apache.hadoop.hive.conf.HiveConf.ConfVars.HIVE_COMPACTOR_DELAY
import static org.apache.hadoop.hive.metastore.HMSHandler.getMSForConf;
import static
org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.getDefaultCatalog;
-import com.codahale.metrics.Counter;
-
/**
* A class to clean directories after compactions. This will run in a
separate thread.
*/
@@ -425,8 +424,10 @@ public class Cleaner extends MetaStoreCompactorThread {
// Including obsolete directories for partitioned tables can result in
data loss.
obsoleteDirs = dir.getAbortedDirectories();
}
-if (obsoleteDirs.isEmpty() && !hasDataBelowWatermark(fs, path,
writeIdList.getHighWatermark())) {
- LOG.info(idWatermark(ci) + " nothing to remove below watermark " +
writeIdList.getHighWatermark() + ", ");
+
+if (obsoleteDirs.isEmpty()
+&& !hasDataBelowWatermark(dir, fs, path,
[hive] branch master updated: HIVE-26010: Compactions not picked up by cleaner if CQ_COMMIT_TIME is null and delayed cleaning is enabled (Laszlo Vegh, reviewed by Karen Coppage)
This is an automated email from the ASF dual-hosted git repository.
klcopp pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git
The following commit(s) were added to refs/heads/master by this push:
new 703a61f HIVE-26010: Compactions not picked up by cleaner if
CQ_COMMIT_TIME is null and delayed cleaning is enabled (Laszlo Vegh, reviewed
by Karen Coppage)
703a61f is described below
commit 703a61f2f6e474699162f26e02b77eaac18cc723
Author: veghlaci05 <90267982+veghlac...@users.noreply.github.com>
AuthorDate: Wed Mar 9 09:28:55 2022 +0100
HIVE-26010: Compactions not picked up by cleaner if CQ_COMMIT_TIME is null
and delayed cleaning is enabled (Laszlo Vegh, reviewed by Karen Coppage)
Closes #3080.
---
.../java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java
index 730b5ec..e691664 100644
---
a/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java
+++
b/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java
@@ -354,7 +354,7 @@ class CompactionTxnHandler extends TxnHandler {
whereClause += " AND (\"CQ_NEXT_TXN_ID\" <= " + minOpenTxnWaterMark
+ " OR \"CQ_NEXT_TXN_ID\" IS NULL)";
}
if (retentionTime > 0) {
- whereClause += " AND \"CQ_COMMIT_TIME\" < (" + getEpochFn(dbProduct)
+ " - " + retentionTime + ")";
+ whereClause += " AND (\"CQ_COMMIT_TIME\" < (" +
getEpochFn(dbProduct) + " - " + retentionTime + ") OR \"CQ_COMMIT_TIME\" IS
NULL)";
}
String s = "SELECT \"CQ_ID\", \"cq1\".\"CQ_DATABASE\",
\"cq1\".\"CQ_TABLE\", \"cq1\".\"CQ_PARTITION\"," +
" \"CQ_TYPE\", \"CQ_RUN_AS\", \"CQ_HIGHEST_WRITE_ID\",
\"CQ_TBLPROPERTIES\"" +
