This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-table-store.git
The following commit(s) were added to refs/heads/master by this push:
new f80659ba [FLINK-28157] Table Store Hive Reader supports Hive3
f80659ba is described below
commit f80659ba2219a4b70b83007f83fef71c304affa0
Author: Jingsong Lee <jingsongl...@gmail.com>
AuthorDate: Wed Oct 26 20:14:02 2022 +0800
[FLINK-28157] Table Store Hive Reader supports Hive3
This closes #329
---
.github/workflows/build-different-versions.yml | 3 +++
docs/content/docs/engines/build.md | 26 ++++++----------------
docs/content/docs/engines/hive.md | 5 +++--
docs/content/docs/engines/overview.md | 1 +
.../flink-table-store-hive-connector/pom.xml | 20 +++++++++++++++++
.../TableStoreDateObjectInspector.java | 0
.../TableStoreTimestampObjectInspector.java | 0
.../TableStoreDateObjectInspector.java | 17 +++++---------
.../TableStoreTimestampObjectInspector.java | 13 +++++------
.../store/hive/TableStoreHiveStorageHandler.java | 2 ++
.../hive/TableStoreHiveStorageHandlerITCase.java | 15 +++++++------
.../TableStoreDateObjectInspectorTest.java | 3 +++
.../TableStoreTimestampObjectInspectorTest.java | 11 +++++----
flink-table-store-hive/pom.xml | 12 ++++++++++
pom.xml | 3 ++-
tools/releasing/create_binary_release.sh | 5 +++++
16 files changed, 85 insertions(+), 51 deletions(-)
diff --git a/.github/workflows/build-different-versions.yml
b/.github/workflows/build-different-versions.yml
index 4d3cd22f..6ce7b590 100644
--- a/.github/workflows/build-different-versions.yml
+++ b/.github/workflows/build-different-versions.yml
@@ -16,6 +16,9 @@ jobs:
- name: Build Flink 1.15
run: |
./mvnw clean install -Dmaven.test.skip=true
+ - name: Build Hive 3.1
+ run: |
+ ./mvnw clean install -Dmaven.test.skip=true -Phive-3.1 -f
flink-table-store-hive
- name: Build Hive 2.2
run: |
./mvnw clean install -Dmaven.test.skip=true -Phive-2.2 -f
flink-table-store-hive
diff --git a/docs/content/docs/engines/build.md
b/docs/content/docs/engines/build.md
index b23fba0e..08c16d7f 100644
--- a/docs/content/docs/engines/build.md
+++ b/docs/content/docs/engines/build.md
@@ -48,25 +48,13 @@ You can find Flink 1.15 bundled jar in
`./flink-table-store-dist/target/flink-ta
## Hive
-To build with Hive 2.3, no special argument is needed.
-
-To build with Hive 2.2, run the following command.
-
-```bash
-mvn clean install -Dmaven.test.skip=true -Phive-2.2
-```
-
-To build with Hive 2.1, run the following command.
-
-```bash
-mvn clean install -Dmaven.test.skip=true -Phive-2.1
-```
-
-To build with Hive 2.1 CDH 6.3, run the following command.
-
-```bash
-mvn clean install -Dmaven.test.skip=true -Phive-2.1-cdh-6.3
-```
+| Version | Command
|
+|:-------------|:----------------------------------------------------------------|
+| 2.3 | mvn clean install -Dmaven.test.skip=true
|
+| 3.1 | mvn clean install -Dmaven.test.skip=true -Phive-3.1
|
+| 2.2 | mvn clean install -Dmaven.test.skip=true -Phive-2.2
|
+| 2.1 | mvn clean install -Dmaven.test.skip=true -Phive-2.1
|
+| 2.1 CDH 6.3 | mvn clean install -Dmaven.test.skip=true -Phive-2.1-cdh-6.3
|
You can find Hive catalog jar in
`./flink-table-store-hive/flink-table-store-hive-catalog/target/flink-table-store-hive-catalog-{{<
version >}}.jar`.
diff --git a/docs/content/docs/engines/hive.md
b/docs/content/docs/engines/hive.md
index b96ebf2b..c4869603 100644
--- a/docs/content/docs/engines/hive.md
+++ b/docs/content/docs/engines/hive.md
@@ -32,7 +32,7 @@ Table Store currently supports the following features related
with Hive:
## Version
-Table Store currently supports Hive 2.1, 2.2 and 2.3.
+Table Store currently supports Hive 2.1, 2.2, 2.3 and 3.1.
## Execution Engine
@@ -47,7 +47,7 @@ Download the jar file with corresponding version.
| |Jar|
|---|---|
-|Hive 2.3|[flink-table-store-hive-catalog-{{< version
>}}_2.3.jar](https://www.apache.org/dyn/closer.lua/flink/flink-table-store-{{<
version >}}/flink-table-store-hive-catalog-{{< version >}}_2.3.jar)|
+|Hive 2.3+|[flink-table-store-hive-catalog-{{< version
>}}_2.3.jar](https://www.apache.org/dyn/closer.lua/flink/flink-table-store-{{<
version >}}/flink-table-store-hive-catalog-{{< version >}}_2.3.jar)|
|Hive 2.2|[flink-table-store-hive-catalog-{{< version
>}}_2.2.jar](https://www.apache.org/dyn/closer.lua/flink/flink-table-store-{{<
version >}}/flink-table-store-hive-catalog-{{< version >}}_2.2.jar)|
|Hive 2.1|[flink-table-store-hive-catalog-{{< version
>}}_2.1.jar](https://www.apache.org/dyn/closer.lua/flink/flink-table-store-{{<
version >}}/flink-table-store-hive-catalog-{{< version >}}_2.1.jar)|
@@ -71,6 +71,7 @@ Download the jar file with corresponding version.
| |Jar|
|---|---|
+|Hive 3.1|[flink-table-store-hive-connector-{{< version
>}}_3.1.jar](https://www.apache.org/dyn/closer.lua/flink/flink-table-store-{{<
version >}}/flink-table-store-hive-connector-{{< version >}}_3.1.jar)|
|Hive 2.3|[flink-table-store-hive-connector-{{< version
>}}_2.3.jar](https://www.apache.org/dyn/closer.lua/flink/flink-table-store-{{<
version >}}/flink-table-store-hive-connector-{{< version >}}_2.3.jar)|
|Hive 2.2|[flink-table-store-hive-connector-{{< version
>}}_2.2.jar](https://www.apache.org/dyn/closer.lua/flink/flink-table-store-{{<
version >}}/flink-table-store-hive-connector-{{< version >}}_2.2.jar)|
|Hive 2.1|[flink-table-store-hive-connector-{{< version
>}}_2.1.jar](https://www.apache.org/dyn/closer.lua/flink/flink-table-store-{{<
version >}}/flink-table-store-hive-connector-{{< version >}}_2.1.jar)|
diff --git a/docs/content/docs/engines/overview.md
b/docs/content/docs/engines/overview.md
index ec3625a5..5a960f50 100644
--- a/docs/content/docs/engines/overview.md
+++ b/docs/content/docs/engines/overview.md
@@ -40,6 +40,7 @@ Apache Hive and Apache Spark.
| Hive | 2.1 CDH 6.3 | read
| Projection, Filter |
| Hive | 2.2 | read
| Projection, Filter |
| Hive | 2.3 | read
| Projection, Filter |
+| Hive | 3.1 | read
| Projection, Filter |
| Spark | 2.4 | read
| Projection, Filter |
| Spark | 3.0 | read
| Projection, Filter |
| Spark | 3.1 | read
| Projection, Filter |
diff --git a/flink-table-store-hive/flink-table-store-hive-connector/pom.xml
b/flink-table-store-hive/flink-table-store-hive-connector/pom.xml
index ee0d5fe4..aff36a59 100644
--- a/flink-table-store-hive/flink-table-store-hive-connector/pom.xml
+++ b/flink-table-store-hive/flink-table-store-hive-connector/pom.xml
@@ -461,6 +461,26 @@ under the License.
<build>
<plugins>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>build-helper-maven-plugin</artifactId>
+ <version>1.7</version>
+ <executions>
+ <execution>
+ <id>add-sources</id>
+ <phase>generate-sources</phase>
+ <goals>
+ <goal>add-source</goal>
+ </goals>
+ <configuration>
+ <sources>
+ <source>src/main/${hive.main.version}</source>
+ </sources>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
diff --git
a/flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspector.java
b/flink-table-store-hive/flink-table-store-hive-connector/src/main/2/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspector.java
similarity index 100%
copy from
flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspector.java
copy to
flink-table-store-hive/flink-table-store-hive-connector/src/main/2/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspector.java
diff --git
a/flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspector.java
b/flink-table-store-hive/flink-table-store-hive-connector/src/main/2/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspector.java
similarity index 100%
copy from
flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspector.java
copy to
flink-table-store-hive/flink-table-store-hive-connector/src/main/2/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspector.java
diff --git
a/flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspector.java
b/flink-table-store-hive/flink-table-store-hive-connector/src/main/3/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspector.java
similarity index 77%
rename from
flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspector.java
rename to
flink-table-store-hive/flink-table-store-hive-connector/src/main/3/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspector.java
index 2a5122fb..b64fcba3 100644
---
a/flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspector.java
+++
b/flink-table-store-hive/flink-table-store-hive-connector/src/main/3/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspector.java
@@ -18,15 +18,12 @@
package org.apache.flink.table.store.hive.objectinspector;
-import org.apache.flink.table.store.utils.DateTimeUtils;
-
-import org.apache.hadoop.hive.serde2.io.DateWritable;
+import org.apache.hadoop.hive.common.type.Date;
+import org.apache.hadoop.hive.serde2.io.DateWritableV2;
import
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveJavaObjectInspector;
import
org.apache.hadoop.hive.serde2.objectinspector.primitive.DateObjectInspector;
import org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory;
-import java.sql.Date;
-
/** {@link AbstractPrimitiveJavaObjectInspector} for DATE type. */
public class TableStoreDateObjectInspector extends
AbstractPrimitiveJavaObjectInspector
implements DateObjectInspector {
@@ -37,22 +34,20 @@ public class TableStoreDateObjectInspector extends
AbstractPrimitiveJavaObjectIn
@Override
public Date getPrimitiveJavaObject(Object o) {
- // Flink stores date as an integer (epoch day, 1970-01-01 = day 0)
- // while constructor of Date accepts epoch millis
- return o == null ? null : DateTimeUtils.toSQLDate((Integer) o);
+ return o == null ? null : Date.ofEpochDay((Integer) o);
}
@Override
- public DateWritable getPrimitiveWritableObject(Object o) {
+ public DateWritableV2 getPrimitiveWritableObject(Object o) {
Date date = getPrimitiveJavaObject(o);
- return date == null ? null : new DateWritable(date);
+ return date == null ? null : new DateWritableV2(date);
}
@Override
public Object copyObject(Object o) {
if (o instanceof Date) {
Date date = (Date) o;
- return new Date(date.getTime());
+ return date.clone();
} else {
return o;
}
diff --git
a/flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspector.java
b/flink-table-store-hive/flink-table-store-hive-connector/src/main/3/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspector.java
similarity index 83%
rename from
flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspector.java
rename to
flink-table-store-hive/flink-table-store-hive-connector/src/main/3/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspector.java
index 7b243b9b..7a7351b8 100644
---
a/flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspector.java
+++
b/flink-table-store-hive/flink-table-store-hive-connector/src/main/3/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspector.java
@@ -20,13 +20,12 @@ package org.apache.flink.table.store.hive.objectinspector;
import org.apache.flink.table.data.TimestampData;
-import org.apache.hadoop.hive.serde2.io.TimestampWritable;
+import org.apache.hadoop.hive.common.type.Timestamp;
+import org.apache.hadoop.hive.serde2.io.TimestampWritableV2;
import
org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveJavaObjectInspector;
import
org.apache.hadoop.hive.serde2.objectinspector.primitive.TimestampObjectInspector;
import org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory;
-import java.sql.Timestamp;
-
/** {@link AbstractPrimitiveJavaObjectInspector} for TIMESTAMP type. */
public class TableStoreTimestampObjectInspector extends
AbstractPrimitiveJavaObjectInspector
implements TimestampObjectInspector {
@@ -37,13 +36,13 @@ public class TableStoreTimestampObjectInspector extends
AbstractPrimitiveJavaObj
@Override
public Timestamp getPrimitiveJavaObject(Object o) {
- return o == null ? null : ((TimestampData) o).toTimestamp();
+ return o == null ? null : Timestamp.ofEpochMilli(((TimestampData)
o).getMillisecond());
}
@Override
- public TimestampWritable getPrimitiveWritableObject(Object o) {
+ public TimestampWritableV2 getPrimitiveWritableObject(Object o) {
Timestamp ts = getPrimitiveJavaObject(o);
- return ts == null ? null : new TimestampWritable(ts);
+ return ts == null ? null : new TimestampWritableV2(ts);
}
@Override
@@ -53,7 +52,7 @@ public class TableStoreTimestampObjectInspector extends
AbstractPrimitiveJavaObj
return o;
} else if (o instanceof Timestamp) {
Timestamp timestamp = (Timestamp) o;
- return new Timestamp(timestamp.getTime());
+ return timestamp.clone();
} else {
return o;
}
diff --git
a/flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/TableStoreHiveStorageHandler.java
b/flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/TableStoreHiveStorageHandler.java
index f93acfc9..533e79bc 100644
---
a/flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/TableStoreHiveStorageHandler.java
+++
b/flink-table-store-hive/flink-table-store-hive-connector/src/main/java/org/apache/flink/table/store/hive/TableStoreHiveStorageHandler.java
@@ -77,6 +77,8 @@ public class TableStoreHiveStorageHandler
TableStoreJobConf.configureInputJobProperties(properties, map);
}
+ public void configureInputJobCredentials(TableDesc tableDesc, Map<String,
String> map) {}
+
@Override
public void configureOutputJobProperties(TableDesc tableDesc, Map<String,
String> map) {}
diff --git
a/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/TableStoreHiveStorageHandlerITCase.java
b/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/TableStoreHiveStorageHandlerITCase.java
index 1301b5e9..3a7d785c 100644
---
a/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/TableStoreHiveStorageHandlerITCase.java
+++
b/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/TableStoreHiveStorageHandlerITCase.java
@@ -874,7 +874,7 @@ public class TableStoreHiveStorageHandlerITCase {
GenericRowData.of(
375, /* 1971-01-11 */
TimestampData.fromLocalDateTime(
- LocalDateTime.of(2022, 5, 17, 17, 29, 20))));
+ LocalDateTime.of(2022, 5, 17, 17, 29, 20,
100_000_000))));
commit.commit("0", write.prepareCommit(true));
write.write(GenericRowData.of(null, null));
commit.commit("1", write.prepareCommit(true));
@@ -882,7 +882,8 @@ public class TableStoreHiveStorageHandlerITCase {
write.write(
GenericRowData.of(
null,
- TimestampData.fromLocalDateTime(LocalDateTime.of(2022,
6, 18, 8, 30, 0))));
+ TimestampData.fromLocalDateTime(
+ LocalDateTime.of(2022, 6, 18, 8, 30, 0,
100_000_000))));
commit.commit("2", write.prepareCommit(true));
write.close();
@@ -894,18 +895,18 @@ public class TableStoreHiveStorageHandlerITCase {
"STORED BY '" +
TableStoreHiveStorageHandler.class.getName() + "'",
"LOCATION '" + path + "'")));
Assert.assertEquals(
- Collections.singletonList("1971-01-11\t2022-05-17 17:29:20.0"),
+ Collections.singletonList("1971-01-11\t2022-05-17 17:29:20.1"),
hiveShell.executeQuery("SELECT * FROM test_table WHERE dt =
'1971-01-11'"));
Assert.assertEquals(
- Collections.singletonList("1971-01-11\t2022-05-17 17:29:20.0"),
+ Collections.singletonList("1971-01-11\t2022-05-17 17:29:20.1"),
hiveShell.executeQuery(
- "SELECT * FROM test_table WHERE ts = '2022-05-17
17:29:20'"));
+ "SELECT * FROM test_table WHERE ts = '2022-05-17
17:29:20.1'"));
Assert.assertEquals(
Collections.singletonList("1971-01-12\tNULL"),
hiveShell.executeQuery("SELECT * FROM test_table WHERE dt =
'1971-01-12'"));
Assert.assertEquals(
- Collections.singletonList("NULL\t2022-06-18 08:30:00.0"),
+ Collections.singletonList("NULL\t2022-06-18 08:30:00.1"),
hiveShell.executeQuery(
- "SELECT * FROM test_table WHERE ts = '2022-06-18
08:30:00'"));
+ "SELECT * FROM test_table WHERE ts = '2022-06-18
08:30:00.1'"));
}
}
diff --git
a/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspectorTest.java
b/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspectorTest.java
index 4c232cd4..e83728ef 100644
---
a/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspectorTest.java
+++
b/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/objectinspector/TableStoreDateObjectInspectorTest.java
@@ -22,6 +22,7 @@ import org.apache.hadoop.hive.serde2.io.DateWritable;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.DisabledIfSystemProperty;
import java.sql.Date;
import java.time.LocalDate;
@@ -31,6 +32,7 @@ import static org.assertj.core.api.Assertions.assertThat;
/** Tests for {@link TableStoreDateObjectInspector}. */
public class TableStoreDateObjectInspectorTest {
+ @DisabledIfSystemProperty(named = "hive.main.version", matches = "3")
@Test
public void testCategoryAndClass() {
TableStoreDateObjectInspector oi = new TableStoreDateObjectInspector();
@@ -61,6 +63,7 @@ public class TableStoreDateObjectInspectorTest {
assertThat(oi.getPrimitiveWritableObject(null)).isNull();
}
+ @DisabledIfSystemProperty(named = "hive.main.version", matches = "3")
@Test
public void testCopyObject() {
TableStoreDateObjectInspector oi = new TableStoreDateObjectInspector();
diff --git
a/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspectorTest.java
b/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspectorTest.java
index ba1f6759..e307219d 100644
---
a/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspectorTest.java
+++
b/flink-table-store-hive/flink-table-store-hive-connector/src/test/java/org/apache/flink/table/store/hive/objectinspector/TableStoreTimestampObjectInspectorTest.java
@@ -24,6 +24,7 @@ import org.apache.hadoop.hive.serde2.io.TimestampWritable;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.DisabledIfSystemProperty;
import java.sql.Timestamp;
import java.time.LocalDateTime;
@@ -33,6 +34,7 @@ import static org.assertj.core.api.Assertions.assertThat;
/** Tests for {@link TableStoreTimestampObjectInspector}. */
public class TableStoreTimestampObjectInspectorTest {
+ @DisabledIfSystemProperty(named = "hive.main.version", matches = "3")
@Test
public void testCategoryAndClass() {
TableStoreTimestampObjectInspector oi = new
TableStoreTimestampObjectInspector();
@@ -49,9 +51,9 @@ public class TableStoreTimestampObjectInspectorTest {
public void testGetPrimitiveJavaObject() {
TableStoreTimestampObjectInspector oi = new
TableStoreTimestampObjectInspector();
- LocalDateTime local = LocalDateTime.of(2022, 4, 27, 15, 0, 0);
+ LocalDateTime local = LocalDateTime.of(2022, 4, 27, 15, 0, 0,
100_000_000);
TimestampData input = TimestampData.fromLocalDateTime(local);
-
assertThat(oi.getPrimitiveJavaObject(input).toString()).isEqualTo("2022-04-27
15:00:00.0");
+
assertThat(oi.getPrimitiveJavaObject(input).toString()).isEqualTo("2022-04-27
15:00:00.1");
assertThat(oi.getPrimitiveJavaObject(null)).isNull();
}
@@ -59,13 +61,14 @@ public class TableStoreTimestampObjectInspectorTest {
public void testGetPrimitiveWritableObject() {
TableStoreTimestampObjectInspector oi = new
TableStoreTimestampObjectInspector();
- LocalDateTime local = LocalDateTime.of(2022, 4, 27, 15, 0, 0);
+ LocalDateTime local = LocalDateTime.of(2022, 4, 27, 15, 0, 0,
100_000_000);
TimestampData input = TimestampData.fromLocalDateTime(local);
assertThat(oi.getPrimitiveWritableObject(input).getTimestamp().toString())
- .isEqualTo("2022-04-27 15:00:00.0");
+ .isEqualTo("2022-04-27 15:00:00.1");
assertThat(oi.getPrimitiveWritableObject(null)).isNull();
}
+ @DisabledIfSystemProperty(named = "hive.main.version", matches = "3")
@Test
public void testCopyObject() {
TableStoreTimestampObjectInspector oi = new
TableStoreTimestampObjectInspector();
diff --git a/flink-table-store-hive/pom.xml b/flink-table-store-hive/pom.xml
index 0c81be0d..db120127 100644
--- a/flink-table-store-hive/pom.xml
+++ b/flink-table-store-hive/pom.xml
@@ -40,6 +40,7 @@ under the License.
</modules>
<properties>
+ <hive.main.version>2</hive.main.version>
<hive.version>2.3.4</hive.version>
<hiverunner.version>4.0.0</hiverunner.version>
<reflections.version>0.9.8</reflections.version>
@@ -49,6 +50,7 @@ under the License.
<profile>
<id>hive-2.1</id>
<properties>
+ <hive.main.version>2</hive.main.version>
<hive.version>2.1.1</hive.version>
</properties>
</profile>
@@ -56,6 +58,7 @@ under the License.
<profile>
<id>hive-2.2</id>
<properties>
+ <hive.main.version>2</hive.main.version>
<hive.version>2.2.0</hive.version>
</properties>
</profile>
@@ -63,6 +66,7 @@ under the License.
<profile>
<id>hive-2.1-cdh-6.3</id>
<properties>
+ <hive.main.version>2</hive.main.version>
<hive.version>2.1.1-cdh6.3.4</hive.version>
</properties>
<repositories>
@@ -72,6 +76,14 @@ under the License.
</repository>
</repositories>
</profile>
+
+ <profile>
+ <id>hive-3.1</id>
+ <properties>
+ <hive.main.version>3</hive.main.version>
+ <hive.version>3.1.2</hive.version>
+ </properties>
+ </profile>
</profiles>
</project>
\ No newline at end of file
diff --git a/pom.xml b/pom.xml
index ec8d17c2..abe5eed7 100644
--- a/pom.xml
+++ b/pom.xml
@@ -431,6 +431,7 @@ under the License.
<!--suppress MavenModelInspection -->
<test.randomization.seed>${test.randomization.seed}</test.randomization.seed>
<flink.version>${flink.version}</flink.version>
+
<hive.main.version>${hive.main.version}</hive.main.version>
</systemPropertyVariables>
<argLine>-Xms256m -Xmx2048m
-Dmvn.forkNumber=${surefire.forkNumber} -XX:+UseG1GC</argLine>
</configuration>
@@ -519,7 +520,7 @@ under the License.
<rules>
<bannedDependencies>
<excludes>
-
<exclude>com.fasterxml.jackson*:*:(,2.9.0]</exclude>
+
<exclude>com.fasterxml.jackson*:*:(,2.7.0]</exclude>
</excludes>
</bannedDependencies>
</rules>
diff --git a/tools/releasing/create_binary_release.sh
b/tools/releasing/create_binary_release.sh
index af13c955..0449c7a3 100755
--- a/tools/releasing/create_binary_release.sh
+++ b/tools/releasing/create_binary_release.sh
@@ -62,6 +62,9 @@ cp
flink-table-store-hive/flink-table-store-hive-connector/target/flink-table-st
cp
flink-table-store-spark/target/flink-table-store-spark-${RELEASE_VERSION}.jar
${RELEASE_DIR}
cp
flink-table-store-spark2/target/flink-table-store-spark2-${RELEASE_VERSION}.jar
${RELEASE_DIR}
+mvn clean install -Dcheckstyle.skip=true -Dgpg.skip -DskipTests -Phive-3.1 -f
flink-table-store-hive
+cp
flink-table-store-hive/flink-table-store-hive-connector/target/flink-table-store-hive-connector-${RELEASE_VERSION}.jar
${RELEASE_DIR}/flink-table-store-hive-connector-${RELEASE_VERSION}_3.1.jar
+
mvn clean install -Dcheckstyle.skip=true -Dgpg.skip -DskipTests -Phive-2.2 -f
flink-table-store-hive
cp
flink-table-store-hive/flink-table-store-hive-catalog/target/flink-table-store-hive-catalog-${RELEASE_VERSION}.jar
${RELEASE_DIR}/flink-table-store-hive-catalog-${RELEASE_VERSION}_2.2.jar
cp
flink-table-store-hive/flink-table-store-hive-connector/target/flink-table-store-hive-connector-${RELEASE_VERSION}.jar
${RELEASE_DIR}/flink-table-store-hive-connector-${RELEASE_VERSION}_2.2.jar
@@ -78,6 +81,7 @@ gpg --armor --detach-sig
"flink-table-store-dist-${RELEASE_VERSION}.jar"
gpg --armor --detach-sig
"flink-table-store-hive-catalog-${RELEASE_VERSION}_2.3.jar"
gpg --armor --detach-sig
"flink-table-store-hive-catalog-${RELEASE_VERSION}_2.2.jar"
gpg --armor --detach-sig
"flink-table-store-hive-catalog-${RELEASE_VERSION}_2.1.jar"
+gpg --armor --detach-sig
"flink-table-store-hive-connector-${RELEASE_VERSION}_3.1.jar"
gpg --armor --detach-sig
"flink-table-store-hive-connector-${RELEASE_VERSION}_2.3.jar"
gpg --armor --detach-sig
"flink-table-store-hive-connector-${RELEASE_VERSION}_2.2.jar"
gpg --armor --detach-sig
"flink-table-store-hive-connector-${RELEASE_VERSION}_2.1.jar"
@@ -89,6 +93,7 @@ $SHASUM "flink-table-store-dist-${RELEASE_VERSION}.jar" >
"flink-table-store-dis
$SHASUM "flink-table-store-hive-catalog-${RELEASE_VERSION}_2.3.jar" >
"flink-table-store-hive-catalog-${RELEASE_VERSION}_2.3.jar.sha512"
$SHASUM "flink-table-store-hive-catalog-${RELEASE_VERSION}_2.2.jar" >
"flink-table-store-hive-catalog-${RELEASE_VERSION}_2.2.jar.sha512"
$SHASUM "flink-table-store-hive-catalog-${RELEASE_VERSION}_2.1.jar" >
"flink-table-store-hive-catalog-${RELEASE_VERSION}_2.1.jar.sha512"
+$SHASUM "flink-table-store-hive-connector-${RELEASE_VERSION}_3.1.jar" >
"flink-table-store-hive-connector-${RELEASE_VERSION}_3.1.jar.sha512"
$SHASUM "flink-table-store-hive-connector-${RELEASE_VERSION}_2.3.jar" >
"flink-table-store-hive-connector-${RELEASE_VERSION}_2.3.jar.sha512"
$SHASUM "flink-table-store-hive-connector-${RELEASE_VERSION}_2.2.jar" >
"flink-table-store-hive-connector-${RELEASE_VERSION}_2.2.jar.sha512"
$SHASUM "flink-table-store-hive-connector-${RELEASE_VERSION}_2.1.jar" >
"flink-table-store-hive-connector-${RELEASE_VERSION}_2.1.jar.sha512"
