[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[ https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=603992=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-603992 ] ASF GitHub Bot logged work on HIVE-25162: - Author: ASF GitHub Bot Created on: 31/May/21 07:37 Start Date: 31/May/21 07:37 Worklog Time Spent: 10m Work Description: lcspinter merged pull request #2317: URL: https://github.com/apache/hive/pull/2317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 603992) Time Spent: 2h 40m (was: 2.5h) > Add support for CREATE TABLE ... STORED BY ICEBERG statements > - > > Key: HIVE-25162 > URL: https://issues.apache.org/jira/browse/HIVE-25162 > Project: Hive > Issue Type: New Feature >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=603843=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-603843
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 29/May/21 17:27
Start Date: 29/May/21 17:27
Worklog Time Spent: 10m
Work Description: lcspinter commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r641963384
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java
##
@@ -93,6 +106,14 @@ public boolean fillStorageFormat(ASTNode child) throws
SemanticException {
return true;
}
+ private String processStorageHandler(String name) throws SemanticException {
+if (StorageHandlerTypes.ICEBERG.name().equalsIgnoreCase(name)) {
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 603843)
Time Spent: 2.5h (was: 2h 20m)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=603842=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-603842
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 29/May/21 17:01
Start Date: 29/May/21 17:01
Worklog Time Spent: 10m
Work Description: lcspinter commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r641960742
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##
@@ -462,7 +493,7 @@ public void testCreateTableWithNotSupportedTypes() {
"Unsupported Hive type", () -> {
shell.executeStatement("CREATE EXTERNAL TABLE not_supported_types
" +
"(not_supported " + notSupportedType + ") " +
-"STORED BY
'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler' " +
+"STORED BY ICEBERG " +
Review comment:
Yes, we have plenty of other test cases (the ones calling the
`TestTables#createTable()`) using the storage handler syntax.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 603842)
Time Spent: 2h 20m (was: 2h 10m)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[ https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=603841=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-603841 ] ASF GitHub Bot logged work on HIVE-25162: - Author: ASF GitHub Bot Created on: 29/May/21 17:00 Start Date: 29/May/21 17:00 Worklog Time Spent: 10m Work Description: lcspinter commented on a change in pull request #2317: URL: https://github.com/apache/hive/pull/2317#discussion_r641960607 ## File path: iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java ## @@ -333,6 +334,12 @@ private static Properties getCatalogProperties(org.apache.hadoop.hive.metastore. properties.put(Catalogs.NAME, TableIdentifier.of(hmsTable.getDbName(), hmsTable.getTableName()).toString()); } +hmsTable.getSd().getSerdeInfo().getParameters().entrySet().stream() Review comment: Good point. Added null check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 603841) Time Spent: 2h 10m (was: 2h) > Add support for CREATE TABLE ... STORED BY ICEBERG statements > - > > Key: HIVE-25162 > URL: https://issues.apache.org/jira/browse/HIVE-25162 > Project: Hive > Issue Type: New Feature >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602785=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602785
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 27/May/21 08:02
Start Date: 27/May/21 08:02
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r640381488
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java
##
@@ -93,6 +106,14 @@ public boolean fillStorageFormat(ASTNode child) throws
SemanticException {
return true;
}
+ private String processStorageHandler(String name) throws SemanticException {
+if (StorageHandlerTypes.ICEBERG.name().equalsIgnoreCase(name)) {
Review comment:
I think the code change to make it generic would be very small, probably
a 4-line for loop, so we might as well add it into this PR, but I don't feel
strongly, just a suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602785)
Time Spent: 2h (was: 1h 50m)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602780=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602780
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 27/May/21 07:58
Start Date: 27/May/21 07:58
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r640378772
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java
##
@@ -93,6 +106,14 @@ public boolean fillStorageFormat(ASTNode child) throws
SemanticException {
return true;
}
+ private String processStorageHandler(String name) throws SemanticException {
+if (StorageHandlerTypes.ICEBERG.name().equalsIgnoreCase(name)) {
Review comment:
I was thinking about HBase / Kudu / Druid, but that probably merits
another PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602780)
Time Spent: 1h 50m (was: 1h 40m)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602777=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602777
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 27/May/21 07:52
Start Date: 27/May/21 07:52
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r640374426
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java
##
@@ -93,6 +106,14 @@ public boolean fillStorageFormat(ASTNode child) throws
SemanticException {
return true;
}
+ private String processStorageHandler(String name) throws SemanticException {
+if (StorageHandlerTypes.ICEBERG.name().equalsIgnoreCase(name)) {
Review comment:
Shall we make this generic so it works for future StorageHandlerTypes
types? i.e. iterate through the enum values and if `name` matches any enum
value, then reset the `name` with `enum.name()` accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602777)
Time Spent: 1h 40m (was: 1.5h)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602772=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602772
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 27/May/21 07:47
Start Date: 27/May/21 07:47
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r640370818
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##
@@ -462,7 +493,7 @@ public void testCreateTableWithNotSupportedTypes() {
"Unsupported Hive type", () -> {
shell.executeStatement("CREATE EXTERNAL TABLE not_supported_types
" +
"(not_supported " + notSupportedType + ") " +
-"STORED BY
'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler' " +
+"STORED BY ICEBERG " +
Review comment:
Did we leave some test cases with the orginal syntax `STORED BY
'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'` to make sure it's still
working?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602772)
Time Spent: 1.5h (was: 1h 20m)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[ https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602769=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602769 ] ASF GitHub Bot logged work on HIVE-25162: - Author: ASF GitHub Bot Created on: 27/May/21 07:44 Start Date: 27/May/21 07:44 Worklog Time Spent: 10m Work Description: marton-bod commented on a change in pull request #2317: URL: https://github.com/apache/hive/pull/2317#discussion_r640368683 ## File path: iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java ## @@ -333,6 +334,12 @@ private static Properties getCatalogProperties(org.apache.hadoop.hive.metastore. properties.put(Catalogs.NAME, TableIdentifier.of(hmsTable.getDbName(), hmsTable.getTableName()).toString()); } +hmsTable.getSd().getSerdeInfo().getParameters().entrySet().stream() Review comment: `SerdeInfo::getParameters` has a `@Nullable` annotation. Shall we guard against that case to be safe? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 602769) Time Spent: 1h 20m (was: 1h 10m) > Add support for CREATE TABLE ... STORED BY ICEBERG statements > - > > Key: HIVE-25162 > URL: https://issues.apache.org/jira/browse/HIVE-25162 > Project: Hive > Issue Type: New Feature >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602299=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602299
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 26/May/21 12:37
Start Date: 26/May/21 12:37
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r639683528
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##
@@ -587,6 +618,7 @@ public void testIcebergAndHmsTableProperties() throws
Exception {
expectedIcebergProperties.put("custom_property", "initial_val");
expectedIcebergProperties.put("EXTERNAL", "TRUE");
expectedIcebergProperties.put("storage_handler",
HiveIcebergStorageHandler.class.getName());
+expectedIcebergProperties.put(serdeConstants.SERIALIZATION_FORMAT, "1");
Review comment:
Thanks for the explanation!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602299)
Time Spent: 1h 10m (was: 1h)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602296=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602296
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 26/May/21 12:35
Start Date: 26/May/21 12:35
Worklog Time Spent: 10m
Work Description: lcspinter commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r639681461
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##
@@ -587,6 +618,7 @@ public void testIcebergAndHmsTableProperties() throws
Exception {
expectedIcebergProperties.put("custom_property", "initial_val");
expectedIcebergProperties.put("EXTERNAL", "TRUE");
expectedIcebergProperties.put("storage_handler",
HiveIcebergStorageHandler.class.getName());
+expectedIcebergProperties.put(serdeConstants.SERIALIZATION_FORMAT, "1");
Review comment:
In this PR, I also added a new logic to the `IcebergMetaHook` to copy
the serdeproperties from HMS table to the catalog properties. This change was
required since the hive syntax allows to provide additional serdeproperties
when creating non-native tables. Like:
`CREATE TABLE STORED BY ICEBERG WITH
SERDEPROPERTIES('my_key'='my_value')`
When we create a table, hive automatically updates the serdeproperties with
the `SERIALIZATION_FORMAT` set to `1`( if it's not overwritten from the DDL).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602296)
Time Spent: 1h (was: 50m)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602295=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602295
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 26/May/21 12:33
Start Date: 26/May/21 12:33
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r639680280
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##
@@ -239,12 +240,42 @@ public void testCreateDropTableNonDefaultCatalog() throws
TException, Interrupte
);
}
+ @Test
+ public void testCreateTableStoredByIceberg() {
+TableIdentifier identifier = TableIdentifier.of("default", "customers");
+String query = String.format("CREATE EXTERNAL TABLE customers (customer_id
BIGINT, first_name STRING, last_name " +
+"STRING) STORED BY %s %s TBLPROPERTIES ('%s'='%s')",
+"iceBeRg",
Review comment:
Probably we should not use `%s` here, just substitute the string by hand
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##
@@ -239,12 +240,42 @@ public void testCreateDropTableNonDefaultCatalog() throws
TException, Interrupte
);
}
+ @Test
+ public void testCreateTableStoredByIceberg() {
+TableIdentifier identifier = TableIdentifier.of("default", "customers");
+String query = String.format("CREATE EXTERNAL TABLE customers (customer_id
BIGINT, first_name STRING, last_name " +
+"STRING) STORED BY %s %s TBLPROPERTIES ('%s'='%s')",
+"iceBeRg",
+testTables.locationForCreateTableSQL(identifier),
+InputFormatConfig.CATALOG_NAME,
+Catalogs.ICEBERG_DEFAULT_CATALOG_NAME);
+shell.executeStatement(query);
+Assert.assertNotNull(testTables.loadTable(identifier));
+ }
+
+ @Test
+ public void testCreateTableStoredByIcebergWithSerdeProperties() {
+TableIdentifier identifier = TableIdentifier.of("default", "customers");
+String query = String.format("CREATE EXTERNAL TABLE customers (customer_id
BIGINT, first_name STRING, last_name " +
+"STRING) STORED BY %s WITH SERDEPROPERTIES('%s'='%s') %s
TBLPROPERTIES ('%s'='%s')",
+"iceberg",
Review comment:
Probably we should not use `%s` here, just substitute the string by hand
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602295)
Time Spent: 50m (was: 40m)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602292=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602292
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 26/May/21 12:25
Start Date: 26/May/21 12:25
Worklog Time Spent: 10m
Work Description: lcspinter commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r639674283
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##
@@ -239,6 +240,36 @@ public void testCreateDropTableNonDefaultCatalog() throws
TException, Interrupte
);
}
+ @Test
+ public void testCreateTableStoredByIceberg() {
+TableIdentifier identifier = TableIdentifier.of("default", "customers");
+String query = String.format("CREATE EXTERNAL TABLE customers (customer_id
BIGINT, first_name STRING, last_name " +
+"STRING) STORED BY %s %s TBLPROPERTIES ('%s'='%s')",
+"ICEBERG",
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602292)
Time Spent: 40m (was: 0.5h)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602202=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602202
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 26/May/21 09:24
Start Date: 26/May/21 09:24
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r639553457
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##
@@ -239,6 +240,36 @@ public void testCreateDropTableNonDefaultCatalog() throws
TException, Interrupte
);
}
+ @Test
+ public void testCreateTableStoredByIceberg() {
+TableIdentifier identifier = TableIdentifier.of("default", "customers");
+String query = String.format("CREATE EXTERNAL TABLE customers (customer_id
BIGINT, first_name STRING, last_name " +
+"STRING) STORED BY %s %s TBLPROPERTIES ('%s'='%s')",
+"ICEBERG",
Review comment:
I would change most of the test to `STORED BY ICEBERG` and keep only a
few with the original class name.
Also I would like to try out `stored by iCeBerG`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602202)
Time Spent: 20m (was: 10m)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[
https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=602203=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-602203
]
ASF GitHub Bot logged work on HIVE-25162:
-
Author: ASF GitHub Bot
Created on: 26/May/21 09:24
Start Date: 26/May/21 09:24
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2317:
URL: https://github.com/apache/hive/pull/2317#discussion_r639554024
##
File path:
iceberg/iceberg-handler/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java
##
@@ -587,6 +618,7 @@ public void testIcebergAndHmsTableProperties() throws
Exception {
expectedIcebergProperties.put("custom_property", "initial_val");
expectedIcebergProperties.put("EXTERNAL", "TRUE");
expectedIcebergProperties.put("storage_handler",
HiveIcebergStorageHandler.class.getName());
+expectedIcebergProperties.put(serdeConstants.SERIALIZATION_FORMAT, "1");
Review comment:
Why is this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 602203)
Time Spent: 0.5h (was: 20m)
> Add support for CREATE TABLE ... STORED BY ICEBERG statements
> -
>
> Key: HIVE-25162
> URL: https://issues.apache.org/jira/browse/HIVE-25162
> Project: Hive
> Issue Type: New Feature
>Reporter: László Pintér
>Assignee: László Pintér
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25162) Add support for CREATE TABLE ... STORED BY ICEBERG statements
[ https://issues.apache.org/jira/browse/HIVE-25162?focusedWorklogId=601729=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-601729 ] ASF GitHub Bot logged work on HIVE-25162: - Author: ASF GitHub Bot Created on: 25/May/21 13:39 Start Date: 25/May/21 13:39 Worklog Time Spent: 10m Work Description: lcspinter opened a new pull request #2317: URL: https://github.com/apache/hive/pull/2317 ### What changes were proposed in this pull request? This PR introduces syntactic shortcut to allow the creation of Iceberg tables without providing the handler class `CREATE TABLE ice_t (a int) STORED BY ICEBERG [WITH SERDEPROPERTIES (...)]` ### Why are the changes needed? At the moment, when we want to create an Iceberg table, we have to provide the fully qualified iceberg storage handler class name. `CREATE TABLE ice_t (a int) STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler' [WITH SERDEPROPERTIES (...)]` ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Unit test, q test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 601729) Remaining Estimate: 0h Time Spent: 10m > Add support for CREATE TABLE ... STORED BY ICEBERG statements > - > > Key: HIVE-25162 > URL: https://issues.apache.org/jira/browse/HIVE-25162 > Project: Hive > Issue Type: New Feature >Reporter: László Pintér >Assignee: László Pintér >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
