[
https://issues.apache.org/jira/browse/HIVE-25179?focusedWorklogId=607821&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-607821
]
ASF GitHub Bot logged work on HIVE-25179:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Jun/21 12:24
Start Date: 07/Jun/21 12:24
Worklog Time Spent: 10m
Work Description: pvary commented on a change in pull request #2333:
URL: https://github.com/apache/hive/pull/2333#discussion_r646538065
##########
File path:
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java
##########
@@ -46,27 +50,60 @@ private IcebergTableUtil() {
* @return an Iceberg table
*/
static Table getTable(Configuration configuration, Properties properties) {
- Table table = null;
- QueryState queryState = null;
String tableIdentifier = properties.getProperty(Catalogs.NAME);
- if (SessionState.get() != null) {
- queryState =
SessionState.get().getQueryState(configuration.get(HiveConf.ConfVars.HIVEQUERYID.varname));
- if (queryState != null) {
- table = (Table) queryState.getResource(tableIdentifier);
- } else {
- LOG.debug("QueryState is not available in SessionState. Loading {}
from configured catalog.", tableIdentifier);
- }
- } else {
- LOG.debug("SessionState is not available. Loading {} from configured
catalog.", tableIdentifier);
- }
+ return SessionStateUtil.getResource(configuration,
tableIdentifier).filter(o -> o instanceof Table)
+ .map(o -> (Table) o).orElseGet(() -> {
+ LOG.debug("Iceberg table {} is not found in QueryState. Loading
table from configured catalog",
+ tableIdentifier);
+ Table tab = Catalogs.loadTable(configuration, properties);
+ SessionStateUtil.addResource(configuration, tableIdentifier, tab);
+ return tab;
+ });
+ }
- if (table == null) {
- table = Catalogs.loadTable(configuration, properties);
- if (queryState != null) {
- queryState.addResource(tableIdentifier, table);
- }
+ /**
+ * Create {@link PartitionSpec} based on the partition information stored in
+ * {@link
org.apache.hadoop.hive.ql.parse.PartitionTransform.PartitionTransformSpec}.
+ * @param configuration a Hadoop configuration
+ * @param schema iceberg table schema
+ * @return iceberg partition spec, always non-null
+ */
+ public static PartitionSpec spec(Configuration configuration, Schema schema)
{
+ List<PartitionTransform.PartitionTransformSpec> partitionTransformSpecList
= SessionStateUtil
+ .getResource(configuration,
hive_metastoreConstants.PARTITION_TRANSFORM_SPEC)
+ .map(o -> (List<PartitionTransform.PartitionTransformSpec>)
o).orElseGet(() -> null);
+
+ if (partitionTransformSpecList == null) {
+ LOG.debug("Iceberg partition transform spec is not found in
QueryState.");
Review comment:
I think this log line will be printed every time when we create or alter
a table and there is no partition spec.
This could be inconvenient
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 607821)
Time Spent: 4h 50m (was: 4h 40m)
> Support all partition transforms for Iceberg in create table
> ------------------------------------------------------------
>
> Key: HIVE-25179
> URL: https://issues.apache.org/jira/browse/HIVE-25179
> Project: Hive
> Issue Type: New Feature
> Reporter: László Pintér
> Assignee: László Pintér
> Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Enhance table create syntax with support to partition transforms:
> {code:sql}
> CREATE TABLE ... PARTITIONED BY SPEC( year(year_field), month(month_field),
> day(day_field), hour(hour_field), truncate(3, truncate_field), bucket(5,
> bucket_field bucket), identity_field ) STORED BY ICEBERG;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
