[
https://issues.apache.org/jira/browse/HIVE-22931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ádám Szita updated HIVE-22931:
------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master, thx for the review Marti!
> HoS dynamic partitioning fails with blobstore optimizations off
> ---------------------------------------------------------------
>
> Key: HIVE-22931
> URL: https://issues.apache.org/jira/browse/HIVE-22931
> Project: Hive
> Issue Type: Bug
> Reporter: Ádám Szita
> Assignee: Ádám Szita
> Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22931.0.patch, HIVE-22931.1.patch,
> HIVE-22931.2.patch, HIVE-22931.3.patch
>
>
> Reproduction steps:
> - Create s3a backed table and normal table.
> {code:java}
> CREATE TABLE source (
> a string,
> b int,
> c int);
>
> CREATE TABLE target (
> a string)
> PARTITIONED BY (
> b int,
> c int)
> STORED AS parquet
> LOCATION
> 's3a://somepath';
> {code}
> - Insert values into normal table.
> {code:java}
> INSERT INTO TABLE source VALUES ("a", "1", "1");
> {code}
> - Do an insert overwrite with dynamic partitions:
> {code:java}
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.blobstore.optimizations.enabled=false;
> set hive.execution.engine=spark;
> INSERT OVERWRITE TABLE target partition (b,c)
> SELECT *
> FROM source;{code}
> This fails only with Spark execution engine + blobstorage optimizations being
> turned off with:
> {code}
> 2020-01-16 15:24:56,064 ERROR hive.ql.metadata.Hive:
> [load-dynamic-partitions-5]: Exception when loading partition with parameters
>
> partPath=hdfs://nameservice1/tmp/hive/hive/6bcee075-b637-429e-9bf0-a2658355415e/hive_2020-01-16_15-24-01_156_4299941251929377815-4/-mr-10000/.hive-staging_hive_2020-01-16_15-24-01_156_4299941251929377815-4/-ext-10002,
> table=email_click_base, partSpec={b=null, c=null}, replace=true,
> listBucketingEnabled=false, isAcid=false,
> hasFollowingStatsTask=trueorg.apache.hadoop.hive.ql.metadata.HiveException:
> MetaException(message:Partition spec is incorrect. {companyid=null,
> eventmonth=null})
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartitionInternal(Hive.java:1666)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
