Ádám Szita created HIVE-22931:
---------------------------------
Summary: HoS dynamic partitioning fails with blobstore
optimizations off
Key: HIVE-22931
URL: https://issues.apache.org/jira/browse/HIVE-22931
Project: Hive
Issue Type: Bug
Reporter: Ádám Szita
Assignee: Ádám Szita
Reproduction steps:
- Create s3a backed table and normal table.
{code:java}
CREATE TABLE source (
a string,
b int,
c int);
CREATE TABLE target (
a string)
PARTITIONED BY (
b int,
c int)
STORED AS parquet
LOCATION
's3a://somepath';
{code}
- Insert values into normal table.
{code:java}
INSERT INTO TABLE source VALUES ("a", "1", "1");
{code}
- Do an insert overwrite with dynamic partitions:
{code:java}
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.blobstore.optimizations.enabled=false;
set hive.execution.engine=spark;
INSERT OVERWRITE TABLE target partition (b,c)
SELECT *
FROM source;{code}
This fails only with Spark execution engine + blobstorage optimizations being
turned off with:
2020-01-16 15:24:56,064 ERROR hive.ql.metadata.Hive:
[load-dynamic-partitions-5]: Exception when loading partition with parameters
partPath=hdfs://nameservice1/tmp/hive/hive/6bcee075-b637-429e-9bf0-a2658355415e/hive_2020-01-16_15-24-01_156_4299941251929377815-4/-mr-10000/.hive-staging_hive_2020-01-16_15-24-01_156_4299941251929377815-4/-ext-10002,
table=email_click_base, partSpec={b=null, c=null}, replace=true,
listBucketingEnabled=false, isAcid=false,
hasFollowingStatsTask=trueorg.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Partition spec is incorrect. {companyid=null,
eventmonth=null})
at
org.apache.hadoop.hive.ql.metadata.Hive.loadPartitionInternal(Hive.java:1666)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
