gavin_shi created HIVE-20064:
--------------------------------

             Summary: Data cannot be inserted into a dynamic partition
                 Key: HIVE-20064
                 URL: https://issues.apache.org/jira/browse/HIVE-20064
             Project: Hive
          Issue Type: Bug
          Components: SQL
            Reporter: gavin_shi
            Assignee: gavin_shi


Data cannot be inserted into a dynamic partition,The relevant table information 
is as follows:

The source table:
{code:java}
//代码占位符
create table customer(id string,name string,gender string,state string) 
partitioned by(country string);

{code}
The source table data:

 
{code:java}
//代码占位符
+--------------+----------------+------------------+-----------------+-------------------+--+
| customer.id | customer.name | customer.gender | customer.state | 
customer.country |
+--------------+----------------+------------------+-----------------+-------------------+--+
| 1 | gavin | 1 | shanghai | china |
+--------------+----------------+------------------+-----------------+-------------------+--+

{code}
 

The target table:

 
{code:java}
//代码占位符
create table customer_bak(id string,name string,gender string,state string) 
partitioned by(country string);

{code}
 

Now I use dynamic techniques to insert data into the customer_bak table:
{code:java}
//代码占位符
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
insert into table customer_bak partition(country) select 
id,name,gender,state,country from customer;
{code}
The program is always executing and neither success nor failure is shown. The 
information in the hive log is as follows:
{code:java}
//代码占位符

2018-06-28T16:01:22,900 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Driver: Executing 
command(queryId=hadoop_20180628160122_f80c5f8e-98a0-4286-bc55-c25efd7a60bc): 
insert into table customer_bak partition(country) select 
id,name,gender,state,country from customer
2018-06-28T16:01:22,901 WARN [HiveServer2-Background-Pool: Thread-97] 
ql.Driver: Hive-on-MR is deprecated in Hive 2 and may not be available in the 
future versions. Consider using a different execution engine (i.e. tez, spark) 
or using Hive 1.X releases.
2018-06-28T16:01:22,902 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Driver: WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available 
in the future versions. Consider using a different execution engine (i.e. tez, 
spark) or using Hive 1.X releases.
2018-06-28T16:01:22,902 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Driver: Query ID = hadoop_20180628160122_f80c5f8e-98a0-4286-bc55-c25efd7a60bc
2018-06-28T16:01:22,903 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Driver: Total jobs = 3
2018-06-28T16:01:22,904 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Driver: Launching Job 1 out of 3
2018-06-28T16:01:22,907 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Driver: Starting task [Stage-1:MAPRED] in serial mode
2018-06-28T16:01:22,908 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: Number of reduce tasks is set to 0 since there's no reduce operator
2018-06-28T16:01:22,929 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Context: New scratch dir is 
hdfs://hadoop-cluster/tmp/hive/hadoop/07fd131c-6590-4081-b7ed-2f0ae5b16551/hive_2018-06-28_16-01-22_402_5116918946052746369-3
2018-06-28T16:01:22,942 INFO [HiveServer2-Background-Pool: Thread-97] 
mr.ExecDriver: Using org.apache.hadoop.hive.ql.io.CombineHiveInputFormat
2018-06-28T16:01:22,943 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Utilities: Processing alias customer
2018-06-28T16:01:22,943 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Utilities: Adding 2 inputs; the first input is 
hdfs://hadoop-cluster/user/hive/warehouse/customer/country=china
2018-06-28T16:01:22,943 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Utilities: Content Summary not cached for 
hdfs://hadoop-cluster/user/hive/warehouse/customer/country=china
2018-06-28T16:01:22,945 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Utilities: Content Summary not cached for 
hdfs://hadoop-cluster/user/hive/warehouse/customer/country=usa
2018-06-28T16:01:22,978 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Utilities: Changed input file 
hdfs://hadoop-cluster/user/hive/warehouse/customer/country=usa to empty file 
hdfs://hadoop-cluster/tmp/hive/hadoop/07fd131c-6590-4081-b7ed-2f0ae5b16551/hive_2018-06-28_16-01-22_402_5116918946052746369-3/-mr-10003/408baabf-a30d-4c00-a196-70e14e172036
 (false)
2018-06-28T16:01:22,979 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Context: New scratch dir is 
hdfs://hadoop-cluster/tmp/hive/hadoop/07fd131c-6590-4081-b7ed-2f0ae5b16551/hive_2018-06-28_16-01-22_402_5116918946052746369-3
2018-06-28T16:01:22,996 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.SerializationUtilities: Serializing MapWork using kryo
2018-06-28T16:01:23,055 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Utilities: Serialized plan (via FILE) - name: null size: 4.86KB
2018-06-28T16:01:29,318 INFO [HiveServer2-Background-Pool: Thread-97] 
fs.FSStatsPublisher: created : 
hdfs://hadoop-cluster/user/hive/warehouse/customer_bak/.hive-staging_hive_2018-06-28_16-01-22_402_5116918946052746369-1/-ext-10001
2018-06-28T16:01:29,395 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Utilities: PLAN PATH = 
hdfs://hadoop-cluster/tmp/hive/hadoop/07fd131c-6590-4081-b7ed-2f0ae5b16551/hive_2018-06-28_16-01-22_402_5116918946052746369-3/-mr-10004/2cb63e68-2c77-4554-817a-0e25d700cf0f/map.xml
2018-06-28T16:01:29,395 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Utilities: PLAN PATH = 
hdfs://hadoop-cluster/tmp/hive/hadoop/07fd131c-6590-4081-b7ed-2f0ae5b16551/hive_2018-06-28_16-01-22_402_5116918946052746369-3/-mr-10004/2cb63e68-2c77-4554-817a-0e25d700cf0f/reduce.xml
2018-06-28T16:01:29,398 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Utilities: No plan file found: 
hdfs://hadoop-cluster/tmp/hive/hadoop/07fd131c-6590-4081-b7ed-2f0ae5b16551/hive_2018-06-28_16-01-22_402_5116918946052746369-3/-mr-10004/2cb63e68-2c77-4554-817a-0e25d700cf0f/reduce.xml
2018-06-28T16:01:29,406 INFO [HiveServer2-Background-Pool: Thread-97] 
client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
2018-06-28T16:01:29,413 WARN [HiveServer2-Background-Pool: Thread-97] 
mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. 
Implement the Tool interface and execute your application with ToolRunner to 
remedy this.
2018-06-28T16:01:31,721 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Utilities: PLAN PATH = 
hdfs://hadoop-cluster/tmp/hive/hadoop/07fd131c-6590-4081-b7ed-2f0ae5b16551/hive_2018-06-28_16-01-22_402_5116918946052746369-3/-mr-10004/2cb63e68-2c77-4554-817a-0e25d700cf0f/map.xml
2018-06-28T16:01:31,722 INFO [HiveServer2-Background-Pool: Thread-97] 
io.CombineHiveInputFormat: Total number of paths: 2, launching 1 threads to 
check non-combinable ones.
2018-06-28T16:01:31,724 INFO [HiveServer2-Background-Pool: Thread-97] 
io.CombineHiveInputFormat: CombineHiveInputSplit creating pool for 
hdfs://hadoop-cluster/user/hive/warehouse/customer/country=china; using filter 
path hdfs://hadoop-cluster/user/hive/warehouse/customer/country=china
2018-06-28T16:01:31,725 INFO [HiveServer2-Background-Pool: Thread-97] 
io.CombineHiveInputFormat: CombineHiveInputSplit: pool is already created for 
hdfs://hadoop-cluster/tmp/hive/hadoop/07fd131c-6590-4081-b7ed-2f0ae5b16551/hive_2018-06-28_16-01-22_402_5116918946052746369-3/-mr-10003/408baabf-a30d-4c00-a196-70e14e172036;
 using filter path 
hdfs://hadoop-cluster/tmp/hive/hadoop/07fd131c-6590-4081-b7ed-2f0ae5b16551/hive_2018-06-28_16-01-22_402_5116918946052746369-3/-mr-10003/408baabf-a30d-4c00-a196-70e14e172036
2018-06-28T16:01:31,738 INFO [HiveServer2-Background-Pool: Thread-97] 
input.FileInputFormat: Total input paths to process : 2
2018-06-28T16:01:31,742 INFO [HiveServer2-Background-Pool: Thread-97] 
input.CombineFileInputFormat: DEBUG: Terminated node allocation with : 
CompletedNodes: 3, size left: 0
2018-06-28T16:01:31,742 INFO [HiveServer2-Background-Pool: Thread-97] 
io.CombineHiveInputFormat: number of splits 1
2018-06-28T16:01:31,743 INFO [HiveServer2-Background-Pool: Thread-97] 
io.CombineHiveInputFormat: Number of all splits 1
2018-06-28T16:01:31,853 INFO [HiveServer2-Background-Pool: Thread-97] 
mapreduce.JobSubmitter: number of splits:1
2018-06-28T16:01:31,982 INFO [HiveServer2-Background-Pool: Thread-97] 
mapreduce.JobSubmitter: Submitting tokens for job: job_1530599349924_0007
2018-06-28T16:01:32,220 INFO [HiveServer2-Background-Pool: Thread-97] 
impl.YarnClientImpl: Submitted application application_1530599349924_0007
2018-06-28T16:01:32,226 INFO [HiveServer2-Background-Pool: Thread-97] 
mapreduce.Job: The url to track the job: 
http://cluster3:8088/proxy/application_1530599349924_0007/
2018-06-28T16:01:32,227 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: Starting Job = job_1530599349924_0007, Tracking URL = 
http://cluster3:8088/proxy/application_1530599349924_0007/
2018-06-28T16:01:32,227 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: Kill Command = /home/hadoop/app/hadoop_2.6/bin/hadoop job -kill 
job_1530599349924_0007
2018-06-28T16:01:56,835 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: Hadoop job information for Stage-1: number of mappers: 1; number of 
reducers: 0
2018-06-28T16:01:56,933 WARN [HiveServer2-Background-Pool: Thread-97] 
mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. 
Use org.apache.hadoop.mapreduce.TaskCounter instead
2018-06-28T16:01:56,934 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: 2018-06-28 16:01:56,933 Stage-1 map = 0%, reduce = 0%
2018-06-28T16:02:05,922 INFO [HiveServer2-Handler-Pool: Thread-30] 
conf.HiveConf: Using the default value passed in for log id: 
07fd131c-6590-4081-b7ed-2f0ae5b16551
2018-06-28T16:02:05,922 INFO [HiveServer2-Handler-Pool: Thread-30] 
session.SessionState: Updating thread name to 
07fd131c-6590-4081-b7ed-2f0ae5b16551 HiveServer2-Handler-Pool: Thread-30
2018-06-28T16:02:05,924 INFO [07fd131c-6590-4081-b7ed-2f0ae5b16551 
HiveServer2-Handler-Pool: Thread-30] conf.HiveConf: Using the default value 
passed in for log id: 07fd131c-6590-4081-b7ed-2f0ae5b16551
2018-06-28T16:02:05,925 INFO [HiveServer2-Handler-Pool: Thread-30] 
session.SessionState: Resetting thread name to HiveServer2-Handler-Pool: 
Thread-30
2018-06-28T16:02:35,131 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: 2018-06-28 16:02:35,130 Stage-1 map = 100%, reduce = 0%, Cumulative 
CPU 2.82 sec
2018-06-28T16:02:39,527 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: MapReduce Total cumulative CPU time: 2 seconds 820 msec
2018-06-28T16:02:39,607 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: Ended Job = job_1530599349924_0007
2018-06-28T16:02:39,643 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.FileSinkOperator: Moving tmp dir: 
hdfs://hadoop-cluster/user/hive/warehouse/customer_bak/.hive-staging_hive_2018-06-28_16-01-22_402_5116918946052746369-1/_tmp.-ext-10002
 to: 
hdfs://hadoop-cluster/user/hive/warehouse/customer_bak/.hive-staging_hive_2018-06-28_16-01-22_402_5116918946052746369-1/-ext-10002
2018-06-28T16:02:39,669 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Driver: Starting task [Stage-7:CONDITIONAL] in serial mode
2018-06-28T16:02:39,688 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: Stage-4 is selected by condition resolver.
2018-06-28T16:02:39,689 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: Stage-3 is filtered out by condition resolver.
2018-06-28T16:02:39,689 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: Stage-5 is filtered out by condition resolver.
2018-06-28T16:02:39,690 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Driver: Starting task [Stage-4:MOVE] in serial mode
2018-06-28T16:02:39,691 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: Moving data to directory 
hdfs://hadoop-cluster/user/hive/warehouse/customer_bak/.hive-staging_hive_2018-06-28_16-01-22_402_5116918946052746369-1/-ext-10000
 from 
hdfs://hadoop-cluster/user/hive/warehouse/customer_bak/.hive-staging_hive_2018-06-28_16-01-22_402_5116918946052746369-1/-ext-10002
2018-06-28T16:02:39,785 INFO [HiveServer2-Background-Pool: Thread-97] 
ql.Driver: Starting task [Stage-0:MOVE] in serial mode
2018-06-28T16:02:39,786 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.Task: Loading data to table default.customer_bak partition (country=null) 
from 
hdfs://hadoop-cluster/user/hive/warehouse/customer_bak/.hive-staging_hive_2018-06-28_16-01-22_402_5116918946052746369-1/-ext-10000
2018-06-28T16:02:39,871 INFO [HiveServer2-Background-Pool: Thread-97] 
exec.MoveTask: Partition is: {country=null}
2018-06-28T16:02:39,879 INFO [HiveServer2-Background-Pool: Thread-97] exec.Task:
{code}
Why doesn't the program continue?

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to