[
https://issues.apache.org/jira/browse/HIVE-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179333#comment-14179333
]
Xiaobing Zhou commented on HIVE-8550:
-------------------------------------
The root reason is SQLServer varchar and nvarchar behaves fundamentally
different when unicode chars are involved. Hive meta data table PARTITION(...,
PART_NAME varchar(767), ...) contains varchar column, but unicode chars like
'tag=䶵' are inserted, which leads to garbled representation like 'tag=?'. Of
course a query will be failed, however, if nvarchar is used for column
creation, N'tag=䶵' with 'N' prefix is correct thing to be inserted, without
'N', also garbled in representation.
I traced this issue down to bottom-most layer of HIVE, that is, DataNucleus
that talks to MSSQLServer as metastore backend, and found DataNucleus returns
NULL partition that fails LOAD DATA as described. I guess this relates
SQLServer nvarchar issue.
I need more time to hack DataNucleus and HIVE metastore DDL/DML script to
confirm my speculation.
> Hive cannot load data into partitioned table with Unicode key
> -------------------------------------------------------------
>
> Key: HIVE-8550
> URL: https://issues.apache.org/jira/browse/HIVE-8550
> Project: Hive
> Issue Type: Bug
> Environment: Windows
> Reporter: Xiaobing Zhou
> Priority: Critical
> Attachments: CreatePartitionedTable.hql,
> LoadIntoPartitionedTable.hql, partitioned.txt
>
>
> Steps to reproduce:
> 1) Copy the file partitioned.txt to the root folder of your HDFS root dir.
> Copy the two hql files to your local directory.
> 2) Open Hive CLI.
> 3) Run:
> hive> source <path to CreatePartitionedTable.hql>;
> 4) Run
> hive> source <path to LoadIntoPartitionedTable.hql>;
> The following error will be shown:
> hive> source C:\Scripts\partition\LoadIntoPartitionedTable.hql;
> Loading data to table default.mypartitioned partition (tag=䶵)
> Failed with exception null
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
