Ádám Szita created HIVE-24151:
---------------------------------
Summary: MultiDelimitSerDe shifts data if strings contain
non-ASCII characters
Key: HIVE-24151
URL: https://issues.apache.org/jira/browse/HIVE-24151
Project: Hive
Issue Type: Bug
Reporter: Ádám Szita
Assignee: Ádám Szita
HIVE-22360 intended to fix another MultiDelimitSerde problem (with NULL last
columns) but introduced a regression: the approach of the fix is pretty much
all wrong, as the existing logic that operated on bytes got replaced by regex
matcher logic which deals in character positions, rather than byte positions.
As some non ASCII characters consist of more than 1 byte, the whole record may
get shifted due to this.
With this ticket I'm going to restore the old logic, and apply the proper fix
on that, but keeping (and extending) the test cases added with HIVE-22360 so
that we have a solution for both issues.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
