[
https://issues.apache.org/jira/browse/HIVE-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105967#comment-17105967
]
Mustafa Iman commented on HIVE-23455:
-------------------------------------
[~ashutoshc] The error happens when populating loop does not visit all the
indices. The loop does not visit all the indices -> there are nulls in the
array. Therefore the extra assertion loop. Regarding TestNegativeCliDriver, I
tried that but it shows only generic task error at the end of the output in
both cases.
Anyway, [~gopalv] thinks this is breaking schema evolution. I still do not see
it as this codepath fails in very next method
call(EncodedTreeReaderFactory#createRootTreeReader) with NullPointerException
anyway.
I also tried the sample query he gave with mini llap drivers:
{code:java}
create external table foo(x int);
insert into foo values(1);
alter table foo add column (y int);
select sum(x) from foo;
{code}
It works for me. So I did not close the ticket but please do not merge yet.
> Improve error message for external orc table
> --------------------------------------------
>
> Key: HIVE-23455
> URL: https://issues.apache.org/jira/browse/HIVE-23455
> Project: Hive
> Issue Type: Improvement
> Reporter: Mustafa Iman
> Assignee: Mustafa Iman
> Priority: Major
> Attachments: HIVE-23455.patch, acid_table_as_external.q,
> acid_table_as_external.q.out
>
>
> Since there is no schema validation for external tables, users may face
> various errors if their orc data and external table schema does not match. If
> orc schema has fewer columns than projection OrcEncodedDataConsumer may
> receive an incomplete TypeDescription array which will manifest itself as
> NullPointerException later.
> We can at least verify that OrcEncodedDataConsumer gets enough
> TypeDescriptions. If assertion fails, user sees there is something wrong with
> the schema and hopefully resolves the problem quickly. If there are enough
> columns in the file but the schema of the query does not match, user
> generally sees a ClassCastException. If there are enough columns and types
> accidentally match, there is nothing we can do as this is an external table.
> We have seen this when trying to use a managed table as external table
> location. Although user facing schemas are the same, managed table has acid
> related metadata. I am adding a q file demonstrating NullPointerException
> with TestMiniLlapLocalCliDriver and the output after the fix. I haven't added
> this to precommit tests as it is hard to assert the exception message from
> mini driver framework and effectively it is just changing the error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
