[
https://issues.apache.org/jira/browse/DRILL-7313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Volodymyr Vysotskyi updated DRILL-7313:
---------------------------------------
Description:
Currently, when an external Hive MaprDB table is queried using hive plugin with
enabled {{store.hive.maprdb_json.optimize_scan_with_native_reader}}, some
queries may fail due to soft schema change, though Hive knows actual data types.
For example, when we have a table with several fields, and one of them has only
several non-null values, queries with grouping by such field will fail due to
schema change.
The goal of this Jira is to allow using types from Hive when a non-existing
field is created, so it will allow avoiding such issues.
In the scope of this Jira is also added new session option
{{store.hive.maprdb_json.all_text_mode}} to read all data from the maprDB JSON
tables as VARCHAR when hive plugin is used and Drill native MaprDB JSON reader
usage is enabled.
was:
Currently, when an external Hive MaprDB table is queried using hive plugin with
enabled {{store.hive.maprdb_json.optimize_scan_with_native_reader}}, some
queries may fail due to soft schema change, though Hive knows actual data types.
For example, when we have a table with several fields, and one of them has only
several non-null values, queries with grouping by such field will fail due to
schema change.
The goal of this Jira is to allow using types from Hive when a non-existing
field is created, so it will allow avoiding such issues.
> Use Hive schema for MaprDB native reader when field was empty and support all
> text mode
> ---------------------------------------------------------------------------------------
>
> Key: DRILL-7313
> URL: https://issues.apache.org/jira/browse/DRILL-7313
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.16.0
> Reporter: Volodymyr Vysotskyi
> Assignee: Volodymyr Vysotskyi
> Priority: Major
> Labels: doc-impacting, ready-to-commit
> Fix For: 1.17.0
>
>
> Currently, when an external Hive MaprDB table is queried using hive plugin
> with enabled {{store.hive.maprdb_json.optimize_scan_with_native_reader}},
> some queries may fail due to soft schema change, though Hive knows actual
> data types.
> For example, when we have a table with several fields, and one of them has
> only several non-null values, queries with grouping by such field will fail
> due to schema change.
> The goal of this Jira is to allow using types from Hive when a non-existing
> field is created, so it will allow avoiding such issues.
> In the scope of this Jira is also added new session option
> {{store.hive.maprdb_json.all_text_mode}} to read all data from the maprDB
> JSON tables as VARCHAR when hive plugin is used and Drill native MaprDB JSON
> reader usage is enabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
