[ https://issues.apache.org/jira/browse/DRILL-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Volodymyr Vysotskyi updated DRILL-6969: --------------------------------------- Labels: doc-impacting ready-to-commit (was: ready-to-commit) > Inconsistent results when reading MaprDB JSON tables using hive plugin when > native reader is enabled > ---------------------------------------------------------------------------------------------------- > > Key: DRILL-6969 > URL: https://issues.apache.org/jira/browse/DRILL-6969 > Project: Apache Drill > Issue Type: Bug > Affects Versions: 1.14.0 > Reporter: Volodymyr Vysotskyi > Assignee: Volodymyr Vysotskyi > Priority: Major > Labels: doc-impacting, ready-to-commit > Fix For: 1.16.0 > > > Steps to reproduce: > 0. Set PST timezone. > 1. Create the table in MaprDB shell: > {code:java} > create /tmp/testtimestamp > insert /tmp/testtimestamp --value '{"_id":"1","datestring":"2018-01-01 > 12:12:12.123","datetimestamp":{"$date":"2018-01-01T20:12:12.123Z"}}' > insert /tmp/testtimestamp --value '{"_id":"2","datestring":"9999-12-31 > 23:59:59.999","datetimestamp":{"$date":"10000-01-01T07:59:59.999Z"}}' > {code} > 2. Create a hive table: > {code:sql} > create external table `testtimestamp` (`_id` string, datestring string, > datetimestamp timestamp) > ROW FORMAT SERDE 'org.apache.hadoop.hive.maprdb.json.serde.MapRDBSerDe' > STORED BY 'org.apache.hadoop.hive.maprdb.json.MapRDBJsonStorageHandler' > TBLPROPERTIES ( 'maprdb.column.id'='_id', > 'maprdb.table.name'='/tmp/testtimestamp'); > {code} > 3. Disable native reader and run the query on the table from Drill using hive > plugin: > {code:sql} > alter session set > store.hive.maprdb_json.optimize_scan_with_native_reader=false; > select * from hive.testtimestamp; > {code} > It returns: > {noformat} > +------+--------------------------+--------------------------+ > | _id | datestring | datetimestamp | > +------+--------------------------+--------------------------+ > | 1 | 2018-01-01 12:12:12.123 | 2018-01-01 12:12:12.123 | > | 2 | 9999-12-31 23:59:59.999 | 9999-12-31 23:59:59.999 | > +------+--------------------------+--------------------------+ > {noformat} > 4. Enable native reader and run the query on the same table: > {code:sql} > alter session set > store.hive.maprdb_json.optimize_scan_with_native_reader=true; > select * from hive.testtimestamp; > {code} > It returns: > {noformat} > +------+--------------------------+---------------------------+ > | _id | datestring | datetimestamp | > +------+--------------------------+---------------------------+ > | 1 | 2018-01-01 12:12:12.123 | 2018-01-01 20:12:12.123 | > | 2 | 9999-12-31 23:59:59.999 | 10000-01-01 07:59:59.999 | > +------+--------------------------+---------------------------+ > {noformat} > h2. For documentation: > Added the following Mapr-DB Format Setting: > ||Option||Description||Value|| > |readTimestampWithZoneOffset|When enabled, Drill converts timestamp values > read form MapR Database from UTC to local timezone. Disabled by > default.|true\|false| > Added the following configuration option: > ||Name||Default||Description|| > |store.hive.maprdb_json.read_timestamp_with_timezone_offset|FALSE|Enables > Drill to read timestamp values with timezone offset when hive plugin is used > and Drill native MaprDB JSON reader usage is enabled. (Drill 1.16+)| -- This message was sent by Atlassian JIRA (v7.6.3#76005)