Hao Zhu created DRILL-3579: ------------------------------ Summary: Drill on Hive query fails if partition table has __HIVE_DEFAULT_PARTITION__ Key: DRILL-3579 URL: https://issues.apache.org/jira/browse/DRILL-3579 Project: Apache Drill Issue Type: Bug Components: Functions - Hive Affects Versions: 1.1.0 Environment: Drill 1.1 on Hive 1.0 Reporter: Hao Zhu Assignee: Mehant Baid
If Hive's partition table has __HIVE_DEFAULT_PARTITION__ in the case of null values in the partition column, Drill on Hive query will fail. Minimum reproduce: 1.Hive: {code} CREATE TABLE h1_testpart2(id INT) PARTITIONED BY(id2 int); set hive.exec.dynamic.partition.mode=nonstrict; INSERT OVERWRITE TABLE h1_testpart2 PARTITION(id2) SELECT 1 as id1 , 20150101 as id2 from h1_passwords limit 1; INSERT OVERWRITE TABLE h1_testpart2 PARTITION(id2) SELECT 1 as id1 , null as id2 from h1_passwords limit 1; {code} 2. Filesystem looks like: {code} h1 h1_testpart2]# ls -altr total 2 drwxrwxrwx 89 mapr mapr 87 Jul 30 00:04 .. drwxr-xr-x 2 mapr mapr 1 Jul 30 00:05 id2=20150101 drwxr-xr-x 2 mapr mapr 1 Jul 30 00:05 id2=__HIVE_DEFAULT_PARTITION__ drwxr-xr-x 4 mapr mapr 2 Jul 30 00:05 . {code} 3.Drill will fail: {code} select * from h1_testpart2; Error: SYSTEM ERROR: NumberFormatException: For input string: "__HIVE_DEFAULT_PARTITION__" Fragment 0:0 [Error Id: 509eb392-db9a-42f3-96ea-fb597425f49f on h1.poc.com:31010] (java.lang.reflect.UndeclaredThrowableException) null org.apache.hadoop.security.UserGroupInformation.doAs():1581 org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():136 org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173 org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():131 org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173 org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106 org.apache.drill.exec.physical.impl.ImplCreator.getExec():81 org.apache.drill.exec.work.fragment.FragmentExecutor.run():235 org.apache.drill.common.SelfCleaningRunnable.run():38 java.util.concurrent.ThreadPoolExecutor.runWorker():1142 java.util.concurrent.ThreadPoolExecutor$Worker.run():617 java.lang.Thread.run():745 Caused By (org.apache.drill.common.exceptions.ExecutionSetupException) Failure while initializing HiveRecordReader: For input string: "__HIVE_DEFAULT_PARTITION__" org.apache.drill.exec.store.hive.HiveRecordReader.init():241 org.apache.drill.exec.store.hive.HiveRecordReader.<init>():138 org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():58 org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():34 org.apache.drill.exec.physical.impl.ImplCreator$2.run():138 org.apache.drill.exec.physical.impl.ImplCreator$2.run():136 java.security.AccessController.doPrivileged():-2 javax.security.auth.Subject.doAs():422 org.apache.hadoop.security.UserGroupInformation.doAs():1566 org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():136 org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173 org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():131 org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173 org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106 org.apache.drill.exec.physical.impl.ImplCreator.getExec():81 org.apache.drill.exec.work.fragment.FragmentExecutor.run():235 org.apache.drill.common.SelfCleaningRunnable.run():38 java.util.concurrent.ThreadPoolExecutor.runWorker():1142 java.util.concurrent.ThreadPoolExecutor$Worker.run():617 java.lang.Thread.run():745 Caused By (java.lang.NumberFormatException) For input string: "__HIVE_DEFAULT_PARTITION__" java.lang.NumberFormatException.forInputString():65 java.lang.Integer.parseInt():580 java.lang.Integer.parseInt():615 org.apache.drill.exec.store.hive.HiveRecordReader.convertPartitionType():605 org.apache.drill.exec.store.hive.HiveRecordReader.init():236 org.apache.drill.exec.store.hive.HiveRecordReader.<init>():138 org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():58 org.apache.drill.exec.store.hive.HiveScanBatchCreator.getBatch():34 org.apache.drill.exec.physical.impl.ImplCreator$2.run():138 org.apache.drill.exec.physical.impl.ImplCreator$2.run():136 java.security.AccessController.doPrivileged():-2 javax.security.auth.Subject.doAs():422 org.apache.hadoop.security.UserGroupInformation.doAs():1566 org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():136 org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173 org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch():131 org.apache.drill.exec.physical.impl.ImplCreator.getChildren():173 org.apache.drill.exec.physical.impl.ImplCreator.getRootExec():106 org.apache.drill.exec.physical.impl.ImplCreator.getExec():81 org.apache.drill.exec.work.fragment.FragmentExecutor.run():235 org.apache.drill.common.SelfCleaningRunnable.run():38 java.util.concurrent.ThreadPoolExecutor.runWorker():1142 java.util.concurrent.ThreadPoolExecutor$Worker.run():617 java.lang.Thread.run():745 (state=,code=0) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)