Hi Martin This is an known issue and its fixed in hive trunk. It should be available in 0.13 release.
https://issues.apache.org/jira/browse/HIVE-5601 Thanks Prasanth Jayachandran On Feb 26, 2014, at 8:55 AM, Martin, Nick <nimar...@pssd.com> wrote: > Hi all, > > (Running Hive 12.0) > > I have two tables and both are stored as ORC. I attempted to insert via > select from tbl1 to tbl2 using ‘BETWEEN’ in my where clause to narrow down > some dates. Something like so: > > “Insert into tbl1 select col1, col2 from tbl2 where col1 between 2 and 4” > > I kept hitting the error pasted below. So, I switched to a different approach > to see if it would work: > > “Insert into tbl1 select col1,col2 from tbl2 where col1>=2 and col1<=4” > > Hit the same error. When I just use “where col1>=2” in the where clause the > insert will run fine. > > Is this expected? > > > > 2014-02-26 11:22:53,755 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: > mapreduce.job.end-notification.max.retry.interval; Ignoring. > 2014-02-26 11:22:53,782 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: > mapreduce.job.end-notification.max.attempts; Ignoring. > 2014-02-26 11:22:53,902 INFO [main] > org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from > hadoop-metrics2.properties > 2014-02-26 11:22:53,930 INFO [main] > org.apache.hadoop.metrics2.impl.MetricsSinkAdapter: Sink ganglia started > 2014-02-26 11:22:53,975 INFO [main] > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period > at 10 second(s). > 2014-02-26 11:22:53,975 INFO [main] > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system > started > 2014-02-26 11:22:53,985 INFO [main] org.apache.hadoop.mapred.YarnChild: > Executing with tokens: > 2014-02-26 11:22:53,985 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: > mapreduce.job, Service: job_1392147432508_1108, Ident: > (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@249c2715) > 2014-02-26 11:22:54,057 INFO [main] org.apache.hadoop.mapred.YarnChild: > Sleeping for 0ms before retrying again. Got null now. > 2014-02-26 11:22:54,352 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: > mapreduce.job.end-notification.max.retry.interval; Ignoring. > 2014-02-26 11:22:54,363 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: > mapreduce.job.end-notification.max.attempts; Ignoring. > 2014-02-26 11:22:54,409 INFO [main] org.apache.hadoop.mapred.YarnChild: > mapreduce.cluster.local.dir for child: > /hdfs/01/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/02/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/03/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/04/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/05/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/06/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/07/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/08/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/09/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/10/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/11/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/12/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108 > 2014-02-26 11:22:54,481 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: > mapreduce.job.end-notification.max.retry.interval; Ignoring. > 2014-02-26 11:22:54,486 WARN [main] org.apache.hadoop.conf.Configuration: > job.xml:an attempt to override final parameter: > mapreduce.job.end-notification.max.attempts; Ignoring. > 2014-02-26 11:22:54,542 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: mapred.task.id is > deprecated. Instead, use mapreduce.task.attempt.id > 2014-02-26 11:22:54,542 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: mapred.task.is.map is > deprecated. Instead, use mapreduce.task.ismap > 2014-02-26 11:22:54,543 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: mapred.local.dir is > deprecated. Instead, use mapreduce.cluster.local.dir > 2014-02-26 11:22:54,543 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: mapred.cache.localFiles is > deprecated. Instead, use mapreduce.job.cache.local.files > 2014-02-26 11:22:54,543 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: mapred.job.id is > deprecated. Instead, use mapreduce.job.id > 2014-02-26 11:22:54,544 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: mapred.tip.id is > deprecated. Instead, use mapreduce.task.id > 2014-02-26 11:22:54,544 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: mapred.task.partition is > deprecated. Instead, use mapreduce.task.partition > 2014-02-26 11:22:54,545 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: job.local.dir is > deprecated. Instead, use mapreduce.job.local.dir > 2014-02-26 11:22:54,546 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: mapred.cache.localArchives > is deprecated. Instead, use mapreduce.job.cache.local.archives > 2014-02-26 11:22:54,710 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. > Instead, use dfs.metrics.session-id > 2014-02-26 11:22:55,007 INFO [main] org.apache.hadoop.mapred.Task: Using > ResourceCalculatorProcessTree : [ ] > 2014-02-26 11:22:55,316 INFO [main] org.apache.hadoop.mapred.MapTask: > Processing split: > Paths:/apps/hive/warehouse/edw.db/penopp_fsm/000000_0:0+116456645,/apps/hive/warehouse/edw.db/penopp_fsm/000001_0:0+116484938,/apps/hive/warehouse/edw.db/penopp_fsm/000002_0:0+116473509InputFormatClass: > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat > > 2014-02-26 11:22:55,370 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: > <PERFLOG method=deserializePlan> > 2014-02-26 11:22:55,370 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: > Deserializing MapWork via kryo > 2014-02-26 11:22:55,574 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: > </PERFLOG method=deserializePlan start=1393431775370 end=1393431775574 > duration=204> > 2014-02-26 11:22:55,708 INFO [main] > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat: included column ids = > 0,1,2,3,4,5,6 > 2014-02-26 11:22:55,708 INFO [main] > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat: included columns names = > col1,col2,col3,col4,col5,col6,col7 > 2014-02-26 11:22:55,728 INFO [main] > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat: ORC pushdown predicate: > leaf-0 = (BETWEEN col7 201301 201312) > expr = leaf-0 > 2014-02-26 11:22:55,897 ERROR [main] > org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException > as:myusername (auth:SIMPLE) cause:java.io.IOException: > java.lang.reflect.InvocationTargetException > 2014-02-26 11:22:55,898 WARN [main] org.apache.hadoop.mapred.YarnChild: > Exception running child : java.io.IOException: > java.lang.reflect.InvocationTargetException > at > org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97) > at > org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57) > at > org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:343) > at > org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:290) > at > org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:404) > at > org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:556) > at > org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:167) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:408) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > at > org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:329) > ... 11 more > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.planReadPartialDataStreams(RecordReaderImpl.java:1962) > at > org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readPartialDataStreams(RecordReaderImpl.java:2118) > at > org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:1775) > at > org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:2148) > at > org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:2190) > at > org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>(RecordReaderImpl.java:130) > at > org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rows(ReaderImpl.java:335) > at > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.<init>(OrcInputFormat.java:100) > at > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:205) > at > org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:65) > ... 16 more > > 2014-02-26 11:22:55,902 INFO [main] org.apache.hadoop.mapred.Task: Runnning > cleanup for the task -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
