[jira] [Created] (HIVE-2685) Unable to query Hive table defined with custom ROW FORMAT remotely using JDBC
Unable to query Hive table defined with custom ROW FORMAT remotely using JDBC
-
Key: HIVE-2685
URL: https://issues.apache.org/jira/browse/HIVE-2685
Project: Hive
Issue Type: Bug
Components: JDBC
Affects Versions: 0.7.1
Environment: Cloudera VM for Linux Red Hat
Reporter: Bobo Vava
Created table using custom format as follows :
create table if not exists FILE_HDFS_TO_HIVE_CUSTOM_NO_STAGE_TGT
(
ID STRING,
NAMESTRING,
DESCR STRING
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
"input.regex" = "(.{3})(.{5})(.{9})",
"output.format.string" = "%1$s %2$s %3$s"
)
STORED AS TEXTFILE
and loaded some data into it.
Now I am trying to query the table from a remote host using JDBC from a java
program.
All the Hive JDBC jars are included in the classpath and I am able to execute
run all kinds
of DDL and DML statements except for this one.
Additionally I use add jar just before executing the query :
add jar
/ade/bosimova_bobo_odi/odi/odiqa/automation/qa_scripts/knowledge_modules/sql/hive_cert/drivers/hive-contrib-0.7.1-cdh3u2.jar
0
select * from file_hdfs_to_hive_custom_no_stage_tgt order by id
Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 2
from org.apache.hadoop.hive.ql.exec.MapRedTask
java.lang.ClassNotFoundException:
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
More details :
2011-12-27 19:59:30,695 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:387)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
... 22 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassNotFoundException:
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:403)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:90)
... 22 more
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:228)
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:359)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly,
[jira] [Created] (HIVE-2683) load data twice from the same hdfs file without overwrite fails with return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
load data twice from the same hdfs file without overwrite fails with return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask Key: HIVE-2683 URL: https://issues.apache.org/jira/browse/HIVE-2683 Project: Hive Issue Type: Bug Components: SQL Affects Versions: 0.7.1 Environment: Cloudera VM for Linux Red Hat Reporter: Bobo Vava I try to load data into a hive table twice from the same hdfs file without overwrite option. I would expect second load to append to existing data or at least replace it. I get the following error message instead. Note that, during the operation Hive is renaming the source file to _copy_, which makes us believe it is trying to avoid erasing the existing file while moving and both files (original and renamed) would be made available to Hive. But this is not the case. Example of code end error stack : load data inpath 'hdfs://0.0.0.0/user/qafiles/SRC_FILE_EMP.dat' into table EMP_DELIMITED_FILE_TRUNC_TBL --- Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask river returned: 9. Errors: Hive history file=/tmp/hue/hive_job_log_hue_201112132213_439347746.txt Loading data to table default.emp_delimited_file_trunc_tbl Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask Failed with exception null 11/12/13 22:13:55 ERROR exec.MoveTask: Failed with exception null java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Hive.copyFiles(Hive.java:1738) at org.apache.hadoop.hive.ql.metadata.Table.copyFiles(Table.java:542) at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1189) at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:197) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:306) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:510) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:499) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:337) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java :1110) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxService Impl.java:499) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java :886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908 ) at java.lang.Thread.run(Thread.java:619) FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask 11/12/13 22:13:55 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask 11/12/13 22:13:55 ERROR beeswax.BeeswaxServiceImpl: Exception while processing query BeeswaxException(message:Driver returned: 9. Errors: Hive history file=/tmp/hue/hive_job_log_hue_201112132213_439347746.txt Loading data to table default.emp_delimited_file_trunc_tbl Failed with exception null FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask , log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f, handle:QueryHandle(id:8e8ea906-9b5a-4bfe-9946-515249cc523f, log_context:8e8ea906-9b5a-4bfe-9946-515249cc523f)) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:313) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:510) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:499) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:337) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java :1110) at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:499) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java :886)
