[
https://issues.apache.org/jira/browse/DRILL-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326276#comment-14326276
]
Hema Kumar S edited comment on DRILL-1075 at 2/18/15 6:21 PM:
--------------------------------------------------------------
[~amitskatti]
I'm using cdh 4.5 and drill 0.7.0 .I'm trying to store file in hdfs, (using
create table),
If I add CDH jars to drill classpath. I got below error.
Java.lang.UnsupportedOperationException: This is supposed to be overridden by
subclasses.
{quote}
at
com.google.protobuf.GeneratedMessage.getUnknownFields(GeneratedMessage.java:180)
~[protobuf-java-2.5.0.jar:na]
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetFileInfoRequestProto.getSerializedSize(ClientNamenodeProtocolProtos.java:30108)
~[hadoop-hdfs-2.0.0-cdh4.5.0.jar:na]
at
com.google.protobuf.AbstractMessageLite.toByteString(AbstractMessageLite.java:49)
~[protobuf-java-2.5.0.jar:na]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.constructRpcRequest(ProtobufRpcEngine.java:149)
~[hadoop-common-2.0.0-cdh4.5.0.jar:na]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:193)
~[hadoop-common-2.0.0-cdh4.5.0.jar:na]
at com.sun.proxy.$Proxy33.getFileInfo(Unknown Source) ~[na:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[na:1.7.0_51]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.7.0_51]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_51]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
~[hadoop-common-2.0.0-cdh4.5.0.jar:na]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
~[hadoop-common-2.0.0-cdh4.5.0.jar:na]
at com.sun.proxy.$Proxy33.getFileInfo(Unknown Source) ~[na:na]
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:629)
~[hadoop-hdfs-2.0.0-cdh4.5.0.jar:na]
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1545)
~[hadoop-hdfs-2.0.0-cdh4.5.0.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:820)
~[hadoop-hdfs-2.0.0-cdh4.5.0.jar:na]
{quote}
if I don't add cdh jars to Drill classpath. It's giving below error.
ERROR o.a.d.e.s.text.DrillTextRecordWriter - Unable to create file:
/tmp/table/1_13_0.csv
{quote}
java.io.IOException: Failed on local exception:
java.nio.channels.ClosedByInterruptException; Host Details : local host is:
"***********"; destination host is: "**********":8020;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.ipc.Client.call(Client.java:1414)
~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.ipc.Client.call(Client.java:1363)
~[hadoop-common-2.4.1.jar:na]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
~[hadoop-common-2.4.1.jar:na]
at com.sun.proxy.$Proxy38.create(Unknown Source) ~[na:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[na:1.7.0_51]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.7.0_51]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_51]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:190)
~[hadoop-common-2.4.1.jar:na]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
~[hadoop-common-2.4.1.jar:na]
at com.sun.proxy.$Proxy38.create(Unknown Source) ~[na:na]
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.create(ClientNamenodeProtocolTranslatorPB.java:258)
~[hadoop-hdfs-2.4.1.jar:na]
at
org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1600)
~[hadoop-hdfs-2.4.1.jar:na]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1465)
~[hadoop-hdfs-2.4.1.jar:na]
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1390)
~[hadoop-hdfs-2.4.1.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:394)
~[hadoop-hdfs-2.4.1.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:390)
~[hadoop-hdfs-2.4.1.jar:na]
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
~[hadoop-common-2.4.1.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:390)
~[hadoop-hdfs-2.4.1.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:334)
~[hadoop-hdfs-2.4.1.jar:na]
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:906)
~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:887)
~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:784)
~[hadoop-common-2.4.1.jar:na]
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:773)
~[hadoop-common-2.4.1.jar:na]
at
org.apache.drill.exec.store.text.DrillTextRecordWriter.startNewSchema(DrillTextRecordWriter.java:81)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.store.StringOutputRecordWriter.updateSchema(StringOutputRecordWriter.java:57)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:162)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
{quote}
Does drill 0.70 support hdfs storage in CDH? or is this a bug?
was (Author: hema.kumar):
[~amitskatti]
I'm using cdh 4.5 and drill 0.7.0 .I'm trying to store file in hdfs, (using
create table), I added all the jars to drill classpath. I got below error.
Java.lang.UnsupportedOperationException: This is supposed to be overridden by
subclasses.
{quote}
at
com.google.protobuf.GeneratedMessage.getUnknownFields(GeneratedMessage.java:180)
~[protobuf-java-2.5.0.jar:na]
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetFileInfoRequestProto.getSerializedSize(ClientNamenodeProtocolProtos.java:30108)
~[hadoop-hdfs-2.0.0-cdh4.5.0.jar:na]
at
com.google.protobuf.AbstractMessageLite.toByteString(AbstractMessageLite.java:49)
~[protobuf-java-2.5.0.jar:na]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.constructRpcRequest(ProtobufRpcEngine.java:149)
~[hadoop-common-2.0.0-cdh4.5.0.jar:na]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:193)
~[hadoop-common-2.0.0-cdh4.5.0.jar:na]
at com.sun.proxy.$Proxy33.getFileInfo(Unknown Source) ~[na:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[na:1.7.0_51]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.7.0_51]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.7.0_51]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_51]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
~[hadoop-common-2.0.0-cdh4.5.0.jar:na]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
~[hadoop-common-2.0.0-cdh4.5.0.jar:na]
at com.sun.proxy.$Proxy33.getFileInfo(Unknown Source) ~[na:na]
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:629)
~[hadoop-hdfs-2.0.0-cdh4.5.0.jar:na]
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1545)
~[hadoop-hdfs-2.0.0-cdh4.5.0.jar:na]
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:820)
~[hadoop-hdfs-2.0.0-cdh4.5.0.jar:na]
at
org.apache.drill.exec.store.dfs.shim.fallback.FallbackFileSystem.getFileStatus(FallbackFileSystem.java:84)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.store.dfs.FileSelection.create(FileSelection.java:135)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
at
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.create(WorkspaceSchemaFactory.java:133)
~[drill-java-exec-0.7.0-rebuffed.jar:0.7.0]
{quote}
> can not create hdfs as connection type in storage engine : server throws http
> 500 error
> ---------------------------------------------------------------------------------------
>
> Key: DRILL-1075
> URL: https://issues.apache.org/jira/browse/DRILL-1075
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Vivian Summers
> Assignee: Jacques Nadeau
> Priority: Critical
> Fix For: 0.4.0
>
>
> Server at 8047 throws:
> HTTP ERROR 500
> Problem accessing /storage/config/update. Reason:
> Request failed.
> configure file:
> {
> "type" : "file",
> "enabled" : true,
> "connection" : "hdfs:///",
> "workspaces" : {
> "root" : {
> "location" : "/",
> "writable" : false,
> "storageformat" : null
> },
> "default" : {
> "location" : "/user/root",
> "writable" : true,
> "storageformat" : null
> },
> "tmp" : {
> "location" : "/tmp",
> "writable" : true,
> "storageformat" : "csv"
> }
> },
> "formats" : {
> "psv" : {
> "type" : "text",
> "extensions" : [ "tbl" ],
> "delimiter" : "|"
> },
> "csv" : {
> "type" : "text",
> "extensions" : [ "csv" ],
> "delimiter" : ","
> },
> "tsv" : {
> "type" : "text",
> "extensions" : [ "tsv" ],
> "delimiter" : "\t"
> },
> "parquet" : {
> "type" : "parquet"
> },
> "json" : {
> "type" : "json"
> }
> }
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
