Hi all, I'm attempting to read a nested JSON file compressed with Snappy using the following query:
select * from hdfs.`/tmp/test/part-r-01195.json.snappy` limit 20 However I am encountering the following error: 2016-06-17 09:36:29,385 [USER-rpc-event-queue] ERROR o.a.d.exec.server.rest.QueryWrapper - Query Failed org.apache.drill.common.exceptions.UserRemoteException: DATA_READ ERROR: Failure reading JSON file - native snappy library not available: this version of libhadoop was built without snappy support. File /tmp/test/part-r-01195.json.snappy Record 1 Fragment 0:0 [Error Id: 4d50001a-4449-4614-b647-686f867ff85e on MSR-HDP-DN0300.redmond.corp.microsoft.com:31010] at org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:119) [drill-java-exec-1.6.0.jar:1.6.0] at org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:113) [drill-java-exec-1.6.0.jar:1.6.0] at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:46) [drill-rpc-1.6.0.jar:1.6.0] at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:31) [drill-rpc-1.6.0.jar:1.6.0] at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:67) [drill-rpc-1.6.0.jar:1.6.0] at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:374) [drill-rpc-1.6.0.jar:1.6.0] at org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:89) [drill-rpc-1.6.0.jar:1.6.0] at org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:252) [drill-rpc-1.6.0.jar:1.6.0] at org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:123) [drill-rpc-1.6.0.jar:1.6.0] at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:285) [drill-rpc-1.6.0.jar:1.6.0] at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:257) [drill-rpc-1.6.0.jar:1.6.0] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89) [netty-codec-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254) [netty-handler-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242) [netty-codec-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847) [netty-transport-4.0.27.Final.jar:4.0.27.Final] at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:618) [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na] at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:329) [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na] at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:250) [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na] at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) [netty-common-4.0.27.Final.jar:4.0.27.Final] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75] Our Hadoop cluster supports Snappy compression, I can read Snappy files using the `hadoop fs -text` command, and `hadoop checknative -a` shows that Snappy is configured: 16/06/17 09:49:38 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native 16/06/17 09:49:38 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library Native library checking: hadoop: true /usr/hdp/2.3.4.0-3485/hadoop/lib/native/libhadoop.so.1.0.0 zlib: true /lib/x86_64-linux-gnu/libz.so.1 snappy: true /usr/hdp/2.3.4.0-3485/hadoop/lib/native/libsnappy.so.1 lz4: true revision:99 bzip2: true /lib/x86_64-linux-gnu/libbz2.so.1 openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)! I really appreciate any suggestions you can give. Thanks, Ben