David Smith created DRILL-5128: ---------------------------------- Summary: Issue with unable to create parquet file on dfs workspace Key: DRILL-5128 URL: https://issues.apache.org/jira/browse/DRILL-5128 Project: Apache Drill Issue Type: Bug Components: Storage - Parquet Affects Versions: 1.9.0 Environment: docker instance built on harisekhon/apache-drill Reporter: David Smith
Unable to write a parquet file to dfs workspace with CTAS from an Oracle query. Oracle querying works fine using ojdbc6.jar in /jars/3rdparty and configured via enabled storage plugin Created dfs "data" folder and storage workspace: {quote} drwxr-xr-x 3 root root 4096 Dec 13 16:28 data {quote} {quote} "data": { "location": "/data", "writable": true, "defaultInputFormat": null }, { {quote} and set up to use it: {quote} use dfs.data; alter session set `store.format` = 'parquet' {quote} and then used CTAS to query Oracle for a test sample to write out: {quote} create table dfs.data.tsdtest1 as select * from enviroi.D_EF_FAC.FRS_INTEREST where INTEREST_TYPE = 'TSD' LIMIT 10; {quote} Which then throws the error {quote} Exception in thread "drill-executor-28" java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy at org.apache.parquet.hadoop.DirectCodecFactory$SnappyCompressor.compress(DirectCodecFactory.java:290) at org.apache.parquet.hadoop.ColumnChunkPageWriteStore$ColumnChunkPageWriter.writePage(ColumnChunkPageWriteStore.java:94) at org.apache.parquet.column.impl.ColumnWriterV1.writePage(ColumnWriterV1.java:154) at org.apache.parquet.column.impl.ColumnWriterV1.flush(ColumnWriterV1.ja va:242) at org.apache.parquet.column.impl.ColumnWriteStoreV1.flush(ColumnWriteStoreV1.java:129) at org.apache.drill.exec.store.parquet.ParquetRecordWriter.flush(Parquet RecordWriter.java:253) at org.apache.drill.exec.store.parquet.ParquetRecordWriter.cleanup(Parqu etRecordWriter.java:381) at org.apache.drill.exec.physical.impl.WriterRecordBatch.closeWriter(Wri terRecordBatch.java:180) at org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(Write rRecordBatch.java:128) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordB atch.java:162) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordB atch.java:119) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordB atch.java:109) at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(Abst ractSingleRecordBatch.java:51) at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerN ext(ProjectRecordBatch.java:135) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordB atch.java:162) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.ja va:104) at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNex t(ScreenCreator.java:81) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.ja va:94) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentEx ecutor.java:232) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:226) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:226) at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor .java:617) at java.lang.Thread.run(Thread.java:745) Error: SYSTEM ERROR: IOException: The file being written is in an invalid state. Probably caused by an error thrown previously. Current state: BLOCK Fragment 0:0 [Error Id: 23f78b7e-f15c-461a-8179-2738ebfcf4a7 on 8637c49541d:31010] (state=,code=0) {quote} The process did create \data\tsdtest1 with a file, 0_0_0.parquet: {quote} bash-4.3# ls -l tsdtest1 total 0 -rw-r--r-- 1 root root 0 Dec 13 16:28 0_0_0.parquet {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)