[jira] [Commented] (DRILL-5983) Unsupported nullable converted type INT_8 for primitive type INT32 error
[ https://issues.apache.org/jira/browse/DRILL-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945849#comment-16945849 ] ASF GitHub Bot commented on DRILL-5983: --- asfgit commented on pull request #1866: DRILL-5983: Add missing nullable Parquet readers for INT and UINT logical types URL: https://github.com/apache/drill/pull/1866 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Unsupported nullable converted type INT_8 for primitive type INT32 error > > > Key: DRILL-5983 > URL: https://issues.apache.org/jira/browse/DRILL-5983 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Data Types >Affects Versions: 1.10.0, 1.11.0 > Environment: NAME="Ubuntu" > VERSION="16.04.2 LTS (Xenial Xerus)" >Reporter: Hakan Sarıbıyık >Assignee: Arina Ielchiieva >Priority: Major > Labels: parquet, read, ready-to-commit, types > Fix For: 1.17.0 > > Attachments: test.parquet > > > When I query a table with byte in it, then it gives an error; > _Query Failed: An Error Occurred > org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: > ExecutionSetupException: Unsupported nullable converted type INT_8 for > primitive type INT32 Fragment 1:6 [Error Id: > 46636b05-cff5-455b-ba25-527217346b3e on bigdata7:31010]_ > Actualy, it has been solved with > [DRILL-4764] - Parquet file with INT_16, etc. logical types not supported by > simple SELECT > according to https://drill.apache.org/docs/apache-drill-1-10-0-release-notes/ > But i tried it with even 1-11-0 it didnt worked. > I am querying parquet formatted file with pySpark > tablo1 > sourceid: byte (nullable = true) > select sourceid from tablo1 > works as expected with pySpark. But not with Drill v1.11.0 > Thanx. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-5983) Unsupported nullable converted type INT_8 for primitive type INT32 error
[ https://issues.apache.org/jira/browse/DRILL-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944470#comment-16944470 ] ASF GitHub Bot commented on DRILL-5983: --- arina-ielchiieva commented on pull request #1866: DRILL-5983: Add missing nullable Parquet readers for INT and UINT logical types URL: https://github.com/apache/drill/pull/1866 Jira - [DRILL-5983](https://issues.apache.org/jira/browse/DRILL-5983). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Unsupported nullable converted type INT_8 for primitive type INT32 error > > > Key: DRILL-5983 > URL: https://issues.apache.org/jira/browse/DRILL-5983 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Data Types >Affects Versions: 1.10.0, 1.11.0 > Environment: NAME="Ubuntu" > VERSION="16.04.2 LTS (Xenial Xerus)" >Reporter: Hakan Sarıbıyık >Assignee: Arina Ielchiieva >Priority: Major > Labels: parquet, read, types > Fix For: 1.17.0 > > Attachments: test.parquet > > > When I query a table with byte in it, then it gives an error; > _Query Failed: An Error Occurred > org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: > ExecutionSetupException: Unsupported nullable converted type INT_8 for > primitive type INT32 Fragment 1:6 [Error Id: > 46636b05-cff5-455b-ba25-527217346b3e on bigdata7:31010]_ > Actualy, it has been solved with > [DRILL-4764] - Parquet file with INT_16, etc. logical types not supported by > simple SELECT > according to https://drill.apache.org/docs/apache-drill-1-10-0-release-notes/ > But i tried it with even 1-11-0 it didnt worked. > I am querying parquet formatted file with pySpark > tablo1 > sourceid: byte (nullable = true) > select sourceid from tablo1 > works as expected with pySpark. But not with Drill v1.11.0 > Thanx. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-5983) Unsupported nullable converted type INT_8 for primitive type INT32 error
[
https://issues.apache.org/jira/browse/DRILL-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874263#comment-16874263
]
Stuart Teasdale commented on DRILL-5983:
I'm seeing basically the same issue with Drill 1.16.0. I've attached the file
that does this
> select * from test;
Error: INTERNAL_ERROR ERROR: Error in parquet record reader.
Message: Failure in setting up reader
Parquet Metadata: ParquetMetaData{FileMetaData{schema: message schema {
optional int32 chrom_int (INT_8);
optional int64 __index_level_0__;
}
, metadata: \{pandas={"index_columns": ["__index_level_0__"], "column_indexes":
[{"name": null, "field_name": null, "pandas_type": "unicode", "numpy_type":
"object", "metadata": {"encoding": "UTF-8"}}], "columns": [\{"name":
"chrom_int", "field_name": "chrom_int", "pandas_type": "int8", "numpy_type":
"int8", "metadata": null}, \{"name": null, "field_name": "__index_level_0__",
"pandas_type": "int64", "numpy_type": "int64", "metadata": null}],
"pandas_version": "0.24.2"}}}, blocks: [BlockMetaData\{1, 140
[ColumnMetaData{SNAPPY [chrom_int] optional int32 chrom_int (INT_8)
[PLAIN_DICTIONARY, RLE, PLAIN], 24}, ColumnMetaData\{SNAPPY [__index_level_0__]
optional int64 __index_level_0__ [PLAIN_DICTIONARY, RLE, PLAIN], 158}]}]}
Fragment 0:0
Please, refer to logs for more information.
[Error Id: c3e0e2ea-0e51-4732-96af-88ffe669b22c on
mephistopheles.londc.genomicsplc.com:31010] (state=,code=0)
and from the logs:
org.apache.drill.common.exceptions.DrillRuntimeException: Error in parquet
record reader.
Message: Failure in setting up reader
Parquet Metadata: ParquetMetaData{FileMetaData{schema: message schema {
optional int32 chrom_int (INT_8);
optional int64 __index_level_0__;
}
, metadata: \{pandas={"index_columns": ["__index_level_0__"], "column_indexes":
[{"name": null, "field_name": null, "pandas_type": "unicode", "numpy_type":
"object", "metadata": {"encoding": "UTF-8"}}], "columns": [\{"name":
"chrom_int", "field_name": "chrom_int", "pandas_type": "int8", "numpy_type":
"int8", "metadata": null}, \{"name": null, "field_name": "__index_level_0__",
"pandas_type": "int64", "numpy_type": "int64", "metadata": null}],
"pandas_version": "0.24.2"}}}, blocks: [BlockMetaData\{1, 140
[ColumnMetaData{SNAPPY [chrom_int] optional int32 chrom_int (INT_8)
[PLAIN_DICTIONARY, RLE, PLAIN], 24}, ColumnMetaData\{SNAPPY [__index_level_0__]
optional int64 __index_level_0__ [PLAIN_DICTIONARY, RLE, PLAIN], 158}]}]}
at
org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.handleException(ParquetRecordReader.java:269)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.setup(ParquetRecordReader.java:253)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.physical.impl.ScanBatch.getNextReaderIfHas(ScanBatch.java:321)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.physical.impl.ScanBatch.internalNext(ScanBatch.java:216)
~[drill-java-exec-1.16.0.jar:1.16.0]
at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:271)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:116)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:141)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83)
~[drill-java-exec-1.16.0.jar:1.16.0]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283)
~[drill-java-exec-1.16.0.jar:1.16.0]
at ...(:0) ~[na:na]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
~[hadoop-common-2.7.4.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:283)
~[drill-java-exec-1.16.0.jar:1.16.0]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
~[drill-common-1.16.0.jar:1.16.0]
at ...(:0) ~[na:na]
Caused by:
[jira] [Commented] (DRILL-5983) Unsupported nullable converted type INT_8 for primitive type INT32 error
[ https://issues.apache.org/jira/browse/DRILL-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844669#comment-16844669 ] Volodymyr Vysotskyi commented on DRILL-5983: [~vezir], [~davlee1...@yahoo.com], could you please share parquet files for which Drill fails, since I tried files from the DRILL-4764, and on Drill 1.16.0 everything works fine. > Unsupported nullable converted type INT_8 for primitive type INT32 error > > > Key: DRILL-5983 > URL: https://issues.apache.org/jira/browse/DRILL-5983 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Data Types >Affects Versions: 1.10.0, 1.11.0 > Environment: NAME="Ubuntu" > VERSION="16.04.2 LTS (Xenial Xerus)" >Reporter: Hakan Sarıbıyık >Priority: Major > Labels: parquet, read, types > > When I query a table with byte in it, then it gives an error; > _Query Failed: An Error Occurred > org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: > ExecutionSetupException: Unsupported nullable converted type INT_8 for > primitive type INT32 Fragment 1:6 [Error Id: > 46636b05-cff5-455b-ba25-527217346b3e on bigdata7:31010]_ > Actualy, it has been solved with > [DRILL-4764] - Parquet file with INT_16, etc. logical types not supported by > simple SELECT > according to https://drill.apache.org/docs/apache-drill-1-10-0-release-notes/ > But i tried it with even 1-11-0 it didnt worked. > I am querying parquet formatted file with pySpark > tablo1 > sourceid: byte (nullable = true) > select sourceid from tablo1 > works as expected with pySpark. But not with Drill v1.11.0 > Thanx. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
