[jira] [Commented] (DRILL-4054) convert_from(,'JSON') gives JsonParseException
[ https://issues.apache.org/jira/browse/DRILL-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997926#comment-14997926 ] Steven Phillips commented on DRILL-4054: (1) I agree the message is bad. If that's the issue, please be explicit. It wasn't clear from the description whether there was an actual bug or just a request for better error message. (2) This doesn't seem to have anything to do with the original bug. In fact, this isn't even a bug. The convert_from function requires a varbinary or varchar input. It is not possible to perform this function against a MAP type. > convert_from(,'JSON') gives JsonParseException > --- > > Key: DRILL-4054 > URL: https://issues.apache.org/jira/browse/DRILL-4054 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Data Types >Affects Versions: 1.3.0 >Reporter: Khurram Faraaz > > convert_from(,'JSON') gives JsonParseException > sys.version => 3a73f098 > Drill 1.3 > 4 node cluster CentOS > {code} > 0: jdbc:drill:schema=dfs.tmp> select columns[3], convert_from(CAST(columns[3] > AS VARCHAR(64)),'JSON') json FROM `allData.csv`; > Error: SYSTEM ERROR: JsonParseException: Unrecognized token > 'AXCB': was expecting > ('true', 'false' or 'null') > at [Source: > org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line: > 1, column: 105] > Fragment 0:0 > [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010] > (state=,code=0) > Stack trace from drillbit.log > [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > JsonParseException: Unrecognized token > 'AXCB': was expecting > ('true', 'false' or 'null') > at [Source: > org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line: > 1, column: 105] > Fragment 0:0 > [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534) > ~[drill-common-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321) > [drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184) > [drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290) > [drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.3.0.jar:1.3.0] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_85] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_85] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85] > Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error > while converting from JSON. > at > org.apache.drill.exec.test.generated.ProjectorGen6.doEval(ProjectorTemplate.java:126) > ~[na:na] > at > org.apache.drill.exec.test.generated.ProjectorGen6.projectRecords(ProjectorTemplate.java:62) > ~[na:na] > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:174) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:131) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:156) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:80) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:250) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at java.security.AccessController.doPrivileged(Native Method) > ~[na:1.7.0_85] > at javax.security.auth.Subject.doAs(Subjec
[jira] [Commented] (DRILL-4054) convert_from(,'JSON') gives JsonParseException
[ https://issues.apache.org/jira/browse/DRILL-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997917#comment-14997917 ] Khurram Faraaz commented on DRILL-4054: --- (1) It is not an easy to understand error message. (2) When we pass a valid JSON as input source we see a SchemaChangeException {code} 0: jdbc:drill:schema=dfs.tmp> select t.c1, convert_from(t.c1,'JSON') jsn from `test.json` t; Error: SYSTEM ERROR: SchemaChangeException: Failure while trying to materialize incoming schema. Errors: Error in expression at index -1. Error: Missing function implementation: [convert_fromjson(MAP-REQUIRED)]. Full expression: --UNKNOWN EXPRESSION--.. Fragment 0:0 [Error Id: dd9c7d11-fb07-436d-adac-c5efd9372cb4 on centos-04.qa.lab:31010] (state=,code=0) {code} Data from test.son file used in above test {code} [root@centos-01 ~]# cat test.json { "c1":{"key":"value1"} } [root@centos-01 ~]# {code} > convert_from(,'JSON') gives JsonParseException > --- > > Key: DRILL-4054 > URL: https://issues.apache.org/jira/browse/DRILL-4054 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Data Types >Affects Versions: 1.3.0 >Reporter: Khurram Faraaz > > convert_from(,'JSON') gives JsonParseException > sys.version => 3a73f098 > Drill 1.3 > 4 node cluster CentOS > {code} > 0: jdbc:drill:schema=dfs.tmp> select columns[3], convert_from(CAST(columns[3] > AS VARCHAR(64)),'JSON') json FROM `allData.csv`; > Error: SYSTEM ERROR: JsonParseException: Unrecognized token > 'AXCB': was expecting > ('true', 'false' or 'null') > at [Source: > org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line: > 1, column: 105] > Fragment 0:0 > [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010] > (state=,code=0) > Stack trace from drillbit.log > [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > JsonParseException: Unrecognized token > 'AXCB': was expecting > ('true', 'false' or 'null') > at [Source: > org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line: > 1, column: 105] > Fragment 0:0 > [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534) > ~[drill-common-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321) > [drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184) > [drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290) > [drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.3.0.jar:1.3.0] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_85] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_85] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85] > Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error > while converting from JSON. > at > org.apache.drill.exec.test.generated.ProjectorGen6.doEval(ProjectorTemplate.java:126) > ~[na:na] > at > org.apache.drill.exec.test.generated.ProjectorGen6.projectRecords(ProjectorTemplate.java:62) > ~[na:na] > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:174) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:131) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:156) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:80) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256)
[jira] [Commented] (DRILL-4054) convert_from(,'JSON') gives JsonParseException
[ https://issues.apache.org/jira/browse/DRILL-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997896#comment-14997896 ] Steven Phillips commented on DRILL-4054: Could you explain what the bug is here? > convert_from(,'JSON') gives JsonParseException > --- > > Key: DRILL-4054 > URL: https://issues.apache.org/jira/browse/DRILL-4054 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Data Types >Affects Versions: 1.3.0 >Reporter: Khurram Faraaz > > convert_from(,'JSON') gives JsonParseException > sys.version => 3a73f098 > Drill 1.3 > 4 node cluster CentOS > {code} > 0: jdbc:drill:schema=dfs.tmp> select columns[3], convert_from(CAST(columns[3] > AS VARCHAR(64)),'JSON') json FROM `allData.csv`; > Error: SYSTEM ERROR: JsonParseException: Unrecognized token > 'AXCB': was expecting > ('true', 'false' or 'null') > at [Source: > org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line: > 1, column: 105] > Fragment 0:0 > [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010] > (state=,code=0) > Stack trace from drillbit.log > [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > JsonParseException: Unrecognized token > 'AXCB': was expecting > ('true', 'false' or 'null') > at [Source: > org.apache.drill.exec.vector.complex.fn.DrillBufInputStream@5441715d; line: > 1, column: 105] > Fragment 0:0 > [Error Id: 7f8cb677-20e9-4e99-bbec-3ada707671ee on centos-03.qa.lab:31010] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534) > ~[drill-common-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321) > [drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184) > [drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290) > [drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.3.0.jar:1.3.0] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_85] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_85] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_85] > Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Error > while converting from JSON. > at > org.apache.drill.exec.test.generated.ProjectorGen6.doEval(ProjectorTemplate.java:126) > ~[na:na] > at > org.apache.drill.exec.test.generated.ProjectorGen6.projectRecords(ProjectorTemplate.java:62) > ~[na:na] > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:174) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:131) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:156) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:80) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:250) > ~[drill-java-exec-1.3.0.jar:1.3.0] > at java.security.AccessController.doPrivileged(Native Method) > ~[na:1.7.0_85] > at javax.security.auth.Subject.doAs(Subject.java:415) ~[na:1.7.0_85] > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595) > ~[hadoop-common-2.7.0-mapr-1506.jar:na] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:250) > [drill-java-exec-1.3.0.jar:1.3.0] > ... 4 common frames omitted > Caused by: com.fasterxml.jackson