[jira] [Commented] (DRILL-8284) Apache SQL Query failing while accessing the Json with complex data model
[
https://issues.apache.org/jira/browse/DRILL-8284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583946#comment-17583946
]
Charles Givre commented on DRILL-8284:
--
[~shubhamsmvdu] This is normal behavior for Drill. The issue you are
encountering is a schema change exception on the `value` field. In both cases,
what is happening is that Drill first encounters one data type and creates a
vector for that, then in the next row, encounters the same field but in a
different data type and throws an exception.
The are a few options:
# If you use the v1 JSON reader, you can enable the UNION data type which
allows heterogeneous data types. We are working on enabling this for the V2
JSON reader, but for the moment, it is not. This is a variable which must be
set at the system level.
# Provide a schema: You can provide a schema for the field `value` and set
`mode` to JSON. I'd have to dig up the documentation for this but what this
does is force the field to a string. If JSON objects are encountered, those
will be rendered as a string.
I'm going to close this as this is expected behavior. Please use github issues
or slack to continue the conversation.
> Apache SQL Query failing while accessing the Json with complex data model
> -
>
> Key: DRILL-8284
> URL: https://issues.apache.org/jira/browse/DRILL-8284
> Project: Apache Drill
> Issue Type: Bug
>Reporter: SHUBHAM KUMAR
>Priority: Major
>
> Apache SQL Query failing while accessing the Json with complex data model.
> Complex Json:
> Map object inside another map object then Array Object.
> Case1: When we have nested objects within array map, and map within map.
> {"attributes": [
> {
> "name": "webBrandName",
> "value": {
> "en-US": "Smashbox"
> }
> },
> {
> "name": "startDate",
> "value": "2011-07-25T15:30:00.000Z"
> }
> ]
> }
> Case2: Having array with multiple map items with diff data types. eg. String
> and Boolean both type.
> {"attributes": [
> {
> "name": "startDate",
> "value": "2011-07-25T15:30:00.000Z"
> },
> {
> "name": "hasCBD",
> "value": false
> }
> ]
> }
> Query:
> select flatten(attributes) as Var from dfs.`/filepath/filename.json`
>
> Error:
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> IndexOutOfBoundsException: readerIndex: 0, writerIndex: 1764642048 (expected:
> 0 <= readerIndex <= writerIndex <= capacity(0)) Fragment: 0:0 Please, refer
> to logs for more information. [Error Id: c5a3b8fa-cad1-4c9a-8673-de5745e9170b
> on GGNUWT461535L.ad.infosys.com:31010]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8284) Apache SQL Query failing while accessing the Json with complex data model
[
https://issues.apache.org/jira/browse/DRILL-8284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17583896#comment-17583896
]
SHUBHAM KUMAR commented on DRILL-8284:
--
Json sample for case1:
{"attributes": [
{
"name": "webBrandName",
"value": {
"en-US": "Smashbox"
}
},
{
"name": "startDate",
"value": "2011-07-25T15:30:00.000Z"
}
]
}
Json sample for case2:
{"attributes": [
{
"name": "startDate",
"value": "2011-07-25T15:30:00.000Z"
},
{
"name": "hasCBD",
"value": false
}
]
}
> Apache SQL Query failing while accessing the Json with complex data model
> -
>
> Key: DRILL-8284
> URL: https://issues.apache.org/jira/browse/DRILL-8284
> Project: Apache Drill
> Issue Type: Bug
>Reporter: SHUBHAM KUMAR
>Priority: Major
>
> Apache SQL Query failing while accessing the Json with complex data model.
> Complex Json:
> Map object inside another map object then Array Object.
> Case1: When we have nested objects within array map, and map within map.
> {"attributes": [
> {
> "name": "webBrandName",
> "value": {
> "en-US": "Smashbox"
> }
> },
> {
> "name": "startDate",
> "value": "2011-07-25T15:30:00.000Z"
> }
> ]
> }
> Case2: Having array with multiple map items with diff data types. eg. String
> and Boolean both type.
> {"attributes": [
> {
> "name": "startDate",
> "value": "2011-07-25T15:30:00.000Z"
> },
> {
> "name": "hasCBD",
> "value": false
> }
> ]
> }
> Query:
> select flatten(attributes) as Var from dfs.`/filepath/filename.json`
>
> Error:
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> IndexOutOfBoundsException: readerIndex: 0, writerIndex: 1764642048 (expected:
> 0 <= readerIndex <= writerIndex <= capacity(0)) Fragment: 0:0 Please, refer
> to logs for more information. [Error Id: c5a3b8fa-cad1-4c9a-8673-de5745e9170b
> on GGNUWT461535L.ad.infosys.com:31010]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
