[
https://issues.apache.org/jira/browse/DRILL-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223913#comment-14223913
]
Jason Altekruse commented on DRILL-1728:
----------------------------------------
I think this fix could happen in two places. The message is coming out of the
Drill Field/Map Writer interfaces that allow us to easily read and write
complex data. I think that these need to produce user consumable messages, as
there are other parts of the Drill code (such as the complex parquet reader,
covertFromJSON, kvgen, etc.) that use this interface and will fail in the same
manner currently.
On top of this fix, it would be useful if the various consumers of this
interface, like the json reader would provide additional context information on
top of the low level (but soon to be more readable) error. For the JSON reader
it would be nice if this could include a filename and line number and field in
the JSON document where the error occurred. For components later in execution
this becomes less straight forward, as the data streaming in from several
different incoming operators will come in a random order. It might be possible
to include whatever information we have about the current or last successfully
handled record in the error message to help users identify the issue in the
dataset.
There is also the outstanding patch for just inserting null in the dataset when
there is an error. [~mehant] I could not find the issue number for this, could
you post it here? I think this needs to be merged, and if it does not return
empty list or empty map appropriately when we have an error operating on a
complex type, it should be enhanced to give the same flexibility if the users
need to interact with imperfect complex data.
> Better error messages on Drill JSON read error
> ----------------------------------------------
>
> Key: DRILL-1728
> URL: https://issues.apache.org/jira/browse/DRILL-1728
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Tomer Shiran
> Assignee: Jason Altekruse
> Fix For: 0.7.0
>
>
> {code}
> 0: jdbc:drill:zk=localhost:2181> SELECT * FROM
> dfs.root.`Users/tshiran/Development/demo/data/yelp/business.json` WHERE true
> and REPEATED_CONTAINS(categories, 'Australian');
> +-------------+--------------+------------+------------+------------+------------+--------------+------------+------------+------------+------------+------------+------------+------------+---------------+
> | business_id | full_address | hours | open | categories | city
> | review_count | name | longitude | state | stars |
> latitude | attributes | type | neighborhoods |
> +-------------+--------------+------------+------------+------------+------------+--------------+------------+------------+------------+------------+------------+------------+------------+---------------+
> Query failed: Query stopeed., You tried to start when you are using a
> ValueWriter of type NullableBitWriterImpl. [
> e5bafa1e-6226-443d-80fd-51e18f330899 on 172.17.3.132:31010 ]
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing
> query.
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
