[
https://issues.apache.org/jira/browse/DRILL-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16666295#comment-16666295
]
ASF GitHub Bot commented on DRILL-6809:
---------------------------------------
paul-rogers opened a new pull request #1513: DRILL-6809: Handle repeated map in
schema inference
URL: https://github.com/apache/drill/pull/1513
It turns out that the RowSet utilities build a repeated map without
including the hidden `$offsets$` vector in the metadata for the map. But, other
parts in Drill do include this vector.
The RowSet behavior might be a bug which can be addressed in another PR.
This PR:
* Adds unit tests for map accessors at the row set level. Looks like these
were never added originally. They are a simplified form of the ResultSetLoader
map tests.
* Verified that the schema inference can infer a schema from a repeated map
(using the RowSet style.)
* Added a test to reproduce the case from the bug.
* Made a tweak to the RowSetBuilder to allow access to the RowSetWriter
which is needed by the new tests.
* Couple of minor clean-ups.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> msgpack test case fails same as with json, problem with testing framework
> -------------------------------------------------------------------------
>
> Key: DRILL-6809
> URL: https://issues.apache.org/jira/browse/DRILL-6809
> Project: Apache Drill
> Issue Type: Bug
> Reporter: jean-claude
> Priority: Minor
>
> I'm trying to write a test case for a repeated map scenario. However testing
> framework is unable to infer the schema of the resultset? I'm using the API
> correctly?
>
> Thanks
> jc
>
> // create a test file
> try (OutputStreamWriter w = new OutputStreamWriter(new
> FileOutputStream(new File(testDir, "test.json")))) {
> w.write("\{\"arrayOfMap\":[{\"type\": 3, \"data\": 44}]}\n");
> }
>
> // read the test file
> String sql = "select * from `dfs.data`.`test.json`";
> RowSet actual = client.queryBuilder().sql(sql).rowSet();
>
> // error outputted
> java.lang.NullPointerException: null
> at
> org.apache.drill.exec.physical.rowSet.model.single.SingleSchemaInference.inferVector(SingleSchemaInference.java:70)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.rowSet.model.single.SingleSchemaInference.inferMapSchema(SingleSchemaInference.java:91)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.rowSet.model.single.SingleSchemaInference.inferVector(SingleSchemaInference.java:73)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.rowSet.model.single.SingleSchemaInference.infer(SingleSchemaInference.java:64)
> ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
> at
> org.apache.drill.test.rowSet.DirectRowSet.fromContainer(DirectRowSet.java:77)
> ~[drill-java-exec-1.15.0-SNAPSHOT-tests.jar:1.15.0-SNAPSHOT]
> at org.apache.drill.test.QueryBuilder.rowSet(QueryBuilder.java:363)
> ~[drill-java-exec-1.15.0-SNAPSHOT-tests.jar:1.15.0-SNAPSHOT]
> at
> org.apache.drill.exec.store.msgpack.TestMsgpackRecordReader.testSchemaRepeatedMapWithFieldWithDifferentTypes(TestMsgpackRecordReader.java:245)
> ~[test-classes/:na]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_181]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
