Hi Charles, Looking at the code in your PR, it seems that you are, in fact, using Drill's JSON reader to decode the message JSON. (See [1]). Is that where you are having problems?
Looks like this reader handles JSON passed as a string or from a file? In either case, get a local copy of the JSON, then use the JsonReader directly. The JSON reader wants a container and other knick-knacks which you can create within a test that extends the SubOperatorTest. That framework gives you things like an allocator so you can create the vectors, allocate memory, and so on. This code uses the old JSON reader, so it is going to be pretty fiddly. This reader would greatly benefit from the newer EVF-based JSON reader, but we can work on that later. Thanks, - Paul [1] https://github.com/apache/drill/pull/1892/files#diff-59df95a0bedb082b25742242eef0bb9c On Friday, November 15, 2019, 12:40:22 PM PST, Charles Givre <cgi...@gmail.com> wrote: > On Nov 15, 2019, at 1:39 PM, Paul Rogers <par0...@yahoo.com.INVALID> wrote: > > Hi Charles, > > A thought on debugging deserialization is to not do it in a query. Capture > the JSON returned from a rest call. Write a simple unit test that > deserializes that by itself from a string or file. Deserialization is a bit > of a black art, and is really a problem separate from Drill itself. So dumb non-dev question... How exactly do I do that? I have SeDe unit test(s), but the query in question is failing in the first part of the unit test. @Test public void testSerDe() throws Exception { String sql = "SELECT COUNT(*) FROM http.`/json?lat=36.7201600&lng=-4.4203400&date=2019-10-02`"; String plan = queryBuilder().sql(sql).explainJson(); long cnt = queryBuilder().physical(plan).singletonLong(); assertEquals("Counts should match",1L, cnt); }
