Yup, I found a Drill class which works similarly to what we need, but for
JSON.
It is in class JsonLoaderImpl, method: boolean readBatch();
It fills in a batch of rows (for efficiency), and returns true if more data
is possible, and false if EOF.
The loop is roughly:
until data is EOF or no
It sounds like we want to implement something very similar to what the
Daffodil CLI calls "streaming mode". Something along the lines of this
(making guesses about what the drill API looks like based on reading this):
val input = new InputSourceDataInputStream(inputStream)
def hasNext():
Mike,
I'll add to Paul's comments. While Drill is expecting an iterator style
reader, that iterator pattern really only applies to batches. This concept
took me a while to wrap my head around, but in any of the batch reader classes,
it's important to remember that the next() method is really
