Yup, I found a Drill class which works similarly to what we need, but for
JSON.
It is in class JsonLoaderImpl, method: boolean readBatch();
It fills in a batch of rows (for efficiency), and returns true if more data
is possible, and false if EOF.
The loop is roughly:
until data is EOF or no
It sounds like we want to implement something very similar to what the
Daffodil CLI calls "streaming mode". Something along the lines of this
(making guesses about what the drill API looks like based on reading this):
val input = new InputSourceDataInputStream(inputStream)
def hasNext():
Mike,
I'll add to Paul's comments. While Drill is expecting an iterator style
reader, that iterator pattern really only applies to batches. This concept
took me a while to wrap my head around, but in any of the batch reader classes,
it's important to remember that the next() method is really
Mike,
This is a complex question and has two answers.
First, the standard enhanced vector framework (EVF) used by most readers
assumes a "pull" model: read each record. This is where the next() comes
in: readers just implement this to read the next record. But, the code
under EVF works with a
Daffodil parsing generates event callbacks to an InfosetOutputter, which is
analogous to a SAX event handler.
Drill is expecting an iterator style of calling next() to advance through
the input, i.e., Drill has the control thread and expects to do pull
parsing. At least from the code I studied in
