Hi Nicolas,
Yet another suggestion, FWIW. We already have tests for writing JSON and CSV.
We also have tests for reading MapR DB. So, try making changes to those and
seeing if you can get those to run. For example, create a test that reads a
file in CSV, write it to JSON, read it as JSON and
ce the writes to JSON with writes to MapRDB.
> At
> >> least this gives you a place to start looking.
> >>
> >>
> >> A more general solution would be to build the writer using some of the
> >> recent additions to Drill such as the row set mechanisms for reading
ning it must handle
> > the
> > >> Volcano iterator protocol. Looks like you can perhaps use
> > WriterRecordBatch
> > >> as the writer operator itself. (See EasyWriterBatchCreator and follow
> > the
> > >> code to understand the plumbing.)
development path might just be to copy the
> >> JsonRecordWriter, and replace the writes to JSON with writes to MapRDB.
> At
> >> least this gives you a place to start looking.
> >>
> >>
> >> A more general solution would be to build the writer using some
looking.
> >>
> >>
> >> A more general solution would be to build the writer using some of the
> >> recent additions to Drill such as the row set mechanisms for reading a
> >> record batch. But, since copying the JSON approach provides a quick &
&g
xpect. If you create your
>> BatchCreator, but not the writer, things won't actually work, but you can
>> set a breakpoint in the getBatch() method to verify the Drill did find your
>> class. And so on.
>>
>>
; recent additions to Drill such as the row set mechanisms for reading a
> >> record batch. But, since copying the JSON approach provides a quick &
> dirty
> >> solution, perhaps that is good enough for this particular use case.
> >>
> >>
> >> In o
is good enough for this particular use case.
>>
>>
>> In our book, we recommend building each step one-by-one and doing a quick
>> test to verify that each step works as you expect. If you create your
>> BatchCreator, but not the writer, things won't actuall
your
> BatchCreator, but not the writer, things won't actually work, but you can
> set a breakpoint in the getBatch() method to verify the Drill did find your
> class. And so on.
>
>
> Thanks,
> - Paul
>
>
>
> On Thursday, May 30, 2019, 3:05:39 AM PDT, Nicolas A Perez
actually work, but you can set a breakpoint in
the getBatch() method to verify the Drill did find your class. And so on.
Thanks,
- Paul
On Thursday, May 30, 2019, 3:05:39 AM PDT, Nicolas A Perez
wrote:
Can anyone give me an overview of how to implement AbstractRecordWriter?
What are
Can anyone give me an overview of how to implement AbstractRecordWriter?
What are the mechanics it follows, what should I do and so on? It will very
helpful.
Best Regards,
Nicolas A Perez
--
Sent
11 matches
Mail list logo
