We wrote a custom reader in C++ since performance of parsing CSV files matters a lot -- we wanted to do multi-threaded execution of conversion steps, also. I don't know what the performance of commons-csv is but it might be worth doing some benchmarks to see.
On Thu, Jul 18, 2019 at 4:35 AM Ji Liu <[email protected]> wrote: > > Hi all, > > Seems there is no adapter to convert CSV data to Arrow data in Java side > which C++ has. Now we already have JDBC adapter, Orc adapter and Avro > adapter (In progress), I think an adapter for CSV would probably also be > nice. > After a brief discuss with @Micah Kornfield, Apache commons-csv [1] seems an > efficient CSV parser that we could potentially leverage but I don't know if > there are other better options. Any inputs and comments would be appreciated. > > Thanks, > Ji Liu[1]https://commons.apache.org/proper/commons-csv/
