Thanks for the suggestions. The data is coming from Flight. We found the process of using read_pandas() and then converting the data to JSON to be an area we might be able to improve on. Getting the data into an Arrow Table (calling read_all() on the FlightStreamReader) took about as long as read_pandas().
On Wed, Dec 7, 2022 at 3:36 PM Jacob Quinn <[email protected]> wrote: > The Julia implementation (Arrow.jl) has the ability to output an > Arrow.Table as a JSON array of objects or object of arrays via the > JSONTables.jl package, like: > > using Arrow, JSONTables > > julia> t = Arrow.Table("tbl.arrow") > Arrow.Table with 4 rows, 1 columns, and schema: > :a String > > julia> arraytable(t) > "[{\"a\":null},{\"a\":\"a\"},{\"a\":\"b\"},{\"a\":\"a\"}]" > > julia> objecttable(t) > "{\"a\":[null,\"a\",\"b\",\"a\"]}" > > > You can also provide an IO argument as the 1st argument to > arraytable/objecttable to write the results out to a file directly. > > In case that helps at all! > > -Jacob > > On Wed, Dec 7, 2022 at 9:20 AM James Duong via user <[email protected]> > wrote: > >> Hi, >> >> Is there a way to write JSON files from Arrow Tables? I see other formats >> such as CSV and Parquet supported for both, but only reading for JSON. >> >> -- >> >> *James Duong* >> Lead Software Developer >> Bit Quill Technologies Inc. >> Direct: +1.604.562.6082 | [email protected] >> https://www.bitquilltech.com >> >> This email message is for the sole use of the intended recipient(s) and >> may contain confidential and privileged information. Any unauthorized >> review, use, disclosure, or distribution is prohibited. If you are not the >> intended recipient, please contact the sender by reply email and destroy >> all copies of the original message. Thank you. >> >
