Hi Andrew, I'm glad you got this working! The javascript library only implements the arrow IPC spec, it doesn't have any special handling for feather and its compression support. It's good to know that you can read uncompressed feather files, but I'd only expect it to read an IPC stream or file. This is what I did for the Intro to Arrow JS notebook [1], see scrabble.py here [2]. Note that python script was written many versions of arrow ago, I'm sure there's less boilerplate required for this in pyarrow 2.0.
Support for feather and compression would certainly be a welcome contribution [1] https://observablehq.com/@theneuralbit/introduction-to-apache-arrow [2] https://gist.github.com/TheNeuralBit/64d8cc13050c9b5743281dcf66059de5 On Thu, Dec 17, 2020 at 10:10 AM Andrew Clancy <[email protected]> wrote: > So, I figured out the issue here - I had to remove compression from the > pyarrow feather.write_feather(compression='uncompressed'). Is there any way > to read a compressed feather file in arrow js? > See the comment under the first answer here: > > https://stackoverflow.com/questions/64629670/how-to-write-a-pandas-dataframe-to-arrow-file/64648955#64648955 > I couldn't find anything in the arrow docs or notebooks on this - I'm > assuming that's related to javascript compression libraries being so > limited. > > On Mon, 14 Dec 2020 at 19:02, Andrew Clancy <[email protected]> wrote: > > > Hi, > > > > I have a simple feather file created via a pandas to_feather with a > > datetime64[ns] column, and cannot get timestamps in javascript > > [email protected] > > > > See this notebook: > > https://observablehq.com/@nite/apache-arrow-timestamp-investigation > > > > I'm guessing I'm missing something, has anyone got any suggestions, or > > decent examples of reading a file created in pandas? I've seen in > examples > > of [email protected] where dates stored as an array of 2 ints. > > > > File was created with: > > > > import pandas as pd > > pd.read_parquet('sample.parquet') > > df.to_feather('sample-seconds.feather') > > > > Final Q: I'm assuming this is the best place for this question? Happy to > > post elsewhere if there's any other forums, or if this should be a JIRA > > ticket? > > > > Thanks! > > Andy > > >
