Re: Standard/Reserved Metadata keys

2021-04-13 Thread Joris Van den Bossche
On Thu, 8 Apr 2021 at 19:52, Micah Kornfield wrote: > 1. Do the standard libraries handle that metadata key automatically ? > > C++, Python and Java have facilities to support them automatically > (extensions needs to register themselves), I'm not sure about other > languages. > > 2. Are there

Re: Standard/Reserved Metadata keys

2021-04-08 Thread Neal Richardson
Like Pandas, in the R package we store R-specific metadata in the schema; this lets us preserve all R class and other attributes when round-tripping data to Feather or Parquet. These are not stored in a way that is readable or useful to other implementations, partly by design. If there were type

Re: Standard/Reserved Metadata keys

2021-04-08 Thread Micah Kornfield
1. Do the standard libraries handle that metadata key automatically ? C++, Python and Java have facilities to support them automatically (extensions needs to register themselves), I'm not sure about other languages. 2. Are there other standard or best practice metadata keys that either people

Standard/Reserved Metadata keys

2021-04-08 Thread Michael Lavina
Hey Team, I noticed that under extension type the Arrow docs specifically call out to `ARROW:extension:name` and `ARROW:extension:metadata` as recommended/reserved metadata keys to handle extension types. 2 quick questions 1. Do the standard libraries handle that metadata key