GitHub user dentiny edited a discussion: Other format converting to arrow
Hi community, in my daily work, I heavily do a bunch of format conversion (i.e., WARC / ATIF to lance, etc), and I use arrow as an intermediate format. I notice there's already some crates doing the conversion work in the repo, for example, [csv](https://github.com/apache/arrow-rs/tree/main/arrow-csv) and [json](https://github.com/apache/arrow-rs/tree/main/arrow-json), I'm wondering if there's a procedure or place so I could contribute to other format conversions, so we could have a centralized place? Thank you! Some of my work - [ATIF](https://docs.harborframework.com/agents/atif) to arrow: https://github.com/dentiny/atif-arrow - [WARC](https://en.wikipedia.org/wiki/WARC_(file_format)) to arrow: https://github.com/dentiny/warc-rs - A DuckDB extension to convert all formats I need to handle, all via arrow: https://github.com/dentiny/duckdb_lance_conversion GitHub link: https://github.com/apache/arrow-rs/discussions/11633 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
