This is pretty great especially as a primer on Parquet structures. Thanks for sharing
On Fri, Aug 15, 2025 at 8:07 AM Andrew Lamb <[email protected]> wrote: > Hello, > > I recently wrote an article [1] about how to use custom external indexes > and avoid reparsing Parquet footer metadata in each query. > > While this email is definitely a bit of self promotion, I think the blog is > relevant to the discussions about adding new indexes and footer structures > to the format, as it addresses common misconceptions of what is possible > with the current definition. > > Thank you for your time, > Andrew > > p.s. Our previous post [2][3] was about embedding such indexes into the > files themselves > > [1]: > https://datafusion.apache.org/blog/2025/08/15/external-parquet-indexes/ > [2]: https://lists.apache.org/thread/81zgxbtxcgk4zyjzs352l5b56kz4b36o > [3]: > https://datafusion.apache.org/blog/2025/07/14/user-defined-parquet-indexes/ >
