divjotarora commented on PR #3709: URL: https://github.com/apache/parquet-java/pull/3709#issuecomment-5496517622
> Thanks @divjotarora for working on this. I'm wondering if it wouldn't be easier to pull the structures from the snapshots repository: https://repository.apache.org/content/groups/snapshots/org/apache/parquet/parquet-format-structures/. It looks like it is missing some versions, but I think it would make sense to just take the snapshot from there and avoid a shell script. @Fokko thanks for taking a look. The suggestion to pull in parquet-format via Maven solves the issue for local development and unblocks building reference implementations against a local copy of `parquet.thrift`. However, it does not make CI pass because CI will still reference a commit hash of parquet-format. FWIW the motivation for this change was that the Rust and C++ Parquet implementations both vendor this file (Rust technically has hand-rolled Thrift structs and a custom parser, so they don't vendor parquet.thrift but they do make custom changes to those structs for reference implementations and it's conceptually the same thing), so there is precedent for this. Based on my investigation, vendoring the file is the most flexible option. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
