I wouldn't be too excited about this. Here are my thoughts: 1. Having the format/ directory in apache/arrow be a submodule would be cumbersome and error-prone for developers. The only submodules we have right now are optional testing dependencies — not having these initialized and updated does not result in a broken project, whereas this change would. We have a copy of parquet.thrift from apache/parquet-format for similar reasons.
2. So based on #1, we would want to maintain a copy of the format files in apache/arrow, even if there were a separate apache/arrow-format repository. The format files are slow-moving enough that I don't think it's burdensome to mirror these into satellite repositories like arrow/arrow-rs. On Tue, Apr 27, 2021 at 10:54 AM Neville Dipale <nevilled...@gmail.com> wrote: > Hi Arrow devs, > > Andy noticed that we carry a copy of the format directory in arrow-rs, > which > is bound to get outdated in the future. > > We would like to propose creating an arrow-format repository, similar to > parquet-format, so that arrow-rs and other future separate repositories > could > add this as a submodule. > > What are your thoughts? > > Regards > Neville >
