OK, originally this was part of https://issues.apache.org/jira/browse/ARROW-12006 and I was going to just add some doc on flatc, but I will make this a new bug because it's a little bigger: https://issues.apache.org/jira/browse/ARROW-12111
On 2021/03/23 23:40:50, Micah Kornfield <emkornfi...@gmail.com> wrote: > > > > I have a concern, though. Four other languages (Java would be five) check > > in the generated flatbuffers code, and it appears (based on a quick scan of > > Git logs) that this is done manually. Is there a danger that the binary > > format could change, but some language might get forgotten, and thus be > > working with the old format? > > The format changes relatively slowly and any changes at this point should > be backwards compatible. > > > > > Or is there enough interop testing that the problem would get caught right > > away? > > In most cases I would expect integration tests to catch these types of > error. > > On Tue, Mar 23, 2021 at 4:26 PM bobtins <bobti...@gmail.com> wrote: > > > I'm happy to check in the generated Java source. I would also update the > > Java build info to reflect this change and document how to regenerate the > > source as needed. > > > > I have a concern, though. Four other languages (Java would be five) check > > in the generated flatbuffers code, and it appears (based on a quick scan of > > Git logs) that this is done manually. Is there a danger that the binary > > format could change, but some language might get forgotten, and thus be > > working with the old format? Or is there enough interop testing that the > > problem would get caught right away? > > > > I'm new to the project and don't know how big of an issue this is in > > practice. Thanks for any enlightenment. > > > > On 2021/03/23 07:39:16, Micah Kornfield <emkornfi...@gmail.com> wrote: > > > I think checking in the java files is fine and probably better then > > relying > > > on a third party package. We should make sure there are instructions on > > > how to regenerate them along with the PR > > > > > > On Monday, March 22, 2021, Antoine Pitrou <anto...@python.org> wrote: > > > > > > > > > > > Le 22/03/2021 à 20:17, bobtins a écrit : > > > > > > > >> TL;DR: The Java implementation doesn't have generated flatbuffers code > > > >> under source control, and the code generation depends on an > > > >> unofficially-maintained Maven artifact. Other language > > implementations do > > > >> check in the generated code; would it make sense for this to be done > > for > > > >> Java as well? > > > >> > > > >> I'm currently focusing on Java development; I started building on > > Windows > > > >> and got a failure under java/format, because I couldn't download the > > > >> flatbuffers compiler (flatc) to generate Java source. > > > >> The artifact for the flatc binary is provided "unofficially" (not by > > the > > > >> flatbuffers project), and there was no Windows version, so I had to > > jump > > > >> through hoops to build it and proceed. > > > >> > > > > > > > > While this does not answer the more general question of checking in the > > > > generated Flatbuffers code (which sounds like a good idea, but I'm not > > a > > > > Java developer), note that you could workaround this by installing the > > > > Conda-provided flatbuffers package: > > > > > > > > $ conda install flatbuffers > > > > > > > > which should get you the `flatc` compiler, even on Windows. > > > > (see https://docs.conda.io/projects/conda/en/latest/ for installing > > conda) > > > > > > > > You may also try other package managers such as Chocolatey: > > > > > > > > https://chocolatey.org/packages/flatc > > > > > > > > Regards > > > > > > > > Antoine. > > > > > > > > > >
