Thank you all for your input. It sounds like we are *not* expanding the features of the bundled dependency system, but instead are focused on better support on major packaging systems. I will close ARROW-17295 [1].
[1] https://issues.apache.org/jira/browse/ARROW-17295 On Sun, Aug 7, 2022 at 7:26 PM Sutou Kouhei <k...@clear-code.com> wrote: > Hi, > > > (a) To expand its functionality and encourage use by downstream projects > > I think that this is not an option for us. > > If we do this, we need to release a security release as soon > as possible when any security vulnerability is found in > bundle libraries. I think that we can't do it at least for > now. > > > (b) Keep as is, and consider it primarily for internal use > > (c) Move away from it in favor of using vcpkg or conan > > I considered them. I think that we can choose (c) in the > long run. But we need to choose (b) at least for now. > > If we choose (c), we can remove many macro(build_XXX) in > cpp/cmake_modules/ThirdpartyToolchain.cmake. This is very > happy. :-) > > But we need to prepare dependencies by other ways for each > packaging platforms instead. Because we can't mix multiple > packaging systems generally. If we mix multiple packaging > systems, some common libraries may be conflicted. For > example, OpenSSL installed by deb and by vcpkg may be > conflicted. > > For example, we send a pull request that adds > google-cloud-cpp to Homebrew because Homebrew doesn't > provide google-cloud-cpp formula yet. Then, we can enable > GCS support in apache-arrow formula. > > We can choose (c) when all (or most) of existing > dependencies are ready for our major supported packaging > systems: > > * Conan > * Conda > * Homebrew > * MSYS2 > * RPM > * deb > * vcpkg > > Regarding RPM/deb, we'll add packages for dependencies to > https://apache.jfrog.io/ui/native/arrow/almalinux/ , > https://apache.jfrog.io/ui/native/arrow/debian/ and so on. > > > Regarding (c)'s backend, I think that Conan is better than > vcpkg because vcpkg doesn't support downloading compiled > binaries from a public server yet: > > > https://github.com/microsoft/vcpkg/blob/master/docs/about/faq.md#will-vcpkg-support-downloading-compiled-binaries-from-a-public-or-private-server > > I think that building all dependencies on each > users/developers' machine should be avoided as much as > possible. > > > Thanks, > -- > kou > > In <cah6bbm3eaaw_rvbmnn4iampjo4tgr8k+2et4toyzcrxm0kn...@mail.gmail.com> > "[C++] Purpose of C++ bundled dependencies" on Wed, 3 Aug 2022 10:57:48 > -0700, > Will Jones <will.jones...@gmail.com> wrote: > > > I was creating this ticket ARROW-17295 [1], but ended up unsure if this > is > > something we'd like to maintain, so I thought I would bring it up for > > discussion. Essentially: should we expand the capabilities of our bundled > > dependency system? Or should we constrain the scope and point users that > > wish for more functionality toward package managers such as vcpkg and > Conan? > > > > My understanding is that our bundled dependency system was created to > fill > > the gaps where package managers failed to provide working builds or a > > recent enough version of a package. More recently, we added support for > > vcpkg and have started to use that in many of our CI and packaging builds > > [2]. And in the past few months, we have worked on adding support for > Conan > > [3]. > > > > So my question is what is the future of Arrow C++'s bundled dependency > > system? Do we want: > > (a) To expand its functionality and encourage use by downstream projects > > (b) Keep as is, and consider it primarily for internal use > > (c) Move away from it in favor of using vcpkg or conan > > > > [1] https://issues.apache.org/jira/browse/ARROW-17295 > > [2] > > > https://github.com/apache/arrow/blob/master/dev/tasks/python-wheels/github.osx.amd64.yml > > [3] https://issues.apache.org/jira/browse/ARROW-16089 >
