Thanks Feng and sorry for the confusion everyone! Feng quoted my original email, but I'll repeat it here for clarity. Hopefully everyone will be able to reply to messages properly this time. :)
-- Hi all, I wanted to kick off a conversation about the future of Protobuf and Abseil/ABSL in Debian/Ubuntu. Laszlo is the Debian maintainer for the Protobuf package, and has also expressed interest in packaging Abseil. Debian currently distributes a binary shared library for protobuf (libprotobuf.so). Debian policy requires that the SONAME is bumped every time there is an ABI-incompatible change to the library: https://www.debian.org/doc/debian-policy/#shared-libraries Since protobuf does not guarantee any kind of binary compatibility, we have traditionally updated our SONAME for every minor release of protobuf. We are currently at major version 15: https://github.com/google/protobuf/blob/0ba8eea655a5e40d19ab95c773192b5d908c5a61/src/Makefile.am#L186 This has worked reasonably well so far. But we want to start depending on Abseil (https://abseil.io) which does not do releases at all (so doesn't even have a major version number to track). How would this work for Debian? Abseil's philosophy is that users will build from source. But Debian is inherently a binary distribution. We have to figure out how to reconcile these two philosophies. I don't know the answer, but I'm going to throw out a few straw man ideas as a starting point for discussion. 1. Distribute libprotobuf and libabsl as static libraries only. Debian policy says that libraries can be distributed in static form only if that is what is intended by upstream: https://www.debian.org/doc/debian-policy/#static-libraries Since both libprotobuf and libabsl are disavowing any binary compatibility guarantees, that implies that they should not be distributed as .so's. Any Debian packages or user programs that want to use protobuf have to link it statically. The tricky part is that libprotobuf.a and libabsl.a would need to match each other exactly. They would need to be compiled from exactly the same ABSL headers. This implies that libprotobuf-dev would need to pin itself to one specific libabsl-dev version -- the one it was compiled against. Other libraries that depend on ABSL (like TensorFlow) would also need to be pinned to exactly the same libabsl-dev version, otherwise it would be impossible to install them all at the same time. So realistically all packages that depend on ABSL would need to be updated at the same time. 2. Distribute libprotobuf and libabsl as shared libraries. This is against the philosophy of ABSL. But if Debian updated its ABSL infrequently enough, it could just treat each ABSL update as a separate major version. Sort of like we do with protobuf now. The tricky part is if two different libraries pass ABSL types across library boundaries. For example, imagine that a user program was developing against libabsl.so.2 but also using libprotobuf.so.15 that was linked against libabsl.so.1. If a user got an absl::string_view from protobuf, this could be unsafe for the user to pass directly to ABSL functions. The representation of absl::string_view could have changed between the two versions of libabsl. The all-static option (1) seems to solve this problem by pinning all of the ABSL dependencies so that they all rev together. But this solution could become unwieldy if too many packages depend on ABSL. Also any library that exposes ABSL types in its API would have to be part of this lock-step ABSL upgrade. 3. Remove protobuf from Debian completely, ask users to build from source. Since the philosophy of ABSL is that users will always build from source, we could ask that users do this if they want to develop against protobuf. Debian packages that depend on protobuf could build protobuf (and ABSL) and statically link against it to build runnable binaries. It would not be possible to have library packages in Debian that depend on protobuf. Thoughts? Josh On Wed, Dec 20, 2017 at 10:58 AM, Feng Xiao <xiaof...@google.com> wrote: > +abseil...@googlegroups.com, +proto...@gogolegroups.com > > CCed public google groups so László can reply to the group. BCCed internal > mailing lists. > > On Tue, Dec 19, 2017 at 8:24 PM, Josh Haberman <haber...@google.com> > wrote: > >> Hi all, >> >> I wanted to kick off a conversation about the future of Protobuf and >> Abseil/ABSL in Debian/Ubuntu. Laszlo is the Debian maintainer for the >> Protobuf package, and has also expressed interest in packaging Abseil. >> >> Debian currently distributes a binary shared library for protobuf >> (libprotobuf.so). Debian policy requires that the SONAME is bumped every >> time there is an ABI-incompatible change to the library: >> >> https://www.debian.org/doc/debian-policy/#shared-libraries >> >> Since protobuf does not guarantee any kind of binary compatibility, we >> have traditionally updated our SONAME for every minor release of protobuf. >> We are currently at major version 15: >> >> https://github.com/google/protobuf/blob/0ba8eea655a5e40d19ab >> 95c773192b5d908c5a61/src/Makefile.am#L186 >> >> This has worked reasonably well so far. But we want to start depending on >> Abseil (https://abseil.io) which does not do releases at all (so doesn't >> even have a major version number to track). How would this work for Debian? >> >> Abseil's philosophy is that users will build from source. But Debian is >> inherently a binary distribution. We have to figure out how to reconcile >> these two philosophies. I don't know the answer, but I'm going to throw out >> a few straw man ideas as a starting point for discussion. >> >> 1. Distribute libprotobuf and libabsl as static libraries only. >> >> Debian policy says that libraries can be distributed in static form only >> if that is what is intended by upstream: >> >> https://www.debian.org/doc/debian-policy/#static-libraries >> >> Since both libprotobuf and libabsl are disavowing any binary >> compatibility guarantees, that implies that they should not be distributed >> as .so's. Any Debian packages or user programs that want to use protobuf >> have to link it statically. >> >> The tricky part is that libprotobuf.a and libabsl.a would need to match >> each other exactly. They would need to be compiled from exactly the same >> ABSL headers. This implies that libprotobuf-dev would need to pin itself to >> one specific libabsl-dev version -- the one it was compiled against. Other >> libraries that depend on ABSL (like TensorFlow) would also need to be >> pinned to exactly the same libabsl-dev version, otherwise it would be >> impossible to install them all at the same time. So realistically all >> packages that depend on ABSL would need to be updated at the same time. >> >> 2. Distribute libprotobuf and libabsl as shared libraries. >> >> This is against the philosophy of ABSL. But if Debian updated its ABSL >> infrequently enough, it could just treat each ABSL update as a separate >> major version. Sort of like we do with protobuf now. >> >> The tricky part is if two different libraries pass ABSL types across >> library boundaries. For example, imagine that a user program was developing >> against libabsl.so.2 but also using libprotobuf.so.15 that was linked >> against libabsl.so.1. If a user got an absl::string_view from protobuf, >> this could be unsafe for the user to pass directly to ABSL functions. The >> representation of absl::string_view could have changed between the two >> versions of libabsl. >> >> The all-static option (1) seems to solve this problem by pinning all of >> the ABSL dependencies so that they all rev together. But this solution >> could become unwieldy if too many packages depend on ABSL. Also any library >> that exposes ABSL types in its API would have to be part of this lock-step >> ABSL upgrade. >> >> 3. Remove protobuf from Debian completely, ask users to build from source. >> >> Since the philosophy of ABSL is that users will always build from source, >> we could ask that users do this if they want to develop against protobuf. >> Debian packages that depend on protobuf could build protobuf (and ABSL) and >> statically link against it to build runnable binaries. It would not be >> possible to have library packages in Debian that depend on protobuf. >> >> Thoughts? >> Josh >> > > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at https://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
