Re: Bug#1035946: RFS: justbuild/1.1.0-1 [ITP] -- Justbuild generic build system
Hi Wookey On Tue, May 16, 2023 at 8:19 PM Wookey wrote: > I used 0~0-1 to start with. 0~ is a quite a good way of > versioning things like this that don't have versions. (that 0~ lets > you switch neatly to real versions in the future should they > appear). Adding git hashes mostly makes for unreadable versions and > doesn't add much IMHO, but we can do that if it's important. Yes, the git hash clutters up the version, but at least you can easily identify which exact commit is packaged. The date alone might not be sufficient, in particular with rebasing. > Debian generally tries to pick a version and make depending packages > work with it, rather than try to suport multiple versions unless it > really is necessary. I do not have a good feel for what the best > approach here is, and would greatly appreciate input from someone more > familiar with the codebase on what the best approach in debian might be. I see. I just wonder how useful such a package is. Many packages might have a hard time using it without significant upstream changes. Just as an example, even gRPC (another Google project itself) uses a 2 year old version, which is incompatible with what Fedora packaged last year, which is already incompatible with current master. It's kind of a mess... > I will put my unfinished project on salsa, file an ITP, and find my > notes, then mail you and we can see if we can sort this with a > reasonable level of effort. Sure, I would be very happy to help. Oliver
Re: Bug#1035946: RFS: justbuild/1.1.0-1 [ITP] -- Justbuild generic build system
On 2023-05-16 14:46 +0200, Oliver Reiche wrote: >I'm basically aware of the build dependencies >policy and all of my binary and header-only dependencies are satisfied >from packages. However, my package additionally depends on 11 proto files >(i.e., architecture-independent interface of data encoding [1]) from >google-apis [2] and bazel-remote-apis [3] as a pure build dependency. ... >3. Taking the longer road and package google-apis and bazel-remote-apis >first. This is the right way to fix this. I noticed in 2021 whilst doing tensorflow packaging that quite a few packages were using parts of these but nearly everyone had just embedded some files. We do have parts of the api packaged (e.g ruby-googleapis-common-prootos-types, and there are also ITPs for python-google-api-core and ruby-google-apis-discovery-v1 from jan 2023) but not the whole thing for all the languages. So I started on fixing it properly, and have a build that works for C, C++, java, python3 and ruby, but some language bindings did not build, and clearly I stalled part-way through the process of fixing those. I'm not sure which bindings we actually care about and which we can leave for now. I think I started an email about this somewhere, but am failing to find it right now, so I can't remember exactly where I got to. >However, that raises a few more questions: > a. google-apis is not versioned/tagged upstream. What version would I >use? I've seen that Fedora uses the version string >"0-1.git". I used 0~0-1 to start with. 0~ is a quite a good way of versioning things like this that don't have versions. (that 0~ lets you switch neatly to real versions in the future should they appear). Adding git hashes mostly makes for unreadable versions and doesn't add much IMHO, but we can do that if it's important. > b. Where should proto files be installed to? I know that libprotobuf-dev >puts it in /usr/include, but /usr/share could be also viable. What is the >recommended location? Good question. The right answer may depend on the language. > c. As the file structure of google-apis changes rather frequently, >should there be any prefix, so multiple versions could be installed in >parallel? Debian generally tries to pick a version and make depending packages work with it, rather than try to suport multiple versions unless it really is necessary. I do not have a good feel for what the best approach here is, and would greatly appreciate input from someone more familiar with the codebase on what the best approach in debian might be. >Could you please comment on which option you would suggest to take, and >also briefly address the potential follow-up questions? I suggest we compare notes on this in a specific ITP bug for google-apis. If you have a bit of time to put into this it would be great to actully (finally) get this fixed for the general case. I can provide debian packaging and build expertise to complement your knowledge of google-apis. (and then maybe we can give bazel-remote-apis a very similar treatment). I will put my unfinished project on salsa, file an ITP, and find my notes, then mail you and we can see if we can sort this with a reasonable level of effort. Wookey -- Principal hats: Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Bug#1035946: RFS: justbuild/1.1.0-1 [ITP] -- Justbuild generic build system
Hi Paul, On Tue, May 16, 2023 at 5:07 AM Paul Wise wrote: > Generally all build dependencies should be packaged separately instead. > > https://wiki.debian.org/EmbeddedCopies > > Many thanks for that hint. I'm basically aware of the build dependencies policy and all of my binary and header-only dependencies are satisfied from packages. However, my package additionally depends on 11 proto files (i.e., architecture-independent interface of data encoding [1]) from google-apis [2] and bazel-remote-apis [3] as a pure build dependency. In the past, it seems that there was made an exception for packages that depend on these proto files: - buildstream and bazel-bootstrap (both only build-dep) - bazel-bootstrap-source (seems temporary/hackish though) - golang-github-gogo-googleapis-dev and golang-github-grpc-ecosystem-grpc-gateway-dev (both even shipping these files in their deb package) Interestingly, none of these packages is mentioned in the list of source packages that embed code from other projects [4]. For that reason, I was initially hoping that embedding these files would be fine for my package as well, albeit as a tarball. Currently, I see 3 options: 1. Keep the tarballs in debian/third_party, which would be the cleanest solution for our bootstrap, _but_ according to your last message this is probably not a viable option for Debian. 2. If the problem is only the tarballs, I could unpack them and embed only the exact 11 proto files that we need and explicitly mention them in the copyrights file of course. 3. Taking the longer road and package google-apis and bazel-remote-apis first. However, that raises a few more questions: a. google-apis is not versioned/tagged upstream. What version would I use? I've seen that Fedora uses the version string "0-1.git". b. Where should proto files be installed to? I know that libprotobuf-dev puts it in /usr/include, but /usr/share could be also viable. What is the recommended location? c. As the file structure of google-apis changes rather frequently, should there be any prefix, so multiple versions could be installed in parallel? Could you please comment on which option you would suggest to take, and also briefly address the potential follow-up questions? Many thanks in advance, I really appreciate your help! Kind regards, Oliver [1] https://protobuf.dev/ [2] https://github.com/googleapis/googleapis [3] https://github.com/bazelbuild/remote-apis [4] https://salsa.debian.org/security-tracker-team/security-tracker/raw/master/data/embedded-code-copies
