Re: [PATCH 4/4] debian: add generic rule file
On Wed, Jul 17, 2019 at 04:16:39PM +0200, Enrico Weigelt, metux IT consult wrote: > > > In practice, that's not going to be a problem for most distributions. > > The traditional way Debian-derived systems have done builds is > > completely outside of git. So there will be a linux_5.2.orig.tar.gz > > and a linux_5.2-1.debian.tar.xz. dpkg_source -x will first unpackage > > the orig.tar.gz, and then the debian.tar.xz, and if the second > > overwrites the first, it's no big deal. > > ACK. IIRC they already filter out debian/ directories when generating > upstream tarballs - other upstreams already provide their debian/ > stuff, too. Well, no, actually they don't. That's because as much as possible they want the upstream tarball to be bit-for-bit identical to the one published on the official upstream distribution site. That allows them to include the detached PGP signature from the upstream maintainer, if one is provided. If there are files in the upstream debian/ directory that they don't need, they can delete in the distro's debian/rules file. Ideally, so we shouldn't include files in the Linux kernel's debian/ directory willy-nilly. But the debian/rules file will *always* be present, and so it will be overwritten by the _.debian.tar.xz file, and so it's no big deal. - Ted
Re: [PATCH 4/4] debian: add generic rule file
On 16.07.19 14:34, Theodore Y. Ts'o wrote: Hi, > In practice, that's not going to be a problem for most distributions. > The traditional way Debian-derived systems have done builds is > completely outside of git. So there will be a linux_5.2.orig.tar.gz > and a linux_5.2-1.debian.tar.xz. dpkg_source -x will first unpackage > the orig.tar.gz, and then the debian.tar.xz, and if the second > overwrites the first, it's no big deal. ACK. IIRC they already filter out debian/ directories when generating upstream tarballs - other upstreams already provide their debian/ stuff, too. > First, once it's checked in, I expect changes to the default > debian/rules file will be relatively rare. ACK. I currently don't see much reasons for future changes. If anybody sees something missing, just let me know, and I'll take it up for the next review round. --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: [PATCH 4/4] debian: add generic rule file
On 15.07.19 21:12, Theodore Y. Ts'o wrote: Hi, > It's possible I'm not remembering some of the feedback, but the only> thing I > recall was the comment I made that I'd really like this use> case:> > make O=/build/linux-build bindeb-pkg ah, I yet wanted to test that - thx for reminding me. > to not break. And as far as I can tell from the proposed patch series> (I haven't had a chance to experimentally verify it yet), I don't> think it should break anything --- I'm assuming that we will still> have a way of creating the debian/rules file in> /build/linux-build/debian/rules when doing a O= build, and that the> intdeb-pkg rule remains the same. At least, it appears to be the case> from my doing a quick look at the patches. Yes (unless i've missed something), everything should remain as it is. One thing that could happen (not checked yet) is that running good old 'make bindeb-pkg' without O=... could overwrite the now already existing debian/rules file. If that's really a problem, we could tweak the machinery to use a different name for the rule file (for now, one the preceeding patch just allows giving a different name for just *generating* the rules file). Another idea could be rewriting the whole process so that no rules file needs to be generated at all. > Yeah, the official Debian debian/rules is optimized for doing a > distribution release, and in addition to the issues Enrico has raised, > last time I tried it, it was S-L-O-W since it was building a fully > generic kernel. It's not at all useable for general developer use. I'm a bit reluctant calling this 'optimized' :p The strangest aspect (IMHO) is they're building several different trees (w/ different huge patch queues) from only one source package. Instead I'd rather: * try to get as much as possible in one tree * have separate source packages if there really need to be separate patche queues (IMHO, these things, like RT stuff, just need proper Kconfig's) * do all the patching in git and skip the text-based patches entirely Haven't found out, why they're actually doing it that complicated way (didn't get any useful answers from debian kernel folks) > It sounds like what Enrico is trying to do is to enable running > "dpkg-buildpackage -us -uc -b" from the the top-level kernel package > as being easier than running "make bindeb-pkg". I suspect this might > be because his goal is to integrate individual kernel builds from > using Debian's hermetic build / chroot systems (e.g., sbuild, pbuilder)? Yes, I'm building all deb's by the same process / infrastructure. In my case it's dck-buildpackage (*1) which runs the build in a docker container (kinda pbuilder w/ docker). It always starts with a fresh (minimal) base image, calls debian/rules to create debian/control (if necessary) deploys the dependencies found in the control file and finally fire's up dpkg-buildpackage - the output is collected in an ready-to-use apt repo. The goal of this is having a canonical build process for all deb packages, not having to care of any special cases anymore. I also have another tool ontop of that, which runs the whole show for dozens of packages and targets (*2). My first approach was trying to use Debian source packages with new kernel trees, but had to give up after a few days. Then I've found out that the kernel already has *almost* what I needed. The difference between almost and fine is this patch queue (minus local .config files) --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: [PATCH 4/4] debian: add generic rule file
On Tue, Jul 16, 2019 at 05:58:49PM +0900, Masahiro Yamada wrote: > I want debian/ to be kept as a drop-in directory > for packagers, without replacing the upstream debian/rules. > > If a check-in source file is modified in anyway, > scripts/setlocalversion would set -dirty flag, > which I want to avoid. In practice, that's not going to be a problem for most distributions. The traditional way Debian-derived systems have done builds is completely outside of git. So there will be a linux_5.2.orig.tar.gz and a linux_5.2-1.debian.tar.xz. dpkg_source -x will first unpackage the orig.tar.gz, and then the debian.tar.xz, and if the second overwrites the first, it's no big deal. More modern Debian package maintainer workflows may be using git, but in that case, all of the "Debianizations" are reflected in a separate branch. So it's not going to set the -dirty flag. There will be potential merge conflicts between Enrico's proposed "upstream default debian/rules" file and the Debian/Ubuntu debian/rules file on their distro branch. However, I don't think that's a big issue, for two reasons. First, once it's checked in, I expect changes to the default debian/rules file will be relatively rare. Secondly, it's easy enough to use gitattributes and defining a custom merge driver so that a distribution can configure things so that they always use the version of debian/rules from their branch, so the merge conflict resolution can be set up to always do the right thing. There are certainly other upstreams which ship their own debian/ directories. E2fsprogs is one such example, but in that case I'm cheating because I'm both the Debian package maintainer as well as the upstream maintainer. :-) However, it's never been an issue for Ubuntu when they choose to ship their own customized debian/rules file. > debian/rules is a hook for packagers to do their jobs in downstream. > "We kindly committed a generic one for you" sounds weird to me. It is weird, and it's not common for upstream packages (which are not native Debian packages) to ship their own debian directory. But it certainly does happen, and it won't cause any problems in actual practice. Regards, - Ted
Re: [PATCH 4/4] debian: add generic rule file
On Tue, Jul 16, 2019 at 4:13 AM Theodore Y. Ts'o wrote: > > On Mon, Jul 15, 2019 at 08:56:25PM +0200, Enrico Weigelt, metux IT consult > wrote: > > On 15.07.19 14:28, Masahiro Yamada wrote: > > > > >> The rule file contains a rule for creating debian/control and > > >> other metadata - this is done similar to the 'deb-pkg' make rule, > > >> scripts/packaging/mkdebian. > > > > > > I saw a similar patch submission before, and negative feedback about it. > > > > Do you recall what negative feedback exactly ? Sorry, my memory was broken. I did not like this patch set from the beginning, but missed to express my opinion strongly. I want debian/ to be kept as a drop-in directory for packagers, without replacing the upstream debian/rules. If a check-in source file is modified in anyway, scripts/setlocalversion would set -dirty flag, which I want to avoid. > It's possible I'm not remembering some of the feedback, but the only > thing I recall was the comment I made that I'd really like this use > case: > > make O=/build/linux-build bindeb-pkg > > to not break. And as far as I can tell from the proposed patch series > (I haven't had a chance to experimentally verify it yet), I don't > think it should break anything --- I'm assuming that we will still > have a way of creating the debian/rules file in > /build/linux-build/debian/rules when doing a O= build, and that the > intdeb-pkg rule remains the same. At least, it appears to be the case > from my doing a quick look at the patches. > > > > Debian maintains its own debian/rules, and it is fine. > > > > Not for me, I don't use it - given up trying to make anything useful > > out of it. It's extremly complex, practically undebuggable and doesn't > > even work w/o lots of external preparations. > > Yeah, the official Debian debian/rules is optimized for doing a > distribution release, and in addition to the issues Enrico has raised, > last time I tried it, it was S-L-O-W since it was building a fully > generic kernel. It's not at all useable for general developer use. It is OK if the package is targeting normal users instead of kernel developers. > It sounds like what Enrico is trying to do is to enable running > "dpkg-buildpackage -us -uc -b" from the the top-level kernel package > as being easier than running "make bindeb-pkg". I suspect this might > be because his goal is to integrate individual kernel builds from > using Debian's hermetic build / chroot systems (e.g., sbuild, pbuilder)? I am OK with generating debian/rules with 'make bindeb-pkg', a shell scripts or whatever, but I dislike to commit it in upstream git tree. debian/rules is a hook for packagers to do their jobs in downstream. "We kindly committed a generic one for you" sounds weird to me. -- Best Regards Masahiro Yamada
Re: [PATCH 4/4] debian: add generic rule file
On Mon, Jul 15, 2019 at 08:56:25PM +0200, Enrico Weigelt, metux IT consult wrote: > On 15.07.19 14:28, Masahiro Yamada wrote: > > >> The rule file contains a rule for creating debian/control and > >> other metadata - this is done similar to the 'deb-pkg' make rule, > >> scripts/packaging/mkdebian. > > > > I saw a similar patch submission before, and negative feedback about it. > > Do you recall what negative feedback exactly ? It's possible I'm not remembering some of the feedback, but the only thing I recall was the comment I made that I'd really like this use case: make O=/build/linux-build bindeb-pkg to not break. And as far as I can tell from the proposed patch series (I haven't had a chance to experimentally verify it yet), I don't think it should break anything --- I'm assuming that we will still have a way of creating the debian/rules file in /build/linux-build/debian/rules when doing a O= build, and that the intdeb-pkg rule remains the same. At least, it appears to be the case from my doing a quick look at the patches. > > Debian maintains its own debian/rules, and it is fine. > > Not for me, I don't use it - given up trying to make anything useful > out of it. It's extremly complex, practically undebuggable and doesn't > even work w/o lots of external preparations. Yeah, the official Debian debian/rules is optimized for doing a distribution release, and in addition to the issues Enrico has raised, last time I tried it, it was S-L-O-W since it was building a fully generic kernel. It's not at all useable for general developer use. It sounds like what Enrico is trying to do is to enable running "dpkg-buildpackage -us -uc -b" from the the top-level kernel package as being easier than running "make bindeb-pkg". I suspect this might be because his goal is to integrate individual kernel builds from using Debian's hermetic build / chroot systems (e.g., sbuild, pbuilder)? - Ted
Re: [PATCH 4/4] debian: add generic rule file
On 15.07.19 14:28, Masahiro Yamada wrote: >> The rule file contains a rule for creating debian/control and >> other metadata - this is done similar to the 'deb-pkg' make rule, >> scripts/packaging/mkdebian. > > I saw a similar patch submission before, and negative feedback about it. Do you recall what negative feedback exactly ? > Debian maintains its own debian/rules, and it is fine. Not for me, I don't use it - given up trying to make anything useful out of it. It's extremly complex, practically undebuggable and doesn't even work w/o lots of external preparations. --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: [PATCH 4/4] debian: add generic rule file
On Tue, Jul 9, 2019 at 6:33 PM Enrico Weigelt, metux IT consult wrote: > > Adding a generic debian rule file, so we can build the directly > via usual Debian package build tools (eg. git-buildpackage, > dck-buildpackage, etc). It expects the .config file already > placed in the source tree. > > The rule file contains a rule for creating debian/control and > other metadata - this is done similar to the 'deb-pkg' make rule, > scripts/packaging/mkdebian. I saw a similar patch submission before, and negative feedback about it. Debian maintains its own debian/rules, and it is fine. I do not like to check-in the one in the kernel tree. > Signed-off-by: Enrico Weigelt, metux IT consult > --- > .gitignore | 1 + > MAINTAINERS | 6 ++ > debian/rules | 27 +++ > 3 files changed, 34 insertions(+) > create mode 100755 debian/rules > > diff --git a/.gitignore b/.gitignore > index 7587ef56..01d742c 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -69,6 +69,7 @@ modules.builtin > # Debian directory (make deb-pkg) > # > /debian/ > +!/debian/rules > > # > # Snap directory (make snap-pkg) > diff --git a/MAINTAINERS b/MAINTAINERS > index 558acf2..56e034c 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -4516,6 +4516,12 @@ F: include/uapi/linux/dccp.h > F: include/linux/tfrc.h > F: net/dccp/ > > +DEBIAN PACKAGING FILES > +M: Enrico Weigelt > +L: linux-kbu...@vger.kernel.org > +S: Maintained > +F: debian/ > + > DECnet NETWORK LAYER > W: http://linux-decnet.sourceforge.net > L: linux-decnet-u...@lists.sourceforge.net > diff --git a/debian/rules b/debian/rules > new file mode 100755 > index 000..c2f0319 > --- /dev/null > +++ b/debian/rules > @@ -0,0 +1,27 @@ > +#!/usr/bin/make -f > +# SPDX-License-Identifier: GPL-2.0 > + > +export MAKE > +export KERNELARCH = $(shell $(MAKE) kernelarch) > +export KERNELRELEASE= $(shell $(MAKE) kernelrelease) > +export KBUILD_DEBARCH = $(shell dpkg-architecture -qDEB_HOST_ARCH) > +export KBUILD_BUILD_VERSION = $(shell $(MAKE) kernellocalversion) > +export KDEB_RULES = debian/rules.auto > +export KDEB_SOURCENAME = linux-source > +export ARCH = $(KERNELARCH) > + > +debian/control debian/changelong debian/arch debian/copyright: > +debian/control: > + ./scripts/package/mkdebian > + > +build: debian/control > + $(MAKE) KERNELRELEASE=$(KERNELRELEASE) ARCH=$(KERNELARCH) > KBUILD_BUILD_VERSION=$(KBUILD_BUILD_VERSION) KBUILD_SRC= > + > +binary-arch: debian/control > + $(MAKE) KERNELRELEASE=$(KERNELRELEASE) ARCH=$(KERNELARCH) > KBUILD_BUILD_VERSION=$(KBUILD_BUILD_VERSION) KBUILD_SRC= intdeb-pkg > + > +clean: > + rm -rf debian/*tmp debian/files debian/changelog debian/control > debian/copyright debian/rules.auto debian/arch > + $(MAKE) clean > + > +binary: binary-arch > -- > 1.9.1 > -- Best Regards Masahiro Yamada
