Re: Bits from /me: A humble draft policy on "deep learning v.s. freedom"
Hi, On 2019-05-21 23:52, Paul Wise wrote: > Has anyone repeated the training of Mozilla DeepSpeech for example? By chance I found a paper from a pile of papers (that attacks AI models) that Berkeley researchers have successfully attacked DeepSpeech: https://arxiv.org/pdf/1801.01944.pdf IHMO Try not to ask AI to deal with any critical task unless one understands the security risk. Maybe attacking AI models will be what future hackers do? ```quote from https://arxiv.org/abs/1801.01944 Abstract We construct targeted audio adversarial examples on automatic speech recognition. Given any audio waveform, we can produce another that is over 99.9% similar, but transcribes as any phrase we choose (recognizing up to 50 characters per second of audio). We apply our white-box iterative optimization-based attack to Mozilla’s implementation DeepSpeech end-to-end, and show it has a 100% success rate. The feasibility of this attack introduce a new domain to study adversarial examples. ```quote
Re: Exclude some architectures from an architecture-independent package ?
On Wed, May 29, 2019 at 8:46 PM Raphaël Halimi wrote: > What would be the "cleanest" solution according to you ? The cleanest solution would be to get this code into Linux mainline. Some discussion of workarounds: dak needs a way to restrict the availability of arch:all packages to certain architecture's Packages files. This would also be useful for interpreted code that only has support for certain kernel interfaces that aren't available on non-Linux kernels. For example for iotop I went with an arch:linux-any package instead of arch:all, the package is fairly small so the duplication isn't a big issue. Personally I think you should hardcode the ACPI architectures and accept the minor duplication. -- bye, pabs https://wiki.debian.org/PaulWise
Re: Why do we take so long to realise good ideas
On Wed, May 29, 2019 at 5:36 PM Mo Zhou wrote: > For example, many years ago I proposed that we could hire some > web developers to rewrite our homepage, to make it more good-looking > (Generally I don't care about superficial stuff but our homepage > is really old enough. Look at Gentoo's homepage and compare it with > the ancient version.) The web team has been working on this at their latest sprint. -- bye, pabs https://wiki.debian.org/PaulWise
Re: Survey: git packaging practices / repository format
Ian Jackson writes: > Can you please look through the table below and see if I have covered > everything you do ? You have covered the workflows I set up where I have the choice. Thanks. -- \ “I am amazed, O Wall, that you have not collapsed and fallen, | `\since you must bear the tedious stupidities of so many | _o__) scrawlers.” —anonymous graffiti, Pompeii, 79 CE | Ben Finney
Re: Exclude some architectures from an architecture-independent package ?
Le 29/05/2019 à 15:41, Ondřej Surý a écrit : > can’t you just “skip” building the module in DKMS when on unsupported > architecture? > > Install the package on that system would be noop then. Well, I was so focused on trying to make the package unavailable on non-ACPI architectures that I didn't think to explore this avenue. It's a good idea, and actually quite easy to implement, since I found after some research that DKMS supports a "BUILD_EXCLUSIVE_ARCH" option that can accept a regexp to match architectures to build for. I'll go this way, thanks a lot for your help. Regards, -- Raphaël Halimi signature.asc Description: OpenPGP digital signature
Bug#929741: ITP: wham -- Wisconsin's High-Throughput Alignment Method
Package: wnpp Severity: wishlist Owner: Steffen Moeller * Package name: wham * URL : http://www.cs.wisc.edu/wham/ * License : GPL Programming Lang: C Description : Wisconsin's High-Throughput Alignment Method To be team-maintained on salsa.debian.org/med-team/wham or wham-align.
Re: Survey: git packaging practices / repository format
Ian Jackson writes:
> Ian Jackson writes ("Re: Survey: git packaging practices / repository
> format"):
>> David Bremner writes ("Re: Survey: git packaging practices / repository
>> format"):
> ...
>> > With unmodified upstream files in the main branch, plus debian/*, but
>> > usually no d/patches, I use git-debcherry to generate a quilt series at
>> > dsc build time.
>>
>> I think I understand this one a bit better than the one above.[1]
>> What constraints are there on the main branch, for this to work ?
>
> Also, how do you move to a new upstream version ?
use git merge, typically from an upstream tag, or from a debian specific
upstream branch with tarballs imported on top of upstream history.
Re: Survey: git packaging practices / repository format
Ian Jackson writes:
> Hi. Thanks for your contributions which I am trying to capture, but I
> don't think I fully understand them.
>
> David Bremner writes ("Re: Survey: git packaging practices / repository
> format"):
>> With modified upstream files in the main branch, plus debian/*, but
>> usually no d/patches I use a seperate (manually
>> rebased) branch for patches, and export those at dsc creation time using
>> a gitpkg hook
>
> Is this the same setup as described by Simon McVittie for xorg
> packages (eg, mesa) ?
I don't think so. My own usage is "purer" and doesn't involve quilt;
the mention of d/patches is probably a red herring here; I only
mentioned because I remember that some users of git-debcherry like(d) to
commit exported patches to be compatible(ish) with gbp.
> If not I don't understand, because you say both that the upstream
> files are modified in your main branch, and that there are patches in
> d/patches but that is in a separate branch.
> Are the same changes
> represented twice, then, on two git branches ?
The patch branch (which is just a regular git branch), is just a linear
sequence of commits on upstream.
> You say "a gitpkg hook". Is the hook script in Debian or is it ad
> hoc ? My table would perhaps want to name it.
yes, it is /usr/share/gitpkg/hooks/quilt-patches-deb-export-hook (in
the package gitpkg).
>
>> With unmodified upstream files in the main branch, plus debian/*, but
>> usually no d/patches, I use git-debcherry to generate a quilt series at
>> dsc build time.
>
> I think I understand this one a bit better than the one above.[1]
> What constraints are there on the main branch, for this to work ?
>
There are no (known) constraints on the main branch, but the degree to
which it "works" varies. It guarantees the same working tree as you
started with, but not a one-to-one mapping between git commits and quilt
patches. In particular there can be a "debcherry-fixup-patch" containing
some changes that could not be nicely linearized into patches.
> [1] git-debcherry solves a very similar problem to dgit's quilt
> linearisation, which is used for example by dgit to construct `3.0
> (quilt)' patches out of the commits made by an NMUer.
Yes, that's why I pointed git-debcherry out to you when you started
designing dgit :P
> And I think git-debrebase branches are always suitable for use with
> git-debcherry, but git-debrebase knows how to make patches itself so
> you don't need git-debcherry then.)
Sure, except if you have a project already using git-debcherry, where I
guess git-debrebase might not work.
git-debcherry itself does not modify history, only generates source
packages. There is a companion script 'git-refresh' that I think
was never packaged (attached for reference). The idea is to bring
patches to the tip of history again, which I guess is a simplified
version of what git-debrebase does.
git-refresh
Description: Binary data
Re: Survey: git packaging practices / repository format
Ian Jackson writes ("Re: Survey: git packaging practices / repository format"):
> David Bremner writes ("Re: Survey: git packaging practices / repository
> format"):
...
> > With unmodified upstream files in the main branch, plus debian/*, but
> > usually no d/patches, I use git-debcherry to generate a quilt series at
> > dsc build time.
>
> I think I understand this one a bit better than the one above.[1]
> What constraints are there on the main branch, for this to work ?
Also, how do you move to a new upstream version ?
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Survey: git packaging practices / repository format
Hi. Thanks for your contributions which I am trying to capture, but I
don't think I fully understand them.
David Bremner writes ("Re: Survey: git packaging practices / repository
format"):
> With modified upstream files in the main branch, plus debian/*, but
> usually no d/patches I use a seperate (manually
> rebased) branch for patches, and export those at dsc creation time using
> a gitpkg hook
Is this the same setup as described by Simon McVittie for xorg
packages (eg, mesa) ?
If not I don't understand, because you say both that the upstream
files are modified in your main branch, and that there are patches in
d/patches but that is in a separate branch. Are the same changes
represented twice, then, on two git branches ?
You say "a gitpkg hook". Is the hook script in Debian or is it ad
hoc ? My table would perhaps want to name it.
> With unmodified upstream files in the main branch, plus debian/*, but
> usually no d/patches, I use git-debcherry to generate a quilt series at
> dsc build time.
I think I understand this one a bit better than the one above.[1]
What constraints are there on the main branch, for this to work ?
Thanks,
Ian.
[1] git-debcherry solves a very similar problem to dgit's quilt
linearisation, which is used for example by dgit to construct `3.0
(quilt)' patches out of the commits made by an NMUer.
And I think git-debrebase branches are always suitable for use with
git-debcherry, but git-debrebase knows how to make patches itself so
you don't need git-debcherry then.)
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Survey: git packaging practices / repository format
On Wed, May 29, 2019 at 12:20:23PM -0400, Sam Hartman wrote: > >> Perhaps we should update policy to say that the .orig tarball may > >> (or even "should") be generated from an upstream release tag > >> where applicable. > Andrey> This conflicts with shipping tarball signatures. > > Sure does. > > I can see the argument for caring about that if you're dealing with an > upstream that does run make dist and publish official signed tarballs. > > There are a lot of upstreams though where the tarball is an afterthought > or entirely not present. > I hope we as a community can decide to go with the git rather than > pressuring such upstreams to care more about tarballs. Yup. I'm still not sure how useful and successful was the campaign for the signed orig tarballs. -- WBR, wRAR signature.asc Description: PGP signature
Re: Survey: git packaging practices / repository format
> "Andrey" == Andrey Rahmatullin writes: Andrey> On Wed, May 29, 2019 at 02:14:09PM +0100, Ben Hutchings wrote: >> [...] > My understanding is that this unusual difference between >> the .orig > tarball and what's in git is an attempt to "square >> the circle" between > two colliding design principles: "the .orig >> tarball should be upstream's > official binary artifact" (in this >> case Automake `make dist` output, > including generated files >> like Makefile.in but not non-critical source > files like >> .gitignore) and "what's in git should match upstream's git > >> repository" (including .gitignore but > not usually Makefile.in). >> [...] >> >> Perhaps we should update policy to say that the .orig tarball may >> (or even "should") be generated from an upstream release tag >> where applicable. Andrey> This conflicts with shipping tarball signatures. Sure does. I can see the argument for caring about that if you're dealing with an upstream that does run make dist and publish official signed tarballs. There are a lot of upstreams though where the tarball is an afterthought or entirely not present. I hope we as a community can decide to go with the git rather than pressuring such upstreams to care more about tarballs. --Sam
Re: Survey: git packaging practices / repository format
On Wed, May 29, 2019 at 02:14:09PM +0100, Ben Hutchings wrote: > [...] > > My understanding is that this unusual difference between the .orig > > tarball and what's in git is an attempt to "square the circle" between > > two colliding design principles: "the .orig tarball should be upstream's > > official binary artifact" (in this case Automake `make dist` output, > > including generated files like Makefile.in but not non-critical source > > files like .gitignore) and "what's in git should match upstream's git > > repository" (including .gitignore but > > not usually Makefile.in). > [...] > > Perhaps we should update policy to say that the .orig tarball may (or > even "should") be generated from an upstream release tag where > applicable. This conflicts with shipping tarball signatures. -- WBR, wRAR signature.asc Description: PGP signature
Re: Survey: git packaging practices / repository format
On Wed, 29 May 2019 at 14:14:09 +0100, Ben Hutchings wrote:
> On Wed, 2019-05-29 at 00:39 +0100, Simon McVittie wrote:
> [...]
> > My understanding is that this unusual difference between the .orig
> > tarball and what's in git is an attempt to "square the circle" between
> > two colliding design principles: "the .orig tarball should be upstream's
> > official binary artifact" (in this case Automake `make dist` output,
> > including generated files like Makefile.in but not non-critical source
> > files like .gitignore) and "what's in git should match upstream's git
> > repository" (including .gitignore but
> > not usually Makefile.in).
> [...]
>
> Perhaps we should update policy to say that the .orig tarball may (or
> even "should") be generated from an upstream release tag where
> applicable.
I couldn't immediately find anything in Policy that contradicts this.
devref §6.7.8 "Best practices for .orig.tar.{gz,bz2,xz} files" seems
to be the closest thing we have to policy on this? (That does currently
mandate use of upstream's official binary artifact unless there is a
very good reason not to, but perhaps priorities have shifted since that
section of devref was written and the reasons given there are no longer
as important as they used to be.)
I also tried dpkg-source(1), but that just describes the mechanics of the
formats, and not how they are meant to be used.
smcv
Re: ZFS in Buster
> "Sam" == Sam Hartman writes: ke the Software Freedom Conservancy (SFC)'s Sam> position Sam> https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/-UUU:**--F1 Sam> *unsent wide reply to Aron Xu* Bot L50 (Message SC:f MML Abbre Ah, that's an interesting artifact of how cut&paste works in my environment.:-( https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/ --Sam
Re: ZFS in Buster
Hi, With my Debian ZoL maintainer hat + FTP trainee hat on, I didn't wish to jump into this topic too early without a in-depth investigation... On 2019-05-29 14:14, Sam Hartman wrote: > And if you take the Software Freedom Conservancy (SFC)'s position > https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/-UUU:**--F1 But I got an HTTP404 there. > I'd rather spend time on other things unless this is something a > significant number of people in our community need. I maintain a bunch of packages for Debian. The only package about which I received many private prodding mails, is exactly ZFS...
Re: ZFS in Buster
I hope that we do not choose to open the zfs discussion at this time. If it does get opened, I think my approach would be to try and educate the community about the various different viewpoints, find text for GRs that would allow key stakeholders to believe their positions were respected and considered (and that we were setting global policy not trying to unreasonably micromanage ftpmaster), and hold a vote. I don't think we could come to a rough consensus on zfs as a project; we have a position that at least at the time was working for ftpmaster and the zfs maintainers. However please consider the following. I think the Software Freedom Law Center (SFLC's) blog post on zfs https://www.softwarefreedom.org/resources/2016/linux-kernel-cddl.html is the most pro-zfs legalish position that seems credible to me. I understand that blog post is not legal advice, but it's the kind of nuanced and complex reasoning you're going to get from a lawyer if you're working to find a legally defensible way to be permissive. That position basically depends on arguing that by their actions the kernel community has interpreted the boundary of derivative works somewhat different than the FSF typically does. I haven't read the blog post recently, but it basically argues that the kernel community could if they choose make the boundary even more clear. But a key take away is that they are arguing that it's the intent of the copyright holders that matters here. Yeah, that sucks for people who want clear answers because the intent of the kernel copyright holders is mixed. Except now we're seeing a different intent expressed. We're seeing the kernel developers choose to close off some interfaces. So, even under a very permissive (but IMHO still legally credible) interpretation, the kernel developers are moving *away* rather than *towards* zfs not being permitted to use the SIMD interfaces. I have really high confidence that even if you reopen the discussion, you're not going to convince the project of a legal position more permissive than that advanced by the SFLC. And if you take the Software Freedom Conservancy (SFC)'s position https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/-UUU:**--F1 *unsent wide reply to Aron Xu* Bot L50(Message SC:f MML Abbre which is a fairly typical position from someone who values a strong GPL over being permissive in what we allow users to do, well, the issue is fairly cut & dry. So if people really are uncomfortable with the position--if getting some sort of real closure would be worth a month or so of putting together educational material, rangling text for a GR, and having a vote, let's do that. I think it will be painful, but I do totally understand that issues like this can be painful if you feel a position was not given adequate consideration. But no matter what we do I suspect we'll find that getting a project level consensus to revert commits so zfs can use certain interfaces ends up being very unlikely. That's only my prediction. As DPL, I'll run the discussion fairly if that's what we need to do. I'd rather spend time on other things unless this is something a significant number of people in our community need. Meanwhile I'd like to thank the zfs maintainers, the former DPL, ftpmaster, and all the parties that contributed to the balance we have today. signature.asc Description: PGP signature
Re: ZFS in Buster
On Wed, 2019-05-29 at 13:43 +0200, Dan wrote: > Hi Jonathan, > > On Tue, May 28, 2019 at 8:50 PM Jonathan Carter wrote: > > On 2019/05/28 18:43, Dan wrote: > > > ZFS 0.8 has been released with lots of improvements, notably encryption. > > > > Yep, it's an exciting feature. > > > > > Sadly the Linux Kernel has introduced a commit in kernel 4.19 and 5.0 > > > that prevents ZFS from using SIMD. The result is that ZFS won't be > > > usable in Buster. See the following issue > > > https://github.com/zfsonlinux/zfs/issues/8793 > > > > Buster ships zfs-dkms version 0.7.12-2 though, which works just fine on > > buster. > > My message was not accurate. I think that the commit has been > introduced in in 4.19.38 (released 2019-05-02). I think that Debian > Buster uses 4.19.37 > > https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.19.38 > > The commit also affects ZFS 0.7 because SIMD is used for checksum operations. > > There might be a performance penalty in ZFS only if Debian Buster > upgrades to 4.19.38. Which we will, some time soon. Ben. -- Ben Hutchings Horngren's Observation: Among economists, the real world is often a special case. signature.asc Description: This is a digitally signed message part
Re: ZFS in Buster
On Wed, May 29, 2019 at 8:55 PM Ian Jackson
wrote:
>
> Ansgar writes ("Re: ZFS in Buster"):
> > Ian Jackson writes:
> > > I think this would be both unwise legally (without seeking additional
> > > legal advice) and rather rude to the kernel upstream whose code is
> > > then being reused without permission - indeed, contrary to their
> > > explicitly stated intent.
> >
> > At least one commercial distribution (Ubuntu) has been distributing ZFS
> > binary modules as part of their Linux packages for some years and didn't
> > have any problems.
>
> That doesn't prove anything other than that no-one felt it was
> politically or financially expedient to sue.
>
> AIUI Debian's position is still that summarised here:
> https://blog.halon.org.uk/2016/01/on-zfs-in-debian/
> (sorry about the pale grey on white disease; it works well in w3m)
>
> Are you trying to reopen that discussion ? If so then you should be
> CCing ftpmaster@ and leader@ probably...
>
(With my Debian ZoL hat on.)
I don't know whether this is correct time to discuss about Debian's
position but it was a compromise. Having Mehdi Dogguy (DPL of the
time) and representatives of Software Freedom Conservancy on the
table, we talked about this very topic at DebConf 16 and agreed to put
ZFS into contrib for the good of our users.
ZFS is free and open source software, perfectly complies with DFSG for
main archive inclusion on its own, and we had another copy of the code
in FreeBSD kernel which is main. While putting it into contrib is a
very expressive language telling the world that Debian and, more
importantly SFC, would like to see that the licensing issue could have
a better resolution at the root. And this is exactly the compromise
that made ZFS possible to go through NEW.
Regards,
Aron
Re: Exclude some architectures from an architecture-independent package ?
> On 29 May 2019, at 14:45, Raphaël Halimi wrote: > > Hi, > > I'm the maintainer of package acpi-call, which is a kernel module > allowing a user to call ACPI methods. Its build is handled by DKMS. > > The module used to build on all architectures, even if it was obviously > useless on those which don't support ACPI. > > Starting with Linux 5.2, what used to be a mere warning about > "acpi_root_dir" set to an undefined value, became an error, and prevents > the module to be built on architectures which don't support ACPI, > meaning every one except i386, amd64, ia64 and arm64. > > Since, AFAIK, the "Architecture" field in debian/control doesn't allow > tuning the "all" wildcard by excluding some architectures, I have > several (IMHO non-optimal) solutions to fix this: > > 1/ Restrict the architectures to i386, amd64, ia64, arm64 in > debian/control. This would fix the problem, but will make the mirrors > hold four copies of nearly identical binary packages, so wasting some > space (even if it's not much: the binary package holding the module > source code for DKMS is only 13 KB in size). > > 2/ Keep "arch:all" and set KBUILD_MODPOST_WARN in the module source > code, to allow build on unsupported architectures. > > 3/ Keep the status quo and close the bug [1] as wontfix (that's what the > reporter of [2], which is a duplicate of [1], did). > > [1] https://bugs.launchpad.net/ubuntu/+source/acpi-call/+bug/1830040 > [2] https://bugs.launchpad.net/ubuntu/+source/acpi-call/+bug/1830787 > > What would be the "cleanest" solution according to you ? > Hi, can’t you just “skip” building the module in DKMS when on unsupported architecture? Install the package on that system would be noop then. Cheers, Ondrej -- Ondřej Surý ond...@sury.org
Re: Survey: git packaging practices / repository format
On Wed, 2019-05-29 at 00:39 +0100, Simon McVittie wrote: [...] > My understanding is that this unusual difference between the .orig > tarball and what's in git is an attempt to "square the circle" between > two colliding design principles: "the .orig tarball should be upstream's > official binary artifact" (in this case Automake `make dist` output, > including generated files like Makefile.in but not non-critical source > files like .gitignore) and "what's in git should match upstream's git > repository" (including .gitignore but > not usually Makefile.in). [...] Perhaps we should update policy to say that the .orig tarball may (or even "should") be generated from an upstream release tag where applicable. Ben. -- Ben Hutchings Horngren's Observation: Among economists, the real world is often a special case. signature.asc Description: This is a digitally signed message part
Re: Survey: git packaging practices / repository format
On Tue, 2019-05-28 at 21:14 +0100, Ian Jackson wrote:
> Simon McVittie writes ("Re: Survey: git packaging practices / repository
> format"):
[...]
> > Debian Linux kernel
> > ===
> >
> > Tree contains: an incomplete debian/ directory, notably without d/control,
> > and no upstream source
> > Changes to upstream source are: d/patches only
> > Baseline upstream: changelog version => .orig tarball
> > Patches managed by: ???
> > Special build tool: there is a pre-build step to generate d/control
>
> Thanks, I will add this to my survey. I assume "patches are managed
> by" is the same as any other only-debian/* tree. The wrinkle is the
> need for a special build tool.
[...]
The build tool is part of the source package. It generates a lot of
other files in the source package beside debian/control.
I have an unmerged branch that changes various things to be compatible
with dgit. It adds debian/control and debian/tests/control to git and
defers generation of other things to build time.
Ben.
--
Ben Hutchings
Horngren's Observation:
Among economists, the real world is often a special case.
signature.asc
Description: This is a digitally signed message part
Bug#929708: ITP: python-geneimpacts -- wraps tools to assess variants in gene sequences
Package: wnpp Severity: wishlist Owner: Steffen Moeller * Package name: python-geneimpacts * URL : https://github.com/brentp/geneimpacts * License : MIT/X Programming Lang: Python Description : wraps tools to assess variants in gene sequences To be team-maintained on salsa.debian.org/med-team/python-geneimpacts It is a runtime dependency for bcbio.
Re: ZFS in Buster
Ansgar writes ("Re: ZFS in Buster"):
> Ian Jackson writes:
> > I think this would be both unwise legally (without seeking additional
> > legal advice) and rather rude to the kernel upstream whose code is
> > then being reused without permission - indeed, contrary to their
> > explicitly stated intent.
>
> At least one commercial distribution (Ubuntu) has been distributing ZFS
> binary modules as part of their Linux packages for some years and didn't
> have any problems.
That doesn't prove anything other than that no-one felt it was
politically or financially expedient to sue.
AIUI Debian's position is still that summarised here:
https://blog.halon.org.uk/2016/01/on-zfs-in-debian/
(sorry about the pale grey on white disease; it works well in w3m)
Are you trying to reopen that discussion ? If so then you should be
CCing ftpmaster@ and leader@ probably...
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Survey: git packaging practices / repository format
> "Ian" == Ian Jackson writes:
Ian> Enrico Weigelt, metux IT consult writes ("Re: Survey: git
Ian> packaging practices / repository format"):
>> I'd call it the 'git-only-workflow' ;-)
Ian> ...
>> It's not in official Debian. I've announced it long go, but
>> nobody here really cared. I couldn't even convice debian
>> maintainers for little less insane scm workflows (just look at
>> the kernel :p), but failed, so I don't waste my time anymore,
>> instead just clean up the mess for those packages that I actually
>> need.
Ian> Oh. I think I have misunderstood. I think you are describing
Ian> a git workflow you use as a *downstream* of Debian, not as a
Ian> maintainer *within* Debian.
Ian> And I think what you are saying is that you don't use source
Ian> packages (.dsc) except maybe in the innards somewhere of your
Ian> machinery. I think that is a good way for a downstream to
Ian> work. Certainly when I modify anything locally I don't bothere
Ian> with that .dsc stuff.
I'm certainly going to look at dck-buildpackage now, because what he
describes is a workflow I'd like to be using within Debian.
For some projects I want to ignore orig tarballs as much as I can. I'm
happy with native packages, or 3.0 quilt with single-debian-patch.
I don't want merge artifacts from Debian packaging on my branches.
I'm happy to need to give the system an upstream tag.
I'm happy for a dsc to fall out the bottom, and so long as it
corresponds to my git tree I don't care how that happens.
I have a slight preference for 3.0 format over 1.0 format packages. 3.0
makes it possible to deal with binaries, better compression and a couple
of things like that. The quilt bits are (in this workflow) an annoyance
to be conquered, not a value.
The thing his approach really seems to have going for it is that he
gives up on the debian history fast forwarding and instead rebases a lot
for a cleaner history.
If we could figure out a way to collaborate on something like that well,
it might be a very interesting tool to have.
--Sam
Exclude some architectures from an architecture-independent package ?
Hi, I'm the maintainer of package acpi-call, which is a kernel module allowing a user to call ACPI methods. Its build is handled by DKMS. The module used to build on all architectures, even if it was obviously useless on those which don't support ACPI. Starting with Linux 5.2, what used to be a mere warning about "acpi_root_dir" set to an undefined value, became an error, and prevents the module to be built on architectures which don't support ACPI, meaning every one except i386, amd64, ia64 and arm64. Since, AFAIK, the "Architecture" field in debian/control doesn't allow tuning the "all" wildcard by excluding some architectures, I have several (IMHO non-optimal) solutions to fix this: 1/ Restrict the architectures to i386, amd64, ia64, arm64 in debian/control. This would fix the problem, but will make the mirrors hold four copies of nearly identical binary packages, so wasting some space (even if it's not much: the binary package holding the module source code for DKMS is only 13 KB in size). 2/ Keep "arch:all" and set KBUILD_MODPOST_WARN in the module source code, to allow build on unsupported architectures. 3/ Keep the status quo and close the bug [1] as wontfix (that's what the reporter of [2], which is a duplicate of [1], did). [1] https://bugs.launchpad.net/ubuntu/+source/acpi-call/+bug/1830040 [2] https://bugs.launchpad.net/ubuntu/+source/acpi-call/+bug/1830787 What would be the "cleanest" solution according to you ? Thanks in advance for your advice. Regards, -- Raphaël Halimi signature.asc Description: OpenPGP digital signature
Re: Survey: git packaging practices / repository format
Enrico Weigelt, metux IT consult writes ("Re: Survey: git packaging practices /
repository format"):
> On 28.05.19 22:08, Ian Jackson wrote:
> > Please can we leave aside discussion of the merits or otherwise of
> > each of these formats/workflows.
> >
> > Perhaps we can talk about that (again!) at some point, but it tends to
> > derail any conversation about git packaging stuff and I don't want
> > this thread derailed.
>
> I understand you point, but I believe we really should discuss this.
> (maybe based on some specific examples)
Not in this thread, please. There have been many other threads about
these issues.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Survey: git packaging practices / repository format
Enrico Weigelt, metux IT consult writes ("Re: Survey: git packaging practices /
repository format"):
> I'd call it the 'git-only-workflow' ;-)
...
> It's not in official Debian. I've announced it long go, but nobody
> here really cared. I couldn't even convice debian maintainers for
> little less insane scm workflows (just look at the kernel :p), but
> failed, so I don't waste my time anymore, instead just clean up the
> mess for those packages that I actually need.
Oh. I think I have misunderstood. I think you are describing a git
workflow you use as a *downstream* of Debian, not as a maintainer
*within* Debian.
And I think what you are saying is that you don't use source packages
(.dsc) except maybe in the innards somewhere of your machinery.
I think that is a good way for a downstream to work. Certainly when I
modify anything locally I don't bothere with that .dsc stuff.
But my aim in this thread was to capture how people work *within*
Debian, where a maintainer is still required to produce a .dsc.
(Sorry that maybe I have wasted your time answering my questions.)
Regards,
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Realizing Good Ideas with Debian Money
[moving a discussion from -devel to -project where it belongs] > "Mo" == Mo Zhou writes: Mo> Hi, Mo> On 2019-05-29 08:38, Raphael Hertzog wrote: >> Use the $300,000 on our bank accounts? So, there were two $300k donations in the last year. One of these was earmarked for a DSA equipment upgrade. DSA has a couple of options to pursue, but it's possible they may actually spend $400k on an equipment refresh. $200k doesn't really go that far in terms of big infrastructure projects like bikeshed or similar. I'm looking for someone who would be willing to guide a discussion of the Money issues Martin brought up in his campaign. I don't have time to guide that effor myself. Real thought needs to be put into it; it will be at least as much work as the discussions I'm leading on packaging practices and git if done correctly. However it could be very valuable for the project. --Sam
Re: ZFS in Buster
Hi Jonathan, On Tue, May 28, 2019 at 8:50 PM Jonathan Carter wrote: > On 2019/05/28 18:43, Dan wrote: > > ZFS 0.8 has been released with lots of improvements, notably encryption. > > Yep, it's an exciting feature. > > > Sadly the Linux Kernel has introduced a commit in kernel 4.19 and 5.0 > > that prevents ZFS from using SIMD. The result is that ZFS won't be > > usable in Buster. See the following issue > > https://github.com/zfsonlinux/zfs/issues/8793 > > Buster ships zfs-dkms version 0.7.12-2 though, which works just fine on > buster. My message was not accurate. I think that the commit has been introduced in in 4.19.38 (released 2019-05-02). I think that Debian Buster uses 4.19.37 https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.19.38 The commit also affects ZFS 0.7 because SIMD is used for checksum operations. There might be a performance penalty in ZFS only if Debian Buster upgrades to 4.19.38. Best, Daniel
Re: Survey: git packaging practices / repository format
Enrico Weigelt, metux IT consult writes ("Re: Survey: git packaging practices /
repository format"):
> hmm, sounds quite complicated ... anyone here who could explain why
> exactly they're doing it that way ?
>
> by the way: that's IMHO an important information we should also collect:
> why exactly some particular workflow was picked
I don't want to get into that in this thread. It is too close to a
discussion of what is the best way to do things. I certainly don't
want to challenge people by asking "why".
I want people to feel free to describe things they have seen, or which
they do, without feeling criticised. "sounds quite complicated"
sounds like a criticism to me. And I don't want the thread derailed.
I would prefer to focus on "what".
(Also, I feel that I personally have a pretty good handle on the
advantages and disadvantages of various approaches and I can usually
see why people have done things, even things that I think are a bad
idea. So I don't need help with that.)
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: ZFS in Buster
On 28.05.19 18:43, Dan wrote: > Sadly the Linux Kernel has introduced a commit in kernel 4.19 and 5.0> that > prevents ZFS from using SIMD. The result is that ZFS won't be> usable in Buster. See the following issue> https://github.com/zfsonlinux/zfs/issues/8793 We recently had this discussion on lkml - yet another case of 3rdparty folks that just don't follow the license rules. It's not the kernel who broke zfs, it's zfs that broke itself. The kernel is GPL, and they just have to follow the rules or go away. OOT modules are conceptionally messy in the first place. It's sometimes okay as an temporary workaround, until things get mainlined. But intentionally keeping things oot for long time is just silly and creates lots of more problems than it creates. And they're even using now *deeply* arch-internal functions directly. > NixOS reverted that particular commit:> https://www.phoronix.com/scan.php?page=news_item&px=NixOS-Linux-5.0-ZFS-FPU-Drop Intentional license violation. Not funny. > Debian is the "Universal Operating System" and gives the user the> option to > choose. It provides "vim and emacs", "Gnome and KDE", If you wanna have something new included, you'll have to sit down and do the actual work. In the end of the day, it's that simple. > Would it be possible to provide an alternative patched linux kernel > that works with ZFS? You mean patching against the license ? > The ZFS developers proposed the Linux developers to rewrite the whole > ZFS code and use GPL, but surprisingly the linux developers didn't > accept. See below: > https://github.com/zfsonlinux/zfs/issues/8314 Wait, no. It's not that we refused anything (actually, I don't even recall any decent discussion on that @lkml). There even wasn't anything to accept or refuse - except the existing code, that is nowhere near a quality where any maintainer likes to even have a closer look at. The major problem is that ZoL always has been oot on purpose, which is the wrong approach to begin with. That also leads to bad code quality (eg. lots of useless wrappers, horrible maintenance, ...) What ZoL folks could do is step by step rewrite it to use mainline functionality where ever technically feasible and work closely with upstream to introduce missing functionality. Obviously, their current proprietary userland interface can't be accepted for mainline - it has to be reworked to be conformant w/ standard uapi (eg. we already have it for things like snapshots, deduplication, quotas, ...) But it's up to ZoL developers to do the actual work and post patches to lkml. There won't be anybody else doing that. --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: Survey: git packaging practices / repository format
Ian Jackson writes: > > Main packaging Delta from upstream Tools for manipulating > git branch represented as delta from upstream, > contains building .dsc, etc. > > Unmodified debian/patches gbp, gbp pq > upstream files,(only)quilt / dquilt > plus debian/* Manual patch editing > incl. d/patches > > Modified Direct changes git merge > upstream files,to upstream files (.dsc: 1.0-with-diff or > plus debian/*. single-debian-patch) > Maybe d/patches, depending. > History has direct merges from upstream. With modified upstream files in the main branch, plus debian/*, but usually no d/patches I use a seperate (manually rebased) branch for patches, and export those at dsc creation time using a gitpkg hook With unmodified upstream files in the main branch, plus debian/*, but usually no d/patches, I use git-debcherry to generate a quilt series at dsc build time.
Re: Survey: git packaging practices / repository format
On 28.05.19 19:31, Simon McVittie wrote: Hi, > Debian Linux kernel > === > > Tree contains: an incomplete debian/ directory, notably without d/control, > and no upstream source > Changes to upstream source are: d/patches only > Baseline upstream: changelog version => .orig tarball > Patches managed by: ??? > Special build tool: there is a pre-build step to generate d/control I'm handling the kernel very differently (actually the offical packages never actually built at my site), similar to what I've described in my other mails - layered branches: * layer 0: upstream tag (linus or greg) * layer 1: generic patches for making upstream's 'make dep-pkg' work with usual debian workflows (eg. not creating debian/rules from there anymore, but a generic one instead) * layer 2: dist and target specific customizations (changelos, .config, etc ...) The whole thing is again is built via dck-buildpackage (dpkg- buildpackage should also work, but I never called it manually anymore, since i've wrote dck-buildpackage). Note that I don't even try to create some one-fits-all superpackage for all archs, flavours, etc. - instead I'm using separate layer 2 branches for that. (for maintaining lots of kernel configs based on some meta config, I've got a separate tool) --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: Survey: git packaging practices / repository format
On 29.05.19 01:39, Simon McVittie wrote: Hi, > You might reasonably assume that, but no, they are not. mesa (and probably > other xorg-team packages) uses v1.0 dpkg-source format combined with > dh --with quilt, so deliberate Debian changes can be either direct > changes to the upstream source code, or quilt patches in d/patches, > or a mixture. Additionally, mesa uses d/source/local-options to ignore > files that only exist in the upstream git tag (which is what gets merged > into the packaging git branch), but not in the upstream `make dist` output > produced from that tag (which is used as the .orig tarball). hmm, sounds quite complicated ... anyone here who could explain why exactly they're doing it that way ? by the way: that's IMHO an important information we should also collect: why exactly some particular workflow was picked > My understanding is that this unusual difference between the .orig > tarball and what's in git is an attempt to "square the circle" between > two colliding design principles: "the .orig tarball should be upstream's > official binary artifact" (in this case Automake `make dist` output, > including generated files like Makefile.in but not non-critical source > files like .gitignore) and "what's in git should match upstream's git > repository" (including .gitignore but > not usually Makefile.in). Since we have git, I've completely given up the orig tarball - I'm just basing on their release tags. And, of course, there shouldn't be anything autogenerated in the git repo - always recreate everything (*especially* autotools-generated stuff). The orig tarball, IMHO, is a long obsolete ancient relic. For upstreams that don't have a git repo yet, I setup an importer first, and call that my upstream. --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: Why do we take so long to realise good ideas
Hi, On 2019-05-29 08:38, Raphael Hertzog wrote: > Use the $300,000 on our bank accounts? I totally support the idea that we should find more valuable usage of our fund. For example, if developers don't have enough time or don't want to do something difficult, we could hire somebody else to fix them. For example, many years ago I proposed that we could hire some web developers to rewrite our homepage, to make it more good-looking (Generally I don't care about superficial stuff but our homepage is really old enough. Look at Gentoo's homepage and compare it with the ancient version.)
Re: Survey: git packaging practices / repository format
On 28.05.19 22:08, Ian Jackson wrote: Hi, > Please can we leave aside discussion of the merits or otherwise of > each of these formats/workflows. > > Perhaps we can talk about that (again!) at some point, but it tends to > derail any conversation about git packaging stuff and I don't want > this thread derailed. I understand you point, but I believe we really should discuss this. (maybe based on some specific examples) OTOH, I'll only participate in such discussions, if I see that it's really going forward ... already tried that several times in recent years, but no success, so I just gave up :( --mtx -- Enrico Weigelt, metux IT consult Free software and Linux embedded engineering i...@metux.net -- +49-151-27565287
Re: Survey: git packaging practices / repository format
On 28.05.19 22:30, Ian Jackson wrote:
> Hi, thanks for replying. You have an interesting workflow which I
> think I need to ask some questions about before I can document it
> fully.
I'd call it the 'git-only-workflow' ;-)
The main reasons behind are:
* i wanna be able to easily rebase onto upstream anytime
* i wanna keep generic changes separate from the distro-specific stuff
(usually I try to do very generic, so it can go into mainline, eg.
instead of directly patching things like pathes, etc, I'm adding
new build options, ...)
* i wanna easily bring generic changes upstream
* i don't ever like to cope w/ text-based patches anymore (all these
apply/unapply cycles really suck :p) - git is much easier to handle,
IMHO
* i wanna have exactly the build tree in my git repo
* i don't wanna versioning patches (reading diffs of diffs are not quite
useful :o)
Actually, the workflow is a tiny bit more complex: i'm using layered
branches (regularily rebasing):
layer 0: upstream releases
layer 1: per release maintenance branches w/ generic (hopefully
upstreamable) fixes - based on the corresponding upstream
release tags (or potentially their maint branches)
layer 2: per distro and release debianized branches
(sometimes some layer 1.5 for really generic deb stuff)
Branches and tags have a canonical naming - ref name prefixes,
canonical version numbers, ... (eg. anyting for debian stretch is
prefixed 'stretch/' ...)
Years ago, I've already tried to form layer 1 into a greater,
cross-distro community, where stabelization efforts are shared between
many distros (kinda send-patches.org but w/ high grade of normalization
and automation. It was called the 'oss-qm' project (github org with same
name). But the interest from dist maintainers was asymptotically
approaching zero, from below.
> Enrico Weigelt, metux IT consult writes ("Re: Survey: git packaging practices
> / repository format"):
>> I'm always cloning the upstream repo, branch off at their release tag
>> and add all necessary chanages as individual git commits - first come
>> the generic (non-deb specific) patches, then the deb specific ones.
>> No text-based patches, or even magic rewriting within the build process.
>> The HEAD is exactly the debianized source tree,
>
> What source format do you use ? What is in debian/source/format, if
> anything ?
Usually "3.0 (quilt)", but I actually don't really care so much. Just
picked that some time, as it just worked, and never really though about
it anymore :p
> Do you handle orig tarballs at all ?
No. I'm exclusively using docker-buildpackage, which directly operates
on the final source tree - no intermediate steps like unpacking,
patching, etc.
One of the fine things (besides simplicity) is that if anything goes
wrong, I can just jump into the container (it intentionally doesn't
clean up failing containers) and directly work from there (the git
repo is also there).
> When you go to a new upstream, you make a new git branch, then ?
git checkout -b
git rebase
And then see it it works, finxing things, etc.
Of course, I'll also care about self-consistent and understandable
commits - git history is documentation, not rotating backup ;-)
> Do you publish this git branch anywhere ?
https://github.com/oss-qm
(from time to time I also send patches upstream)
>> which is then fed to dck-buildpackage.
>
> What is that ?
https://github.com/metux/docker-buildpackage
It's a little tool that sets up build containers (also creates base
images on-demand), including build tools, extra repos, etc, runs
the build in the container and finally pulls out the debs.
The main audience are folks that maintain extra repos (eg.
customizations, backports, etc) - that's one of the things I'm
regularily doing for my clients.
I've got another toolkit ontop of that, which helps maintaining
whole repos, including managing git repos and their remotes,
dependency handling, etc. It's actually not a standalone tool,
but a fundation for easily setting up your own customized build
environment. I'm using it for all my customers who get apt repos,
but also for backports and depotterization.
(Note: the 'master' branch currently is crappy, more a playgound, w/
lot's of things that have to be cleaned up ... for production use
fork from 'base' branch.)
> manpages.debian.org wasn't any help.
It's not in official Debian. I've announced it long go, but nobody
here really cared. I couldn't even convice debian maintainers for
little less insane scm workflows (just look at the kernel :p), but
failed, so I don't waste my time anymore, instead just clean up the
mess for those packages that I actually need.
--mtx
--
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
i...@metux.net -- +49-151-27565287
Re: Why do we take so long to realise good ideas (Was: Difficult Packaging Practices)
On Wed, 29 May 2019, Ansgar wrote:
> On Wed, 2019-05-29 at 10:38 +0200, Raphael Hertzog wrote:
> > On Wed, 29 May 2019, Andrey Rahmatullin wrote:
> > > One of the popular answers to this and some other problems is "nobody sat
> > > down and wrote the code". Not sure what can we do about this class of
> > > reasons.
> >
> > Use the $300,000 on our bank accounts?
>
> I heard that this didn't work out well the last time ("dunc tank"),
> though that was before the time I followed Debian development.
Yes, I was there (and mention it briefly in the questions of the talk I
gave) but it's been a long time ago. There are things to learn
from this failed experiment (such as "don't let the DPL decide alone who
gets paid") but there are also many reasons to believe that we are no
longer in the same situation. At that time, the number of persons working
on open source as part of their paid work was rather low and the jealousy
aspect was likely more problematic than it would be today. We have been
getting used to have Debian contributors being paid (such as on LTS) and
we know that with appropriate rules, the social impact of the use of money
is acceptable.
The topic still needs to be approached carefully but I believe that we
should aim to have this discussion and build some framework where we can
leverage money to complete projects and tasks that we find important
but that have not gone forward through volunteer work.
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Support Debian LTS: https://www.freexian.com/services/debian-lts.html
Learn to master Debian: https://debian-handbook.info/get/
Re: Why do we take so long to realise good ideas (Was: Difficult Packaging Practices)
On Wed, 2019-05-29 at 10:38 +0200, Raphael Hertzog wrote:
> On Wed, 29 May 2019, Andrey Rahmatullin wrote:
> > One of the popular answers to this and some other problems is "nobody sat
> > down and wrote the code". Not sure what can we do about this class of
> > reasons.
>
> Use the $300,000 on our bank accounts?
I heard that this didn't work out well the last time ("dunc tank"),
though that was before the time I followed Debian development.
Ansgar
Re: Why do we take so long to realise good ideas (Was: Difficult Packaging Practices)
Hi, On Wed, 29 May 2019, Andrey Rahmatullin wrote: > One of the popular answers to this and some other problems is "nobody sat > down and wrote the code". Not sure what can we do about this class of > reasons. Use the $300,000 on our bank accounts? https://lists.debian.org/debian-news/2019/msg2.html And I heard of another $300,000 donation from Google (through Thomas Koch) although I can't find any reference to it. FWIW, I gave a talk on LTS and the topic of funding Debian work at the minidebconf in Marseille (30 minutes): http://meetings-archive.debian.net/pub/debian-meetings/2019/miniconf-marseille/2019-05-25/5_years_lts_funding.webm My slides are here: https://wiki.debian.org/DebianEvents/fr/2019/Marseille?action=AttachFile&do=view&target=debian-lts-5-years-of-funding.pdf Cheers, -- Raphaël Hertzog ◈ Debian Developer Support Debian LTS: https://www.freexian.com/services/debian-lts.html Learn to master Debian: https://debian-handbook.info/get/ signature.asc Description: PGP signature
