Re: Derivatives, MongoDB and freezes
On Sat, Apr 20, 2013 at 7:37 PM, Daniel Pocock wrote: > Nonetheless, with derivatives and Debian itself having different release > cycles, and wearing my upstream developer hat, I can't help wondering: > how can upstreams ensure that the freshest versions of their package > propagate to the derivatives without duplicating effort? You can push those versions to experimental and ask Ubuntu to sync from there (they do that during freeze but not automatically) using requestsync from ubuntu-dev-tools (available in Debian). For other derivatives there aren't any specific sync procedures, you'll need to contact their development fora, some are listed in the census pages. http://wiki.ubuntu.com/SyncRequestProcess http://wiki.ubuntu.com/UbuntuDevelopment/Merging http://wiki.debian.org/Derivatives/Census -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAKTje6FG=sluznjrwqdwxymtmlnphujjn7ryx-lbr7ez9-0...@mail.gmail.com
Re: Derivatives, MongoDB and freezes
On 2013-04-20 04:37, Daniel Pocock wrote: I came across this on Planet Debian http://rb.doesntexist.org/blog//posts/lack_of_cooperation_from_ubuntu/ I'm guessing that Ubuntu may not have pushed the changes to sid because of the freeze, that may well be the answer to Rogério's questions. Those changes are extremely recent. For instance, MongoDB 2.2 was only uploaded to Ubuntu on March 21. The ARM changes are also just from this recent raring cycle which only began in October. MongoDB is getting a lot of attention because it is at the core of Juju. Normally this much delta would be avoided, but some new features from 2.2 are needed and ARM is an important platform for Ubuntu server. Typically Ubuntu takes a pattern of re-converging with Debian as much as possible early in each 6 month cycle. So I would expect that we will get quite a few patches pushed toward Debian unstable right about the time wheezy releases. Nonetheless, with derivatives and Debian itself having different release cycles, and wearing my upstream developer hat, I can't help wondering: how can upstreams ensure that the freshest versions of their package propagate to the derivatives without duplicating effort? These patches, by and large should not require duplication. If you look, some of the arm support patches happened in Fedora, not Ubuntu. Now, I think I might scold my colleagues in Ubuntu for not annotating their patches with bug reports so we can make sure they get forwarded to upstream and closed. But as long as these issues are being forwarded upstream, they should not be "duplicated" at all. For example, to respect the Debian release process, I've avoided uploading the latest versions of my packages to sid, so it appears that newer versions of those packages missed the boat when Ubuntu started their freeze. This means that both Debian and Ubuntu will release with versions of the packages that are old and don't have the latest bug fixes and/or any manual effort to work around that takes away time that could be spent on more bug fixes or features Debian is focused on stable releases. There's nothing "wrong" with this model, but it does mean you don't get the latest upstreams without diverging. IMO this is why upstream packaging should be embraced and enhanced rather than focusing on dpkg. I once worked on the 'pkgme' project for Ubuntu and there have been others, but never followed through. The idea was just to build debian source packages from upstream sources. Upstreams should be able to release a package which serves their needs, and Debian should be able to consume these almost directly. Where Debian's efforts should be focused is on things like license verification and helping bug reports and fixes get to upstream. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/da27206329b6ad09d56602e8d78b4...@secure.spamaps.org
Re: Derivatives, MongoDB and freezes
On 20 April 2013 12:37, Daniel Pocock wrote: > > > I came across this on Planet Debian > > http://rb.doesntexist.org/blog//posts/lack_of_cooperation_from_ubuntu/ > > I'm guessing that Ubuntu may not have pushed the changes to sid because > of the freeze, that may well be the answer to Rogério's questions. > That is mostly correct. The other part is that packages & fixes not suitable for sid/wheezy were not applied & uploaded to experimental as aggressively as Ubuntu pace would want. I also posted on Debian Planet, how to find patches applied in Ubuntu via Debian PTS together with categories of useful fixes that are relevant to Jessie and may be already solved/patched in Ubuntu. [1] I perceived that blog post as a partial personal stab at me, by the way. Which I found unjust. > Nonetheless, with derivatives and Debian itself having different release > cycles, and wearing my upstream developer hat, I can't help wondering: > how can upstreams ensure that the freshest versions of their package > propagate to the derivatives without duplicating effort? > > For example, to respect the Debian release process, I've avoided > uploading the latest versions of my packages to sid, so it appears that > newer versions of those packages missed the boat when Ubuntu started > their freeze. This means that both Debian and Ubuntu will release with > versions of the packages that are old and don't have the latest bug > fixes and/or any manual effort to work around that takes away time that > could be spent on more bug fixes or features. You can look at glibc, gcc, python which all got packages in Debian VCS and/or uploaded into Ubuntu from Debian VCS or sycned/merged from Debian Experimental uploads. These workflows can be done, but require cooperation and are not automatic (e.g. sync-from-experimental is always manual and not automatic as e.g. syncing from sid or testing). Ideally, i'd like to see debian to branch or use t-p-u, such that sid can continue accepting new uploads and not freeze. E.g. something similar to how fedora operates. I vaguely recall that something like that has been proposed to debian in the past, and didn't get much traction, since developers can get distracted by continious uploads instead of working on releasing the frozen part of the archive. Regards, Dmitrijs. [1] http://blog.surgut.co.uk/2013/04/ftbfs-fixes-and-other-patches-available.html -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/canbhlujh1-g77ffrhruganszhmx06gjzuexn5zc6hmoh7np...@mail.gmail.com
Re: Derivatives, MongoDB and freezes
Le Sat, Apr 20, 2013 at 11:05:29AM -0700, Clint Byrum a écrit : > > These patches, by and large should not require duplication. If you > look, some of the arm support patches happened in Fedora, not > Ubuntu. Now, I think I might scold my colleagues in Ubuntu for not > annotating their patches with bug reports so we can make sure they > get forwarded to upstream and closed. But as long as these issues > are being forwarded upstream, they should not be "duplicated" at > all. Hi Clint, this is a key point: the one who makes the patch should forward it upstream directly. This way it benefits everybody. For a large number of bug reports forwarded to Debian from Ubuntu, I wonder why one is asking me to forward Upstream, intead of doing it himself, especially in some race-conditions situations (new compilers, new linkers, etc.) where Upstreams would have adapted their software themselves if we had given them a bit more time. Cheers, -- Charles Plessy Debian Med packaging team, http://www.debian.org/devel/debian-med Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130421012815.gb5...@falafel.plessy.net
Re: Derivatives, MongoDB and freezes
On Sat, Apr 20, 2013 at 07:22:05PM +0100, Dmitrijs Ledkovs wrote: > I also posted on Debian Planet, how to find patches applied in Ubuntu > via Debian PTS together with categories of useful fixes that are > relevant to Jessie and may be already solved/patched in Ubuntu. [1] > > I perceived that blog post as a partial personal stab at me, by the > way. Which I found unjust. Rogério Brito wrote his post on the 19th¹, and you wrote your post on the 20th², so I fail to see how it could be a personal stab at you, or at least in relation to your post. I couldn't figure out how to comment on your post, because I wanted to complain about "Not uploading patches because they don't apply to Debian yet, is silly as they will apply to Debian eventually", which I think is unfair. It seems to me you want Debian people to make Ubuntu's life easier, whilst Rogério wants Ubuntu people to make Debian's life easier. Not all Debian developers have the time ot care about derivatives, and I'm the reverse is also true. ¹ http://blog.surgut.co.uk/2013/04/ftbfs-fixes-and-other-patches-available.html ² http://rb.doesntexist.org/blog//posts/lack_of_cooperation_from_ubuntu/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130422122021.GA11230@debian
Re: Derivatives, MongoDB and freezes
On 22 April 2013 13:20, Jonathan Dowland wrote: > On Sat, Apr 20, 2013 at 07:22:05PM +0100, Dmitrijs Ledkovs wrote: >> I also posted on Debian Planet, how to find patches applied in Ubuntu >> via Debian PTS together with categories of useful fixes that are >> relevant to Jessie and may be already solved/patched in Ubuntu. [1] >> >> I perceived that blog post as a partial personal stab at me, by the >> way. Which I found unjust. > > Rogério Brito wrote his post on the 19th¹, and you wrote your post on > the 20th², so I fail to see how it could be a personal stab at you, or > at least in relation to your post. > My post was in response to his. I have been partially involved in mongodb work in Ubuntu. > I couldn't figure out how to comment on your post, because I wanted to > complain > about "Not uploading patches because they don't apply to Debian yet, is silly > as they will apply to Debian eventually", which I think is unfair. It seems > to me you want Debian people to make Ubuntu's life easier, whilst Rogério > wants Ubuntu people to make Debian's life easier. Not all Debian developers > have the time ot care about derivatives, and I'm the reverse is also true. > yes, I agree. Both stances are equal and opposite. I guess my post was more about promoting DDs to check PTS once in a while. And to promote and advertise my position =) Thank you for your feedback. I also got feedback that patches.u.c generates full-diff instead of debdiff which can be rather large. Regards, Dmitrijs. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CANBHLUjR_QeUGpo50D9Q6mPF=2xtwawhvv1ibjbkpa+3b9o...@mail.gmail.com
Re: Derivatives, MongoDB and freezes
On Sat, Apr 20, 2013 at 07:22:05PM +0100, Dmitrijs Ledkovs wrote: > Ideally, i'd like to see debian to branch or use t-p-u, such that sid > can continue accepting new uploads and not freeze. E.g. something > similar to how fedora operates. I vaguely recall that something like > that has been proposed to debian in the past, and didn't get much > traction, since developers can get distracted by continious uploads > instead of working on releasing the frozen part of the archive. We already have a technical solution to developing for $release+n (n≥2): the DELAYED queue. We just need it to be expanded a little bit and for future releases to have their date pinned long in advance; we can then just upload new versions of packages to (say) DELAYED-700! (This is a joke and my penance will be to fix an RC bug.) -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130422131217.GA13404@debian
Re: Derivatives, MongoDB and freezes
On 04/20/2013 07:37 PM, Daniel Pocock wrote: > I came across this on Planet Debian > > http://rb.doesntexist.org/blog//posts/lack_of_cooperation_from_ubuntu/ > > I'm guessing that Ubuntu may not have pushed the changes to sid because > of the freeze, that may well be the answer to Rogério's questions. Thinking that this is the only reason is very naive. It's simply not the case. In some areas, Canonical guys believe they need to differentiate, even if that means making the life of maintainers of both distributions harder. The areas are where they focus commercially: cloud computing stacks, desktop with Unity and MIR, and I guess soon phone and TV sets. These are commercial decisions on some very specific parts of Ubuntu though, and we shall care to not at all generalize this sentiment. I am not aware that this is the case with MongoDB though, but it could well be, since that is in the cloud thingy, which Canonical is deeply involved commercially. It also well could be that Ubuntu maintainers are just busy, and/or didn't care enough (eg: this could be an individual issue rather than a company policy). Hard to know, really... We shouldn't think too bad about such corporate decisions, this is a commercial entity that we are dealing with, and it is very normal that they do have an agenda (and releasing on time is always part of this agenda). So don't try to guess. Just remember such things can happen, and try to deal with it in the best way possible, trying to push for more collaboration when you can. That is the policy that I am trying to apply to myself, and I hope it will be appreciated from both sides (eg: Debian and Ubuntu) in the long run. > Nonetheless, with derivatives and Debian itself having different release > cycles, and wearing my upstream developer hat, I can't help wondering: > how can upstreams ensure that the freshest versions of their package > propagate to the derivatives without duplicating effort? IMO, they can't. It is an endless effort. Though the most easy way is to do the work in Debian so that it has more chances to propagate in all derivatives. > For example, to respect the Debian release process, I've avoided > uploading the latest versions of my packages to sid, so it appears that > newer versions of those packages missed the boat when Ubuntu started > their freeze. As others stated, you should have uploaded these to Experimental. > This means that both Debian and Ubuntu will release with > versions of the packages that are old and don't have the latest bug > fixes and/or any manual effort to work around that takes away time that > could be spent on more bug fixes or features. Bugs can be fixed, even after releases... For the latest versions, we have SID / Experimental, so it is easy to cherry-pick what you need, even during the freeze. Then we have backports. It is to be noted that even Ubuntu people are deeply thinking about having backports repositories for their LTS releases. I'm not sure how far they are in that process though. Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/5175c05a.5030...@debian.org
Re: Derivatives, MongoDB and freezes
On Tuesday, April 23, 2013 06:57:30 AM Thomas Goirand wrote: > On 04/20/2013 07:37 PM, Daniel Pocock wrote: > > I came across this on Planet Debian > > > > http://rb.doesntexist.org/blog//posts/lack_of_cooperation_from_ubuntu/ > > > > I'm guessing that Ubuntu may not have pushed the changes to sid because > > of the freeze, that may well be the answer to Rogério's questions. > > Thinking that this is the only reason is very naive. It's simply not the > case. > > In some areas, Canonical guys believe they need to differentiate, even > if that means making the life of maintainers of both distributions harder. > The areas are where they focus commercially: cloud computing stacks, > desktop with Unity and MIR, and I guess soon phone and TV sets. > > These are commercial decisions on some very specific parts of Ubuntu > though, and we shall care to not at all generalize this sentiment. I am > not aware that this is the case with MongoDB though, but it could well > be, since that is in the cloud thingy, which Canonical is deeply involved > commercially. It also well could be that Ubuntu maintainers are just busy, > and/or didn't care enough (eg: this could be an individual issue rather > than a company policy). Hard to know, really... > > We shouldn't think too bad about such corporate decisions, this is a > commercial entity that we are dealing with, and it is very normal that > they do have an agenda (and releasing on time is always part of this > agenda). > > So don't try to guess. Just remember such things can happen, and try > to deal with it in the best way possible, trying to push for more > collaboration when you can. That is the policy that I am trying to apply > to myself, and I hope it will be appreciated from both sides (eg: Debian > and Ubuntu) in the long run. This entire discussion was started by someone, who is not the maintainer or uploader of a package, complaining on Planet Debian that Ubuntu had changes that Debian didn't. It was entirely a guess about why, since the he didn't bother to check with anyone involved with the package in either Debian or Ubuntu. I agree guessing is bad. I spoke with one of the Ubuntu people involved and he told me he's been in contact both with the Debian maintainer and upstream about the very issues that were whined about in the blog. So now we get to have the pointless discussion about it spill over into -devel too. In case anyone is confused, if someone would like to actually change the behavior of developers in a Debian derivative, the method for doing so that's likely to be effective is to actually communicate with them. Posting on Planet Debian (where a derivative developer who isn't paying attention to Debian probably isn't reading) is just pointless whining. As far as I can tell, out of all the people conversing on the topic, I'm the only one to go ask one of the involved parties what was happening. Random whining in blog posts may feel good, but so do other solo activities. Scott K -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/3736674.uP4Rc8B9ug@scott-latitude-e6320
Re: Derivatives, MongoDB and freezes
On Tue, Apr 23, 2013 at 06:57:30AM +0800, Thomas Goirand wrote: > On 04/20/2013 07:37 PM, Daniel Pocock wrote: > > I came across this on Planet Debian > > http://rb.doesntexist.org/blog//posts/lack_of_cooperation_from_ubuntu/ > > I'm guessing that Ubuntu may not have pushed the changes to sid because > > of the freeze, that may well be the answer to Rogério's questions. > Thinking that this is the only reason is very naive. It's simply not the > case. > In some areas, Canonical guys believe they need to differentiate, even > if that means making the life of maintainers of both distributions harder. > The areas are where they focus commercially: cloud computing stacks, > desktop with Unity and MIR, and I guess soon phone and TV sets. There is a difference between "needing to differentiate" and "needing to deliver a product in the most efficient way possible". In cases where Debian and Ubuntu are moving in the same direction and/or sharing technologies, the most efficient way to deliver Ubuntu as a product is to do so in collaboration with Debian; and so there's a standing policy that changes to packages do get pushed back to Debian from Ubuntu, as others have commented. But for new packages, where Canonical is striking out on its own to deliver significant new functionality and the folks working on these packages are not DDs, there's a clear pragmatic argument for doing the work directly in Ubuntu rather than blocking the work on finding folks able to upload to Debian and willing to maintain the packages there. That doesn't mean Canonical is not interested in having such software in Debian. I can't speak to the cloud packages in particular, but I know there's been an ITP for Unity in the past, which unfortunately hasn't made it to completion for one reason or another. I think the Unity team would welcome having Unity available for Debian users - it just doesn't make sense for them to try to push Debian to accept such packages, or to volunteer to maintain them. Likewise, while the goal of Mir is first and foremost to be a platform for Unity and the Ubuntu client offerings, and the team's priorities will naturally reflect this, I'm sure that if the Debian community found Mir useful for their own purposes the Mir team would be happy to see Mir in Debian. > So don't try to guess. Just remember such things can happen, and try > to deal with it in the best way possible, trying to push for more > collaboration when you can. That is the policy that I am trying to apply > to myself, and I hope it will be appreciated from both sides (eg: Debian > and Ubuntu) in the long run. Well said. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: Digital signature
Re: Derivatives, MongoDB and freezes
Steve Langasek wrote: > But for new packages, where Canonical is striking out on its own > to deliver significant new functionality and the folks working on these > packages are not DDs, there's a clear pragmatic argument for doing the work > directly in Ubuntu rather than blocking the work on finding folks able to > upload to Debian and willing to maintain the packages there. To be a devil's advocate: when the Debian Developers that a company has been able to contact (inside or outside the company) do not consider a package to be ready for upload, it is not hard to contribute the packaging to Debian in an RFP bug to avoid duplication of effort. If the company considers the package suitable for ongoing use (for example via inclusion in Ubuntu universe), it's probably suitable for Debian experimental, too, and it doesn't take much time to upload and sync from there. So I don't see the delay as an important reason not to cooperate. Some important reasons not to cooperate publically would be avoiding revealing business plans too early, or lack of expertise regarding Debian processes by the people working on a particular package. Jonathan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130423065832.GA1390@elie.Belkin
Re: Derivatives, MongoDB and freezes
On Mon, Apr 22, 2013 at 11:58:33PM -0700, Jonathan Nieder wrote: > Steve Langasek wrote: > > But for new packages, where Canonical is striking out on its own > > to deliver significant new functionality and the folks working on these > > packages are not DDs, there's a clear pragmatic argument for doing the work > > directly in Ubuntu rather than blocking the work on finding folks able to > > upload to Debian and willing to maintain the packages there. > > To be a devil's advocate: when the Debian Developers that a company > has been able to contact (inside or outside the company) do not > consider a package to be ready for upload, it is not hard to > contribute the packaging to Debian in an RFP bug to avoid duplication > of effort. > Indeed, this answers the first point, but the second is more significant - willing to maintian the packages there. Neil -- signature.asc Description: Digital signature
Re: Derivatives, MongoDB and freezes
Hi Daniel and others.
On Apr 20 2013, Daniel Pocock wrote:
> I came across this on Planet Debian
>
> http://rb.doesntexist.org/blog//posts/lack_of_cooperation_from_ubuntu/
>
> I'm guessing that Ubuntu may not have pushed the changes to sid because
> of the freeze, that may well be the answer to Rogério's questions.
First of all, thank you (Daniel) very much for CCing me on this post.
Second, thank you for bringing this to a more technical audience.
*** From now on, occurrences of 'YOU' do not necessarily apply to Daniel. ***
Third, I will elaborate this in a longer fashion as another post in my blog.
In fact, it will be a lot longer, but I just want to address some points
here in a more specific form, quoting the e-mails directly.
Fourth, I would like to apologize for not getting in contact with the Ubuntu
maintainers of the MongoDB package before posting to my blog.
Fifth, I would *not* like to apologize for the people that have *not* pushed
their work even *before* wheezy was in a freeze. In fact, for *more* than 1
year. That's the time for Ubuntu's 12.04 LTS, 12.10, and 13.04 releases.
You had a *lot* of time to get things pushed to Debian (or, even better,
directly with upstream). So much for "cadence, cadence, cadence".
Sixth, I may not be a Debian Developer (who cares, there's a lot of work
being done for Debian coming from people that are *not* Debian Developers),
but I know a good deal about Debian, *and* I care about the distribution
that I use.
Please, correct me if I am wrong, but, apparently, this is a strong signal
that I do care *more* about Debian than some people that don't distribute
their work to their base distribution.
Seventh, despite not being a Debian Developer, I am a Debian Maintainer, and
I know very well that even if I am not listed in the Uploaders field of a
package or have unlimited uploading rights, I *can* (and, morally, *should*,
as should you who already had a fix!) work towards making the software that
we rely on being better.
Please, don't tell me that "This entire discussion was started by someone,
who is not the maintainer or uploader of a package, complaining on Planet
Debian that Ubuntu had changes that Debian didn't.".
Again, please educate me if I am wrong, but the "knowing what your role
is" is, BTW, one of the rationales (if not *the* rationale) behind the NMU
thing that governs the weekly Release Critical bugfixes that people like
Gregor Herrmann (thanks, gregoa and others!) try to maintain.
If I am wrong here, please, do wake me up from my misguided dream and tell
me that Debian is not about technical excellence---I should, then,
reconsider where I spend my time. More than anything, I think that
willing to maintain the packages is of high importance and (thanks,
Eight, having a blob of 522KB of patches (when you could integrate things
earlier) is a lot. Especially when it has really trivial things that could
have been pushed way back then (Remember? at least two of your releases went
by!).
Ninth, I don't really have to tell others about my work, but in the case of
those asking "who are you", then you can know about me if you try to see
some of my work in packages that others have considered a "hard nut to
crack", like the HandBrake video transcoder (among others).
Tenth, the technical person in me is puzzled to know how some magical things
are working in a not-so-intended way and, in the course of the investigation
discover more than 4 embedded libraries in a package (which are, now,
already killed in the Debian archives for unstable, but which are not in
wheezy).
At the risk of omitting some names, I would like to thank (even if you
didn't realize that you were agreeing with me in thread---but you were)
Jonathan Dowland, Charles Plessy, Wouter Verhelst, Gergely Nagy, Guillem
Jover, Daniel Pocock, Thomas Goirand, Russ Allbery.
OK, I can certainly go on and on. Again, I apologize for some of my behavior
(the form of it, namely, a blog post in general), but not for all of it (the
content).
Regards,
Rogério.
P.S.: I'm not subscribed to debian-devel at the moment and would appreciate
being CC'ed.
--
Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFC
http://rb.doesntexist.org/blog : Projects : https://github.com/rbrito/
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130430222308.ga4...@ime.usp.br
Upstream packaging (was Re: Derivatives, MongoDB and freezes)
Hi! On Sat, 2013-04-20 at 11:05:29 -0700, Clint Byrum wrote: > [...]. IMO this is why upstream packaging should be > embraced and enhanced rather than focusing on dpkg. I'm not sure if you refer to the tool here, or to the packaging work, doesn't change much anyway. > I once worked on the 'pkgme' project for Ubuntu and there have been > others, but never followed through. The idea was just to build > debian source packages from upstream sources. Upstreams should be > able to release a package which serves their needs, and Debian > should be able to consume these almost directly. Respectfully, I think you've entirely missed the point of a distribution (and sadly I'm seeing this trend more often than before now, with all the hype around app stores and similar...). Packaging and maintaining a consistent and unified distribution cannot be delegated to upstreams (and I'm talking as an upstream developer here too), because that entails a bit more than slapping some files somewhere, tarring the thing up and calling it a day. Building a nice distribution requires a high-level view and QA of the entire system; requires curating sane namespaces, be them on the package/project names, on the version strings, on the filesystem (by avoiding file collisions, using alternatives or diversions), on exposed programming interfaces, etc; requires making sure a diverse set of programs interact correctly with each other; performing security updates; ideally keeping single runtime versions; doing global transitions to use other or newer runtimes, programs, libraries or packages; an unified way to build from sources to cope with the endless and interesting different upstream build systems; porting and building for different binary architectures, not just what upstream might have around; documentation; translation; tagging stuff with metadata to allow for easy searches; excision of the embedded code copies cancer; even stuff like how the Delete and Backspace keys should behave; sets a qualify bar for upstream projects, stuff of low quality will not be accepted most of the time; license checks; etc, etc... And all this, upstream will never be able to provide (at least not as defaults), because each distribution has its own policies, some are better, some are worse, and some are just different. In general I'd never trust the packaging produced by an upstream for a distribution they are not involved in. But most of the time there will be no such packages for the desired distribution anyway. Although there's been work on creating distribution-neutral specs that some upstreams have picked up, there's always going to be something new, the specs will just not cover all needed things, the specs might not be liked by some/many people, or might not have been fully adopted by all upstreams. A distribution (any, most) is the gel that binds and gives an unified and coherent shape to the software ecosystem. An app store is just like a scrapyard, you might find magnificent and isolated gems there, but most of it will probably be junk, or not combine with the other scrap parts. Also if people thought that distributions are unneeded, then the amount of them would reflect that, or start decreasing, which I'm not seeing. Distributions will exist as long as there's FLOSS, because by its decentralized nature, there's no single coordination point; people who are distraught by the amount of distributions or by their mere existence might probably only be able to find peace in closed-systems instead, with their centralized control points. > Where Debian's efforts should be focused is on things like license > verification and helping bug reports and fixes get to upstream. So basically, getting rid of most of the fun stuff and turning it into a lawyerish play-ground and support center... I'd venture to say, not the most attractive work for most people here if it was the only thing to be done, which we do because we think it's important non the less. Regards, Guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130424194348.ga32...@gaara.hadrons.org
Re: Upstream packaging (was Re: Derivatives, MongoDB and freezes)
On Wed, Apr 24, 2013 at 09:43:48PM +0200, Guillem Jover wrote: > ... > A distribution (any, most) is the gel that binds and gives an unified > and coherent shape to the software ecosystem. +1 (to everything even the cutted part) > An app store is just like > a scrapyard, you might find magnificent and isolated gems there, but most > of it will probably be junk, or not combine with the other scrap parts. I honestly wonder if there is some more general definition for the term "app store" besides what according to[1] certain companies have made out of it. Following the logic that "app store" is crap because some companies provide it as such we are terribly lucky that they did not choose the term "distribution" for their crap. I personally can not find something wrong in a *generic* term "application store" and I would not seriously object if someone would call Debian an app store done the right way. > > Where Debian's efforts should be focused is on things like license > > verification and helping bug reports and fixes get to upstream. > > So basically, getting rid of most of the fun stuff and turning it into > a lawyerish play-ground and support center... I'd venture to say, not > the most attractive work for most people here if it was the only > thing to be done, which we do because we think it's important non the > less. +1 Kind regards Andreas. [1] http://en.wikipedia.org/wiki/App_Store -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130425072059.gd23...@an3as.eu
Re: Upstream packaging (was Re: Derivatives, MongoDB and freezes)
On 2013-04-24 12:43, Guillem Jover wrote: Hi! On Sat, 2013-04-20 at 11:05:29 -0700, Clint Byrum wrote: [...]. IMO this is why upstream packaging should be embraced and enhanced rather than focusing on dpkg. I'm not sure if you refer to the tool here, or to the packaging work, doesn't change much anyway. I'm referring to delivering software in general, most of which falls into a few categories which are not "a programming language", "plumbing", and "system development tools". I once worked on the 'pkgme' project for Ubuntu and there have been others, but never followed through. The idea was just to build debian source packages from upstream sources. Upstreams should be able to release a package which serves their needs, and Debian should be able to consume these almost directly. Respectfully, I think you've entirely missed the point of a distribution (and sadly I'm seeing this trend more often than before now, with all the hype around app stores and similar...). Packaging and maintaining a consistent and unified distribution cannot be delegated to upstreams (and I'm talking as an upstream developer here too), because that entails a bit more than slapping some files somewhere, tarring the thing up and calling it a day. Indeed, there is a great deal of hard work in putting together an operating system. But for the bulk of software, things like MongoDB included, I see very little point in distributions spending a lot of time tweaking and poking and prodding the software to work into a grand policy. For the most part, developers ship and test a good tarball that builds with some pretty standard and easy to detect options. The bits where that doesn't work ought be fixed upstream, and I know sometimes they are, but in the mean time, the distribution maintainers (myself included) spend their precious time conforming to the broken upstream, instead of the other way around. Building a nice distribution requires a high-level view and QA of the entire system; requires curating sane namespaces, be them on the package/project names, on the version strings, on the filesystem (by avoiding file collisions, using alternatives or diversions), on exposed programming interfaces, etc; requires making sure a diverse set of programs interact correctly with each other; performing security updates; ideally keeping single runtime versions; doing global transitions to use other or newer runtimes, programs, libraries or packages; an unified way to build from sources to cope with the endless and interesting different upstream build systems; porting and building for different binary architectures, not just what upstream might have around; documentation; translation; tagging stuff with metadata to allow for easy searches; excision of the embedded code copies cancer; even stuff like how the Delete and Backspace keys should behave; sets a qualify bar for upstream projects, stuff of low quality will not be accepted most of the time; license checks; etc, etc... All of the things you mention are huge accomplishments, but the scope is what I am suggesting has gotten out of hand. Do we really need "a high level view" and "QA of the entire system" for MongoDB? It is a daemon and some client tools. If I install it from tarball, I know this is shocking, but it actually works fine and doesn't interfere with anything else. If it does, I will complain to upstream. My suggestion is not to stop packaging, but to shift focus from "make an awesome package" to "make an awesome upstream" that results in an automatically generated awesome package. Debhelper and many of the other tools definitely help with this, but we can do more. And all this, upstream will never be able to provide (at least not as defaults), because each distribution has its own policies, some are better, some are worse, and some are just different. In general I'd never trust the packaging produced by an upstream for a distribution they are not involved in. But most of the time there will be no such packages for the desired distribution anyway. Debian wants to provide end-to-end generalized tight integration. Thus packages lagging behind upstream. I'm suggesting that this tight integration ideal actually *limits* the scope of Debian. Although there's been work on creating distribution-neutral specs that some upstreams have picked up, there's always going to be something new, the specs will just not cover all needed things, the specs might not be liked by some/many people, or might not have been fully adopted by all upstreams. Right, and Debian maintainers can, and will keep bending over backward to get all the upstreams into Debian. That doesn't mean its a good use of someone's time. A distribution (any, most) is the gel that binds and gives an unified and coherent shape to the software ecosystem. An app store is just like a scrapyard, you might find magnificent and isolated gems there, but most of it will prob
Re: Upstream packaging (was Re: Derivatives, MongoDB and freezes)
Le Thu, Apr 25, 2013 at 01:50:58AM -0700, Clint Byrum a écrit : > > My suggestion is not to stop packaging, but to shift focus > from "make an awesome package" to "make an awesome upstream" that > results in an automatically generated awesome package. Debhelper and > many of the other tools definitely help with this, but we can do > more. Hi Clint, indeed, we do more: http://wiki.debian.org/UpstreamGuide Have a nice day, -- Charles Plessy Debian Med packaging team, http://www.debian.org/devel/debian-med Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130425094629.gf21...@falafel.plessy.net
Re: Upstream packaging (was Re: Derivatives, MongoDB and freezes)
Quoting Andreas Tille (2013-04-25 09:20:59) > On Wed, Apr 24, 2013 at 09:43:48PM +0200, Guillem Jover wrote: > > ... > > A distribution (any, most) is the gel that binds and gives an unified > > and coherent shape to the software ecosystem. > > +1 (to everything even the cutted part) > > > An app store is just like a scrapyard, you might find magnificent > > and isolated gems there, but most of it will probably be junk, or > > not combine with the other scrap parts. > > I honestly wonder if there is some more general definition for the > term "app store" besides what according to[1] certain companies have > made out of it. Following the logic that "app store" is crap because > some companies provide it as such we are terribly lucky that they did > not choose the term "distribution" for their crap. I personally can > not find something wrong in a *generic* term "application store" and I > would not seriously object if someone would call Debian an app store > done the right way. If by "not seriously object" you mean that you would object but with a smile and continue your great meal together, rather than leave the party and file a lawsuit, then I agree with you... To me, calling Debian an "app store" would be only misleading. To me an "app store" is for "topping op" a system, not the system itself. Debian is no app store: content is not only apps nor app-centric, and aim is not to archive nor to sell, but to streamline and distribute. When I need an alternative term to describe what "distribution" means, I use "library": Like librarians we care not only about the individual "books" of software, and not only about the "novels", but about long-term cohesive structure of the library as a whole. Libraries also has the connotation of "collecting dust" which arguably is descriptive for (stable) Debian as well ;-) - Jonas -- * Jonas Smedegaard - idealist & Internet-arkitekt * Tlf.: +45 40843136 Website: http://dr.jones.dk/ [x] quote me freely [ ] ask before reusing [ ] keep private -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130425100011.12767.1...@bastian.jones.dk
Re: Upstream packaging (was Re: Derivatives, MongoDB and freezes)
On 25-04-13 10:50, Clint Byrum wrote: > On 2013-04-24 12:43, Guillem Jover wrote: > All of the things you mention are huge accomplishments, but the scope is > what I am suggesting has gotten out of hand. Do we really need "a high > level view" and "QA of the entire system" for MongoDB? We don't need it for MongoDB, but we do need it for Debian. The difference might seem to be minor, but it isn't. > It is a daemon > and some client tools. If I install it from tarball, I know this is > shocking, but it actually works fine and doesn't interfere with anything > else. That's fine, great, and dandy. But it's besides the point. I would hope that all upstream software "works fine" and "doesn't interfere with anything else" if installed from tarball. If it doesn't, they seriously need to check what they're doing. But Debian packages go beyond "work fine". They are integrated. If there are other packages out there that do similar functionality to whatever it is that "MongoDB" does (I don't have a clue, but it doesn't matter for this discussion; I guess it's some sort of database), then it's reasonable to expect that a MongoDB package would need to behave similarly to those other packages. For a database server, this could include things like: - making sure the data is stored in the right location in the file system; e.g., postgresql upstream assumes all its stuff (binaries, configuration, data files) are stored in a single directory. Given their background (they support way more systems than we do; e.g., they also support Windows) this makes sense, but that doesn't mean we should, too. - If it's an SQL database, integrating everything with dbconfig-common. This is a Debian-specific thing to make installing database-using packages easier; it makes no sense whatsoever to push this upstream. - ... probably others, but I'm not very fluent in packaging of database systems. > If it does, I will complain to upstream. My suggestion is not to > stop packaging, but to shift focus from "make an awesome package" to > "make an awesome upstream" that results in an automatically generated > awesome package. Debhelper and many of the other tools definitely help > with this, but we can do more. Yes, we do have a guideline that says you should push improvements upstream if and when they make sense there, and that is more often the case than you'd think if you were just looking at the number of patches that flow upstream. But making a distribution is so much more than just checking licenses and converting source into installable packages. Take, for instance, the apache packages. If you were to just take the upstream source, compile, and install it, then it would also Just Work. But it wouldn't have the same flexibility that the Debian packages have. On Debian, you can just install some libapache-mod-foo package, and due to how the configuration files are arranged, it will simply work. This kind of integration isn't something that can (or should!) be done upstream; it's squarely inside the realm of what a distribution does. The amount of such integration is part of what makes a distribution unique; some distributions tend to do this more than others, and at one extreme of this are Arch and Slackware, who make it a policy to ship upstream source as pristine as possible. Debian is clearly not at that extreme, and it should not be, IMO; the high level of integration, the expectation that I can have from the behaviour of a package that's part of Debian is what attracted me to Debian in the first place, and if we ever lose that I'll probably start looking for a different distribution. >> And all this, upstream will never be able to provide (at least not as >> defaults), because each distribution has its own policies, some are >> better, some are worse, and some are just different. In general I'd >> never trust the packaging produced by an upstream for a distribution >> they are not involved in. But most of the time there will be no such >> packages for the desired distribution anyway. >> > > Debian wants to provide end-to-end generalized tight integration. Thus > packages lagging behind upstream. I'm suggesting that this tight > integration ideal actually *limits* the scope of Debian. Sometimes packages are lagging behind, yes, and that's a problem; but I don't agree that the tight integration lies at the root of that problem. When done right, this kind of integration consists of just a few files inside a debian/ directory; they should not interfere with packaging a new upstream version. Yes, there are exceptions, and we should strive to eliminate such cases. Sometimes this is because upstream is no longer active; sometimes this happens because the maintainer isn't doing a good (enough?) job. Whatever the reason, it's something we should try to fix. But just throwing our hands up in the air and saying that we should therefore give up on trying to integrate isn't the right way forward, I'd say. [...] >> Although there's been
Re: Upstream packaging (was Re: Derivatives, MongoDB and freezes)
Hi, On Thu, Apr 25, 2013 at 12:00:11PM +0200, Jonas Smedegaard wrote: > Quoting Andreas Tille (2013-04-25 09:20:59) > > I honestly wonder if there is some more general definition for the > > term "app store" besides what according to[1] certain companies have > > made out of it. Following the logic that "app store" is crap because > > some companies provide it as such we are terribly lucky that they did > > not choose the term "distribution" for their crap. I personally can > > not find something wrong in a *generic* term "application store" and I > > would not seriously object if someone would call Debian an app store > > done the right way. > > If by "not seriously object" you mean that you would object but with a > smile and continue your great meal together, rather than leave the > party and file a lawsuit, then I agree with you... It depends with whom I share that great meal. If it is my father I would simply say "yes". If it is my son I would mind explaining the difference. Rationale: I have installed Debian on my fathers box and told him how he can install additional applications. Once you have a basic system running the Debian mirror serves as an app store and it makes actually the point for this kind of user (and a lot of other users as well). > To me, calling Debian an "app store" would be only misleading. Yes. That's why I would not call Debian an "app store" to *you*. > To me an > "app store" is for "topping op" a system, not the system itself. There are actually users who do not "see" the system but just the topping. I would never try to blame the user about this. For my father it is even easier to understand what I'm doing in Debian because I spend most of my time with leaf packages (if I do not care for Blends infrastructure stuff). So telling him "Debian is an app store and my work on it is adding apps to the store" this is an very easy to understand explanation which leaves out the part that is not understandable to him: What is an operating system. (Hey, also Windows is no operating system - it is just a kick-starter for Windows, Excel, a browser and a mail client, right?) > Debian is no app store: content is not only apps nor app-centric, and > aim is not to archive nor to sell, but to streamline and distribute. If you do not like the selling part: Store in the sense of some "storage of goods" is not necessarily about bying (at least of my understanding). Or tweak it like this: We are selling our stuff but the price tag says 0€/$. Feel free to blame me about oversimplification > When I need an alternative term to describe what "distribution" means, I > use "library": Like librarians we care not only about the individual > "books" of software, and not only about the "novels", but about > long-term cohesive structure of the library as a whole. > > Libraries also has the connotation of "collecting dust" which arguably > is descriptive for (stable) Debian as well ;-) Nice alternative explanation. Kind regards Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130425124038.ga7...@an3as.eu
Re: Upstream packaging (was Re: Derivatives, MongoDB and freezes)
On Thu, Apr 25, 2013 at 02:40:38PM +0200, Andreas Tille wrote: > understandable to him: What is an operating system. (Hey, also Windows > is no operating system - it is just a kick-starter for Windows, Excel, a > browser and a mail client, right?) s/for Windows, Excel/for Word, Excel/ Kind regards Andreas. -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130425125614.gb7...@an3as.eu
Re: Upstream packaging (was Re: Derivatives, MongoDB and freezes)
On 04/25/2013 09:40 AM, Andreas Tille wrote: > There are actually users who do not "see" the system but just the > topping. Yes, but I don't think we should encourage any users in this skewed view of the system. > I would never try to blame the user about this. Nor would I. However, I would not use this as an excuse not to educate them. > For my father > it is even easier to understand what I'm doing in Debian because I spend > most of my time with leaf packages (if I do not care for Blends > infrastructure stuff). So telling him "Debian is an app store and my > work on it is adding apps to the store" this is an very easy to > understand explanation which leaves out the part that is not > understandable to him: What is an operating system. (Hey, also Windows > is no operating system - it is just a kick-starter for Windows, Excel, a > browser and a mail client, right?) I understand using app store as an analogy, as long as it is explained as such, and qualified as being an imperfect analogy. But the common conception of an "app store" as involving you solely as consumer and not as participant in the free software ecosystem encourages poor relationship between users and producers (or distributors) of free software. So long as users continue to see themselves as a tiny, insignificant recipient at the end of the production of the software with no input into the system, you've stripped them of the power to change the software to meet their needs. So using the "app store" analogy is walking the fine line and really needs to be qualified to avoid doing damage to the user's relationship to the community. > If you do not like the selling part: Store in the sense of some > "storage of goods" is not necessarily about bying (at least of my > understanding). Or tweak it like this: We are selling our stuff but > the price tag says 0€/$. Feel free to blame me about oversimplification Honestly, I don't think the selling part is the most offensive part of the concept of an "app store". It's the one-way nature of the transactions carried out with them. Ben -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51792a53.7080...@sanctuary.nslug.ns.ca
