Re: Debian development and release: always releasable (essay)

[Vincent Bernat]

 ❦  9 mai 2013 21:49 CEST, Lars Wirzenius  :

> * Remove RC buggy packages sooner rather than later. An RC buggy
>   package should be removed at soon as possible: when the bug
>   is identified, allow a bit of time for the bug to be verified
>   (was it actually an RC bug?), but after that, remove the package
>   from testing, preferably automatically.  If the package has
>   reverse dependencies, remove those as well. This keeps testing
>   releasable. The removed package can and will re-enter testing once
>   it gets fixed.

I think that's a very good idea. A threshold on the number of source
packages that can be removed from testing due to an RC bug could be
something like 10. The release team should be entitled to delay the
removal if the bug is quite complex.

I think that the other things you propose are too complicated:

> A package that is not included in one or more of the reference
> installations is a package we want to include in the release, but we
> will not delay the release for its sake. We should have a low threshold
> for removing such a package from testing: it could perhaps even be
> removed automatically one week after an RC bug is filed against it
> (assuming the bug affects the version in testing).

If a package is important, an RC bug will get noticed and someone will
step to fix the RC bug or ask for a delay. This avoids unnecessary
debate on what is important and what is not and people fighting to get
their packages in the right list.

> Tests for running reference installation might include the following:
>
> * Basic networking setup works: System responds to ping from the outside.
> * Mail server responds appropriate on the SMTP, submission, IMAPS, and POPS
>   ports.
> * LAMP server responds on the HTTP and HTTPS ports.
> * A desktop system that automatically logs in a test user has the right
>   processes running, and can start some common applications.
> * In each case, it's possible to log in remotely with ssh and run
>   "sudo apt-get install hello".

Many people run unstable and a bug that would fail those kind of tests
would get immediatly noticed and many bug reports are usually opened at
once for each of those bugs. Maybe those tests would catch them one hour
earlier but is it worth spending time to conceive those tests?

> These are trivial, even simplistic tests. However, if they pass, we know
> that at least the basic, fundamental things in the system are not horribly
> broken: networking, system administration, and the software that is meant
> to start in that reference installation. Furthermore, we know that the
> debian-installer works. That's a good foundation for further hacking.

Maybe the installer is less daily tested but did we already have a major
bug (one that does not pass the test you described) gone unnoticed?
-- 
 /*
  *   Should be panic but... (Why are BSD people panic obsessed ??)
  */
2.0.38 /usr/src/linux/net/ipv4/ip_fw.c


pgpQIIIqPZawA.pgp
Description: PGP signature

Re: Debian development and release: always releasable (essay)

[Russ Allbery]

Vincent Bernat  writes:
>  ❦  9 mai 2013 21:49 CEST, Lars Wirzenius  :

>> A package that is not included in one or more of the reference
>> installations is a package we want to include in the release, but we
>> will not delay the release for its sake. We should have a low threshold
>> for removing such a package from testing: it could perhaps even be
>> removed automatically one week after an RC bug is filed against it
>> (assuming the bug affects the version in testing).

> If a package is important, an RC bug will get noticed and someone will
> step to fix the RC bug or ask for a delay. This avoids unnecessary
> debate on what is important and what is not and people fighting to get
> their packages in the right list.

Personally, I think it's important to be more transparent and systematic
about how we designate packages as important or not important; it will add
clarity and it will let us be more efficient and encourage people to be
bold.  The fact of the matter is that we already have those distinctions:
we will remove random leaf packages from testing as soon as the release
gets close, but we're not going to remove cron or bash.  But the
distinctions are fuzzy and require people to make (and then argue about)
judgement calls.  We can't eliminate the arguments entirely, but I think
we can approach the process in a cleaner way that will require less
constant debate.

Package sets for critical purposes are useful in their own right.  They
make it clear why a package is important (and also why it may *cease* to
be important), and they also provide a basis for automated testing.

Personally, I'd be inclined to unify optional and extra (which at this
point don't really represent a meaningful difference) and possibly even
further simplify our use of priorities (it's not clear to me that standard
is really buying us anything any more), replacing most of that with
package sets for key use cases that we want to support.

One interesting possibly that this would also open up (and please
understand that I don't know if this would happen and I'm not positive
that it's a good idea; I'm just throwing it out there) is that the teams
who most care about a particular reference package set could also do some
of the release management duties for that package set.  For example,
suppose we had a LAMP team of experts in that package set.  Could they
possibly take on, not just maintaining the list of packages, but doing
freeze reviews and transition coordination for transitions that are
contained within that set?

To some extent, the desktop packaging groups (GNOME, KDE, etc.) already do
some of this for those desktop environments, and I think that's quite
helpful and might be useful to formalize further.

> Many people run unstable and a bug that would fail those kind of tests
> would get immediatly noticed and many bug reports are usually opened at
> once for each of those bugs. Maybe those tests would catch them one hour
> earlier but is it worth spending time to conceive those tests?

Yes, absolutely.  Because when you have the tests, it opens up all sorts
of possibilities for automation.  For example, you could, in the future,
put newly-uploaded packages in a holding area until the tests pass and not
allow them into the archive until they do.

Breakage bugs in unstable are very valuable, but they also represent
*breakage* and take someone's time and attention to write up.  Anything we
can catch on an automated basis saves precious human resources.

>> These are trivial, even simplistic tests. However, if they pass, we
>> know that at least the basic, fundamental things in the system are not
>> horribly broken: networking, system administration, and the software
>> that is meant to start in that reference installation. Furthermore, we
>> know that the debian-installer works. That's a good foundation for
>> further hacking.

> Maybe the installer is less daily tested but did we already have a major
> bug (one that does not pass the test you described) gone unnoticed?

We've had system-breaking bugs in core packages like libc6 enter the
archive in the past.  They don't make it through to testing, no, but they
do take up people's time and attention in unstable, and it's all more
resources that we can't spend on other things that are more useful.  And,
over time, the tests can become more sophisticated.  That's the great part
about building a testing framework: once you have a good framework in
place, it becomes much easier to add more tests, and you can build
something like Lintian (which is now quite extensive) from a modest
beginning.

-- 
Russ Allbery (r...@debian.org)   


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/8761yrn9sf@windlord.stanford.edu

Re: Debian development and release: always releasable (essay)

[Paul Wise]

On Fri, May 10, 2013 at 3:49 AM, Lars Wirzenius wrote:

> * An attitude change: we decide that releases are important, and that
>   they're the job of the entire project, not just the release team.

I already believe that. I would find it quite surprising if people
actually believe that releases are solely the job of the release team.
Do you have any data about what proportion of Debian contributors
don't believe that?

Folks obviously care about stable to varying levels though, some
probably don't even use stable.

> * Keep testing free of RC bugs.

I keep packages I (co-)maintain free of RC bugs.

> * We should use automatic testing much more extensively, to find
>   problems as early as possible.

I already do pay attention to that. I look at PTS pages before doing
uploads and run a bunch of automated tests before uploading or sending
a package review on debian-mentors. I also try to remember check sites
with QA/etc info that aren't yet integrated into the PTS (like the
derivatives patches).

http://wiki.debian.org/HowToPackageForDebian#Check_points_for_any_package

> * We should limit the number of packages we strongly care about for
>   a release.

I don't agree with this.

> * Remove RC buggy packages sooner rather than later. An RC buggy
>   package should be removed at soon as possible: when the bug

The release team already do this using a semi-automated method.

> We should have an explicit list of such reference installations
> and declare them as crucial for the release:

How about:

all the tasks
all the blends

> if they work, we can release, and if they don't, we can't.

How would you define "work"?

Presumably: no RC bugs, no piuparts issues?

> Use automatic testing extensively
> -
>
> We have some automatic testing tools specifically for Debian: lintian,
> piuparts, adequate, autopkgtest, and probably more. We should use
> these much more extensively, and let them guide the migration of
> packages into testing.

Some more automatic tests folks might like to run:

http://wiki.debian.org/HowToPackageForDebian#Check_points_for_any_package

> Imagine a continuous integration system for Debian: for every new
> package upload to unstable, it builds and tests all the reference
> installations.  If all builds succeed, and all tests pass, the package
> can be moved into testing at once. When you, a developer, upload a
> new package, you get notified about test results, and testing migration,
> within minutes.

Different tests are more important than others. I don't think every
test is important enough to block testing migration. The only tests I
can think of that should do that are puiparts failures.

Thanks for your thoughts, hopefully the release team will be willing
to adopt some of them, especially the puiparts failures one.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAKTje6HWdxgFjDJapAc_+eUjmC+U6X=N5q+_ffqUnreR-=z...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Russ Allbery]

Paul Wise  writes:
> On Fri, May 10, 2013 at 3:49 AM, Lars Wirzenius wrote:

>> * Keep testing free of RC bugs.

> I keep packages I (co-)maintain free of RC bugs.

The point isn't what individual developers do, particularly developers who
are extremely well-engaged with the project.  The point is to find ways to
do this at another level up.  Obviously, given the number of RC bugs that
we had to fix *after* the freeze, this isn't already being done at the
level required for a timely release process.  I don't think we can solve
that problem by saying "well, developers really *should*."

>> * We should limit the number of packages we strongly care about for
>>   a release.

> I don't agree with this.

Care to expand the thought?

>> * Remove RC buggy packages sooner rather than later. An RC buggy
>>   package should be removed at soon as possible: when the bug

> The release team already do this using a semi-automated method.

But by and large they only do this on a large scale during the freeze, at
which point, in a way, it's too late.  We've already built a huge backlog
of work, and everyone is anxious to release.  I think we should be doing
this continuously during the release and much more aggressively than we've
been willing to do in the past.

I suspect, though, that mini-freezes whenever the RC bug count rises above
a certain level will be as effective or more so, since I know that package
removal very quickly involves deeply tangled dependencies, and one has to
sometimes remove vast swathes of packages to remove a particular RC-buggy
package.

>> We should have an explicit list of such reference installations and
>> declare them as crucial for the release:

> How about:

> all the tasks
> all the blends

That's certainly a good start, although I think it's worth asking the
blends maintainers whether all of the packages they include are
release-critical.  I don't think it captures all of the interesting use
cases, though; there are common patterns that we want Debian to support
that aren't captured in a task or a blend.

>> if they work, we can release, and if they don't, we can't.

> How would you define "work"?

> Presumably: no RC bugs, no piuparts issues?

I would limit it to just no RC bugs (and therefore no test failures where
we're pretty sure that test failure would indicate an RC bug).  If we feel
that piuparts is testing things that should be RC, that would certainly
include piuparts.

Bear in mind, however, that the core problem is that we don't keep testing
sufficiently free of RC bugs to be able to release in a timely fashion
after a freeze.  That means we're not dealing well with the bugs we
already have, so I would be a bit hestitant to create new classes of RC
bugs until we have that situation under control.  While looking for new
classes of bugs is something that we should always be open to, I think the
most important step is to try to catch the bugs we were already catching
earlier in the process and to be more aggressive about dealing with them.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87vc6qrh7k@windlord.stanford.edu

Re: Debian development and release: always releasable (essay)

[Paul Wise]

On Fri, May 10, 2013 at 11:24 PM, Russ Allbery wrote:

> The point isn't what individual developers do, particularly developers who
> are extremely well-engaged with the project.  The point is to find ways to
> do this at another level up.  Obviously, given the number of RC bugs that
> we had to fix *after* the freeze, this isn't already being done at the
> level required for a timely release process.  I don't think we can solve
> that problem by saying "well, developers really *should*."

The problem this thread is trying to solve essentially comes down to
"people don't do enough work and we want them to do more". There are
various factors at play here; time, motivation, demotivation,
knowledge, confidence and probably more.

> Care to expand the thought?

The diversity of software in Debian is an advantage, I would prefer
that we strongly care about all packages for the release. I expect
that very few people in Debian and none/few of the QA or release team
members in recent years who share my opinion here though, I accept
that and avoid expressing this opinion in general. I also acknowledge
that we just don't have enough people to properly maintain all the
packages in Debian which means that my opinion is also unrealistic.

> But by and large they only do this on a large scale during the freeze, at
> which point, in a way, it's too late.  We've already built a huge backlog
> of work, and everyone is anxious to release.  I think we should be doing
> this continuously during the release and much more aggressively than we've
> been willing to do in the past.

Agreed, I've been poking various RT folks about this over the years,
mainly I suggested completely automated removals for all packages.
That was a bit extreme though since core packages like
apt/toolchain/etc can have RC bugs open for a long time.

> I suspect, though, that mini-freezes whenever the RC bug count rises above
> a certain level will be as effective or more so, since I know that package
> removal very quickly involves deeply tangled dependencies, and one has to
> sometimes remove vast swathes of packages to remove a particular RC-buggy
> package.

I think this is a much better idea than removals.

> That's certainly a good start, although I think it's worth asking the
> blends maintainers whether all of the packages they include are
> release-critical.  I don't think it captures all of the interesting use
> cases, though; there are common patterns that we want Debian to support
> that aren't captured in a task or a blend.

Agreed. Do you have any example use-cases that should block releases
but aren't in blends or tasks? Perhaps we need to start some new
blends or add new tasks.

> I would limit it to just no RC bugs (and therefore no test failures where
> we're pretty sure that test failure would indicate an RC bug).  If we feel
> that piuparts is testing things that should be RC, that would certainly
> include piuparts.

piuparts folks generally already file RC bugs as they are able.

> Bear in mind, however, that the core problem is that we don't keep testing
> sufficiently free of RC bugs to be able to release in a timely fashion
> after a freeze.  That means we're not dealing well with the bugs we
> already have, so I would be a bit hestitant to create new classes of RC
> bugs until we have that situation under control.  While looking for new
> classes of bugs is something that we should always be open to, I think the
> most important step is to try to catch the bugs we were already catching
> earlier in the process and to be more aggressive about dealing with them.

Ack.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/caktje6glnfpt4xftsdekv1rv3_ze+9oktf18xbzwjz54z6g...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Russ Allbery]

Paul Wise  writes:
> On Fri, May 10, 2013 at 11:24 PM, Russ Allbery wrote:

>> The point isn't what individual developers do, particularly developers
>> who are extremely well-engaged with the project.  The point is to find
>> ways to do this at another level up.  Obviously, given the number of RC
>> bugs that we had to fix *after* the freeze, this isn't already being
>> done at the level required for a timely release process.  I don't think
>> we can solve that problem by saying "well, developers really *should*."

> The problem this thread is trying to solve essentially comes down to
> "people don't do enough work and we want them to do more".  There are
> various factors at play here; time, motivation, demotivation, knowledge,
> confidence and probably more.

Well, sort of -- we are also proposing an alternative: not shipping those
packages with the next stable release, but making them available to users
in some other way.  So, people don't have to do more work, but instead of
then freezing for months so that everyone else in the project fixes those
packages, we pull them from stable early.  If keeping them in stable is
more work than anyone is willing to do, then they won't be in the release,
and we'll know that much earlier on.  Furthermore, what does need to be
done to keep them in stable will be clearer.

In other words, the proposal is an attempt to fail faster, instead of
accumulating work (which grows larger with each release of Debian) until
the freeze and then trying to fix it all then, across the entire project.

I see below that you generally agree with that part of the proposal
anyway, though.  :)

> The diversity of software in Debian is an advantage, I would prefer that
> we strongly care about all packages for the release. I expect that very
> few people in Debian and none/few of the QA or release team members in
> recent years who share my opinion here though, I accept that and avoid
> expressing this opinion in general. I also acknowledge that we just
> don't have enough people to properly maintain all the packages in Debian
> which means that my opinion is also unrealistic.

I do agree with this opinion (the breadth of Debian is why I'm using it
instead of Red Hat for work), but it's the second part that I'm worried
about.  I do think that some additional clarity and failing faster in
pulling things out of testing sooner will help people focus their efforts
and help with making tradeoff decisions.

> Agreed. Do you have any example use-cases that should block releases but
> aren't in blends or tasks? Perhaps we need to start some new blends or
> add new tasks.

I would need to do some research, since I'm not personally familiar with
everything that's in a blend or a task at the moment.  Just off the top of
my head, though, to pick an area of personal expertise, I don't think
there's an existing blend or task for a Kerberos KDC or, more generally,
an authentication and identity management infrastructure.  That's one that
I'd be willing to tackle creating a package list for if we went this
route.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87li7m2q5u@windlord.stanford.edu

Re: Debian development and release: always releasable (essay)

[Paul Wise]

On Sat, May 11, 2013 at 10:42 AM, Russ Allbery wrote:

> I would need to do some research, since I'm not personally familiar with
> everything that's in a blend or a task at the moment.  Just off the top of
> my head, though, to pick an area of personal expertise, I don't think
> there's an existing blend or task for a Kerberos KDC or, more generally,
> an authentication and identity management infrastructure.  That's one that
> I'd be willing to tackle creating a package list for if we went this
> route.

Sounds like something that would be suitable for a task package, I'm
not sure if d-i folks will be happy about including more tasks though.
This is a conversation we need to have anyway, blends folks have been
talking about ways to integrate blends with d-i (and Debian in
general), so we need better ways to expose blends and tasks in d-i
that cope with the large amount of diversity of usage we have in
Debian.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAKTje6EY7=2pt_w2zcannnajs+hpljhfvha1y+b-ipgrwpj...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Helmut Grohne]

On Thu, May 09, 2013 at 08:49:51PM +0100, Lars Wirzenius wrote:
> * Remove RC buggy packages sooner rather than later. An RC buggy
>   package should be removed at soon as possible: when the bug
>   is identified, allow a bit of time for the bug to be verified
>   (was it actually an RC bug?), but after that, remove the package
>   from testing, preferably automatically.  If the package has
>   reverse dependencies, remove those as well. This keeps testing
>   releasable. The removed package can and will re-enter testing once
>   it gets fixed.

The value of package removal, is to clarify that the release will not
wait for this package. I fear that having a flaky testing distribution
with packages frequently removed and added again would cause problems on
its own merits. On the other hand maybe removal is not the thing we are
aiming for? I am aware of at least one case where a removal notice (the
actual removal never happened) caused a third party to fix a package,
because maintaining it himself would have been more work. Maybe making
those removal notices more visible would help getting further people
involved? What about feeding removal notices to planet.d.o? And yeah,
more of them should help as well. To that end improving the rc-alert
output (and making this utility more visible to end users) could improve
the situation as well.

>   To reduce the sting of optional packages missing the release, we
>   should consider whether we're willing to introduce new packages
>   in stable point releases.  Obviously, only packages that have
>   no new dependencies could be introduced that way, so things that
>   require newer versions of the packages already in stable would not
>   be eligible.  But it means that if a package was in the previous
>   stable but missed the current stable due to unresolved issues at
>   the time of the releease, we could still get it back in and it
>   wouldn't have to wait another year or two.

This idea has a negative side effect. If my pet package is missing from
a stable release I am probably tempted to wait with upgrading, because
there is a chance for it to be reintroduced. This even applies if the
package does not end up being added due to the scarce availability of
time machines. The current policy of not extending a stable release
actually provides a benefit: clarity.

Helmut


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130511055427.ga19...@alf.mars

Re: Debian development and release: always releasable (essay)

[Andreas Tille]

Hi

On Fri, May 10, 2013 at 07:42:21PM -0700, Russ Allbery wrote:
> Paul Wise  writes:
> > Agreed. Do you have any example use-cases that should block releases but
> > aren't in blends or tasks? Perhaps we need to start some new blends or
> > add new tasks.
> 
> I would need to do some research, since I'm not personally familiar with
> everything that's in a blend or a task at the moment.  Just off the top of
> my head, though, to pick an area of personal expertise,

>From my point of view one main point in Blends is that you can (if you
care enough) assemble some manpower behind a certain set of packages.
For Debian Med I have some proof of this statement which are those ten
developers that inserted "yes" in the column "DD because Debian Med
exists" in the developers questionaire I did[1].  In other words:
Because there is a Blend we do have the people working on its packages.

I admit that running a Blend is also extra work to explain it to people
but extra manpower of 10 people (which is one per year of the projects
life time) was worth the effort.

> I don't think
> there's an existing blend or task for a Kerberos KDC or, more generally,
> an authentication and identity management infrastructure.  That's one that
> I'd be willing to tackle creating a package list for if we went this
> route.

IMHO this could be kick-started from Debian Enterprise.

Kind regards

  Andreas.

[1] https://wiki.debian.org/DebianMed/Developers 

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130511060824.gc28...@an3as.eu

Re: Debian development and release: always releasable (essay)

[Tollef Fog Heen]

]] Paul Wise 

> Agreed. Do you have any example use-cases that should block releases
> but aren't in blends or tasks? Perhaps we need to start some new
> blends or add new tasks.

(Where can I look up what tasks or blends use a given package?)

I don't know if, say, puppet is in a task or a blend, but it's one of
the packages where I'm fairly sure that lots of people (DSA included)
would be less than impressed if it was missing from a stable release.
I'd also not be surprised if it wasn't in an existing blend or task.

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87zjw12a4d@xoog.err.no

Re: Debian development and release: always releasable (essay)

[Paul Wise]

On Sat, May 11, 2013 at 4:28 PM, Tollef Fog Heen wrote:

> (Where can I look up what tasks or blends use a given package?)

For the blends part, we plan to add that to the PTS (#703402). I
should extend that to tasks too I think.

Until then, use your favourite rdepends viewer, the aptitude curses
interface is mine.

> I don't know if, say, puppet is in a task or a blend, but it's one of
> the packages where I'm fairly sure that lots of people (DSA included)
> would be less than impressed if it was missing from a stable release.
> I'd also not be surprised if it wasn't in an existing blend or task.

aptitude says it doesn't have any reverse dependencies so it isn't in
any task/blend (both use metapackages).

Lucas created a script that displays a list of "important" packages,
puppet isn't on that either:

http://udd.debian.org/cgi-bin/important_packages.cgi

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAKTje6Ep2=t1qqvqof9ycgpkk-puipmn7gnirod7f-wk0xj...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Andrei POPESCU]

On Jo, 09 mai 13, 20:49:51, Lars Wirzenius wrote:
> 
> The fundamental change is to start keeping our "testing" branch
> as close to releasable as possible, at all times. 

It's probably obvious for debian-devel readers, but I think it is worth 
saying it out loud: this would also give us CUT/rolling/etc. for free.

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser
Offtopic discussions among Debian users and developers:
http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic


signature.asc
Description: Digital signature

Re: Debian development and release: always releasable (essay)

[Johannes Schauer]

Hi,

Quoting Paul Wise (2013-05-11 10:40:18)
> Lucas created a script that displays a list of "important" packages, puppet
> isn't on that either:
> 
> http://udd.debian.org/cgi-bin/important_packages.cgi

Not surprising as the algorithm (from what can be read in the comments)
executes what we call build_closure algorithm in this paper [1].  During
bootstrapping we execute the same algorithm with the only difference that we do
not pull in source packages that only contribute arch:all packages (for obvious
reasons).

While we also recognized this selection of packages as important from a
bootstrapping point of view (as it contains the largest strongly connected
component in the dependency graph) it is not surprising that puppet is not in
it. Instead, puppet is just a leaf package in the dependency graph.

So while the set of source packages found by the build_closure algorithm should
certainly be part of the "important" packages, this also shows an observation
that we made during dependency graph analysis: The syntax of the dependency
graph is not enough to make semantic conclusions of the contained packages.

So instead, the important packages should be the union of:

 - the result of the build_closure algorithm
 - the transitive dependencies of all tasks and all blends
 - ???

Maybe the puppet question can just be solved by introducing an openstack task?
Adding new tasks could help codify the set of features that are deemed
"important".

cheers, josch

[1] http://mancoosi.org/~abate/bootstrapping-software-distributions


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130511093758.32278.6057@hoothoot

Re: Debian development and release: always releasable (essay)

[Thomas Goirand]

Hi Lars,

I do like a lot the idea of running things like piuparts and such at
upload time.
If you have time to work this out with the FTP masters, that would be a very
good idea IMO, and I warmly welcome you to do that. However...

On 05/10/2013 03:49 AM, Lars Wirzenius wrote:
> Tests for running reference installation might include the following:
>
> * Basic networking setup works: System responds to ping from the outside.
> * Mail server responds appropriate on the SMTP, submission, IMAPS, and POPS
>   ports.
> * LAMP server responds on the HTTP and HTTPS ports.
> * A desktop system that automatically logs in a test user has the right
>   processes running, and can start some common applications.
> * In each case, it's possible to log in remotely with ssh and run
>   "sudo apt-get install hello".

These wont help. They were not the RC bugs we had during the release cycle.
I believe for example, that Apache, MySQL, and PHP were quite well
maintained
and didn't suffer from RC bugs that stayed for a long time. I didn't see
bugs
for Exim, Postfix, ssh or Bind either. We had problems with Dovecot though,
but they were well identified, and having tests against it wouldn't have
help
in any ways.

If you want to find out which tests would help, you would have to conduct
a better analysis of the problems we had during the freeze, IMO.

Thomas


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/518e1e35.1070...@debian.org

Re: Debian development and release: always releasable (essay)

[Tollef Fog Heen]

]] Johannes Schauer 

> Maybe the puppet question can just be solved by introducing an openstack task?

puppet isn't important because it's used by/part of openstack (which I
don't think it is?)  It's important because it's a tool lots of
sysadmins use to automate their infrastructures.  Also, it's generally a
bigger problem if something goes away than if it was never shipped.
Going away means leaving users hanging.  Not ever having something just
means, well, we didn't have it and those who wanted it had to install it
from elsewhere.

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87vc6p23fc@xoog.err.no

Re: Debian development and release: always releasable (essay)

[Antonio Terceiro]

On Sat, May 11, 2013 at 06:32:21PM +0800, Thomas Goirand wrote:
> Hi Lars,
> 
> I do like a lot the idea of running things like piuparts and such at
> upload time.
> If you have time to work this out with the FTP masters, that would be a very
> good idea IMO, and I warmly welcome you to do that. However...
> 
> On 05/10/2013 03:49 AM, Lars Wirzenius wrote:
> > Tests for running reference installation might include the following:
> >
> > * Basic networking setup works: System responds to ping from the outside.
> > * Mail server responds appropriate on the SMTP, submission, IMAPS, and POPS
> >   ports.
> > * LAMP server responds on the HTTP and HTTPS ports.
> > * A desktop system that automatically logs in a test user has the right
> >   processes running, and can start some common applications.
> > * In each case, it's possible to log in remotely with ssh and run
> >   "sudo apt-get install hello".
> 
> These wont help. They were not the RC bugs we had during the release
> cycle.  I believe for example, that Apache, MySQL, and PHP were quite
> well maintained and didn't suffer from RC bugs that stayed for a long
> time. I didn't see bugs for Exim, Postfix, ssh or Bind either. We had
> problems with Dovecot though, but they were well identified, and
> having tests against it wouldn't have help in any ways.
> 
> If you want to find out which tests would help, you would have to conduct
> a better analysis of the problems we had during the freeze, IMO.

You can't assume that just because something works today, it will work
forever. Having such tests in place will help to automatically catch
regressions if they arise. If they don't, fine, but you never know.

While I agree with you that there are a lot more we can test, I don't
think it's useless to include tests for stuff we know is working. Even
"trivial" tests have their value with so many moving parts.

-- 
Antonio Terceiro 


signature.asc
Description: Digital signature

Re: Debian development and release: always releasable (essay)

[Mehdi Dogguy]

Le 2013-05-10 17:24, Russ Allbery a écrit :


But by and large they only do this on a large scale during the freeze, 
at
which point, in a way, it's too late.  We've already built a huge 
backlog
of work, and everyone is anxious to release.  I think we should be 
doing
this continuously during the release and much more aggressively than 
we've

been willing to do in the past.



The idea was to do it even during the development cycle. It was mainly 
an

issue of manpower that we did it only during the freeze, I think.

I'm fairly confident that we will be able to do it during the 
development

cycle this time.

--
Mehdi


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/a9216453b89dd7358222efbc2734d...@dogguy.org

Re: Debian development and release: always releasable (essay)

[Neil Williams]

On Sat, 11 May 2013 10:31:09 +0800
Paul Wise  wrote:

> On Fri, May 10, 2013 at 11:24 PM, Russ Allbery wrote:
> 
> > The point isn't what individual developers do, particularly developers who
> > are extremely well-engaged with the project.  The point is to find ways to
> > do this at another level up.  Obviously, given the number of RC bugs that
> > we had to fix *after* the freeze, this isn't already being done at the
> > level required for a timely release process.  I don't think we can solve
> > that problem by saying "well, developers really *should*."
> 
> The problem this thread is trying to solve essentially comes down to
> "people don't do enough work and we want them to do more". There are
> various factors at play here; time, motivation, demotivation,
> knowledge, confidence and probably more.

The problem we should be trying to solve is "people are not getting the
work done, let's break down the problems and make working on them
easier or the solutions more obvious".

i.e.
0: Improve detection (more frequent and more varied QA)

1: Improve triage (more data, more criteria / tags / meta data)

2: Improve fixability (more conformance across packages so that
understanding / fixing the package is trivial, ensure packages always
build twice cleanly, mandate consistent patch handling)

3: Improve automation (remove stalled packages, alert maintainers of
reverse dependencies about problems other than wnpp issues.)

> The diversity of software in Debian is an advantage, I would prefer
> that we strongly care about all packages for the release.

Modulo dropping packages from Debian when it is obvious that actually
nobody cares sufficiently about those specific packages. Make the
archive consist of packages at least someone cares about, then we have
an archive which we, collectively, care about releasing.

> I expect
> that very few people in Debian and none/few of the QA or release team
> members in recent years who share my opinion here though, I accept
> that and avoid expressing this opinion in general. I also acknowledge
> that we just don't have enough people to properly maintain all the
> packages in Debian which means that my opinion is also unrealistic.

So drop more packages, get down to a realistic set.
 
> Agreed, I've been poking various RT folks about this over the years,
> mainly I suggested completely automated removals for all packages.
> That was a bit extreme though since core packages like
> apt/toolchain/etc can have RC bugs open for a long time.

Automated removals of leaf packages (or packages where the only reverse
dependencies are themselves leaf / under maintained) still helps the
release as a whole. It's easy to implement a check that removals are
considered only when the entire dependency chain to be removed is less
than N levels deep or involves less than X source packages.

> > I suspect, though, that mini-freezes whenever the RC bug count rises above
> > a certain level will be as effective or more so, since I know that package
> > removal very quickly involves deeply tangled dependencies, and one has to
> > sometimes remove vast swathes of packages to remove a particular RC-buggy
> > package.
> 
> I think this is a much better idea than removals.

(Doesn't discount removals where dependencies are less problematic.)
 
Existing transition support could be extended to implement a
mini-freeze on a particular chain of packages. The problem, as ever,
will be imposing such a mini-freeze and ensuring that it is maintained
and respected. Making that "policing" role part of the release team
workload is not going to help anyone.

> > Bear in mind, however, that the core problem is that we don't keep testing
> > sufficiently free of RC bugs to be able to release in a timely fashion
> > after a freeze.  That means we're not dealing well with the bugs we
> > already have, so I would be a bit hestitant to create new classes of RC
> > bugs until we have that situation under control.

Helping people sift the number of RC bugs to more easily locate the
ones which can be fixed and which ones to ignore will also help bring
the RC count, as a whole, under control. New meta data about those bugs
will help people get the list itself under control.

>  While looking for new
> > classes of bugs is something that we should always be open to, I think the
> > most important step is to try to catch the bugs we were already catching
> > earlier in the process and to be more aggressive about dealing with them.
> 
> Ack.

Ack.

-- 


Neil Williams
=
http://www.linux.codehelp.co.uk/



pgpgf7vOPeyZb.pgp
Description: PGP signature

Re: Debian development and release: always releasable (essay)

[Paul Wise]

On Sun, May 12, 2013 at 5:22 PM, Neil Williams wrote:

> The problem we should be trying to solve is "people are not getting the
> work done, let's break down the problems and make working on them
> easier or the solutions more obvious".

Sounds good to me.

> Modulo dropping packages from Debian when it is obvious that actually
> nobody cares sufficiently about those specific packages. Make the
> archive consist of packages at least someone cares about, then we have
> an archive which we, collectively, care about releasing.

I don't think that is the right conclusion. A better one might be that
the intersection between people who have time, care about the software
and have the skills to maintain the software is low. There may be
users who use the software every day, have some spare time but do not
have the skills to develop or package software. There may be
developers who have a low amount of time they can commit to the
package.

> So drop more packages, get down to a realistic set.

I don't think that is the right conclusion either. A better one might
be to spend time recruiting more developers or pinging existing
developers who have expressed interest in the past and folks involved
in upstream projects. I've learned over the years that removals
usually happen without the latter two and that Debian isn't
particularly good at the former.

> Automated removals of leaf packages (or packages where the only reverse
> dependencies are themselves leaf / under maintained) still helps the
> release as a whole. It's easy to implement a check that removals are
> considered only when the entire dependency chain to be removed is less
> than N levels deep or involves less than X source packages.

That would be good, I think Neils has been working on this.

> Existing transition support could be extended to implement a
> mini-freeze on a particular chain of packages. The problem, as ever,
> will be imposing such a mini-freeze and ensuring that it is maintained
> and respected. Making that "policing" role part of the release team
> workload is not going to help anyone.

The release team already does "policing"; they complain when people
start uncoordinated transitions or upload unsuitable changes during
release freezes.

> Helping people sift the number of RC bugs to more easily locate the
> ones which can be fixed and which ones to ignore will also help bring
> the RC count, as a whole, under control. New meta data about those bugs
> will help people get the list itself under control.

That would be nice.

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAKTje6FyZd23ZrCdHQ=uzoqpygb1veh_h+aufk5pxbdzd1j...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Neil Williams]

On Sun, 12 May 2013 17:43:57 +0800
Paul Wise  wrote:

> On Sun, May 12, 2013 at 5:22 PM, Neil Williams wrote:
> 
> > The problem we should be trying to solve is "people are not getting the
> > work done, let's break down the problems and make working on them
> > easier or the solutions more obvious".
> 
> Sounds good to me.
> 
> > Modulo dropping packages from Debian when it is obvious that actually
> > nobody cares sufficiently about those specific packages. Make the
> > archive consist of packages at least someone cares about, then we have
> > an archive which we, collectively, care about releasing.
> 
> I don't think that is the right conclusion. A better one might be that
> the intersection between people who have time, care about the software
> and have the skills to maintain the software is low.

There remain packages where that is not just low but absent. Those
packages are candidates for removal, subject to the dependency checks
below which we appear to agree upon.

Where removals are concerned, there is always the check of N levels
deep and X packages involved. `rm *` doesn't actually make for a good
release.

> I don't think that is the right conclusion either. A better one might
> be to spend time recruiting more developers or pinging existing
> developers who have expressed interest in the past and folks involved
> in upstream projects. I've learned over the years that removals
> usually happen without the latter two and that Debian isn't
> particularly good at the former.
> 
> > Automated removals of leaf packages (or packages where the only reverse
> > dependencies are themselves leaf / under maintained) still helps the
> > release as a whole. It's easy to implement a check that removals are
> > considered only when the entire dependency chain to be removed is less
> > than N levels deep or involves less than X source packages.
> 
> That would be good, I think Neils has been working on this.
> 
> > Existing transition support could be extended to implement a
> > mini-freeze on a particular chain of packages. The problem, as ever,
> > will be imposing such a mini-freeze and ensuring that it is maintained
> > and respected. Making that "policing" role part of the release team
> > workload is not going to help anyone.
> 
> The release team already does "policing"; they complain when people
> start uncoordinated transitions or upload unsuitable changes during
> release freezes.

Exactly, the release team is at full capacity doing that, therefore we
cannot expect the release team to do more of it. They rightly complain
about uncoordinated transitions because that makes their work harder and
slower. Although the transition mechanism could help with mini-freezes,
policing those freezes needs to be done by someone else or the
mini-freeze will be chaotic. The release team have enough to do
already, but learn from their methods if we are to create another area
where a "police" role is required. This is the weakpoint of a
mini-freeze idea, I don't think a mini-freeze will be observed and we
don't have a team with sufficient time to take the "policing" role.
Saying "no" in a volunteer project is never an easy thing to do, even
when it is absolutely the right thing to do for the benefit of the
project as a whole.

-- 


Neil Williams
=
http://www.linux.codehelp.co.uk/



pgpHweXHPx2im.pgp
Description: PGP signature

Re: Debian development and release: always releasable (essay)

[Lucas Nussbaum]

On 11/05/13 at 11:37 +0200, Johannes Schauer wrote:
> Hi,
> 
> Quoting Paul Wise (2013-05-11 10:40:18)
> > Lucas created a script that displays a list of "important" packages, puppet
> > isn't on that either:
> > 
> > http://udd.debian.org/cgi-bin/important_packages.cgi
> 
> Not surprising as the algorithm (from what can be read in the comments)
> executes what we call build_closure algorithm in this paper [1].  During
> bootstrapping we execute the same algorithm with the only difference that we 
> do
> not pull in source packages that only contribute arch:all packages (for 
> obvious
> reasons).
> 
> While we also recognized this selection of packages as important from a
> bootstrapping point of view (as it contains the largest strongly connected
> component in the dependency graph) it is not surprising that puppet is not in
> it. Instead, puppet is just a leaf package in the dependency graph.
> 
> So while the set of source packages found by the build_closure algorithm 
> should
> certainly be part of the "important" packages, this also shows an observation
> that we made during dependency graph analysis: The syntax of the dependency
> graph is not enough to make semantic conclusions of the contained packages.
> 
> So instead, the important packages should be the union of:
> 
>  - the result of the build_closure algorithm
>  - the transitive dependencies of all tasks and all blends
>  - ???

The algorithm also includes "popular" (as in popularity-contest)
packages in the list, not just the ones required to bootstrap Debian.

But generally, I agree that the list of packages should probably be
extended using more criterias, such as inclusion in tasks/blends, or
importance for Debian infrastructure.

Lucas


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130512102310.ga18...@xanadu.blop.info

Re: Debian development and release: always releasable (essay)

[Thomas Goirand]

On 05/11/2013 06:53 PM, Tollef Fog Heen wrote:
> ]] Johannes Schauer 
>
>> Maybe the puppet question can just be solved by introducing an openstack 
>> task?
> puppet isn't important because it's used by/part of openstack (which I
> don't think it is?)
Puppet isn't used by OpenStack. Though it's used by operators to setup
some OpenStack based cloud.

I don't think there's the need of an OpenStack task. There is already
"openstack-meta-packages" that I created, which helps setting-up
either a "controler" node, or a compute node. Having it used as part
of a custom CD would be nice though (I've been looking at using
live-build to do that, though I failed to find enough time to do it yet).

I know very little about blends. Maybe it is something I should have
a look into? Would it be a good idea to use a task instead of my
openstack-meta-packages?

Cheers,

Thomas


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/518f7ed5.1000...@debian.org

Re: Debian development and release: always releasable (essay)

[Thomas Goirand]

On 05/12/2013 02:03 AM, Antonio Terceiro wrote:
> You can't assume that just because something works today, it will work
> forever.

True, though it's been at least 2 release cycle (maybe 3?) that this
set of packages were maintained quite well. I don't remember
seeing complains or bad bugs. Do you?

> Having such tests in place will help to automatically catch
> regressions if they arise.

My point is: I don't think so, and I think we are better off focusing
on other (real?) problems.

Thomas


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/518f8117.7050...@debian.org

Re: Debian development and release: always releasable (essay)

[Michael Gilbert]

On Thu, May 9, 2013 at 3:49 PM, Lars Wirzenius wrote:
> The wheezy freeze has been much too long. At ten months, it's four
> months longer than what we've gotten used to in several previous
> releases. Had we managed to keep the freeze at six months, it would
> still have been too long. I believe there is something wrong in how
> we develop Debian, and how we do releases, and that by fixing them,
> we can have much shorter releases, with an increase in their quality.

I disagree that there is something fundamentally wrong with how
development is done.  The primary problems with this cycle were that
there were something like 400 or 500 extra rc bugs due to a concerted
effort to report all serious issues found via piuparts, and then the
existential problem of not enough rc squashers, which in and of itself
is not all that rewarding.  You address the former with the more
automated testing comment below.  The latter could possibly be
addressed by bring in more DDs, and that means doing better with
-mentors.

Anyway, those are the two problems that I believe need focus.

> We should aim for a short freeze, perhaps as short as two weeks,
> and certainly not longer than two months. This would remove the
> frustration, and fix the other problems related to a long freeze.
> However, to achieve a short freeze, we need to change how develop
> Debian.

A certain freeze length is inevitable simply because it takes a
certain time minimum for people to test and find all of the
significant compatibility issues with the set of software chosen to be
a part of the release.  Debian's high quality standard cannot be met
with a two-week testing period.  Somewhere between 3 and 6 months is
much more reasonable.

> The fundamental change is to start keeping our "testing" branch
> as close to releasable as possible, at all times. For individual
> projects, this corresponds to keeping the master or trunk branch
> in version control ready to be released. Practitioners of agile
> development models, for example, do this quite successfully, by
> applying continuous integration, automatic testing, and by having
> a development culture that if there's a severe bug in master,
> fixing that gets highest priority.

There are simply too many rc bugs all the time.  Jessie is already
over 500 after one week of development:
http://bugs.debian.org/release-critical

That, I think is the real problem.  How did testing go from a minimal
(70-ish) rc bug wheezy release to over 500 in one week?  Why didn't
britney prevent the migration of all those rc-buggy packages?

> Imagine a continuous integration system for Debian: for every new
> package upload to unstable, it builds and tests all the reference
> installations.  If all builds succeed, and all tests pass, the package
> can be moved into testing at once. When you, a developer, upload a
> new package, you get notified about test results, and testing migration,
> within minutes.

I am in total agreement.  This would be wonderful.

Best wishes,
Mike


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CANTw=MMbJe=Hp4NP0a374X7PeLTQf=eqtruujeph-asyzwc...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Russ Allbery]

Michael Gilbert  writes:

> I disagree that there is something fundamentally wrong with how
> development is done.  The primary problems with this cycle were that
> there were something like 400 or 500 extra rc bugs due to a concerted
> effort to report all serious issues found via piuparts, and then the
> existential problem of not enough rc squashers, which in and of itself
> is not all that rewarding.

This is what we say every release, for various values of "piuparts"
(archive-wide rebuilds, license audits, etc.).  And then every release we
have the same problem.  Let's be sure that we're not just trying the same
thing over and over again and expecting different results.

> A certain freeze length is inevitable simply because it takes a certain
> time minimum for people to test and find all of the significant
> compatibility issues with the set of software chosen to be a part of the
> release.  Debian's high quality standard cannot be met with a two-week
> testing period.  Somewhere between 3 and 6 months is much more
> reasonable.

Six months would be an improvement, but I think it's still much too long,
long enough to have negative distortive effects.  I think three months is
a better target to aim for.  (That said, if we could reliably get the
release freeze down to six months, that would certainly be a worthwhile
improvement.)

> There are simply too many rc bugs all the time.  Jessie is already over
> 500 after one week of development:
> http://bugs.debian.org/release-critical

Right.  Which is why we should immediately (for definitions of immediately
that involve the release team taking a much-deserved break, but not for
definitions of immediately that mean "six months from now") freeze testing
again until we're back down under 50-100 RC bugs, whether via fixes and
transitions or whether by kicking out a bunch of packages.  Now is also
the ideal time to kick packages out of testing so that people are aware
that they're in trouble very early in the release cycle.

We always have this surge immediately after the release, but we don't
stomp on it immediately.  We therefore get up to 1000 RC bugs before we
know it, and never get back on top of them without a long and horribly
painful freeze.

> That, I think is the real problem.  How did testing go from a minimal
> (70-ish) rc bug wheezy release to over 500 in one week?  Why didn't
> britney prevent the migration of all those rc-buggy packages?

Because they're not from migrations.  They're from transitions.  They're
all the "this is going to break with a new version of libc6" and the like
bugs that were filed before the release at priority important and were
mass-upgraded after the release.

This is fine and normal, but it means that we should not now be diving
into all new upstream, all the time development immediately.  We should
stop, take a breath, work through these transitions first, get that RC bug
count down to something releasonable again, and then tackle the next thing
that surges RC bug counts.

That's going to require some discipline.  We know from long experience
with the release process that we can use the bully pulpit until we're blue
in the face, but this is a huge project with a lot of developers, many of
whom just don't read mailing lists.  People will keep uploading unless we
put something in place that actually prevents them from doing so, such as
freezing testing migration right now except for pre-arranged transitions
and RC bug fixes.

(It's quite possible that, to implement this properly without drowning the
release team in work, we'll need better tools to manage that sort of
freeze and migration process.)

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87sj1sdpvs@windlord.stanford.edu

Re: Debian development and release: always releasable (essay)

[Michael Gilbert]

On Sun, May 12, 2013 at 2:17 PM, Russ Allbery wrote:
> Michael Gilbert writes:
>
>> I disagree that there is something fundamentally wrong with how
>> development is done.  The primary problems with this cycle were that
>> there were something like 400 or 500 extra rc bugs due to a concerted
>> effort to report all serious issues found via piuparts, and then the
>> existential problem of not enough rc squashers, which in and of itself
>> is not all that rewarding.
>
> This is what we say every release, for various values of "piuparts"
> (archive-wide rebuilds, license audits, etc.).  And then every release we
> have the same problem.  Let's be sure that we're not just trying the same
> thing over and over again and expecting different results.

I am not saying that is going away with jessie.  In fact, I am totally
expecting another 400-500 serious or more piuparts bugs.  The reason
that I mention that as the biggest issue is that for one the concerted
effort to report all of those bugs is new, and could possibly be
identified as the catalyst for the +4 months compared to squeeze.

That is why I think the automated testing infrastructure needs to be
implement (as soon as possible) this cycle, in order to keep all
piuparts-broken packages from ever migrating to testing, resulting in
a later large amount of the rc count.

>> There are simply too many rc bugs all the time.  Jessie is already over
>> 500 after one week of development:
>> http://bugs.debian.org/release-critical
>
> Right.  Which is why we should immediately (for definitions of immediately
> that involve the release team taking a much-deserved break, but not for
> definitions of immediately that mean "six months from now") freeze testing
> again until we're back down under 50-100 RC bugs, whether via fixes and
> transitions or whether by kicking out a bunch of packages.

People just spent 10 months fixing issues in packages that were not
their own.  I don't think they want to be pulled away from the fun
stuff so quickly.

> Now is also
> the ideal time to kick packages out of testing so that people are aware
> that they're in trouble very early in the release cycle.

That would be good.

>> That, I think is the real problem.  How did testing go from a minimal
>> (70-ish) rc bug wheezy release to over 500 in one week?  Why didn't
>> britney prevent the migration of all those rc-buggy packages?
>
> Because they're not from migrations.  They're from transitions.  They're
> all the "this is going to break with a new version of libc6" and the like
> bugs that were filed before the release at priority important and were
> mass-upgraded after the release.

But those shouldn't affect testing yet, right?  All of that stuff
needs staging in unstable first.  Are bug filers not tagging their
reports correctly?  If so, that's quite misleading, and actually
should be quite easy although tedious to fix.

Best wishes,
Mike


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CANTw=MMMXsirPPYLbhSe3Nwf5wEkb=5cdt5As=j3dmfy+bc...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Russ Allbery]

Michael Gilbert  writes:
> On Sun, May 12, 2013 at 2:17 PM, Russ Allbery wrote:

>> Right.  Which is why we should immediately (for definitions of
>> immediately that involve the release team taking a much-deserved break,
>> but not for definitions of immediately that mean "six months from now")
>> freeze testing again until we're back down under 50-100 RC bugs,
>> whether via fixes and transitions or whether by kicking out a bunch of
>> packages.

> People just spent 10 months fixing issues in packages that were not
> their own.  I don't think they want to be pulled away from the fun
> stuff so quickly.

And that's why releases are so painful, at least IMO.

We have to break this cycle at some point or it will just get worse.  To
break the cycle, we're going to have to keep doing bug fixing after we're
exhausted of doing bug fixing so that we don't build up a huge backlog.

The good part is that if we actually can break that cycle, the freeze
will be much less painful and we won't be as sick of fixing bugs the next
time around.

Think of this in software development terms.  We're currently following a
development model where, immediately following a release, there's a nearly
complete free-for-all without much enforcement of bugs or regressions.  We
do that for a year, and then we try to stabilize the results.  Most free
software projects that used to follow this model (and there have been
quite a number of them) have had similar struggles with the extended
stabilization process this requires.  That's part of the shift towards
test-driven development, continuous integration, and constantly-usable
master branches.

>> Because they're not from migrations.  They're from transitions.
>> They're all the "this is going to break with a new version of libc6"
>> and the like bugs that were filed before the release at priority
>> important and were mass-upgraded after the release.

> But those shouldn't affect testing yet, right?  All of that stuff
> needs staging in unstable first.  Are bug filers not tagging their
> reports correctly?  If so, that's quite misleading, and actually
> should be quite easy although tedious to fix.

The bug affects the version of the package in testing.  I see what you're
saying, but I don't think this is something the BTS can represent.  And
those bugs *are* all release-critical: they have to be fixed before we can
release jessie, at least unless we're going to abort the transition, which
seems unlikely.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87txm855p7@windlord.stanford.edu

Re: Debian development and release: always releasable (essay)

[Michael Gilbert]

On Sun, May 12, 2013 at 4:00 PM, Russ Allbery wrote:
> Michael Gilbert writes:
>> On Sun, May 12, 2013 at 2:17 PM, Russ Allbery wrote:
>
>>> Right.  Which is why we should immediately (for definitions of
>>> immediately that involve the release team taking a much-deserved break,
>>> but not for definitions of immediately that mean "six months from now")
>>> freeze testing again until we're back down under 50-100 RC bugs,
>>> whether via fixes and transitions or whether by kicking out a bunch of
>>> packages.
>
>> People just spent 10 months fixing issues in packages that were not
>> their own.  I don't think they want to be pulled away from the fun
>> stuff so quickly.
>
> And that's why releases are so painful, at least IMO.
>
> We have to break this cycle at some point or it will just get worse.  To
> break the cycle, we're going to have to keep doing bug fixing after we're
> exhausted of doing bug fixing so that we don't build up a huge backlog.
>
> The good part is that if we actually can break that cycle, the freeze
> will be much less painful and we won't be as sick of fixing bugs the next
> time around.

Or the project could end up in a perpetual freeze.  Every time the
floodgates are opened, another 1,000 bugs could get reported due to
all of the new transitions, and another freeze will need to happen to
get those down.

> Think of this in software development terms.  We're currently following a
> development model where, immediately following a release, there's a nearly
> complete free-for-all without much enforcement of bugs or regressions.  We
> do that for a year, and then we try to stabilize the results.  Most free
> software projects that used to follow this model (and there have been
> quite a number of them) have had similar struggles with the extended
> stabilization process this requires.  That's part of the shift towards
> test-driven development, continuous integration, and constantly-usable
> master branches.

Those other distributions have also given up on producing high-quality
releases.  Only Debian and RHEL remain in that category.

>>> Because they're not from migrations.  They're from transitions.
>>> They're all the "this is going to break with a new version of libc6"
>>> and the like bugs that were filed before the release at priority
>>> important and were mass-upgraded after the release.
>
>> But those shouldn't affect testing yet, right?  All of that stuff
>> needs staging in unstable first.  Are bug filers not tagging their
>> reports correctly?  If so, that's quite misleading, and actually
>> should be quite easy although tedious to fix.
>
> The bug affects the version of the package in testing.  I see what you're
> saying, but I don't think this is something the BTS can represent.  And
> those bugs *are* all release-critical: they have to be fixed before we can
> release jessie, at least unless we're going to abort the transition, which
> seems unlikely.

There is the "sid" tag, which indicates that the issue only affects
unstable.  Although I think a "triggered-by" function is needed in the
bts.  For example, bugs with "triggered-by gcc 4.8-0" would not be
marked as affected in suites that have gcc versions prior to 4.8-0
(i.e. jessie would not yet be affected by the gcc transition).

Best wishes,
Mike


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CANTw=mo7tswbxi5jwhzy8woyy4v-4_0mhm1q91nnotxxq+y...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Russ Allbery]

Michael Gilbert  writes:

> Or the project could end up in a perpetual freeze.  Every time the
> floodgates are opened, another 1,000 bugs could get reported due to all
> of the new transitions, and another freeze will need to happen to get
> those down.

I would like to see us try it and see if that happens.  If that does
happen, I think that's a fascinating data point, and one that would be
well worth the few months of difficult process in order to confirm.
Knowing for certain whether or not that would be the outcome would be
wonderfully clarifying and helpful in narrowing the possible solution
space.

> On Sun, May 12, 2013 at 4:00 PM, Russ Allbery wrote:

>> Think of this in software development terms.  We're currently following
>> a development model where, immediately following a release, there's a
>> nearly complete free-for-all without much enforcement of bugs or
>> regressions.  We do that for a year, and then we try to stabilize the
>> results.  Most free software projects that used to follow this model
>> (and there have been quite a number of them) have had similar struggles
>> with the extended stabilization process this requires.  That's part of
>> the shift towards test-driven development, continuous integration, and
>> constantly-usable master branches.

> Those other distributions have also given up on producing high-quality
> releases.  Only Debian and RHEL remain in that category.

I don't believe that's true; in fact, it's the exact opposite of the
outcomes I've observed in practice.  Most projects that use test-driven
development, continuous integration, and constantly-usable master branches
maintain a significantly higher level of quality than the projects that
use development methodologies like Debian's (at the cost of forcing
disruptive changes to be done more slowly and with more planning, or to be
postponed if people aren't available to do them properly).

>> The bug affects the version of the package in testing.  I see what
>> you're saying, but I don't think this is something the BTS can
>> represent.  And those bugs *are* all release-critical: they have to be
>> fixed before we can release jessie, at least unless we're going to
>> abort the transition, which seems unlikely.

> There is the "sid" tag, which indicates that the issue only affects
> unstable.  Although I think a "triggered-by" function is needed in the
> bts.  For example, bugs with "triggered-by gcc 4.8-0" would not be
> marked as affected in suites that have gcc versions prior to 4.8-0
> (i.e. jessie would not yet be affected by the gcc transition).

If there are no RC bugs affecting the version of a package in testing,
that should mean that nothing has to be done to that package in order to
release.  If the package has to be fixed due to a transition that will be
in the next release, that is an RC bug affecting the version in testing
and should be counted accordingly.  The package cannot be left unchanged
and still go into the release, which is the definition of RC.

In other words, I see no point in doing what you describe.  So far as I
can tell, all it does is artificially lower the RC bug count in the graph,
while in no way reducing the amount of work that has to happen before
jessie is releasable.  In other words, it just hides a bunch of RC bugs
from the statistics without actually changing their RC status.  That seems
like an even worse state: we have just as much work to do but now we're
hiding that work from ourselves.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87y5bk3p68@windlord.stanford.edu

Re: Debian development and release: always releasable (essay)

[Michael Gilbert]

On Sun, May 12, 2013 at 4:42 PM, Russ Allbery wrote:
> Michael Gilbert writes:
>
>> Or the project could end up in a perpetual freeze.  Every time the
>> floodgates are opened, another 1,000 bugs could get reported due to all
>> of the new transitions, and another freeze will need to happen to get
>> those down.
>
> I would like to see us try it and see if that happens.  If that does
> happen, I think that's a fascinating data point, and one that would be
> well worth the few months of difficult process in order to confirm.
> Knowing for certain whether or not that would be the outcome would be
> wonderfully clarifying and helpful in narrowing the possible solution
> space.

I personally vote no more freeze, but I have no intent to stand in the
way if that is really the consensus of the rest of the project.

>>> Think of this in software development terms.  We're currently following
>>> a development model where, immediately following a release, there's a
>>> nearly complete free-for-all without much enforcement of bugs or
>>> regressions.  We do that for a year, and then we try to stabilize the
>>> results.  Most free software projects that used to follow this model
>>> (and there have been quite a number of them) have had similar struggles
>>> with the extended stabilization process this requires.  That's part of
>>> the shift towards test-driven development, continuous integration, and
>>> constantly-usable master branches.
>
>> Those other distributions have also given up on producing high-quality
>> releases.  Only Debian and RHEL remain in that category.
>
> I don't believe that's true; in fact, it's the exact opposite of the
> outcomes I've observed in practice.  Most projects that use test-driven
> development, continuous integration, and constantly-usable master branches
> maintain a significantly higher level of quality than the projects that
> use development methodologies like Debian's (at the cost of forcing
> disruptive changes to be done more slowly and with more planning, or to be
> postponed if people aren't available to do them properly).

I wasn't responding to the continuous testing sentence.  I'm in
complete agreement on the importance of continuous testing.

I was responding to the comments on the development model.  Other
distributions have moved to time-based and given up on quality.
Debian has maintained quality by not compromising on time.

>>> The bug affects the version of the package in testing.  I see what
>>> you're saying, but I don't think this is something the BTS can
>>> represent.  And those bugs *are* all release-critical: they have to be
>>> fixed before we can release jessie, at least unless we're going to
>>> abort the transition, which seems unlikely.
>
>> There is the "sid" tag, which indicates that the issue only affects
>> unstable.  Although I think a "triggered-by" function is needed in the
>> bts.  For example, bugs with "triggered-by gcc 4.8-0" would not be
>> marked as affected in suites that have gcc versions prior to 4.8-0
>> (i.e. jessie would not yet be affected by the gcc transition).
>
> If there are no RC bugs affecting the version of a package in testing,
> that should mean that nothing has to be done to that package in order to
> release.  If the package has to be fixed due to a transition that will be
> in the next release, that is an RC bug affecting the version in testing
> and should be counted accordingly.  The package cannot be left unchanged
> and still go into the release, which is the definition of RC.
>
> In other words, I see no point in doing what you describe.  So far as I
> can tell, all it does is artificially lower the RC bug count in the graph,
> while in no way reducing the amount of work that has to happen before
> jessie is releasable.  In other words, it just hides a bunch of RC bugs
> from the statistics without actually changing their RC status.  That seems
> like an even worse state: we have just as much work to do but now we're
> hiding that work from ourselves.

Unless that information is then used to figure out when its really ok
to let the gcc (or whatever) transition happen, or even to decide to
put an end to a particular buggy transition before it starts affecting
testing.

Right now, there are just large numbers every where, in testing, in
unstable.  That lack of information is counter-productive and
misleading, and the large number is discouraging to anyone potentially
interesting in knocking a couple bugs off.

Best wishes,
Mike


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CANTw=MPQaG5uf=u2dntzxafz1n0zjhlqc5j60vx+deoeapd...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Julien Cristau]

On Sun, May 12, 2013 at 14:48:36 -0400, Michael Gilbert wrote:

> But those shouldn't affect testing yet, right?  All of that stuff
> needs staging in unstable first.  Are bug filers not tagging their
> reports correctly?  If so, that's quite misleading, and actually
> should be quite easy although tedious to fix.
> 
NAK.  These bugs do affect testing, and the BTS state should reflect
that.  Don't go add bogus sid tags, and remove the ones you just added,
TIA.

Cheers,
Julien


signature.asc
Description: Digital signature

Re: Debian development and release: always releasable (essay)

[Kurt Roeckx]

On Thu, May 09, 2013 at 08:49:51PM +0100, Lars Wirzenius wrote:
> 
> Releases are important
> --
> 
> Releases are important to many, perhaps most, of our users. Hackers
> and hardcore powerusers don't necessarily care about them, of course,
> but most others do. A released version of Debian implies that the
> operating system works: there's a working installer, for example.
> It also implies that all the packages are expected to work together:
> there's no transitions going on, for example, that might break
> dependencies or reverse dependencies.

One thing I'm wondering about, and you don't seem to talk about is
what versions end up in a release.

Some upstreams have a testing branch of there software and a
release branch.  It's sometimes useful to have people test the
version in from the testing branch, and having it available in
Debian makes it easier for people to test it.

The question is, to what do I upload it?  If I just upload this
to experimental, I'm not going to get any real users, only people
who really want to use this newer version for whatever reason.
But it's sometimes more interesting to have a wider audience use
it.  This of course depends on how stable the version really is.
So what people now do is upload this to unstable, and even let it
migrate to testing.  But it might not be diserable to actually
have this unreleased version in a Debian release.  It might have
bugs that aren't RC, and you might be better of with the previous
release.

Do you have a suggestion on how to deal with that?



Kurt


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130515222911.ga22...@roeckx.be

Re: Debian development and release: always releasable (essay)

[Lars Wirzenius]

On Thu, May 16, 2013 at 12:29:11AM +0200, Kurt Roeckx wrote:
> One thing I'm wondering about, and you don't seem to talk about is
> what versions end up in a release.
> 
> Some upstreams have a testing branch of there software and a
> release branch.  It's sometimes useful to have people test the
> version in from the testing branch, and having it available in
> Debian makes it easier for people to test it.
> 
> The question is, to what do I upload it?  If I just upload this
> to experimental, I'm not going to get any real users, only people
> who really want to use this newer version for whatever reason.
> But it's sometimes more interesting to have a wider audience use
> it.  This of course depends on how stable the version really is.
> So what people now do is upload this to unstable, and even let it
> migrate to testing.  But it might not be diserable to actually
> have this unreleased version in a Debian release.  It might have
> bugs that aren't RC, and you might be better of with the previous
> release.
> 
> Do you have a suggestion on how to deal with that?

On the assumption that the testing branch is not suitable for users of
the software...

I'd use a PPA-style package repository of some sort, and then advertise
it to people might want to try that version of the package. That doesn't
give you as much exposure as uploading it to Debian unstable (and letting
it migrate to Debian testing), but also doesn't impact Debian releases
and doesn't surprise Debian users.

-- 
http://www.cafepress.com/trunktees -- geeky funny T-shirts
http://gtdfh.branchable.com/ -- GTD for hackers


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130516070332.gz2...@havelock.liw.fi

Re: Debian development and release: always releasable (essay)

[Neil McGovern]

On Thu, May 16, 2013 at 12:29:11AM +0200, Kurt Roeckx wrote:
> Some upstreams have a testing branch of there software and a
> release branch.  It's sometimes useful to have people test the
> version in from the testing branch, and having it available in
> Debian makes it easier for people to test it.
> 

A couple of options seem to present itself (under current scheme) - 
a) upload to experimental and publically call for testing
b) upload to unstable and file an RC bug yourself so it doesn't migrate
c) upload to unstable, wait for migration, then file an RC bug so we
don't release with it.

Neil
-- 


signature.asc
Description: Digital signature

Re: Debian development and release: always releasable (essay)

[Andrei POPESCU]

On Jo, 16 mai 13, 10:52:05, Neil McGovern wrote:
> On Thu, May 16, 2013 at 12:29:11AM +0200, Kurt Roeckx wrote:
> > Some upstreams have a testing branch of there software and a
> > release branch.  It's sometimes useful to have people test the
> > version in from the testing branch, and having it available in
> > Debian makes it easier for people to test it.
> 
> A couple of options seem to present itself (under current scheme) - 
> a) upload to experimental and publically call for testing
> b) upload to unstable and file an RC bug yourself so it doesn't migrate
> c) upload to unstable, wait for migration, then file an RC bug so we
> don't release with it.

d) maintain a separate -unstable package (see wine and wine-unstable), 
use a), b) or c) as apropiate if the package is not suitable for a 
stable release.

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser
Offtopic discussions among Debian users and developers:
http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic


signature.asc
Description: Digital signature

Re: Debian development and release: always releasable (essay)

[Kurt Roeckx]

On Thu, May 16, 2013 at 08:03:33AM +0100, Lars Wirzenius wrote:
> 
> I'd use a PPA-style package repository of some sort, and then advertise
> it to people might want to try that version of the package.

Then it makes more sense to upload it to experimental to me.


Kurt


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130516164348.ga17...@roeckx.be

Re: Debian development and release: always releasable (essay)

[Michael Gilbert]

On Thu, May 16, 2013 at 5:52 AM, Neil McGovern wrote:
> On Thu, May 16, 2013 at 12:29:11AM +0200, Kurt Roeckx wrote:
>> Some upstreams have a testing branch of there software and a
>> release branch.  It's sometimes useful to have people test the
>> version in from the testing branch, and having it available in
>> Debian makes it easier for people to test it.
>>
>
> A couple of options seem to present itself (under current scheme) -
> a) upload to experimental and publically call for testing
> b) upload to unstable and file an RC bug yourself so it doesn't migrate
> c) upload to unstable, wait for migration, then file an RC bug so we
> don't release with it.

Approach c) should really be considered socially unacceptable.  In
that case, if the maintainer doesn't get around to solving the
problem, he/she has in effect selfishly forced the rest of the project
into solving problems that they otherwise wouldn't have to deal with
(if that package had never migrated).

Best wishes,
Mike


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CANTw=MMktMU4u=qomc_4tanvh3zj6xb15igs9xtdk7-ki_s...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Ondřej Surý]

On Sun, May 12, 2013 at 1:46 PM, Thomas Goirand  wrote:
> On 05/12/2013 02:03 AM, Antonio Terceiro wrote:
>> You can't assume that just because something works today, it will work
>> forever.
>
> True, though it's been at least 2 release cycle (maybe 3?) that this
> set of packages were maintained quite well. I don't remember
> seeing complains or bad bugs. Do you?

Part of the problem is that some of those important packages don't
have enough manpower.

As an example: There's a RFH bug filled on PHP for a long time, some
people came, but in the end I had to poke Ubuntu PHP maintainer
several times to help me write php5{en,dis}mod, which I could
integrate with PHP. (To see the situation, you can look at ohloh.net:
https://www.ohloh.net/p/pkg-php/contributors/summary)

This (in the end) leads to inevitable situations where I do upload
packages with bugs sometimes, or I just don't have enough time to fix
bugs in time and other (sometimes more virtual than real) team members
are also in slumber.

So apart from the more hands on some packages with high priority, it
would really help me to have some automated tests which would be run
before uploading to testing.

One thing which immediatelly comes to the mind is the install & upgrade test.

1. install every built package
2. try upgrade from stable for every built package
3. try upgrade from testing for every built package
4. try upgrade from unstable for every built package

Optionally:
5. do some testing with all r-deps (treeish?)

I am not sure if we need to do that for all packages in the archive,
but I am writing this from the perspective of maintainer/only active
team member of PHP5 (php5 5.3 -> 5.4 transition), Berkeley DB
(db4.7+db4.8 -> db5.1 transition), Cyrus SASL (cyrus-sasl2), Cyrus
IMAPd (cyrus-imapd-2.2 -> cyrus-imapd-2.4 transition with some crazy
database backends change). And I already have pu for three of four of
them (not very proud of that).

And rails 2.3+3.2, but thanks to Antonio Terceiro (and rest of
pkg-ruby-extra), I don't feel that alone there.

Ondrej
--
Ondřej Surý 


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/caljhhg_d5e+2qopnput1hjvq7urjd7zmswlrtnn0cjmln_t...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Paul Wise]

On Mon, May 20, 2013 at 12:39 AM, Ondřej Surý wrote:

> So apart from the more hands on some packages with high priority, it
> would really help me to have some automated tests which would be run
> before uploading to testing.
>
> One thing which immediatelly comes to the mind is the install & upgrade test.
>
> 1. install every built package
> 2. try upgrade from stable for every built package
> 3. try upgrade from testing for every built package
> 4. try upgrade from unstable for every built package

Perhaps you missed the existence of piuparts.d.o?

http://piuparts.debian.org/

Failures of installation in sid are advertised on the PTS, we are
hoping to extend this to the other tests soon (#696094).

> Optionally:
> 5. do some testing with all r-deps (treeish?)

We don't yet have any systems running autopkgtest, but Ubuntu does so
you could look at their instance. In the meantime check out DEP-8:

http://dep.debian.net/deps/dep8/

Other tests may be suitable for the Debian Jenkins instance:

http://jenkins.debian.net/

-- 
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/caktje6efj0gdcf3uuzai4uquhq6scowfxrbndda9ekmzrsy...@mail.gmail.com

Re: Debian development and release: always releasable (essay)

[Petter Reinholdtsen]

[Russ Allbery]
> I would need to do some research, since I'm not personally familiar
> with everything that's in a blend or a task at the moment.  Just off
> the top of my head, though, to pick an area of personal expertise, I
> don't think there's an existing blend or task for a Kerberos KDC or,
> more generally, an authentication and identity management
> infrastructure.

Actually, I believe there is.  The Debian Edu blend contain the
education-main-server task and metapackage, which include a Kerberos
KDC.  It also contain the LDAP server as KDC backend storage and
user/group/etc lookup.

-- 
Happy hacking
Petter Reinholdtsen


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/2flsj1ag54h@login1.uio.no

Re: Debian development and release: always releasable (essay)

[Holger Levsen]

Hi,

On Samstag, 25. Mai 2013, Petter Reinholdtsen wrote:
> Actually, I believe there is.  The Debian Edu blend contain the
> education-main-server task and metapackage, which include a Kerberos
> KDC.  It also contain the LDAP server as KDC backend storage and
> user/group/etc lookup.

there is also the Debian-LAN which is described as "The goal of Debian-LAN is 
to make setting up a local network with centralized user and machine 
management, intranet, etc. as easy as possible in Debian."

see ://lists.debian.org/20120408083121.GB9680@flashgordon


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Re: Debian development and release: always releasable (essay)

[Andreas Tille]

Hi,

On Sun, May 26, 2013 at 03:10:07PM +0200, Holger Levsen wrote:
> Hi,
> 
> On Samstag, 25. Mai 2013, Petter Reinholdtsen wrote:
> > Actually, I believe there is.  The Debian Edu blend contain the
> > education-main-server task and metapackage, which include a Kerberos
> > KDC.  It also contain the LDAP server as KDC backend storage and
> > user/group/etc lookup.
> 
> there is also the Debian-LAN which is described as "The goal of Debian-LAN is 
> to make setting up a local network with centralized user and machine 
> management, intranet, etc. as easy as possible in Debian."
> 
> see ://lists.debian.org/20120408083121.GB9680@flashgordon

If I where Andreas Mundt I would try to do this inside the Debian
Enterprise effort.  While there is barely any traffic on this list you
just need to start with such an effort somehow and IMHO the topic does
fit.

Kind regards

   Andreas.

-- 
http://fam-tille.de


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130527121417.gc16...@an3as.eu

Re: Debian development and release: always releasable (essay)

[Andreas B. Mundt]

Hi all,

On Mon, May 27, 2013 at 02:14:17PM +0200, Andreas Tille wrote:
> On Sun, May 26, 2013 at 03:10:07PM +0200, Holger Levsen wrote:
> > On Samstag, 25. Mai 2013, Petter Reinholdtsen wrote:
> > > Actually, I believe there is.  The Debian Edu blend contain the
> > > education-main-server task and metapackage, which include a Kerberos
> > > KDC.  It also contain the LDAP server as KDC backend storage and
> > > user/group/etc lookup.
> >
> > there is also the Debian-LAN which is described as "The goal of Debian-LAN 
> > is
> > to make setting up a local network with centralized user and machine
> > management, intranet, etc. as easy as possible in Debian."
> >
> > see ://lists.debian.org/20120408083121.GB9680@flashgordon
>
> If I where Andreas Mundt I would try to do this inside the Debian
> Enterprise effort.  While there is barely any traffic on this list you
> just need to start with such an effort somehow and IMHO the topic does
> fit.

First, many thanks to Holger for mentioning the Debian-LAN project
here and thereby pointing me to the ongoing discussion. I try to
follow -devel as time permits, but I'm not subscribed, please keep me
in cc.

@Andreas:  I announced Debian-LAN on several lists, including
debian-enterprice [1], and debian-news mentioned it too [2].  It's
planned to send another announcement message as soon as the latest
additions to the debian-lan-config package are well tested and
uploaded to wheezy-backports.  A DebConf talk is under way.  In other
words:  Anybody is invited to make use of and/or contribute to the
Debian-LAN project.  I would be rather astonished if there are already
efforts in Debian Enterprise currently working on the same issue.  If
this is the case, we should of course not do the work twice.

But back to the topic.  I appreciate the ideas outlined by Lars and
Russ very much, and I would love to see debian-lan helping to make
them reality.

The Debian-LAN system provides a basic Debian Desktop installation in
combination with a full-featured server providing a Kerberos KDC with
kerberized services: ssh, NFSv4, apache, LDAP, exim, dovecot, ...

FAI provides a very structured and extremely flexible way to compose
the system. For any service (== FAI class), the implementation is
straight forward: Packages, preseedings, and if unavoidable extra
files and/or 'manual' configuration tweaks.  All this in combination
with the thorough logging included in FAI proved to be a valuable
concept:  It's almost always clear what's gone wrong and causes
problems.

Since I started to work on the Debian-LAN project at DebConf11, it
turned out with every implementation of a new service/feature that
using FAI is a very good approach.

I am not familiar with jenkins, but I cannot imagine a problem running
automatic test as Holger already does with a slightly adapted
Debian-LAN system.  The goal of Debian-LAN is to provide a Debian
local area network out-of-the box, this covers exactly all relevant
tests that a stable Debian should pass.

If Debian-LAN can be part of such automatic testing it would be really
great!

Best regards,

 Andi



[1] https://lists.debian.org/debian-enterprise/2012/04/msg0.html
[2] https://lists.debian.org/debian-news/2012/msg00015.html


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130527220655.GA438@fuzi

simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))

[Andreas Beckmann]

@all maintainers: How would you like to run piuparts s.t. it easily
integrates into your workflow and allows improving Debian's quality?
This is something we could improve right now. Integrating piuparts into
the ftp-master/buildd side will take a much longer way.

On 2013-05-19 18:39, Ondřej Surý wrote:
[...]
> So apart from the more hands on some packages with high priority, it
> would really help me to have some automated tests which would be run
> before uploading to testing.
> 
> One thing which immediatelly comes to the mind is the install & upgrade test.
> 
> 1. install every built package
> 2. try upgrade from stable for every built package
> 3. try upgrade from testing for every built package
> 4. try upgrade from unstable for every built package

Let me see if we can find a way to simplify running piuparts for all
these ...

Since you are aiming to test transitions, this will need to cover
testing multiple source packages at the same time and a lot of
dependencies between them. May I assume you have a local repository of
these packages available? (You probably already have one to build
updated packages for the transition.) A file:// URL with some .debs and
a Packages file will be fine.
Or would you prefer to feed a bunch of .changes files to some script?

Once you run that yet-to-be-written script, you will get a bunch of
logfiles (for k .debs and 4 tests that would be 4*k logfiles), some
failed, some success. Since piuparts is very chatty, you probably want
to have a summary and a list of failures at the end ...
The failing logfiles will need to be analyzed manually.

Next question: Do you want to do incremental testing? Assume you
identified an incorrectly versioned dependency, fixed that, rebuilt the
package, updated the repository. Do you want to
* retest a specific failure
* retest all failures
* retest everything (you probably only want to do this at the end as it
may be very time consuming).
I would probably go for "test everything that does not have a success
log", so you could retest arbitrary subsets by just deleting the
corresponding (success-)logs.

The good thing is: by now piuparts should have all the needed options to
do this, it just needs a new driver script to simplify running it on a
bunch of local packages. (And running it on each package from a (set of)
.changes file(s) individually, not only installing all at the same time)

In http://bugs.debian.org/700849 I posted a script skeleton that
(requires manual adjustment to configure it and) allows to quickly run
one of the tests listed above for a local (or remote) repository.

This yet-to-be-written script should not be specific for uploads to sid,
but work for pu, opu or backports as well. Experimental may be a bit
more difficult as this is "frequently broken" (as in "requiring a very
careful selection of the install set mixing sid and experimental")


> Optionally:
> 5. do some testing with all r-deps (treeish?)

That may take some time to run ... I also don't know how to quickly
build the rdep tree. But if you already have the local repository and a
list of "interesting" packages (not in the local repo), it should be
quite easy to check how they behave (install/upgrade wise) in a
sid+prospective environment.

That's just my interpretation how this could work ...


Andreas


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/519b4309.3040...@debian.org

Re: simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))

[Charles Plessy]

Le Tue, May 21, 2013 at 11:48:57AM +0200, Andreas Beckmann a écrit :
> @all maintainers: How would you like to run piuparts s.t. it easily
> integrates into your workflow and allows improving Debian's quality?
> This is something we could improve right now. Integrating piuparts into
> the ftp-master/buildd side will take a much longer way.

Hi Andreas,

it is not fully related to your original question, but do you think that 
piuparts
could support running Autopkgtests as well ?

Cheers,

-- 
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130522040503.gb21...@falafel.plessy.net

Re: simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))

[Holger Levsen]

Hi,

On Mittwoch, 22. Mai 2013, Charles Plessy wrote:
> it is not fully related to your original question, but do you think that
> piuparts could support running Autopkgtests as well ?

I think we need another setup for this. Autopkgtests may destroy their 
environment (and might need more than a chroot) so they cannot be fully tested 
in our current piuparts.d.o setup.

I'd rather do autopkgtests with jenkins.d.n (and be very happy to help anyone 
working on them).


cheers,
Holger




signature.asc
Description: This is a digitally signed message part.

Re: simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))

[Jakub Wilk]

* Holger Levsen , 2013-05-22, 13:26:
it is not fully related to your original question, but do you think 
that piuparts could support running Autopkgtests as well ?
I think we need another setup for this. Autopkgtests may destroy their 
environment (and might need more than a chroot) so they cannot be fully 
tested in our current piuparts.d.o setup.


FWIW, most of the packages don't need anything more than a chroot.
Out of 116 packages that have DEP-8 tests, only 2 declare the 
breaks-testbed restrictions. (And, AFAICS, even the two with 
breaks-testbed don't currently need stronger isolation.)


--
Jakub Wilk


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130522121418.ga3...@jwilk.net

Re: simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))

[Holger Levsen]

On Mittwoch, 22. Mai 2013, Jakub Wilk wrote:
> FWIW, most of the packages don't need anything more than a chroot.

Interesting, thanks.


signature.asc
Description: This is a digitally signed message part.

Re: [Piuparts-devel] simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))

[Holger Levsen]

Hi,

On Dienstag, 21. Mai 2013, Andreas Beckmann wrote:
> @all maintainers: How would you like to run piuparts s.t. it easily
> integrates into your workflow and allows improving Debian's quality?

have an option to run piuparts automatically by debuild, after or before 
lintian.


cheers,
Holger


signature.asc
Description: This is a digitally signed message part.

Re: [Piuparts-devel] simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))

[Ondřej Surý]

On Tue, May 21, 2013 at 12:05 PM, Holger Levsen  wrote:
> Hi,
>
> On Dienstag, 21. Mai 2013, Andreas Beckmann wrote:
>> @all maintainers: How would you like to run piuparts s.t. it easily
>> integrates into your workflow and allows improving Debian's quality?
>
> have an option to run piuparts automatically by debuild, after or before
> lintian.

Also integrate it with git-pbuilder/pbuilder/cowbuilder to run
piuparts inside the created clean(ish) chroot, so it's less time
consuming.

O.
--
Ondřej Surý 


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/caljhhg-zkfs5vxnic+e3_bzdskpwqa7bko+sng0m6p3k_mh...@mail.gmail.com

Re: [Piuparts-devel] simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))

[Didier 'OdyX' Raboud]

Le mardi, 21 mai 2013 12.35:47, Ondřej Surý a écrit :
> On Tue, May 21, 2013 at 12:05 PM, Holger Levsen  
wrote:
> > have an option to run piuparts automatically by debuild, after or before
> > lintian.
> 
> Also integrate it with git-pbuilder/pbuilder/cowbuilder to run
> piuparts inside the created clean(ish) chroot, so it's less time
> consuming.

Actually, doing it in sbuild (and hence hypothetically on the buildds) would 
be awesome:
 - deinstall build-depends
 - install previous version (+ dependencies), then upgrade
 - remove
 - purge
 - …

A failure at each of these stages could fail the build.

(The problem becomes tricky with source packages building mutually-
incompatible binary packages).

Cheers,

OdyX

P.S. I'd be interested in getting something along those lines working but am 
not sure to have enough time on my hands, unfortunately.


signature.asc
Description: This is a digitally signed message part.