Re: Build sheriffs for GNOME

[Michael Catanzaro]

On Sat, 2016-01-23 at 03:14 +0900, Tristan Van Berkom wrote:
> I do not support the vague notion that a "sheriff" can come along and
> revert an entire branch I've just landed which changes the API
> contract
> early enough in the release cycle for depending modules to catch on
> and
> adapt to.

Ah, yeah, we should not be reverting entire large sequences of
commits... I agree that would be excessive. In cases like this, we can
tag your module in Continuous and branch it in jhbuild, until other
modules have caught up with your API changes. Build sheriffs should try
the least-disruptive option possible to get things fixed.

In practice, I don't think this is very often an issue; usually it's
just little things that break. The most recent example comparable to
your scenario would be grilo-3.0. In such cases, it usually makes more
sense to tag/branch the module that broke API for a bit, or to try
fixing the build in the affected modules.

Michael
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Michael Catanzaro]

On Fri, 2016-01-22 at 13:32 -0600, Michael Catanzaro wrote:
> Ah, yeah, we should not be reverting entire large sequences of
> commits... I agree that would be excessive. In cases like this, we
> can
> tag your module in Continuous and branch it in jhbuild, until other
> modules have caught up with your API changes. Build sheriffs should
> try
> the least-disruptive option possible to get things fixed.

We could also keep a list of maintainers who request that we not touch
their modules directly, so we know to branch and tag your module when
something breaks instead of committing to it directly. I suspect not
many maintainers will opt-in to this, but you've made it clear there is
demand!

Michael
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Tristan Van Berkom]

On Fri, 2016-01-22 at 11:40 -0600, Michael Catanzaro wrote:
> On Fri, 2016-01-22 at 09:25 -0800, Jasper St. Pierre wrote:
> > It's more frequent than you might think.
> > 
> > In the past week, alone, we've had to tag glib, gnome-calculator,
> > e-d-s, gstreamer, and more, all because of broken builds. Take a
> > look
> > at how busy gnome-continuous is as a project:
> > https://git.gnome.org/browse/gnome-continuous/log/
> 
> Absolutely. I should be more clear: it doesn't happen so often for
> any
> PARTICULAR module, so the git history of any particular module is not
> likely to get polluted by many revert commits. (And even if it does
> --
> so what if your history doesn't look quite as nice as you'd like.)
> But
> it happens a LOT for GNOME as a whole, and it's a big problem, which
> is
> why I'm so supportive of Emmanuele's plan.

I am still a bit baffled by this, as I expect a higher standard of
quality from GNOME contributors than I would for your regular corporate
r lab.

So let me reiterate what I mentioned in my first email, I am not
against this proposal for the stable branches, and, for master: I think
Emmanuele's proposal needs clarification.

To clarify what I mean by that, is it should be clear what a sheriff
has, and has not the right to revert. If I committed a spelling error
which obviously broke the build of my own module, *doh*, of course,
it's a single commit, a stupid one, and I really couldnt care less if
it were to be reverted by someone else who caught the error. Jasper
seems to indicate that this kind of thing actually happens more than I
would expect, this is indeed sloppy and should be addressed.

I do not support the vague notion that a "sheriff" can come along and
revert an entire branch I've just landed which changes the API contract
early enough in the release cycle for depending modules to catch on and
adapt to.

As I have elaborated enough in my reply to Shaun, this kind of thing
happens, frequently enough, and there are transitional periods where
building all of GNOME and expecting everything to build is just not
rational, responsible maintainers will be careful to introduce churn as
early as possible in the cycle.

Best Regards,
    -Tristan

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Tristan Van Berkom]

On Fri, 2016-01-22 at 12:12 -0500, Shaun McCance wrote:
> On Fri, 2016-01-22 at 16:03 +0900, Tristan Van Berkom wrote:
> > On Thu, 2016-01-21 at 14:54 +, Emmanuele Bassi wrote:
> > [...]
> > > This is not enough, and it does not raise the bar in keeping
> > > GNOME
> > > in
> > > a buildable state. It actually lowers it a fair , to the
> > > effective
> > > point that *nobody* cares about Continuous builds.
> > > 
> > > I want this to change. I want to be able to revert failing
> > > commits
> > > on
> > > the offending modules, if they are hosted on GNOME
> > > infrastructure,
> > > if
> > > they fail for more than N hours, and *then* open a bug about it.
> > 
> > Hi Emmanuele,
> > 
> > In GNOME, we have got along well for many years with many projects
> > sharing the same infrastructure mostly because we find that in
> > practice, people are generally respectful enough of eachothers work
> > to
> > recognize a module maintainer's word as scripture. As such we do
> > not
> > commit changes to others modules without expressed permission
> > unless
> > it
> > is for a minor change that we are absolutely sure about.
> 
> It has always been my understanding that master (née trunk née HEAD)
> is
> supposed to be buildable in GNOME, and that the release team has the
> power to make commits to ensure it is.

And it has always been my understanding that it is simply not the case,
notably because of said cross module API changes.

It annoys me that in recent years, notably after the introduction of
jhbuild, which has saved us a lot of trouble of setting up local
environment scripts and building everything by hand, that this
expectation has somehow arisen, this expectation is just causing
friction, it simply cannot be expected within reason.

> I am sympathetic to the difficulty of synchronizing cross-module API
> changes. But is there any reason those changes can't all be done in
> development branches in each module, with all of them merged to
> master
> at the same time when they're ready?

Yes.

Here is one example:

Michael Catanzaro writes in this very thread:
> Milan, you have full commit access to every repo on git.gnome.org.
> Any breakage that could register on Continuous is fully within your 
> ability to fix.

I find this statement very disconcerting, and it relates and is a good
example.

Milan maintains EDS, and EDS is a relatively *huge* and complex source
base, I personally know Milan to be considerate about API breakages and
stability in general and I trust that if he has to change the API
contract, it is in the interest of long term maintainability of this
very complex project which has gone though a *lot* of churn over the
last decade.

Milan does not maintain libfolks, and he does not maintain gnome-
contacts, if he is going to spend weeks refactoring EDS and Evolution,
I am not going to tell him that his word is not the law, his word on
the EDS contract *is* law, and I'm confident that he will announce it,
and that maintainers of related modules will have a heads up with
enough time to adjust for the next stable GNOME release.

I am not going to tell Milan that hey, I know you spent weeks or months
of your life refactoring and making EDS better, but I will not accept
that you land your work in your own module unless you spend another
week modifying libfolks and/or gnome-contacts.

Changes like this need to be introduced as soon as they are ready in
master, which is the unstable development branch until the next stable
release is made. It cannot be delayed until the maintainer of libfolks
returns from vacation and wraps his head around the new changes, then
we lose time and we cause friction with other patches landing in
master, accumulating conflicts and generating more work - this work is
not necessary, because nobody every guaranteed that "all of GNOME
master" would build harmoniously together at a given time.

Here is another example, Benjemin Otte one week or two ago landed some
huge refactoring in GTK+ related to drawing - as a result, Glade is
entirely unusable. It's entirely possible that I may have had an
installable unit test in Glade to raise a flag on this.

Are you, really, going to go and revert the landing of his branch
because the unit test flagged GTK+ as broken ?

Sure, for a single project which builds regularly against a stable
platform, the expectation is always that master builds. But this
analogy of a single project cannot not carry on to GNOME, which is an
assembly of distinctly separately run projects which are developed
under the same infrastructure, integration issues will obviously arise
and the best that we can hope for is introducing those issues as early
as possible in the cycle and try our best to stabilize by the end of
the cycle, when churn comes to a halt and individual modules stabilize.

Best Regards,
    -Tristan

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org

Re: Build sheriffs for GNOME

[Milan Crha]

On Fri, 2016-01-22 at 16:03 +0900, Tristan Van Berkom wrote:
> In summary, I am not opposed to applying your proposal as is to the
> stable builds, there is no justification *ever* for breakage in stable
> branches.
> 
> For master, I only think this needs to be detailed properly, perhaps it
> would be enough to ensure we had policy ensuring that intentional
> breakage is announced (on this list ?) and that "sheriffs" are
> responsible for following this list and not reverting commits which
> break things intentionally in a transitional period.

Hi,
I agree with Tristan. I was just about to write something along his
lines. Seeing semi-automated reverts in the projects would be quite
depressing, especially when the semi-automat has no idea about the
project and what the change was meant for.

It makes sense to make sure the build works *before the release*, but
when we are in the middle of two releases, then the build break might
be just a red flag, not an excuse to break someone's work by the
revert.

I sometimes find it hard to plan some larger work due to too often
releases (it's for a really large work, which can take several weeks to
accomplish, then some more to fine tune in the real world usage). If
you add this precedence, to always let someone ask: "oh, well, can I
commit it? It's large, would anyone revert it? My change touches API
and influences like 5+ projects, where I have only 3 under my direct
control, thus the main commit is almost immediately followed by
corresponding fixes in those projects I've under control, but some
still can be broken", then I'm afraid the "hostile environment" would
be a very nice word about GNOME and its infrastructure (people would
use much much worse in the case you'd really make your proposal alive).

This is also about different environments, you partly mentioned it. I
believe most of the maintainers build before commit. I do, but I'm a
human and mistakes happen. Some mistakes are independent of the build
environment (I'm sorry for the before-Christmas breakage of the eds),
some are not. I recall broken builds even in jhbuild when API changes
happened (clean build was required to make it work, there was nothing
wrong about the sources).

Even for example Ubuntu environment is different from that I use,
causing undefined symbols on places which work just fine for me. What
would you revert then in my project? I know, Continuous, it built an
hour earlier, it should build now too.

Anyway, doing any reverts might not be implicit, not without a previous
discussion with the respective maintainers (unless they are not
reachable), and only a day before the release - or on the Friday when
the notification mail is sent, though for Weekend Contributors it would
make more sense for Monday morning of the release day? I do not know,
people can fix things just before the release too. It's because the
GNOME should build from the release, not necessarily from the git
snapshot.

The Continuous idea is great to discover build breaks early, but it's
not an excuse to damage anyone's work.
Bye,
Milan
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: libgnomekbd development

[Antonio Ospite]

On Thu, 21 Jan 2016 14:27:28 -0600
Michael Catanzaro  wrote:

> Is anybody interested in taking over maintainership of libgnomekbd?
> To my knowledge, it's only used by gnome-shell and gnome-control-center 
> to display the keyboard layout dialog, so it should be very low-
> maintenance.
>

The keyboard widget is also very useful when testing xkb geometries and
layouts, so it makes sense to keep it alive and maintained IMHO.

For instance I am using it to experiment with custom keyboard layouts
and it's a lot nicer then xkbprint :)

I have some more patches (autotools updates, some gtk deprecation fixes,
a fix for a SIGSEGV) and a question about how to use
gkbd_keyboard_drawing_set_groups_levels() from python.

I'll open other bugzilla reports hoping that someone finds the time to
go through them.

Ciao ciao,
   Antonio

-- 
Antonio Ospite
http://ao2.it

A: Because it messes up the order in which people normally read text.
   See http://en.wikipedia.org/wiki/Posting_style
Q: Why is top-posting such a bad thing?
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: libgnomekbd development

[Коростіль Данило]

On 22.01.16 14:57, Antonio Ospite wrote:

The keyboard widget is also very useful when testing xkb geometries and
layouts, so it makes sense to keep it alive and maintained IMHO.

For instance I am using it to experiment with custom keyboard layouts
and it's a lot nicer then xkbprint:)

I have some more patches (autotools updates, some gtk deprecation fixes,
a fix for a SIGSEGV) and a question about how to use
gkbd_keyboard_drawing_set_groups_levels() from python.

I'll open other bugzilla reports hoping that someone finds the time to
go through them.

Ciao ciao,
My guess is you're are most probably the only user and active developer 
in the same time at the moment, so it's okay if you maintain and push 
your patches yourself. Don't be hesitate to ask for corresponding 
permissions here.

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Simon McVittie]

On 21/01/16 14:54, Emmanuele Bassi wrote:
> In short, I want to ensure that GNOME maintainers become
> a bit more proactive in giving a crap about their modules breaking on
> something that is not their own computers.

Is there a way in which contributors/maintainers can reproduce something
equivalent to what Continuous would build (or a large enough subset of
the dependency stack to reach their packages), in a local virtual
machine where they can debug issues interactively?

In the autobuild systems that I've worked with, I've found that the key
feature that makes them most useful is to avoid "magic" centralized
infrastructure that only a few sysadmins/maintainers can access.

For instance, one of the things I really like about Debian's autopkgtest
initiative (a package-oriented and potentially system-level equivalent
of GNOME installed-tests) is that it's quite well-documented how to get
a local mirror of the environment in which tests are run on the official
infrastructure[1] and how to get something basically equivalent as a
virtual machine image[2].

[1] https://ci.debian.net/doc/file.MAINTAINERS.html
[2] http://manpages.debian.org/cgi-bin/man.cgi?query=adt-virt-qemu

-- 
Simon McVittie
Collabora Ltd. 

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Michael Catanzaro]

On Fri, 2016-01-22 at 10:55 +0100, Milan Crha wrote:
>   Hi,
> I agree with Tristan. I was just about to write something along his
> lines. Seeing semi-automated reverts in the projects would be quite
> depressing, especially when the semi-automat has no idea about the
> project and what the change was meant for.

Why does it matter what the change was meant for? If people cannot
build GNOME, that is a much more serious problem than whatever your
commit is fixing.

> It makes sense to make sure the build works *before the release*, but
> when we are in the middle of two releases, then the build break might
> be just a red flag, not an excuse to break someone's work by the
> revert.

Let's take e-d-s as a specific example. If we were building tarball
releases of e-d-s in jhbuild and Continuous, this would make perfect
sense. We're not doing that. We build e-d-s from git master because it
is a core component of GNOME, and we want jhbuild to build the latest
version of everything, and we want contributors to be able to
contribute to e-d-s after all!, hence the need for a git repo. master
needs to be buildable at all times. This isn't for you, it's for
*everyone else* trying to build GNOME.

Now, you could choose to work around this if you want, by tagging e-d-s 
in Continuous and branching it in jhbuild, then updating that branch
with every release. Or, you could work in a "next" branch that gets
merged into master each release. Personally, I would continue to work
in master, and accept that once in a while a change of mine will get
reverted until I have figured out what is wrong with it

> I sometimes find it hard to plan some larger work due to too often
> releases (it's for a really large work, which can take several weeks
> to
> accomplish, then some more to fine tune in the real world usage). If
> you add this precedence, to always let someone ask: "oh, well, can I
> commit it? It's large, would anyone revert it? My change touches API
> and influences like 5+ projects, where I have only 3 under my direct
> control, thus the main commit is almost immediately followed by
> corresponding fixes in those projects I've under control, but some
> still can be broken", then I'm afraid the "hostile environment" would
> be a very nice word about GNOME and its infrastructure (people would
> use much much worse in the case you'd really make your proposal
> alive).

Milan, you have full commit access to every repo on git.gnome.org. Any
breakage that could register on Continuous is fully within your ability
to fix.

> This is also about different environments, you partly mentioned it. I
> believe most of the maintainers build before commit. I do, but I'm a
> human and mistakes happen. Some mistakes are independent of the build
> environment (I'm sorry for the before-Christmas breakage of the eds),
> some are not. I recall broken builds even in jhbuild when API changes
> happened (clean build was required to make it work, there was nothing
> wrong about the sources).

Yeah, we don't expect incremental builds to always work. There's just
no way to make that possible. We do expect clean builds to work.

> Even for example Ubuntu environment is different from that I use,
> causing undefined symbols on places which work just fine for me. What
> would you revert then in my project? I know, Continuous, it built an
> hour earlier, it should build now too.

The value of Continuous is that it is a neutral environment. I'd like
to set up additional jhbuild builders to do CI in Ubuntu, Fedora, Arch,
etc. so that we can catch build breakage in more environments.

> Anyway, doing any reverts might not be implicit, not without a
> previous
> discussion with the respective maintainers (unless they are not
> reachable), and only a day before the release - or on the Friday when
> the notification mail is sent, though for Weekend Contributors it
> would
> make more sense for Monday morning of the release day? I do not know,
> people can fix things just before the release too. It's because the
> GNOME should build from the release, not necessarily from the git
> snapshot.
> 
> The Continuous idea is great to discover build breaks early, but it's
> not an excuse to damage anyone's work.

Milan, I honestly feel that you might be better off using a "next"
branch if you really don't want any commits to be reverted. (That might
be the compromise solution here.) Or change jhbuild and gnome-
continuous to build your stable branch instead.

Michael
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Alberto Ruiz]

2016-01-22 16:55 GMT+00:00 Tristan Van Berkom :

> Of course, when you commit something and your *own* code does not
> compile, this is a very, very rare thing, rare enough to not be worth
> discussion I think, it that's happening, it's a problem.
>
> But it's common enough with many moving parts that the breakage is
> indeed intentional, in which case screwing with the history is only
> intrusive.
>

If you're breaking things intentionally, work on a wip/myfeature branch
until it doesn't break anymore? What's the big deal here?

If we want to keep the history clean  working on a wip branch that you can
rebase interactively squashing unmeaninful commits is what we should be
doing anyway.

-- 
Cheers,
Alberto Ruiz
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Shaun McCance]

On Fri, 2016-01-22 at 16:03 +0900, Tristan Van Berkom wrote:
> On Thu, 2016-01-21 at 14:54 +, Emmanuele Bassi wrote:
> [...]
> > This is not enough, and it does not raise the bar in keeping GNOME
> > in
> > a buildable state. It actually lowers it a fair , to the effective
> > point that *nobody* cares about Continuous builds.
> > 
> > I want this to change. I want to be able to revert failing commits
> > on
> > the offending modules, if they are hosted on GNOME infrastructure,
> > if
> > they fail for more than N hours, and *then* open a bug about it.
> 
> Hi Emmanuele,
> 
> In GNOME, we have got along well for many years with many projects
> sharing the same infrastructure mostly because we find that in
> practice, people are generally respectful enough of eachothers work
> to
> recognize a module maintainer's word as scripture. As such we do not
> commit changes to others modules without expressed permission unless
> it
> is for a minor change that we are absolutely sure about.

It has always been my understanding that master (née trunk née HEAD) is
supposed to be buildable in GNOME, and that the release team has the
power to make commits to ensure it is.

I am sympathetic to the difficulty of synchronizing cross-module API
changes. But is there any reason those changes can't all be done in
development branches in each module, with all of them merged to master
at the same time when they're ready?

>  There is/was a
> page describing this policy to those who receive commit access
> explaining the responsibility that comes with being a committer, it
> has
> either gone missing or I cannot currently find the text.
> 
> In that context, I admit that I first interpreted your proposal as
> hostile, at least in it's current form, because it violates this
> policy
> which has allowed our projects to exist in harmony for so many years
> under the GNOME umbrella, by awarding some kind of revert power to an
> arbitrary group of individuals. I can see much opportunity here for
> unnecessary dispute, if special care is not taken, we may risk
> incentivizing maintainers to host their projects elsewhere.
> 
> On the other hand, it is desirable to have things building regularly,
> of course the master branch is inherently unstable - that's what it's
> for, but I would however support your proposal without hesitation
> were
> it to be applied to stable branches only.
> 
> Were we to apply your proposed policy to master, I think we need
> further thought and clarification before carrying that out:
> 
>   o We are a volunteer driven project with contributors distributed
> across timezones, who have dayjobs, it is ridiculous to expect
> that
> maintainers can be reachable within a 3 hour turnaround period.
> 
> This leads me to believe that you fully expect to cause friction
> by
> reverting commits before said maintainers have the time to even
> respond, and doing so on the unstable master branch.
> 
> The master branch should ideally not be littered with reverts and
> reverts of reverts, it is the main history of project development
> and adding such noise to master should be avoided as much as
> possible for the purpose of preserving the integrity and value of
> the history.
> 
> I think this is unfriendly at best and I would prescribe a one
> week
> period for master. We should really wait for the maintainers word
> before intruding and reverting commits which introduce breakages
> which may have even been intentional.
> 
> 
>   o Not all libraries advertize API/ABI stability, especially when it
> comes to libraries whos only consumers are GNOME applications, we
> allow ourselves much more leniency here than with platform
> libraries which are also relevant outside of the GNOME ecosystem.
> 
> During the course of unstable development, it is entirely common
> for a library to introduce a new API and then to later change it,
> even more than once before it is ready for a stable release.
> For instance, EDS's backend facing APIs are not stable, and there
> have been regular issues in the vala API it exposes and libfolks
> consumes.
> 
> There are periods where it must be accepted that things are just
> broken in a transitional phase.
> 
> My interpretation of your proposal is that you would want to
> revert
> this api break or behavioral change if it were found to break the
> build (or the tests) of another module, during this unstable
> development period.
> 
> This would obviously be wrong and would just be an obstacle to
> progress in the cases where breakage is intentional and the only
> correct solution is to have applications adapt to the new API or
> behavior before the next stable release.
> 
> In summary, I am not opposed to applying your proposal as is to the
> stable builds, there is no justification *ever* for breakage in
> stable
> branches.
> 
> For master, I only think 

Re: Build sheriffs for GNOME

[Lasse Schuirmann]

One ideal end result IMO would be that some service similar to GNOME
continuous that validates any commit or patch ahead of time. If a
commit wasn't validated and somebody tried to push it to master, the
server should reject it. (GitLab does this implicitly with protected
branches and merge requests, GitHub lately got something similar too.)
I personally would love to have a bot that listens to bugzilla and
tests all patches automatically; also any pushed branch should be
tested automatically.

Testing code and buildability should be as simple as submitting or
pushing a patch.

2016-01-22 14:13 GMT+01:00 Alberto Ruiz :
> Another thing that would be really useful in this line is to be able to
> submit specific branches for testing. Say I'm working on wip/foobar to avoid
> breaking master, and before pulling that stuff from master I want to make
> sure I won't break the builds.
>
> Is anything like this possible at all?
>
> 2016-01-22 13:03 GMT+00:00 Simon McVittie :
>>
>> On 21/01/16 14:54, Emmanuele Bassi wrote:
>> > In short, I want to ensure that GNOME maintainers become
>> > a bit more proactive in giving a crap about their modules breaking on
>> > something that is not their own computers.
>>
>> Is there a way in which contributors/maintainers can reproduce something
>> equivalent to what Continuous would build (or a large enough subset of
>> the dependency stack to reach their packages), in a local virtual
>> machine where they can debug issues interactively?
>>
>> In the autobuild systems that I've worked with, I've found that the key
>> feature that makes them most useful is to avoid "magic" centralized
>> infrastructure that only a few sysadmins/maintainers can access.
>>
>> For instance, one of the things I really like about Debian's autopkgtest
>> initiative (a package-oriented and potentially system-level equivalent
>> of GNOME installed-tests) is that it's quite well-documented how to get
>> a local mirror of the environment in which tests are run on the official
>> infrastructure[1] and how to get something basically equivalent as a
>> virtual machine image[2].
>>
>> [1] https://ci.debian.net/doc/file.MAINTAINERS.html
>> [2] http://manpages.debian.org/cgi-bin/man.cgi?query=adt-virt-qemu
>>
>> --
>> Simon McVittie
>> Collabora Ltd. 
>>
>> ___
>> desktop-devel-list mailing list
>> desktop-devel-list@gnome.org
>> https://mail.gnome.org/mailman/listinfo/desktop-devel-list
>
>
>
>
> --
> Cheers,
> Alberto Ruiz
>
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Emmanuele Bassi]

Hi;

On 22 January 2016 at 13:03, Simon McVittie
 wrote:
> On 21/01/16 14:54, Emmanuele Bassi wrote:
>> In short, I want to ensure that GNOME maintainers become
>> a bit more proactive in giving a crap about their modules breaking on
>> something that is not their own computers.
>
> Is there a way in which contributors/maintainers can reproduce something
> equivalent to what Continuous would build (or a large enough subset of
> the dependency stack to reach their packages), in a local virtual
> machine where they can debug issues interactively?

Continuous generates VM images that you can easily install inside
Boxes, and keep them updated (they are based on OSTree and allow
atomic updates).

The latest successfully built image is always at:
http://build.gnome.org/continuous/buildmaster/images/z/current/

On build failures you get the full log and environment of what was
built and how: 
http://build.gnome.org/continuous/buildmaster/builds/2016/01/22/37/build/

If you want to, you can also run gnome-continuous on your local
machine: https://git.gnome.org/browse/gnome-continuous/tree/README-build.md

Ciao,
 Emmanuele.

-- 
https://www.bassi.io
[@] ebassi [@gmail.com]
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Alberto Ruiz]

Another thing that would be really useful in this line is to be able to
submit specific branches for testing. Say I'm working on wip/foobar to
avoid breaking master, and before pulling that stuff from master I want to
make sure I won't break the builds.

Is anything like this possible at all?

2016-01-22 13:03 GMT+00:00 Simon McVittie :

> On 21/01/16 14:54, Emmanuele Bassi wrote:
> > In short, I want to ensure that GNOME maintainers become
> > a bit more proactive in giving a crap about their modules breaking on
> > something that is not their own computers.
>
> Is there a way in which contributors/maintainers can reproduce something
> equivalent to what Continuous would build (or a large enough subset of
> the dependency stack to reach their packages), in a local virtual
> machine where they can debug issues interactively?
>
> In the autobuild systems that I've worked with, I've found that the key
> feature that makes them most useful is to avoid "magic" centralized
> infrastructure that only a few sysadmins/maintainers can access.
>
> For instance, one of the things I really like about Debian's autopkgtest
> initiative (a package-oriented and potentially system-level equivalent
> of GNOME installed-tests) is that it's quite well-documented how to get
> a local mirror of the environment in which tests are run on the official
> infrastructure[1] and how to get something basically equivalent as a
> virtual machine image[2].
>
> [1] https://ci.debian.net/doc/file.MAINTAINERS.html
> [2] http://manpages.debian.org/cgi-bin/man.cgi?query=adt-virt-qemu
>
> --
> Simon McVittie
> Collabora Ltd. 
>
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list
>



-- 
Cheers,
Alberto Ruiz
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Emmanuele Bassi]

Hi Alberto;

you're asking for a try server, which is a fairly complex piece of machinery.

Mozilla has one, based on buildbot; it may be possible to have a
machine dedicated to try builds, but that's not what Continuous does,
and it would not be what Continuous aims to be.

Ciao,
 Emmanuele.


On 22 January 2016 at 13:13, Alberto Ruiz  wrote:
> Another thing that would be really useful in this line is to be able to
> submit specific branches for testing. Say I'm working on wip/foobar to avoid
> breaking master, and before pulling that stuff from master I want to make
> sure I won't break the builds.
>
> Is anything like this possible at all?
>
> 2016-01-22 13:03 GMT+00:00 Simon McVittie :
>>
>> On 21/01/16 14:54, Emmanuele Bassi wrote:
>> > In short, I want to ensure that GNOME maintainers become
>> > a bit more proactive in giving a crap about their modules breaking on
>> > something that is not their own computers.
>>
>> Is there a way in which contributors/maintainers can reproduce something
>> equivalent to what Continuous would build (or a large enough subset of
>> the dependency stack to reach their packages), in a local virtual
>> machine where they can debug issues interactively?
>>
>> In the autobuild systems that I've worked with, I've found that the key
>> feature that makes them most useful is to avoid "magic" centralized
>> infrastructure that only a few sysadmins/maintainers can access.
>>
>> For instance, one of the things I really like about Debian's autopkgtest
>> initiative (a package-oriented and potentially system-level equivalent
>> of GNOME installed-tests) is that it's quite well-documented how to get
>> a local mirror of the environment in which tests are run on the official
>> infrastructure[1] and how to get something basically equivalent as a
>> virtual machine image[2].
>>
>> [1] https://ci.debian.net/doc/file.MAINTAINERS.html
>> [2] http://manpages.debian.org/cgi-bin/man.cgi?query=adt-virt-qemu
>>
>> --
>> Simon McVittie
>> Collabora Ltd. 
>>
>> ___
>> desktop-devel-list mailing list
>> desktop-devel-list@gnome.org
>> https://mail.gnome.org/mailman/listinfo/desktop-devel-list
>
>
>
>
> --
> Cheers,
> Alberto Ruiz
>
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list



-- 
https://www.bassi.io
[@] ebassi [@gmail.com]
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Emmanuele Bassi]

Hi;

On 21 January 2016 at 18:19, Michael Catanzaro  wrote:

> The next step is to convert either jhbuild or gnome-continuous to use
> the others' modulesets. Currently the jhbuild modulesets are the
> official "what we release,"

"What we release" and "what we actually run" are two fairly difference concepts.

I'd contend that the GNOME Continuous manifest is actually much more
close to what distributions ship, and thus what we run on a daily
basis, than what we put into the configure arguments when we
distcheck, or the arguments we put into the moduleset. Mostly, because
nobody, except the release team, smoketest a system composed by
jhbuild, whereas Continuous does smoketesting on a fully composed file
system image.

Continuous also builds the whole stack down to systemd, whereas
JHBuild does not.

JHBuild is meant to be assisted and attended by a human being;
Continuous is mostly meant for automation.

There's also the issue that the JHBuild modulesets cater to a fairly
different crowd, and have flexibility to build on OSX or Windows, with
conditionals and suggested packages.

We could make JHBuild read the Continuous manifest, and start from a
specific module, but the two systems are *not* interchangeable. Each
one has its own requirements and goals, and confusing the two is going
to leave use cases out.

We should certainly ensure that what goes into JHBuild also goes into
Continuous, and possibly with minimal fuss. Ideally, we should move
the JHBuild modulesets and the Continuous manifest out of their
respective repositories, and into a shared location.

> and they are more important because they
> affect the ability of new contributors to build GNOME, so that's what I
> focus on maintaining. Continuous is useful to catch many build
> failures, but it misses many, many more failures than could only be
> detected were it to use the official modulesets.

If Continuous fails to catch build failures then you should update the
manifest until you trigger them. That's the whole point of having
automated builds.

> Continuous is a great first step (the response time is *stunningly*
> good) and we're better off now that we have it, but it's not good
> enough as it's not testing the build tool we use for development, it's
> not testing what we release.

If you're releasing something different than what you have in master
then you're doing something impressively wrong.

You (and by "you" I mean "maintainers") should check what flags and
dependencies are listed in the JHBuild moduleset and which ones are in
the Continuous manifest, and try to make them close or identical.

Bugs will happen, and I consider keeping the Continuous manifest up to
date also part of the job description of a build sheriff.

>>  Many are not even on
>> #gnome-hackers, which means they miss the notification that the
>> something broke the build.
>
> Anecdote: I am on #gnome-hackers so you can ping me, but I don't turn
> on notifications for every message in the room (too many!),

The IRC bot does not notify you personally. You can add a notification
on a specific substring, like 'continuous:build * failed'; or you
could just look at the red message that says that the build failed.
:-)

> so it's
> rare that I notice a failure in one of my modules unless I happen to be
> looking at #gnome-hackers at the time, or someone else pings me. It
> would be really awesome if Continuous could learn to guess who to blame
> for breakage and include the IRC name in the channel;

As I said, it can be incredibly hard to determine that from an
automation standpoint.

Ciao,
 Emmanuele.

-- 
https://www.bassi.io
[@] ebassi [@gmail.com]
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Michael Catanzaro]

On Fri, 2016-01-22 at 16:03 +0900, Tristan Van Berkom wrote:
>   o We are a volunteer driven project with contributors distributed
>     across timezones, who have dayjobs, it is ridiculous to expect
> that
>     maintainers can be reachable within a 3 hour turnaround period.
> 
> This leads me to believe that you fully expect to cause friction
> by
>     reverting commits before said maintainers have the time to even
>     respond, and doing so on the unstable master branch.
> 
> The master branch should ideally not be littered with reverts and
>     reverts of reverts, it is the main history of project development
> and adding such noise to master should be avoided as much as
>     possible for the purpose of preserving the integrity and value of
>     the history.

I really don't think it's a big deal to have a few reverts in the git
history. And anyway, reverts should be relatively rare, because build
breakage should be relatively rare.

The master branch may be unstable, but that doesn't mean build breakage
is ever acceptable. If your code is not buildable at all times in
Continuous, keep it on a *side branch* -- otherwise, tag your module in
Continuous, branch it in jhbuild, and do whatever you want in master.
But as long as you have an official GNOME module, if it's not branched
and tagged, master needs to build. This isn't the wild west.


> My interpretation of your proposal is that you would want to
> revert
>     this api break or behavioral change if it were found to break the
>     build (or the tests) of another module, during this unstable
>     development period.
> 
> This would obviously be wrong and would just be an obstacle to
> progress in the cases where breakage is intentional and the only
>     correct solution is to have applications adapt to the new API or
>     behavior before the next stable release.

It's a big problem that some modules break for days or weeks at a time
during API transitions; this makes it very difficult for new
contributors to build and run GNOME, and it's inconvenient for
experienced contributors as well. We need to land these transitions
better, by ensuring either that they occur in side branches, where all
modules' side branches are merged into master at the same time, or else
by tagging the relevant modules in Continuous and branching them in
jhbuild until the transition is completed.

To be clear: if you've branched and tagged your module, then you do
whatever you want in master without hurting anyone else. But otherwise,
master needs to be buildable.

Michael
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Tristan Van Berkom]

On Fri, 2016-01-22 at 10:37 -0600, Michael Catanzaro wrote:
[...]
> I really don't think it's a big deal to have a few reverts in the git
> history. And anyway, reverts should be relatively rare, because build
> breakage should be relatively rare.
> 
> The master branch may be unstable, but that doesn't mean build
> breakage
> is ever acceptable. If your code is not buildable at all times in
> Continuous, keep it on a *side branch* -- otherwise, tag your module
> in
> Continuous, branch it in jhbuild, and do whatever you want in master.
> But as long as you have an official GNOME module, if it's not
> branched
> and tagged, master needs to build. This isn't the wild west.

Of course, when you commit something and your *own* code does not
compile, this is a very, very rare thing, rare enough to not be worth
discussion I think, it that's happening, it's a problem.

But it's common enough with many moving parts that the breakage is
indeed intentional, in which case screwing with the history is only
intrusive.

> 
> > My interpretation of your proposal is that you would want to
> > revert
> >     this api break or behavioral change if it were found to break
> > the
> >     build (or the tests) of another module, during this unstable
> >     development period.
> > 
> > This would obviously be wrong and would just be an obstacle to
> > progress in the cases where breakage is intentional and the
> > only
> >     correct solution is to have applications adapt to the new API
> > or
> >     behavior before the next stable release.
> 
> It's a big problem that some modules break for days or weeks at a
> time
> during API transitions; this makes it very difficult for new
> contributors to build and run GNOME, and it's inconvenient for
> experienced contributors as well.

This kind of breakage is just a fact of life, you cannot sweep it under
the rug and pretend it does not exist.

Sure, it's a barrier of entry for newcomers and an annoyance for
experienced developers who realize the facts of life. But that's just
the way it is, there is no reason to complain about it.

Software development is difficult, integration is difficult when you
have many moving parts that are in a development stage before being
release ready.

[...]
> To be clear: if you've branched and tagged your module, then you do
> whatever you want in master without hurting anyone else. But
> otherwise,
> master needs to be buildable.

And to be doubly clear: the master branch is *not* what is advertised
by upstream maintainers as stable, it is the bleeding edge, it can have
churn, it may not always match up to other modules perfectly, other
modules may have not yet adapted to the churn. Building master may be
asking for trouble, again, this is just a fact of life if you are
trying to integrate with all bleeding edge component in the middle of
their development cycle, instead of what maintainers have published as
stable.

If you want something that you are sure will build, take the stable
release tag and build that.

Best,
    -Tristan

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Shaun McCance]

On Fri, 2016-01-22 at 13:31 +, Emmanuele Bassi wrote:
> Hi;
> 

> > Continuous is a great first step (the response time is *stunningly*
> > good) and we're better off now that we have it, but it's not good
> > enough as it's not testing the build tool we use for development,
> > it's
> > not testing what we release.
> 
> If you're releasing something different than what you have in master
> then you're doing something impressively wrong.

Surely s/master/master or a stable release branch/ ?

--
Shaun

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Michael Catanzaro]

On Fri, 2016-01-22 at 13:38 +, Emmanuele Bassi wrote:
> Continuous generates VM images that you can easily install inside
> Boxes, and keep them updated (they are based on OSTree and allow
> atomic updates).
> 
> The latest successfully built image is always at:
> http://build.gnome.org/continuous/buildmaster/images/z/current/

Would be awesome to advertise this on the build.gnome.org landing page,
so people can find it.
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Michael Catanzaro]

To be clear on what I'd like to see:

The most important requirement for our CI system is to run 'jhbuild
build' for each module with a clean install prefix, to find missing
dependencies in our moduleset. Not having to maintain jhbuild
modulesets separate from the Continuous manifest would be good too.

Michael
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Jasper St. Pierre]

It's more frequent than you might think.

In the past week, alone, we've had to tag glib, gnome-calculator,
e-d-s, gstreamer, and more, all because of broken builds. Take a look
at how busy gnome-continuous is as a project:
https://git.gnome.org/browse/gnome-continuous/log/

We currently have a team of people (usually Michael, Emmanuele, Colin
and Javier) who babysit gnome-continuous to file bugs for broken
components, tag the relevant component in GNOME Continuous, but it's
not a great workflow, in that it doesn't tend to keep master
buildable.

The first mail didn't really say how often build breakages in GNOME
actually happen -- they happen *a lot*.

On Fri, Jan 22, 2016 at 8:55 AM, Tristan Van Berkom
 wrote:
> On Fri, 2016-01-22 at 10:37 -0600, Michael Catanzaro wrote:
> [...]
>> I really don't think it's a big deal to have a few reverts in the git
>> history. And anyway, reverts should be relatively rare, because build
>> breakage should be relatively rare.
>>
>> The master branch may be unstable, but that doesn't mean build
>> breakage
>> is ever acceptable. If your code is not buildable at all times in
>> Continuous, keep it on a *side branch* -- otherwise, tag your module
>> in
>> Continuous, branch it in jhbuild, and do whatever you want in master.
>> But as long as you have an official GNOME module, if it's not
>> branched
>> and tagged, master needs to build. This isn't the wild west.
>
> Of course, when you commit something and your *own* code does not
> compile, this is a very, very rare thing, rare enough to not be worth
> discussion I think, it that's happening, it's a problem.
>
> But it's common enough with many moving parts that the breakage is
> indeed intentional, in which case screwing with the history is only
> intrusive.
>
>>
>> > My interpretation of your proposal is that you would want to
>> > revert
>> > this api break or behavioral change if it were found to break
>> > the
>> > build (or the tests) of another module, during this unstable
>> > development period.
>> >
>> > This would obviously be wrong and would just be an obstacle to
>> > progress in the cases where breakage is intentional and the
>> > only
>> > correct solution is to have applications adapt to the new API
>> > or
>> > behavior before the next stable release.
>>
>> It's a big problem that some modules break for days or weeks at a
>> time
>> during API transitions; this makes it very difficult for new
>> contributors to build and run GNOME, and it's inconvenient for
>> experienced contributors as well.
>
> This kind of breakage is just a fact of life, you cannot sweep it under
> the rug and pretend it does not exist.
>
> Sure, it's a barrier of entry for newcomers and an annoyance for
> experienced developers who realize the facts of life. But that's just
> the way it is, there is no reason to complain about it.
>
> Software development is difficult, integration is difficult when you
> have many moving parts that are in a development stage before being
> release ready.
>
> [...]
>> To be clear: if you've branched and tagged your module, then you do
>> whatever you want in master without hurting anyone else. But
>> otherwise,
>> master needs to be buildable.
>
> And to be doubly clear: the master branch is *not* what is advertised
> by upstream maintainers as stable, it is the bleeding edge, it can have
> churn, it may not always match up to other modules perfectly, other
> modules may have not yet adapted to the churn. Building master may be
> asking for trouble, again, this is just a fact of life if you are
> trying to integrate with all bleeding edge component in the middle of
> their development cycle, instead of what maintainers have published as
> stable.
>
> If you want something that you are sure will build, take the stable
> release tag and build that.
>
> Best,
> -Tristan
>
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list



-- 
  Jasper
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Michael Catanzaro]

On Sat, 2016-01-23 at 01:55 +0900, Tristan Van Berkom wrote:
> This kind of breakage is just a fact of life, you cannot sweep it
> under
> the rug and pretend it does not exist.
>
> Sure, it's a barrier of entry for newcomers and an annoyance for
> experienced developers who realize the facts of life. But that's just
> the way it is, there is no reason to complain about it.
> 
> Software development is difficult, integration is difficult when you
> have many moving parts that are in a development stage before being
> release ready.

Of course we can't pretend it does not exist, but we certainly can work
on side branches to keep things building. It's pretty easy and it does
solve the problem

> And to be doubly clear: the master branch is *not* what is advertised
> by upstream maintainers as stable, it is the bleeding edge, it can
> have
> churn, it may not always match up to other modules perfectly, other
> modules may have not yet adapted to the churn. Building master may be
> asking for trouble, again, this is just a fact of life if you are
> trying to integrate with all bleeding edge component in the middle of
> their development cycle, instead of what maintainers have published
> as
> stable.
> 
> If you want something that you are sure will build, take the stable
> release tag and build that.

But master is what other GNOME contributors need to be able to build in
order to contribute to your module or any modules that depend on your
module. I would very much appreciate it if you would work in a
sidebranch if you need to make a change you know will break the build
for some module.

Thanks,

Michael
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Build sheriffs for GNOME

[Michael Catanzaro]

On Fri, 2016-01-22 at 09:25 -0800, Jasper St. Pierre wrote:
> It's more frequent than you might think.
> 
> In the past week, alone, we've had to tag glib, gnome-calculator,
> e-d-s, gstreamer, and more, all because of broken builds. Take a look
> at how busy gnome-continuous is as a project:
> https://git.gnome.org/browse/gnome-continuous/log/

Absolutely. I should be more clear: it doesn't happen so often for any
PARTICULAR module, so the git history of any particular module is not
likely to get polluted by many revert commits. (And even if it does --
so what if your history doesn't look quite as nice as you'd like.) But
it happens a LOT for GNOME as a whole, and it's a big problem, which is
why I'm so supportive of Emmanuele's plan.
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list