Re: How to chainbuild in a build override?

[Stephen Gallagher]

On Fri, Sep 22, 2017 at 3:08 AM Ralf Corsepius  wrote:

> Hi,
>
> I am trying to build a chain of packages in a build override?
>
> I.e. a series of packages: A->B->C
>
> I set up a build override for A, and B built successfully. Now, I would
> have expect building C to pickup B from the A-override.
>
> This does not seem to apply. C fails to build, apparently because
> building doesn't pickup "B" from A's build override.
>
> Is this a timing issue (Do I need to wait)? Do I have to setup another
> build-override for B?
>
>

This is actually one of the places where the Modularity initiative really
shines. With the module metadata, you specify all of the source packages
that your module is going to need, along with an optional "buildorder"
value that is basically a chainbuild indicator. All things with buildorder
== 0 (or buildorder unspecified) will build roughly in parallel (as
builders are available), then will generate a repo to supplement the
buildroot for things with buildorder == 1 and so on.

So in essence, every module build is capable of being a chainbuild.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: How to chainbuild in a build override?

[Kevin Kofler]

Neal Gompa wrote:
> Each tag is a full repository that Koji must track and consume for
> generating buildroots. I don't know if the contents go away when they
> get merged into the main tag (I don't think they do, if my
> understanding of Koji is correct), so it just adds to the pressure for
> builders to resolve and the storage backend to hold.
> 
> It's similar to how COPR and OBS maintain all the projects' individual
> repositories. And in OBS, submitting packages from one project to
> another does not eliminate the old project unless you request it in
> the submission.
> 
> Both Koji and OBS have a garbage collection process, but as far as I
> know, that's run manually...

That is only the storage part, which is not the biggest issue. The biggest 
issue is that each time a package is pushed for the standard buildroot 
(which for Rawhide means as soon as it is built and autosigned, whereas for 
releases, it means as soon as it is pushed to stable or filed as a buildroot 
override), Koji has to regenerate the repo not only for the standard 
buildroot, but also for all the custom ones that are based on it. So this 
multiplies the number of newRepo tasks that it has to run each time. That 
said, this cost goes away as soon as the custom tag is deleted, which is why 
you should request it deleted as soon as you no longer need it (in 
principle, you can delete the custom tag as soon as you tagged all the built 
packages into the appropriate f*-updates-candidate tag, but at the latest, 
you should delete it when your updates went stable).

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: How to chainbuild in a build override?

[Neal Gompa]

On Sun, Sep 24, 2017 at 9:02 AM, Till Hofmann
 wrote:
>
>
> On 09/23/2017 02:05 AM, Kevin Kofler wrote:
>>
>> Petr Pisar wrote:
>>>
>>> In rawhide, if you want a longer time isolation, you ask relengs for
>>> a sige tag, you do all builds there without affecting others and then
>>> you ask relengs to merge the builds back to rawhide. Of course this has
>>> same races and one needs to reaply all rawhide changes into the side tag
>>> otherwise a mismatches can happen after the merge.
>>>
>>> I don't know if this procedure is acceptable for f27.
>>
>>
>> It is possible to do this (request a side tag) in a release. The KDE SIG
>> frequently does this for larger updates, the GNOME team sometimes does
>> too.
>>
>> That said, side tags do put a strain on the infrastructure and thus should
>> only be used where it makes sense.
>>
>
> Why is that? Asking purely out of curiosity.

Each tag is a full repository that Koji must track and consume for
generating buildroots. I don't know if the contents go away when they
get merged into the main tag (I don't think they do, if my
understanding of Koji is correct), so it just adds to the pressure for
builders to resolve and the storage backend to hold.

It's similar to how COPR and OBS maintain all the projects' individual
repositories. And in OBS, submitting packages from one project to
another does not eliminate the old project unless you request it in
the submission.

Both Koji and OBS have a garbage collection process, but as far as I
know, that's run manually...


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: How to chainbuild in a build override?

[Till Hofmann]

On 09/23/2017 02:05 AM, Kevin Kofler wrote:

Petr Pisar wrote:

In rawhide, if you want a longer time isolation, you ask relengs for
a sige tag, you do all builds there without affecting others and then
you ask relengs to merge the builds back to rawhide. Of course this has
same races and one needs to reaply all rawhide changes into the side tag
otherwise a mismatches can happen after the merge.

I don't know if this procedure is acceptable for f27.


It is possible to do this (request a side tag) in a release. The KDE SIG
frequently does this for larger updates, the GNOME team sometimes does too.

That said, side tags do put a strain on the infrastructure and thus should
only be used where it makes sense.



Why is that? Asking purely out of curiosity.

Regards,
Till
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: How to chainbuild in a build override?

[Kevin Kofler]

Petr Pisar wrote:
> In rawhide, if you want a longer time isolation, you ask relengs for
> a sige tag, you do all builds there without affecting others and then
> you ask relengs to merge the builds back to rawhide. Of course this has
> same races and one needs to reaply all rawhide changes into the side tag
> otherwise a mismatches can happen after the merge.
> 
> I don't know if this procedure is acceptable for f27.

It is possible to do this (request a side tag) in a release. The KDE SIG 
frequently does this for larger updates, the GNOME team sometimes does too.

That said, side tags do put a strain on the infrastructure and thus should 
only be used where it makes sense.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: How to chainbuild in a build override?

[Ralf Corsepius]

On 09/22/2017 01:07 PM, Petr Pisar wrote:

On 2017-09-22, Ralf Corsepius  wrote:

How to build a sub tree of packages in fc27, when the root of this tree
changed SONAME?


In rawhide, if you want a longer time isolation, you ask relengs for
a sige tag, you do all builds there without affecting others and then
you ask relengs to merge the builds back to rawhide. Of course this has
same races and one needs to reaply all rawhide changes into the side tag
otherwise a mismatches can happen after the merge.


This certainly makes sense for larger number of packages.


I don't know if this procedure is acceptable for f27.

I would not want to do this.

However, I meanwhile seem to have resolved my problem.

The trick was to use several build overrides, one for each "tree level" 
of the package tree being involved.


In this case, the tree was 2 levels deep, comprising 6 packages, with a 
dependency graph like this:


A1 -> B1
   -> B2
   -> B3 -> C1
 -> C2

One build override on "A1" and one build override on "B3".

As Tom said, painful, but doable due to the fairly small size of 
packages being involved ;)



In my opinion the whole idea of chaning a SONAME in f27 is wrong.


Nah, it's just a more or less a last minute update (Remember F27 is 
still in testing!), which seemed useful because A1's upstream released 
the first major bugfix update for ca. 2 years. "Just in time"/"last 
minute" to make it into Fedora.


In former Fedora times, I would have updated A1 by brute force and then 
gradually have let the other packages catch up.


Ralf
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: How to chainbuild in a build override?

[Petr Pisar]

On 2017-09-22, Ralf Corsepius  wrote:
> How to build a sub tree of packages in fc27, when the root of this tree 
> changed SONAME?
>
In rawhide, if you want a longer time isolation, you ask relengs for
a sige tag, you do all builds there without affecting others and then
you ask relengs to merge the builds back to rawhide. Of course this has
same races and one needs to reaply all rawhide changes into the side tag
otherwise a mismatches can happen after the merge.

I don't know if this procedure is acceptable for f27.

In my opinion the whole idea of chaning a SONAME in f27 is wrong.

-- Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: How to chainbuild in a build override?

[Tom Hughes]

On 22/09/17 08:50, Ralf Corsepius wrote:

How to build a sub tree of packages in fc27, when the root of this tree 
changed SONAME?


Well it's painful - you have to build each one then add an override for 
it and wait for that to appear and then move on to the next one etc...


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: How to chainbuild in a build override?

[Ralf Corsepius]

On 09/22/2017 09:32 AM, Tom Hughes wrote:

On 22/09/17 08:03, Ralf Corsepius wrote:


I am trying to build a chain of packages in a build override?

I.e. a series of packages: A->B->C

I set up a build override for A, and B built successfully. Now, I 
would have expect building C to pickup B from the A-override.


You seem to be confused about what an override is ;-)


Quite likely, I rarely used them and have always found the tooling to be 
hard to use ;)


It's not a separate environment just for you - if it was then you would 
have to say which override you wanted to use when building.

Correct. That's a issue I never understood about build overrides.

Rather it just adds that package to the global build environment that 
everybody sees.


But in order for B to appear in the build environment for C to use you 
would need to add an override for B and then wait for it to appear 
there.


That's what I had presumed. But this implies me to deliberately break deps.

I believe chainbuild knows how to do the waiting but not how to 
request an override.


So in summary chainbuild is I believe only useful in rawhide, and in 
branched before bodhi is enabled.
That's clear. I was using the term chain-building in the meaning of 
building a hierarchy of packages.


So let me rephrase my question:

How to build a sub tree of packages in fc27, when the root of this tree 
changed SONAME?



Ralf
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: How to chainbuild in a build override?

[Tom Hughes]

On 22/09/17 08:03, Ralf Corsepius wrote:


I am trying to build a chain of packages in a build override?

I.e. a series of packages: A->B->C

I set up a build override for A, and B built successfully. Now, I would 
have expect building C to pickup B from the A-override.


You seem to be confused about what an override is ;-)

It's not a separate environment just for you - if it was then you would 
have to say which override you wanted to use when building.


Rather it just adds that package to the global build environment that 
everybody sees.


But in order for B to appear in the build environment for C to use you 
would need to add an override for B and then wait for it to appear 
there. I believe chainbuild knows how to do the waiting but not how to 
request an override.


So in summary chainbuild is I believe only useful in rawhide, and in 
branched before bodhi is enabled.


Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org