Re: CI Requirements - Lessons Not Learnt?

2017-01-06 Thread Scarlett Clark
Hello all,
First my apologies, yes the CI system is behind, I had done significant
work on a new revamp and then life, work and those other incredibly
annoying things got in the way. The CI team is rather small and we are
trying our best to move things forward. We are now getting back on track,
but it will still be some time to get this out the door. The PIM team has
worked with me directly and we have had very good luck getting their
requirements satisfied working together. If anyone needs something extra,
like a dependency built from source, just ask. Despite my best efforts,
mailing lists, I simply can't keep up. Ping me, directly CC me, telegram
me, anything. I am always accommodating. Again, sorry this has all come to
this point.
Thank you,
Scarlett


On Thu, Jan 5, 2017 at 12:44 AM, Ben Cooksley  wrote:

> Hi all,
>
> It seems that my previous vocal complaints about system level /
> serious impact dependency bumps on the CI system have gone completely
> unnoticed by (some) members of our Community.
>
> This was demonstrated earlier this week when components of Plasma
> bumped their version requirements for XKBCommon and Appstream-Qt -
> without even a thought about notifying Sysadmin or checking which
> version the CI had, until their builds broke.
>
> Neither of these is easy to fix at this stage, as the system base is
> now too old to receive updates such as these. Base upgrades require a
> full rebuild of everything on the CI system, and usually involve
> significant additional churn and is a process that must be done
> roughly twice a year, depending on dependency bump demands.
>
> Does anyone have any suggestions as to how we may avoid this in the future?
>
> At this point i'm in favour of if you don't follow the rules your
> dependency bump just gets reverted out of existence, then you get to
> go through the process properly...
>
> Regards,
> Ben
>


Re: CI Requirements - Lessons Not Learnt?

2017-01-06 Thread David Edmundson
>This was demonstrated earlier this week when components of Plasma
bumped their version requirements for XKBCommon and Appstream-Qt

FWIW, Appstream-Qt became optional later that same day, because of how new
the dependency was. Though obviously it could have been hanlded better.

Does anyone have any suggestions as to how we may avoid this in the future?
>
>
Yes

Is there a way to find out what version of libFoo is available on the CI
without pestering a  sysadmin?

At the moment if I do bump a version (or review a version bump) my best
option is simply to try it and find out.


Re: CI Requirements - Lessons Not Learnt?

2017-01-06 Thread Ben Cooksley
On Fri, Jan 6, 2017 at 9:52 PM, Martin Gräßlin  wrote:
> Am 2017-01-06 05:57, schrieb Ben Cooksley:
>>
>> On Fri, Jan 6, 2017 at 12:38 AM, Martin Gräßlin 
>> wrote:
>>>
>>> Am 2017-01-05 11:20, schrieb Ben Cooksley:


 On Thu, Jan 5, 2017 at 10:28 PM, Martin Gräßlin
  wrote:
>
>
> Am 2017-01-05 09:44, schrieb Ben Cooksley:
>>
>>
>>
>> Hi all,
>>
>> It seems that my previous vocal complaints about system level /
>> serious impact dependency bumps on the CI system have gone completely
>> unnoticed by (some) members of our Community.
>>
>> This was demonstrated earlier this week when components of Plasma
>> bumped their version requirements for XKBCommon and Appstream-Qt -
>> without even a thought about notifying Sysadmin or checking which
>> version the CI had, until their builds broke.
>>
>> Neither of these is easy to fix at this stage, as the system base is
>> now too old to receive updates such as these. Base upgrades require a
>> full rebuild of everything on the CI system, and usually involve
>> significant additional churn and is a process that must be done
>> roughly twice a year, depending on dependency bump demands.
>>
>> Does anyone have any suggestions as to how we may avoid this in the
>> future?
>
>
>
>
> I have a few questions here:
>
> 1) Where is this requirement to check with sysadmins codified? So far I
> was
> only aware of dependency freeze.



 It's been codified since the PIM Qt 5.6 / WebEngine debacle, where
 Sysadmin had to rush delivery of Qt 5.6 to the CI system because the
 whole of PIM broke when they started using QtWebEngine. That was
 around March/April 2016, my mail archives can't seem to find the exact
 thread though.
>>>
>>>
>>>
>>> I'm sorry Ben, but I fear "sending out a mail about an issue with PIM"
>>> doesn't
>>> qualify as codifying it. Given what we have it looks like this did not
>>> reach
>>> the
>>> target audience. And neither will this thread.
>>
>>
>> Uhm, it was far more than a single email. It was quite the thread, and
>> if memory serves was on at least one of the mailing lists this thread
>> was on.
>> One of the key things that came out of that thread was to ask for
>> things to be bumped well in advance if a newer version is needed.
>>
>> I'm disappointed that you think that email threads have insufficient
>> reach given it's one of our communities principal means of
>> communication.
>
>
> Email threads don't work to codify such requirements. What we need is
> something like an "announce new dependency to sysadmin freeze" prior to
> the dependency freeze in the release schedule. That's what I mean with
> codifying it. We need to have it in a way where devs actually check.
> It needs to be part of the process. An old email thread cannot be part of
> the process.

Announcing new dependencies to Sysadmin as part of a release schedule
doesn't really work... because dependencies can be bumped at any time
(as long as a dependency freeze is not in effect for a project). Also,
CI builds for far more than just Plasma, including for software that
doesn't have a formal release schedule, including software in
Extragear. CI also doesn't really care for release schedules at all,
it just builds the latest state of the repository.

If you want to specify a time it has to be X before the commits
introducing the dependency land.

>
>>
>>>
>>> This needs to change the process, the way KDE develops software. It needs
>>> to
>>> be
>>> listed in the release schedule (is not, I checked), maybe reviews need to
>>> be
>>> acked by release managers, etc.
>>
>>
>> It seems to be part of the process for many others already, so not
>> sure what needs to change. The gpgme transition went quite well for
>> PIM, and other applications developers have reached out and asked
>> about version upgrades to various libraries which were in their area
>> of interest which we have sorted out easily enough.
>
>
> See above. Part of release schedule.
>
>
>>
>>>

> 2) How can we easily check what build.kde.org has? Looking at cmake
> output
> is not a sufficient way as it gives me wrong information



 If CMake is outputting wrong information, then your CMakeLists.txt
 can't make the appropriate decisions as to whether the available
 version is suitable, so i'd say you've got bigger problems here that
 need to be addressed first.
>>>
>>>
>>>
>>> Cmake feature summary says: "required version >= 0.5" and that's for all
>>> found
>>> depeendencies. Unfortunately no information at all in the feature summary
>>> about
>>> the actual version.
>>
>>
>> Quoting the KWin CMake log...
>>
>> 15:08:41 -- Found Wayland_Client:
>> /usr/lib/x86_64-linux-gnu/libwayland-client.so (found version
>> "1.12.90")
>> 15:08:41 -- Found Wayland_Cursor:
>> /usr/lib/x86_64-linux-gnu/libwayland-curso

Re: CI Requirements - Lessons Not Learnt?

2017-01-06 Thread Martin Gräßlin

Am 2017-01-06 05:57, schrieb Ben Cooksley:
On Fri, Jan 6, 2017 at 12:38 AM, Martin Gräßlin  
wrote:

Am 2017-01-05 11:20, schrieb Ben Cooksley:


On Thu, Jan 5, 2017 at 10:28 PM, Martin Gräßlin
 wrote:


Am 2017-01-05 09:44, schrieb Ben Cooksley:



Hi all,

It seems that my previous vocal complaints about system level /
serious impact dependency bumps on the CI system have gone 
completely

unnoticed by (some) members of our Community.

This was demonstrated earlier this week when components of Plasma
bumped their version requirements for XKBCommon and Appstream-Qt -
without even a thought about notifying Sysadmin or checking which
version the CI had, until their builds broke.

Neither of these is easy to fix at this stage, as the system base 
is
now too old to receive updates such as these. Base upgrades require 
a

full rebuild of everything on the CI system, and usually involve
significant additional churn and is a process that must be done
roughly twice a year, depending on dependency bump demands.

Does anyone have any suggestions as to how we may avoid this in the
future?




I have a few questions here:

1) Where is this requirement to check with sysadmins codified? So 
far I

was
only aware of dependency freeze.



It's been codified since the PIM Qt 5.6 / WebEngine debacle, where
Sysadmin had to rush delivery of Qt 5.6 to the CI system because the
whole of PIM broke when they started using QtWebEngine. That was
around March/April 2016, my mail archives can't seem to find the 
exact

thread though.



I'm sorry Ben, but I fear "sending out a mail about an issue with PIM"
doesn't
qualify as codifying it. Given what we have it looks like this did not 
reach

the
target audience. And neither will this thread.


Uhm, it was far more than a single email. It was quite the thread, and
if memory serves was on at least one of the mailing lists this thread
was on.
One of the key things that came out of that thread was to ask for
things to be bumped well in advance if a newer version is needed.

I'm disappointed that you think that email threads have insufficient
reach given it's one of our communities principal means of
communication.


Email threads don't work to codify such requirements. What we need is
something like an "announce new dependency to sysadmin freeze" prior to
the dependency freeze in the release schedule. That's what I mean with
codifying it. We need to have it in a way where devs actually check.
It needs to be part of the process. An old email thread cannot be part 
of

the process.





This needs to change the process, the way KDE develops software. It 
needs to

be
listed in the release schedule (is not, I checked), maybe reviews need 
to be

acked by release managers, etc.


It seems to be part of the process for many others already, so not
sure what needs to change. The gpgme transition went quite well for
PIM, and other applications developers have reached out and asked
about version upgrades to various libraries which were in their area
of interest which we have sorted out easily enough.


See above. Part of release schedule.








2) How can we easily check what build.kde.org has? Looking at cmake
output
is not a sufficient way as it gives me wrong information



If CMake is outputting wrong information, then your CMakeLists.txt
can't make the appropriate decisions as to whether the available
version is suitable, so i'd say you've got bigger problems here that
need to be addressed first.



Cmake feature summary says: "required version >= 0.5" and that's for 
all

found
depeendencies. Unfortunately no information at all in the feature 
summary

about
the actual version.


Quoting the KWin CMake log...

15:08:41 -- Found Wayland_Client:
/usr/lib/x86_64-linux-gnu/libwayland-client.so (found version
"1.12.90")
15:08:41 -- Found Wayland_Cursor:
/usr/lib/x86_64-linux-gnu/libwayland-cursor.so (found version
"1.12.90")
15:08:42 -- Found Wayland_Egl:
/usr/lib/x86_64-linux-gnu/libwayland-egl.so (found version "11.0.2")
15:08:42 -- Found Wayland:
/usr/lib/x86_64-linux-gnu/libwayland-client.so;/usr/lib/x86_64-linux-gnu/libwayland-cursor.so;/usr/lib/x86_64-linux-gnu/libwayland-egl.so
(found suitable version "1.12.90", minimum required is "1.2") found
components:  Cursor Egl
15:08:42 -- Could NOT find XKB: Found unsuitable version "0.5.0", but
required is at least "0.7.0" (found
/usr/lib/x86_64-linux-gnu/libxkbcommon.so)
15:08:42 -- Found Libinput: /usr/lib/x86_64-linux-gnu/libinput.so
(found suitable version "1.5.2", minimum required is "1.5")
15:08:42 -- Found UDev: /usr/include
15:08:42 -- Found Libdrm: /usr/lib/x86_64-linux-gnu/libdrm.so (found
suitable version "2.4.64", minimum required is "2.4.62")
15:08:42 -- Found gbm: /usr/lib/x86_64-linux-gnu/libgbm.so (found
version "11.0.2")

The summary may not be of any use, but the rest of the output has the
information you're after.





In any case, you can see the Docker log of the container being
generated at https://build.kde.org/job/crea

Re: CI Requirements - Lessons Not Learnt?

2017-01-05 Thread Ben Cooksley
On Fri, Jan 6, 2017 at 12:38 AM, Martin Gräßlin  wrote:
> Am 2017-01-05 11:20, schrieb Ben Cooksley:
>>
>> On Thu, Jan 5, 2017 at 10:28 PM, Martin Gräßlin
>>  wrote:
>>>
>>> Am 2017-01-05 09:44, schrieb Ben Cooksley:


 Hi all,

 It seems that my previous vocal complaints about system level /
 serious impact dependency bumps on the CI system have gone completely
 unnoticed by (some) members of our Community.

 This was demonstrated earlier this week when components of Plasma
 bumped their version requirements for XKBCommon and Appstream-Qt -
 without even a thought about notifying Sysadmin or checking which
 version the CI had, until their builds broke.

 Neither of these is easy to fix at this stage, as the system base is
 now too old to receive updates such as these. Base upgrades require a
 full rebuild of everything on the CI system, and usually involve
 significant additional churn and is a process that must be done
 roughly twice a year, depending on dependency bump demands.

 Does anyone have any suggestions as to how we may avoid this in the
 future?
>>>
>>>
>>>
>>> I have a few questions here:
>>>
>>> 1) Where is this requirement to check with sysadmins codified? So far I
>>> was
>>> only aware of dependency freeze.
>>
>>
>> It's been codified since the PIM Qt 5.6 / WebEngine debacle, where
>> Sysadmin had to rush delivery of Qt 5.6 to the CI system because the
>> whole of PIM broke when they started using QtWebEngine. That was
>> around March/April 2016, my mail archives can't seem to find the exact
>> thread though.
>
>
> I'm sorry Ben, but I fear "sending out a mail about an issue with PIM"
> doesn't
> qualify as codifying it. Given what we have it looks like this did not reach
> the
> target audience. And neither will this thread.

Uhm, it was far more than a single email. It was quite the thread, and
if memory serves was on at least one of the mailing lists this thread
was on.
One of the key things that came out of that thread was to ask for
things to be bumped well in advance if a newer version is needed.

I'm disappointed that you think that email threads have insufficient
reach given it's one of our communities principal means of
communication.

>
> This needs to change the process, the way KDE develops software. It needs to
> be
> listed in the release schedule (is not, I checked), maybe reviews need to be
> acked by release managers, etc.

It seems to be part of the process for many others already, so not
sure what needs to change. The gpgme transition went quite well for
PIM, and other applications developers have reached out and asked
about version upgrades to various libraries which were in their area
of interest which we have sorted out easily enough.

>
>>
>>> 2) How can we easily check what build.kde.org has? Looking at cmake
>>> output
>>> is not a sufficient way as it gives me wrong information
>>
>>
>> If CMake is outputting wrong information, then your CMakeLists.txt
>> can't make the appropriate decisions as to whether the available
>> version is suitable, so i'd say you've got bigger problems here that
>> need to be addressed first.
>
>
> Cmake feature summary says: "required version >= 0.5" and that's for all
> found
> depeendencies. Unfortunately no information at all in the feature summary
> about
> the actual version.

Quoting the KWin CMake log...

15:08:41 -- Found Wayland_Client:
/usr/lib/x86_64-linux-gnu/libwayland-client.so (found version
"1.12.90")
15:08:41 -- Found Wayland_Cursor:
/usr/lib/x86_64-linux-gnu/libwayland-cursor.so (found version
"1.12.90")
15:08:42 -- Found Wayland_Egl:
/usr/lib/x86_64-linux-gnu/libwayland-egl.so (found version "11.0.2")
15:08:42 -- Found Wayland:
/usr/lib/x86_64-linux-gnu/libwayland-client.so;/usr/lib/x86_64-linux-gnu/libwayland-cursor.so;/usr/lib/x86_64-linux-gnu/libwayland-egl.so
(found suitable version "1.12.90", minimum required is "1.2") found
components:  Cursor Egl
15:08:42 -- Could NOT find XKB: Found unsuitable version "0.5.0", but
required is at least "0.7.0" (found
/usr/lib/x86_64-linux-gnu/libxkbcommon.so)
15:08:42 -- Found Libinput: /usr/lib/x86_64-linux-gnu/libinput.so
(found suitable version "1.5.2", minimum required is "1.5")
15:08:42 -- Found UDev: /usr/include
15:08:42 -- Found Libdrm: /usr/lib/x86_64-linux-gnu/libdrm.so (found
suitable version "2.4.64", minimum required is "2.4.62")
15:08:42 -- Found gbm: /usr/lib/x86_64-linux-gnu/libgbm.so (found
version "11.0.2")

The summary may not be of any use, but the rest of the output has the
information you're after.

>
>>
>> In any case, you can see the Docker log of the container being
>> generated at https://build.kde.org/job/create_ubuntu_slave-ange/
>
>
> and where do I find this information if I would not have asked in a mail?
> This is very related to properly codifying and documenting such
> requirements.

Pretty sure i've pointed it out to you (among others) on IRC more than once.

Re: CI Requirements - Lessons Not Learnt?

2017-01-05 Thread Martin Gräßlin

Am 2017-01-05 11:20, schrieb Ben Cooksley:

On Thu, Jan 5, 2017 at 10:28 PM, Martin Gräßlin
 wrote:

Am 2017-01-05 09:44, schrieb Ben Cooksley:


Hi all,

It seems that my previous vocal complaints about system level /
serious impact dependency bumps on the CI system have gone completely
unnoticed by (some) members of our Community.

This was demonstrated earlier this week when components of Plasma
bumped their version requirements for XKBCommon and Appstream-Qt -
without even a thought about notifying Sysadmin or checking which
version the CI had, until their builds broke.

Neither of these is easy to fix at this stage, as the system base is
now too old to receive updates such as these. Base upgrades require a
full rebuild of everything on the CI system, and usually involve
significant additional churn and is a process that must be done
roughly twice a year, depending on dependency bump demands.

Does anyone have any suggestions as to how we may avoid this in the
future?



I have a few questions here:

1) Where is this requirement to check with sysadmins codified? So far 
I was

only aware of dependency freeze.


It's been codified since the PIM Qt 5.6 / WebEngine debacle, where
Sysadmin had to rush delivery of Qt 5.6 to the CI system because the
whole of PIM broke when they started using QtWebEngine. That was
around March/April 2016, my mail archives can't seem to find the exact
thread though.


I'm sorry Ben, but I fear "sending out a mail about an issue with PIM" 
doesn't
qualify as codifying it. Given what we have it looks like this did not 
reach the

target audience. And neither will this thread.

This needs to change the process, the way KDE develops software. It 
needs to be
listed in the release schedule (is not, I checked), maybe reviews need 
to be

acked by release managers, etc.



2) How can we easily check what build.kde.org has? Looking at cmake 
output

is not a sufficient way as it gives me wrong information


If CMake is outputting wrong information, then your CMakeLists.txt
can't make the appropriate decisions as to whether the available
version is suitable, so i'd say you've got bigger problems here that
need to be addressed first.


Cmake feature summary says: "required version >= 0.5" and that's for all 
found
depeendencies. Unfortunately no information at all in the feature 
summary about

the actual version.



In any case, you can see the Docker log of the container being
generated at https://build.kde.org/job/create_ubuntu_slave-ange/


and where do I find this information if I would not have asked in a 
mail?
This is very related to properly codifying and documenting such 
requirements.


You cannot tell people "don't introduce new dependencies", without 
telling them

how to check.




3) What should we do when build.kde.org does not have the requirement?


You have to file a Sysadmin ticket, also tagging the project
'build.kde.org' at the same time.


And then? What's the process then? How long do we have to expect this to 
go?
Would it allow to block a finished feature or an important bug fix? 
Would we be

forced to write ifdef hackery?

Sorry, but I'm not thrilled by this process.

What matters to me is getting out good software to our users. And for 
that I have

a hard requirement I have to hit: dependency freeze.





It should be rather obvious that we don't introduce new dependencies 
because

we like to. There is a very important software reason to it.
That's the case for the xkbcommon dependency increase. Should I have 
let the

code broken as it was, expecting half a year of bug reports till
build.kde.org has the base upgraded to Ubuntu 16.04?


That's what #ifdef is for...


I see you volunteer to:
1. write the ifdef
2. adjust the unit test to skip
3. Inform distros that the reported minimum version is wrong, that in 
truth it

  requires a newer version than reported
4. handle all the bug reports related to it

If not, please don't suggest ifdef. We all know that it comes with a 
huge cost.
A cost I decided is too high in this case. After all I had many people 
complain

about it and you can imagine how annoyed I am about the broken build.

If it were as simple as an ifdef, we would have done it, wouldn't we?





If I have to degrade the quality of the product for serving the CI, I 
and
all users have a problem. And this is currently the only alternative. 
The

quality of our product is highly at risk as our changes are no longer
compile tested. This is a huge problem for the release of Plasma 5.9. 
On the
other hand I cannot revert the dependency change as that would break 
tests
or introduce the broken code again. So actually we are caught between 
a hard

and a rock place.

When I increased the dependency I had the dependency freeze of Plasma 
5.9 in
mind. That's the one target I have to hit from release process 
currently.
Also I had to consider a social aspect here. I asked xkbcommon devs to 
do
the release. I would have feeled ashamed if we asked for the

Re: CI Requirements - Lessons Not Learnt?

2017-01-05 Thread Ben Cooksley
On Thu, Jan 5, 2017 at 10:28 PM, Martin Gräßlin
 wrote:
> Am 2017-01-05 09:44, schrieb Ben Cooksley:
>>
>> Hi all,
>>
>> It seems that my previous vocal complaints about system level /
>> serious impact dependency bumps on the CI system have gone completely
>> unnoticed by (some) members of our Community.
>>
>> This was demonstrated earlier this week when components of Plasma
>> bumped their version requirements for XKBCommon and Appstream-Qt -
>> without even a thought about notifying Sysadmin or checking which
>> version the CI had, until their builds broke.
>>
>> Neither of these is easy to fix at this stage, as the system base is
>> now too old to receive updates such as these. Base upgrades require a
>> full rebuild of everything on the CI system, and usually involve
>> significant additional churn and is a process that must be done
>> roughly twice a year, depending on dependency bump demands.
>>
>> Does anyone have any suggestions as to how we may avoid this in the
>> future?
>
>
> I have a few questions here:
>
> 1) Where is this requirement to check with sysadmins codified? So far I was
> only aware of dependency freeze.

It's been codified since the PIM Qt 5.6 / WebEngine debacle, where
Sysadmin had to rush delivery of Qt 5.6 to the CI system because the
whole of PIM broke when they started using QtWebEngine. That was
around March/April 2016, my mail archives can't seem to find the exact
thread though.

> 2) How can we easily check what build.kde.org has? Looking at cmake output
> is not a sufficient way as it gives me wrong information

If CMake is outputting wrong information, then your CMakeLists.txt
can't make the appropriate decisions as to whether the available
version is suitable, so i'd say you've got bigger problems here that
need to be addressed first.

In any case, you can see the Docker log of the container being
generated at https://build.kde.org/job/create_ubuntu_slave-ange/

> 3) What should we do when build.kde.org does not have the requirement?

You have to file a Sysadmin ticket, also tagging the project
'build.kde.org' at the same time.

>
> It should be rather obvious that we don't introduce new dependencies because
> we like to. There is a very important software reason to it.
> That's the case for the xkbcommon dependency increase. Should I have let the
> code broken as it was, expecting half a year of bug reports till
> build.kde.org has the base upgraded to Ubuntu 16.04?

That's what #ifdef is for...

>
> If I have to degrade the quality of the product for serving the CI, I and
> all users have a problem. And this is currently the only alternative. The
> quality of our product is highly at risk as our changes are no longer
> compile tested. This is a huge problem for the release of Plasma 5.9. On the
> other hand I cannot revert the dependency change as that would break tests
> or introduce the broken code again. So actually we are caught between a hard
> and a rock place.
>
> When I increased the dependency I had the dependency freeze of Plasma 5.9 in
> mind. That's the one target I have to hit from release process currently.
> Also I had to consider a social aspect here. I asked xkbcommon devs to do
> the release. I would have feeled ashamed if we asked for the release and
> then don't use it. For me it was from a social point of view a very high
> requirement to ship with the dependency in the next release after xkbcommon
> release.
>
> If we have to wait an arbitrary time till build.kde.org has upgraded the
> base, maybe the choice of the base is not sufficient. E.g. I asked openSUSE
> about this dependency weeks ago. Actually a few days after xkbcommon had the
> release and it was already shipped in tumbleweed. Similar for Mesa 13 which
> I'm also eagerly waiting for build.kde.org to fetch it.

Mesa 13 is news to me.

Base upgrades are a major, major piece of effort. Ignoring changes to
packaging made by the distros, everything on the CI has to be fully
rebuilt due to broken binary compatibility (GLIBC usually changes)

Even if it were kept, as soon as you get new builds using new features
while old build artifacts are still using old ones, it'll start to
break (Cue wave to Qt's plugin loader & Akonadi with even patch level
version bumps to Qt). This problem is exacerbated by us often ending
up using PPA's and other third party repositories to provide certain
version bumped dependencies - which of course are packaged
differently, leading to not only potential naming differences but also
different sets of compiler flags (ABI compatibility says hi again).

In terms of the rebuild - that's everything from Qt, up through
Frameworks, then all of the libraries that aren't in Frameworks but
everyone uses, then finally into Applications/Frameworks/Extragear -
easily a solid 24 hours of building and test runs. During the time the
CI is completely unavailable, and we usually spend a good 2-3 days
afterwards mopping up various breakages in tests, etc.

That time frame also dates

Re: CI Requirements - Lessons Not Learnt?

2017-01-05 Thread Martin Gräßlin

Sorry picked wrong from address

Am 2017-01-05 10:28, schrieb Martin Gräßlin:

Am 2017-01-05 09:44, schrieb Ben Cooksley:

Hi all,

It seems that my previous vocal complaints about system level /
serious impact dependency bumps on the CI system have gone completely
unnoticed by (some) members of our Community.

This was demonstrated earlier this week when components of Plasma
bumped their version requirements for XKBCommon and Appstream-Qt -
without even a thought about notifying Sysadmin or checking which
version the CI had, until their builds broke.

Neither of these is easy to fix at this stage, as the system base is
now too old to receive updates such as these. Base upgrades require a
full rebuild of everything on the CI system, and usually involve
significant additional churn and is a process that must be done
roughly twice a year, depending on dependency bump demands.

Does anyone have any suggestions as to how we may avoid this in the 
future?


I have a few questions here:

1) Where is this requirement to check with sysadmins codified? So far
I was only aware of dependency freeze.
2) How can we easily check what build.kde.org has? Looking at cmake
output is not a sufficient way as it gives me wrong information
3) What should we do when build.kde.org does not have the requirement?

It should be rather obvious that we don't introduce new dependencies
because we like to. There is a very important software reason to it.
That's the case for the xkbcommon dependency increase. Should I have
let the code broken as it was, expecting half a year of bug reports
till build.kde.org has the base upgraded to Ubuntu 16.04?

If I have to degrade the quality of the product for serving the CI, I
and all users have a problem. And this is currently the only
alternative. The quality of our product is highly at risk as our
changes are no longer compile tested. This is a huge problem for the
release of Plasma 5.9. On the other hand I cannot revert the
dependency change as that would break tests or introduce the broken
code again. So actually we are caught between a hard and a rock place.

When I increased the dependency I had the dependency freeze of Plasma
5.9 in mind. That's the one target I have to hit from release process
currently. Also I had to consider a social aspect here. I asked
xkbcommon devs to do the release. I would have feeled ashamed if we
asked for the release and then don't use it. For me it was from a
social point of view a very high requirement to ship with the
dependency in the next release after xkbcommon release.

If we have to wait an arbitrary time till build.kde.org has upgraded
the base, maybe the choice of the base is not sufficient. E.g. I asked
openSUSE about this dependency weeks ago. Actually a few days after
xkbcommon had the release and it was already shipped in tumbleweed.
Similar for Mesa 13 which I'm also eagerly waiting for build.kde.org
to fetch it.

Cheers
Martin