[Development] Google Summer of Code 2019
Hi all, Some of us have been discussing on IRC whether to apply again to the Google Summer of Code 2019, as The Qt Project. The deadline for organizations to send applications is February 6th. The current idea is that, unless there are strong objections against applying, we will try to submit an application on behalf of The Qt Project. So, if you agree and have ideas please use the following wiki page to propose your project and/or assign yourself as a possible mentor in other interesting projects: https://wiki.qt.io/index.php?title=Google_Summer_of_Code/2019/Project_Ideas A lesson learned from last year's participation is that the candidate's skills are essential, so the goal will be to prioritize top candidates. Please join also #qt-gsoc on Freenode to discuss the participation of The Qt Project to the Google Summer of Code 2019. Once the application will be submitted and accepted, we would use the #qt-gsoc IRC channel for questions about procedure and projects. Qt-related questions should still go to #qt-labs on IRC/freenode. This mailing list is also a good place to propose or discuss GSoC projects. I have also created a tentative landing page: https://wiki.qt.io/Google_Summer_of_Code/2019 Best regards, Jesús ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Allow to do parallel integration (was: Proposal: New branch model)
On Wednesday, January 30, 2019 1:55:41 PM CET Allan Jensen wrote: > On Wednesday, 30 January 2019 13:38:46 CET Jedrzej Nowacki wrote: > > On Wednesday, January 30, 2019 11:48:40 AM CET Allan Jensen wrote: > > > On Wednesday, 30 January 2019 11:19:07 CET Edward Welbourne wrote: > > > > Jedrzej Nowacki (30 January 2019 11:08) > > > > > > > > > Mårten pointed out that we can check for conflicts up front. Not > > > > > only > > > > > against HEAD of the target branch, but also against all build > > > > > branches. > > > > > That is even nicer as there is no need to start a job that would > > > > > likely > > > > > to be rejected at the end. > > > > > > > > That'll only find VC-level conflicts. > > > > It's no help in finding the kinds of breakage that only testing > > > > reveals. > > > > It'll also sometimes lead to rejecting a change needlessly, because > > > > the > > > > already-integrating branch it conflicts with is about to fail its > > > > testing. > > > > Still, I guess restaging a little later can fix that ... > > > > > > We could do speculative merging. Guess if the previous commit will merge > > > on > > not, and stage the following commit based on that guess. Or stage two > > > > versions of the next commit, one for success of the previous and one for > > > failure. That latter would use 3x the VMs for 2x the throughput. > > > > > > 'Allan > > > > We could. In such case we do not allow regressions, but we waste ~50% of > > speculative builds. I believe that would be much better accepting the > > risks, > the throughput maybe N times higher, when N is count of parallel build > > > branches. In the end reverts are just one click. Btw. after revert coin > > will likely detect that the revision was build and it would just use > > cached > > results, so it may even avoid running tests. > > At the current pass rates it would be a better speculation to assume that > the other commits do not merge. > > 'Allan Not to my knowledge (last 7 days): total percentage Passed 153 47.37% Failed 147 45.51% Cancelled 23 7.12% ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Allow to do parallel integration (was: Proposal: New branch model)
On Wednesday, 30 January 2019 13:38:46 CET Jedrzej Nowacki wrote: > On Wednesday, January 30, 2019 11:48:40 AM CET Allan Jensen wrote: > > > On Wednesday, 30 January 2019 11:19:07 CET Edward Welbourne wrote: > > > > > Jedrzej Nowacki (30 January 2019 11:08) > > > > > > > > > > Mårten pointed out that we can check for conflicts up front. Not only > > > > against HEAD of the target branch, but also against all build > > > > branches. > > > > That is even nicer as there is no need to start a job that would > > > > likely > > > > to be rejected at the end. > > > > > > > > > That'll only find VC-level conflicts. > > > It's no help in finding the kinds of breakage that only testing > > > reveals. > > > It'll also sometimes lead to rejecting a change needlessly, because the > > > already-integrating branch it conflicts with is about to fail its > > > testing. > > > Still, I guess restaging a little later can fix that ... > > > > > > We could do speculative merging. Guess if the previous commit will merge > > on not, and stage the following commit based on that guess. Or stage two > > versions of the next commit, one for success of the previous and one for > > failure. That latter would use 3x the VMs for 2x the throughput. > > > > 'Allan > > > We could. In such case we do not allow regressions, but we waste ~50% of > speculative builds. I believe that would be much better accepting the risks, > the throughput maybe N times higher, when N is count of parallel build > branches. In the end reverts are just one click. Btw. after revert coin > will likely detect that the revision was build and it would just use cached > results, so it may even avoid running tests. > At the current pass rates it would be a better speculation to assume that the other commits do not merge. 'Allan ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Allow to do parallel integration (was: Proposal: New branch model)
On Wednesday, January 30, 2019 11:48:40 AM CET Allan Jensen wrote: > On Wednesday, 30 January 2019 11:19:07 CET Edward Welbourne wrote: > > Jedrzej Nowacki (30 January 2019 11:08) > > > > > Mårten pointed out that we can check for conflicts up front. Not only > > > against HEAD of the target branch, but also against all build branches. > > > That is even nicer as there is no need to start a job that would likely > > > to be rejected at the end. > > > > That'll only find VC-level conflicts. > > It's no help in finding the kinds of breakage that only testing reveals. > > It'll also sometimes lead to rejecting a change needlessly, because the > > already-integrating branch it conflicts with is about to fail its testing. > > Still, I guess restaging a little later can fix that ... > > We could do speculative merging. Guess if the previous commit will merge on > not, and stage the following commit based on that guess. Or stage two > versions of the next commit, one for success of the previous and one for > failure. That latter would use 3x the VMs for 2x the throughput. > > 'Allan We could. In such case we do not allow regressions, but we waste ~50% of speculative builds. I believe that would be much better accepting the risks, the throughput maybe N times higher, when N is count of parallel build branches. In the end reverts are just one click. Btw. after revert coin will likely detect that the revision was build and it would just use cached results, so it may even avoid running tests. Cheers, Jędrek ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Allow to do parallel integration (was: Proposal: New branch model)
On Wednesday, January 30, 2019 11:19:07 AM CET Edward Welbourne wrote: > > Mårten pointed out that we can check for conflicts up front. Not only > > against HEAD of the target branch, but also against all build branches. > > That is even nicer as there is no need to start a job that would likely > > to be rejected at the end. > > That'll only find VC-level conflicts. > It's no help in finding the kinds of breakage that only testing reveals. Of course. That is exactly what I'm proposing. Cost: CI would allow breakages caused by patches that are being integrated in parallel. Gain: CI would allow running many parallel integrations. > It'll also sometimes lead to rejecting a change needlessly, because the > already-integrating branch it conflicts with is about to fail its testing. We have that already, you can not stage if your change conflicts with changes that are being integrated. Gerrit gives you a nice and complex message. Maybe it will work out of the box, with multiple build branches. I do not know. Some research is needed. > Still, I guess restaging a little later can fix that ... Exactly. Hopefully less restaging. As cherry on top it may increases chances that the content sha1(*) optimization kicks in, so re-staging may be faster. Cheers, Jędrek (*) Coin works on the content sha1 level not on the commit sha1. That way commit message change or different commit order is irrelevant from it's perspective and it can re-use caches. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' started
Good point. I only just now saw & read the entire tooltip for WebEngine. I agree that it feels like user error, but it's too easy to make this error. Also the cost is quite high, I only discovered the issue after a full Qt download & install & rebuild failed on my code. That's half a day burned, considering I have to uninstall again and then reinstall msvc 2017 package (luckily I have that option, or I'd be more upset)... Mark -Original Message- From: Development On Behalf Of Michal Klocek Sent: 30 January 2019 10:41 To: development@qt-project.org Subject: Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' started This suggestion is already in JIRA see https://bugreports.qt.io/browse/QTBUG-71579 Br Michal On 01/30/2019 11:29 AM, Kai Koehne wrote: >> -Original Message- >> From: Development On Behalf Of >> Mark De Wit >> Sent: Wednesday, January 30, 2019 11:13 AM >> To: development@qt-project.org >> Subject: Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' >> started >> >> Selecting MSVC 2015 64-bit + Qt WebEngine for 5.12.1 in the installer does >> not >> give me WebEngine. Is that intentional? > > Yes, this is intentional. Qt WebEngine in 5.12 only supports MSVC 2017, that > was the case already for Qt 5.12.0. > >> Should the installer perhaps warn >> that the selected combination is unsupported? > > Maybe, though getting a warning every time might be a bit too much, too. > Also, I'm not sure how and whether such a warning can be implemented ... > > At least the description of the Qt WebEngine module says > > On Windows, the module is only available under MSVC 2017 > > Kai > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development > ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Allow to do parallel integration (was: Proposal: New branch model)
On Wednesday, 30 January 2019 11:19:07 CET Edward Welbourne wrote: > Jedrzej Nowacki (30 January 2019 11:08) > > > Mårten pointed out that we can check for conflicts up front. Not only > > against HEAD of the target branch, but also against all build branches. > > That is even nicer as there is no need to start a job that would likely > > to be rejected at the end. > > That'll only find VC-level conflicts. > It's no help in finding the kinds of breakage that only testing reveals. > It'll also sometimes lead to rejecting a change needlessly, because the > already-integrating branch it conflicts with is about to fail its testing. > Still, I guess restaging a little later can fix that ... > We could do speculative merging. Guess if the previous commit will merge on not, and stage the following commit based on that guess. Or stage two versions of the next commit, one for success of the previous and one for failure. That latter would use 3x the VMs for 2x the throughput. 'Allan ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' started
This suggestion is already in JIRA see https://bugreports.qt.io/browse/QTBUG-71579 Br Michal On 01/30/2019 11:29 AM, Kai Koehne wrote: >> -Original Message- >> From: Development On Behalf Of >> Mark De Wit >> Sent: Wednesday, January 30, 2019 11:13 AM >> To: development@qt-project.org >> Subject: Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' >> started >> >> Selecting MSVC 2015 64-bit + Qt WebEngine for 5.12.1 in the installer does >> not >> give me WebEngine. Is that intentional? > > Yes, this is intentional. Qt WebEngine in 5.12 only supports MSVC 2017, that > was the case already for Qt 5.12.0. > >> Should the installer perhaps warn >> that the selected combination is unsupported? > > Maybe, though getting a warning every time might be a bit too much, too. > Also, I'm not sure how and whether such a warning can be implemented ... > > At least the description of the Qt WebEngine module says > > On Windows, the module is only available under MSVC 2017 > > Kai > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development > ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' started
> -Original Message- > From: Development On Behalf Of > Mark De Wit > Sent: Wednesday, January 30, 2019 11:13 AM > To: development@qt-project.org > Subject: Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' started > > Selecting MSVC 2015 64-bit + Qt WebEngine for 5.12.1 in the installer does not > give me WebEngine. Is that intentional? Yes, this is intentional. Qt WebEngine in 5.12 only supports MSVC 2017, that was the case already for Qt 5.12.0. > Should the installer perhaps warn > that the selected combination is unsupported? Maybe, though getting a warning every time might be a bit too much, too. Also, I'm not sure how and whether such a warning can be implemented ... At least the description of the Qt WebEngine module says On Windows, the module is only available under MSVC 2017 Kai ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Allow to do parallel integration (was: Proposal: New branch model)
Jedrzej Nowacki (30 January 2019 11:08) > Mårten pointed out that we can check for conflicts up front. Not only against > HEAD of the target branch, but also against all build branches. That is even > nicer as there is no need to start a job that would likely to be rejected at > the end. That'll only find VC-level conflicts. It's no help in finding the kinds of breakage that only testing reveals. It'll also sometimes lead to rejecting a change needlessly, because the already-integrating branch it conflicts with is about to fail its testing. Still, I guess restaging a little later can fix that ... Eddy. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' started
Selecting MSVC 2015 64-bit + Qt WebEngine for 5.12.1 in the installer does not give me WebEngine. Is that intentional? Should the installer perhaps warn that the selected combination is unsupported? I shall try again with MSVC 2017... KR, Mark -Original Message- From: Development On Behalf Of Jani Heikkinen Sent: 29 January 2019 10:32 To: ekke ; development@qt-project.org Subject: Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' started Hi, We have "RC" packages under testing. So far all seems to be pretty much OK so if nothing new & serious enough found we should be able to get the release out during this week. Let's keep our fingers crossed. br, Jani btw, there is Qt 5.12.1 "RC" snapshot available via online installer if you want to take a tour. It is almost what we expect to release; only couple of changes is in after that one From: Development on behalf of ekke Sent: Tuesday, January 29, 2019 12:24 PM To: development@qt-project.org Subject: Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' started Am 23.01.19 um 17:03 schrieb Kai Koehne: >> -Original Message- >> From: Development On Behalf Of ekke >> Sent: Wednesday, January 23, 2019 9:45 AM >> To: development@qt-project.org >> Subject: Re: [Development] HEADS-UP: Branching from '5.12' to '5.12.1' >> started >> >> seems this time it's a rocky path to release Qt 5.12.1 > We're still struggling with a change in configure that made system library > paths absolute - good for local installations, but problematic for the binary > packages, which are supposed to work in different setups: > > https://bugreports.qt.io/browse/QTBUG-72903 > > Kai > what do you think - is there a chance to get Qt 5.12.1 this week ? thx ekke ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt modules, API changes and Qt 6
Robin Burchell (30 January 2019 10:13) > I will admit that a monorepo has a _different_ set of problems > (including but not limited to: longer build times, longer test times, > flaky tests in unrelated areas blocking changes), It also makes it easier to cope with API changes, which is great where it's public APIs that haven't yet been shipped, but also makes it easier to get away with using private APIs between components that really shouldn't do that. One of the classic reasons for modularisation at the VC layer is that it makes this sort of thing harder, which means it happens less, which is good. There's also the problem of scope - which things go in the monorepo, which should be outside. We have that today, with qt5/, and we should probably hoist some of its pieces outside, if only to force ourselves to make it easy for a sizeable component to live happily outside; that would enable folk in our ecosystem to live happily alongside, rather than inside, Qt. If we insist on solving that as part of a switch to a monorepo, then we win (even if we could have done it without the switch), if only because a major upheaval is an opportunity to make other needed changes. But if we move to a monorepo without solving that problem, there's a significant risk we'll be making things harder for those who work outside but close to Qt. > but those problems are not complex, and can be fixed with some > dedicated application of smarter scripting at build/test time I remain to be convinced. > (for instance: if change is doc only, don't run any test that _isn't_ > related to documentation, to cover one complaint from earlier in this > thread). This sort of thing [*] sounds terribly sensible and feasible, until you start running into changes that the submitter and reviewers all *think* should only have impact in a bounded area, but that turns out to break stuff in surprising places outside those bounds. That's probably rare but when it happens it'll gum up the works - in a seemingly not very related area that's been caught in the cross-fire. In particular, this sort of thing happens more readily when disparate things use each others private APIs, as sketched above. [*] The case of doc fixes is probably relatively safe, of course; but if this is applied to other changes, we can't be assured of as much safety. One of the scripts involved in my API change review generator knows to ignore various changes that "make no difference"; we could apply something like that to changes to say "needs minimal testing"; but I'd still worry about the cases where a change makes more difference than the script maintainer is aware of. Once we get to "this only changes network code, we don't need to test graphics" (or vice versa) you can start to expect sporadic surprises. Not that the present state of affairs entirely avoids that situation. In commit qtbase's 641eb4a965 I reverted the introduction of GPU blacklisting, since it's no longer used; in the process, I renamed QTestPrivate::checkBlackLists() back to the singular name it'd had before GPU blcaklists were introduced, confidently expecting that to have no impact outside QtTestLib. That broke qtdeclarative, because it actually uses this private API (in implementing the QML test framework), resulting in a crisis that Liang fixed with qtbase's af6d4d068. That would have been avoided by a monorepo, but not if we were only building and testing the parts we believed should be affected. So we need to be deliberate about refraining from and objecting to cross-component use of private parts, all the more so if we're going to a monorepo. We should also document, alongside each private API, any known violations of its privacy; normally, those are done by friend declarations, but non-class cases (like the QTestPrivate namespace) need comments about such things (and those comments need to be specific enough that someone finding them years later can determine whether they've gone out of date). Eddy. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Mailing list archive annoyance
Il 30/01/19 10:29, Christian Gagneraud ha scritto:
I do not know what is going on with the mailing list archive, but this
is getting frustrating.
Trying to move forward and keep positive, i would like to report on
a(nother) use case here:https://bugreports.qt.io/browse/QTBUG-71225
This bug report starts with a couple of links to the Qt Interest ML
archive, in an attempt to give some background.This qt.io Jira ticket
is linked into our own bug tracking system. Yet, all these mailing
list archive hyperlinks yield a so-called HTTP 404.This is quite
annoying. History seems to be gone for ever.
This has already been noted in another thread ("Archiving is working").
I'd handle out a link to it, but chances are it's going to break for the
very same reason, so I better don't -- just find the thread yourself
please :-)
Cheers,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts
smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development
Re: [Development] Allow to do parallel integration (was: Proposal: New branch model)
On Wednesday, January 30, 2019 10:41:02 AM CET Jedrzej Nowacki wrote: > Cheap (a.k.a reasonably fast to implement) proposal: > - Stage button would create a stage branch as currently (through cherry-pick > on top of target branch) > - 3-5 minutes old stage branch would changed to be a build branche and all > patches in it would be changed to "integrating" state. CI would start > testing the build branch. At that time a new stage branch may be created. > As opposite to now, multiple build branches may be tested in parallel. > - Once a build branch testing passed we cherry-pick / merges (depends what > gerrit does) changes on top of the target branch, in case of conflicts we > mark it as failed. Mårten pointed out that we can check for conflicts up front. Not only against HEAD of the target branch, but also against all build branches. That is even nicer as there is no need to start a job that would likely to be rejected at the end. Cheers, Jędrek ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Mailing list archive annoyance
> -Original Message- > From: Development On Behalf Of > Christian Gagneraud > Sent: Wednesday, 30 January 2019 10:30 AM > To: interest ; development project.org> > Subject: [Development] Mailing list archive annoyance > > Hi all, > > I do not know what is going on with the mailing list archive, but this is > getting > frustrating. > Trying to move forward and keep positive, i would like to report on > a(nother) use case here: https://bugreports.qt.io/browse/QTBUG-71225 > This bug report starts with a couple of links to the Qt Interest ML archive, > in > an attempt to give some background.This qt.io Jira ticket is linked into our > own bug tracking system. Yet, all these mailing list archive hyperlinks yield > a > so-called HTTP 404.This is quite annoying. History seems to be gone for ever. > > Can you guys recover the mailing list archive, like nobody will ever notice > what has happened? Can you do that? Yes or No? > > Or should we switch (back) the Qt mailing list management to the > community? > > Who is in charge of the qt-project.org domain? It's being looked into apparently: https://bugreports.qt.io/browse/QTQAINFRA-2553 > Chris > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt modules, API changes and Qt 6
On tirsdag 29. januar 2019 11:07:55 CET Alex Blasche wrote: > >From: Development on behalf of > >Frederik Gladhorn > 2 Heads: use the latest revision of each branch (the system we used to have > in the past) > > >3 Modules containing pinned dependency sha1s > > > >Each module is completely self-contained, that means qt5 is not > > > >required as such (but may still exist as a collection of all modules, for > >convenience and releases). In each module we have a list of dependencies > >and their sha1. > > > >Updates for a release (and also otherwise) must happen regularly > >(e.g. > > > >nightly), moving each module forward towards newer dependencies, this would > >be implemented as bot which updates the above mentioned requirements file. > I like this one. As you mentioned, we kind of had it already with > sync.profiles.And really, if you implement this option you can almost > implicitly run option 2 above too. In fact some modules might even want to > do that if you permit SHA definition based on HEAD of some other > repo/branch. > > There is one big question though. I vaguely recall one of the reasons for > going to today's model was to limit the number of potential builds. This > model could have 10 different modules/repos using different SHA's for all > of its dependencies. Doesn't this increase the amount of different module > builds which you have to store for later CI runs or build e.g. qtbase more > often? Do we have the capacity/storage for it? Yes, this is what we ended up with as well, module pinning seems like the way to go. It seems to solve a bunch of issues and we'd like to try implementing it (a few lines in how the CI resolves dependencies and writing the bot that bumps dependencies forward). The capacity is something to look at, but considering how many failed qt5.git updates we have, I'm actually hoping that we get a better success rate with this model. Other things to figure out are provisioning (how we prepare the VMs on which we build and test) and what the files specifying the dependencies should look like. So option 3 is the favorite, but we need to clarify a few details. Cheers, Frederik > > -- > Alex > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
[Development] Allow to do parallel integration (was: Proposal: New branch model)
On Friday, January 25, 2019 11:08:52 AM CET Lars Knoll wrote: > To me this means we need to seriously rethink that part of our CI system, > and ideally test changes (or patch series) individually and in parallel. So > maybe we should adjust our CI system that way: > > * test changes (or patch series) individually > * If they pass CI merge or cherry-pick them into the branch > * reject only if the merge/cherry-pick gives conflicts > > This adds a very small risk that two parallel changes don’t conflict during > the merge/cherry-pick process, but cause a test regression together. To > help with that, we can simply run a regular status check on the repo. If > this happens the repo will be blocked for further testing until someone > submits a fix or reverts an offending change, which is acceptable. > > Another advantage is that this would pave the road towards a system where CI > testing happens before human review, so we can in the longer term avoid > duplicated review/approval work. Hi, I do agree with it. Personally, I'm convinced that we are doing CI gate- keeping in a wrong way. CI is about balancing costs vs. gains. We can not test everything, we should find as many as possible bugs, while running in reasonable time without using too much resources. That is the function to optimize. We really should test things in parallel. The risk of conflicts / regression caused by parallel integration in project as big as qt is minimal. One can be even proactive and not stage changes while other similar ones are integrating. While broken repository state is bad, it is also easy to detect; nothing integrates because of one test, which is not known to be flaky (testrestults.qt.io/grafana...). Cheap (a.k.a reasonably fast to implement) proposal: - Stage button would create a stage branch as currently (through cherry-pick on top of target branch) - 3-5 minutes old stage branch would changed to be a build branche and all patches in it would be changed to "integrating" state. CI would start testing the build branch. At that time a new stage branch may be created. As opposite to now, multiple build branches may be tested in parallel. - Once a build branch testing passed we cherry-pick / merges (depends what gerrit does) changes on top of the target branch, in case of conflicts we mark it as failed. Advantages: - changes are tested in smaller batches => smaller chances to fail - we can use hardware in a more efficient way => more code integrated per hour - fairly easy to implement => we would not need to wait for it to happen years - that would be a step forward in to direction of testing before reviewing Disadvantages: - risk of breakages going through CI - someone work needs to be done to enable it Cheers, Jędrek ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt modules, API changes and Qt 6
On onsdag 30. januar 2019 09:17:11 CET Alex Blasche wrote: > >From: Development on behalf of Jedrzej > >Nowacki > > >Personally, I also do like the idea of monolithic repo, while keeping > > > >modularization on the logical / build level. In our current state I see two > >major problems: > >- our build is quite monolithic in practice. For example qtbase always > >needs to be build. CI currently caches builds on repository level (caches > >results of make install) with monolithic repository the optimization would > >need to be reconstructed on the build level. Conceptually it is good, but > >someone would need to do the work. > >- as a consequence of a partial build, the repository may be in a broken > >state, for example not compiling. It can be solved by time based world > >rebuilding and tagging known good revisions, but some policies would need > >to be created. > > I am clearly missing something here. Could you please outline why such a > monolithic repo would be good? As you pointed out yourself, the conceptual > problem that different libraries won't walk in lockstep remains and > segmented builds continue to be desired. Sure, repo checkout is easier but > as long as segmentation exists I fail to see the advantage. A mono repo is good because it allows changing API across modules in on go. There are a bunch of things speaking in favor of this, but the practicalities are quite hard - how would we do CI for this? How would we merge between the mono repo and the existing individual repos? In the end we discarded the idea as impractical for the time being. It would be possible for someone to champion it by showing a lot of git magic and coming up with a good testing strategy where things work together nicely, but I fear that the complexity is quite high. With unlimited resources we could of course test everything in the repo for each change. Other options would be to try to figure out which tests are relevant for which change. Doing that cross- platform in a sensible way seems quite non-trivial as well. Cheers, Frederik > > -- > Alex > ___ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
[Development] Mailing list archive annoyance
Hi all, I do not know what is going on with the mailing list archive, but this is getting frustrating. Trying to move forward and keep positive, i would like to report on a(nother) use case here: https://bugreports.qt.io/browse/QTBUG-71225 This bug report starts with a couple of links to the Qt Interest ML archive, in an attempt to give some background.This qt.io Jira ticket is linked into our own bug tracking system. Yet, all these mailing list archive hyperlinks yield a so-called HTTP 404.This is quite annoying. History seems to be gone for ever. Can you guys recover the mailing list archive, like nobody will ever notice what has happened? Can you do that? Yes or No? Or should we switch (back) the Qt mailing list management to the community? Who is in charge of the qt-project.org domain? Chris ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt modules, API changes and Qt 6
On Wed, Jan 30, 2019, at 9:17 AM, Alex Blasche wrote: > I am clearly missing something here. Could you please outline why such a > monolithic repo would be good? As you pointed out yourself, the > conceptual problem that different libraries won't walk in lockstep > remains and segmented builds continue to be desired. Sure, repo checkout > is easier but as long as segmentation exists I fail to see the > advantage. Split repositories have a lot of problems, both social, and technical, but I'd say that it all boils down to extra friction. On the technical side, you have the well-known problem of API changes. When you make one, you have to make it in a backwards-compatible way, integrate that into qt5, then change everything, integrate that into qt5, then remove the backwards compatibility. That's a long process, even if everything goes perfectly (reviews are easy, happen quickly, CI isn't flaky, etc, etc). Then you have the social side of it. Basically, I think I can summarize it as being this: a repository boundary generally makes a problem into "someone else's problem" rather than "my problem" or "our problem", because the perception is it's easier that way, even if the cohesive whole ends up suffering as a result. I will admit that a monorepo has a _different_ set of problems (including but not limited to: longer build times, longer test times, flaky tests in unrelated areas blocking changes), but those problems are not complex, and can be fixed with some dedicated application of smarter scripting at build/test time (for instance: if change is doc only, don't run any test that _isn't_ related to documentation, to cover one complaint from earlier in this thread). I don't think that changing the technology behind how we contribute today would necessarily fix any of this split repo stuff either, I think a lot of it is just inherent to how such a system works. For instance, a project I am currently working on uses a completely different toolset, yet suffered from (a version of) exactly the problems outlined here -- and once we realised just how hard that was making life, and merged a bunch of repositories together, suddenly, things became a lot easier. The repository merge, in our case, helped to inspire actually applying work to _fix_ those problems rather than just working around them by splitting things into smaller pieces, too. Another real life anecdote: when I merged together our bunch of repositories, I was annoyed by ~20 minute clean build times of one component in our CI pipeline. I found out that it was due to heavy (ab)use of templates, which I managed to work around in a largely source-compatible way, and dropped the build time of that component down to ~3 minutes, which certainly made the people working on that component happy. I wouldn't have done it without the repository merge, partly because it wasn't "my problem" before, and partly because I just didn't notice, as I didn't work on that component: I still don't, really, but because I live near it, I see the code in grep results. I see the impact it has on build/test times, and so on. Robin ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] [Releasing] HEADS-UP: Branching '5.13' from 'dev' started
Hi, Very little content yet at: https://wiki.qt.io/New_Features_in_Qt_5.13 There has been many changes worth mentioning pushed to dev during the past 6 months. Please add these to wiki. I hope that by end of this week we have much more content in the page as Alpha release milestone is approaching soon. Yours, Tuukka On 28/01/2019, 18.42, "Releasing on behalf of Jani Heikkinen" wrote: Hi, We have soft branched '5.13' from 'dev' now. Final downmerge and Qt 5.13 Feature Freeze will happen Friday 1st February. So please start using '5.13' now for new changes targeted to Qt 5.13 release br, Jani Heikkinen Release Manager ___ Releasing mailing list releas...@qt-project.org https://lists.qt-project.org/listinfo/releasing ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Qt modules, API changes and Qt 6
>From: Development on behalf of Jedrzej >Nowacki >Personally, I also do like the idea of monolithic repo, while keeping >modularization on the logical / build level. In our current state I see two >major problems: >- our build is quite monolithic in practice. For example qtbase always needs >to be build. CI currently caches builds on repository level (caches results of >make install) with monolithic repository the optimization would need to be >reconstructed on the build level. Conceptually it is good, but someone would >need to do the work. >- as a consequence of a partial build, the repository may be in a broken >state, for example not compiling. It can be solved by time based world >rebuilding and tagging known good revisions, but some policies would need to >be created. I am clearly missing something here. Could you please outline why such a monolithic repo would be good? As you pointed out yourself, the conceptual problem that different libraries won't walk in lockstep remains and segmented builds continue to be desired. Sure, repo checkout is easier but as long as segmentation exists I fail to see the advantage. -- Alex ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
