Hi, On fredag 15. februar 2019 07:31:33 CET Lars Knoll wrote: > Summing up the discussion here. It looks like people overall agree that the > pinned dependency approach (option 3) sounds better than what we currently > have. The main concern was CI capacity, but Frederik believes that with > enough storage capacity for build artefacts this will not be worse than > what we have today with all the failed qt5.git integrations. > The only other option that was seriously considered was a more monolithic > repo (option 4). Disadvantages here are that it would require more work on > the CI front to make testing times bearable, and it doesn’t give us a very > flexible approach with regards to adding/removing modules. This is > something that option 3 will nicely give us, as it treats all repositories > the same way. Option 3 does btw not exclude that we later on merge some > repositories if we feel that makes sense. > So let’s go with option 3. Frederik will be leading the work to get this > implemented. > As far as I can see, it requires some changes to CI so that we have the > dependencies encoded in each repository, a bot to automatically push sha1’s > of dependencies forward and some monitoring to alert us if modules get left > behind.
I have indeed been playing with this, and I think I have an implementation that works when it comes to resolving dependencies based on the idea. After some pondering and starting with ini files, I settled on yaml, since we probably want to extend the data later on and I would like this to be easily human and machine read- and writable. My current files look like this: dependencies: qt/qtbase: ref: 698078680fc5a6870177af285fa50c0d8a7c0bc3 url: code.qt.io qt/qtsvg: ref: 2ae3c52fc061f574c9582bf58963fb3996724fbf url: code.qt.io This is for qt/qtdeclarative, sha1's taken from qt5.git at some point. Then there is the question of what the file name should be. Should it be at the top-level or in a coin subdirectory? dependencies.yaml? I have not yet worked on the bot doing the module updates (which may turn out to be the harder part). I don't think this can go live before we have that written as well. Cheers, Frederik > Cheers, > Lars > > > > On 30 Jan 2019, at 11:11, Edward Welbourne <edward.welbou...@qt.io> > > wrote: > > > > Robin Burchell (30 January 2019 10:13) > > > >> I will admit that a monorepo has a _different_ set of problems > >> (including but not limited to: longer build times, longer test times, > >> flaky tests in unrelated areas blocking changes), > > > > > > It also makes it easier to cope with API changes, which is great where > > it's public APIs that haven't yet been shipped, but also makes it easier > > to get away with using private APIs between components that really > > shouldn't do that. One of the classic reasons for modularisation at the > > VC layer is that it makes this sort of thing harder, which means it > > happens less, which is good. > > > > There's also the problem of scope - which things go in the monorepo, > > which should be outside. We have that today, with qt5/, and we should > > probably hoist some of its pieces outside, if only to force ourselves to > > make it easy for a sizeable component to live happily outside; that > > would enable folk in our ecosystem to live happily alongside, rather > > than inside, Qt. If we insist on solving that as part of a switch to a > > monorepo, then we win (even if we could have done it without the > > switch), if only because a major upheaval is an opportunity to make > > other needed changes. But if we move to a monorepo without solving that > > problem, there's a significant risk we'll be making things harder for > > those who work outside but close to Qt. > > > > > >> but those problems are not complex, and can be fixed with some > >> dedicated application of smarter scripting at build/test time > > > > > > I remain to be convinced. > > > > > >> (for instance: if change is doc only, don't run any test that _isn't_ > >> related to documentation, to cover one complaint from earlier in this > >> thread). > > > > > > This sort of thing [*] sounds terribly sensible and feasible, until you > > start running into changes that the submitter and reviewers all *think* > > should only have impact in a bounded area, but that turns out to break > > stuff in surprising places outside those bounds. That's probably rare > > but when it happens it'll gum up the works - in a seemingly not very > > related area that's been caught in the cross-fire. In particular, this > > sort of thing happens more readily when disparate things use each > > others private APIs, as sketched above. > > > > [*] The case of doc fixes is probably relatively safe, of course; but if > > this is applied to other changes, we can't be assured of as much safety. > > One of the scripts involved in my API change review generator knows to > > ignore various changes that "make no difference"; we could apply > > something like that to changes to say "needs minimal testing"; but I'd > > still worry about the cases where a change makes more difference than > > the script maintainer is aware of. Once we get to "this only changes > > network code, we don't need to test graphics" (or vice versa) you can > > start to expect sporadic surprises. > > > > Not that the present state of affairs entirely avoids that situation. > > In commit qtbase's 641eb4a965 I reverted the introduction of GPU > > blacklisting, since it's no longer used; in the process, I renamed > > QTestPrivate::checkBlackLists() back to the singular name it'd had > > before GPU blcaklists were introduced, confidently expecting that to > > have no impact outside QtTestLib. That broke qtdeclarative, because it > > actually uses this private API (in implementing the QML test framework), > > resulting in a crisis that Liang fixed with qtbase's af6d4d068. That > > would have been avoided by a monorepo, but not if we were only building > > and testing the parts we believed should be affected. > > > > So we need to be deliberate about refraining from and objecting to > > cross-component use of private parts, all the more so if we're going to > > a monorepo. We should also document, alongside each private API, any > > known violations of its privacy; normally, those are done by friend > > declarations, but non-class cases (like the QTestPrivate namespace) need > > comments about such things (and those comments need to be specific > > enough that someone finding them years later can determine whether > > they've gone out of date). > > > > > > Eddy. > > > > _______________________________________________ > > Development mailing list > > Development@qt-project.org > > https://lists.qt-project.org/listinfo/development > > > _______________________________________________ > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development _______________________________________________ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development