Re: Time for drm-ci-next?
On Tue, Sep 24, 2024 at 05:57:07PM GMT, Vignesh Raman wrote: > Hi, > > On 12/09/24 11:18, Dmitry Baryshkov wrote: > > On Mon, Sep 09, 2024 at 07:34:04AM GMT, Rob Clark wrote: > > > On Mon, Sep 9, 2024 at 2:54 AM Dmitry Baryshkov > > > wrote: > > > > > > > > On Mon, 9 Sept 2024 at 10:50, Maxime Ripard wrote: > > > > > > > > > > Hi, > > > > > > > > > > On Tue, Jul 09, 2024 at 01:27:51AM GMT, Dmitry Baryshkov wrote: > > > > > > On Mon, 8 Jul 2024 at 21:38, Rob Clark wrote: > > > > > > > > > > > > > > On Mon, Jul 8, 2024 at 1:52 AM Daniel Vetter > > > > > > > wrote: > > > > > > > > > > > > > > > > On Fri, Jul 05, 2024 at 12:31:36PM -0700, Rob Clark wrote: > > > > > > > > > On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote: > > > > > > > > > > > On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark > > > > > > > > > > > > wrote: > > > > > > > > > > > > > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, > > > > > > > > > > > > > > > > Dmitry Baryshkov wrote: > > > > > > > > > > > > > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, > > > > > > > > > > > > > > > > > Daniel Vetter wrote: > > > > > > > > > > > > > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, > > > > > > > > > > > > > > > > > > Helen Koike wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote: > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote: > > > > > > > > > > > > > > > > > > > > > Basically, I often find myself > > > > > > > > > > > > > > > > > > > > > needing to merge CI patches on top of > > > > > > > > > > > > > > > > > > > > > msm-next in order to run CI, and then > > > > > > > > > > > > > > > > > > > > > after a clean CI run, reset HEAD > > > > > > > > > > > > > > > > > > > > > back before the merge and force-push. > > > > > > > > > > > > > > > > > > > > > Which isn't really how things > > > > > > > > > > > > > > > > > > > > > should work. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sounds more like you want an > > > > > > > > > > > > > > > > > > integration tree like drm-tip. Get msm > > > > > > > > > > > > > > > > > > branches integrated there, done. Backmerges > > > > > > > > > > > > > > > > > > just for integration testing > > > > > > > > > > > > > > > > > > are not a good idea indeed. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > But AFAIU this doesn't help for pre-merge > > > > > > > > > > > > > > > testing, ie. prior to a > > > > > > > > > > > > > > > patch landing in msm-next > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > My idea was to have a drm-ci-next managed similar > > > > > > > > > > > > > > > to drm-misc-next, if > > > > > > > > > > > > > > > we have needed drm/ci patches we could push them > > > > > > > > > > > > > > > to drm-ci-next, and > > > > > > > > > > > > > > > then merge that into the driver tree (along with > > > > > > > > > > > > > > > a PR from drm-ci-next > > > > > > > > > > > > > > > to Dave). > > > > > > > > > > > > > > > > > > > > > > > > > > > > I guess I'm confused about what kind of pre-merge > > > > > > > > > > > > > > testing we're talking > > > > > > > > > > > > > > about then ... Or maybe this just doesn't work too > > > > > > > > > > > > > > well with the linux > > > > > > > > > > > > > > kernel. The model is that you have some pile of > > > > > > > > > > > > > > trees, they're split up, > > > > > > > > > > > > > > and testing of all the trees together is done in > > > > > > > > > > > > > > integration trees like > > > > > > > > > > > > > > linux-next or drm-tip. > > > > > > > > > > > > > > > > > > > > > > > > > > pre-merge: for msm we've been collecting up patches > > > > > > > > > > > > > from list into a > > > > > > > > > > > > > fast-forward MR which triggers CI before merging to > > > > > > > > > > > > > msm-next/msm-fixes > > > > > > > > > > > > > > > > > > > > > > > > > > Ideally drm-misc and other trees would do similar, > > > > > > > > > > > > > we'd catch more > > > > > > > > > > > > > regressions that way. For example, in msm-next the > > > > > > > > > > > > > nodebugfs build is > > > > > > > > > > > > > currently broken, because we merge
Re: Time for drm-ci-next?
Hi,
On 12/09/24 11:18, Dmitry Baryshkov wrote:
On Mon, Sep 09, 2024 at 07:34:04AM GMT, Rob Clark wrote:
On Mon, Sep 9, 2024 at 2:54 AM Dmitry Baryshkov
wrote:
On Mon, 9 Sept 2024 at 10:50, Maxime Ripard wrote:
Hi,
On Tue, Jul 09, 2024 at 01:27:51AM GMT, Dmitry Baryshkov wrote:
On Mon, 8 Jul 2024 at 21:38, Rob Clark wrote:
On Mon, Jul 8, 2024 at 1:52 AM Daniel Vetter wrote:
On Fri, Jul 05, 2024 at 12:31:36PM -0700, Rob Clark wrote:
On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter wrote:
On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote:
On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter wrote:
On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote:
On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter wrote:
On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote:
On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter wrote:
On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov wrote:
On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote:
On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote:
On 24/06/2024 02:34, Vignesh Raman wrote:
Hi,
On 15/03/24 22:50, Rob Clark wrote:
Basically, I often find myself needing to merge CI patches on top of
msm-next in order to run CI, and then after a clean CI run, reset HEAD
back before the merge and force-push. Which isn't really how things
should work.
This sounds more like you want an integration tree like drm-tip. Get msm
branches integrated there, done. Backmerges just for integration testing
are not a good idea indeed.
But AFAIU this doesn't help for pre-merge testing, ie. prior to a
patch landing in msm-next
My idea was to have a drm-ci-next managed similar to drm-misc-next, if
we have needed drm/ci patches we could push them to drm-ci-next, and
then merge that into the driver tree (along with a PR from drm-ci-next
to Dave).
I guess I'm confused about what kind of pre-merge testing we're talking
about then ... Or maybe this just doesn't work too well with the linux
kernel. The model is that you have some pile of trees, they're split up,
and testing of all the trees together is done in integration trees like
linux-next or drm-tip.
pre-merge: for msm we've been collecting up patches from list into a
fast-forward MR which triggers CI before merging to msm-next/msm-fixes
Ideally drm-misc and other trees would do similar, we'd catch more
regressions that way. For example, in msm-next the nodebugfs build is
currently broken, because we merged drm-misc-next at a time when
komeda was broken:
https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520
If drm-misc was using pre-merge CI this would have been caught and the
offending patch dropped.
That sounds more like we should push on the drm-misc pre-merge CI boulder
to move it uphill, than add even more trees to make it even harder to get
there long term ...
Short term it helps locally to have finer trees, but only short term and
only very locally.
The path to have fewer trees (ideally only one for all of drm) is to
use gitlab MRs to land everything :-)
drm-ci-next is only a stop-gap.. but one that we need. The
${branchname}-external-fixes trick covers _most_ cases where we need
unrelated patches (ie. to fix random ToT breakage outside of drm or in
core drm), but it doesn't help when the needed changes are yml
(because gitlab processes all the yml before merging the
-external-fixes branch). This is where we need drm-ci-next, otherwise
we are having to create a separate MR which cherry-picks drm/ci
patches for doing the CI. This is a rather broken process.
So what I don't get is ... if we CI drm-misc, how does that not help
improve the situation here? Step one would be post-merge (i.e. just enable
CI in the repo), then get MRs going.
I guess post-merge is better than nothing.. but pre-merge is better.
post-merge can work if you have a "sheriff" system where someone
(perhaps on a rotation) is actively monitoring results and "revert and
ask questions later" when something breaks. Pre-merge ensures the
interested party is involved in the process ;-)
So ... make that happen? And it doesn't have to be for all of drm-misc,
mesa after all switched over to MR also on a bit a driver/area basis. So
agreeing among all drm-ci folks to use gitlab MR in drm-misc for pre-merge
testing shouldn't be that hard to make happen. And unlike a separate
branch it's not some kind of detour with a good chance to get stuck in a
local optimum.
Tree vs branch doesn't really have much in the way of distinction,
modulo gitlab permissions. In that it doesn't do much good if drm/ci
patches are landing on a different branch.
I guess what you are suggesting is that we have a single tree/branch
that drm/ci + drm/msm + (plus whoever else wants to get in on the
drm/ci, so probably at least vkms) lands patches into via gitlab MRs?
This again brings a separate CI-enabled tree. I think it has been
suggested to start with enabling DRM CI for drm-misc branches. Then we
can possibly
Re: Time for drm-ci-next?
On Mon, Sep 09, 2024 at 07:34:04AM GMT, Rob Clark wrote: > On Mon, Sep 9, 2024 at 2:54 AM Dmitry Baryshkov > wrote: > > > > On Mon, 9 Sept 2024 at 10:50, Maxime Ripard wrote: > > > > > > Hi, > > > > > > On Tue, Jul 09, 2024 at 01:27:51AM GMT, Dmitry Baryshkov wrote: > > > > On Mon, 8 Jul 2024 at 21:38, Rob Clark wrote: > > > > > > > > > > On Mon, Jul 8, 2024 at 1:52 AM Daniel Vetter > > > > > wrote: > > > > > > > > > > > > On Fri, Jul 05, 2024 at 12:31:36PM -0700, Rob Clark wrote: > > > > > > > On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter > > > > > > > wrote: > > > > > > > > > > > > > > > > On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote: > > > > > > > > > On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote: > > > > > > > > > > > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark > > > > > > > > > > > > wrote: > > > > > > > > > > > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry > > > > > > > > > > > > > > Baryshkov wrote: > > > > > > > > > > > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel > > > > > > > > > > > > > > > Vetter wrote: > > > > > > > > > > > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen > > > > > > > > > > > > > > > > Koike wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote: > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote: > > > > > > > > > > > > > > > > > > > Basically, I often find myself needing to > > > > > > > > > > > > > > > > > > > merge CI patches on top of > > > > > > > > > > > > > > > > > > > msm-next in order to run CI, and then > > > > > > > > > > > > > > > > > > > after a clean CI run, reset HEAD > > > > > > > > > > > > > > > > > > > back before the merge and force-push. > > > > > > > > > > > > > > > > > > > Which isn't really how things > > > > > > > > > > > > > > > > > > > should work. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sounds more like you want an integration > > > > > > > > > > > > > > > > tree like drm-tip. Get msm > > > > > > > > > > > > > > > > branches integrated there, done. Backmerges > > > > > > > > > > > > > > > > just for integration testing > > > > > > > > > > > > > > > > are not a good idea indeed. > > > > > > > > > > > > > > > > > > > > > > > > > > But AFAIU this doesn't help for pre-merge testing, > > > > > > > > > > > > > ie. prior to a > > > > > > > > > > > > > patch landing in msm-next > > > > > > > > > > > > > > > > > > > > > > > > > > My idea was to have a drm-ci-next managed similar to > > > > > > > > > > > > > drm-misc-next, if > > > > > > > > > > > > > we have needed drm/ci patches we could push them to > > > > > > > > > > > > > drm-ci-next, and > > > > > > > > > > > > > then merge that into the driver tree (along with a PR > > > > > > > > > > > > > from drm-ci-next > > > > > > > > > > > > > to Dave). > > > > > > > > > > > > > > > > > > > > > > > > I guess I'm confused about what kind of pre-merge > > > > > > > > > > > > testing we're talking > > > > > > > > > > > > about then ... Or maybe this just doesn't work too well > > > > > > > > > > > > with the linux > > > > > > > > > > > > kernel. The model is that you have some pile of trees, > > > > > > > > > > > > they're split up, > > > > > > > > > > > > and testing of all the trees together is done in > > > > > > > > > > > > integration trees like > > > > > > > > > > > > linux-next or drm-tip. > > > > > > > > > > > > > > > > > > > > > > pre-merge: for msm we've been collecting up patches from > > > > > > > > > > > list into a > > > > > > > > > > > fast-forward MR which triggers CI before merging to > > > > > > > > > > > msm-next/msm-fixes > > > > > > > > > > > > > > > > > > > > > > Ideally drm-misc and other trees would do similar, we'd > > > > > > > > > > > catch more > > > > > > > > > > > regressions that way. For example, in msm-next the > > > > > > > > > > > nodebugfs build is > > > > > > > > > > > currently broken, because we merged drm-misc-next at a > > > > > > > > > > > time when > > > > > > > > > > > komeda was broken: > > > > > > > > > > > > > > > > > > > > > > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520 > > > > > > > > > > > > > > > > > > > > > > If drm-misc was using pre-merge CI this would have been > > > > > > > > > > > caught and the > > > > > > > > > > > offending patch dropped. > > > > > > > > > > > > > > > > > > > > That sounds more like we should push on the drm-misc > > > > > > > > > > pre-merge CI bo
Re: Time for drm-ci-next?
On Mon, Sep 9, 2024 at 2:54 AM Dmitry Baryshkov wrote: > > On Mon, 9 Sept 2024 at 10:50, Maxime Ripard wrote: > > > > Hi, > > > > On Tue, Jul 09, 2024 at 01:27:51AM GMT, Dmitry Baryshkov wrote: > > > On Mon, 8 Jul 2024 at 21:38, Rob Clark wrote: > > > > > > > > On Mon, Jul 8, 2024 at 1:52 AM Daniel Vetter > > > > wrote: > > > > > > > > > > On Fri, Jul 05, 2024 at 12:31:36PM -0700, Rob Clark wrote: > > > > > > On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter > > > > > > wrote: > > > > > > > > > > > > > > On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote: > > > > > > > > On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote: > > > > > > > > > > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote: > > > > > > > > > > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry > > > > > > > > > > > > > Baryshkov wrote: > > > > > > > > > > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel > > > > > > > > > > > > > > Vetter wrote: > > > > > > > > > > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen > > > > > > > > > > > > > > > Koike wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote: > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote: > > > > > > > > > > > > > > > > > > Basically, I often find myself needing to > > > > > > > > > > > > > > > > > > merge CI patches on top of > > > > > > > > > > > > > > > > > > msm-next in order to run CI, and then after > > > > > > > > > > > > > > > > > > a clean CI run, reset HEAD > > > > > > > > > > > > > > > > > > back before the merge and force-push. > > > > > > > > > > > > > > > > > > Which isn't really how things > > > > > > > > > > > > > > > > > > should work. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sounds more like you want an integration > > > > > > > > > > > > > > > tree like drm-tip. Get msm > > > > > > > > > > > > > > > branches integrated there, done. Backmerges just > > > > > > > > > > > > > > > for integration testing > > > > > > > > > > > > > > > are not a good idea indeed. > > > > > > > > > > > > > > > > > > > > > > > > But AFAIU this doesn't help for pre-merge testing, ie. > > > > > > > > > > > > prior to a > > > > > > > > > > > > patch landing in msm-next > > > > > > > > > > > > > > > > > > > > > > > > My idea was to have a drm-ci-next managed similar to > > > > > > > > > > > > drm-misc-next, if > > > > > > > > > > > > we have needed drm/ci patches we could push them to > > > > > > > > > > > > drm-ci-next, and > > > > > > > > > > > > then merge that into the driver tree (along with a PR > > > > > > > > > > > > from drm-ci-next > > > > > > > > > > > > to Dave). > > > > > > > > > > > > > > > > > > > > > > I guess I'm confused about what kind of pre-merge testing > > > > > > > > > > > we're talking > > > > > > > > > > > about then ... Or maybe this just doesn't work too well > > > > > > > > > > > with the linux > > > > > > > > > > > kernel. The model is that you have some pile of trees, > > > > > > > > > > > they're split up, > > > > > > > > > > > and testing of all the trees together is done in > > > > > > > > > > > integration trees like > > > > > > > > > > > linux-next or drm-tip. > > > > > > > > > > > > > > > > > > > > pre-merge: for msm we've been collecting up patches from > > > > > > > > > > list into a > > > > > > > > > > fast-forward MR which triggers CI before merging to > > > > > > > > > > msm-next/msm-fixes > > > > > > > > > > > > > > > > > > > > Ideally drm-misc and other trees would do similar, we'd > > > > > > > > > > catch more > > > > > > > > > > regressions that way. For example, in msm-next the > > > > > > > > > > nodebugfs build is > > > > > > > > > > currently broken, because we merged drm-misc-next at a time > > > > > > > > > > when > > > > > > > > > > komeda was broken: > > > > > > > > > > > > > > > > > > > > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520 > > > > > > > > > > > > > > > > > > > > If drm-misc was using pre-merge CI this would have been > > > > > > > > > > caught and the > > > > > > > > > > offending patch dropped. > > > > > > > > > > > > > > > > > > That sounds more like we should push on the drm-misc > > > > > > > > > pre-merge CI boulder > > > > > > > > > to move it uphill, than add even more trees to make it even > > > > > > > > > harder to get > > > > > > > > > there long term ... > > > > > > > > > > > > > > > > > > Short term it helps locally to have finer trees, but only > > > > > > > > > short term
Re: Time for drm-ci-next?
On Mon, 9 Sept 2024 at 10:50, Maxime Ripard wrote: > > Hi, > > On Tue, Jul 09, 2024 at 01:27:51AM GMT, Dmitry Baryshkov wrote: > > On Mon, 8 Jul 2024 at 21:38, Rob Clark wrote: > > > > > > On Mon, Jul 8, 2024 at 1:52 AM Daniel Vetter > > > wrote: > > > > > > > > On Fri, Jul 05, 2024 at 12:31:36PM -0700, Rob Clark wrote: > > > > > On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter > > > > > wrote: > > > > > > > > > > > > On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote: > > > > > > > On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter > > > > > > > wrote: > > > > > > > > > > > > > > > > On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote: > > > > > > > > > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote: > > > > > > > > > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry > > > > > > > > > > > > Baryshkov wrote: > > > > > > > > > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen > > > > > > > > > > > > > > Koike wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote: > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote: > > > > > > > > > > > > > > > > > Basically, I often find myself needing to > > > > > > > > > > > > > > > > > merge CI patches on top of > > > > > > > > > > > > > > > > > msm-next in order to run CI, and then after a > > > > > > > > > > > > > > > > > clean CI run, reset HEAD > > > > > > > > > > > > > > > > > back before the merge and force-push. Which > > > > > > > > > > > > > > > > > isn't really how things > > > > > > > > > > > > > > > > > should work. > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sounds more like you want an integration tree > > > > > > > > > > > > > > like drm-tip. Get msm > > > > > > > > > > > > > > branches integrated there, done. Backmerges just > > > > > > > > > > > > > > for integration testing > > > > > > > > > > > > > > are not a good idea indeed. > > > > > > > > > > > > > > > > > > > > > > But AFAIU this doesn't help for pre-merge testing, ie. > > > > > > > > > > > prior to a > > > > > > > > > > > patch landing in msm-next > > > > > > > > > > > > > > > > > > > > > > My idea was to have a drm-ci-next managed similar to > > > > > > > > > > > drm-misc-next, if > > > > > > > > > > > we have needed drm/ci patches we could push them to > > > > > > > > > > > drm-ci-next, and > > > > > > > > > > > then merge that into the driver tree (along with a PR > > > > > > > > > > > from drm-ci-next > > > > > > > > > > > to Dave). > > > > > > > > > > > > > > > > > > > > I guess I'm confused about what kind of pre-merge testing > > > > > > > > > > we're talking > > > > > > > > > > about then ... Or maybe this just doesn't work too well > > > > > > > > > > with the linux > > > > > > > > > > kernel. The model is that you have some pile of trees, > > > > > > > > > > they're split up, > > > > > > > > > > and testing of all the trees together is done in > > > > > > > > > > integration trees like > > > > > > > > > > linux-next or drm-tip. > > > > > > > > > > > > > > > > > > pre-merge: for msm we've been collecting up patches from list > > > > > > > > > into a > > > > > > > > > fast-forward MR which triggers CI before merging to > > > > > > > > > msm-next/msm-fixes > > > > > > > > > > > > > > > > > > Ideally drm-misc and other trees would do similar, we'd catch > > > > > > > > > more > > > > > > > > > regressions that way. For example, in msm-next the nodebugfs > > > > > > > > > build is > > > > > > > > > currently broken, because we merged drm-misc-next at a time > > > > > > > > > when > > > > > > > > > komeda was broken: > > > > > > > > > > > > > > > > > > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520 > > > > > > > > > > > > > > > > > > If drm-misc was using pre-merge CI this would have been > > > > > > > > > caught and the > > > > > > > > > offending patch dropped. > > > > > > > > > > > > > > > > That sounds more like we should push on the drm-misc pre-merge > > > > > > > > CI boulder > > > > > > > > to move it uphill, than add even more trees to make it even > > > > > > > > harder to get > > > > > > > > there long term ... > > > > > > > > > > > > > > > > Short term it helps locally to have finer trees, but only short > > > > > > > > term and > > > > > > > > only very locally. > > > > > > > > > > > > > > The path to have fewer trees (ideally only one for all of drm) is > > > > > > > to > > > > > > > use gitlab MRs to land everything :-) > > > > > > > > > > > > > > drm-ci-next is only a stop-g
Re: Time for drm-ci-next?
Hi,
On Tue, Jul 09, 2024 at 01:27:51AM GMT, Dmitry Baryshkov wrote:
> On Mon, 8 Jul 2024 at 21:38, Rob Clark wrote:
> >
> > On Mon, Jul 8, 2024 at 1:52 AM Daniel Vetter wrote:
> > >
> > > On Fri, Jul 05, 2024 at 12:31:36PM -0700, Rob Clark wrote:
> > > > On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter
> > > > wrote:
> > > > >
> > > > > On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote:
> > > > > > On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter
> > > > > > wrote:
> > > > > > >
> > > > > > > On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote:
> > > > > > > > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote:
> > > > > > > > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry
> > > > > > > > > > > Baryshkov wrote:
> > > > > > > > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote:
> > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote:
> > > > > > > > > > > > > > > > Basically, I often find myself needing to merge
> > > > > > > > > > > > > > > > CI patches on top of
> > > > > > > > > > > > > > > > msm-next in order to run CI, and then after a
> > > > > > > > > > > > > > > > clean CI run, reset HEAD
> > > > > > > > > > > > > > > > back before the merge and force-push. Which
> > > > > > > > > > > > > > > > isn't really how things
> > > > > > > > > > > > > > > > should work.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This sounds more like you want an integration tree
> > > > > > > > > > > > > like drm-tip. Get msm
> > > > > > > > > > > > > branches integrated there, done. Backmerges just for
> > > > > > > > > > > > > integration testing
> > > > > > > > > > > > > are not a good idea indeed.
> > > > > > > > > >
> > > > > > > > > > But AFAIU this doesn't help for pre-merge testing, ie.
> > > > > > > > > > prior to a
> > > > > > > > > > patch landing in msm-next
> > > > > > > > > >
> > > > > > > > > > My idea was to have a drm-ci-next managed similar to
> > > > > > > > > > drm-misc-next, if
> > > > > > > > > > we have needed drm/ci patches we could push them to
> > > > > > > > > > drm-ci-next, and
> > > > > > > > > > then merge that into the driver tree (along with a PR from
> > > > > > > > > > drm-ci-next
> > > > > > > > > > to Dave).
> > > > > > > > >
> > > > > > > > > I guess I'm confused about what kind of pre-merge testing
> > > > > > > > > we're talking
> > > > > > > > > about then ... Or maybe this just doesn't work too well with
> > > > > > > > > the linux
> > > > > > > > > kernel. The model is that you have some pile of trees,
> > > > > > > > > they're split up,
> > > > > > > > > and testing of all the trees together is done in integration
> > > > > > > > > trees like
> > > > > > > > > linux-next or drm-tip.
> > > > > > > >
> > > > > > > > pre-merge: for msm we've been collecting up patches from list
> > > > > > > > into a
> > > > > > > > fast-forward MR which triggers CI before merging to
> > > > > > > > msm-next/msm-fixes
> > > > > > > >
> > > > > > > > Ideally drm-misc and other trees would do similar, we'd catch
> > > > > > > > more
> > > > > > > > regressions that way. For example, in msm-next the nodebugfs
> > > > > > > > build is
> > > > > > > > currently broken, because we merged drm-misc-next at a time when
> > > > > > > > komeda was broken:
> > > > > > > >
> > > > > > > > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520
> > > > > > > >
> > > > > > > > If drm-misc was using pre-merge CI this would have been caught
> > > > > > > > and the
> > > > > > > > offending patch dropped.
> > > > > > >
> > > > > > > That sounds more like we should push on the drm-misc pre-merge CI
> > > > > > > boulder
> > > > > > > to move it uphill, than add even more trees to make it even
> > > > > > > harder to get
> > > > > > > there long term ...
> > > > > > >
> > > > > > > Short term it helps locally to have finer trees, but only short
> > > > > > > term and
> > > > > > > only very locally.
> > > > > >
> > > > > > The path to have fewer trees (ideally only one for all of drm) is to
> > > > > > use gitlab MRs to land everything :-)
> > > > > >
> > > > > > drm-ci-next is only a stop-gap.. but one that we need. The
> > > > > > ${branchname}-external-fixes trick covers _most_ cases where we need
> > > > > > unrelated patches (ie. to fix random ToT breakage outside of drm or
> > > > > > in
> > > > > > core drm), but it doesn't help when the needed changes are yml
> > > > > > (bec
Re: Time for drm-ci-next?
On Mon, 8 Jul 2024 at 21:38, Rob Clark wrote:
>
> On Mon, Jul 8, 2024 at 1:52 AM Daniel Vetter wrote:
> >
> > On Fri, Jul 05, 2024 at 12:31:36PM -0700, Rob Clark wrote:
> > > On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter
> > > wrote:
> > > >
> > > > On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote:
> > > > > On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter
> > > > > wrote:
> > > > > >
> > > > > > On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote:
> > > > > > > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote:
> > > > > > > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov
> > > > > > > > > > wrote:
> > > > > > > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter
> > > > > > > > > > > wrote:
> > > > > > > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote:
> > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote:
> > > > > > > > > > > > > > > Basically, I often find myself needing to merge
> > > > > > > > > > > > > > > CI patches on top of
> > > > > > > > > > > > > > > msm-next in order to run CI, and then after a
> > > > > > > > > > > > > > > clean CI run, reset HEAD
> > > > > > > > > > > > > > > back before the merge and force-push. Which
> > > > > > > > > > > > > > > isn't really how things
> > > > > > > > > > > > > > > should work.
> > > > > > > > > > > >
> > > > > > > > > > > > This sounds more like you want an integration tree like
> > > > > > > > > > > > drm-tip. Get msm
> > > > > > > > > > > > branches integrated there, done. Backmerges just for
> > > > > > > > > > > > integration testing
> > > > > > > > > > > > are not a good idea indeed.
> > > > > > > > >
> > > > > > > > > But AFAIU this doesn't help for pre-merge testing, ie. prior
> > > > > > > > > to a
> > > > > > > > > patch landing in msm-next
> > > > > > > > >
> > > > > > > > > My idea was to have a drm-ci-next managed similar to
> > > > > > > > > drm-misc-next, if
> > > > > > > > > we have needed drm/ci patches we could push them to
> > > > > > > > > drm-ci-next, and
> > > > > > > > > then merge that into the driver tree (along with a PR from
> > > > > > > > > drm-ci-next
> > > > > > > > > to Dave).
> > > > > > > >
> > > > > > > > I guess I'm confused about what kind of pre-merge testing we're
> > > > > > > > talking
> > > > > > > > about then ... Or maybe this just doesn't work too well with
> > > > > > > > the linux
> > > > > > > > kernel. The model is that you have some pile of trees, they're
> > > > > > > > split up,
> > > > > > > > and testing of all the trees together is done in integration
> > > > > > > > trees like
> > > > > > > > linux-next or drm-tip.
> > > > > > >
> > > > > > > pre-merge: for msm we've been collecting up patches from list
> > > > > > > into a
> > > > > > > fast-forward MR which triggers CI before merging to
> > > > > > > msm-next/msm-fixes
> > > > > > >
> > > > > > > Ideally drm-misc and other trees would do similar, we'd catch more
> > > > > > > regressions that way. For example, in msm-next the nodebugfs
> > > > > > > build is
> > > > > > > currently broken, because we merged drm-misc-next at a time when
> > > > > > > komeda was broken:
> > > > > > >
> > > > > > > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520
> > > > > > >
> > > > > > > If drm-misc was using pre-merge CI this would have been caught
> > > > > > > and the
> > > > > > > offending patch dropped.
> > > > > >
> > > > > > That sounds more like we should push on the drm-misc pre-merge CI
> > > > > > boulder
> > > > > > to move it uphill, than add even more trees to make it even harder
> > > > > > to get
> > > > > > there long term ...
> > > > > >
> > > > > > Short term it helps locally to have finer trees, but only short
> > > > > > term and
> > > > > > only very locally.
> > > > >
> > > > > The path to have fewer trees (ideally only one for all of drm) is to
> > > > > use gitlab MRs to land everything :-)
> > > > >
> > > > > drm-ci-next is only a stop-gap.. but one that we need. The
> > > > > ${branchname}-external-fixes trick covers _most_ cases where we need
> > > > > unrelated patches (ie. to fix random ToT breakage outside of drm or in
> > > > > core drm), but it doesn't help when the needed changes are yml
> > > > > (because gitlab processes all the yml before merging the
> > > > > -external-fixes branch). This is where we need drm-ci-next, otherwise
> > > > > we are having to create a separate MR which cherry-picks drm/ci
> > > > > patches for doing the CI. This is a rather broken process.
> > > >
> > > > So what I d
Re: Time for drm-ci-next?
On Mon, Jul 8, 2024 at 1:52 AM Daniel Vetter wrote:
>
> On Fri, Jul 05, 2024 at 12:31:36PM -0700, Rob Clark wrote:
> > On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter wrote:
> > >
> > > On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote:
> > > > On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter
> > > > wrote:
> > > > >
> > > > > On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote:
> > > > > > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter
> > > > > > wrote:
> > > > > > >
> > > > > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote:
> > > > > > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov
> > > > > > > > > wrote:
> > > > > > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote:
> > > > > > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote:
> > > > > > > > > > > > > Hi,
> > > > > > > > > > > > >
> > > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote:
> > > > > > > > > > > > > > Basically, I often find myself needing to merge CI
> > > > > > > > > > > > > > patches on top of
> > > > > > > > > > > > > > msm-next in order to run CI, and then after a clean
> > > > > > > > > > > > > > CI run, reset HEAD
> > > > > > > > > > > > > > back before the merge and force-push. Which isn't
> > > > > > > > > > > > > > really how things
> > > > > > > > > > > > > > should work.
> > > > > > > > > > >
> > > > > > > > > > > This sounds more like you want an integration tree like
> > > > > > > > > > > drm-tip. Get msm
> > > > > > > > > > > branches integrated there, done. Backmerges just for
> > > > > > > > > > > integration testing
> > > > > > > > > > > are not a good idea indeed.
> > > > > > > >
> > > > > > > > But AFAIU this doesn't help for pre-merge testing, ie. prior to
> > > > > > > > a
> > > > > > > > patch landing in msm-next
> > > > > > > >
> > > > > > > > My idea was to have a drm-ci-next managed similar to
> > > > > > > > drm-misc-next, if
> > > > > > > > we have needed drm/ci patches we could push them to
> > > > > > > > drm-ci-next, and
> > > > > > > > then merge that into the driver tree (along with a PR from
> > > > > > > > drm-ci-next
> > > > > > > > to Dave).
> > > > > > >
> > > > > > > I guess I'm confused about what kind of pre-merge testing we're
> > > > > > > talking
> > > > > > > about then ... Or maybe this just doesn't work too well with the
> > > > > > > linux
> > > > > > > kernel. The model is that you have some pile of trees, they're
> > > > > > > split up,
> > > > > > > and testing of all the trees together is done in integration
> > > > > > > trees like
> > > > > > > linux-next or drm-tip.
> > > > > >
> > > > > > pre-merge: for msm we've been collecting up patches from list into a
> > > > > > fast-forward MR which triggers CI before merging to
> > > > > > msm-next/msm-fixes
> > > > > >
> > > > > > Ideally drm-misc and other trees would do similar, we'd catch more
> > > > > > regressions that way. For example, in msm-next the nodebugfs build
> > > > > > is
> > > > > > currently broken, because we merged drm-misc-next at a time when
> > > > > > komeda was broken:
> > > > > >
> > > > > > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520
> > > > > >
> > > > > > If drm-misc was using pre-merge CI this would have been caught and
> > > > > > the
> > > > > > offending patch dropped.
> > > > >
> > > > > That sounds more like we should push on the drm-misc pre-merge CI
> > > > > boulder
> > > > > to move it uphill, than add even more trees to make it even harder to
> > > > > get
> > > > > there long term ...
> > > > >
> > > > > Short term it helps locally to have finer trees, but only short term
> > > > > and
> > > > > only very locally.
> > > >
> > > > The path to have fewer trees (ideally only one for all of drm) is to
> > > > use gitlab MRs to land everything :-)
> > > >
> > > > drm-ci-next is only a stop-gap.. but one that we need. The
> > > > ${branchname}-external-fixes trick covers _most_ cases where we need
> > > > unrelated patches (ie. to fix random ToT breakage outside of drm or in
> > > > core drm), but it doesn't help when the needed changes are yml
> > > > (because gitlab processes all the yml before merging the
> > > > -external-fixes branch). This is where we need drm-ci-next, otherwise
> > > > we are having to create a separate MR which cherry-picks drm/ci
> > > > patches for doing the CI. This is a rather broken process.
> > >
> > > So what I don't get is ... if we CI drm-misc, how does that not help
> > > improve the situation here? Step one would be post-merge (i.e. just enable
> > > CI in the repo), then get MRs going.
> >
> > I guess post-merge is better than nothing.. but pre-merge is better.
> >
> > post-merge can work if you have a
Re: Time for drm-ci-next?
On Fri, Jul 05, 2024 at 12:31:36PM -0700, Rob Clark wrote:
> On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter wrote:
> >
> > On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote:
> > > On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter
> > > wrote:
> > > >
> > > > On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote:
> > > > > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter
> > > > > wrote:
> > > > > >
> > > > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote:
> > > > > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov
> > > > > > > > wrote:
> > > > > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote:
> > > > > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote:
> > > > > > > > > > > > Hi,
> > > > > > > > > > > >
> > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote:
> > > > > > > > > > > > > Basically, I often find myself needing to merge CI
> > > > > > > > > > > > > patches on top of
> > > > > > > > > > > > > msm-next in order to run CI, and then after a clean
> > > > > > > > > > > > > CI run, reset HEAD
> > > > > > > > > > > > > back before the merge and force-push. Which isn't
> > > > > > > > > > > > > really how things
> > > > > > > > > > > > > should work.
> > > > > > > > > >
> > > > > > > > > > This sounds more like you want an integration tree like
> > > > > > > > > > drm-tip. Get msm
> > > > > > > > > > branches integrated there, done. Backmerges just for
> > > > > > > > > > integration testing
> > > > > > > > > > are not a good idea indeed.
> > > > > > >
> > > > > > > But AFAIU this doesn't help for pre-merge testing, ie. prior to a
> > > > > > > patch landing in msm-next
> > > > > > >
> > > > > > > My idea was to have a drm-ci-next managed similar to
> > > > > > > drm-misc-next, if
> > > > > > > we have needed drm/ci patches we could push them to drm-ci-next,
> > > > > > > and
> > > > > > > then merge that into the driver tree (along with a PR from
> > > > > > > drm-ci-next
> > > > > > > to Dave).
> > > > > >
> > > > > > I guess I'm confused about what kind of pre-merge testing we're
> > > > > > talking
> > > > > > about then ... Or maybe this just doesn't work too well with the
> > > > > > linux
> > > > > > kernel. The model is that you have some pile of trees, they're
> > > > > > split up,
> > > > > > and testing of all the trees together is done in integration trees
> > > > > > like
> > > > > > linux-next or drm-tip.
> > > > >
> > > > > pre-merge: for msm we've been collecting up patches from list into a
> > > > > fast-forward MR which triggers CI before merging to msm-next/msm-fixes
> > > > >
> > > > > Ideally drm-misc and other trees would do similar, we'd catch more
> > > > > regressions that way. For example, in msm-next the nodebugfs build is
> > > > > currently broken, because we merged drm-misc-next at a time when
> > > > > komeda was broken:
> > > > >
> > > > > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520
> > > > >
> > > > > If drm-misc was using pre-merge CI this would have been caught and the
> > > > > offending patch dropped.
> > > >
> > > > That sounds more like we should push on the drm-misc pre-merge CI
> > > > boulder
> > > > to move it uphill, than add even more trees to make it even harder to
> > > > get
> > > > there long term ...
> > > >
> > > > Short term it helps locally to have finer trees, but only short term and
> > > > only very locally.
> > >
> > > The path to have fewer trees (ideally only one for all of drm) is to
> > > use gitlab MRs to land everything :-)
> > >
> > > drm-ci-next is only a stop-gap.. but one that we need. The
> > > ${branchname}-external-fixes trick covers _most_ cases where we need
> > > unrelated patches (ie. to fix random ToT breakage outside of drm or in
> > > core drm), but it doesn't help when the needed changes are yml
> > > (because gitlab processes all the yml before merging the
> > > -external-fixes branch). This is where we need drm-ci-next, otherwise
> > > we are having to create a separate MR which cherry-picks drm/ci
> > > patches for doing the CI. This is a rather broken process.
> >
> > So what I don't get is ... if we CI drm-misc, how does that not help
> > improve the situation here? Step one would be post-merge (i.e. just enable
> > CI in the repo), then get MRs going.
>
> I guess post-merge is better than nothing.. but pre-merge is better.
>
> post-merge can work if you have a "sheriff" system where someone
> (perhaps on a rotation) is actively monitoring results and "revert and
> ask questions later" when something breaks. Pre-merge ensures the
> interested party is involved in the process ;-)
So ... make that happen? And it doesn't have to be for all of drm-misc,
mesa after all switched over to MR also on a bit
Re: Time for drm-ci-next?
On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter wrote:
>
> On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote:
> > On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter wrote:
> > >
> > > On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote:
> > > > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter wrote:
> > > > >
> > > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote:
> > > > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter
> > > > > > wrote:
> > > > > > >
> > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov wrote:
> > > > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote:
> > > > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote:
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote:
> > > > > > > > > > > > Basically, I often find myself needing to merge CI
> > > > > > > > > > > > patches on top of
> > > > > > > > > > > > msm-next in order to run CI, and then after a clean CI
> > > > > > > > > > > > run, reset HEAD
> > > > > > > > > > > > back before the merge and force-push. Which isn't
> > > > > > > > > > > > really how things
> > > > > > > > > > > > should work.
> > > > > > > > >
> > > > > > > > > This sounds more like you want an integration tree like
> > > > > > > > > drm-tip. Get msm
> > > > > > > > > branches integrated there, done. Backmerges just for
> > > > > > > > > integration testing
> > > > > > > > > are not a good idea indeed.
> > > > > >
> > > > > > But AFAIU this doesn't help for pre-merge testing, ie. prior to a
> > > > > > patch landing in msm-next
> > > > > >
> > > > > > My idea was to have a drm-ci-next managed similar to drm-misc-next,
> > > > > > if
> > > > > > we have needed drm/ci patches we could push them to drm-ci-next, and
> > > > > > then merge that into the driver tree (along with a PR from
> > > > > > drm-ci-next
> > > > > > to Dave).
> > > > >
> > > > > I guess I'm confused about what kind of pre-merge testing we're
> > > > > talking
> > > > > about then ... Or maybe this just doesn't work too well with the linux
> > > > > kernel. The model is that you have some pile of trees, they're split
> > > > > up,
> > > > > and testing of all the trees together is done in integration trees
> > > > > like
> > > > > linux-next or drm-tip.
> > > >
> > > > pre-merge: for msm we've been collecting up patches from list into a
> > > > fast-forward MR which triggers CI before merging to msm-next/msm-fixes
> > > >
> > > > Ideally drm-misc and other trees would do similar, we'd catch more
> > > > regressions that way. For example, in msm-next the nodebugfs build is
> > > > currently broken, because we merged drm-misc-next at a time when
> > > > komeda was broken:
> > > >
> > > > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520
> > > >
> > > > If drm-misc was using pre-merge CI this would have been caught and the
> > > > offending patch dropped.
> > >
> > > That sounds more like we should push on the drm-misc pre-merge CI boulder
> > > to move it uphill, than add even more trees to make it even harder to get
> > > there long term ...
> > >
> > > Short term it helps locally to have finer trees, but only short term and
> > > only very locally.
> >
> > The path to have fewer trees (ideally only one for all of drm) is to
> > use gitlab MRs to land everything :-)
> >
> > drm-ci-next is only a stop-gap.. but one that we need. The
> > ${branchname}-external-fixes trick covers _most_ cases where we need
> > unrelated patches (ie. to fix random ToT breakage outside of drm or in
> > core drm), but it doesn't help when the needed changes are yml
> > (because gitlab processes all the yml before merging the
> > -external-fixes branch). This is where we need drm-ci-next, otherwise
> > we are having to create a separate MR which cherry-picks drm/ci
> > patches for doing the CI. This is a rather broken process.
>
> So what I don't get is ... if we CI drm-misc, how does that not help
> improve the situation here? Step one would be post-merge (i.e. just enable
> CI in the repo), then get MRs going.
I guess post-merge is better than nothing.. but pre-merge is better.
post-merge can work if you have a "sheriff" system where someone
(perhaps on a rotation) is actively monitoring results and "revert and
ask questions later" when something breaks. Pre-merge ensures the
interested party is involved in the process ;-)
BR,
-R
> > I could conceivably see a scenario where we are landing both drm/ci
> > and drm/msm changes via the same tree. But only if we were using
> > gitlab MRs and CI for landing all changes in that tree.
>
> Yeah that's kinda the world I'm hoping for.
> -Sima
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
Re: Time for drm-ci-next?
On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote:
> On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter wrote:
> >
> > On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote:
> > > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter wrote:
> > > >
> > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote:
> > > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter
> > > > > wrote:
> > > > > >
> > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov wrote:
> > > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote:
> > > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote:
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote:
> > > > > > > > > > > Basically, I often find myself needing to merge CI
> > > > > > > > > > > patches on top of
> > > > > > > > > > > msm-next in order to run CI, and then after a clean CI
> > > > > > > > > > > run, reset HEAD
> > > > > > > > > > > back before the merge and force-push. Which isn't really
> > > > > > > > > > > how things
> > > > > > > > > > > should work.
> > > > > > > >
> > > > > > > > This sounds more like you want an integration tree like
> > > > > > > > drm-tip. Get msm
> > > > > > > > branches integrated there, done. Backmerges just for
> > > > > > > > integration testing
> > > > > > > > are not a good idea indeed.
> > > > >
> > > > > But AFAIU this doesn't help for pre-merge testing, ie. prior to a
> > > > > patch landing in msm-next
> > > > >
> > > > > My idea was to have a drm-ci-next managed similar to drm-misc-next, if
> > > > > we have needed drm/ci patches we could push them to drm-ci-next, and
> > > > > then merge that into the driver tree (along with a PR from drm-ci-next
> > > > > to Dave).
> > > >
> > > > I guess I'm confused about what kind of pre-merge testing we're talking
> > > > about then ... Or maybe this just doesn't work too well with the linux
> > > > kernel. The model is that you have some pile of trees, they're split up,
> > > > and testing of all the trees together is done in integration trees like
> > > > linux-next or drm-tip.
> > >
> > > pre-merge: for msm we've been collecting up patches from list into a
> > > fast-forward MR which triggers CI before merging to msm-next/msm-fixes
> > >
> > > Ideally drm-misc and other trees would do similar, we'd catch more
> > > regressions that way. For example, in msm-next the nodebugfs build is
> > > currently broken, because we merged drm-misc-next at a time when
> > > komeda was broken:
> > >
> > > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520
> > >
> > > If drm-misc was using pre-merge CI this would have been caught and the
> > > offending patch dropped.
> >
> > That sounds more like we should push on the drm-misc pre-merge CI boulder
> > to move it uphill, than add even more trees to make it even harder to get
> > there long term ...
> >
> > Short term it helps locally to have finer trees, but only short term and
> > only very locally.
>
> The path to have fewer trees (ideally only one for all of drm) is to
> use gitlab MRs to land everything :-)
>
> drm-ci-next is only a stop-gap.. but one that we need. The
> ${branchname}-external-fixes trick covers _most_ cases where we need
> unrelated patches (ie. to fix random ToT breakage outside of drm or in
> core drm), but it doesn't help when the needed changes are yml
> (because gitlab processes all the yml before merging the
> -external-fixes branch). This is where we need drm-ci-next, otherwise
> we are having to create a separate MR which cherry-picks drm/ci
> patches for doing the CI. This is a rather broken process.
So what I don't get is ... if we CI drm-misc, how does that not help
improve the situation here? Step one would be post-merge (i.e. just enable
CI in the repo), then get MRs going.
> I could conceivably see a scenario where we are landing both drm/ci
> and drm/msm changes via the same tree. But only if we were using
> gitlab MRs and CI for landing all changes in that tree.
Yeah that's kinda the world I'm hoping for.
-Sima
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
Re: Time for drm-ci-next?
On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter wrote:
>
> On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote:
> > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter wrote:
> > >
> > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote:
> > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter wrote:
> > > > >
> > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov wrote:
> > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote:
> > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote:
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > On 15/03/24 22:50, Rob Clark wrote:
> > > > > > > > > > Basically, I often find myself needing to merge CI patches
> > > > > > > > > > on top of
> > > > > > > > > > msm-next in order to run CI, and then after a clean CI run,
> > > > > > > > > > reset HEAD
> > > > > > > > > > back before the merge and force-push. Which isn't really
> > > > > > > > > > how things
> > > > > > > > > > should work.
> > > > > > >
> > > > > > > This sounds more like you want an integration tree like drm-tip.
> > > > > > > Get msm
> > > > > > > branches integrated there, done. Backmerges just for integration
> > > > > > > testing
> > > > > > > are not a good idea indeed.
> > > >
> > > > But AFAIU this doesn't help for pre-merge testing, ie. prior to a
> > > > patch landing in msm-next
> > > >
> > > > My idea was to have a drm-ci-next managed similar to drm-misc-next, if
> > > > we have needed drm/ci patches we could push them to drm-ci-next, and
> > > > then merge that into the driver tree (along with a PR from drm-ci-next
> > > > to Dave).
> > >
> > > I guess I'm confused about what kind of pre-merge testing we're talking
> > > about then ... Or maybe this just doesn't work too well with the linux
> > > kernel. The model is that you have some pile of trees, they're split up,
> > > and testing of all the trees together is done in integration trees like
> > > linux-next or drm-tip.
> >
> > pre-merge: for msm we've been collecting up patches from list into a
> > fast-forward MR which triggers CI before merging to msm-next/msm-fixes
> >
> > Ideally drm-misc and other trees would do similar, we'd catch more
> > regressions that way. For example, in msm-next the nodebugfs build is
> > currently broken, because we merged drm-misc-next at a time when
> > komeda was broken:
> >
> > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520
> >
> > If drm-misc was using pre-merge CI this would have been caught and the
> > offending patch dropped.
>
> That sounds more like we should push on the drm-misc pre-merge CI boulder
> to move it uphill, than add even more trees to make it even harder to get
> there long term ...
>
> Short term it helps locally to have finer trees, but only short term and
> only very locally.
The path to have fewer trees (ideally only one for all of drm) is to
use gitlab MRs to land everything :-)
drm-ci-next is only a stop-gap.. but one that we need. The
${branchname}-external-fixes trick covers _most_ cases where we need
unrelated patches (ie. to fix random ToT breakage outside of drm or in
core drm), but it doesn't help when the needed changes are yml
(because gitlab processes all the yml before merging the
-external-fixes branch). This is where we need drm-ci-next, otherwise
we are having to create a separate MR which cherry-picks drm/ci
patches for doing the CI. This is a rather broken process.
I could conceivably see a scenario where we are landing both drm/ci
and drm/msm changes via the same tree. But only if we were using
gitlab MRs and CI for landing all changes in that tree.
BR,
-R
> -Sima
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
Re: Time for drm-ci-next?
On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote: > On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter wrote: > > > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote: > > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter wrote: > > > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov wrote: > > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote: > > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote: > > > > > > > > > > > > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote: > > > > > > > > Hi, > > > > > > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote: > > > > > > > > > Basically, I often find myself needing to merge CI patches on > > > > > > > > > top of > > > > > > > > > msm-next in order to run CI, and then after a clean CI run, > > > > > > > > > reset HEAD > > > > > > > > > back before the merge and force-push. Which isn't really how > > > > > > > > > things > > > > > > > > > should work. > > > > > > > > > > > > This sounds more like you want an integration tree like drm-tip. > > > > > > Get msm > > > > > > branches integrated there, done. Backmerges just for integration > > > > > > testing > > > > > > are not a good idea indeed. > > > > > > But AFAIU this doesn't help for pre-merge testing, ie. prior to a > > > patch landing in msm-next > > > > > > My idea was to have a drm-ci-next managed similar to drm-misc-next, if > > > we have needed drm/ci patches we could push them to drm-ci-next, and > > > then merge that into the driver tree (along with a PR from drm-ci-next > > > to Dave). > > > > I guess I'm confused about what kind of pre-merge testing we're talking > > about then ... Or maybe this just doesn't work too well with the linux > > kernel. The model is that you have some pile of trees, they're split up, > > and testing of all the trees together is done in integration trees like > > linux-next or drm-tip. > > pre-merge: for msm we've been collecting up patches from list into a > fast-forward MR which triggers CI before merging to msm-next/msm-fixes > > Ideally drm-misc and other trees would do similar, we'd catch more > regressions that way. For example, in msm-next the nodebugfs build is > currently broken, because we merged drm-misc-next at a time when > komeda was broken: > > https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520 > > If drm-misc was using pre-merge CI this would have been caught and the > offending patch dropped. That sounds more like we should push on the drm-misc pre-merge CI boulder to move it uphill, than add even more trees to make it even harder to get there long term ... Short term it helps locally to have finer trees, but only short term and only very locally. -Sima -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: Time for drm-ci-next?
On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter wrote: > > On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote: > > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter wrote: > > > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov wrote: > > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote: > > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote: > > > > > > > > > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote: > > > > > > > Hi, > > > > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote: > > > > > > > > Basically, I often find myself needing to merge CI patches on > > > > > > > > top of > > > > > > > > msm-next in order to run CI, and then after a clean CI run, > > > > > > > > reset HEAD > > > > > > > > back before the merge and force-push. Which isn't really how > > > > > > > > things > > > > > > > > should work. > > > > > > > > > > This sounds more like you want an integration tree like drm-tip. Get > > > > > msm > > > > > branches integrated there, done. Backmerges just for integration > > > > > testing > > > > > are not a good idea indeed. > > > > But AFAIU this doesn't help for pre-merge testing, ie. prior to a > > patch landing in msm-next > > > > My idea was to have a drm-ci-next managed similar to drm-misc-next, if > > we have needed drm/ci patches we could push them to drm-ci-next, and > > then merge that into the driver tree (along with a PR from drm-ci-next > > to Dave). > > I guess I'm confused about what kind of pre-merge testing we're talking > about then ... Or maybe this just doesn't work too well with the linux > kernel. The model is that you have some pile of trees, they're split up, > and testing of all the trees together is done in integration trees like > linux-next or drm-tip. pre-merge: for msm we've been collecting up patches from list into a fast-forward MR which triggers CI before merging to msm-next/msm-fixes Ideally drm-misc and other trees would do similar, we'd catch more regressions that way. For example, in msm-next the nodebugfs build is currently broken, because we merged drm-misc-next at a time when komeda was broken: https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520 If drm-misc was using pre-merge CI this would have been caught and the offending patch dropped. BR, -R > Criss-cross merging of trees just for integration testing is no-go. And > that seems to be the only reason you want drm-ci-next? > > Also, this sounds more like msm being in a separate tree is the pain point > here, and solving "we have too many trees" by adding more isn't a good > idea ... > > Or I'm just totally confused. > -Sima > -- > Daniel Vetter > Software Engineer, Intel Corporation > http://blog.ffwll.ch
Re: Time for drm-ci-next?
On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote: > On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter wrote: > > > > On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov wrote: > > > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote: > > > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote: > > > > > > > > > > > > > > > On 24/06/2024 02:34, Vignesh Raman wrote: > > > > > > Hi, > > > > > > > > > > > > On 15/03/24 22:50, Rob Clark wrote: > > > > > > > Basically, I often find myself needing to merge CI patches on top > > > > > > > of > > > > > > > msm-next in order to run CI, and then after a clean CI run, reset > > > > > > > HEAD > > > > > > > back before the merge and force-push. Which isn't really how > > > > > > > things > > > > > > > should work. > > > > > > > > This sounds more like you want an integration tree like drm-tip. Get msm > > > > branches integrated there, done. Backmerges just for integration testing > > > > are not a good idea indeed. > > But AFAIU this doesn't help for pre-merge testing, ie. prior to a > patch landing in msm-next > > My idea was to have a drm-ci-next managed similar to drm-misc-next, if > we have needed drm/ci patches we could push them to drm-ci-next, and > then merge that into the driver tree (along with a PR from drm-ci-next > to Dave). I guess I'm confused about what kind of pre-merge testing we're talking about then ... Or maybe this just doesn't work too well with the linux kernel. The model is that you have some pile of trees, they're split up, and testing of all the trees together is done in integration trees like linux-next or drm-tip. Criss-cross merging of trees just for integration testing is no-go. And that seems to be the only reason you want drm-ci-next? Also, this sounds more like msm being in a separate tree is the pain point here, and solving "we have too many trees" by adding more isn't a good idea ... Or I'm just totally confused. -Sima -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: Time for drm-ci-next?
On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter wrote:
>
> On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov wrote:
> > On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote:
> > > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote:
> > > >
> > > >
> > > > On 24/06/2024 02:34, Vignesh Raman wrote:
> > > > > Hi,
> > > > >
> > > > > On 15/03/24 22:50, Rob Clark wrote:
> > > > > > Basically, I often find myself needing to merge CI patches on top of
> > > > > > msm-next in order to run CI, and then after a clean CI run, reset
> > > > > > HEAD
> > > > > > back before the merge and force-push. Which isn't really how things
> > > > > > should work.
> > >
> > > This sounds more like you want an integration tree like drm-tip. Get msm
> > > branches integrated there, done. Backmerges just for integration testing
> > > are not a good idea indeed.
But AFAIU this doesn't help for pre-merge testing, ie. prior to a
patch landing in msm-next
My idea was to have a drm-ci-next managed similar to drm-misc-next, if
we have needed drm/ci patches we could push them to drm-ci-next, and
then merge that into the driver tree (along with a PR from drm-ci-next
to Dave).
BR,
-R
> >
> > Is it fine to get drm/msm{-fixes,-next} into drm-tip?
>
> Should be.
>
> > > What exactly is the issue in backmerging drm-misc-next (well through
> > > drm-next really)?
> >
> > drm-misc-next is its own tree with separate cadence, its own bugs and
> > misfeatures. But probably just picking up drm-next for the tests should
> > be fine.
>
> Well, more CI should make the situation better for everyone. And I don't
> think you can avoid issues with topic branches, since usually there's
> enough stuff going on that you still often need parts of drm-next. The
> clean topic branches only tend to happen with other subsystems, where the
> interactions are much less.
>
> I think aim for more integration testing first with something like
> drm-tip, one-off topic branches if really needed for e.g. the gitlab
> version upgrade (but still prefer backmerges if that's enough) and see
> where it goes?
>
> If other trees introduce bugs it's imo much better to hit them early than
> in the middle of the next merge window, which is what you'd get with
> maximum amount of topic branches and tree separation. And the merge window
> is already really wobbly, we need to make that better.
>
> Cheers, Sima
>
> > > Also if there is an issue, generally we do ad-hoc topic branches.
> > >
> > > I'm very very skeptical of boutique trees with tiny focus, we've had that
> > > before drm-misc, it's a mess. Definitely no enthusiasm for getting back
> > > to that kind of world.
> > > -Sima
> >
> > --
> > With best wishes
> > Dmitry
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
Re: Time for drm-ci-next?
On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov wrote:
> On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote:
> > On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote:
> > >
> > >
> > > On 24/06/2024 02:34, Vignesh Raman wrote:
> > > > Hi,
> > > >
> > > > On 15/03/24 22:50, Rob Clark wrote:
> > > > > Basically, I often find myself needing to merge CI patches on top of
> > > > > msm-next in order to run CI, and then after a clean CI run, reset HEAD
> > > > > back before the merge and force-push. Which isn't really how things
> > > > > should work.
> >
> > This sounds more like you want an integration tree like drm-tip. Get msm
> > branches integrated there, done. Backmerges just for integration testing
> > are not a good idea indeed.
>
> Is it fine to get drm/msm{-fixes,-next} into drm-tip?
Should be.
> > What exactly is the issue in backmerging drm-misc-next (well through
> > drm-next really)?
>
> drm-misc-next is its own tree with separate cadence, its own bugs and
> misfeatures. But probably just picking up drm-next for the tests should
> be fine.
Well, more CI should make the situation better for everyone. And I don't
think you can avoid issues with topic branches, since usually there's
enough stuff going on that you still often need parts of drm-next. The
clean topic branches only tend to happen with other subsystems, where the
interactions are much less.
I think aim for more integration testing first with something like
drm-tip, one-off topic branches if really needed for e.g. the gitlab
version upgrade (but still prefer backmerges if that's enough) and see
where it goes?
If other trees introduce bugs it's imo much better to hit them early than
in the middle of the next merge window, which is what you'd get with
maximum amount of topic branches and tree separation. And the merge window
is already really wobbly, we need to make that better.
Cheers, Sima
> > Also if there is an issue, generally we do ad-hoc topic branches.
> >
> > I'm very very skeptical of boutique trees with tiny focus, we've had that
> > before drm-misc, it's a mess. Definitely no enthusiasm for getting back
> > to that kind of world.
> > -Sima
>
> --
> With best wishes
> Dmitry
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
Re: Time for drm-ci-next?
On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote:
> On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote:
> >
> >
> > On 24/06/2024 02:34, Vignesh Raman wrote:
> > > Hi,
> > >
> > > On 15/03/24 22:50, Rob Clark wrote:
> > > > Basically, I often find myself needing to merge CI patches on top of
> > > > msm-next in order to run CI, and then after a clean CI run, reset HEAD
> > > > back before the merge and force-push. Which isn't really how things
> > > > should work.
>
> This sounds more like you want an integration tree like drm-tip. Get msm
> branches integrated there, done. Backmerges just for integration testing
> are not a good idea indeed.
Is it fine to get drm/msm{-fixes,-next} into drm-tip?
> What exactly is the issue in backmerging drm-misc-next (well through
> drm-next really)?
drm-misc-next is its own tree with separate cadence, its own bugs and
misfeatures. But probably just picking up drm-next for the tests should
be fine.
> Also if there is an issue, generally we do ad-hoc topic branches.
>
> I'm very very skeptical of boutique trees with tiny focus, we've had that
> before drm-misc, it's a mess. Definitely no enthusiasm for getting back
> to that kind of world.
> -Sima
--
With best wishes
Dmitry
Re: Time for drm-ci-next?
On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote: > > > On 24/06/2024 02:34, Vignesh Raman wrote: > > Hi, > > > > On 15/03/24 22:50, Rob Clark wrote: > > > On Fri, Mar 15, 2024 at 2:28 AM Jani Nikula > > > wrote: > > > > > > > > On Thu, 14 Mar 2024, Rob Clark wrote: > > > > > When we first merged drm/ci I was unsure if it would need it's own > > > > > -next branch. But after using it for a couple releases, a few times > > > > > I've found myself wanting to backmerge drm/ci changes without > > > > > necessarily backmerging all of drm-misc-next. > > > > > > > > > > So, maybe it makes some sense to have a drm-ci-next branch that > > > > > driver-maintainers could back-merge as-needed? > > > > > > > > That's a crossmerge instead of a backmerge, and I feel that could get > > > > messy. What if folks crossmerge drm-ci-next but it gets rejected for > > > > drm-next? Or the baselines are different, and the crossmerge pulls in > > > > way more stuff than it should? > > > > > > Yeah, it would defeat the point a bit of drm-ci-next was on too new of > > > a baseline, the whole point is to be able to merge CI changes without > > > pulling in unrelated changes. So drm-ci-next would need to base on > > > something older, like the previous kernel release tag. > > > > > > > IMO the route should be drm-ci-next -> pull request to drm-next -> > > > > backmerge drm-next to drivers and drm-misc-next. > > > > > > > > I'm not opposed to having drm-ci-next at all, mainly indifferent, but I > > > > question the merge flows. And then the question becomes, does my > > > > suggested merge flow complicate your original goal? > > > > > > > > > > I guess we could avoid merging drm-ci-next until it had been merged > > > into drm-next? Yes, either dedicated topic branch or only backmerging drm-next please, that's how we're handling the flow for all other subtrees too. > > > Basically, I often find myself needing to merge CI patches on top of > > > msm-next in order to run CI, and then after a clean CI run, reset HEAD > > > back before the merge and force-push. Which isn't really how things > > > should work. This sounds more like you want an integration tree like drm-tip. Get msm branches integrated there, done. Backmerges just for integration testing are not a good idea indeed. > > There are many CI patches merged recently to drm-misc-next. > > With the GitLab 18.0 release, CI/CD pipeline configurations must > > transition from using the deprecated CI_JOB_JWT to the new id_tokens > > method, as the former will be removed. > > > > Without the below commit kernel-build job pipelines fail in drm-ci, > > https://gitlab.freedesktop.org/drm/misc/kernel/-/commit/cc806b74466672a9bbd4e9a04265d44eb506b686 > > > > We need to cherry pick only this commit to fix this issue. > > So it would be beneficial to have a drm-ci-next branch. > > > > Regards, > > Vignesh > > > I don't mind using a drm-ci-next branch if it is created. What exactly is the issue in backmerging drm-misc-next (well through drm-next really)? Also if there is an issue, generally we do ad-hoc topic branches. I'm very very skeptical of boutique trees with tiny focus, we've had that before drm-misc, it's a mess. Definitely no enthusiasm for getting back to that kind of world. -Sima -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: Time for drm-ci-next?
On 24/06/2024 02:34, Vignesh Raman wrote: Hi, On 15/03/24 22:50, Rob Clark wrote: On Fri, Mar 15, 2024 at 2:28 AM Jani Nikula wrote: On Thu, 14 Mar 2024, Rob Clark wrote: When we first merged drm/ci I was unsure if it would need it's own -next branch. But after using it for a couple releases, a few times I've found myself wanting to backmerge drm/ci changes without necessarily backmerging all of drm-misc-next. So, maybe it makes some sense to have a drm-ci-next branch that driver-maintainers could back-merge as-needed? That's a crossmerge instead of a backmerge, and I feel that could get messy. What if folks crossmerge drm-ci-next but it gets rejected for drm-next? Or the baselines are different, and the crossmerge pulls in way more stuff than it should? Yeah, it would defeat the point a bit of drm-ci-next was on too new of a baseline, the whole point is to be able to merge CI changes without pulling in unrelated changes. So drm-ci-next would need to base on something older, like the previous kernel release tag. IMO the route should be drm-ci-next -> pull request to drm-next -> backmerge drm-next to drivers and drm-misc-next. I'm not opposed to having drm-ci-next at all, mainly indifferent, but I question the merge flows. And then the question becomes, does my suggested merge flow complicate your original goal? I guess we could avoid merging drm-ci-next until it had been merged into drm-next? Basically, I often find myself needing to merge CI patches on top of msm-next in order to run CI, and then after a clean CI run, reset HEAD back before the merge and force-push. Which isn't really how things should work. There are many CI patches merged recently to drm-misc-next. With the GitLab 18.0 release, CI/CD pipeline configurations must transition from using the deprecated CI_JOB_JWT to the new id_tokens method, as the former will be removed. Without the below commit kernel-build job pipelines fail in drm-ci, https://gitlab.freedesktop.org/drm/misc/kernel/-/commit/cc806b74466672a9bbd4e9a04265d44eb506b686 We need to cherry pick only this commit to fix this issue. So it would be beneficial to have a drm-ci-next branch. Regards, Vignesh I don't mind using a drm-ci-next branch if it is created. Thanks Helen
Re: Time for drm-ci-next?
Hi, On 15/03/24 22:50, Rob Clark wrote: On Fri, Mar 15, 2024 at 2:28 AM Jani Nikula wrote: On Thu, 14 Mar 2024, Rob Clark wrote: When we first merged drm/ci I was unsure if it would need it's own -next branch. But after using it for a couple releases, a few times I've found myself wanting to backmerge drm/ci changes without necessarily backmerging all of drm-misc-next. So, maybe it makes some sense to have a drm-ci-next branch that driver-maintainers could back-merge as-needed? That's a crossmerge instead of a backmerge, and I feel that could get messy. What if folks crossmerge drm-ci-next but it gets rejected for drm-next? Or the baselines are different, and the crossmerge pulls in way more stuff than it should? Yeah, it would defeat the point a bit of drm-ci-next was on too new of a baseline, the whole point is to be able to merge CI changes without pulling in unrelated changes. So drm-ci-next would need to base on something older, like the previous kernel release tag. IMO the route should be drm-ci-next -> pull request to drm-next -> backmerge drm-next to drivers and drm-misc-next. I'm not opposed to having drm-ci-next at all, mainly indifferent, but I question the merge flows. And then the question becomes, does my suggested merge flow complicate your original goal? I guess we could avoid merging drm-ci-next until it had been merged into drm-next? Basically, I often find myself needing to merge CI patches on top of msm-next in order to run CI, and then after a clean CI run, reset HEAD back before the merge and force-push. Which isn't really how things should work. There are many CI patches merged recently to drm-misc-next. With the GitLab 18.0 release, CI/CD pipeline configurations must transition from using the deprecated CI_JOB_JWT to the new id_tokens method, as the former will be removed. Without the below commit kernel-build job pipelines fail in drm-ci, https://gitlab.freedesktop.org/drm/misc/kernel/-/commit/cc806b74466672a9bbd4e9a04265d44eb506b686 We need to cherry pick only this commit to fix this issue. So it would be beneficial to have a drm-ci-next branch. Regards, Vignesh
Re: Time for drm-ci-next?
On Fri, Mar 15, 2024 at 2:28 AM Jani Nikula wrote: > > On Thu, 14 Mar 2024, Rob Clark wrote: > > When we first merged drm/ci I was unsure if it would need it's own > > -next branch. But after using it for a couple releases, a few times > > I've found myself wanting to backmerge drm/ci changes without > > necessarily backmerging all of drm-misc-next. > > > > So, maybe it makes some sense to have a drm-ci-next branch that > > driver-maintainers could back-merge as-needed? > > That's a crossmerge instead of a backmerge, and I feel that could get > messy. What if folks crossmerge drm-ci-next but it gets rejected for > drm-next? Or the baselines are different, and the crossmerge pulls in > way more stuff than it should? Yeah, it would defeat the point a bit of drm-ci-next was on too new of a baseline, the whole point is to be able to merge CI changes without pulling in unrelated changes. So drm-ci-next would need to base on something older, like the previous kernel release tag. > IMO the route should be drm-ci-next -> pull request to drm-next -> > backmerge drm-next to drivers and drm-misc-next. > > I'm not opposed to having drm-ci-next at all, mainly indifferent, but I > question the merge flows. And then the question becomes, does my > suggested merge flow complicate your original goal? > I guess we could avoid merging drm-ci-next until it had been merged into drm-next? Basically, I often find myself needing to merge CI patches on top of msm-next in order to run CI, and then after a clean CI run, reset HEAD back before the merge and force-push. Which isn't really how things should work. BR, -R > > BR, > Jani. > > > -- > Jani Nikula, Intel
Re: Time for drm-ci-next?
On Thu, 14 Mar 2024, Rob Clark wrote: > When we first merged drm/ci I was unsure if it would need it's own > -next branch. But after using it for a couple releases, a few times > I've found myself wanting to backmerge drm/ci changes without > necessarily backmerging all of drm-misc-next. > > So, maybe it makes some sense to have a drm-ci-next branch that > driver-maintainers could back-merge as-needed? That's a crossmerge instead of a backmerge, and I feel that could get messy. What if folks crossmerge drm-ci-next but it gets rejected for drm-next? Or the baselines are different, and the crossmerge pulls in way more stuff than it should? IMO the route should be drm-ci-next -> pull request to drm-next -> backmerge drm-next to drivers and drm-misc-next. I'm not opposed to having drm-ci-next at all, mainly indifferent, but I question the merge flows. And then the question becomes, does my suggested merge flow complicate your original goal? BR, Jani. -- Jani Nikula, Intel
Time for drm-ci-next?
When we first merged drm/ci I was unsure if it would need it's own -next branch. But after using it for a couple releases, a few times I've found myself wanting to backmerge drm/ci changes without necessarily backmerging all of drm-misc-next. So, maybe it makes some sense to have a drm-ci-next branch that driver-maintainers could back-merge as-needed? Thoughts? BR, -R
