Re: proposal: remove "opportunity to carry out other desirable work" from the Exceptional Cases considerations in the blocker bug SOP

[Sudhir D]

+1 to the proposal.

On 10/21/20 11:37 PM, Matthew Miller wrote:

In https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process#Exceptional_cases,
there is a list of factors we may consider when dealing with last minute or
difficult to resolve blockers. One item on the list is:

* Whether delaying the release may give us an opportunity to carry out other 
desirable work

I would like to remove this from the list, and perhaps go so far as to include 
it
as something we _shouldn't_ consider.

There is always an infinite amount of desirable work, and it's tempting to
push the schedule just to get one more thing in. But! Admiral Ackbar meme
gif! that temptation is a trap! Any such additional work increases the risk
of further delay, and actually pushes off getting all the other work that
*did* make the release from getting to users.

If something isn't ready, there is always the next release, and often there
is the possibility of providing whatever other desirable work via an update
-- or through an alternate module stream after the the release.

What do you think? Reasonable? Thanks!


___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: Fedora 30 updates-testing report

[Sudhir D]

On 5/10/20 12:35 PM, upda...@fedoraproject.org wrote:

304https://bodhi.fedoraproject.org/updates/FEDORA-2019-c05e4425d1
dash-0.5.10.2-3.fc30



This issue is lingering around for long now. Status of BZ#1691825 
 is ON_QA which 
does not seem right since the issue seem unresolved in F31. Can we move 
this bz to correct status?



___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org

Re: 'No More Alphas': wiki revision drafts

[Sudhir D]

On 08/22/2017 07:34 PM, Matthew Miller wrote:

On Tue, Aug 22, 2017 at 07:18:04PM +0530, Sudhir D wrote:

On a slightly different thought, if we run all existing Alpha
criteria tests in rawhide, we can then probably look at existing
Alpha blocker as Branch blocker.. i.e, we don't branch unless the
blockers are fixed and thereby keeping rawhide at Alpha quality all
the time. We might want to call 'Basic Release Criteria' as 'Basic
Branch Criteria'.

H. I guess this would avoid needing to fix problems on both the
branch and rawhide. But, sometimes the fix should be different (e.g.
backport or quick fix on branch, update to new version or other long
term solution on rawhide).



Such fix must then be tracked separately for branched version.
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org

Re: 'No More Alphas': wiki revision drafts

[Sudhir D]

On 08/03/2017 05:46 AM, Adam Williamson wrote:

* My proposal for 'what do we do about release criteria / validation'
is basically: the 'Fedora 27 Alpha Release Criteria' page gets renamed
'Basic Release Criteria' (note: not versioned, I don't think it should
be), and we document that *all* composes - not just Beta and Final
candidate composes, but also Rawhide and Branched nightlies - will be
automatically tested for (and release of them gated on) compliance with
them. Which is more or less what's proposed in the Change. All external
references to the 'Alpha' criteria get changed to 'Basic' (e.g. this is
what changed on the other criteria pages, and on the test matrix
template pages). Practically speaking, we currently have automated
testing for *most* of the existing Alpha criteria, but a few items
aren't covered. We can choose to move these to the Beta criteria, or
leave them on Basic and just accept that *actual* coverage doesn't
quite meet what we aspire to. I do *NOT* propose to have any kind of
blocker tracking bug for the Basic release criteria; it doesn't seem to
fit in the process, there is no Alpha release to block, and we can't
realistically block nightly composes on manual test results. So a
tracker bug wouldn't really have any reason to exist. In the case where
a violation of the Basic criteria makes it into composes despite the
automated testing, it should be marked as a Beta blocker.




On a slightly different thought, if we run all existing Alpha criteria 
tests in rawhide, we can then probably look at existing Alpha blocker as 
Branch blocker.. i.e, we don't branch unless the blockers are fixed and 
thereby keeping rawhide at Alpha quality all the time. We might want to 
call 'Basic Release Criteria' as 'Basic Branch Criteria'.


Regards,
Sudhir
___
test mailing list -- test@lists.fedoraproject.org
To unsubscribe send an email to test-le...@lists.fedoraproject.org

Re: On vacation for a couple of weeks

[Sudhir D]

On 07/13/2016 01:29 AM, Adam Williamson wrote:


pschindl has done those when I've been away before, so he might be OK
with picking it up again, or if anyone else would like to try that's
fine. We'd only need a compose request if there's something we want to
test that requires a compose but which we don't want to push stable
before testing.


Anyone conducting today's meeting? Please nominate yourself :)

Cheers,
Sudhir


I'll be checking mail periodically while I'm away, so if you really
need me for anything, just drop a mail and I'll get back soon enough!

Thanks everyone :)

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: Requiring package test instructions

[Sudhir D]

On 07/13/2016 08:17 AM, Siddhesh Poyarekar wrote:


But then we're setting the bar too low by allowing *anyone* to set
karma for the sake of it.  You might as well just let developers push
packages to stable if they're 'confident' about it.


That is painting with too broad of a brush for one or two instance. We 
have to be sensitive to contributions coming from new members. It is our 
duty to show them the right way and not scram them away.




we need a way to put in control mechanisms (and maybe have a mentorship 
program) when we see
a pattern of incorrect behaviour.


Correct. There are many ways and having basic functionality testing 
details is one of them, which the proposal is about. Adam and others are 
doing their bit by having on-boarding calls etc.. to show the new 
joiners how to do Fedora testing the right way.


Cheers,
Sudhir
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

[New Members] Welcome and things to keep in mind.

[Sudhir D]

Hello to all new members!

You are guaranteed to have exciting time working with and testing 
Fedora. Enjoy the journey :)


There are some basic guidelines while responding on mailing list that is 
in-place for best and effective communication. You can find them @ 
https://fedoraproject.org/wiki/Mailing_list_guidelines
It is a bit long but it is also a one time read that will help you 
communicate better/effectively with other community members. Spend some 
time reading it and one of the important things to note there is to  
*not* top post :)


Happy testing and welcome again!

Cheers,
Sudhir
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://lists.fedoraproject.org/admin/lists/test@lists.fedoraproject.org

Re: Non-image blocker process change proposal

[Sudhir D]

On 11/19/2015 11:40 PM, Adam Williamson wrote:

On Thu, 2015-11-19 at 19:09 +0530, Sudhir D wrote:

I suggest we have only one ZeroDay i.e., for Final and do away with
intermediate ones.
As I see it, ZeroDay comes with cost and we also need to have basic
sanity testcases automated to ensure ZeroDay fixes won't
introduce/regress blocker.

How about automatically qualifying any freeze exception in current phase
as blocker for next phase and keep 0day only for RC?
AlphaBlocker --> AlphaFreezeException --> BetaBlocker -->
BetaFreezeException --> FinalBlocker --> FinalBlockerException --> ZeroDay

This would mean we will be not so liberal in allowing blockers linger
around in a phase for more time, but I think that is okay tradeoff.

  From tracking perspective, I think we may just want to have trackers
for phaseBlocker for each milestone and FinalBlocker and 0Day for Final
along with backPortfix tracker one for the pending release, and one for
previous stable releases.

Well, the thing is, the criteria are organized by milestone, and we hit
this situation quite often at Beta: the upgrade criteria kick in at
Beta, for instance. So if upgrade from F23 to F24 Beta is completely
broken, but the fix has to go out as an F23 update, we should really be
tracking that to make sure it does. If we only make sure the fix goes
out by Final, are we really honouring the criteria properly?


If a blocker bug breaks phase criteria, then there is no phase exit 
unless the bug is fixed. Unless we are ready to risk as it might happen 
in certain cases earlier in cycle but such instances should be zero once 
we are in Beta. That way, we would still be honoring the phase exit 
criteria. As a definition, BlockerExceptions should not contain any 
phase exit criteria bugs; these can be related to an important feature 
which is partially broken. For the Final phase though, all identified 
blockers and blockerExceptions that were carried from earlier phase are 
fixed before GA and if there is any exception in this phase out of that 
list (after risk assessment), we can consider them for 0day.




I don't think it's appropriate to turn FEs into blockers automatically,
in fact there are obvious cases where it certainly wouldn't be
appropriate: bugs in non-blocking desktops are typically taken as FEs,
for instance, as are bugs in secondary arches. Neither of those can
ever be blockers by policy.


Ok. We should probably stop calling them as FEs in that case :) and have 
a mechanism to track them on basis of priority and have them fixed 
before RC.


Regards,
Sudhir

Regards,
Sudhir

--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Non-image blocker process change proposal

[Sudhir D]

On 11/19/2015 06:06 AM, Adam Williamson wrote:

Hi, folks! It's been a recurring issue in the blocker review / release
validation process in recent times that we run across bugs that qualify
as blockers, but for which the fix does not need to be in the final
frozen media or install trees.

Common cases are bugs related to upgrading, especially since the
introduction of fedup and even more so of dnf-system-upgrade; most
upgrade-related issues can now be sufficiently fixed by package updates
to either the source or target release. Bugs to do with writing USB
media from the previous release, for instance, also often fall in this
category.

Up until now we've been sort of handwaving these as 'special blockers',
following the regular blocker process but noting in comments that they
don't block the compose. We haven't been tracking very hard if they
actually *are* being fixed with updates promptly, we've just been sort
of waving a magic wand and assuming it will happen. I just found one
which was supposed to be fixed with a 0-day update for Beta, but hadn't
been fixed yet: https://bugzilla.redhat.com/show_bug.cgi?id=1263230

So, we kinda need to do this better.

Up top I'd like to note there are really kind of two buckets of
'special blockers' for any given release. If the release being
validated is N, they are:

1) Bugs for which the fix must be in the 0-day update set for N
2) Bugs for which the fix must be stable in N-1 and N-2 by N release day

There will be a lot of process nerd detail involved in any fix, but
before any detailed drafts I'd like to suggest two broad possible
approaches and see what people think:

#1 Separate trackers


As a sort of on-the-spot PoC for F23 Beta, I created a new tracker bug
for bucket 1: https://bugzilla.redhat.com/show_bug.cgi?id=1264167

We could formalize that approach, and have a '0-day' blocker tracker
for each milestone. We could either just have one '0-day' tracker and
throw bugs for both the pending release and previous stable releases on
the same tracker and keep track of what needs updating where in each
bug, or we could have two 0-day trackers for each milestone, one for
the pending release, and one for previous stable releases.

So we'd have something like:

F24AlphaBlocker
F24AlphaFreezeException
F24Alpha0Day
(F24AlphaStable) (? - better alias suggestions welcome)

and so on. This is a lot of bugs, but there is a script to create them,
so we're not adding a bunch of onerous work.

So far as proposing bugs goes, I think we'd probably want to extend the
current somewhat flexible approach; formally you would nominate a bug
as a particular type of blocker/FE (by marking it as blocking the
appropriate tracker), but we would move things around in blocker review
meetings sensibly, as we currently do (when something is nominated as
FE but should really be a blocker, or vice versa). In blocker review
we'd go through all bugs nominated for any of the trackers, probably
starting with 'Blocker', then '0Day' and 'Stable', then
'FreezeException'.

#2 MOAR METADATA


The alternative is to make the existing Blocker trackers do more work.
In this model we wouldn't add any new tracker bugs; we'd just add new
'magic words' in the Whiteboard field. Right now, an accepted blocker
is identified by the string 'AcceptedBlocker' appearing in the
whiteboard field. We could simply add some more magical strings like
that: 'Accepted0Day' and 'AcceptedStable', say (better suggestions
welcome).

I kind of like this idea as it's less change and involves creating
fewer new bugs. We'd have to make some changes to blockerbugs either
way - tflink can say if either approach would be more work in
blockerbugs, but I'm gonna guess they'd be fairly similar.

With either approach, the basic goal is to make it more feasible to
keep an eye on each of the different categories of 'release blocker'
bugs; right now we have solid processes in place for ensuring the
'normal' blockers are all addressed in the release media, but we don't
have any processes in place for ensuring 0Day and Stable bugs actually
get updates shipped when we say they must.

My suggestion would be that we make sure 'blockerbugs' includes lists
of each type of blocker. Ahead of and at Go/No-Go meetings, we would
want to have a formal assurance from the person responsible for fixing
the bug that the fix would be provided by a certain time - say, one day
or two days ahead of the release date - and it would be QA's
responsibility to ensure the updates are tested promptly, and releng's
responsibility to ensure they are pushed on time after being tested. I
would suggest the Program Manager ought to have overall responsibility
for keeping an eye on the 0Day and Stable blocker lists and making sure
the maintainer, QA, and releng all did their jobs on time.

It'd be great if folks could post their general thoughts on this, and
any preference for option 1 or option 2. Thanks!


I suggest we have only one 

Re: Alpha Criterion Discussion: Desktop Backgrounds

[Sudhir D]

On 08/07/2015 10:41 PM, Kevin Fenzi wrote:

On Fri, 07 Aug 2015 10:03:34 -0700
Adam Williamson adamw...@fedoraproject.org wrote:


This is exactly what I *don't* want to happen, and what rather annoys
me about this whole business.

The blocker process is not a tool for the rest of the project to use
as a reminder system. It doesn't work very well for that. The
expectation the blocker process is built around is that people should
be working towards its requirements *all the time*, ideally well in
advance, and the blocker process is there to catch the relatively
*few* cases where things break unexpectedly. It's absolutely not
supposed to work like this:

* QA tests Thing and finds no-one ever even started working on it
* QA files bug and marks it blocker
* Team Thing starts working on Thing

I realize that's an exaggerated example, but it's just to make the
point clear. That's not a sane development process.

+1000.


+1k1.



I'd like to propose something somewhat shocking.

How about we get these things done in rawhide, so we are a release
ahead of things instead of scrambling to get them done at the last
minute all the time.

Cannot the design team work on F24 wallpaper _right_ now? get it in
rawhide and test that it's all there and then when we branch it's done,
they can work on f25 in rawhide and so on.


+1.

In this process we would have to work on both 24 and 25 simultaneously, 
with higher priority on 24. Cause on the day we branch from rawhide, 
they should have 25 wallpaper ready so that people don't get confused 
again :)


Another proposal would be to have a rawhide wallpaper with watermark 
(probably the same wallpaper as of of n+1 release with watermark saying 
rawhide). That way, design team would need not worry about 2 
simultaneous release wallpaper, if that helps.


Regards,
Sudhir


Just a thought...

kevin




-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: QA talks at Flock

[Sudhir D]

On 05/01/2015 03:01 AM, Adam Williamson wrote:

I was considering proposing a session which would just cover the 'new
bits' we've put in place since F21 - the revisions to the validation
test process (including relval/wikitcms), openQA, and some of the
taskotron stuff (as the taskotron session is a workshop rather than a
talk), but since we already have three proposals, I don't want to pile
on further...
I agree. I think we should be selective in submitting talks based on 
conference location. Places where Fedora community is active and there 
is general awareness among people who are attending, we should look at 
some advance topics while submitting talks.


Regards,
Sudhir
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Fedora QA subproject status report for council (in June)

[Sudhir D]

On 04/20/2015 11:18 PM, Matthew Miller wrote:

Whoo, beta is (almost) out the door — time for congratulations and
thanks to all of the QA team (and hopefully a nice deep breath).

I'm looking at planning future meetings for the Fedora Council, and one
thing we'd like to have is regular status reports from various
subprojects. (This, it turns out, is even part of the _definition_ of a
Fedora subproject, although in practice I don't think it's really been
happening, if it ever did.)

The idea is to have one group report monthly, with a prepared
slideshow. It doesn't need to be long or very formal, but something
more organized than an unstructured chat. It'd cover:

   - the current state of the subproject
   - future plans
   - things the team needs from the rest of the project
 - any blockers we can help unblock
 - big resource requests?

and I'm hoping to experiment with doing these as recorded video
sessions. (See
https://lists.fedoraproject.org/pipermail/council-discuss/2015-April/013298.html)
for more background and discussion.)

Is anyone from QA interested and willing to volunteer? There's an open
slot in the plan on June 8th, which, fingers crossed and knock on wood
and all of the superstitions, will be after the F22 release.



This is great initiative. Adam has this line item in QA meeting and we 
should have a name to represent soon.


Cheers,
Sudhir






--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test