Re: Release criterion proposal: upgrade methods

2012-10-08 Thread Kamil Paral
> I actually think it's fine, now I finally got my head around
> that - if you forget all the troubles we had with the rewrite, and
> forget the rest of my mail, and just read the final proposal I came
> up
> with, is it actually very difficult to understand?

No, it is not that difficult. I was speaking more generically, because we hit 
"all/any" issues very often with different criteria. The idea was not tied to 
this particular one.
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-10-05 Thread Adam Williamson
On Fri, 2012-10-05 at 04:55 -0400, Kamil Paral wrote:

> > I think we have to rejig the whole thing somehow. Um.
> > 
> > For each one of the release-blocking package sets ('minimal', and the
> > package sets for each one of the release-blocking desktops), it must
> > be
> > possible to successfully complete an upgrade from a fully updated
> > installation of the previous stable Fedora release with that package
> > set
> > installed, using any officially recommended upgrade mechanism. The
> > upgraded system must meet all release criteria."
> > 
> > I _think_ that gets it (with, again, the obvious corresponding
> > criterion
> > for Final). English, eh. Who'd use it.
> 
> That almost begs for a mathematical formula instead.
> 
> Now seriously - after reading the email, I wonder whether non-native
> English speakers will ever be able to distinguish these subtleties. I
> wanted to say "especially for those outside the QA team, who don't
> read this list often", but then I realized that I'll forget the
> correct meaning soon myself, and I'll have to ask for confirmation
> again and again in the future.
> 
> If we want to engage more community and link to these criteria, so
> that they tag blocker bugs, they have to be as accessible as possible.
> Maybe we could put explanations right into the criteria text? Like
> "any of supported mechanism (all of them must work)" or "any of
> supported mechanism (at least one must work)". It doesn't look pretty
> and it makes the text longer, but it makes it pretty clear.
> 
> Of course if we intend to use these criteria only in the core QA team,
> and ask community to tag blocker bugs intuitively (without really
> reading those criteria), then it's probably fine. But one day, one
> day, we will be deciding blocker bug status and no English language
> expert will be around, and then it's gonna be tough.
> 
> I'd rather have it longer and clearer.

So I think you have some good points for sure. Talking about the
criteria in general, I'd say we have the problem that we have four
requirements for the criteria which are sort of competing against each
other:

* Clarity / ease of understanding
* Precision
* Length
* Comprehensiveness

It's very hard to write criteria that are short, clear, precise and
comprehensive. We started out after the big revision for F13 with
criteria that were short and clear:

https://fedoraproject.org/wiki/Fedora_13_Alpha_Release_Criteria

but the problem is that over time it became clear they weren't precise
enough. We want the criteria to say what they mean, so when we had to
take a tricky case and decide exactly what we meant to cover with the
criteria, we've written that into the criteria, and we've added quite a
few over the years.

So I think we've definitely come to a point where the simple format we
currently have - a single numbered list of plain text rules - is getting
a little hard to manage, and it's certainly becoming a bit of a
wall-o-text. I think what we probably need to do is look at doing a
reshuffle of the overall presentation of the criteria, after F18. I'll
put it on the retrospective so someone can take it as a task for F19.
I'm happy to do it, but if anyone else wants to, please do - the
criteria have been kind of an 'adamw thing' and I think it's usually
healthy when you get a fresh perspective on this kind of thing.

Ideas I've had include splitting the criteria up into groups, having a
proper glossary of those terms we've given specific meaning, and having
a 'concordance' which explains particular conventions we've developed in
interpreting the criteria, like the conventions we have for
hardware-specific bugs and so on.

Coming back to this particular case, I left my thought process in there
for a bit of fun, but I don't think the final draft I came up with is
terrible. Looking at it linguistically, I think the worries about 'or'
and 'each' and 'any' and so on were almost red herrings: the key problem
is that we need to address in the criterion a somewhat complex
hypothetical scenario. The way the criterion was written at first, from
which we were trying to 'patch' it, was looking at a hypothetical
scenario where there's a single notional installation of Fedora 17 and
we're talking about the conditions for upgrading it to Fedora 18, say.
It started out with the text "It must be possible to successfully
complete an upgrade from a fully updated installation of the previous
stable Fedora release". The mental picture that gives you is, well, just
as it says: 'a fully updated installation...'. One single fully updated
installation.

All the trouble we had with the update came down to that limitation of
the original wording, because that's not actually the concept we really
want to express. We want to express the hypothetical scenario of
_several_ fully updated installations of the previous stable release,
and apply a certain condition to each one of them: the mental picture we
needed the criterion to express is 'you're sitting in a

Re: Release criterion proposal: upgrade methods

2012-10-05 Thread Matthew Miller
On Thu, Oct 04, 2012 at 06:50:18PM -0700, Adam Williamson wrote:
> For each one of the release-blocking package sets ('minimal', and the
> package sets for each one of the release-blocking desktops), it must be
> possible to successfully complete an upgrade from a fully updated
> installation of the previous stable Fedora release with that package set
> installed, using any officially recommended upgrade mechanism. The
> upgraded system must meet all release criteria."
> 
> I _think_ that gets it (with, again, the obvious corresponding criterion
> for Final). English, eh. Who'd use it.

+1 It's not pretty but it works.


-- 
Matthew Miller  ☁☁☁  Fedora Cloud Architect  ☁☁☁  
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-10-05 Thread Kamil Paral
> Doesn't work. English being the ridiculous language it is, 'any' in
> fact
> has precisely the same problem as 'or'. It can mean 'any one of
> (these
> things)' as well as 'all of (these things)'. You can read 'any
> release
> blocking package set' as 'any one release blocking package set'. In
> fact, we use 'any' in this sense *right there in the same criterion*
> -
> for Beta, we say 'any officially recommended upgrade mechanism',
> meaning
> any _one_ officially recommended upgrade mechanism (only one has to
> work). English, you can't beat it.
> 
> We can't use 'all', either, like we do for the mechanisms for Final,
> because it could be read as 'you have to be able to do an upgrade
> from a
> system with both KDE and GNOME installed'.
> 
> I _think_ - think, mind - that 'each' would work:
> 
> "It must be possible to successfully complete an upgrade from a fully
> updated installation of the previous stable Fedora release with each
> release-blocking package set ('minimal', and the package sets for all
> release-blocking desktop environments), using any officially
> recommended
> upgrade mechanism. The upgraded system must meet all release
> criteria."
> 
> and the corresponding thing for Final, obviously.
> 
> Actually, god, no, that doesn't work either, it has the same problem
> as
> 'all'. This is freaking hard. Quick, somebody call a university.
> 
> I think we have to rejig the whole thing somehow. Um.
> 
> For each one of the release-blocking package sets ('minimal', and the
> package sets for each one of the release-blocking desktops), it must
> be
> possible to successfully complete an upgrade from a fully updated
> installation of the previous stable Fedora release with that package
> set
> installed, using any officially recommended upgrade mechanism. The
> upgraded system must meet all release criteria."
> 
> I _think_ that gets it (with, again, the obvious corresponding
> criterion
> for Final). English, eh. Who'd use it.

That almost begs for a mathematical formula instead.

Now seriously - after reading the email, I wonder whether non-native English 
speakers will ever be able to distinguish these subtleties. I wanted to say 
"especially for those outside the QA team, who don't read this list often", but 
then I realized that I'll forget the correct meaning soon myself, and I'll have 
to ask for confirmation again and again in the future.

If we want to engage more community and link to these criteria, so that they 
tag blocker bugs, they have to be as accessible as possible. Maybe we could put 
explanations right into the criteria text? Like "any of supported mechanism 
(all of them must work)" or "any of supported mechanism (at least one must 
work)". It doesn't look pretty and it makes the text longer, but it makes it 
pretty clear.

Of course if we intend to use these criteria only in the core QA team, and ask 
community to tag blocker bugs intuitively (without really reading those 
criteria), then it's probably fine. But one day, one day, we will be deciding 
blocker bug status and no English language expert will be around, and then it's 
gonna be tough.

I'd rather have it longer and clearer.
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-10-04 Thread Adam Williamson
On Wed, 2012-10-03 at 08:10 -0600, Tim Flink wrote:
> On Mon, 1 Oct 2012 08:21:13 -0600
> Tim Flink  wrote:
> > It sounds like we're pretty much OK with these release criteria
> > changes
> > - if there are no objections, I'll make the wiki changes later tonight
> >   or tomorrow morning.
> 
> There was some concern about the interpretation of 'or' in the criteria
> that we accepted in the QA meeting on Monday, so I propose a slightly
> different wording to make the intent a bit more clear
> 
> For beta:
> 
> "It must be possible to successfully complete an upgrade from a
> fully updated installation of the previous stable Fedora release
> with any release blocking package set ('minimal' or any release-blocking
> desktop environment), using any officially recommended upgrade
> mechanism. The upgraded system must meet all release criteria."
> 
> For final:
> 
> "It must be possible to successfully complete an upgrade from a
> fully updated installation of the previous stable Fedora release
> with any release blocking package set ('minimal' or any release-blocking
> desktop environment), using all officially recommended upgrade
> mechanisms. The upgraded system must meet all release criteria."
> 
> Any thoughts?

Doesn't work. English being the ridiculous language it is, 'any' in fact
has precisely the same problem as 'or'. It can mean 'any one of (these
things)' as well as 'all of (these things)'. You can read 'any release
blocking package set' as 'any one release blocking package set'. In
fact, we use 'any' in this sense *right there in the same criterion* -
for Beta, we say 'any officially recommended upgrade mechanism', meaning
any _one_ officially recommended upgrade mechanism (only one has to
work). English, you can't beat it.

We can't use 'all', either, like we do for the mechanisms for Final,
because it could be read as 'you have to be able to do an upgrade from a
system with both KDE and GNOME installed'.

I _think_ - think, mind - that 'each' would work:

"It must be possible to successfully complete an upgrade from a fully
updated installation of the previous stable Fedora release with each
release-blocking package set ('minimal', and the package sets for all
release-blocking desktop environments), using any officially recommended
upgrade mechanism. The upgraded system must meet all release criteria."

and the corresponding thing for Final, obviously.

Actually, god, no, that doesn't work either, it has the same problem as
'all'. This is freaking hard. Quick, somebody call a university.

I think we have to rejig the whole thing somehow. Um.

For each one of the release-blocking package sets ('minimal', and the
package sets for each one of the release-blocking desktops), it must be
possible to successfully complete an upgrade from a fully updated
installation of the previous stable Fedora release with that package set
installed, using any officially recommended upgrade mechanism. The
upgraded system must meet all release criteria."

I _think_ that gets it (with, again, the obvious corresponding criterion
for Final). English, eh. Who'd use it.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-10-03 Thread Tim Flink
On Mon, 1 Oct 2012 08:21:13 -0600
Tim Flink  wrote:

> It sounds like we're pretty much OK with these release criteria
> changes
> - if there are no objections, I'll make the wiki changes later tonight
>   or tomorrow morning.

Per agreement at the 2012-10-01 Fedora QA meeting [1], I made changes to
the beta and final release requirement pages.

[1]http://meetbot.fedoraproject.org/fedora-meeting/2012-10-01/fedora-qa.2012-10-01-15.01.html

However, the wording change modifications to remove ambiguity around
the package sets is still pending. Once we have agreement on that,
we'll change the wording again.

Tim


signature.asc
Description: PGP signature
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-10-03 Thread Tim Flink
On Mon, 1 Oct 2012 08:21:13 -0600
Tim Flink  wrote:
> It sounds like we're pretty much OK with these release criteria
> changes
> - if there are no objections, I'll make the wiki changes later tonight
>   or tomorrow morning.

There was some concern about the interpretation of 'or' in the criteria
that we accepted in the QA meeting on Monday, so I propose a slightly
different wording to make the intent a bit more clear

For beta:

"It must be possible to successfully complete an upgrade from a
fully updated installation of the previous stable Fedora release
with any release blocking package set ('minimal' or any release-blocking
desktop environment), using any officially recommended upgrade
mechanism. The upgraded system must meet all release criteria."

For final:

"It must be possible to successfully complete an upgrade from a
fully updated installation of the previous stable Fedora release
with any release blocking package set ('minimal' or any release-blocking
desktop environment), using all officially recommended upgrade
mechanisms. The upgraded system must meet all release criteria."

Any thoughts?

Tim


signature.asc
Description: PGP signature
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-10-01 Thread Tim Flink
On Wed, 26 Sep 2012 08:41:48 -0600
Tim Flink  wrote:

> On Mon, 24 Sep 2012 10:32:01 -0600
> Tim Flink  wrote:
> 
> > As currently written, the upgrade criterion in the Fedora 18 beta
> > release requirements [1] reads:
> > 
> > The installer must be able to successfully complete an upgrade
> > installation from a clean, fully updated default installation (from
> > any official install medium) of the previous stable Fedora release,
> > either via preupgrade or by booting to the installer manually. The
> > upgraded system must meet all release criteria.
> 
> Wow, I'm glad that I didn't just change the wiki without sending
> something out to test@ - I thought this was a minor, uncontroversial
> change ... I was wrong :)
> 
> From the discussion in this thread, it sounds like we're proposing to
> change the one release criterion to two - one for beta, one for final.
> 
> For beta:
> 
> "It must be possible to successfully complete an upgrade from a fully
> updated installation of the previous stable Fedora release with the
> 'minimal' package set or the package set for a release-blocking
> desktop, using any officially recommended upgrade mechanism. The
> upgraded system must meet all release criteria."
> 
> For final:
> 
> "It must be possible to successfully complete an upgrade from a fully
> updated installation of the previous stable Fedora release with the
> 'minimal' package set or the package set for a release-blocking
> desktop, using all officially recommended upgrade mechanisms. The
> upgraded system must meet all release criteria."
> 
> The implication being that at least one upgrade method has to work for
> beta and all have to work for final.

It sounds like we're pretty much OK with these release criteria changes
- if there are no objections, I'll make the wiki changes later tonight
  or tomorrow morning.

Tim


signature.asc
Description: PGP signature
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread Matthew Miller
On Tue, Sep 25, 2012 at 08:20:48PM -0700, Adam Williamson wrote:
> I can kind of see arguments both ways; on the one hand, the burden of
> testing upgrades to the strictly limited extent we currently do is not a
> terribly harsh one, and it at least gives us some confidence that the
> basic upgrade mechanism is not irretrievably broken. On the other hand,
> the practical benefits of the testing are fairly marginal: that 'we know
> it's not completely impossible' is pretty much all we get out of it.
> 
> Any more thoughts down that road?

I think the practical benefits go beyond that, because testing can uncover
possible specific problems, which might either have workarounds or be
conditions we can document as "sorry, we know this will keep it from
working".

-- 
Matthew Miller  ☁☁☁  Fedora Cloud Architect  ☁☁☁  
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread Adam Williamson
On Wed, 2012-09-26 at 08:46 -0700, Adam Williamson wrote:
> On Wed, 2012-09-26 at 08:50 -0600, Tim Flink wrote:
> > On Wed, 26 Sep 2012 16:47:16 +0200
> > drago01  wrote:
> > 
> > > > Would we OK with shipping beta with only yum upgrades working? While
> > > > it's not currently a 'recommended' method for upgrades right now as
> > > > far as I know, that could certainly change.
> > > 
> > > No that way the upgrade method used by most users (which is even new
> > > code this time) will get even less testing.
> > 
> > Yeah, I'm pretty much in agreement with you here. Regardless of which
> > method is used by most users, I'm wary of leaving a new upgrade method
> > for final instead of getting most of the bugs out in beta even if that
> > does mean that we slip a bit.
> > 
> > I'm not sure how to get around this risk unless we require 'all' to
> > work for beta or pretend that we control which upgrade methods are
> > 'recommended'. Any suggestions?
> 
> I don't see this as a practical problem. There is no danger that we're
> somehow going to overturn years of Fedora history and declare between
> now and F18 GA that yum is a 'recommended upgrade method'.

To be clear on this - the scenario Tim is worried about, apparently, is
one where, say, for F18 Beta the new upgrade tool is broken, and there
is pressure to simply say 'yum is the recommended upgrade method' for
F18 Beta.

To be clear on my position, I would resist such a fudge. I'm only happy
with the 'any' wording so long as we don't allow the definition of the
'recommended upgrade methods' to be fudged from release to release. We
would have to insist that it refers to the project's permanent ongoing
recommendations and that they can't simply be changed from release to
release for convenience's sake.

I don't see the development team being happy with declaring yum a
permanently 'recommended' upgrade method, so I think as long as we stuck
to that line, the 'any' wording would be OK. But if we are worried that
line wouldn't be holdable, then I'd accept the 'any' wording is a
problem.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread Adam Williamson
On Wed, 2012-09-26 at 09:26 -0600, Stephen John Smoogen wrote:

>  A user can upgrade your XP -> Vista, XP -> 7?, 7->8 etc and it will
> mostly work. 

Well there's that, and also...I don't recall the details and I may be
wrong, but I thought I'd read that the 'upgrade' to 7 from anything but
Vista wasn't really an upgrade at all, it was 'clean install and we'll
try to keep some of your configuration'.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread Adam Williamson
On Wed, 2012-09-26 at 08:50 -0600, Tim Flink wrote:
> On Wed, 26 Sep 2012 16:47:16 +0200
> drago01  wrote:
> 
> > > Would we OK with shipping beta with only yum upgrades working? While
> > > it's not currently a 'recommended' method for upgrades right now as
> > > far as I know, that could certainly change.
> > 
> > No that way the upgrade method used by most users (which is even new
> > code this time) will get even less testing.
> 
> Yeah, I'm pretty much in agreement with you here. Regardless of which
> method is used by most users, I'm wary of leaving a new upgrade method
> for final instead of getting most of the bugs out in beta even if that
> does mean that we slip a bit.
> 
> I'm not sure how to get around this risk unless we require 'all' to
> work for beta or pretend that we control which upgrade methods are
> 'recommended'. Any suggestions?

I don't see this as a practical problem. There is no danger that we're
somehow going to overturn years of Fedora history and declare between
now and F18 GA that yum is a 'recommended upgrade method'.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread Adam Williamson
On Wed, 2012-09-26 at 08:41 -0600, Tim Flink wrote:

> Would we OK with shipping beta with only yum upgrades working? While
> it's not currently a 'recommended' method for upgrades right now as far
> as I know, that could certainly change.

If we started considering it a recommended method, then sure, I'd be
fine with that. I don't think that's likely, though.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread Stephen John Smoogen
On 26 September 2012 01:10, drago01  wrote:
> On Wed, Sep 26, 2012 at 5:20 AM, Adam Williamson  wrote:
>> On Tue, 2012-09-25 at 23:01 -0400, Richard Ryniker wrote:
>>
>>> Maybe someone with more fortitude (intellectual honesty? discipline?)
>>> than I will kill upgrade, and make the world a better place.  Or at least
>>> document that "upgrade" is offered only on a "good effort" basis, with no
>>> guarantee or support.
>>
>> That's more or less my take on it, and why I'd like to use the word
>> 'recommended' rather than 'supported'. I agree that it's very difficult
>> to convincingly suggest that upgrades are in any reasonable definition
>> 'supported'.
>>
>> (As a sidebar, it's worth noting that major version upgrades are
>> unsupported for RHEL, and Microsoft rarely offers true 'upgrades'
>> between Windows builds any more, and I think never recommended them for
>> enterprise use: vastly better funded and more conservative operating
>> system projects than Fedora nevertheless have the same problems. It all
>> rather indicates to me that 'supporting' major version upgrades of
>> operating systems is rather close to being an impossibility.)
>
> I have been always upgrading my systems, I do never reinstall (I never
> tried to skip a release but from N-1 to N always has been fine for me;
> the only time I did that was to move from i386 to x86_64 years ago).
> Also Vista -> 7 upgrade worked just fine for me. Same for OSX
> upgrades. Other linux distributions manage to support that as well.

Word definition problem here. What you mean by supported is not what
Adam andt others mean by supported. I think getting those definitions
in order would help get this conversation going versus sidelined.

The word supported has various legal meanings usually that when it is
said there is a guarentee of it working and that it will be fixed if
it does not work.

Microsoft, Apple and other companies have upgrade methods that mostly
work. They don't "support" them without an expensive support contract
and in many cases without a field engineer looking at what you are
going to upgrade and doing the work themselves (depending on the
system and level of contract.)

 A user can upgrade your XP -> Vista, XP -> 7?, 7->8 etc and it will
mostly work. On the offhand case that the box does not boot afterwords
or has problems.. they will be told to backup the system, install a
fresh copy of the OS, reinstall any 3rd party applications and then
restore user data from backups. In their test cases they will do
general upgrade testing but they have shipped where it just doesn't
work on some systems (Apple is easier to test for since they control
the hardware and have made installation of software very sandboxed so
that it mostly works.)



-- 
Stephen J Smoogen.
"Don't derail a useful feature for the 99% because you're not in it."
Linus Torvalds
"Years ago my mother used to say to me,... Elwood, you must be oh
so smart or oh so pleasant. Well, for years I was smart. I
recommend pleasant. You may quote me."  —James Stewart as Elwood P. Dowd
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread drago01
On Wed, Sep 26, 2012 at 4:50 PM, Tim Flink  wrote:
> On Wed, 26 Sep 2012 16:47:16 +0200
> drago01  wrote:
>
>> > Would we OK with shipping beta with only yum upgrades working? While
>> > it's not currently a 'recommended' method for upgrades right now as
>> > far as I know, that could certainly change.
>>
>> No that way the upgrade method used by most users (which is even new
>> code this time) will get even less testing.
>
> Yeah, I'm pretty much in agreement with you here. Regardless of which
> method is used by most users, I'm wary of leaving a new upgrade method
> for final instead of getting most of the bugs out in beta even if that
> does mean that we slip a bit.
>
> I'm not sure how to get around this risk unless we require 'all' to
> work for beta or pretend that we control which upgrade methods are
> 'recommended'. Any suggestions?

Simply require all to work by beta time. I see no benefit from the
beta / final distinction here.
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread Jóhann B. Guðmundsson

On 09/26/2012 02:47 PM, drago01 wrote:

No that way the upgrade method used by most users (which is even new
code this time) will get even less testing.


Yum upgrades ( even thou officially unsupported ) is the preferred 
method to upgrade Fedora in this country so I guess it's depends on 
where you are located which method is used by most user but regardless 
of that "official way" always takes precedence over "unofficial way" of 
doing things so if the new code is broken we just have to slip some more...


JBG
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread Jóhann B. Guðmundsson

On 09/26/2012 02:41 PM, Tim Flink wrote:

Would we OK with shipping beta with only yum upgrades working? While
it's not currently a 'recommended' method for upgrades right now as far
as I know, that could certainly change.


Ack
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread Tim Flink
On Wed, 26 Sep 2012 16:47:16 +0200
drago01  wrote:

> > Would we OK with shipping beta with only yum upgrades working? While
> > it's not currently a 'recommended' method for upgrades right now as
> > far as I know, that could certainly change.
> 
> No that way the upgrade method used by most users (which is even new
> code this time) will get even less testing.

Yeah, I'm pretty much in agreement with you here. Regardless of which
method is used by most users, I'm wary of leaving a new upgrade method
for final instead of getting most of the bugs out in beta even if that
does mean that we slip a bit.

I'm not sure how to get around this risk unless we require 'all' to
work for beta or pretend that we control which upgrade methods are
'recommended'. Any suggestions?

Tim


signature.asc
Description: PGP signature
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread drago01
On Wed, Sep 26, 2012 at 4:41 PM, Tim Flink  wrote:
> On Mon, 24 Sep 2012 10:32:01 -0600
> Tim Flink  wrote:
>
>> As currently written, the upgrade criterion in the Fedora 18 beta
>> release requirements [1] reads:
>>
>> The installer must be able to successfully complete an upgrade
>> installation from a clean, fully updated default installation (from
>> any official install medium) of the previous stable Fedora release,
>> either via preupgrade or by booting to the installer manually. The
>> upgraded system must meet all release criteria.
>
> Wow, I'm glad that I didn't just change the wiki without sending
> something out to test@ - I thought this was a minor, uncontroversial
> change ... I was wrong :)
>
> From the discussion in this thread, it sounds like we're proposing to
> change the one release criterion to two - one for beta, one for final.
>
> For beta:
>
> "It must be possible to successfully complete an upgrade from a fully
> updated installation of the previous stable Fedora release with the
> 'minimal' package set or the package set for a release-blocking desktop,
> using any officially recommended upgrade mechanism. The upgraded system
> must meet all release criteria."
>
> For final:
>
> "It must be possible to successfully complete an upgrade from a fully
> updated installation of the previous stable Fedora release with the
> 'minimal' package set or the package set for a release-blocking desktop,
> using all officially recommended upgrade mechanisms. The upgraded system
> must meet all release criteria."
>
> The implication being that at least one upgrade method has to work for
> beta and all have to work for final.
>
> Would we OK with shipping beta with only yum upgrades working? While
> it's not currently a 'recommended' method for upgrades right now as far
> as I know, that could certainly change.

No that way the upgrade method used by most users (which is even new
code this time) will get even less testing.
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread Tim Flink
On Mon, 24 Sep 2012 10:32:01 -0600
Tim Flink  wrote:

> As currently written, the upgrade criterion in the Fedora 18 beta
> release requirements [1] reads:
> 
> The installer must be able to successfully complete an upgrade
> installation from a clean, fully updated default installation (from
> any official install medium) of the previous stable Fedora release,
> either via preupgrade or by booting to the installer manually. The
> upgraded system must meet all release criteria.

Wow, I'm glad that I didn't just change the wiki without sending
something out to test@ - I thought this was a minor, uncontroversial
change ... I was wrong :)

From the discussion in this thread, it sounds like we're proposing to
change the one release criterion to two - one for beta, one for final.

For beta:

"It must be possible to successfully complete an upgrade from a fully
updated installation of the previous stable Fedora release with the
'minimal' package set or the package set for a release-blocking desktop,
using any officially recommended upgrade mechanism. The upgraded system
must meet all release criteria."

For final:

"It must be possible to successfully complete an upgrade from a fully
updated installation of the previous stable Fedora release with the
'minimal' package set or the package set for a release-blocking desktop,
using all officially recommended upgrade mechanisms. The upgraded system
must meet all release criteria."

The implication being that at least one upgrade method has to work for
beta and all have to work for final.

Would we OK with shipping beta with only yum upgrades working? While
it's not currently a 'recommended' method for upgrades right now as far
as I know, that could certainly change.

Tim


signature.asc
Description: PGP signature
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-26 Thread drago01
On Wed, Sep 26, 2012 at 5:20 AM, Adam Williamson  wrote:
> On Tue, 2012-09-25 at 23:01 -0400, Richard Ryniker wrote:
>
>> Maybe someone with more fortitude (intellectual honesty? discipline?)
>> than I will kill upgrade, and make the world a better place.  Or at least
>> document that "upgrade" is offered only on a "good effort" basis, with no
>> guarantee or support.
>
> That's more or less my take on it, and why I'd like to use the word
> 'recommended' rather than 'supported'. I agree that it's very difficult
> to convincingly suggest that upgrades are in any reasonable definition
> 'supported'.
>
> (As a sidebar, it's worth noting that major version upgrades are
> unsupported for RHEL, and Microsoft rarely offers true 'upgrades'
> between Windows builds any more, and I think never recommended them for
> enterprise use: vastly better funded and more conservative operating
> system projects than Fedora nevertheless have the same problems. It all
> rather indicates to me that 'supporting' major version upgrades of
> operating systems is rather close to being an impossibility.)

I have been always upgrading my systems, I do never reinstall (I never
tried to skip a release but from N-1 to N always has been fine for me;
the only time I did that was to move from i386 to x86_64 years ago).
Also Vista -> 7 upgrade worked just fine for me. Same for OSX
upgrades. Other linux distributions manage to support that as well.

> To bring this back to practicalities: I'd say this discussion represents
> a rather strong consensus that we don't see much value in
> *strengthening* the release criteria and validation testing as concerns
> upgrades. We are left with the option of preserving the existing status
> quo, wherein we effectively guarantee that precisely two fairly
> artificial cases will work, or of simply dropping the release criterion
> relating to upgrading and demoting the test cases to 'optional' status.

I don't see a reason why we should change anything. The status quo has
been fine and making it optional would just result into broken
upgrades (i.e not blocking for upgrade bugs etc).

> I can kind of see arguments both ways; on the one hand, the burden of
> testing upgrades to the strictly limited extent we currently do is not a
> terribly harsh one, and it at least gives us some confidence that the
> basic upgrade mechanism is not irretrievably broken. On the other hand,
> the practical benefits of the testing are fairly marginal: that 'we know
> it's not completely impossible' is pretty much all we get out of it.

I don't understand what you mean by "impossible"  ... we can test
specific cases like the usrmove migration pretty well (upgrade from
F16 to 17); we can test anaconda/preugrade/whatever ... everything
else is more or less the same as a regular update with just more
packages.

Sure we cannot test every single package whether it upgrades properly
but I don't see any reason why we should just say "it is impossible
lets just stop testing anything".
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-25 Thread Adam Williamson
On Tue, 2012-09-25 at 23:01 -0400, Richard Ryniker wrote:

> Maybe someone with more fortitude (intellectual honesty? discipline?)
> than I will kill upgrade, and make the world a better place.  Or at least
> document that "upgrade" is offered only on a "good effort" basis, with no
> guarantee or support.

That's more or less my take on it, and why I'd like to use the word
'recommended' rather than 'supported'. I agree that it's very difficult
to convincingly suggest that upgrades are in any reasonable definition
'supported'.

(As a sidebar, it's worth noting that major version upgrades are
unsupported for RHEL, and Microsoft rarely offers true 'upgrades'
between Windows builds any more, and I think never recommended them for
enterprise use: vastly better funded and more conservative operating
system projects than Fedora nevertheless have the same problems. It all
rather indicates to me that 'supporting' major version upgrades of
operating systems is rather close to being an impossibility.)

To bring this back to practicalities: I'd say this discussion represents
a rather strong consensus that we don't see much value in
*strengthening* the release criteria and validation testing as concerns
upgrades. We are left with the option of preserving the existing status
quo, wherein we effectively guarantee that precisely two fairly
artificial cases will work, or of simply dropping the release criterion
relating to upgrading and demoting the test cases to 'optional' status.

I can kind of see arguments both ways; on the one hand, the burden of
testing upgrades to the strictly limited extent we currently do is not a
terribly harsh one, and it at least gives us some confidence that the
basic upgrade mechanism is not irretrievably broken. On the other hand,
the practical benefits of the testing are fairly marginal: that 'we know
it's not completely impossible' is pretty much all we get out of it.

Any more thoughts down that road?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-25 Thread Richard Ryniker
"Upgrade" installation is a bizarre beast, because the result is not well
defined.  Yes, a newer set of packages is installed, but a new install
does that.  The reason "upgrade" is so seductive is the notion all one's
configuration and personalization is carried into the upgraded system,
whereas a new installation loses that.

If the only personalization is creation of one userid, that's pretty easy
and separate /home makes it even easier.  On the other hand, a system
with multiple users, complex firewall, e-mail, DHCP server, print, udev,
or other configurations (e.g. the whole /etc/alternatives structure) is
problematic.  The old files preserved by an "upgrade" installation may
not mean what they used to mean... new data or different formats may be
needed, there may even be a new component that replaced what used to be
configured, and this new component uses completely different data.

Think "rpmnew" on steroids.

The sheer number of possibilities and possible effects makes "supported"
a lie (in the sense a user would like it to mean).  The change to systemd
is a fine example, where a user would like "supported" to mean his
initscripts, upstart, or whatever is magically converted to systemd
formats that do exactly what used to be done.  Hah!

In practical terms, upgrade installation "support" means:

  You'll get something that resembles what you used to have, but is
  different.  If you do not notice any differences (except newer versions
  of packages), you are extremely lucky.  If something brakes, you can
  either try to repair the pieces, or perform a new installation.  You are
  welcome to report bugs, but if these cannot be reproduced in a new
  installation, it is likely they will be ignored ("not reproducible" or
  "won't fix", or simply languish until end-of-life).

I remark that "upgrade installation is only supported from the prior
release" simply means "upgrade from F14 to F15; update; upgrade from
F15 to F16; update; upgrade from F16 to F17; update; ..." is the
"supported" path.  Well, that will increase the proportion of new
installations; at least it is good in that respect.

Personally, I am as susceptible to the lure of "upgrade" as most, even
though I "know better."  I have gotten away with upgrade installations,
even through more than one release, but always something eventually
breaks, and a new installation is the proper solution.  I try now to put
more effort into good configuration records, and tools to help me
replicate configurations into new installations... and less into analysis
of what went awry in the last upgrade.

Intellectually, I understand "upgrade" is a snake in the grass, waiting
to bite.  Realistically, I expect soon to again think "Why not try an
upgrade, and see if it works?" and take the (initially) easier road.

Maybe someone with more fortitude (intellectual honesty? discipline?)
than I will kill upgrade, and make the world a better place.  Or at least
document that "upgrade" is offered only on a "good effort" basis, with no
guarantee or support.

Meanwhile, it is like that old, threadbare and torn shirt, overdue for
the dustbin, but "oh! so familiar and comfortable," that still hangs in
my closet and is my first choice when something better is not required.


-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-25 Thread Adam Williamson
On Tue, 2012-09-25 at 19:41 -0400, Matthew Miller wrote:
> On Tue, Sep 25, 2012 at 11:24:58PM +, "Jóhann B. Guðmundsson" wrote:
> > >It'd be kind of awesome if we also *knew* each release if three or four
> > >releases back is likely to work, even if the answer is "no".
> > Personally I think we should limit our "support" not go further back
> > then the release we are about to eol so we would support F16 being
> > upgrade to F18 but not F15 being upgraded to F18.
> 
> We want to encourage people to get off those old releases without abandoning
> Fedora. It's reasonable to say we can't *support* all those upgrade
> scenarios, but we should generally be predisposed to having them working.
> 
> If we chase all the users away, the whole excercise becomes rather
> pointless.

But I'd get to play so much more golf...
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-25 Thread Matthew Miller
On Tue, Sep 25, 2012 at 11:24:58PM +, "Jóhann B. Guðmundsson" wrote:
> >It'd be kind of awesome if we also *knew* each release if three or four
> >releases back is likely to work, even if the answer is "no".
> Personally I think we should limit our "support" not go further back
> then the release we are about to eol so we would support F16 being
> upgrade to F18 but not F15 being upgraded to F18.

We want to encourage people to get off those old releases without abandoning
Fedora. It's reasonable to say we can't *support* all those upgrade
scenarios, but we should generally be predisposed to having them working.

If we chase all the users away, the whole excercise becomes rather
pointless.


-- 
Matthew Miller  ☁☁☁  Fedora Cloud Architect  ☁☁☁  
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-25 Thread Jóhann B. Guðmundsson

On 09/25/2012 11:02 PM, Matthew Miller wrote:

On Tue, Sep 25, 2012 at 12:59:40PM -0700, Adam Williamson wrote:

Yeah. I have one system where I run rawhide, and I leap releases for
everything else -- but, usually I accept that I'm going to reinstall. I
don't think Fedora (or Red Hat) have *ever* promised that an upgrade from
anything but the previous release will work.

Richard pointed out that we actually do suggest (not promise, but...)
this in the install guide. The example cited suggests it's fine to
upgrade across like three releases. We probably ought to change that.

It'd be kind of awesome if we also *knew* each release if three or four
releases back is likely to work, even if the answer is "no".



Personally I think we should limit our "support" not go further back 
then the release we are about to eol so we would support F16 being 
upgrade to F18 but not F15 being upgraded to F18.


JBG
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-25 Thread Matthew Miller
On Tue, Sep 25, 2012 at 12:59:40PM -0700, Adam Williamson wrote:
> > Yeah. I have one system where I run rawhide, and I leap releases for
> > everything else -- but, usually I accept that I'm going to reinstall. I
> > don't think Fedora (or Red Hat) have *ever* promised that an upgrade from
> > anything but the previous release will work.
> Richard pointed out that we actually do suggest (not promise, but...)
> this in the install guide. The example cited suggests it's fine to
> upgrade across like three releases. We probably ought to change that.

It'd be kind of awesome if we also *knew* each release if three or four
releases back is likely to work, even if the answer is "no".

-- 
Matthew Miller  ☁☁☁  Fedora Cloud Architect  ☁☁☁  
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-25 Thread Adam Williamson
On Tue, 2012-09-25 at 08:15 -0400, Matthew Miller wrote:
> On Mon, Sep 24, 2012 at 04:06:23PM -0700, Adam Williamson wrote:
> > You could certainly make the case, yeah. Given that our excuse for
> > people who say 'you have to upgrade every six months' is to say 'no you
> > don't, because we support releases for 12, you can just upgrade every
> > second release!', so we kinda do tell people that.
> 
> Yeah. I have one system where I run rawhide, and I leap releases for
> everything else -- but, usually I accept that I'm going to reinstall. I
> don't think Fedora (or Red Hat) have *ever* promised that an upgrade from
> anything but the previous release will work.

Richard pointed out that we actually do suggest (not promise, but...)
this in the install guide. The example cited suggests it's fine to
upgrade across like three releases. We probably ought to change that.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-25 Thread Matthew Miller
On Mon, Sep 24, 2012 at 04:06:23PM -0700, Adam Williamson wrote:
> You could certainly make the case, yeah. Given that our excuse for
> people who say 'you have to upgrade every six months' is to say 'no you
> don't, because we support releases for 12, you can just upgrade every
> second release!', so we kinda do tell people that.

Yeah. I have one system where I run rawhide, and I leap releases for
everything else -- but, usually I accept that I'm going to reinstall. I
don't think Fedora (or Red Hat) have *ever* promised that an upgrade from
anything but the previous release will work.

I'm very much in favor of changing that given what you say above, but I
think maybe that in itself is a Feature.

-- 
Matthew Miller  ☁☁☁  Fedora Cloud Architect  ☁☁☁  
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Richard Ryniker
You have convinced me, Adam.  How much does it contribute to release
quality if excellent test criteria are perfectly validated, but the
documentation the end-user reads says Fedora does something different?
To that user, what he sees is clearly a fault.

QA may not be explicitly requested to vet end-user documentation, but
your admonition to have one shared description, not multiple and possibly
divergent descriptions, makes a lot of sense.  

If a situation arises where inconsistent documentation cannot be
reconciled, that is the time to seek a modification to QA criteria to
make clear what tests will be performed and what function validated..

You have played trump cards, and converted my problems into benefits.
Wow.

I worry some documentaiton pages may not make clear exactly what Fedora
version or time they describe, especially when they are located by search
engines that process queries with no specification about what Fedora
version is of interest, but this may actually make your position more
relevant.
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Adam Williamson
On Mon, 2012-09-24 at 18:15 -0700, Adam Williamson wrote:

> Another way to put it...I'd say that interlinking the criteria and the
> installation guide, as you outline it above, has provided us with a
> *benefit*: we have identified an inconsistency between what the
> installation guide reckons is reliable and what QA and devel are making
> any kind of effort to ensure actually is reliable. If we just wrote the
> criteria in some kind of silo where we had our own definitions of
> everything, it wouldn't make that problem go away, would it? The
> installation guide would still be out there and would still be advising
> people to do something that maybe it shouldn't be advising people to do.
> Given that Fedora's a collaborative effort, I'd say we should be
> consciously trying to interlink between teams as much as possible as a
> way to ensure we're all on the same page, not silo'ing off our efforts
> from each other because we're scared of what we might find out from
> working together...

Not to hammer the point too much, but there's another reason, now I come
to think of it. It's _precisely_ the same reason we require packages to
use shared system libraries, not static linking.

Let's say instead of referring to the 'Upgrading' wiki page for the
definition of Fedora's 'officially recommended upgrade methods', we just
read it once and write whatever it says into the criteria pages instead.
What did we just do? We static linked the officially recommended upgrade
methods.

Now, if that's the way Fedora as a project is going to do things, we're
not going to be the only ones! The installation guide will likely do the
same. I'm sure releng has some document somewhere which refers to
upgrade methods; that one will static link them too. The forums will
probably do the same thing in a sticky thread somewhere.

Now imagine we as a project decide to change our officially recommended
installation method - as, indeed, it appears we are currently doing.
What has to happen? Same exact problem you have when there's a
vulnerability in a statically linked library: we have to go around and
change every damn instance. We have to change the Upgrading page's text,
the criteria page's text, the releng page's text, the installation
guide, the forum sticky thread...

It's a different area but the same exact problem. As a general
principle, we should have *one* canonical reference point for things
like this, which everything else should refer to. Then when it changes,
you only have to update the canonical definition.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Adam Williamson
On Mon, 2012-09-24 at 20:21 -0400, Richard Ryniker wrote:
> >I think we could link to
> >https://fedoraproject.org/wiki/Upgrading to define
> >'supported/recommendation upgrade mechanism'
> 
> OK, but to illustrate the problem with indirect references:  the
> "Upgrading" page you cite above says to read
>   
> http://docs.fedoraproject.org/en-US/Fedora/17/html/Installation_Guide/index.html
> for details.  This document says (Chapter 20):
> 
>   Before upgrading to Fedora 17 you should first bring your current version
>   up to date. However, it is not then necessary to upgrade to intermediate
>   versions. For example, you can upgrade from Fedora 14 to Fedora 17
>   directly.
> 
> Therefore, it seems your recent post to this list:
>   http://lists.fedoraproject.org/pipermail/test/2012-September/110331.html
> may be incorrect.  I think you are right, but the quotation above
> contradicts your statement:
> 
>   Only 17-18: using the definite article 'the' rather than the indefinite
>   article 'a' implies this. It says 'the previous stable Fedora release' -
>   which strictly means "only the single preceding release"
> 
> My point here is that details in the QA criteria that specify explicitly
> what operations must work are valuable.  Indirect references to dynamic
> documents (which you properly describe as not owned by QA) mean somebody
> else, who may have different interests and objectives than QA and does
> not intend to write a QA criterion, defines what your criteria are.

Well, to an extent, yeah. To me that's not really a problem as much as
it is an opportunity, though. ;) It's a left hand, right hand issue; the
one should know what the other is doing. As you explain above, obviously
that's not quite the case there. I don't think that's an inherent
weakness of the idea as much as it is a case where we should correct
something, though. Either we should start testing upgrades from ancient
releases and taking it seriously when they fail, or we should stop
advising people to do so in the installation guide.

Another way to put it...I'd say that interlinking the criteria and the
installation guide, as you outline it above, has provided us with a
*benefit*: we have identified an inconsistency between what the
installation guide reckons is reliable and what QA and devel are making
any kind of effort to ensure actually is reliable. If we just wrote the
criteria in some kind of silo where we had our own definitions of
everything, it wouldn't make that problem go away, would it? The
installation guide would still be out there and would still be advising
people to do something that maybe it shouldn't be advising people to do.
Given that Fedora's a collaborative effort, I'd say we should be
consciously trying to interlink between teams as much as possible as a
way to ensure we're all on the same page, not silo'ing off our efforts
from each other because we're scared of what we might find out from
working together...

> >'official install media' is not quite as obvious; maybe it
> >should be a link to the Deliverables SOP when I or someone else finally
> >gets that done (my last draft is still at
> >https://fedoraproject.org/wiki/User:Adamwill/Draft_releng_SOP_deliverables
> 
> Your draft makes no mention of LiveUSB Creator or livecd-tools.  It only
> mentions "'dd' or similar tools".  Does this mean persistent storage,
> encrypted /home, and other features, are no longer supported on live
> media (or perhaps are simply not of concern to QA)?  

Before this goes viral, let me provide the all-important context. This
is the text Richard's referring to:

"All image files for PC architectures should be prepared for writing to
USB directly with 'dd' or similar tools, and should be prepared for both
EFI and BIOS booting whether written to an optical disc or a USB disk."

The answer to your question is no, it doesn't mean I don't care about
litd or livecd-creator. It just means there isn't any 'preparation' work
that has to be done to an image to make it writeable via
livecd-iso-to-disk. Our images are litd-able as they pop out of the
creator. That's not the case for dd; releng has to run mkisohybrid on
the images at some point to make them dd'able. Once or twice this hasn't
got done, hence I decided to specify it in the SOP.

> I gather there has
> been a lot of "back and forth" in this area of late - perhaps another
> case where explicit language in QA criteria may be valuable, even if it
> has to track evolving decisions about what sophisticated live system
> features will be "supported".

We do in fact have this. At Alpha:

"The installer must boot (if appropriate) and run on all primary
architectures, with all system firmware types that are common on those
architectures, from default live image, DVD, and boot.iso install media
when written to an optical disc and when written to a USB stick with at
least one of the officially supported methods"

('at least one' is in boldface, and 'officially supported methods' li

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Richard Ryniker
>I think we could link to
>https://fedoraproject.org/wiki/Upgrading to define
>'supported/recommendation upgrade mechanism'

OK, but to illustrate the problem with indirect references:  the
"Upgrading" page you cite above says to read
  
http://docs.fedoraproject.org/en-US/Fedora/17/html/Installation_Guide/index.html
for details.  This document says (Chapter 20):

  Before upgrading to Fedora 17 you should first bring your current version
  up to date. However, it is not then necessary to upgrade to intermediate
  versions. For example, you can upgrade from Fedora 14 to Fedora 17
  directly.

Therefore, it seems your recent post to this list:
  http://lists.fedoraproject.org/pipermail/test/2012-September/110331.html
may be incorrect.  I think you are right, but the quotation above
contradicts your statement:

  Only 17-18: using the definite article 'the' rather than the indefinite
  article 'a' implies this. It says 'the previous stable Fedora release' -
  which strictly means "only the single preceding release"

My point here is that details in the QA criteria that specify explicitly
what operations must work are valuable.  Indirect references to dynamic
documents (which you properly describe as not owned by QA) mean somebody
else, who may have different interests and objectives than QA and does
not intend to write a QA criterion, defines what your criteria are.

>'official install media' is not quite as obvious; maybe it
>should be a link to the Deliverables SOP when I or someone else finally
>gets that done (my last draft is still at
>https://fedoraproject.org/wiki/User:Adamwill/Draft_releng_SOP_deliverables

Your draft makes no mention of LiveUSB Creator or livecd-tools.  It only
mentions "'dd' or similar tools".  Does this mean persistent storage,
encrypted /home, and other features, are no longer supported on live
media (or perhaps are simply not of concern to QA)?  I gather there has
been a lot of "back and forth" in this area of late - perhaps another
case where explicit language in QA criteria may be valuable, even if it
has to track evolving decisions about what sophisticated live system
features will be "supported".

It is unlikely there will ever be perfect QA criteria, but these criteria
are important: the value of the QA effort depends in significant ways on
their quality.  Whatever their eventual form, I hope my comments help to
make them a little better.
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Stephen John Smoogen
On 24 September 2012 17:06, Adam Williamson  wrote:
> On Mon, 2012-09-24 at 21:52 +, "Jóhann B. Guðmundsson" wrote:
>
>> Do you know if we keep log in our infrastructure that shows how many are
>> actually upgrading on which version they do it from?
>
> I don't know that, no. I don't think we do. I suppose it might be
> possible to infer such information from the yum records, with a careful
> analysis, by looking at installations with reliable IP addresses and
> seeing their upgrade patterns. That might actually be kinda interesting,
> but I don't know if it's really possible. In general Fedora is pretty
> conservative about logging user information. As a F/OSS project, you run
> the risk of a bad case of Slashdotitis if you do anything else =)

We do not have a simple way of tracking upgrade methods nor users to
do so. Mainly for the reasons that IPs change a lot, what looks like
an upgrade turns out to be users behind a NAT, etc etc.


> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
> http://www.happyassassin.net
>
> --
> test mailing list
> test@lists.fedoraproject.org
> To unsubscribe:
> https://admin.fedoraproject.org/mailman/listinfo/test



-- 
Stephen J Smoogen.
"Don't derail a useful feature for the 99% because you're not in it."
Linus Torvalds
"Years ago my mother used to say to me,... Elwood, you must be oh
so smart or oh so pleasant. Well, for years I was smart. I
recommend pleasant. You may quote me."  —James Stewart as Elwood P. Dowd
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Adam Williamson
On Mon, 2012-09-24 at 21:52 +, "Jóhann B. Guðmundsson" wrote:
> On 09/24/2012 09:43 PM, Adam Williamson wrote:
> > Only 17-18: using the definite article 'the' rather than the indefinite
> > article 'a' implies this. It says 'the previous stable Fedora release' -
> > which strictly means "only the single preceding release" - not 'a
> > previous stable Fedora release' or 'any previous stable Fedora release'
> > or anything like that. I suppose it's a distinction which is clearer to
> > a native speaker, admittedly, it's a bit of a fine point in English.
> > That's definitely why it's written that way, though.
> 
> Arguably we should actually be covering both GA release. ( everyone I 
> know do it at EOL time )

You could certainly make the case, yeah. Given that our excuse for
people who say 'you have to upgrade every six months' is to say 'no you
don't, because we support releases for 12, you can just upgrade every
second release!', so we kinda do tell people that.

> Do you know if we keep log in our infrastructure that shows how many are 
> actually upgrading on which version they do it from?

I don't know that, no. I don't think we do. I suppose it might be
possible to infer such information from the yum records, with a careful
analysis, by looking at installations with reliable IP addresses and
seeing their upgrade patterns. That might actually be kinda interesting,
but I don't know if it's really possible. In general Fedora is pretty
conservative about logging user information. As a F/OSS project, you run
the risk of a bad case of Slashdotitis if you do anything else =)
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Jóhann B. Guðmundsson

On 09/24/2012 09:43 PM, Adam Williamson wrote:

Only 17-18: using the definite article 'the' rather than the indefinite
article 'a' implies this. It says 'the previous stable Fedora release' -
which strictly means "only the single preceding release" - not 'a
previous stable Fedora release' or 'any previous stable Fedora release'
or anything like that. I suppose it's a distinction which is clearer to
a native speaker, admittedly, it's a bit of a fine point in English.
That's definitely why it's written that way, though.


Arguably we should actually be covering both GA release. ( everyone I 
know do it at EOL time )


Do you know if we keep log in our infrastructure that shows how many are 
actually upgrading on which version they do it from?


JBG
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Adam Williamson
On Mon, 2012-09-24 at 20:02 +, "Jóhann B. Guðmundsson" wrote:
> On 09/24/2012 07:54 PM, Matthew Miller wrote:
> > On Mon, Sep 24, 2012 at 12:34:34PM -0700, Adam Williamson wrote:
> >> "It must be possible to successfully complete an upgrade from a fully
> >> updated installation of the previous stable Fedora release with the
> >> 'minimal' package set or the package set for a release-blocking desktop,
> >> using any officially recommended upgrade mechanism. The upgraded system
> >> must meet all release criteria."
> > +1
> >
> > Is it worth leaving an out for corner cases? What about situations like "oh,
> > we don't support a separate /usr anymore"? What level of (possibly-crazy)
> > customization is allowed between the initial installation + updates and
> > upgrading?
> >
> 
> None we only support package selection that we have *pre* selected for 
> the user to choose from in the software spoke ( which should be the same 
> as what we hand out in the form of live media at various events thus we 
> kill two birds with one criteria ;) ).
> 
> I'm not sure how far back that release wise that support is suppose to 
> go as in do we support ( or should support since package selection might 
> differ between release ) F15 --> F18 ( GA + 1 unsupported release ) or 
> F16 -->  F18 ( Between GA releases ) or only F17 --> F18 ( latest GA 
> release  )

Only 17-18: using the definite article 'the' rather than the indefinite
article 'a' implies this. It says 'the previous stable Fedora release' -
which strictly means "only the single preceding release" - not 'a
previous stable Fedora release' or 'any previous stable Fedora release'
or anything like that. I suppose it's a distinction which is clearer to
a native speaker, admittedly, it's a bit of a fine point in English.
That's definitely why it's written that way, though.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Adam Williamson
On Mon, 2012-09-24 at 15:54 -0400, Matthew Miller wrote:
> On Mon, Sep 24, 2012 at 12:34:34PM -0700, Adam Williamson wrote:
> > "It must be possible to successfully complete an upgrade from a fully
> > updated installation of the previous stable Fedora release with the
> > 'minimal' package set or the package set for a release-blocking desktop,
> > using any officially recommended upgrade mechanism. The upgraded system
> > must meet all release criteria."
> 
> +1
> 
> Is it worth leaving an out for corner cases? What about situations like "oh,
> we don't support a separate /usr anymore"? What level of (possibly-crazy)
> customization is allowed between the initial installation + updates and
> upgrading?

Oh, I think I dropped the word 'clean', which was code for 'we'll reject
all the stuff Matthew just wrote about'. We can add that back in. We've
always interpreted the criterion quite strictly, in practice.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Jóhann B. Guðmundsson

On 09/24/2012 07:54 PM, Matthew Miller wrote:

On Mon, Sep 24, 2012 at 12:34:34PM -0700, Adam Williamson wrote:

"It must be possible to successfully complete an upgrade from a fully
updated installation of the previous stable Fedora release with the
'minimal' package set or the package set for a release-blocking desktop,
using any officially recommended upgrade mechanism. The upgraded system
must meet all release criteria."

+1

Is it worth leaving an out for corner cases? What about situations like "oh,
we don't support a separate /usr anymore"? What level of (possibly-crazy)
customization is allowed between the initial installation + updates and
upgrading?



None we only support package selection that we have *pre* selected for 
the user to choose from in the software spoke ( which should be the same 
as what we hand out in the form of live media at various events thus we 
kill two birds with one criteria ;) ).


I'm not sure how far back that release wise that support is suppose to 
go as in do we support ( or should support since package selection might 
differ between release ) F15 --> F18 ( GA + 1 unsupported release ) or 
F16 -->  F18 ( Between GA releases ) or only F17 --> F18 ( latest GA 
release  )


JBG
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Matthew Miller
On Mon, Sep 24, 2012 at 12:34:34PM -0700, Adam Williamson wrote:
> "It must be possible to successfully complete an upgrade from a fully
> updated installation of the previous stable Fedora release with the
> 'minimal' package set or the package set for a release-blocking desktop,
> using any officially recommended upgrade mechanism. The upgraded system
> must meet all release criteria."

+1

Is it worth leaving an out for corner cases? What about situations like "oh,
we don't support a separate /usr anymore"? What level of (possibly-crazy)
customization is allowed between the initial installation + updates and
upgrading?

-- 
Matthew Miller  ☁☁☁  Fedora Cloud Architect  ☁☁☁  
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Jóhann B. Guðmundsson

On 09/24/2012 07:34 PM, Adam Williamson wrote:

"It must be possible to successfully complete an upgrade from a fully
updated installation of the previous stable Fedora release with the
'minimal' package set or the package set for a release-blocking desktop,
using any officially recommended upgrade mechanism. The upgraded system
must meet all release criteria."


I would like to replace/drop "release-blocking desktop" and or add 
something that covers all the official media that get handed out at 
various events.


I kinda feel that we are obligated to cover that since the projects 
representatives are putting that directly in the hands of end users


JBG
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Adam Williamson
On Mon, 2012-09-24 at 19:18 +, "Jóhann B. Guðmundsson" wrote:
> On 09/24/2012 05:35 PM, Adam Williamson wrote:
> > It must be possible to successfully complete an upgrade from a clean,
> > fully updated default installation of the previous stable Fedora
> > release, using any officially recommended upgrade mechanism. The
> > upgraded system must meet all release criteria.
> 
> The default should be removed and changed to "offered DE install ( 
> excluding any additional group individual packages user might select in 
> the new software spoke ) to reflect changes to the installer ( which 
> should also cover "live" ) + minimal installs

Right, for 18 the 'default installation' wording would work because
there was still a 'default installation' of Fedora 17, but it wouldn't
work after 18, so we should change it.

"It must be possible to successfully complete an upgrade from a fully
updated installation of the previous stable Fedora release with the
'minimal' package set or the package set for a release-blocking desktop,
using any officially recommended upgrade mechanism. The upgraded system
must meet all release criteria."

?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Jóhann B. Guðmundsson

On 09/24/2012 05:35 PM, Adam Williamson wrote:

It must be possible to successfully complete an upgrade from a clean,
fully updated default installation of the previous stable Fedora
release, using any officially recommended upgrade mechanism. The
upgraded system must meet all release criteria.


The default should be removed and changed to "offered DE install ( 
excluding any additional group individual packages user might select in 
the new software spoke ) to reflect changes to the installer ( which 
should also cover "live" ) + minimal installs


JBG
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Adam Williamson
On Mon, 2012-09-24 at 13:33 -0400, Richard Ryniker wrote:
> >The installer must be able to successfully complete an upgrade
> >installation from a clean, fully updated default installation (from any
> >official install medium) of the previous stable Fedora release via any
> >officially supported upgrade mechanism. The upgraded system must meet
> >all release criteria.
> 
> Sounds like a political statement - good words with almost no content.
> It would be nearly as useful, and easier to say "Everything that has to
> work will work."
> 
> If there is no reasonable way to explicitly say in this criterion what
> should work, at least add references to specific documents that define
> "official install media" and "officially supported upgrade mechanism."
> 
> I think it better to accept that criteria may have to be revised for a
> new release than to craft criteria so general they never need to change,
> or so empty of detail they have little direct value and require research
> to understand what they actually mean.

Is the concept of 'official install media' and a 'supported/recommended
upgrade mechanism' really so vague as to be meaningless? I don't think
I'd agree with that. It seems fairly clear to me. I do think in
principle we should have documents that define what currently fulfils
generic definitions like that, but the problem is that when you think
about it, those documents very rarely ought to be ones QA 'owns', so
it's not always straightforward to ensure it happens. For this specific
case though, I think we could link to
https://fedoraproject.org/wiki/Upgrading to define
'supported/recommendation upgrade mechanism' - that's clearly the
'official' page for such things and should be updated whenever it
changes. 'official install media' is not quite as obvious; maybe it
should be a link to the Deliverables SOP when I or someone else finally
gets that done (my last draft is still at
https://fedoraproject.org/wiki/User:Adamwill/Draft_releng_SOP_deliverables ).
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Jóhann B. Guðmundsson

On 09/24/2012 05:20 PM, Adam Williamson wrote:

is extremely vague and really gave no one in the project (this affects
all groups, really, not just QA) an understanding that things like 'no
more root password by default' and 'completely different upgrade system'
were coming. Some of that information might have been buried somewhere
inhttps://fedoraproject.org/wiki/Anaconda/UX_Redesign  , but I'd really
expect the feature page to be more detailed and organized, I don't think
regular Fedorans should be expected to dig into the background
documentation for any given feature to understand broadly what it
involves.

It's easy to point fingers, but I think FESCo might want to take a
lesson from this newUI process for future releases and that lesson
should be that major disruptive features should have_much_  better and
more definite feature pages.


I would be happy to have a single finger point me to the discussion that 
took place when that decision was made.


The main problem I have is that we weren't even included in the 
discussion so we could not even properly prepare for it to be officially 
supported.


Today it matters less since we are a bit better prepare I just hope that 
they have gather some input from the front line ( #fedora ) on how the 
upgrade process has been turning out for people, What have been the 
major issue people have had etc. to take into account when developing 
the new upgrade process that is as you have pointed extremely vague and 
to be honest I'm a bit vary of given Anconda's rough start this 
development cycle to me this news is coming as a bit of surprise.


JBG
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Adam Williamson
On Mon, 2012-09-24 at 11:03 -0600, Tim Flink wrote:
> On Mon, 24 Sep 2012 18:40:08 +0200
> drago01  wrote:
> 
> > > The installer must be able to successfully complete an upgrade
> > > installation from a clean, fully updated default installation (from
> > > any official install medium) of the previous stable Fedora release,
> > > either via preupgrade or by booting to the installer manually. The
> > > upgraded system must meet all release criteria.
> 
> 
> > Neither will the installer. For F18 a new tool will be written that
> > acts like preupgrade (downloads packages; reboots; upgrades), it might
> > use preupgrade but this isn't decided yet.
> > So I suggest to rewrite the text to say "The upgrade tool".
> 
> Point taken - there are few details available (that I'm aware of) which
> describe how any upgrades will work for F18. How about:
> 
> A clean, fully updated default installation (done with any official
> install method) of the previous stable Fedora release must be
> upgradable via any officially supported upgrade mechanism. The upgraded
> system must meet all release criteria.

I was thinking along the same lines, but I think I'd prefer:

It must be possible to successfully complete an upgrade from a clean,
fully updated default installation of the previous stable Fedora
release, using any officially recommended upgrade mechanism. The
upgraded system must meet all release criteria.

I just like it more that way around, still. I'm also favouring
'officially recommended' over 'officially supported' because, let's be
honest, our 'support' for upgrades is pretty nominal (the testing that
backs this criterion is pretty much the extent of our 'support').

It's a bit bikeshed-y I know, in the end I'd be okay with either.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Richard Ryniker
>The installer must be able to successfully complete an upgrade
>installation from a clean, fully updated default installation (from any
>official install medium) of the previous stable Fedora release via any
>officially supported upgrade mechanism. The upgraded system must meet
>all release criteria.

Sounds like a political statement - good words with almost no content.
It would be nearly as useful, and easier to say "Everything that has to
work will work."

If there is no reasonable way to explicitly say in this criterion what
should work, at least add references to specific documents that define
"official install media" and "officially supported upgrade mechanism."

I think it better to accept that criteria may have to be revised for a
new release than to craft criteria so general they never need to change,
or so empty of detail they have little direct value and require research
to understand what they actually mean.


-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Adam Williamson
On Mon, 2012-09-24 at 17:17 +, "Jóhann B. Guðmundsson" wrote:
> On 09/24/2012 05:05 PM, drago01 wrote:
> > s/any/all/ (in case we happen to support multiple methods).
> 
> If we start supporting more then one official way of upgrading the 
> distribution then we should be using either or in all release blocking 
> criteria.

I think Tim's 'any' was intentional. I think the idea would be that for
Beta we should require 'any' method to work, and for Final we should
require 'all' methods to work. Along the model we decided on for USB
boot methods at Alpha/Beta.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Adam Williamson
On Mon, 2012-09-24 at 16:56 +, "Jóhann B. Guðmundsson" wrote:
> On 09/24/2012 04:32 PM, Tim Flink wrote:
> > As we aren't sure if preupgrade will continue to be an officially
> > supported upgrade mechanism for F18, I propose to change that criterion
> > such that specific upgrade mechanisms are omitted.
> 
> Does not QA need sanction what ever upgrade mechanism they are coming up 
> with this time?
> ( When preupgrade was accepted by whomever that took decision they did 
> not even bother to a) ask us b) prepare for it )

I'm sure we've had this go-round before, but my opinion is no. QA does
not get to veto engineering decisions. However, there is a more general
requirement for transparency in new features that I'm really not
convinced has been properly followed for newUI. The feature page does
not at present contain any description or discussion of several fairly
significant features of the new design, including this new upgrade tool
and the change in how the root account is handled. I'd say if there's a
problem here it's less that QA should get specific sign-off on things
like this and more that:

https://fedoraproject.org/wiki/Features/NewInstallerUI

is extremely vague and really gave no one in the project (this affects
all groups, really, not just QA) an understanding that things like 'no
more root password by default' and 'completely different upgrade system'
were coming. Some of that information might have been buried somewhere
in https://fedoraproject.org/wiki/Anaconda/UX_Redesign , but I'd really
expect the feature page to be more detailed and organized, I don't think
regular Fedorans should be expected to dig into the background
documentation for any given feature to understand broadly what it
involves.

It's easy to point fingers, but I think FESCo might want to take a
lesson from this newUI process for future releases and that lesson
should be that major disruptive features should have _much_ better and
more definite feature pages.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Jóhann B. Guðmundsson

On 09/24/2012 05:05 PM, drago01 wrote:

s/any/all/ (in case we happen to support multiple methods).


If we start supporting more then one official way of upgrading the 
distribution then we should be using either or in all release blocking 
criteria.


JBG
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread drago01
On Mon, Sep 24, 2012 at 7:03 PM, Tim Flink  wrote:
> On Mon, 24 Sep 2012 18:40:08 +0200
> drago01  wrote:
>
>> > The installer must be able to successfully complete an upgrade
>> > installation from a clean, fully updated default installation (from
>> > any official install medium) of the previous stable Fedora release,
>> > either via preupgrade or by booting to the installer manually. The
>> > upgraded system must meet all release criteria.
>
>
>> Neither will the installer. For F18 a new tool will be written that
>> acts like preupgrade (downloads packages; reboots; upgrades), it might
>> use preupgrade but this isn't decided yet.
>> So I suggest to rewrite the text to say "The upgrade tool".
>
> Point taken - there are few details available (that I'm aware of) which
> describe how any upgrades will work for F18.

https://fedorahosted.org/fesco/ticket/946
See "2012-09-05 FESCo" meeting logs for details.

> How about:
>
> A clean, fully updated default installation (done with any official
> install method) of the previous stable Fedora release must be
> upgradable via any officially supported upgrade mechanism. The upgraded
> system must meet all release criteria.

s/any/all/ (in case we happen to support multiple methods).
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Tim Flink
On Mon, 24 Sep 2012 18:40:08 +0200
drago01  wrote:

> > The installer must be able to successfully complete an upgrade
> > installation from a clean, fully updated default installation (from
> > any official install medium) of the previous stable Fedora release,
> > either via preupgrade or by booting to the installer manually. The
> > upgraded system must meet all release criteria.


> Neither will the installer. For F18 a new tool will be written that
> acts like preupgrade (downloads packages; reboots; upgrades), it might
> use preupgrade but this isn't decided yet.
> So I suggest to rewrite the text to say "The upgrade tool".

Point taken - there are few details available (that I'm aware of) which
describe how any upgrades will work for F18. How about:

A clean, fully updated default installation (done with any official
install method) of the previous stable Fedora release must be
upgradable via any officially supported upgrade mechanism. The upgraded
system must meet all release criteria.

Tim


signature.asc
Description: PGP signature
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread Jóhann B. Guðmundsson

On 09/24/2012 04:32 PM, Tim Flink wrote:

As we aren't sure if preupgrade will continue to be an officially
supported upgrade mechanism for F18, I propose to change that criterion
such that specific upgrade mechanisms are omitted.


Does not QA need sanction what ever upgrade mechanism they are coming up 
with this time?
( When preupgrade was accepted by whomever that took decision they did 
not even bother to a) ask us b) prepare for it )


JBG
--
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Re: Release criterion proposal: upgrade methods

2012-09-24 Thread drago01
On Mon, Sep 24, 2012 at 6:32 PM, Tim Flink  wrote:
> As currently written, the upgrade criterion in the Fedora 18 beta
> release requirements [1] reads:
>
> The installer must be able to successfully complete an upgrade
> installation from a clean, fully updated default installation (from any
> official install medium) of the previous stable Fedora release, either
> via preupgrade or by booting to the installer manually. The upgraded
> system must meet all release criteria.
>
> As we aren't sure if preupgrade will continue to be an officially
> supported upgrade mechanism for F18, I propose to change that criterion
> such that specific upgrade mechanisms are omitted.

Neither will the installer. For F18 a new tool will be written that
acts like preupgrade (downloads packages; reboots; upgrades), it might
use preupgrade but this isn't decided yet.
So I suggest to rewrite the text to say "The upgrade tool".
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test

Release criterion proposal: upgrade methods

2012-09-24 Thread Tim Flink
As currently written, the upgrade criterion in the Fedora 18 beta
release requirements [1] reads:

The installer must be able to successfully complete an upgrade
installation from a clean, fully updated default installation (from any
official install medium) of the previous stable Fedora release, either
via preupgrade or by booting to the installer manually. The upgraded
system must meet all release criteria.

As we aren't sure if preupgrade will continue to be an officially
supported upgrade mechanism for F18, I propose to change that criterion
such that specific upgrade mechanisms are omitted.

The installer must be able to successfully complete an upgrade
installation from a clean, fully updated default installation (from any
official install medium) of the previous stable Fedora release via any
officially supported upgrade mechanism. The upgraded system must meet
all release criteria.


Since this is a pretty minor change, I'll wait one week before changing
the wiki page. If there are no objections by then, I'll go through and
change it.

Tim

[1] https://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria


signature.asc
Description: PGP signature
-- 
test mailing list
test@lists.fedoraproject.org
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test