Re: To RHEL or Not to RHEL?

2015-05-13 Thread Kamil Paral
 * Will need to be more diligent about keeping dev/stg on
   updates-testing so that we don't get any nasty surprises in production

I don't have much advice about the other points, but this one caught my 
attention. Do we really need to use updates-testing for dev/stg? That might be 
quite problematic, because anyone can submit anything, no matter how broken, 
into updates-testing. Wouldn't be a safer approach to update dev daily (and stg 
e.g. every other day) from stable updates? And production would be updated 
weekly or bi-weekly (or however often we need it), with the exception of 
security updates. Security updates would be applied to dev/stg immediately and 
after a few jobs were successfully executed, it would be applied to production. 
Would this approach work?

I guess the approach with security updates would be the same, no matter whether 
it's Fedora or RHEL. So the only difference in the volume and speed of standard 
updates.

This doesn't mean I'm in favor of running Fedora, I think you have much more 
experienced view on this. I'm just thinking aloud about some of the details.
___
qa-devel mailing list
qa-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/qa-devel


Re: To RHEL or Not to RHEL?

2015-05-13 Thread Martin Krizek
- Original Message -
 From: Tim Flink tfl...@redhat.com
 To: qa-devel@lists.fedoraproject.org
 Sent: Saturday, May 9, 2015 8:05:04 PM
 Subject: To RHEL or Not to RHEL?

...snip...
 While virt-in-virt is possible, I'd prefer to avoid the extra
 complexity and performance penalty and figure that running on bare
 metal makes more sense. If we disable local task execution, there
 should be little risk of one task disrupting other stuff on that
 virthost that can't be easily reverted.
 

Does it make sense not to disable local execution on one or more
buildslave? I wonder if some tasks could benefit from not running in
vm. Or it might be waste of resources to run tasks like rpmlint
on a disposable client?


Martin
___
qa-devel mailing list
qa-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/qa-devel


Re: CI for Taskotron and Related Projects

2015-05-13 Thread Kamil Paral
 I've just re-started the CI on qadevel (the future one, not cloud) that
 mkrizek set up a while back. It was destroyed when I started working on
 replacing qadevel.cloud but never finished it.
 
 The link is:
 
 https://qadevel.fedoraproject.org/buildmaster/waterfall

Great. Just to refresh my memory, will we receive some kind of a notification 
when some project fails to build?

 
 Note that there will be ssl cert warnings since it's self signed. Once
 the thing gets rebuilt for real, those ssl warnings will go away.

Not sure if it is intentional, but plain http does not work, the connection is 
refused. Not an issue, just saying.
___
qa-devel mailing list
qa-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/qa-devel


Re: CI for Taskotron and Related Projects

2015-05-13 Thread Martin Krizek
- Original Message -
 From: Kamil Paral kpa...@redhat.com
 To: Fedora QA Development qa-devel@lists.fedoraproject.org
 Sent: Wednesday, May 13, 2015 3:38:41 PM
 Subject: Re: CI for Taskotron and Related Projects
 
  I've just re-started the CI on qadevel (the future one, not cloud) that
  mkrizek set up a while back. It was destroyed when I started working on
  replacing qadevel.cloud but never finished it.
  
  The link is:
  
  https://qadevel.fedoraproject.org/buildmaster/waterfall
 
 Great. Just to refresh my memory, will we receive some kind of a notification
 when some project fails to build?

IIRC it doesn't send notifications. We might enable sending e-mail to the
author of a commit that breaks tests (if possible). Thoughts?

Martin
___
qa-devel mailing list
qa-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/qa-devel


Re: CI for Taskotron and Related Projects

2015-05-13 Thread Tim Flink
On Wed, 13 May 2015 09:38:41 -0400 (EDT)
Kamil Paral kpa...@redhat.com wrote:

snip

  Note that there will be ssl cert warnings since it's self signed.
  Once the thing gets rebuilt for real, those ssl warnings will go
  away.
 
 Not sure if it is intentional, but plain http does not work, the
 connection is refused. Not an issue, just saying.

It's not quite intentional but only port 443 is open to that machine
and thus, https only.

I didn't think it was much of a problem, so I didn't ask for port 80 to
be opened but it occurs to me that we can't have http to https
redirection on that machine unless port 80 is open

Tim


pgpqkdVD0pCYo.pgp
Description: OpenPGP digital signature
___
qa-devel mailing list
qa-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/qa-devel


Re: To RHEL or Not to RHEL?

2015-05-13 Thread Tim Flink
On Tue, 12 May 2015 17:34:24 -0600
Kevin Fenzi ke...@scrye.com wrote:

 On Mon, 11 May 2015 17:16:09 -0600
 Tim Flink tfl...@redhat.com wrote:
 
  On Mon, 11 May 2015 13:09:33 -0600
  Kevin Fenzi ke...@scrye.com wrote:
  
   On Sat, 9 May 2015 12:05:04 -0600
   Tim Flink tfl...@redhat.com wrote:
   
This was brought up a little while ago and we decided to put off
the discussion a little bit but I'd like to re-start the
conversation before we get too much farther with disposable
clients.

My plan for how our hosts would be set up once we deploy support
for disposable clients is this:
  - virthosts would have N buildslave processes running on them
  - each buildslave would launch VMs for disposable clients as
needed
  - each virthost would have access to a shared filesystem used
to store at least VM images, maybe logs and other data
   
   Thats each virthost, not all virthosts having the same storage
   right?
  
  All of the virthosts (at least in each group of dev/stg/prod) would
  have a chunk of shared storage used to store the canonical VM images
  that we use to boot the disposable clients (the disk changes would
  be done locally to the virthosts). This way we only have to build
  them once instead of once per virthost.
 
 Would it be feasable to just build them once and rsync them between
 hosts? Or would you prefer shared storage?

Honestly, that hadn't even occurred to me. I think it may end up
depending on how we kick off the image builds but that sounds much
easier and more reliable than a shared filesystem.

  In the back of my head, I'm thinking that it may make sense to store
  logs and artifacts on a chunk of shared storage instead of
  transferring everything to the taskotron master using buildbot. I
  figure that may make sense if the shared storage is already set up
  but this hasn't gotten past the thinking about it stage yet :).
 
 Yeah. We may be able to do a netapp nfs volume, we will have to see
 what all we can do once we move to our new c-mode filer. 

I think it may be a little while before we're ready to look at doing
something different for the logs and shared storage - still at step 1
make it work in this whole process but will keep it in mind.

Thanks,

Tim


pgp11RcpQ0I5L.pgp
Description: OpenPGP digital signature
___
qa-devel mailing list
qa-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/qa-devel


Re: To RHEL or Not to RHEL?

2015-05-13 Thread Tim Flink
On Wed, 13 May 2015 08:45:08 -0400 (EDT)
Kamil Paral kpa...@redhat.com wrote:

  * Will need to be more diligent about keeping dev/stg on
updates-testing so that we don't get any nasty surprises in
  production
 
 I don't have much advice about the other points, but this one caught
 my attention. Do we really need to use updates-testing for dev/stg?
 That might be quite problematic, because anyone can submit anything,
 no matter how broken, into updates-testing. Wouldn't be a safer
 approach to update dev daily (and stg e.g. every other day) from
 stable updates? And production would be updated weekly or bi-weekly
 (or however often we need it), with the exception of security
 updates. Security updates would be applied to dev/stg immediately and
 after a few jobs were successfully executed, it would be applied to
 production. Would this approach work?

Yeah, I'm not dead set on using updates-testing in that scenario - it
was just the easiest way to express the test updates on dev/stg before
they make it to production. I probably could have been more specific

Something like that could work as long as we were careful about only
applying non-security updates on prod that had been sufficiently tested
on dev/stg. At the moment, security updates are applied automatically
on our fedora machines via cron job.

The one thing I'd like to improve on if we continue to use fedora is
regular updates. At the moment, I try to apply updates to everything
every couple of weeks but it's not a set schedule and I'd like to
improve that. I'm open to suggestions on what that schedule should be
and how to implement it (reminders to folks with access, cron-ish,
etc.) if we go that route.

 I guess the approach with security updates would be the same, no
 matter whether it's Fedora or RHEL. So the only difference in the
 volume and speed of standard updates.

Yeah, that leads into one of the disadvantages of running Fedora - more
frequent updates and especially more frequent kernel updates that
require a reboot.

 This doesn't mean I'm in favor of running Fedora, I think you have
 much more experienced view on this. I'm just thinking aloud about
 some of the details.

Yeah, it's a good point. Thanks for pointing this out.


pgpfGyEudvGcr.pgp
Description: OpenPGP digital signature
___
qa-devel mailing list
qa-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/qa-devel


Re: To RHEL or Not to RHEL?

2015-05-13 Thread Tim Flink
On Wed, 13 May 2015 09:31:48 -0400 (EDT)
Martin Krizek mkri...@redhat.com wrote:

 - Original Message -
  From: Tim Flink tfl...@redhat.com
  To: qa-devel@lists.fedoraproject.org
  Sent: Saturday, May 9, 2015 8:05:04 PM
  Subject: To RHEL or Not to RHEL?
 
 ...snip...
  While virt-in-virt is possible, I'd prefer to avoid the extra
  complexity and performance penalty and figure that running on bare
  metal makes more sense. If we disable local task execution, there
  should be little risk of one task disrupting other stuff on that
  virthost that can't be easily reverted.
 
 Does it make sense not to disable local execution on one or more
 buildslave? I wonder if some tasks could benefit from not running in
 vm. Or it might be waste of resources to run tasks like rpmlint
 on a disposable client?

Yeah, that had occurred to me but hadn't gotten much farther with it
than that.

It's something that we should probably look into. I suspect that
you're right that it'd be more efficient to run some if not all
of our regular tasks on non-dispoable clients. It makes
triggering a bit more complicated but I don't think it would be
too terrible to have a new non-disposable builder and trigger
certain tasks on that instead of the regular builder.

Another option would be to maintain some of the vm buildslaves that
we're currently using instead of running tasks on bare metal. I've
filed a task for investigating this once we have a minimal system
working:

https://phab.qadevel.cloud.fedoraproject.org/T480

Thanks,

Tim


pgpCRteLneq3I.pgp
Description: OpenPGP digital signature
___
qa-devel mailing list
qa-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/qa-devel