Re: To RHEL or Not to RHEL?
On Wed, 13 May 2015 09:31:48 -0400 (EDT) Martin Krizek wrote: > - Original Message - > > From: "Tim Flink" > > To: qa-devel@lists.fedoraproject.org > > Sent: Saturday, May 9, 2015 8:05:04 PM > > Subject: To RHEL or Not to RHEL? > > ...snip... > > While virt-in-virt is possible, I'd prefer to avoid the extra > > complexity and performance penalty and figure that running on bare > > metal makes more sense. If we disable local task execution, there > > should be little risk of one task disrupting other stuff on that > > virthost that can't be easily reverted. > > Does it make sense not to disable local execution on one or more > buildslave? I wonder if some tasks could benefit from not running in > vm. Or it might be waste of resources to run tasks like rpmlint > on a disposable client? Yeah, that had occurred to me but hadn't gotten much farther with it than that. It's something that we should probably look into. I suspect that you're right that it'd be more efficient to run some if not all of our regular tasks on non-dispoable clients. It makes triggering a bit more complicated but I don't think it would be too terrible to have a new "non-disposable" builder and trigger certain tasks on that instead of the regular builder. Another option would be to maintain some of the vm buildslaves that we're currently using instead of running tasks on bare metal. I've filed a task for investigating this once we have a minimal system working: https://phab.qadevel.cloud.fedoraproject.org/T480 Thanks, Tim pgpCRteLneq3I.pgp Description: OpenPGP digital signature ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
Re: To RHEL or Not to RHEL?
On Wed, 13 May 2015 08:45:08 -0400 (EDT) Kamil Paral wrote: > > * Will need to be more diligent about keeping dev/stg on > > updates-testing so that we don't get any nasty surprises in > > production > > I don't have much advice about the other points, but this one caught > my attention. Do we really need to use updates-testing for dev/stg? > That might be quite problematic, because anyone can submit anything, > no matter how broken, into updates-testing. Wouldn't be a safer > approach to update dev daily (and stg e.g. every other day) from > stable updates? And production would be updated weekly or bi-weekly > (or however often we need it), with the exception of security > updates. Security updates would be applied to dev/stg immediately and > after a few jobs were successfully executed, it would be applied to > production. Would this approach work? Yeah, I'm not dead set on using updates-testing in that scenario - it was just the easiest way to express the "test updates on dev/stg before they make it to production. I probably could have been more specific Something like that could work as long as we were careful about only applying non-security updates on prod that had been sufficiently tested on dev/stg. At the moment, security updates are applied automatically on our fedora machines via cron job. The one thing I'd like to improve on if we continue to use fedora is regular updates. At the moment, I try to apply updates to everything every couple of weeks but it's not a set schedule and I'd like to improve that. I'm open to suggestions on what that schedule should be and how to implement it (reminders to folks with access, cron-ish, etc.) if we go that route. > I guess the approach with security updates would be the same, no > matter whether it's Fedora or RHEL. So the only difference in the > volume and speed of standard updates. Yeah, that leads into one of the disadvantages of running Fedora - more frequent updates and especially more frequent kernel updates that require a reboot. > This doesn't mean I'm in favor of running Fedora, I think you have > much more experienced view on this. I'm just thinking aloud about > some of the details. Yeah, it's a good point. Thanks for pointing this out. pgpfGyEudvGcr.pgp Description: OpenPGP digital signature ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
Re: To RHEL or Not to RHEL?
On Tue, 12 May 2015 17:34:24 -0600 Kevin Fenzi wrote: > On Mon, 11 May 2015 17:16:09 -0600 > Tim Flink wrote: > > > On Mon, 11 May 2015 13:09:33 -0600 > > Kevin Fenzi wrote: > > > > > On Sat, 9 May 2015 12:05:04 -0600 > > > Tim Flink wrote: > > > > > > > This was brought up a little while ago and we decided to put off > > > > the discussion a little bit but I'd like to re-start the > > > > conversation before we get too much farther with disposable > > > > clients. > > > > > > > > My plan for how our hosts would be set up once we deploy support > > > > for disposable clients is this: > > > > - virthosts would have N buildslave processes running on them > > > > - each buildslave would launch VMs for disposable clients as > > > > needed > > > > - each virthost would have access to a shared filesystem used > > > > to store at least VM images, maybe logs and other data > > > > > > Thats each virthost, not all virthosts having the same storage > > > right? > > > > All of the virthosts (at least in each group of dev/stg/prod) would > > have a chunk of shared storage used to store the canonical VM images > > that we use to boot the disposable clients (the disk changes would > > be done locally to the virthosts). This way we only have to build > > them once instead of once per virthost. > > Would it be feasable to just build them once and rsync them between > hosts? Or would you prefer shared storage? Honestly, that hadn't even occurred to me. I think it may end up depending on how we kick off the image builds but that sounds much easier and more reliable than a shared filesystem. > > In the back of my head, I'm thinking that it may make sense to store > > logs and artifacts on a chunk of shared storage instead of > > transferring everything to the taskotron master using buildbot. I > > figure that may make sense if the shared storage is already set up > > but this hasn't gotten past the "thinking about it" stage yet :). > > Yeah. We may be able to do a netapp nfs volume, we will have to see > what all we can do once we move to our new c-mode filer. I think it may be a little while before we're ready to look at doing something different for the logs and shared storage - still at step 1 "make it work" in this whole process but will keep it in mind. Thanks, Tim pgp11RcpQ0I5L.pgp Description: OpenPGP digital signature ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
Re: CI for Taskotron and Related Projects
On Wed, 13 May 2015 09:38:41 -0400 (EDT) Kamil Paral wrote: > > Note that there will be ssl cert warnings since it's self signed. > > Once the thing gets rebuilt for real, those ssl warnings will go > > away. > > Not sure if it is intentional, but plain http does not work, the > connection is refused. Not an issue, just saying. It's not quite intentional but only port 443 is open to that machine and thus, https only. I didn't think it was much of a problem, so I didn't ask for port 80 to be opened but it occurs to me that we can't have http to https redirection on that machine unless port 80 is open Tim pgpqkdVD0pCYo.pgp Description: OpenPGP digital signature ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
Re: CI for Taskotron and Related Projects
> IIRC it doesn't send notifications. We might enable sending e-mail to the > author of a commit that breaks tests (if possible). Thoughts? That would be a great feature! :-) I'd prefer to also CC qa-devel (might be obtrusive for subscribers) or sysadmin-qa-members, just to be sure we're informed when that commit author is not around for some time. ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
Re: CI for Taskotron and Related Projects
- Original Message - > From: "Kamil Paral" > To: "Fedora QA Development" > Sent: Wednesday, May 13, 2015 3:38:41 PM > Subject: Re: CI for Taskotron and Related Projects > > > I've just re-started the CI on qadevel (the future one, not cloud) that > > mkrizek set up a while back. It was destroyed when I started working on > > replacing qadevel.cloud but never finished it. > > > > The link is: > > > > https://qadevel.fedoraproject.org/buildmaster/waterfall > > Great. Just to refresh my memory, will we receive some kind of a notification > when some project fails to build? IIRC it doesn't send notifications. We might enable sending e-mail to the author of a commit that breaks tests (if possible). Thoughts? Martin ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
Re: CI for Taskotron and Related Projects
> I've just re-started the CI on qadevel (the future one, not cloud) that > mkrizek set up a while back. It was destroyed when I started working on > replacing qadevel.cloud but never finished it. > > The link is: > > https://qadevel.fedoraproject.org/buildmaster/waterfall Great. Just to refresh my memory, will we receive some kind of a notification when some project fails to build? > > Note that there will be ssl cert warnings since it's self signed. Once > the thing gets rebuilt for real, those ssl warnings will go away. Not sure if it is intentional, but plain http does not work, the connection is refused. Not an issue, just saying. ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
Re: To RHEL or Not to RHEL?
- Original Message - > From: "Tim Flink" > To: qa-devel@lists.fedoraproject.org > Sent: Saturday, May 9, 2015 8:05:04 PM > Subject: To RHEL or Not to RHEL? ...snip... > While virt-in-virt is possible, I'd prefer to avoid the extra > complexity and performance penalty and figure that running on bare > metal makes more sense. If we disable local task execution, there > should be little risk of one task disrupting other stuff on that > virthost that can't be easily reverted. > Does it make sense not to disable local execution on one or more buildslave? I wonder if some tasks could benefit from not running in vm. Or it might be waste of resources to run tasks like rpmlint on a disposable client? Martin ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
Re: To RHEL or Not to RHEL?
> * Will need to be more diligent about keeping dev/stg on > updates-testing so that we don't get any nasty surprises in production I don't have much advice about the other points, but this one caught my attention. Do we really need to use updates-testing for dev/stg? That might be quite problematic, because anyone can submit anything, no matter how broken, into updates-testing. Wouldn't be a safer approach to update dev daily (and stg e.g. every other day) from stable updates? And production would be updated weekly or bi-weekly (or however often we need it), with the exception of security updates. Security updates would be applied to dev/stg immediately and after a few jobs were successfully executed, it would be applied to production. Would this approach work? I guess the approach with security updates would be the same, no matter whether it's Fedora or RHEL. So the only difference in the volume and speed of standard updates. This doesn't mean I'm in favor of running Fedora, I think you have much more experienced view on this. I'm just thinking aloud about some of the details. ___ qa-devel mailing list qa-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/qa-devel
