Re: Freeze break request: push mirrormanager2-0.1.0-3
+1 kevin pgpoARNX2pg0T.pgp Description: OpenPGP digital signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request: Use varnish for mirrormanager2 publiclist
+1 kevin pgprC8oA1Be69.pgp Description: OpenPGP digital signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Fed-clou02 migration
Hi, as you know we have new Fedora Cloud instance. And we still have the *old* Fedora Cloud instance. I hereby declare fed-cloud02 a.k.a old Fedora Cloud as deprecated. There is currently 67 machines in running state. And bunch of VM in shutdown state. I would kindly ask all owners to: * not create new VM on fed-cloud02, but rather use fed-cloud09 * migrate your machines from fed-cloud02 to fed-cloud09 * terminate your machines on fed-cloud02, which you do not use (especially those under transient tenant). There is no hurry, we are under no press. However I would like to set up some dead line. Let say during June and July. During July I would like to gather list of remaining VMs and write personal email to its owners. In August - if there would be no reaction - I would suggest to power off (not terminate!) those remaining VMs and keep them for brief period. Sometime during fall terminate all machines and wipe old Fedora Cloud instance. Once again - this time-frame is just proposal as I would like to avoid having old Cloud instance running infinitely. If you have reason to have running it there and not migrating it, please raise your voice and we can alter the schedule. -- Miroslav Suchy, RHCA Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Moving the codecs sticky post link on Ask Fedora to a sidebar to make it easier to find
On Tue, 2015-05-12 at 10:08 -0600, Kevin Fenzi wrote: Fine with me. Done :D signature.asc Description: This is a digitally signed message part ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Freeze break request: Use varnish for mirrormanager2 publiclist
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
Any +1s for the below patch?
As noted in the commit message, this is mostly preferred
after a new release is made (or the change in PR#80 is
hotfixed), though it will work perfectly without, just
confuse the user slightly, because whenever they're in
/mirrors, they will always see like being logged out.
This is currently already live in stg.
commit e1a85426b83739e46c1e4f59e3b962738d0cbc24
Author: Patrick Uiterwijk puiterw...@redhat.com
Date: Tue May 19 16:32:38 2015 +
Use varnish for mirrormanager2 publiclist
This change makes us use varnish for the mm2 publiclist
and configures varnish to ignore cookies on the /mirrors
subpath.
The cookie ignore is only valid after the master-noauth
Pull Request #80 is merged, as that hides all authed
information from the publiclist pages.
Signed-off-by: Patrick Uiterwijk puiterw...@redhat.com
diff --git a/playbooks/include/proxies-reverseproxy.yml
b/playbooks/include/proxies-reverseproxy.yml
index a55472f..904428a 100644
- --- a/playbooks/include/proxies-reverseproxy.yml
+++ b/playbooks/include/proxies-reverseproxy.yml
@@ -216,15 +216,6 @@
proxyurl: http://localhost:10009
- role: httpd/reverseproxy
- -when: env != staging
- -website: admin.fedoraproject.org
- -destname: mirrormanager
- -remotepath: /mirrormanager
- -localpath: /mirrormanager
- -proxyurl: http://localhost:10008
- -
- - - role: httpd/reverseproxy
- -when: env == staging
website: admin.fedoraproject.org
destname: mirrormanager
remotepath: /mirrormanager
diff --git a/roles/varnish/files/proxy.vcl b/roles/varnish/files/proxy.vcl
index 37ca3da..ed8333b 100644
- --- a/roles/varnish/files/proxy.vcl
+++ b/roles/varnish/files/proxy.vcl
@@ -187,6 +187,10 @@ sub vcl_recv {
unset req.http.cookie;
set req.url = regsub(req.url, \?.*, );
}
+if (req.url ~ ^/mirrormanager/mirrors) {
+unset req.http.cookie;
+set req.url = regsub(req.url, \?.*, );
+}
}
if (req.url ~ ^/mirrormanager2/) {
set req.backend_hint = mirrormanager2;
@@ -299,3 +303,13 @@ sub vcl_recv {
#unset beresp.http.set-cookie;
#}
#}
+
+
+# Make sure mirrormanager/mirrors doesn't set any cookies
+# (Setting cookies would make varnish store a HIT-FOR-PASS
+# making it always fetch from backend)
+sub vcl_backend_response {
+if (bereq.url ~ ^/mirrormanager/mirrors) {
+unset beresp.http.set-cookie;
+}
+}
-BEGIN PGP SIGNATURE-
Version: GnuPG v1
iQIcBAEBCgAGBQJVW2deAAoJEIZXmA2atR5QX58QAIyOKoRCasd7lWxkpHg8vlvr
emasDLQ6+bG8UwERMU0Xbk85njzzAWBsUG6wbQKPkJRE+fHtoA2ZFOxrXD4o/ioF
Uv+w6EwG8peltm3l/s9NOLxgzKOZoS/lULhfGshjxQVYXFcyexkNM5W82TyE7D9i
51wQ6pqcLiXeiWyzJGAA/MIA6zAm32bt+TAu7rq0uQSFcbKShhx9A6IHPlKRuELX
U9ORY6Nh5HYqwE+FEkq5kQKkpiFMrohpffLBea4Q5urAxozb1uE1Nj4W5dP8b41H
ZCPr3pbyFZEENJCC4MJ3wfhY7t4wMKUOEjyoEGte2l+rNT8mwchx/TMajOms803v
/wTwG6Q4gbkaPbbxxEgjX9cNfzrvsahZE3n3oJm5ul55pVpMxBDqO6nBPfK1qqoi
cVAurKTta6R6l7CIQC+t4BKFx3O6sfld/8eD4wiigHG0q7e61e5iZudGJkVaAy7m
bz38ZgCHMwTzhjLAu4va0gNZZqhtJqpMnypR1ymbIrmsMi9/kMFh5QhnyI6CbR7r
OY95yNxbAA2SyM7V+Ee9+L5FNd+6aX0jFG8bBZU06o4rZrZEg3BzdnLbNPApZoJs
Q15RWRh9HWyZ8wMAg6u9PthPG4r3GrryksYSQ3s5SuArSvGWejZg9MX1r8wjbryP
YetR6PlqZsGMs1/j55el
=L8bm
-END PGP SIGNATURE-
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
[release] pagure: 0.1.4, 0.1.5 and 0.1.6
Good morning everyone, Over the last two days I have been working on getting the milter up and running for pagure. This took a little more time than expected and a little more fixes as well. I had to change or add ome items in the configuration of pagure. While replying to new emails will work fine and add them to the ticket/PR as expected, replying to older emails won't work (as it did before). So from today on, you should be able to reply to a comment made on a ticket or a pull-request by directly replying to the email. If this does not work, a) ensure you have added the email your are using in pagure (cf your user settings page) and b) let me know! :) Here are the changelog for the three releases I made over the last two days to fix these issues and some other: * Wed May 20 2015 Pierre-Yves Chibon pin...@pingoured.fr - 0.1.6-1 - Update to 0.1.6 - Fix sending notification emails to multiple users, avoid sending private into to all of them * Tue May 19 2015 Pierre-Yves Chibon pin...@pingoured.fr - 0.1.5-1 - Update to 0.1.5 - Bug fix on the milter and the internal API endpoint * Tue May 19 2015 Pierre-Yves Chibon pin...@pingoured.fr - 0.1.4-1 - Update to 0.1.4 - Fix loading requests and tickets from git (allows syncing projects between pagure instances) - Add to the template .wsgi file a way to re-locate the tmp folder to work around a bug in libgit2 - Fix unit-tests suite - Adjust the spec file to install all the files required for the milters - Fix the `View` button on the pull-request pages Thanks, Pierre pgpYNFfvTDsqZ.pgp Description: PGP signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request: push mirrormanager2-0.1.0-3
+1 With kind regards, Patrick Uiterwijk Fedora Infra - Original Message - Hi all, Since MirrorManager2, the flask application incorporates the mirrorlist. We have seen this causing problem once in a while as the query that generates the list of mirror is pretty heavy. Patrick has been working on changing a little bit the layout of the mirrorlist so that we could cache it with varnish (the change is basically to remove the `login`/`logged in as XX | logout` from the top right corner of the templates). This way, people can login from the front page and will see all the pages as being logged in, or they can just see the cached pages or the mirrorlist. We were leaning to wait for after the freeze to push this, but Adrian Reber reported that the issue of mirrorlist being sometime un-available is causing problem with the report-mirror script. So I would like to ask for a freeze-break to push to MirrorManager2 the changes made by Patrick: https://github.com/fedora-infra/mirrormanager2/pull/80 While at it, I would like to push another fix, by Adrian, allowing to always mark as up to date, mirrors that are always up to date: https://github.com/fedora-infra/mirrormanager2/pull/67 These changes have been prepared via a 0.1.0-3 RPM release: http://koji.fedoraproject.org/koji/taskinfo?taskID=9804162 built from: https://github.com/fedora-infra/mirrormanager2/commit/b368e3aa8988367fcf10148d1a06fb3e4c224357 Thanks, Pierre ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request: Use varnish for mirrormanager2 publiclist
On Tue, May 19, 2015 at 06:39:58PM +0200, Patrick Uiterwijk wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512
Any +1s for the below patch?
As noted in the commit message, this is mostly preferred
after a new release is made (or the change in PR#80 is
hotfixed), though it will work perfectly without, just
confuse the user slightly, because whenever they're in
/mirrors, they will always see like being logged out.
This is currently already live in stg.
+1 for me for the changes below.
This does not include the fedmsg-based clearing of the cache though, does it?
So how long would the cache be stored now?
Pierre
commit e1a85426b83739e46c1e4f59e3b962738d0cbc24
Author: Patrick Uiterwijk puiterw...@redhat.com
Date: Tue May 19 16:32:38 2015 +
Use varnish for mirrormanager2 publiclist
This change makes us use varnish for the mm2 publiclist
and configures varnish to ignore cookies on the /mirrors
subpath.
The cookie ignore is only valid after the master-noauth
Pull Request #80 is merged, as that hides all authed
information from the publiclist pages.
Signed-off-by: Patrick Uiterwijk puiterw...@redhat.com
diff --git a/playbooks/include/proxies-reverseproxy.yml
b/playbooks/include/proxies-reverseproxy.yml
index a55472f..904428a 100644
- --- a/playbooks/include/proxies-reverseproxy.yml
+++ b/playbooks/include/proxies-reverseproxy.yml
@@ -216,15 +216,6 @@
proxyurl: http://localhost:10009
- role: httpd/reverseproxy
- -when: env != staging
- -website: admin.fedoraproject.org
- -destname: mirrormanager
- -remotepath: /mirrormanager
- -localpath: /mirrormanager
- -proxyurl: http://localhost:10008
- -
- - - role: httpd/reverseproxy
- -when: env == staging
website: admin.fedoraproject.org
destname: mirrormanager
remotepath: /mirrormanager
diff --git a/roles/varnish/files/proxy.vcl b/roles/varnish/files/proxy.vcl
index 37ca3da..ed8333b 100644
- --- a/roles/varnish/files/proxy.vcl
+++ b/roles/varnish/files/proxy.vcl
@@ -187,6 +187,10 @@ sub vcl_recv {
unset req.http.cookie;
set req.url = regsub(req.url, \?.*, );
}
+if (req.url ~ ^/mirrormanager/mirrors) {
+unset req.http.cookie;
+set req.url = regsub(req.url, \?.*, );
+}
}
if (req.url ~ ^/mirrormanager2/) {
set req.backend_hint = mirrormanager2;
@@ -299,3 +303,13 @@ sub vcl_recv {
#unset beresp.http.set-cookie;
#}
#}
+
+
+# Make sure mirrormanager/mirrors doesn't set any cookies
+# (Setting cookies would make varnish store a HIT-FOR-PASS
+# making it always fetch from backend)
+sub vcl_backend_response {
+if (bereq.url ~ ^/mirrormanager/mirrors) {
+unset beresp.http.set-cookie;
+}
+}
-BEGIN PGP SIGNATURE-
Version: GnuPG v1
iQIcBAEBCgAGBQJVW2deAAoJEIZXmA2atR5QX58QAIyOKoRCasd7lWxkpHg8vlvr
emasDLQ6+bG8UwERMU0Xbk85njzzAWBsUG6wbQKPkJRE+fHtoA2ZFOxrXD4o/ioF
Uv+w6EwG8peltm3l/s9NOLxgzKOZoS/lULhfGshjxQVYXFcyexkNM5W82TyE7D9i
51wQ6pqcLiXeiWyzJGAA/MIA6zAm32bt+TAu7rq0uQSFcbKShhx9A6IHPlKRuELX
U9ORY6Nh5HYqwE+FEkq5kQKkpiFMrohpffLBea4Q5urAxozb1uE1Nj4W5dP8b41H
ZCPr3pbyFZEENJCC4MJ3wfhY7t4wMKUOEjyoEGte2l+rNT8mwchx/TMajOms803v
/wTwG6Q4gbkaPbbxxEgjX9cNfzrvsahZE3n3oJm5ul55pVpMxBDqO6nBPfK1qqoi
cVAurKTta6R6l7CIQC+t4BKFx3O6sfld/8eD4wiigHG0q7e61e5iZudGJkVaAy7m
bz38ZgCHMwTzhjLAu4va0gNZZqhtJqpMnypR1ymbIrmsMi9/kMFh5QhnyI6CbR7r
OY95yNxbAA2SyM7V+Ee9+L5FNd+6aX0jFG8bBZU06o4rZrZEg3BzdnLbNPApZoJs
Q15RWRh9HWyZ8wMAg6u9PthPG4r3GrryksYSQ3s5SuArSvGWejZg9MX1r8wjbryP
YetR6PlqZsGMs1/j55el
=L8bm
-END PGP SIGNATURE-
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
pgpd_qAre0m6L.pgp
Description: PGP signature
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Freeze break request: push mirrormanager2-0.1.0-3
Hi all, Since MirrorManager2, the flask application incorporates the mirrorlist. We have seen this causing problem once in a while as the query that generates the list of mirror is pretty heavy. Patrick has been working on changing a little bit the layout of the mirrorlist so that we could cache it with varnish (the change is basically to remove the `login`/`logged in as XX | logout` from the top right corner of the templates). This way, people can login from the front page and will see all the pages as being logged in, or they can just see the cached pages or the mirrorlist. We were leaning to wait for after the freeze to push this, but Adrian Reber reported that the issue of mirrorlist being sometime un-available is causing problem with the report-mirror script. So I would like to ask for a freeze-break to push to MirrorManager2 the changes made by Patrick: https://github.com/fedora-infra/mirrormanager2/pull/80 While at it, I would like to push another fix, by Adrian, allowing to always mark as up to date, mirrors that are always up to date: https://github.com/fedora-infra/mirrormanager2/pull/67 These changes have been prepared via a 0.1.0-3 RPM release: http://koji.fedoraproject.org/koji/taskinfo?taskID=9804162 built from: https://github.com/fedora-infra/mirrormanager2/commit/b368e3aa8988367fcf10148d1a06fb3e4c224357 Thanks, Pierre pgpga4mZP5xxk.pgp Description: PGP signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request: Use varnish for mirrormanager2 publiclist
Currently, the cache is stored for up to 120 seconds. I will submit a patch to add the regen script and bump the cache TTL when I ported the script. With kind regards, Patrick Uiterwijk Fedora Infra - Original Message - +1 for me for the changes below. This does not include the fedmsg-based clearing of the cache though, does it? So how long would the cache be stored now? Pierre ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request: push mirrormanager2-0.1.0-3
On Wed, May 20, 2015 at 10:11:15AM +0200, Pierre-Yves Chibon wrote: Since MirrorManager2, the flask application incorporates the mirrorlist. We have seen this causing problem once in a while as the query that generates the list of mirror is pretty heavy. Patrick has been working on changing a little bit the layout of the mirrorlist so that we could cache it with varnish (the change is basically to remove the `login`/`logged in as XX | logout` from the top right corner of the templates). This way, people can login from the front page and will see all the pages as being logged in, or they can just see the cached pages or the mirrorlist. We were leaning to wait for after the freeze to push this, but Adrian Reber reported that the issue of mirrorlist being sometime un-available is causing problem with the report-mirror script. So I would like to ask for a freeze-break to push to MirrorManager2 the changes made by Patrick: https://github.com/fedora-infra/mirrormanager2/pull/80 While at it, I would like to push another fix, by Adrian, allowing to always mark as up to date, mirrors that are always up to date: https://github.com/fedora-infra/mirrormanager2/pull/67 This is already active as a hotfix. So this does not introduce actual changes. It just moves the changes from the hotfix to the RPM. These changes have been prepared via a 0.1.0-3 RPM release: http://koji.fedoraproject.org/koji/taskinfo?taskID=9804162 built from: https://github.com/fedora-infra/mirrormanager2/commit/b368e3aa8988367fcf10148d1a06fb3e4c224357 +1 from me as almost every second run of report_mirror is now failing and the mirrorlist (publiclist) viewable in the browser used to be generated once every 12 hours with MM1. It is not as important being up to date as the mirrorlist/metalink handed out to yum/dnf. Adrian pgp3sZPts0CpL.pgp Description: PGP signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Fed-clou02 migration
On Wed, May 20, 2015 at 02:03:44PM +0200, Miroslav Suchý wrote: as you know we have new Fedora Cloud instance. Fedora Infrastructure Cloud? :) :) :) -- Matthew Miller mat...@fedoraproject.org Fedora Project Leader ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request - change haproxy check URL for MirrorManager
On Wed, May 20, 2015 at 06:25:54AM -0600, Kevin Fenzi wrote: On Wed, 20 May 2015 14:20:47 +0200 Adrian Reber adr...@lisas.de wrote: Can I get two +1 for this change? Switch to another URL for mirrormanager haproxy check The haproxy check URL for the MirrorManager web frontend was a URL which resulted in a large DB query. Every proxy, every minute. This resulted in two much memory and CPU consumption. This switches the check to a small static file to reduce the load on mm-frontend01. Well, the problem then is that we don't test that the db is working at all and just that loading a static file works. ;( So, haproxy could mark the service up when the db is down and the page doesn't work. Yes, but that was how it was before (I know that is bad argument ;-). The old MM landing page was a simple login page without any DB access. The current constellation is good that it tests that the whole thing is actually working, but it does it from all proxies. One check if the DB is working would be enough. The new nagios check for the publiclist which is redirected to /mirrormanager/ is also testing if it correctly works. Most accesses to /mirrormanager are from the haproxies and they are basically taking down MM every few hours. Adrian pgphZzgRgjWO6.pgp Description: PGP signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request - change haproxy check URL for MirrorManager
On Wed, 20 May 2015 14:31:28 +0200 Adrian Reber adr...@lisas.de wrote: Yes, but that was how it was before (I know that is bad argument ;-). The old MM landing page was a simple login page without any DB access. The current constellation is good that it tests that the whole thing is actually working, but it does it from all proxies. One check if the DB is working would be enough. The new nagios check for the publiclist which is redirected to /mirrormanager/ is also testing if it correctly works. Most accesses to /mirrormanager are from the haproxies and they are basically taking down MM every few hours. ok. So, perhaps we do this change for now and down the road we add a url that does a db check and just returns 'ok' or something ? So, +1 to this for now, but I'd like a better solution longer term if we can come up with one. kevin pgp3iSqY0Ziv7.pgp Description: OpenPGP digital signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Freeze break request - change haproxy check URL for MirrorManager
Can I get two +1 for this change? Switch to another URL for mirrormanager haproxy check The haproxy check URL for the MirrorManager web frontend was a URL which resulted in a large DB query. Every proxy, every minute. This resulted in two much memory and CPU consumption. This switches the check to a small static file to reduce the load on mm-frontend01. diff --git a/roles/haproxy/files/haproxy.cfg b/roles/haproxy/files/haproxy.cfg index 75bcf17..7a4b6cd 100644 --- a/roles/haproxy/files/haproxy.cfg +++ b/roles/haproxy/files/haproxy.cfg @@ -75,7 +75,7 @@ listen voting 0.0.0.0:10007 listen mirrormanager 0.0.0.0:10008 balance hdr(appserver) server mm-frontend01 mm-frontend01:80 check inter 60s rise 2 fall 3 -option httpchk GET /mirrormanager +option httpchk GET /mirrormanager/static/mirrormanager2.css listen bodhi 0.0.0.0:10009 balance hdr(appserver) diff --git a/roles/haproxy/files/haproxy.cfg.stg b/roles/haproxy/files/haproxy.cfg.stg index ef77c0d..6acc0fd 100644 --- a/roles/haproxy/files/haproxy.cfg.stg +++ b/roles/haproxy/files/haproxy.cfg.stg @@ -65,7 +65,7 @@ listen voting 0.0.0.0:10007 listen mirrormanager 0.0.0.0:10008 balance hdr(appserver) server mm-frontend01 mm-frontend01:80 check inter 60s rise 2 fall 3 -option httpchk GET /mirrormanager/ +option httpchk GET /mirrormanager/static/mirrormanager2.css listen bodhi 0.0.0.0:10009 balance hdr(appserver) Adrian pgpp0iVufnOZI.pgp Description: PGP signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request - change haproxy check URL for MirrorManager
On Wed, 20 May 2015 14:20:47 +0200 Adrian Reber adr...@lisas.de wrote: Can I get two +1 for this change? Switch to another URL for mirrormanager haproxy check The haproxy check URL for the MirrorManager web frontend was a URL which resulted in a large DB query. Every proxy, every minute. This resulted in two much memory and CPU consumption. This switches the check to a small static file to reduce the load on mm-frontend01. Well, the problem then is that we don't test that the db is working at all and just that loading a static file works. ;( So, haproxy could mark the service up when the db is down and the page doesn't work. kevin pgp9wKLCyABtL.pgp Description: OpenPGP digital signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Fed-clou02 migration
On Wed, 20 May 2015 14:03:44 +0200 Miroslav Suchý msu...@redhat.com wrote: Hi, as you know we have new Fedora Cloud instance. And we still have the *old* Fedora Cloud instance. I hereby declare fed-cloud02 a.k.a old Fedora Cloud as deprecated. There is currently 67 machines in running state. And bunch of VM in shutdown state. I would kindly ask all owners to: * not create new VM on fed-cloud02, but rather use fed-cloud09 * migrate your machines from fed-cloud02 to fed-cloud09 * terminate your machines on fed-cloud02, which you do not use (especially those under transient tenant). There is no hurry, we are under no press. However I would like to set up some dead line. Let say during June and July. During July I would like to gather list of remaining VMs and write personal email to its owners. In August - if there would be no reaction - I would suggest to power off (not terminate!) those remaining VMs and keep them for brief period. Sometime during fall terminate all machines and wipe old Fedora Cloud instance. Once again - this time-frame is just proposal as I would like to avoid having old Cloud instance running infinitely. If you have reason to have running it there and not migrating it, please raise your voice and we can alter the schedule. Yes, Patrick and I have been talking about and doing some planning on this. ;) https://fedoraproject.org/wiki/Infrastructure_private_cloud_icehouse_migration 0. We were going to look at replacing the existing ssl cert to avoid that anoying urllib3 warning later this week. 1. Patrick is going to mail users we setup in our new cloud playbook gpg encrypted versions of their initial password. This email will also include how to add the ca so the cert is trusted and ask people to reset their password to something only they know. 2. We control a number of the instances in the persistent tenant, we just need to start moving them over. Things like jenkins and such. When we do this we should check and make sure whatever we move is still being used, and also make sure to call it in ansible by name (we have several in there with ip, and it makes it really hard to see what they are off hand, so I want to move everything to name based). 3. We also host buildbot instances for twisted folks. We need to reach out to them and schedule migrating their buildbots. 4. After all those are moved, we can see whats left. Hopefully not much at all. We always told folks that instances can be terminiated at any point, so I don't think we need a really long ramp here. 5. After things are gone on the old cloud, we should reinstall and add fed-cloud03 - fed-cloud08 to the new cloud as compute nodes, and reinstall fed-cloud01/02 as a new cloud and a compute node. kevin pgpilmjS1pXIe.pgp Description: OpenPGP digital signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze Break request: add 2 cpus to proxy11
+1 With kind regards, Patrick Uiterwijk Fedora Infra - Original Message - This morning we had a few reports of mirrorlists timing out, and noticed that load on proxy11 was very high. It was using all 4 of it's cpus 100% and may have been dropping or timing out requests. I'd like to add 2 to it: +1s? kevin -- diff --git a/inventory/host_vars/proxy11.fedoraproject.org b/inventory/host_vars/proxy11.fedoraproject.org index 4c412de..d07e698 100644 --- a/inventory/host_vars/proxy11.fedoraproject.org +++ b/inventory/host_vars/proxy11.fedoraproject.org @@ -2,7 +2,7 @@ nm: 255.255.255.0 gw: 67.219.144.1 dns: 8.8.8.8 -num_cpus: 4 +num_cpus: 6 ks_url: http://209.132.181.6/repo/rhel/ks/kvm-rhel-7-ext ks_repo: http://209.132.181.6/repo/rhel/RHEL7-x86_64/ ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Freeze Break request: add 2 cpus to proxy11
This morning we had a few reports of mirrorlists timing out, and noticed that load on proxy11 was very high. It was using all 4 of it's cpus 100% and may have been dropping or timing out requests. I'd like to add 2 to it: +1s? kevin -- diff --git a/inventory/host_vars/proxy11.fedoraproject.org b/inventory/host_vars/proxy11.fedoraproject.org index 4c412de..d07e698 100644 --- a/inventory/host_vars/proxy11.fedoraproject.org +++ b/inventory/host_vars/proxy11.fedoraproject.org @@ -2,7 +2,7 @@ nm: 255.255.255.0 gw: 67.219.144.1 dns: 8.8.8.8 -num_cpus: 4 +num_cpus: 6 ks_url: http://209.132.181.6/repo/rhel/ks/kvm-rhel-7-ext ks_repo: http://209.132.181.6/repo/rhel/RHEL7-x86_64/ pgpLc3TvxOvNc.pgp Description: OpenPGP digital signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request - change haproxy check URL for MirrorManager
Until we get /ping, I'm +1 on this. With kind regards, Patrick Uiterwijk Fedora Infra - Original Message - On Wed, May 20, 2015 at 06:57:57AM -0600, Kevin Fenzi wrote: On Wed, 20 May 2015 14:31:28 +0200 Adrian Reber adr...@lisas.de wrote: Yes, but that was how it was before (I know that is bad argument ;-). The old MM landing page was a simple login page without any DB access. The current constellation is good that it tests that the whole thing is actually working, but it does it from all proxies. One check if the DB is working would be enough. The new nagios check for the publiclist which is redirected to /mirrormanager/ is also testing if it correctly works. Most accesses to /mirrormanager are from the haproxies and they are basically taking down MM every few hours. ok. So, perhaps we do this change for now and down the road we add a url that does a db check and just returns 'ok' or something ? So, +1 to this for now, but I'd like a better solution longer term if we can come up with one. Yes, puiterwijk was suggesting a /ping page especially for the proxy check. I opened a ticket for this: https://github.com/fedora-infra/mirrormanager2/issues/83 Adrian ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request: push mirrormanager2-0.1.0-3
On Wed, May 20, 2015 at 10:11:15AM +0200, Pierre-Yves Chibon wrote: Hi all, Since MirrorManager2, the flask application incorporates the mirrorlist. We have seen this causing problem once in a while as the query that generates the list of mirror is pretty heavy. Patrick has been working on changing a little bit the layout of the mirrorlist so that we could cache it with varnish (the change is basically to remove the `login`/`logged in as XX | logout` from the top right corner of the templates). This way, people can login from the front page and will see all the pages as being logged in, or they can just see the cached pages or the mirrorlist. We were leaning to wait for after the freeze to push this, but Adrian Reber reported that the issue of mirrorlist being sometime un-available is causing problem with the report-mirror script. So I would like to ask for a freeze-break to push to MirrorManager2 the changes made by Patrick: https://github.com/fedora-infra/mirrormanager2/pull/80 While at it, I would like to push another fix, by Adrian, allowing to always mark as up to date, mirrors that are always up to date: https://github.com/fedora-infra/mirrormanager2/pull/67 These changes have been prepared via a 0.1.0-3 RPM release: http://koji.fedoraproject.org/koji/taskinfo?taskID=9804162 built from: https://github.com/fedora-infra/mirrormanager2/commit/b368e3aa8988367fcf10148d1a06fb3e4c224357 We are now running 0.1.0-3 Thanks! Pierre pgpuCbNe9YJ5y.pgp Description: PGP signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request - change haproxy check URL for MirrorManager
On Wed, May 20, 2015 at 06:57:57AM -0600, Kevin Fenzi wrote: On Wed, 20 May 2015 14:31:28 +0200 Adrian Reber adr...@lisas.de wrote: Yes, but that was how it was before (I know that is bad argument ;-). The old MM landing page was a simple login page without any DB access. The current constellation is good that it tests that the whole thing is actually working, but it does it from all proxies. One check if the DB is working would be enough. The new nagios check for the publiclist which is redirected to /mirrormanager/ is also testing if it correctly works. Most accesses to /mirrormanager are from the haproxies and they are basically taking down MM every few hours. ok. So, perhaps we do this change for now and down the road we add a url that does a db check and just returns 'ok' or something ? So, +1 to this for now, but I'd like a better solution longer term if we can come up with one. Yes, puiterwijk was suggesting a /ping page especially for the proxy check. I opened a ticket for this: https://github.com/fedora-infra/mirrormanager2/issues/83 Adrian pgpQn9XoHZ_KF.pgp Description: PGP signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Plan for tomorrow's Fedora Infrastructure meeting (2015-05-21)
The infrastructure team will be having it's weekly meeting tomorrow, 2015-05-21 at 18:00 UTC in #fedora-meeting on the freenode network. This week we are continuing to try something new. We have a gobby document (see: https://fedoraproject.org/wiki/Gobby ) fedora-infrastructure-meeting-next is the document. Please try and review and edit that document before the meeting and we will use it to have our agenda of things to discuss. A copy as of this morning is included in this email. If you have something to discuss, add the topic to the discussion area with your name. If you would like to teach other folks about some application or setup in our infrastructure, please add that topic and your name to the learn about section. kevin -- = Introduction = This shared document is for the next fedora infrastructure meeting. We will use it over the week before the meeting to gather status and info and discussion items and so forth, then use it in the irc meeting to transfer information to the meetbot logs. = Meeting start stuff = #startmeeting Infrastructure (2015-05-21) #meetingname infrastructure #topic aloha #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk pbrobinson #topic New folks introductions / Apprentice feedback = Status / information / Trivia / Announcements = (We put things here we want others on the team to know, but don't need to discuss) (Please use #info the thing - your name) #topic announcements and information #info pagure now being backed up and in production - kevin #info download-ib02 was full, dropped 22-alpha/beta content - kevin #info Lots of work on mirrormanager2 to make it more robust - patrick/adrianr/pingou #info copr now moved to the new cloud - msuchy #info Added cpu's to proxy11 after it was not keeping up - kevin #info = Things we should discuss = We use this section to bring up discussion topics. Things we want to talk about as a group and come up with some consensus or decision or just brainstorm a problem or issue. If there are none of these we skip this section. (Use #topic your discussion topic - your username) #topic What are we going to do about meetbot (whither supybot)? - ralph #topic = Learn about some application or setup in infrastructure = (This section, each week we get 1 person to talk about an application or setup that we have. Just going over what it is, how to contribute, ideas for improvement, etc. Whoever would like to do this, just add the info in this section. In the event we don't find someone to teach about something, we skip this section and just move on to open floor.) #topic Learn about: Fedora People server - kevin = Meeting end stuff = #topic Open Floor #endmeeting pgp5hVpDvTKrB.pgp Description: OpenPGP digital signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Freeze break request: Add nagios internal monitoring for mm2 publiclist and fix mm2 static alias
Hi all,
Any +1s for these changes?
The static change is because currently all static assets are served through
Flask (which is highly inefficient and disables caching as it sends session
cookies).
With kind regards,
Patrick Uiterwijk
Fedora Infra
diff --git a/roles/mirrormanager/frontend2/templates/mirrormanager.conf
b/roles/mirrormanager/frontend2/templates/mirrormanager.conf
index ce6b082..6f12c24 100644
--- a/roles/mirrormanager/frontend2/templates/mirrormanager.conf
+++ b/roles/mirrormanager/frontend2/templates/mirrormanager.conf
@@ -1,6 +1,6 @@
# Apache configuration file for mirrormanager2
-Alias /mirrormanager2/static
/usr/lib/python2.7/site-packages/mirrormanager2/static/fedora/
+Alias /mirrormanager/static
/usr/lib/python2.7/site-packages/mirrormanager2/static/fedora/
WSGIDaemonProcess mirrormanager user=apache maximum-requests=1000
display-name=mirrormanager processes=2 threads=1
WSGISocketPrefix run/wsgi
diff --git a/roles/nagios_server/files/nagios/services/websites.cfg
b/roles/nagios_server/files/nagios/services/websites.cfg
index 67f49e9..a9f4e70 100644
--- a/roles/nagios_server/files/nagios/services/websites.cfg
+++ b/roles/nagios_server/files/nagios/services/websites.cfg
@@ -44,6 +44,14 @@ define service {
}
define service {
+ host_name mm-frontend01,mm-frontend01.stg
+ service_description mm-publiclist-internal
+ check_command check_website!localhost!/mirrormanager/
+ use internalwebsitetemplate
+ event_handler restart_httpd
+}
+
+define service {
host_name proxy01-wildcard, proxy02-wildcard, proxy03-fpo,
proxy04-fpo, proxy06-fpo, proxy07-wildcard, proxy08-wildcard, proxy09-wildcard,
proxy05-fpo, proxy10-fpo, proxy11-fpo
service_description start.fedoraproject.org
check_command check_website!start.fedoraproject.org!/!The Fedora
Project is maintained
___
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze Break request: add 2 cpus to proxy11
On Wed, May 20, 2015 at 07:57:46AM -0600, Kevin Fenzi wrote: This morning we had a few reports of mirrorlists timing out, and noticed that load on proxy11 was very high. It was using all 4 of it's cpus 100% and may have been dropping or timing out requests. I'd like to add 2 to it: +1s? kevin -- diff --git a/inventory/host_vars/proxy11.fedoraproject.org b/inventory/host_vars/proxy11.fedoraproject.org index 4c412de..d07e698 100644 --- a/inventory/host_vars/proxy11.fedoraproject.org +++ b/inventory/host_vars/proxy11.fedoraproject.org @@ -2,7 +2,7 @@ nm: 255.255.255.0 gw: 67.219.144.1 dns: 8.8.8.8 -num_cpus: 4 +num_cpus: 6 ks_url: http://209.132.181.6/repo/rhel/ks/kvm-rhel-7-ext ks_repo: http://209.132.181.6/repo/rhel/RHEL7-x86_64/ +1 for me as well, Pierre pgpXYMUBc6QfV.pgp Description: PGP signature ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request: push mirrormanager2-0.1.0-3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 05/20/2015 04:11 AM, Pierre-Yves Chibon wrote: We were leaning to wait for after the freeze to push this, but Adrian Reber reported that the issue of mirrorlist being sometime un-available is causing problem with the report-mirror script. As an aside, I reported a problem with the report_mirror script crashing when the mirrorlist is unavailable some months ago as BZ#1162275, where a trivial patch is available. -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: GPGTools - http://gpgtools.org iQIcBAEBCAAGBQJVXKHtAAoJEJICkBIKCqxcnIQQANgxhnuPYJowVYMlZ6rD2vE5 NBR2CyPNZuaxzCOhS6yL/rwzK7rDpSPc2LvGeQb7ysWsPtEBP5SKqffRZBUOzcYH /wH4uOByhB1CkZnkT2bAoxp8HLdDopiHtjrCNtPHORMft9xusS2ESchLGSp3XzDI AUzL+qb8RkdQdHS450BymRb6sUXdgjAzscCMk3LmMzSTilCwAEoWaW+St9vHMccR YVkTbFYDZTCZ4GxXJvXRNrRXhZ1c/kYcf+xiWDsThgeE3c6fl3pNLFsDywoO1IKu GK+dGEgxIjqFLzZIf4NZxxMFVpFzkjIRV15pW5318YOOBjPzbr6tT2U3iBBcz8z0 Cmj7gcyGQn7vuM9HPA2E1KvFMdKslV+FeeC6ZOCWB5oa/qQ6n8UczfmoSkMfIJeU A/U1dsm6JaZFKdzNj7D3ot8Lk+SR2vKWyKh6EyVjnkKcl/0HvpvTJtnC7iJ7AhuX wCAkxe0/jmZfaI2XEphgAn4a6EOgNf9VXKpCpsQwfQspaeSLnlXfK6JwMYavkSYs d5VWE5CStUfW5Cjytbr++snRhN2nNAIy4UHW89aA+vOforz54Zcg4WUlZeIC34+d 9Nq3PyfTCT6ZmG330zGjhpVqxp4XwwikoFEqdJz9nkQ1KQTx74/W0gz9dnqao5/N 7PfoQLKBH/IrvWBDsXcg =YH1a -END PGP SIGNATURE- ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Freeze break request - change haproxy check URL for MirrorManager
Thanks. Pushed. On Wed, May 20, 2015 at 09:57:33AM -0400, Patrick Uiterwijk wrote: Until we get /ping, I'm +1 on this. With kind regards, Patrick Uiterwijk Fedora Infra - Original Message - On Wed, May 20, 2015 at 06:57:57AM -0600, Kevin Fenzi wrote: On Wed, 20 May 2015 14:31:28 +0200 Adrian Reber adr...@lisas.de wrote: Yes, but that was how it was before (I know that is bad argument ;-). The old MM landing page was a simple login page without any DB access. The current constellation is good that it tests that the whole thing is actually working, but it does it from all proxies. One check if the DB is working would be enough. The new nagios check for the publiclist which is redirected to /mirrormanager/ is also testing if it correctly works. Most accesses to /mirrormanager are from the haproxies and they are basically taking down MM every few hours. ok. So, perhaps we do this change for now and down the road we add a url that does a db check and just returns 'ok' or something ? So, +1 to this for now, but I'd like a better solution longer term if we can come up with one. Yes, puiterwijk was suggesting a /ping page especially for the proxy check. I opened a ticket for this: https://github.com/fedora-infra/mirrormanager2/issues/83 Adrian ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
