Re: [openstack-dev] Apache2 vs uWSGI vs ...
There is no reason why the wsgi app container matters. This is simply a "we should document use if uwsgi and/or gunicorn as an alternative to mod_wsgi". If one solution is better for the gate it will be used there and each deployment will make the determination of what they want to use. Adam's point remains regardless of what wsgi solution is used. > On Sep 25, 2015, at 09:23, Adam Young wrote: > >> On 09/25/2015 07:09 AM, Sergii Golovatiuk wrote: >> Hi, >> >> Morgan gave the perfect case why operators want to use uWSGI. Let's imagine >> a future when all openstack services will work as mod_wsgi processes under >> apache. It's like to put all eggs in one basket. If you need to reconfigure >> one service on controller it may affect another service. For instance, >> sometimes operators need to increase number of Threads/Processes for wsgi or >> add new virtual host to apache. That will require graceful or cold restart >> of apache. It affects other services. Another case, internal problems in >> mod_wsgi where it may lead to apache crash affecting all services. >> >> uWSGI/gunicorn model is safer as in this case apache is reverse_proxy only. >> This model gives flexibility for operators. They may use apache/nginx as >> proxy or load balancer. Stop or crash of one service won't lead to downtime >> of other services. The complexity of OpenStack management will be easier and >> friendly. > > There are some fallacies here: > > 1. OpenStack services should all be on the same machine. > 2. OpenStack web services should run on ports other than 443. > > I think both of these are ideas who's time have come and gone. > > If you have a single machine, run them out of separate containers. That > allows different services to work with different versions of the libraries. > It lets you mix a newer Keystone with older everything else. > Often the APIs are deployed on a common set of nodes. > If everything is on port 443, you need a single web server at the front end > to multiplex it; uWSGI or any other one does not obviate that. > ++ > > There are no good ports left in /etc/services; stop trying to reserve new > ones for the web. If you need to run on a web service, you need to be able > to get through firewalls. You need to run on standard ports. Run on 443. > > Keystone again is a great example of this: it has two ports: 5000 and 35357. > > port 5000 in /etc/services is > > commplex-main 5000/tcp > > and port 35357 is smack dab in the middle of the ephemeral range. > This is a disconnect between linux and IANA. IANA has said 35357 is not ephemeral, linux defaults to say it is. > > Again, so long as the web server supports the cryptographic secure > mechanisms, I don't care what one you chose. But The idea of use going to > Keystone and getting a bearer token as the basis for security is immature; > we should be doing the following on every call: > > 1. TLS > 2. Cryptographic authentication. > > > They can be together or split up. > > So, lets get everything running inside Apache, and, at the same time, push > our other favorite web servers to support the necessary pieces to make > OpenStack and the Web secure. > ++. We should do this and also document alternatives for wsgi which has no impact on this goal. Lets try and keep focused on the different initiatives and not cross the reasons for them. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
On 09/25/2015 07:09 AM, Sergii Golovatiuk wrote: Hi, Morgan gave the perfect case why operators want to use uWSGI. Let's imagine a future when all openstack services will work as mod_wsgi processes under apache. It's like to put all eggs in one basket. If you need to reconfigure one service on controller it may affect another service. For instance, sometimes operators need to increase number of Threads/Processes for wsgi or add new virtual host to apache. That will require graceful or cold restart of apache. It affects other services. Another case, internal problems in mod_wsgi where it may lead to apache crash affecting all services. uWSGI/gunicorn model is safer as in this case apache is reverse_proxy only. This model gives flexibility for operators. They may use apache/nginx as proxy or load balancer. Stop or crash of one service won't lead to downtime of other services. The complexity of OpenStack management will be easier and friendly. There are some fallacies here: 1. OpenStack services should all be on the same machine. 2. OpenStack web services should run on ports other than 443. I think both of these are ideas who's time have come and gone. If you have a single machine, run them out of separate containers. That allows different services to work with different versions of the libraries. It lets you mix a newer Keystone with older everything else. If everything is on port 443, you need a single web server at the front end to multiplex it; uWSGI or any other one does not obviate that. There are no good ports left in /etc/services; stop trying to reserve new ones for the web. If you need to run on a web service, you need to be able to get through firewalls. You need to run on standard ports. Run on 443. Keystone again is a great example of this: it has two ports: 5000 and 35357. port 5000 in /etc/services is commplex-main 5000/tcp and port 35357 is smack dab in the middle of the ephemeral range. Again, so long as the web server supports the cryptographic secure mechanisms, I don't care what one you chose. But The idea of use going to Keystone and getting a bearer token as the basis for security is immature; we should be doing the following on every call: 1. TLS 2. Cryptographic authentication. They can be together or split up. So, lets get everything running inside Apache, and, at the same time, push our other favorite web servers to support the necessary pieces to make OpenStack and the Web secure. -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Fri, Sep 18, 2015 at 3:44 PM, Morgan Fainberg mailto:morgan.fainb...@gmail.com>> wrote: There is and has been desire to support uWSGI and other alternatives to mod_wsgi. There are a variety of operational reasons to consider uWSGI and/or gunicorn behind apache most notably to facilitate easier management of the processes independently of the webserver itself. With mod_wsgi the processes are directly tied to the apache server where as with uWSGI and gunicorn you can manage the various services independently and/or with differing VENVs more easily. There are potential other concerns that must be weighed when considering which method of deployment to use. I hope we have clear documentation within the next cycle (and possible choices for the gate) for utilizing uWSGI and/or gunicorn. --Morgan Sent via mobile On Sep 18, 2015, at 06:12, Adam Young mailto:ayo...@redhat.com>> wrote: On 09/17/2015 10:04 PM, Jim Rollenhagen wrote: On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: In the fuel project, we recently ran into a couple of issues with Apache2 + mod_wsgi as we switched Keystone to run . Please see [1] and [2]. Looking deep into Apache2 issues specifically around "apache2ctl graceful" and module loading/unloading and the hooks used by mod_wsgi [3]. I started wondering if Apache2 + mod_wsgi is the "right" solution and if there was something else better that people are already using. One data point that keeps coming up is, all the CI jobs use Apache2 + mod_wsgi so it must be the best solutionIs it? If not, what is? Disclaimer: it's been a while since I've cared about performance with a web server in front of a Python app. IIRC, mod_wsgi was abandoned for a while, but I think it's being worked on again. In general, I seem to remember it being thought of as a bit old and crusty, but mostly working. I am not aware of that. It has been the workhorse of the Python/wsgi world for a while, and we use it heavily. At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] and saw a significant performance increase. This was a Django app. uwsgi is fairly straightforward to operate and comes loaded with a myriad of options[1] to help folks make the most of it. I've played with Ironic behi
Re: [openstack-dev] Apache2 vs uWSGI vs ...
OK, sorry I mixed up nginx and uwsgi :) A. On Fri, Sep 25, 2015 at 2:54 PM, David Stanek wrote: > > On Fri, Sep 25, 2015 at 8:25 AM Adam Heczko wrote: > >> Are we discussing mod_wsgi and Keystone or OpenStack as a general? >> If Keystone specific use case, then probably Apache provides broadest >> choice of tested external authenticators. >> I'm not against uwsgi at all, but to be honest expectation that nginx >> could substitute Apache in terms of authentication providers is simply >> unrealistic. >> > > uwsgi isn't a replacement for Apache. It's a replacement for mod_wsgi. It > just so happens that it does let you use user web servers if that's what > your usecase dictates. > > As a Keystone developer I don't want to tell deployers that they have to > use Apache. It should be their choice. Since Apache is the most common web > server in our community I think we should continue to provide example > configurations and guidance for it. > > > -- David > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Adam Heczko Security Engineer @ Mirantis Inc. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
David, I am thinking how all OpenStack components may coexist as mod_wsgi processes under apache. In Fuel we stepped into problem with deployment using graceful restart [1] thus this thread was raised to have a good WSGI alternatives. [1] https://github.com/GrahamDumpleton/mod_wsgi/pull/95 -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
On Fri, Sep 25, 2015 at 8:25 AM Adam Heczko wrote: > Are we discussing mod_wsgi and Keystone or OpenStack as a general? > If Keystone specific use case, then probably Apache provides broadest > choice of tested external authenticators. > I'm not against uwsgi at all, but to be honest expectation that nginx > could substitute Apache in terms of authentication providers is simply > unrealistic. > uwsgi isn't a replacement for Apache. It's a replacement for mod_wsgi. It just so happens that it does let you use user web servers if that's what your usecase dictates. As a Keystone developer I don't want to tell deployers that they have to use Apache. It should be their choice. Since Apache is the most common web server in our community I think we should continue to provide example configurations and guidance for it. -- David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
Are we discussing mod_wsgi and Keystone or OpenStack as a general? If Keystone specific use case, then probably Apache provides broadest choice of tested external authenticators. I'm not against uwsgi at all, but to be honest expectation that nginx could substitute Apache in terms of authentication providers is simply unrealistic. A. On Fri, Sep 25, 2015 at 1:24 PM, Sergii Golovatiuk wrote: > Alexandr, > > oauth, shibboleth & openid support are very keystone specific features. > Many other openstack projects don't need these modules at all but they may > require faster HTTP server (lighthttp/nginx). > > For all projects we may use "HTTP server -> uwsgi" model and leave apache > for keystone as " HTTP server -> apache -> uwsgi/mod_wsgi". However, I > would like to think about whole Openstack ecosystem in general. In that > case we'll minimize the number of programs operator should have knowledge. > > > > > -- > Best regards, > Sergii Golovatiuk, > Skype #golserge > IRC #holser > > On Fri, Sep 18, 2015 at 4:28 PM, Alexander Makarov > wrote: > >> Please consider that we use some apache mods - does >> nginx/uwsgi/gunicorn have oauth, shibboleth & openid support? >> >> On Fri, Sep 18, 2015 at 4:54 PM, Vladimir Kuklin >> wrote: >> > Folks >> > >> > I think we do not need to switch to nginx-only or consider any kind of >> war >> > between nginx and apache adherents. Everyone should be able to use >> > web-server he or she needs without being pinned to the unwanted one. It >> is >> > like Postgres vs MySQL war. Why not support both? >> > >> > May be someone does not need something that apache supports and nginx >> not >> > and needs nginx features which apache does not support. Let's let our >> users >> > decide what they want. >> > >> > And the first step should be simple here - support for uwsgi. It will >> allow >> > for usage of any web-server that can work with uwsgi. It will allow >> also us >> > to check for the support of all apache-like bindings like SPNEGO or >> whatever >> > and provide our users with enough info on making decisions. I did not >> > personally test nginx modules for SAML and SPNEGO, but I am pretty >> confident >> > about TLS/SSL parts of nginx. >> > >> > Moreover, nginx will allow you to do things you cannot do with apache, >> e.g. >> > do smart load balancing, which may be crucial for high-loaded >> installations. >> > >> > >> > On Fri, Sep 18, 2015 at 4:12 PM, Adam Young wrote: >> >> >> >> On 09/17/2015 10:04 PM, Jim Rollenhagen wrote: >> >> >> >> On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: >> >> >> >> In the fuel project, we recently ran into a couple of issues with >> Apache2 >> >> + >> >> mod_wsgi as we switched Keystone to run . Please see [1] and [2]. >> >> >> >> Looking deep into Apache2 issues specifically around "apache2ctl >> graceful" >> >> and module loading/unloading and the hooks used by mod_wsgi [3]. I >> started >> >> wondering if Apache2 + mod_wsgi is the "right" solution and if there >> was >> >> something else better that people are already using. >> >> >> >> One data point that keeps coming up is, all the CI jobs use Apache2 + >> >> mod_wsgi so it must be the best solutionIs it? If not, what is? >> >> >> >> Disclaimer: it's been a while since I've cared about performance with a >> >> web server in front of a Python app. >> >> >> >> IIRC, mod_wsgi was abandoned for a while, but I think it's being worked >> >> on again. In general, I seem to remember it being thought of as a bit >> >> old and crusty, but mostly working. >> >> >> >> >> >> I am not aware of that. It has been the workhorse of the Python/wsgi >> >> world for a while, and we use it heavily. >> >> >> >> At a previous job, we switched from Apache2 + mod_wsgi to nginx + >> uwsgi[0] >> >> and saw a significant performance increase. This was a Django app. >> uwsgi >> >> is fairly straightforward to operate and comes loaded with a myriad of >> >> options[1] to help folks make the most of it. I've played with Ironic >> >> behind uwsgi and it seemed to work fine, though I haven't done any sort >> >> of load testing. I'd encourage folks to give it a shot. :) >> >> >> >> >> >> Again, switching web servers is as likely to introduce as to solve >> >> problems. If there are performance issues: >> >> >> >> 1. Idenitfy what causes them >> >> 2. Change configuration settings to deal with them >> >> 3. Fix upstream bugs in the underlying system. >> >> >> >> >> >> Keystone is not about performance. Keystone is about security. The >> cloud >> >> is designed to scale horizontally first. Before advocating switching >> to a >> >> difference web server, make sure it supports the technologies required. >> >> >> >> >> >> 1. TLS at the latest level >> >> 2. Kerberos/GSSAPI/SPNEGO >> >> 3. X509 Client cert validation >> >> 4. SAML >> >> >> >> OpenID connect would be a good one to add to the list; Its been >> requested >> >> for a while. >> >> >> >> If Keystone is having performance issues,
Re: [openstack-dev] Apache2 vs uWSGI vs ...
Alexandr, oauth, shibboleth & openid support are very keystone specific features. Many other openstack projects don't need these modules at all but they may require faster HTTP server (lighthttp/nginx). For all projects we may use "HTTP server -> uwsgi" model and leave apache for keystone as " HTTP server -> apache -> uwsgi/mod_wsgi". However, I would like to think about whole Openstack ecosystem in general. In that case we'll minimize the number of programs operator should have knowledge. -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Fri, Sep 18, 2015 at 4:28 PM, Alexander Makarov wrote: > Please consider that we use some apache mods - does > nginx/uwsgi/gunicorn have oauth, shibboleth & openid support? > > On Fri, Sep 18, 2015 at 4:54 PM, Vladimir Kuklin > wrote: > > Folks > > > > I think we do not need to switch to nginx-only or consider any kind of > war > > between nginx and apache adherents. Everyone should be able to use > > web-server he or she needs without being pinned to the unwanted one. It > is > > like Postgres vs MySQL war. Why not support both? > > > > May be someone does not need something that apache supports and nginx not > > and needs nginx features which apache does not support. Let's let our > users > > decide what they want. > > > > And the first step should be simple here - support for uwsgi. It will > allow > > for usage of any web-server that can work with uwsgi. It will allow also > us > > to check for the support of all apache-like bindings like SPNEGO or > whatever > > and provide our users with enough info on making decisions. I did not > > personally test nginx modules for SAML and SPNEGO, but I am pretty > confident > > about TLS/SSL parts of nginx. > > > > Moreover, nginx will allow you to do things you cannot do with apache, > e.g. > > do smart load balancing, which may be crucial for high-loaded > installations. > > > > > > On Fri, Sep 18, 2015 at 4:12 PM, Adam Young wrote: > >> > >> On 09/17/2015 10:04 PM, Jim Rollenhagen wrote: > >> > >> On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: > >> > >> In the fuel project, we recently ran into a couple of issues with > Apache2 > >> + > >> mod_wsgi as we switched Keystone to run . Please see [1] and [2]. > >> > >> Looking deep into Apache2 issues specifically around "apache2ctl > graceful" > >> and module loading/unloading and the hooks used by mod_wsgi [3]. I > started > >> wondering if Apache2 + mod_wsgi is the "right" solution and if there was > >> something else better that people are already using. > >> > >> One data point that keeps coming up is, all the CI jobs use Apache2 + > >> mod_wsgi so it must be the best solutionIs it? If not, what is? > >> > >> Disclaimer: it's been a while since I've cared about performance with a > >> web server in front of a Python app. > >> > >> IIRC, mod_wsgi was abandoned for a while, but I think it's being worked > >> on again. In general, I seem to remember it being thought of as a bit > >> old and crusty, but mostly working. > >> > >> > >> I am not aware of that. It has been the workhorse of the Python/wsgi > >> world for a while, and we use it heavily. > >> > >> At a previous job, we switched from Apache2 + mod_wsgi to nginx + > uwsgi[0] > >> and saw a significant performance increase. This was a Django app. uwsgi > >> is fairly straightforward to operate and comes loaded with a myriad of > >> options[1] to help folks make the most of it. I've played with Ironic > >> behind uwsgi and it seemed to work fine, though I haven't done any sort > >> of load testing. I'd encourage folks to give it a shot. :) > >> > >> > >> Again, switching web servers is as likely to introduce as to solve > >> problems. If there are performance issues: > >> > >> 1. Idenitfy what causes them > >> 2. Change configuration settings to deal with them > >> 3. Fix upstream bugs in the underlying system. > >> > >> > >> Keystone is not about performance. Keystone is about security. The > cloud > >> is designed to scale horizontally first. Before advocating switching > to a > >> difference web server, make sure it supports the technologies required. > >> > >> > >> 1. TLS at the latest level > >> 2. Kerberos/GSSAPI/SPNEGO > >> 3. X509 Client cert validation > >> 4. SAML > >> > >> OpenID connect would be a good one to add to the list; Its been > requested > >> for a while. > >> > >> If Keystone is having performance issues, it is most likely at the > >> database layer, not the web server. > >> > >> > >> > >> "Programmers waste enormous amounts of time thinking about, or worrying > >> about, the speed of noncritical parts of their programs, and these > attempts > >> at efficiency actually have a strong negative impact when debugging and > >> maintenance are considered. We should forget about small efficiencies, > say > >> about 97% of the time: premature optimization is the root of all evil. > Yet > >> we should not pass up our opportunities in that critical 3%."
Re: [openstack-dev] Apache2 vs uWSGI vs ...
Hi, Morgan gave the perfect case why operators want to use uWSGI. Let's imagine a future when all openstack services will work as mod_wsgi processes under apache. It's like to put all eggs in one basket. If you need to reconfigure one service on controller it may affect another service. For instance, sometimes operators need to increase number of Threads/Processes for wsgi or add new virtual host to apache. That will require graceful or cold restart of apache. It affects other services. Another case, internal problems in mod_wsgi where it may lead to apache crash affecting all services. uWSGI/gunicorn model is safer as in this case apache is reverse_proxy only. This model gives flexibility for operators. They may use apache/nginx as proxy or load balancer. Stop or crash of one service won't lead to downtime of other services. The complexity of OpenStack management will be easier and friendly. -- Best regards, Sergii Golovatiuk, Skype #golserge IRC #holser On Fri, Sep 18, 2015 at 3:44 PM, Morgan Fainberg wrote: > There is and has been desire to support uWSGI and other alternatives to > mod_wsgi. There are a variety of operational reasons to consider uWSGI > and/or gunicorn behind apache most notably to facilitate easier management > of the processes independently of the webserver itself. With mod_wsgi the > processes are directly tied to the apache server where as with uWSGI and > gunicorn you can manage the various services independently and/or with > differing VENVs more easily. > > There are potential other concerns that must be weighed when considering > which method of deployment to use. I hope we have clear documentation > within the next cycle (and possible choices for the gate) for utilizing > uWSGI and/or gunicorn. > > --Morgan > > Sent via mobile > > On Sep 18, 2015, at 06:12, Adam Young wrote: > > On 09/17/2015 10:04 PM, Jim Rollenhagen wrote: > > On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: > > In the fuel project, we recently ran into a couple of issues with Apache2 + > mod_wsgi as we switched Keystone to run . Please see [1] and [2]. > > Looking deep into Apache2 issues specifically around "apache2ctl graceful" > and module loading/unloading and the hooks used by mod_wsgi [3]. I started > wondering if Apache2 + mod_wsgi is the "right" solution and if there was > something else better that people are already using. > > One data point that keeps coming up is, all the CI jobs use Apache2 + > mod_wsgi so it must be the best solutionIs it? If not, what is? > > Disclaimer: it's been a while since I've cared about performance with a > web server in front of a Python app. > > IIRC, mod_wsgi was abandoned for a while, but I think it's being worked > on again. In general, I seem to remember it being thought of as a bit > old and crusty, but mostly working. > > > I am not aware of that. It has been the workhorse of the Python/wsgi > world for a while, and we use it heavily. > > At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] > and saw a significant performance increase. This was a Django app. uwsgi > is fairly straightforward to operate and comes loaded with a myriad of > options[1] to help folks make the most of it. I've played with Ironic > behind uwsgi and it seemed to work fine, though I haven't done any sort > of load testing. I'd encourage folks to give it a shot. :) > > > Again, switching web servers is as likely to introduce as to solve > problems. If there are performance issues: > > 1. Idenitfy what causes them > 2. Change configuration settings to deal with them > 3. Fix upstream bugs in the underlying system. > > > Keystone is not about performance. Keystone is about security. The cloud > is designed to scale horizontally first. Before advocating switching to a > difference web server, make sure it supports the technologies required. > > > 1. TLS at the latest level > 2. Kerberos/GSSAPI/SPNEGO > 3. X509 Client cert validation > 4. SAML > > OpenID connect would be a good one to add to the list; Its been requested > for a while. > > If Keystone is having performance issues, it is most likely at the > database layer, not the web server. > > > > "Programmers waste enormous amounts of time thinking about, or worrying > about, the speed of noncritical parts of their programs, and these attempts > at efficiency actually have a strong negative impact when debugging and > maintenance are considered. We *should* forget about small efficiencies, > say about 97% of the time: *premature optimization is the root of all > evil.* Yet we should not pass up our opportunities in that critical > 3%." --Donald Knuth > > > > Of course, uwsgi can also be ran behind Apache2, if you'd prefer. > > gunicorn[2] is another good option that may be worth investigating; I > personally don't have any experience with it, but I seem to remember > hearing it has good eventlet support. > > // jim > > [0] https://uwsgi-docs.readthedocs.org/en/latest/ > [1] h
Re: [openstack-dev] Apache2 vs uWSGI vs ...
On Fri, Sep 18, 2015 at 11:09 AM, Vladimir Kuklin wrote: > I just suggested to untie keystone from wsgi and implement uwsgi support. > And then let the user decide what he or she wants. > Keystone is not tied to Apache or mod_wsgi, if that's what you mean. We provide a sample configuration for Apache + mod_wsgi because deployers tend to be more familiar with Apache than other web servers, and there happens to be a mature ecosystem of auth modules that keystone can utilize. It's working documentation, just like devstack itself. Use whatever you want to deploy keystone. Choose the thing that you're familiar with, can support effectively, run reliably, and has sufficient performance. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
The conversations around alternative wsgi containers (uwsgi, gunicorn, etc) still would be tied to apache or nginx. While it is possible to deploy without the webservers in some cases, jt would not be recommended nor do I see that being tested in gate. We want to leverage the auth support of the webserver. While uWSGI has interesting features, it wouldnt really replace nginx or apache. It is just an alternative deployment strategy that makes managing the keystone (or any OpenStack service) processes independent of the web server. I do not expect any of these choices to have a performance impact. It will ease some operational concerns. There is no reason uwsgi or gunicorn wont work today (I have run keystone with both in a devstack locally). The documentation and communicating the reasons to pick one or the other is all that is needed. From a gate perspective configuration and restarts of the web server could be made easier in devstack with uWSGI. It would not prevent use of mod_wsgi. --Morgan Sent via mobile > On Sep 18, 2015, at 09:53, Yee, Guang wrote: > > I am with Adam, I kinda doubt Apache cause performance issues for Keystone, > especially since all we have are just simple REST APIs. For other services > with specific needs, like large file streaming, there may be some arguments > to pick one over the other. We haven’t had the need to use Apache for dynamic > routing or proxying at deployment. I am sure there are better tools out there > that can do the job. > > Making Keystone web service independent is a fine goal. However, since > Keystone is moving away from being an identity provider, federation and > external auth will play a major part going forward. Some of them are depended > on the Apache at the moment. We may have to refactor Keystone to isolate the > authentication service from the rest, then figure out how to gate it with > Apache. I don’t think that’s trivial work though. > > At this point, I don’t know what we are really gaining by ripping out Apache, > comparing to amount of work to make that happen. > > > Guang > > > From: Vladimir Kuklin [mailto:vkuk...@mirantis.com] > Sent: Friday, September 18, 2015 9:09 AM > To: OpenStack Development Mailing List (not for usage questions) > Subject: [openstack-dev] Apache2 vs uWSGI vs ... > > Folks > > I just suggested to untie keystone from wsgi and implement uwsgi support. And > then let the user decide what he or she wants. > > There is a plenty of auth modules for nginx also. > > Nginx us much better as a proxy server and you know it. > > Regarding mod wsgi and apache we already saw that it cannot handle simple > restart. I think this is not in any way acceptable from operations point if > view. > > 18 сент. 2015 г. 18:59 пользователь "Fox, Kevin M" > написал: > Part of the reason to use Apache though is the diverse set of authentication > mechanisms it supports. Operators have the desire to plugin Keystone into > their existing authentication systems and Apache tends to be easier to do > that then others. > > Thanks, > Kevin > > From: Jim Rollenhagen [j...@jimrollenhagen.com] > Sent: Thursday, September 17, 2015 7:04 PM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] Apache2 vs uWSGI vs ... > > On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: > > In the fuel project, we recently ran into a couple of issues with Apache2 + > > mod_wsgi as we switched Keystone to run . Please see [1] and [2]. > > > > Looking deep into Apache2 issues specifically around "apache2ctl graceful" > > and module loading/unloading and the hooks used by mod_wsgi [3]. I started > > wondering if Apache2 + mod_wsgi is the "right" solution and if there was > > something else better that people are already using. > > > > One data point that keeps coming up is, all the CI jobs use Apache2 + > > mod_wsgi so it must be the best solutionIs it? If not, what is? > > Disclaimer: it's been a while since I've cared about performance with a > web server in front of a Python app. > > IIRC, mod_wsgi was abandoned for a while, but I think it's being worked > on again. In general, I seem to remember it being thought of as a bit > old and crusty, but mostly working. > > At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] > and saw a significant performance increase. This was a Django app. uwsgi > is fairly straightforward to operate and comes loaded with a myriad of > options[1] to help folks make the most of it. I've played with Ironic > behind uwsgi and it seemed to work fine, t
Re: [openstack-dev] Apache2 vs uWSGI vs ...
I am with Adam, I kinda doubt Apache cause performance issues for Keystone, especially since all we have are just simple REST APIs. For other services with specific needs, like large file streaming, there may be some arguments to pick one over the other. We haven’t had the need to use Apache for dynamic routing or proxying at deployment. I am sure there are better tools out there that can do the job. Making Keystone web service independent is a fine goal. However, since Keystone is moving away from being an identity provider, federation and external auth will play a major part going forward. Some of them are depended on the Apache at the moment. We may have to refactor Keystone to isolate the authentication service from the rest, then figure out how to gate it with Apache. I don’t think that’s trivial work though. At this point, I don’t know what we are really gaining by ripping out Apache, comparing to amount of work to make that happen. Guang From: Vladimir Kuklin [mailto:vkuk...@mirantis.com] Sent: Friday, September 18, 2015 9:09 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] Apache2 vs uWSGI vs ... Folks I just suggested to untie keystone from wsgi and implement uwsgi support. And then let the user decide what he or she wants. There is a plenty of auth modules for nginx also. Nginx us much better as a proxy server and you know it. Regarding mod wsgi and apache we already saw that it cannot handle simple restart. I think this is not in any way acceptable from operations point if view. 18 сент. 2015 г. 18:59 пользователь "Fox, Kevin M" mailto:kevin@pnnl.gov>> написал: Part of the reason to use Apache though is the diverse set of authentication mechanisms it supports. Operators have the desire to plugin Keystone into their existing authentication systems and Apache tends to be easier to do that then others. Thanks, Kevin From: Jim Rollenhagen [j...@jimrollenhagen.com<mailto:j...@jimrollenhagen.com>] Sent: Thursday, September 17, 2015 7:04 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] Apache2 vs uWSGI vs ... On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: > In the fuel project, we recently ran into a couple of issues with Apache2 + > mod_wsgi as we switched Keystone to run . Please see [1] and [2]. > > Looking deep into Apache2 issues specifically around "apache2ctl graceful" > and module loading/unloading and the hooks used by mod_wsgi [3]. I started > wondering if Apache2 + mod_wsgi is the "right" solution and if there was > something else better that people are already using. > > One data point that keeps coming up is, all the CI jobs use Apache2 + > mod_wsgi so it must be the best solutionIs it? If not, what is? Disclaimer: it's been a while since I've cared about performance with a web server in front of a Python app. IIRC, mod_wsgi was abandoned for a while, but I think it's being worked on again. In general, I seem to remember it being thought of as a bit old and crusty, but mostly working. At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] and saw a significant performance increase. This was a Django app. uwsgi is fairly straightforward to operate and comes loaded with a myriad of options[1] to help folks make the most of it. I've played with Ironic behind uwsgi and it seemed to work fine, though I haven't done any sort of load testing. I'd encourage folks to give it a shot. :) Of course, uwsgi can also be ran behind Apache2, if you'd prefer. gunicorn[2] is another good option that may be worth investigating; I personally don't have any experience with it, but I seem to remember hearing it has good eventlet support. // jim [0] https://uwsgi-docs.readthedocs.org/en/latest/ [1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html [2] http://gunicorn.org/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Apache2 vs uWSGI vs ...
Folks I just suggested to untie keystone from wsgi and implement uwsgi support. And then let the user decide what he or she wants. There is a plenty of auth modules for nginx also. Nginx us much better as a proxy server and you know it. Regarding mod wsgi and apache we already saw that it cannot handle simple restart. I think this is not in any way acceptable from operations point if view. 18 сент. 2015 г. 18:59 пользователь "Fox, Kevin M" написал: > Part of the reason to use Apache though is the diverse set of > authentication mechanisms it supports. Operators have the desire to plugin > Keystone into their existing authentication systems and Apache tends to be > easier to do that then others. > > Thanks, > Kevin > > From: Jim Rollenhagen [j...@jimrollenhagen.com] > Sent: Thursday, September 17, 2015 7:04 PM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] Apache2 vs uWSGI vs ... > > On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: > > In the fuel project, we recently ran into a couple of issues with > Apache2 + > > mod_wsgi as we switched Keystone to run . Please see [1] and [2]. > > > > Looking deep into Apache2 issues specifically around "apache2ctl > graceful" > > and module loading/unloading and the hooks used by mod_wsgi [3]. I > started > > wondering if Apache2 + mod_wsgi is the "right" solution and if there was > > something else better that people are already using. > > > > One data point that keeps coming up is, all the CI jobs use Apache2 + > > mod_wsgi so it must be the best solutionIs it? If not, what is? > > Disclaimer: it's been a while since I've cared about performance with a > web server in front of a Python app. > > IIRC, mod_wsgi was abandoned for a while, but I think it's being worked > on again. In general, I seem to remember it being thought of as a bit > old and crusty, but mostly working. > > At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] > and saw a significant performance increase. This was a Django app. uwsgi > is fairly straightforward to operate and comes loaded with a myriad of > options[1] to help folks make the most of it. I've played with Ironic > behind uwsgi and it seemed to work fine, though I haven't done any sort > of load testing. I'd encourage folks to give it a shot. :) > > Of course, uwsgi can also be ran behind Apache2, if you'd prefer. > > gunicorn[2] is another good option that may be worth investigating; I > personally don't have any experience with it, but I seem to remember > hearing it has good eventlet support. > > // jim > > [0] https://uwsgi-docs.readthedocs.org/en/latest/ > [1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html > [2] http://gunicorn.org/ > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
I thoughts below mention Keystone, but in reality I would apply the same logic to any OpenStack service. On Fri, Sep 18, 2015 at 10:32 AM, Boris Bobrov wrote: > There are 2 dimensions this discussion should happen in: web server and > application server. Now we use apache2 as web server and mod_wsgi as app > server. > This is exactly true and Keystone should be deployable in an WSGI compliant application server. If it's not I would consider it a bug. > > I don't have a specific opinion on the app server (mod_wsgi vs uwsgi) and I > don't really care. > > Regarding apache2 vs nginx. I don't see any reasons for the switch. > Apache2 is > well known to deployers and sysadmins. It is very rich for modules. I > wonder > if there are customer-written modules. > Keystone doesn't use or require Apache. We recommend that it is deployed using Apache, but there is no requirement to do that if you don't need to use any Apache modules. For example, at least one of my devstack nodes happily runs Keystone's manage_all. [snip] > There are things in keystone that work under apache. They are not tested. > They > were written to work under apache because it's the simplest and the most > standard way to do. Making them work in nginx means forcing developers > write > some code. You're ready to do that? > This should only be true for optional features and currently require Apache modules. > > > May be someone does not need something that apache supports and nginx not > > and needs nginx features which apache does not support. Let's let our > users > > decide what they want. > > > > And the first step should be simple here - support for uwsgi. > > Why uwsgi? Why not gunicorn? Cherrypy? Twisted? > uwsgi and gunicorn should both work fine, as should any WSGI application server. CherryPy and Twister are more framework than application server, so I would not expect them to work. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
Part of the reason to use Apache though is the diverse set of authentication mechanisms it supports. Operators have the desire to plugin Keystone into their existing authentication systems and Apache tends to be easier to do that then others. Thanks, Kevin From: Jim Rollenhagen [j...@jimrollenhagen.com] Sent: Thursday, September 17, 2015 7:04 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] Apache2 vs uWSGI vs ... On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: > In the fuel project, we recently ran into a couple of issues with Apache2 + > mod_wsgi as we switched Keystone to run . Please see [1] and [2]. > > Looking deep into Apache2 issues specifically around "apache2ctl graceful" > and module loading/unloading and the hooks used by mod_wsgi [3]. I started > wondering if Apache2 + mod_wsgi is the "right" solution and if there was > something else better that people are already using. > > One data point that keeps coming up is, all the CI jobs use Apache2 + > mod_wsgi so it must be the best solutionIs it? If not, what is? Disclaimer: it's been a while since I've cared about performance with a web server in front of a Python app. IIRC, mod_wsgi was abandoned for a while, but I think it's being worked on again. In general, I seem to remember it being thought of as a bit old and crusty, but mostly working. At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] and saw a significant performance increase. This was a Django app. uwsgi is fairly straightforward to operate and comes loaded with a myriad of options[1] to help folks make the most of it. I've played with Ironic behind uwsgi and it seemed to work fine, though I haven't done any sort of load testing. I'd encourage folks to give it a shot. :) Of course, uwsgi can also be ran behind Apache2, if you'd prefer. gunicorn[2] is another good option that may be worth investigating; I personally don't have any experience with it, but I seem to remember hearing it has good eventlet support. // jim [0] https://uwsgi-docs.readthedocs.org/en/latest/ [1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html [2] http://gunicorn.org/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
There are 2 dimensions this discussion should happen in: web server and application server. Now we use apache2 as web server and mod_wsgi as app server. I don't have a specific opinion on the app server (mod_wsgi vs uwsgi) and I don't really care. Regarding apache2 vs nginx. I don't see any reasons for the switch. Apache2 is well known to deployers and sysadmins. It is very rich for modules. I wonder if there are customer-written modules. On Friday 18 September 2015 16:54:02 Vladimir Kuklin wrote: > Folks > > I think we do not need to switch to nginx-only or consider any kind of war > between nginx and apache adherents. Everyone should be able to use > web-server he or she needs without being pinned to the unwanted one. It is > like Postgres vs MySQL war. Why not support both? Why nginx? Why not lighttpd? OpenLitespeed? Litespeed? ? What do you understand by "support both"? I understand it as "both are tested in devstack". Apache2 is supported because you can set up devstack and everything works. There are things in keystone that work under apache. They are not tested. They were written to work under apache because it's the simplest and the most standard way to do. Making them work in nginx means forcing developers write some code. You're ready to do that? > May be someone does not need something that apache supports and nginx not > and needs nginx features which apache does not support. Let's let our users > decide what they want. > > And the first step should be simple here - support for uwsgi. Why uwsgi? Why not gunicorn? Cherrypy? Twisted? > It will allow > for usage of any web-server that can work with uwsgi. It will allow also us > to check for the support of all apache-like bindings like SPNEGO or > whatever and provide our users with enough info on making decisions. I did > not personally test nginx modules for SAML and SPNEGO, but I am pretty > confident about TLS/SSL parts of nginx. > > Moreover, nginx will allow you to do things you cannot do with apache, e.g. > do smart load balancing, which may be crucial for high-loaded installations. > On Fri, Sep 18, 2015 at 4:12 PM, Adam Young wrote: > > On 09/17/2015 10:04 PM, Jim Rollenhagen wrote: > > > > On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: > > > > In the fuel project, we recently ran into a couple of issues with Apache2 > > + > > mod_wsgi as we switched Keystone to run . Please see [1] and [2]. > > > > Looking deep into Apache2 issues specifically around "apache2ctl graceful" > > and module loading/unloading and the hooks used by mod_wsgi [3]. I started > > wondering if Apache2 + mod_wsgi is the "right" solution and if there was > > something else better that people are already using. > > > > One data point that keeps coming up is, all the CI jobs use Apache2 + > > mod_wsgi so it must be the best solutionIs it? If not, what is? > > > > Disclaimer: it's been a while since I've cared about performance with a > > web server in front of a Python app. > > > > IIRC, mod_wsgi was abandoned for a while, but I think it's being worked > > on again. In general, I seem to remember it being thought of as a bit > > old and crusty, but mostly working. > > > > > > I am not aware of that. It has been the workhorse of the Python/wsgi > > world for a while, and we use it heavily. > > > > > > At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] > > and saw a significant performance increase. This was a Django app. uwsgi > > is fairly straightforward to operate and comes loaded with a myriad of > > options[1] to help folks make the most of it. I've played with Ironic > > behind uwsgi and it seemed to work fine, though I haven't done any sort > > of load testing. I'd encourage folks to give it a shot. :) > > > > > > Again, switching web servers is as likely to introduce as to solve > > problems. If there are performance issues: > > > > 1. Idenitfy what causes them > > 2. Change configuration settings to deal with them > > 3. Fix upstream bugs in the underlying system. > > > > > > Keystone is not about performance. Keystone is about security. The cloud > > is designed to scale horizontally first. Before advocating switching to a > > difference web server, make sure it supports the technologies required. > > > > > > 1. TLS at the latest level > > 2. Kerberos/GSSAPI/SPNEGO > > 3. X509 Client cert validation > > 4. SAML > > > > OpenID connect would be a good one to add to the list; Its been requested > > for a while. > > > > If Keystone is having performance issues, it is most likely at the > > database layer, not the web server. > > > > > > > > "Programmers waste enormous amounts of time thinking about, or worrying > > about, the speed of noncritical parts of their programs, and these > > attempts > > at efficiency actually have a strong negative impact when debugging and > > maintenance are considered. We *should* forget about small efficiencies, > > say about 97% of
Re: [openstack-dev] Apache2 vs uWSGI vs ...
Please consider that we use some apache mods - does nginx/uwsgi/gunicorn have oauth, shibboleth & openid support? On Fri, Sep 18, 2015 at 4:54 PM, Vladimir Kuklin wrote: > Folks > > I think we do not need to switch to nginx-only or consider any kind of war > between nginx and apache adherents. Everyone should be able to use > web-server he or she needs without being pinned to the unwanted one. It is > like Postgres vs MySQL war. Why not support both? > > May be someone does not need something that apache supports and nginx not > and needs nginx features which apache does not support. Let's let our users > decide what they want. > > And the first step should be simple here - support for uwsgi. It will allow > for usage of any web-server that can work with uwsgi. It will allow also us > to check for the support of all apache-like bindings like SPNEGO or whatever > and provide our users with enough info on making decisions. I did not > personally test nginx modules for SAML and SPNEGO, but I am pretty confident > about TLS/SSL parts of nginx. > > Moreover, nginx will allow you to do things you cannot do with apache, e.g. > do smart load balancing, which may be crucial for high-loaded installations. > > > On Fri, Sep 18, 2015 at 4:12 PM, Adam Young wrote: >> >> On 09/17/2015 10:04 PM, Jim Rollenhagen wrote: >> >> On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: >> >> In the fuel project, we recently ran into a couple of issues with Apache2 >> + >> mod_wsgi as we switched Keystone to run . Please see [1] and [2]. >> >> Looking deep into Apache2 issues specifically around "apache2ctl graceful" >> and module loading/unloading and the hooks used by mod_wsgi [3]. I started >> wondering if Apache2 + mod_wsgi is the "right" solution and if there was >> something else better that people are already using. >> >> One data point that keeps coming up is, all the CI jobs use Apache2 + >> mod_wsgi so it must be the best solutionIs it? If not, what is? >> >> Disclaimer: it's been a while since I've cared about performance with a >> web server in front of a Python app. >> >> IIRC, mod_wsgi was abandoned for a while, but I think it's being worked >> on again. In general, I seem to remember it being thought of as a bit >> old and crusty, but mostly working. >> >> >> I am not aware of that. It has been the workhorse of the Python/wsgi >> world for a while, and we use it heavily. >> >> At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] >> and saw a significant performance increase. This was a Django app. uwsgi >> is fairly straightforward to operate and comes loaded with a myriad of >> options[1] to help folks make the most of it. I've played with Ironic >> behind uwsgi and it seemed to work fine, though I haven't done any sort >> of load testing. I'd encourage folks to give it a shot. :) >> >> >> Again, switching web servers is as likely to introduce as to solve >> problems. If there are performance issues: >> >> 1. Idenitfy what causes them >> 2. Change configuration settings to deal with them >> 3. Fix upstream bugs in the underlying system. >> >> >> Keystone is not about performance. Keystone is about security. The cloud >> is designed to scale horizontally first. Before advocating switching to a >> difference web server, make sure it supports the technologies required. >> >> >> 1. TLS at the latest level >> 2. Kerberos/GSSAPI/SPNEGO >> 3. X509 Client cert validation >> 4. SAML >> >> OpenID connect would be a good one to add to the list; Its been requested >> for a while. >> >> If Keystone is having performance issues, it is most likely at the >> database layer, not the web server. >> >> >> >> "Programmers waste enormous amounts of time thinking about, or worrying >> about, the speed of noncritical parts of their programs, and these attempts >> at efficiency actually have a strong negative impact when debugging and >> maintenance are considered. We should forget about small efficiencies, say >> about 97% of the time: premature optimization is the root of all evil. Yet >> we should not pass up our opportunities in that critical 3%." --Donald >> Knuth >> >> >> >> Of course, uwsgi can also be ran behind Apache2, if you'd prefer. >> >> gunicorn[2] is another good option that may be worth investigating; I >> personally don't have any experience with it, but I seem to remember >> hearing it has good eventlet support. >> >> // jim >> >> [0] https://uwsgi-docs.readthedocs.org/en/latest/ >> [1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html >> [2] http://gunicorn.org/ >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> __ >> OpenStack Development M
Re: [openstack-dev] Apache2 vs uWSGI vs ...
On 18/09/15 06:44 -0700, Morgan Fainberg wrote: There is and has been desire to support uWSGI and other alternatives to mod_wsgi. There are a variety of operational reasons to consider uWSGI and/or gunicorn behind apache most notably to facilitate easier management of the processes independently of the webserver itself. With mod_wsgi the processes are directly tied to the apache server where as with uWSGI and gunicorn you can manage the various services independently and/or with differing VENVs more easily. There are potential other concerns that must be weighed when considering which method of deployment to use. I hope we have clear documentation within the next cycle (and possible choices for the gate) for utilizing uWSGI and/or gunicorn. +1 FWIW, Zaqar has always been shipped as a wsgi app and the container the team has recommended ever since it was put in production for the first time has been uWSGI. uWSGI is already used by Zaqar in the gate but it's being installed independently. Flavio --Morgan Sent via mobile On Sep 18, 2015, at 06:12, Adam Young wrote: On 09/17/2015 10:04 PM, Jim Rollenhagen wrote: On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: In the fuel project, we recently ran into a couple of issues with Apache2 + mod_wsgi as we switched Keystone to run . Please see [1] and [2]. Looking deep into Apache2 issues specifically around "apache2ctl graceful" and module loading/unloading and the hooks used by mod_wsgi [3]. I started wondering if Apache2 + mod_wsgi is the "right" solution and if there was something else better that people are already using. One data point that keeps coming up is, all the CI jobs use Apache2 + mod_wsgi so it must be the best solutionIs it? If not, what is? Disclaimer: it's been a while since I've cared about performance with a web server in front of a Python app. IIRC, mod_wsgi was abandoned for a while, but I think it's being worked on again. In general, I seem to remember it being thought of as a bit old and crusty, but mostly working. I am not aware of that. It has been the workhorse of the Python/wsgi world for a while, and we use it heavily. At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] and saw a significant performance increase. This was a Django app. uwsgi is fairly straightforward to operate and comes loaded with a myriad of options[1] to help folks make the most of it. I've played with Ironic behind uwsgi and it seemed to work fine, though I haven't done any sort of load testing. I'd encourage folks to give it a shot. :) Again, switching web servers is as likely to introduce as to solve problems. If there are performance issues: 1. Idenitfy what causes them 2. Change configuration settings to deal with them 3. Fix upstream bugs in the underlying system. Keystone is not about performance. Keystone is about security. The cloud is designed to scale horizontally first. Before advocating switching to a difference web server, make sure it supports the technologies required. 1. TLS at the latest level 2. Kerberos/GSSAPI/SPNEGO 3. X509 Client cert validation 4. SAML OpenID connect would be a good one to add to the list; Its been requested for a while. If Keystone is having performance issues, it is most likely at the database layer, not the web server. "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%." --Donald Knuth Of course, uwsgi can also be ran behind Apache2, if you'd prefer. gunicorn[2] is another good option that may be worth investigating; I personally don't have any experience with it, but I seem to remember hearing it has good eventlet support. // jim [0] https://uwsgi-docs.readthedocs.org/en/latest/ [1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html [2] http://gunicorn.org/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ
Re: [openstack-dev] Apache2 vs uWSGI vs ...
Folks I think we do not need to switch to nginx-only or consider any kind of war between nginx and apache adherents. Everyone should be able to use web-server he or she needs without being pinned to the unwanted one. It is like Postgres vs MySQL war. Why not support both? May be someone does not need something that apache supports and nginx not and needs nginx features which apache does not support. Let's let our users decide what they want. And the first step should be simple here - support for uwsgi. It will allow for usage of any web-server that can work with uwsgi. It will allow also us to check for the support of all apache-like bindings like SPNEGO or whatever and provide our users with enough info on making decisions. I did not personally test nginx modules for SAML and SPNEGO, but I am pretty confident about TLS/SSL parts of nginx. Moreover, nginx will allow you to do things you cannot do with apache, e.g. do smart load balancing, which may be crucial for high-loaded installations. On Fri, Sep 18, 2015 at 4:12 PM, Adam Young wrote: > On 09/17/2015 10:04 PM, Jim Rollenhagen wrote: > > On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: > > In the fuel project, we recently ran into a couple of issues with Apache2 + > mod_wsgi as we switched Keystone to run . Please see [1] and [2]. > > Looking deep into Apache2 issues specifically around "apache2ctl graceful" > and module loading/unloading and the hooks used by mod_wsgi [3]. I started > wondering if Apache2 + mod_wsgi is the "right" solution and if there was > something else better that people are already using. > > One data point that keeps coming up is, all the CI jobs use Apache2 + > mod_wsgi so it must be the best solutionIs it? If not, what is? > > Disclaimer: it's been a while since I've cared about performance with a > web server in front of a Python app. > > IIRC, mod_wsgi was abandoned for a while, but I think it's being worked > on again. In general, I seem to remember it being thought of as a bit > old and crusty, but mostly working. > > > I am not aware of that. It has been the workhorse of the Python/wsgi > world for a while, and we use it heavily. > > > At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] > and saw a significant performance increase. This was a Django app. uwsgi > is fairly straightforward to operate and comes loaded with a myriad of > options[1] to help folks make the most of it. I've played with Ironic > behind uwsgi and it seemed to work fine, though I haven't done any sort > of load testing. I'd encourage folks to give it a shot. :) > > > Again, switching web servers is as likely to introduce as to solve > problems. If there are performance issues: > > 1. Idenitfy what causes them > 2. Change configuration settings to deal with them > 3. Fix upstream bugs in the underlying system. > > > Keystone is not about performance. Keystone is about security. The cloud > is designed to scale horizontally first. Before advocating switching to a > difference web server, make sure it supports the technologies required. > > > 1. TLS at the latest level > 2. Kerberos/GSSAPI/SPNEGO > 3. X509 Client cert validation > 4. SAML > > OpenID connect would be a good one to add to the list; Its been requested > for a while. > > If Keystone is having performance issues, it is most likely at the > database layer, not the web server. > > > > "Programmers waste enormous amounts of time thinking about, or worrying > about, the speed of noncritical parts of their programs, and these attempts > at efficiency actually have a strong negative impact when debugging and > maintenance are considered. We *should* forget about small efficiencies, > say about 97% of the time: *premature optimization is the root of all > evil.* Yet we should not pass up our opportunities in that critical > 3%." --Donald Knuth > > > > > Of course, uwsgi can also be ran behind Apache2, if you'd prefer. > > gunicorn[2] is another good option that may be worth investigating; I > personally don't have any experience with it, but I seem to remember > hearing it has good eventlet support. > > // jim > > [0] https://uwsgi-docs.readthedocs.org/en/latest/ > [1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html > [2] http://gunicorn.org/ > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Yours Faithfully, Vladimir Kuklin, Fuel Library Tech Lead, Mirantis, Inc. +7 (495) 640-49-04 +7 (926) 702-3
Re: [openstack-dev] Apache2 vs uWSGI vs ...
There is and has been desire to support uWSGI and other alternatives to mod_wsgi. There are a variety of operational reasons to consider uWSGI and/or gunicorn behind apache most notably to facilitate easier management of the processes independently of the webserver itself. With mod_wsgi the processes are directly tied to the apache server where as with uWSGI and gunicorn you can manage the various services independently and/or with differing VENVs more easily. There are potential other concerns that must be weighed when considering which method of deployment to use. I hope we have clear documentation within the next cycle (and possible choices for the gate) for utilizing uWSGI and/or gunicorn. --Morgan Sent via mobile > On Sep 18, 2015, at 06:12, Adam Young wrote: > >> On 09/17/2015 10:04 PM, Jim Rollenhagen wrote: >>> On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: >>> In the fuel project, we recently ran into a couple of issues with Apache2 + >>> mod_wsgi as we switched Keystone to run . Please see [1] and [2]. >>> >>> Looking deep into Apache2 issues specifically around "apache2ctl graceful" >>> and module loading/unloading and the hooks used by mod_wsgi [3]. I started >>> wondering if Apache2 + mod_wsgi is the "right" solution and if there was >>> something else better that people are already using. >>> >>> One data point that keeps coming up is, all the CI jobs use Apache2 + >>> mod_wsgi so it must be the best solutionIs it? If not, what is? >> Disclaimer: it's been a while since I've cared about performance with a >> web server in front of a Python app. >> >> IIRC, mod_wsgi was abandoned for a while, but I think it's being worked >> on again. In general, I seem to remember it being thought of as a bit >> old and crusty, but mostly working. > > I am not aware of that. It has been the workhorse of the Python/wsgi world > for a while, and we use it heavily. > >> At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] >> and saw a significant performance increase. This was a Django app. uwsgi >> is fairly straightforward to operate and comes loaded with a myriad of >> options[1] to help folks make the most of it. I've played with Ironic >> behind uwsgi and it seemed to work fine, though I haven't done any sort >> of load testing. I'd encourage folks to give it a shot. :) > > Again, switching web servers is as likely to introduce as to solve problems. > If there are performance issues: > > 1. Idenitfy what causes them > 2. Change configuration settings to deal with them > 3. Fix upstream bugs in the underlying system. > > > Keystone is not about performance. Keystone is about security. The cloud is > designed to scale horizontally first. Before advocating switching to a > difference web server, make sure it supports the technologies required. > > > 1. TLS at the latest level > 2. Kerberos/GSSAPI/SPNEGO > 3. X509 Client cert validation > 4. SAML > > OpenID connect would be a good one to add to the list; Its been requested > for a while. > > If Keystone is having performance issues, it is most likely at the database > layer, not the web server. > > > > "Programmers waste enormous amounts of time thinking about, or worrying > about, the speed of noncritical parts of their programs, and these attempts > at efficiency actually have a strong negative impact when debugging and > maintenance are considered. We should forget about small efficiencies, say > about 97% of the time: premature optimization is the root of all evil. Yet we > should not pass up our opportunities in that critical 3%." --Donald Knuth > > > >> Of course, uwsgi can also be ran behind Apache2, if you'd prefer. >> >> gunicorn[2] is another good option that may be worth investigating; I >> personally don't have any experience with it, but I seem to remember >> hearing it has good eventlet support. >> >> // jim >> >> [0] https://uwsgi-docs.readthedocs.org/en/latest/ >> [1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html >> [2] http://gunicorn.org/ >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
On 09/17/2015 10:04 PM, Jim Rollenhagen wrote: On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: In the fuel project, we recently ran into a couple of issues with Apache2 + mod_wsgi as we switched Keystone to run . Please see [1] and [2]. Looking deep into Apache2 issues specifically around "apache2ctl graceful" and module loading/unloading and the hooks used by mod_wsgi [3]. I started wondering if Apache2 + mod_wsgi is the "right" solution and if there was something else better that people are already using. One data point that keeps coming up is, all the CI jobs use Apache2 + mod_wsgi so it must be the best solutionIs it? If not, what is? Disclaimer: it's been a while since I've cared about performance with a web server in front of a Python app. IIRC, mod_wsgi was abandoned for a while, but I think it's being worked on again. In general, I seem to remember it being thought of as a bit old and crusty, but mostly working. I am not aware of that. It has been the workhorse of the Python/wsgi world for a while, and we use it heavily. At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] and saw a significant performance increase. This was a Django app. uwsgi is fairly straightforward to operate and comes loaded with a myriad of options[1] to help folks make the most of it. I've played with Ironic behind uwsgi and it seemed to work fine, though I haven't done any sort of load testing. I'd encourage folks to give it a shot. :) Again, switching web servers is as likely to introduce as to solve problems. If there are performance issues: 1. Idenitfy what causes them 2. Change configuration settings to deal with them 3. Fix upstream bugs in the underlying system. Keystone is not about performance. Keystone is about security. The cloud is designed to scale horizontally first. Before advocating switching to a difference web server, make sure it supports the technologies required. 1. TLS at the latest level 2. Kerberos/GSSAPI/SPNEGO 3. X509 Client cert validation 4. SAML OpenID connect would be a good one to add to the list; Its been requested for a while. If Keystone is having performance issues, it is most likely at the database layer, not the web server. "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We /should/ forget about small efficiencies, say about 97% of the time: *premature optimization is the root of all evil.* Yet we should not pass up our opportunities in that critical 3%." --Donald Knuth Of course, uwsgi can also be ran behind Apache2, if you'd prefer. gunicorn[2] is another good option that may be worth investigating; I personally don't have any experience with it, but I seem to remember hearing it has good eventlet support. // jim [0] https://uwsgi-docs.readthedocs.org/en/latest/ [1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html [2] http://gunicorn.org/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
On Thu, Sep 17, 2015 at 06:48:50PM -0400, Davanum Srinivas wrote: > In the fuel project, we recently ran into a couple of issues with Apache2 + > mod_wsgi as we switched Keystone to run . Please see [1] and [2]. > > Looking deep into Apache2 issues specifically around "apache2ctl graceful" > and module loading/unloading and the hooks used by mod_wsgi [3]. I started > wondering if Apache2 + mod_wsgi is the "right" solution and if there was > something else better that people are already using. > > One data point that keeps coming up is, all the CI jobs use Apache2 + > mod_wsgi so it must be the best solutionIs it? If not, what is? Disclaimer: it's been a while since I've cared about performance with a web server in front of a Python app. IIRC, mod_wsgi was abandoned for a while, but I think it's being worked on again. In general, I seem to remember it being thought of as a bit old and crusty, but mostly working. At a previous job, we switched from Apache2 + mod_wsgi to nginx + uwsgi[0] and saw a significant performance increase. This was a Django app. uwsgi is fairly straightforward to operate and comes loaded with a myriad of options[1] to help folks make the most of it. I've played with Ironic behind uwsgi and it seemed to work fine, though I haven't done any sort of load testing. I'd encourage folks to give it a shot. :) Of course, uwsgi can also be ran behind Apache2, if you'd prefer. gunicorn[2] is another good option that may be worth investigating; I personally don't have any experience with it, but I seem to remember hearing it has good eventlet support. // jim [0] https://uwsgi-docs.readthedocs.org/en/latest/ [1] https://uwsgi-docs.readthedocs.org/en/latest/Options.html [2] http://gunicorn.org/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Apache2 vs uWSGI vs ...
On 09/17/2015 06:48 PM, Davanum Srinivas wrote: In the fuel project, we recently ran into a couple of issues with Apache2 + mod_wsgi as we switched Keystone to run . Please see [1] and [2]. Looking deep into Apache2 issues specifically around "apache2ctl graceful" and module loading/unloading and the hooks used by mod_wsgi [3]. I started wondering if Apache2 + mod_wsgi is the "right" solution and if there was something else better that people are already using. One data point that keeps coming up is, all the CI jobs use Apache2 + mod_wsgi so it must be the best solutionIs it? If not, what is? Thanks, Dims I'd be surprised if switching web servers fixed more problems than it causes. The issues with Apache seem to be issues that are solvable; is there any reason to think that they are not? [1] https://bugs.launchpad.net/mos/+bug/1491576 [2] https://bugs.launchpad.net/fuel/+bug/1493372 [3] https://bugs.launchpad.net/fuel/+bug/1493372/comments/35 -- Davanum Srinivas :: https://twitter.com/dims __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Apache2 vs uWSGI vs ...
In the Fuel project, we recently ran into few issues with Apache2 + mod_wsgi as we switched Keystone to run in that environment and we started a large battery of tests and ended up seeing these issues. Please see [1] and [2] for examples. Looking deep into Apache2 issues specifically around "apache2ctl graceful" and module loading/unloading and the hooks used by mod_wsgi [3]. I started wondering if Apache2 + mod_wsgi is the "right" solution and if there was something else better that people are already using. One data point that keeps coming up is, all the CI jobs use Apache2 + mod_wsgi so it must be the best solutionIs it? If not, what is? Thanks, Dims PS: I will leave issues with memcached + keystone for another email later :) [1] https://bugs.launchpad.net/mos/+bug/1491576 [2] https://bugs.launchpad.net/fuel/+bug/1493372 [3] https://bugs.launchpad.net/fuel/+bug/1493372/comments/35 -- Davanum Srinivas :: https://twitter.com/dims __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Apache2 vs uWSGI vs ...
In the fuel project, we recently ran into a couple of issues with Apache2 + mod_wsgi as we switched Keystone to run . Please see [1] and [2]. Looking deep into Apache2 issues specifically around "apache2ctl graceful" and module loading/unloading and the hooks used by mod_wsgi [3]. I started wondering if Apache2 + mod_wsgi is the "right" solution and if there was something else better that people are already using. One data point that keeps coming up is, all the CI jobs use Apache2 + mod_wsgi so it must be the best solutionIs it? If not, what is? Thanks, Dims [1] https://bugs.launchpad.net/mos/+bug/1491576 [2] https://bugs.launchpad.net/fuel/+bug/1493372 [3] https://bugs.launchpad.net/fuel/+bug/1493372/comments/35 -- Davanum Srinivas :: https://twitter.com/dims __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
