Sorry for the slow reply. Has been far from a normal week this week. On 2 Apr 2015, at 11:40 pm, Justin Dearing <[email protected]> wrote:
> I have two flask apps. One is python 3, the other is python 2. I'd like to > run both through mod_wsgi on the same apache instance on the same ip/port > combo. Is this possible? So the basic limitation here is that the one running Apache instance can only handle either Python 2 or Python 3 code. This is because the Python interpreter itself is embedded inside of the Apache processes through indirect dynamic linking of the Python interpreter shared library when the mod_wsgi module is loaded into Apache. How library linking works means that it isn’t possible to load two different versions of the same dynamic shared library into the one process at the same time. The only solution therefore is to have two separate instances of Apache running, one for Python 2 and one for Python 3 code. The issue is how one achieves that given that systems are generally set up with only one Apache installation and for the normal user it is too large of a challenge to work out how to use that with two separate configurations and different builds of Apache modules such as mod_wsgi. Ignoring that latter point initially, lets consider that if one could easily run two separate Apache instances, one with mod_wsgi for Python 2 and another for Python 3, how would you even run them so they appear on the same ip/port combo. Right now, the only way to achieve that is through the use of a front end web proxy. What would normally be done is run a web server which accepts all requests on the public ip/port combination and which then based on either the virtual host name if separate sites, or based on sub URL if part of the same host, forward the request on to a backend instance in which the Python web application is running. As soon as one mentions a proxy, most people would run to nginx and use that, but Apache itself can be used as a proxy as well since you would already have that installed and setup to listen on the required ip/port combination. Whatever proxy is used, the proxy would then normally forward requests received to the separate instances of the Python web applications running Python 2 and 3. For the case of Apache/mod_wsgi there is another option. Since it is also already loading mod_wsgi, the primary Python web application can still run within the main Apache and you only proxy requests which need to be handled by the secondary application. In other words, the main Apache/mod_wsgi instance could be setup for Python 2 and handle the web application requiring Python 2. Any requests then destined for the Python 3 web application would instead be forwarded onto a second backend instance on a separate port running that application. Whatever way you do it, you can’t therefore avoid proxying for at least one of the Python web applications. Even if you were not using mod_wsgi, this restriction would still apply due to the basic limitation that you cannot run both a Python 2 and Python 3 application in the same process. The issue then is actually how can we easily run both a Python 2 and Python 3 web application at the same time with Apache/mod_wsgi even if in separate processes. At this point most people would give up and run one of the Python web applications with gunicorn, running it separately and forwarding on requests just for that one. This is because of the difficulty of running multiple Apache instances from the one Apache installation. Yes, you could manually install a separate Apache installation with mod_wsgi, but that is a pain. The better way at this point is actually to use mod_wsgi-express. Where mod_wsgi-express is useful is that it allows you to install mod_wsgi as part of a specific Python installation or virtual environment. You can therefore install it separately into both Python 2 and Python 3 installations. You can then run mod_wsgi-express out of each respective Python installation and they will use that version of Python. Key in this is that it will use your existing Apache installation and it will not touch or interfere with the system wide Apache configuration. The first option therefore is that you remove mod_wsgi from the main Apache installation and instead configure it simply as a proxy which will forward requests on to the two separate mod_wsgi-express instances. If you want you could even disable the main Apache installation, but not uninstall it, and run nginx as the front end proxy instead. You wouldn’t uninstall it, as mod_wsgi-express wants to still use it, just not with the system wide configuration. Instead mod_wsgi-express generates its own Apache configuration automatically for the purpose it is being used. Now I keep mentioning mod_wsgi-express but haven’t actually said what it is. If you haven’t been following my recent blog posts about it, then have a read of: http://blog.dscpl.com.au/2015/04/introducing-modwsgi-express.html https://pypi.python.org/pypi/mod_wsgi In short, mod_wsgi-express allows you to easily run multiple instances of Apache/mod_wsgi using different configurations and different versions of Python. So mod_wsgi-express solves the problem of how to run both Python 2 and Python 3 web applications at the same time, albeit by using separate Apache instances. As explained above, it is then just a matter of gluing them together using a proxy. I am not going to go into specific configurations for setting up the proxies here, but instead want too focus on one more specific issue. This issue is that as soon as you start talking about having multiple web applications running as distinct instances, you need to deal with how you get them started up when the system starts. Web servers such as Apache and nginx which you might use as a proxy are usually already integrated into system startup scripts. The issue is how you get your backend Python web applications running. Your choices here are to write the OS specific startup scripts for init.d, monit, systemd or whatever system is used to manage the processes. Alternatively, if supervisord is already setup as a secondary process management system, you could use that instead. What I want to talk about here is another trick which doesn’t require you to write any system startup scripts or use supervisord. What you would do is use your existing Apache/mod_wsgi installation to run your Python 2 application. You would then use the daemon mode capabilities of mod_wsgi to define a new daemon process group with a single process. This daemon process will be defined so it is run, but you will never setup mod_wsgi to run a WSGI application directly in it. That is, mod_wsgi itself would not use its own internal protocols and mechanisms to forward requests to it. As far as mod_wsgi is concerned it is an unused process. What instead we are going to do is use the fact that Apache/mod_wsgi will manage that process and keep it running. That is, if that process is killed it will be restarted. If Apache is restarted, the process will be restarted and if Apache is stopped, the process will be shutdown. Thus we have a simple process manager. What we can now do is tell mod_wsgi to import a specific Python script file into that special daemon process group process at the time it is started. In this Python script we can add code which will exec() a separate application. That separate application will then be managed by Apache and kept running. This separate process could be anything. We could even exec() an instance of Redis or Memcached and Apache would manage it. In this case what we can do though is exec() an instance of mod_wsgi-express which is installed for your Python 3 application. With the main Apache then managing that separate mod_wsgi-express instance, you would then also setup the main Apache to proxy requests for just the sub URL for that application, to whatever port the mod_wsgi-express instance is listening on. To summarise, we have your existing main Apache installation still running mod_wsgi but for Python 2. You install mod_wsgi-express against Python 3 and work out how to run it for your Python 3 application. You use a dummy mod_wsgi daemon process group to exec() the Python 3 instance of mod_wsgi-express for your application listening on a separate port. The main Apache is then setup to forward requests for the Python 3 part of your overall site, to the mod_wsgi-express instance. Now if you hadn’t worked it out, this actually means your main Apache instance, is via the mod_wsgi daemon process, managing a separate instance of Apache. This may sound scary, but it actually does work and is simpler to setup than this description may suggest. So I leave it up to you absorb all that I have said and work out what you might want to do. If you are interested in using mod_wsgi-express then try experimenting with it and get it running with your Python 3 application. I can then explain what you need to do to configure your existing Python 2 Apache instance to run the managed mod_wsgi-express and setup proxying for it properly. Graham -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
