Hi Graham, Thank you for all the help you've provided, it's been crucial to fixing my installation and configuration of mod_wsgi. I managed to get a little help from a developer who proceeded to test the wsgi.py file we utilise and after a little debugging he found out that the issue stemmed from the inclusion of newrelic within the script. I proceeded to upgrade the package from version 2.16.0.12 to 2.80.1.61 which fixed the issue completely and page load times went from ~40 to ~5 seconds . I also checked the Apache access logs where I saw the line that points toward mod_wsgi loading the WSGI script.
Many Thanks, David On Monday, 6 March 2017 23:02:46 UTC, Graham Dumpleton wrote: > > If you have info set for LogLevel, you should see a line like: > > mod_wsgi (pid=85737, process='localhost:8000', application=''): Loading > WSGI script '/tmp/mod_wsgi-localhost:8000:502/handler.wsgi'. > > Are you saying you do not see that in the error log? > > When pre-loading is setup, you should see that when any process is first > loaded. If you are only see it on the first request, that isn't right. > > So take note of where you see that. > > Wait a full minute before doing the first request so can see whether it is > the global import of the WSGI script file on process start which takes a > long time and delaying first request because initial import still not > finished. > > Graham > > On 7 Mar 2017, at 12:48 AM, David <[email protected] <javascript:>> wrote: > > Hi Graham, > > I had mistyped the python-home directory location when I submitted my > response yesterday. The python-home configuration setting in the virtual > host file is indeed correct, sorry for the confusion. I've looked into and > fixed the "No module named site" issue thanks to the help of your advice > on another post using the commands seen below. > > ./configure --with-python=/usr/local/bin/python2.7 > LD_RUN_PATH=/usr/local/lib make > make install > > The installation procedure that configured it incorrectly before was via > configuration management software which I will now update. The overall > configuration is loadable and will respond to requests so it's great to > have everything setup as correctly as it can be now. > > I have still to find out what the issue is regarding slow response times > when a request is made to the application after the processes have been > created. I have included snippets from Apaches logs to hopefully provide > some further information in case it shines a light on an issue that I can't > see for myself. > > The logs show me restarting the httpd service using the "systemctl > restart httpd" command at 13:32:51. I then make a request at 13:33:03 at > which point I wait for roughly 30 seconds before I receive the response > from the server showing the page requested. It should be noted that the log > entry in the xxx.com.access.log file at 13:33:03 didn't actually appear > until 13:33:33 or so, so it's a bit misleading at a glance as well as the > wait time can be between 30 - 60 seconds but in the example below it is > roughly 30 seconds. > > ==> /var/log/httpd/error_log <== > [Mon Mar 06 13:32:51.981992 2017] [core:info] [pid 11691:tid > 140191937927296] AH00096: removed PID file /run/httpd/httpd.pid (pid=11691) > [Mon Mar 06 13:32:51.982022 2017] [mpm_worker:notice] [pid 11691:tid > 140191937927296] AH00296: caught SIGWINCH, shutting down gracefully > [Mon Mar 06 13:32:52.999652 2017] [wsgi:info] [pid 11691:tid > 140191937927296] mod_wsgi (pid=11692): Process 'xxx.com' to be > deregistered, as server is restarting or being shutdown. > [Mon Mar 06 13:32:52.999708 2017] [wsgi:info] [pid 11691:tid > 140191937927296] mod_wsgi (pid=11692): Process 'xxx.com' has been > deregistered and will no longer be monitored. > [Mon Mar 06 13:32:54.583209 2017] [auth_digest:notice] [pid 11996:tid > 140146629888128] AH01757: generating secret for digest authentication ... > [Mon Mar 06 13:32:54.588361 2017] [mpm_worker:notice] [pid 11996:tid > 140146629888128] AH00292: Apache/2.4.6 (CentOS) mod_wsgi/4.5.14 Python/2.7 > configured -- resuming normal operations > [Mon Mar 06 13:32:54.588382 2017] [mpm_worker:info] [pid 11996:tid > 140146629888128] AH00293: Server built: Nov 14 2016 18:04:44 > [Mon Mar 06 13:32:54.588397 2017] [core:notice] [pid 11996:tid > 140146629888128] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND' > [Mon Mar 06 13:32:54.602488 2017] [wsgi:info] [pid 11997:tid > 140146629888128] mod_wsgi (pid=11997): Starting process 'xxx.com' with > uid=1001, gid=1003 and threads=2. > [Mon Mar 06 13:32:54.602891 2017] [wsgi:info] [pid 11997:tid > 140146629888128] mod_wsgi (pid=11997): Python home /srv/django/xxx.com. > [Mon Mar 06 13:32:54.602915 2017] [wsgi:info] [pid 11997:tid > 140146629888128] mod_wsgi (pid=11997): Initializing Python. > [Mon Mar 06 13:32:54.616328 2017] [wsgi:info] [pid 11998:tid > 140146629888128] mod_wsgi (pid=11998): Starting process 'xxx.com' with > uid=1001, gid=1003 and threads=2. > [Mon Mar 06 13:32:54.616745 2017] [wsgi:info] [pid 11998:tid > 140146629888128] mod_wsgi (pid=11998): Python home /srv/django/xxx.com. > [Mon Mar 06 13:32:54.616767 2017] [wsgi:info] [pid 11998:tid > 140146629888128] mod_wsgi (pid=11998): Initializing Python. > > ==> /var/log/httpd/xxx.com.access.log <== > xxx.com 222.222.222.222 "111.111.111.111" - [06/Mar/2017:13:33:03 +0000] > "GET / HTTP/1.1" 200 66126 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X > 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 > Safari/537.36" "-" > xxx.com 222.222.222.222 "111.111.111.111" - [06/Mar/2017:13:33:37 +0000] > "GET /request2/ HTTP/1.1" 200 17570 "https://xxx.com/"; "Mozilla/5.0 > (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) > Chrome/56.0.2924.87 Safari/537.36" "-" > > Thanks, > David > > On Monday, 6 March 2017 00:06:45 UTC, Graham Dumpleton wrote: >> >> >> On 6 Mar 2017, at 10:10 AM, David <[email protected]> wrote: >> >> Hi Graham, >> >> First off thank you very much for the support you have given, I really >> appreciate the time you've taken to help me. I read over the changes you >> kindly pointed out and have now amended the virtual host file to use the >> following configuration. >> >> *virtual host file* >> WSGIRestrictEmbedded On >> <VirtualHost *:80> >> >> ... >> >> WSGIPassAuthorization On >> >> WSGIDaemonProcess xxx.com user=django group=admin processes=2 >> threads=2 display-name=xxx.com python-home=/srv/django/xxx >> >> >> Did you get the argument to python-home wrong here. Doesn't match below. >> >> WSGIScriptAlias / /srv/django/xxx.com/xxx/xxx/wsgi.py process-group= >> xxx.com application-group=%{GLOBAL} >> >> <Directory /srv/django/xxx.com/xxx/> >> Order allow,deny >> Allow from all >> </Directory> >> >> </VirtualHost> >> >> >> Once I had this configuration in place I received the following error. >> >> [wsgi:info] [pid 19595:tid 140504746649728] mod_wsgi (pid=19595): Python >> home /srv/django/xxx.com. >> [wsgi:info] [pid 19595:tid 140504746649728] mod_wsgi (pid=19595): >> Initializing Python. >> ImportError: No module named site >> >> >> When using python-home, this can indicate that your mod_wsgi may not be >> compiled with the same Python base installation as the virtual environment >> was created from. >> >> >> http://modwsgi.readthedocs.io/en/develop/user-guides/checking-your-installation.html#python-installation-in-use >> >> or that is not picking up correct Python shared library. >> >> I double checked my installation of mod_wsgi and confirmed that it was >> indeed configured against my alternative installed python version that I >> had wanted and then checked my virtual environment was using the same >> alternative installed Python version and it was. >> >> >> The problem is where your system Python installation is same major/minor >> version as an alternate Python installation. If you didn't override which >> Python library was being used, it could be picking up system Python library >> and not that for alternative installation. >> >> I then decided to pip install the mod_wsgi package within the virtual >> environment and then modified the "LoadModules" path for mod_wsgi to that >> of the virtual environments as can be seen below. >> >> LoadModule wsgi_module /srv/django/ >> xxx.com/lib/python2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so >> >> >> If using that method, ensure you use the output of running: >> >> mod_wsgi-express module-config >> >> This alleviated the issue of "No module named site" >> >> >> When using pip install method, it tries to ensure that correct Python >> shared library is linked explicitly into the .so file. To be extra safe, >> don't calculate path of .so yourself. Include all the output from running: >> >> mod_wsgi-express module-config >> >> into Apache configuration. This can include a LoadFile line for force >> loads correct Python library. >> >> Also includes a global WSGIPythonHome to ensure using correct Python >> runtime directory. >> >> but it still left me page load times of roughly ~1 minute upon request. >> >> >> Ensure you have LogLevel in Apache set to 'info' and not 'warn' or >> 'error'. >> >> The 'info' level will show when mod_wsgi loads WSGI script file initially >> so can verify occurs on process start. >> >> I restarted the server and waited roughly a minute before requesting a >> page, hoping to verify that the application had loaded into the processes >> by then but instead of seeing a response of 3 seconds or less I got ~1 >> minute. This is leaving me to think that the application isn't being loaded >> into the processes until a request has been made whereas I though the >> configuration allowed the application to preload before a request was made? >> >> Is there any where else to look at this point that I may be overlooking, >> I have a feeling that it may be in and around the area that led me to pip >> install mod_wsgi instead of using my source installed version? >> >> >> Check things about, but also enable logging at 'info' level so can see >> when things are loaded. Will then work out where to go next. >> >> Many Thanks, >> David >> >> On Saturday, 4 March 2017 22:21:36 UTC, Graham Dumpleton wrote: >>> >>> >>> On 5 Mar 2017, at 7:42 AM, David <[email protected]> wrote: >>> >>> I've found an issue in heavily increased loading time when mod_wsgi >>> loads the application for the first time. This has started since since >>> upgrading from my old setup of CentOS 6, Python 2.7.5, Apache 2.2.15, and >>> mod_wsgi 3.4 to my new setup of CentOS 7, Python 2.7.13, Apache 2.4.6, and >>> mod_wsgi 4.5.13. >>> >>> The change has caused the initial loading time of the application into >>> memory from ~5 seconds to ~1 minute. I have followed the "Checking Your >>> Installation" as much as I could, coming up with a blank as to what could >>> be causing this increase. >>> >>> The mod_wsgi and Apache configuration is the same across both the older >>> and new setup, configured to use the worker mpm module running in daemon >>> mode. Please find the configuration of both the virtualhost file and the >>> worker configuration found in the main httpd.conf file below. >>> >>> *httpd.conf* >>> <IfModule mpm_worker_module> >>> StartServers 16 >>> MaxClients 250 >>> MinSpareThreads 10 >>> MaxSpareThreads 10 >>> ThreadsPerChild 25 >>> MaxRequestsPerChild 0 >>> </IfModule> >>> >>> >>> *virtualhost configuration* >>> >>> WSGISocketPrefix run/wsgi >>> WSGILazyInitialization On >>> >>> >>> This is the default, you don't need this directive. >>> >>> WSGIRestrictEmbedded Off >>> >>> >>> You want to set this to On, not Off. >>> >>> <VitualHost *:80> >>> >>> ... >>> >>> WSGIPassAuthorization On >>> WSGIDaemonProcess xxx.com user=django group=admin processes=2 >>> threads=2 display-name=xxx.com >>> python-path=/srv/django/xxx/lib/python2.7/site-packages >>> >>> >>> Use: >>> >>> WSGIDaemonProcess xxx.com user=django group=admin processes=2 >>> threads=2 display-name=xxx.com python-home=/srv/django/xxx >>> >>> You should not use python-path for setting up a virtual environment, use >>> python-home instead. >>> >>> WSGIProcessGroup xxx.com >>> >>> >>> You don't need WSGIProcessGroup if set process-group as option to >>> WSGIScriptAlias. See below. >>> >>> WSGIScriptAlias / /srv/django/xxx/xxx/xxx/wsgi.py >>> WSGIImportScript /srv/django/xxx/xxx/xxx/wsgi.py process-group= >>> xxx.com application-group=xxx.com >>> >>> >>> It is better to use: >>> >>> WSGIScriptAlias / /srv/django/xxx/xxx/xxx/wsgi.py process-group= >>> xxx.com application-group=%{GLOBAL} >>> >>> to force preloading. Remove the WSGIImportScript directive. >>> >>> <Directory /srv/django/xxx/xxx/> >>> WSGIApplicationGroup xxx.com >>> >>> >>> Since you only have the one application and are using daemon mode, you >>> should use: >>> >>> WSGIApplicationGroup %{GLOBAL} >>> >>> But then you don't need this directive at all if you are forcing >>> pre-loading and have supplied application-group option to WSGIScriptAlias. >>> See above. >>> >>> WSGIScriptReloading On >>> >>> >>> This is the default, you don't need this directive. >>> >>> Order allow,deny >>> Allow from all >>> </Directory> >>> >>> ... >>> >>> </VirtualHost> >>> >>> >>> Any help would be greatly appreciated and if anyone can point me in the >>> right direction as to how to debug this issue further I will gladly oblige >>> to the best of my ability! >>> >>> >>> Make those changes and see how it goes. >>> >>> Because you have been force pre-loading, the WSGI application should be >>> getting loaded when daemon processes start, not on first request. >>> >>> Graham >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/modwsgi. >> For more options, visit https://groups.google.com/d/optout. >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] <javascript:> > . > Visit this group at https://groups.google.com/group/modwsgi. > For more options, visit https://groups.google.com/d/optout. > > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
