Graham,
I had seen a post on issues with ITK MPM... So at least we found a "new"
issue and way to enhance the abilities of mod_wsgi. I certainly understand
you need some time to determine how you want to adjust things. Obviously I
could remove mod_ruid2 and change how my Apache deals with PHP, etc, but
honestly I would be more interested in seeing the options in mod_wsgi
because it just makes it even more flexible for people.
I am not sure what your development cycle is like and when new versions
come out as well as when this could even be brought into the code tree....
I would hope it isn't like 6+ months away. :)
In any event I guess I will try to watch for when the feature might be made
available and until then just keep running in normal embedded mode.
Thank you for the help and at least understanding that it wasn't me making
some stupid mistake in the config. :)
On Wednesday, 10 December 2014 21:33:53 UTC-8, Graham Dumpleton wrote:
>
> The mod_ruid2 module would indeed cause problems.
>
> It is similar to the perchild and ITK MPMs in that for each web request
> handled by Apache it switches the uid to a specific user away from the
> Apache user.
>
> RUidGid user1 group1
>
> So this definition would cause the request to be handled as the 'user1'
> user instead of nobody. That user then wouldn't have access to the UNIX
> socket in order to connect to the daemon process group.
>
> The code for mod_wsgi has:
>
> /*
> * Set the ownership of the UNIX listener socket. This would
> * normally be the Apache user that the Apache server child
> * processes run as, as they are the only processes that
> * would connect to the sockets. In the case of ITK MPM,
> * having them owned by Apache user is useless as at the
> * time the request is to be proxied, the Apache server
> * child process will have uid corresponding to the user
> * whose request they are handling. For ITK, thus set the
> * ownership to be the same as the daemon processes. This is
> * still restrictive, in that can only connect to daemon
> * process group running under same user, but most of the
> * time that is what you would want anyway when using ITK
> * MPM.
> */
>
> if (!geteuid()) {
> #if defined(MPM_ITK) || defined(ITK_MPM)
> if (chown(process->socket_path, process->uid, -1) < 0) {
> #else
> if (chown(process->socket_path, ap_unixd_config.user_id, -1) < 0) {
> #endif
> ap_log_error(APLOG_MARK, APLOG_ALERT, errno, wsgi_server,
> "mod_wsgi (pid=%d): Couldn't change owner of unix
> "
> "domain socket '%s'.", getpid(),
> process->socket_path);
> return -1;
> }
> }
>
> So it has special handling for the ITK MPM and will in this case set the
> ownership of the UNIX socket to be whatever is specified for the mod_wsgi
> daemon process group.
>
> That was as long as the mod_wsgi daemon process matches what the Apache
> child worker is switched to, it could connect.
>
> Now this is specific code for ITK MPM though and is a compile time thing.
> In order to get something to work for ITK MPM, it would be necessary for me
> to implement an option to WSGIDaemonProcess which allows one to override
> dynamically at configuration time what user the UNIX socket should be set
> to. Specifically allow one to set it to the user the mod_wsgi daemon
> process group runs as.
>
> If I do this then it would allow things to work with mod_ruid2.
>
> I will need to have a bit of a think about it and whether I have the
> option only allow it to be set to the daemon process group user, thus
> requiring the mod_ruid2 user to match, or whether I allow it to be set to
> any user. The latter may be more flexible, but I will have to think about
> the security implications of that.
>
> Graham
>
> On 11/12/2014, at 3:26 PM, Christiaan Stoudt <[email protected]
> <javascript:>> wrote:
>
> Graham,
>
> Thank you for the detailed post. Does going to WSGIDaemon really help
> that much on resources and other things? Or is it just a "nice to have"?
>
> Based on your steps and what I am seeing, it seems that the failure
> happens when that final request his apache and its passed off to the
> WSGIDaemon during that last connection.
>
> I had been running as LogLevel Debug, but was unaware of the
> WSGIVerboseDebugging. I enabled that and will provide the logs at the
> bottom of my message (I cleaned up and showed what seemed relevant instead
> of just posting everything.) So you are aware, the redirecting to
> /tmp/wsgi failed as well.
>
> I did open a support ticket to KnownHost to see if I could get any
> additional information to help. They really run a very basic config with
> nothing special. The tech did mention one thing that he thought could be
> an issue because was one of the mods installed in Apache --- *mod_ruid2*
> He said he has seen this conflict with Tomcat because of how it handles
> PHP and user requests. I am not very familiar with that mod. Have you had
> any experience with it and could it cause conflicts?
>
> Just for the record, here is all the mods I have installed:
>
> core_module (static) authn_file_module (static) authn_core_module
> (static) authz_host_module (static) authz_groupfile_module (static)
> authz_user_module (static) authz_core_module (static)
> access_compat_module (static) auth_basic_module (static)
> socache_shmcb_module (static) socache_dbm_module (static) so_module
> (static) include_module (static) filter_module (static)
> deflate_module (static) http_module (static) mime_module (static)
> log_config_module (static) logio_module (static) expires_module
> (static) headers_module (static) unique_id_module (static)
> setenvif_module (static) proxy_module (static) proxy_connect_module
> (static) proxy_http_module (static) slotmem_shm_module (static)
> ssl_module (static) mpm_prefork_module (static) unixd_module
> (static) status_module (static) autoindex_module (static)
> info_module (static) suexec_module (static) cgi_module (static)
> negotiation_module (static) dir_module (static) actions_module
> (static) userdir_module (static) alias_module (static)
> rewrite_module (static) bwlimited_module (shared) ruid2_module
> (shared) php5_module (shared) security2_module (shared)
> wsgi_module (shared)
>
> Here are the logs with what seemed relevant. If you want everything with
> a full log, I can certainly provide it... From what I can tell the
> WSGIDaemon process is starting up and waiting with all the threads and
> settings I have set. So it appears to be during that last hand off that it
> freaks out.....
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> [Wed Dec 10 19:56:10.002390 2014] [ssl:info] [pid 24734] AH01876:
> mod_ssl/2.4.10 compiled against Server: Apache/2.4.10, Library:
> OpenSSL/1.0.1e
> [Wed Dec 10 19:56:10.018768 2014] [suexec:notice] [pid 24734] AH01232:
> suEXEC mechanism enabled (wrapper: /usr/local/apache/bin/suexec)
> [Wed Dec 10 19:56:10.018878 2014] [:notice] [pid 24734] ModSecurity for
> Apache/2.8.0 (http://www.modsecurity.org/) configured.
> [Wed Dec 10 19:56:10.018900 2014] [:notice] [pid 24734] ModSecurity: APR
> compiled version="1.5.1"; loaded version="1.5.1"
> [Wed Dec 10 19:56:10.018915 2014] [:notice] [pid 24734] ModSecurity: PCRE
> compiled version="8.36 "; loaded version="8.36 2014-09-26"
> [Wed Dec 10 19:56:10.018929 2014] [:notice] [pid 24734] ModSecurity: LUA
> compiled version="Lua 5.1"
> [Wed Dec 10 19:56:10.018944 2014] [:notice] [pid 24734] ModSecurity:
> LIBXML compiled version="2.9.2"
> [Wed Dec 10 19:56:10.018957 2014] [:notice] [pid 24734] Status engine is
> currently disabled, enable it by set SecStatusEngine to On.
>
> Wed Dec 10 19:56:11.094046 2014] [:debug] [pid 24737] mod_wsgi.c(10488):
> mod_wsgi (pid=24737): Socket for 'mydomain_com' is
> '/wsgisock/wsgi.24737.0.1.sock'.
> [Wed Dec 10 19:56:11.094960 2014] [:info] [pid 24742] mod_wsgi
> (pid=24742): Starting process 'mydomain_com' with uid=99, gid=99 and
> threads=10.
>
> [Wed Dec 10 19:56:11.097400 2014] [mpm_prefork:notice] [pid 24737]
> AH00163: Apache/2.4.10 (Unix) OpenSSL/1.0.1e-fips mod_bwlimited/1.4
> PHP/5.4.34 mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal
> operations
> [Wed Dec 10 19:56:11.097463 2014] [:info] [pid 24746] mod_wsgi
> (pid=24746): Initializing Python.
> [Wed Dec 10 19:56:11.097501 2014] [mpm_prefork:debug] [pid 24737]
> prefork.c(995): AH00165: Accept mutex: sysvsem (default: sysvsem)
> [Wed Dec 10 19:56:11.108088 2014] [proxy:debug] [pid 24747]
> proxy_util.c(1771): AH00925: initializing worker proxy:reverse shared
> [Wed Dec 10 19:56:11.108152 2014] [proxy:debug] [pid 24747]
> proxy_util.c(1813): AH00927: initializing worker proxy:reverse local
> [Wed Dec 10 19:56:11.108216 2014] [proxy:debug] [pid 24747]
> proxy_util.c(1864): AH00931: initialized single connection worker in child
> 24747 for (*)
> [Wed Dec 10 19:56:11.108268 2014] [:debug] [pid 24742] mod_wsgi.c(11873):
> mod_wsgi (pid=24742): Process 'mydomain_com' logging to '
> clientsupportsystem.com'.
> [Wed Dec 10 19:56:11.108348 2014] [:info] [pid 24747] mod_wsgi
> (pid=24747): Initializing Python.
> [Wed Dec 10 19:56:11.108363 2014] [:info] [pid 24742] mod_wsgi
> (pid=24742): Attach interpreter ''.
> [Wed Dec 10 19:56:11.108804 2014] [:debug] [pid 24742] mod_wsgi.c(11253):
> mod_wsgi (pid=24742): Starting 10 threads in daemon process 'mydomain_com'.
> [Wed Dec 10 19:56:11.108812 2014] [:debug] [pid 24742] mod_wsgi.c(11057):
> mod_wsgi (pid=24742): Enable deadlock thread in process 'mydomain_com'.
> [Wed Dec 10 19:56:11.108810 2014] [:debug] [pid 24742] mod_wsgi.c(11089):
> mod_wsgi (pid=24742): Enable monitor thread in process 'mydomain_com'.
> [Wed Dec 10 19:56:11.108831 2014] [:debug] [pid 24742] mod_wsgi.c(11262):
> mod_wsgi (pid=24742): Starting thread 1 in daemon process 'mydomain_com'.
> [Wed Dec 10 19:56:11.108849 2014] [:debug] [pid 24742] mod_wsgi.c(11093):
> mod_wsgi (pid=24742): Deadlock timeout is 300.
> [Wed Dec 10 19:56:11.108869 2014] [:debug] [pid 24742] mod_wsgi.c(11096):
> mod_wsgi (pid=24742): Inactivity timeout is 300.
> [Wed Dec 10 19:56:11.108876 2014] [:debug] [pid 24742] mod_wsgi.c(11262):
> mod_wsgi (pid=24742): Starting thread 2 in daemon process 'mydomain_com'.
>
> [Wed Dec 10 19:58:40.871293 2014] [authz_core:debug] [pid 24743]
> mod_authz_core.c(802): [client 76.168.56.154:39988] AH01626:
> authorization result of Require all granted: granted
> [Wed Dec 10 19:58:40.871314 2014] [authz_core:debug] [pid 24743]
> mod_authz_core.c(802): [client 76.168.56.154:39988] AH01626:
> authorization result of <RequireAny>: granted
>
> [Wed Dec 10 19:58:40.871414 2014] [:error] [pid 24743] (13)Permission
> denied: [client 76.168.56.154:39988] mod_wsgi (pid=24743): Unable to
> connect to WSGI daemon process 'mydomain_com' on
> '/wsgisock/wsgi.24737.0.1.sock' after multiple attempts.
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Thank you again for looking at this... I appreciate the help so much!
>
> Regards,
> C
>
>
> On Wednesday, 10 December 2014 18:35:15 UTC-8, Graham Dumpleton wrote:
>>
>> For good measure, set:
>>
>> LogLevel debug
>> WSGIVerboseDebugging On
>>
>> in the Apache configuration. Start the server and make a single request
>> and capture the mod_wsgi logging or anything related and send it. Will see
>> if that shows anything.
>>
>> Also try:
>>
>> WSGISocketPrefix /tmp/wsgi
>>
>> It is not generally advisable to use /tmp, at least if a shared system,
>> but if your own system is not so bad.
>>
>> Other than looking at that, the only other thing can think of is that the
>> UNIX socket likely will not work in a NFS file system. Unless they are
>> doing something really strange with the type of VPS they run, then that
>> should not be the case though for a subdirectory of '/'.
>>
>> Anyway, as to the sequence of events.
>>
>> Apache would normally start as root so that it can bind port 80.
>>
>> In the Apache parent process, at the point it goes to create the mod_wsgi
>> daemon process group, it will first create as root the UNIX socket. This is
>> the one you are changing the location of with WSGISocketPrefix.
>>
>> That UNIX socket will have its ownership changed to the Apache user as
>> dictated by the Apache User directive. Would typically be www-data, _www or
>> nobody.
>>
>> The permissions of the socket when created will also be 0400 so that it
>> should only be accessible to root or the Apache user which is the owner of
>> the file.
>>
>> All the directories down to that UNIX socket will also though need to be
>> readable/searchable to the Apache user.
>>
>> When the daemon process group process are now forked off the Apache
>> parent process, they will listen for new connections on it.
>>
>> When a request is now received by Apache, they will be received by the
>> Apache child worker processes on port 80 (or other public HTTP/HTTPS port).
>> These will run as the Apache user. If the request is destined for the WSGI
>> application running in a daemon process group, the Apache child worker
>> process will create a connection to the UNIX socket. A mod_wsgi daemon
>> process will accept that connection and then handle the proxied request in
>> the separate daemon process.
>>
>> Now one final thing you could do as a separate sanity check is to try
>> using mod_wsgi-express on that system and see if that even works.
>>
>> https://pypi.python.org/pypi/mod_wsgi
>>
>> It defaults to using /tmp for generated configuration and the location of
>> the UNIX socket. Also therefore try running it with the option:
>>
>> --server-root /some/path
>>
>> to see if works in fails in other directories besides /tmp.
>>
>> Make sure you create that directory first if the parent directory can't
>> be written to by the user you run mod_wsgi-express as, and ensure the
>> directory is writable by the user you are running mod_wsgi-express as.
>>
>> Graham
>>
>> On 10/12/2014, at 11:30 PM, Christiaan Stoudt <[email protected]> wrote:
>>
>> Hi there...
>>
>> From what I can tell it is completely disabled. I do a "getenforce" and
>> it comes back as Disabled. I will open a ticket with KnownHost to see if
>> they have anything else that could block things but the hardest thing is I
>> don't know what file the error is talking about... because how can we have
>> a permission error with the wsgi.sock file when it is being created?????
>>
>> Honestly what might help is if you could quickly explain the series of
>> events that are happening. I could never find an article or anything that
>> actually explained the sequence of how mod_wsgi was loaded and executed.
>> Basically when the Daemon is called what is it doing... do it execute the
>> alias and wsgi.py in the public_html after it creates the wsgi.sock file or
>> before, etc? Or is apache doing it or what?
>>
>> I noticed that when I did the daemon as a user I could get a new apache
>> process running under that user but still it failed even if the wsgi.sock
>> and wsgi.py files were in a folder off of root that the user could read.
>>
>> Since I don't understand some of the basic steps the applications and
>> apache takes I can't even help check for a config issue myself or try other
>> things... so a basic step sequence would help...
>>
>> Thanks for the continued help!
>>
>> On Tuesday, 9 December 2014 20:59:28 UTC-8, Graham Dumpleton wrote:
>>>
>>> Can you double check that SELinux isn't causing an issue as the only
>>> thing that usually causes this is SELinux.
>>>
>>> Try temporarily disabling SELinux by following steps in:
>>>
>>> http://www.crypt.gen.nz/selinux/disable_selinux.html
>>>
>>> Graham
>>>
>>> On 10/12/2014, at 2:28 PM, Christiaan Stoudt <[email protected]> wrote:
>>>
>>> Hi there...
>>>
>>> I was trying to cover all the options in my first message that I tried
>>> and I guess I wasn't clear enough in one of the sentences! The
>>> /usr/local/apache/logs is root:root and I was aware of the permissions
>>> issue on the folder. So I also tried that by creating a directory straight
>>> off the root of the drive called wsgisock.
>>>
>>> drwxr-xr-x 2 nobody nobody 4096 Dec 9 15:15 wsgisock/
>>>
>>> Then I edited the WSGISocketPrefix like this--- WSGISocketPrefix
>>> /wsgisock/wsgi
>>> and I still had the issue even when I saw a wsgi.xxx.0.1.sock file being
>>> created in /wsgisock/
>>>
>>> In fact I just tried it again to make sure I wasn't being crazy. So
>>> technically the only think I did not do was change the permissions on
>>> /var/run/ to nobody:nobody because there are other programs using it for
>>> httpd, etc and I didn't want to mess them up with permission issues. So I
>>> figured creating a new directory off of root should basically provide the
>>> same solution.
>>>
>>> I guess explains why I am at a loss on what else I missed.
>>>
>>>
>>>
>>> On Tuesday, 9 December 2014 16:33:59 UTC-8, Graham Dumpleton wrote:
>>>>
>>>> It isn't the permissions on the socket file which can be the issue, the
>>>> directory that the socket file is in must be readable/searchable by the
>>>> 'nobody' user. Same applies to any directories all the way from '/' down
>>>> to
>>>> that directory.
>>>>
>>>> What do you get for:
>>>>
>>>> ls -las /usr/local/apache/logs
>>>>
>>>> Graham
>>>>
>>>> On 10/12/2014, at 11:06 AM, Christiaan Stoudt <[email protected]> wrote:
>>>>
>>>> Hello...
>>>>
>>>> I was able to get my Django site working with mod_wsgi and Apache after
>>>> learning all the configuration, etc. Thank you for the great
>>>> documentation
>>>> and help. So I want it clearly stated that I have mod_wsgi working.
>>>>
>>>> Unfortunately I am running out of RAM so I decided to switch over to a
>>>> WSGIDaemon configuration. This is where my problem is... I have hit a
>>>> dead
>>>> end and no matter how deep I search I continue to have no success. I hope
>>>> I can get some help here. Below are all the details I have in hopes to get
>>>> a response...
>>>>
>>>> Error: (13)Permission denied: [client xx.xx.xx.xx:xxxx] mod_wsgi
>>>> (pid=4570): Unable to connect to WSGI daemon process 'mydomain_com' on
>>>> '/usr/local/apache/logs/wsgi.4560.0.1.sock' after multiple attempts.
>>>>
>>>> Server details:
>>>> -- VPS Provider - KnownHost
>>>> -- OS Version - CentOS 6.6 (final)
>>>> -- Python 2.7.5
>>>> -- VirtualENV 1.11.6
>>>> -- Django 1.7.1
>>>> -- mod_wsgi 3.4
>>>> -- httpd -V
>>>> ---- Server version: Apache/2.4.10 (Unix)
>>>> ---- Architecture: 32-bit
>>>> ---- Server MPM: prefork / threaded: no / forked:
>>>> yes (variable process count)
>>>> -- Note: Apache runs as NOBODY for the chile processes
>>>> -- SELinux getenforce = Disabled
>>>>
>>>> -- In the pre.virtualhost.global.conf file I have these settings (this
>>>> gets merged with httpd.con):
>>>> LoadModule wsgi_module /usr/local/apache/extramodules/mod_wsgi.so
>>>> AddHandler wsgi-script .wsgi
>>>> WSGISocketPrefix /var/run/wsgi
>>>>
>>>> -- In the virtual host conf I have these settings:
>>>> WSGIDaemonProcess mydomain_com threads=10 inactivity-timeout=300
>>>> maximum-requests=2000 display-name=%{GROUP}
>>>> WSGIProcessGroup mydomain_com
>>>> WSGIScriptAlias / /home/mydomain/public_html/d171p275/mydomain
>>>> _com/wsgi.py
>>>>
>>>> -- In my wsgi.py file I have these settings:
>>>> import os, sys
>>>> sys.path.append('/home/mydomain/public_html/d171p275')
>>>> sys.path.append('/home/mydomain
>>>> /venv/d171p275/lib/python2.7/site-packages/')
>>>> os.environ["DJANGO_SETTINGS_MODULE"] = "mydomain_com.settings"
>>>> from django.core.wsgi import get_wsgi_application
>>>> application = get_wsgi_application()
>>>>
>>>> ~~~~~~~~~
>>>>
>>>> I see that the wsgi.xx.sock file was originally created in the
>>>> /etc/httpd/logs/ folder with nobody:root as permissions and a 0 size.
>>>> After putting in the WSGISocketPrefix setting, it moved to the /var/run
>>>> folder but the error persists. I have also tried to create a folder off
>>>> of
>>>> the / folder with permissions: nobody:nobody and I still get the error. I
>>>> have also tried to add the user and group entries in WSGIDaemonProcess for
>>>> both the "nobody" account as well as the "mydomain" account that the
>>>> virtual host domain was created on.
>>>>
>>>> Also I have moved the WSGI.PY file into various other folders (even the
>>>> same one the wsgi.xx.sock file sat in) to make sure the apache spawned
>>>> process could see it. It is not a SELinux or MPM issue. I don't have a
>>>> python-path in the WSGIDaemon process because it seems that WSGI is
>>>> finding
>>>> the wsgi.py file in the public_html folder for the domain just fine.
>>>>
>>>> Honestly I just have NO OTHER IDEAS!! Since I am still new to this I
>>>> wouldn't doubt it is something stupid. :) Any suggestions?
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "modwsgi" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To post to this group, send email to [email protected].
>>>> Visit this group at http://groups.google.com/group/modwsgi.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "modwsgi" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To post to this group, send email to [email protected].
>>> Visit this group at http://groups.google.com/group/modwsgi.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "modwsgi" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To post to this group, send email to [email protected].
>> Visit this group at http://groups.google.com/group/modwsgi.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected] <javascript:>.
> To post to this group, send email to [email protected] <javascript:>
> .
> Visit this group at http://groups.google.com/group/modwsgi.
> For more options, visit https://groups.google.com/d/optout.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
