Only other thing can suggest if you know point at which it is hanging, is to test and extract C stack trace by attaching gdb to the running process. This may or may not work depending on your system setup and what tools are installed.
https://modwsgi.readthedocs.io/en/master/user-guides/debugging-techniques.html#debugging-crashes-with-gdb That may give some clues. Graham > On 8 Feb 2023, at 8:42 am, Graham Dumpleton <graham.dumple...@gmail.com> > wrote: > > Is any of your code using ctypes module for Apache, or any other module which > is used to call into C code by writing Python code only? > > Graham > >> On 8 Feb 2023, at 2:55 am, Carsten Fuchs <carsten.fu...@cafu.de> wrote: >> >> Hello, >> >> hmmm. There are no .so files there: >> >> $ pwd >> /home/carsten/.virtualenvs/HallCam-web/lib/python3.10/site-packages >> >> $ dir >> insgesamt 120 >> drwxrwxr-x 3 carsten carsten 4096 Feb 6 20:41 asgiref >> drwxrwxr-x 2 carsten carsten 4096 Feb 6 20:41 asgiref-3.6.0.dist-info >> drwxrwxr-x 6 carsten carsten 4096 Feb 6 20:41 dateutil >> drwxrwxr-x 3 carsten carsten 4096 Feb 6 20:40 _distutils_hack >> drwxrwxr-x 18 carsten carsten 4096 Feb 6 20:41 django >> drwxrwxr-x 2 carsten carsten 4096 Feb 6 20:41 Django-4.1.5.dist-info >> drwxrwxr-x 2 carsten carsten 4096 Feb 6 20:41 >> django_improved_user-2.0a2.dist-info >> drwxrwxr-x 2 carsten carsten 4096 Feb 6 20:41 >> django_widget_tweaks-1.4.12.dist-info >> drwxrwxr-x 4 carsten carsten 4096 Feb 6 20:41 improved_user >> drwxrwxr-x 5 carsten carsten 4096 Feb 6 20:40 pip >> drwxrwxr-x 2 carsten carsten 4096 Feb 6 20:40 pip-22.0.2.dist-info >> drwxrwxr-x 6 carsten carsten 4096 Feb 6 20:40 pkg_resources >> drwxrwxr-x 2 carsten carsten 4096 Feb 6 20:41 __pycache__ >> drwxrwxr-x 2 carsten carsten 4096 Feb 6 20:41 >> python_dateutil-2.8.2.dist-info >> drwxrwxr-x 7 carsten carsten 4096 Feb 6 20:40 setuptools >> drwxrwxr-x 2 carsten carsten 4096 Feb 6 20:40 setuptools-59.6.0.dist-info >> drwxrwxr-x 2 carsten carsten 4096 Feb 6 20:41 six-1.16.0.dist-info >> drwxrwxr-x 5 carsten carsten 4096 Feb 6 20:41 sqlparse >> drwxrwxr-x 2 carsten carsten 4096 Feb 6 20:41 sqlparse-0.4.3.dist-info >> drwxrwxr-x 4 carsten carsten 4096 Feb 6 20:41 widget_tweaks >> -rw-rw-r-- 1 carsten carsten 152 Feb 6 20:40 distutils-precedence.pth >> -rw-rw-r-- 1 carsten carsten 34549 Feb 6 20:41 six.py >> >> $ find -iname "*.so" >> >> The last command returned no output, but I additionally looked into the >> directories also in many other ways to make sure that I didn't run a wrong >> search. Hmmm. >> >> Best regards, >> Carsten >> >> >> Am 07.02.23 um 00:55 schrieb Graham Dumpleton: >>> Go into the lib/python3.10/site-packages directory of your Python virtual >>> environment. What .so files exist in that directory? These are the packages >>> with C extensions. It is only code in C extensions that should >>> normally be able to trigger a Python deadlock. >>> >>> Graham >>> >>>> On 7 Feb 2023, at 7:14 am, Carsten Fuchs <carsten.fu...@cafu.de> >>>> <mailto:carsten.fu...@cafu.de> wrote: >>>> >>>> Hello Graham, >>>> >>>> many thanks for your detailed reply! >>>> >>>> I must add that my Django project is a simple webcam application: A webcam >>>> sends POST requests to upload the images, the views show them. A single >>>> webcam is connected over a very slow internet connection (32 kBit/s), so a >>>> single POST can take several minutes. However, that never was and still >>>> doesn't seem to be a problem: Uploads still succeed, it's only the GET >>>> requests that fail. The GET requests in turn do more than just sending >>>> some HTML with the image URLs: especially they use the Pillow library to >>>> lazily produce thumbnails, put timestamps into the images, etc. >>>> >>>> Therefore I thought that Pillow (9.4.0) might be the culprit, so I removed >>>> the related code, made a new virtuelenv without Pillow from scratch and >>>> restarted Apache. Unfortunately, it didn't help – please see the error log >>>> below. The timeout occurs only after the browser has received a large >>>> portion, possibly all, of the view's response. >>>> >>>> `pip list` shows >>>> >>>> Package Version >>>> -------------------- ------- >>>> asgiref 3.6.0 >>>> Django 4.1.5 >>>> django-improved-user 2.0a2 >>>> django-widget-tweaks 1.4.12 >>>> pip 22.0.2 >>>> python-dateutil 2.8.2 >>>> setuptools 59.6.0 >>>> six 1.16.0 >>>> sqlparse 0.4.3 >>>> >>>> >>>> Is any of these know for causing problems like numpy? >>>> >>>> Best regards, >>>> Carsten >>>> >>>> The above mentioned error log: >>>> >>>> [Mon Feb 06 20:52:20.329545 2023] [mpm_event:notice] [pid 25844:tid >>>> 140714434877312] AH00489: Apache/2.4.52 (Ubuntu) >>>> mod_wsgi/4.9.0 Python/3.10 configured -- resuming normal operations >>>> [Mon Feb 06 20:52:20.330207 2023] [core:notice] [pid 25844:tid >>>> 140714434877312] AH00094: Command line: '/usr/sbin/apache2' >>>> [Mon Feb 06 20:52:20.380741 2023] [wsgi:info] [pid 25845:tid >>>> 140714434877312] mod_wsgi (pid=25845): Attach interpreter ''. >>>> [Mon Feb 06 20:52:20.395024 2023] [wsgi:info] [pid 25846:tid >>>> 140714434877312] mod_wsgi (pid=25846): Attach interpreter ''. >>>> [Mon Feb 06 20:52:20.395771 2023] [wsgi:info] [pid 25845:tid >>>> 140714434877312] mod_wsgi (pid=25845): Adding '/home/carsten/HallCam/web' >>>> to path. >>>> [Mon Feb 06 20:52:20.411711 2023] [wsgi:info] [pid 25846:tid >>>> 140714434877312] mod_wsgi (pid=25846): Adding '/home/carsten/HallCam/web' >>>> to path. >>>> [Mon Feb 06 20:52:48.621284 2023] [wsgi:info] [pid 25845:tid >>>> 140714369189440] mod_wsgi (pid=25845): Create interpreter >>>> '192.168.1.222:32228|'. >>>> [Mon Feb 06 20:52:48.634464 2023] [wsgi:info] [pid 25845:tid >>>> 140714369189440] mod_wsgi (pid=25845): Adding '/home/carsten/HallCam/web' >>>> to path. >>>> [Mon Feb 06 20:52:48.635248 2023] [wsgi:info] [pid 25845:tid >>>> 140714369189440] [remote 88.75.25.178:42162] mod_wsgi (pid=25845, >>>> process='cf-hallcam-site', application='192.168.1.222:32228|'): Loading >>>> Python script file '/home/carsten/HallCam/web/HallCam/wsgi.py'. >>>> [Mon Feb 06 20:57:40.661043 2023] [wsgi:info] [pid 25845:tid >>>> 140714419545664] mod_wsgi (pid=25845): Daemon process request time limit >>>> exceeded, stopping process 'cf-hallcam-site'. >>>> [Mon Feb 06 20:57:40.661184 2023] [wsgi:info] [pid 25845:tid >>>> 140714434877312] mod_wsgi (pid=25845): Shutdown requested >>>> 'cf-hallcam-site'. >>>> [Mon Feb 06 20:57:45.661421 2023] [wsgi:info] [pid 25845:tid >>>> 140714177934912] mod_wsgi (pid=25845): Aborting process 'cf-hallcam-site'. >>>> [Mon Feb 06 20:57:45.661464 2023] [wsgi:info] [pid 25845:tid >>>> 140714177934912] mod_wsgi (pid=25845): Exiting process 'cf-hallcam-site'. >>>> [Mon Feb 06 20:57:45.666987 2023] [wsgi:error] [pid 25847:tid >>>> 140714226689600] [client 37.81.109.237:49940] Truncated or oversized >>>> response headers received from daemon process 'cf-hallcam-site': >>>> /home/carsten/HallCam/web/HallCam/wsgi.py >>>> [Mon Feb 06 20:57:45.667005 2023] [wsgi:error] [pid 25848:tid >>>> 140714335794752] (104)Connection reset by peer: [client >>>> 88.75.25.178:42162] mod_wsgi (pid=25848): Failed to proxy response from >>>> daemon., referer: http://vdzuggmrroo5k7e9.myfritz.net:32228/ >>>> [Mon Feb 06 20:57:46.621804 2023] [wsgi:info] [pid 25977:tid >>>> 140714434877312] mod_wsgi (pid=25977): Attach interpreter ''. >>>> [Mon Feb 06 20:57:46.633564 2023] [wsgi:info] [pid 25977:tid >>>> 140714434877312] mod_wsgi (pid=25977): Adding '/home/carsten/HallCam/web' >>>> to path. >>>> [Mon Feb 06 21:00:01.811479 2023] [wsgi:info] [pid 25846:tid >>>> 140714360796736] mod_wsgi (pid=25846): Create interpreter >>>> '192.168.1.222:32228|'. >>>> [Mon Feb 06 21:00:01.824795 2023] [wsgi:info] [pid 25846:tid >>>> 140714360796736] mod_wsgi (pid=25846): Adding '/home/carsten/HallCam/web' >>>> to path. >>>> [Mon Feb 06 21:00:01.825535 2023] [wsgi:info] [pid 25846:tid >>>> 140714360796736] [remote 37.81.109.237:49942] mod_wsgi (pid=25846, >>>> process='cf-hallcam-site', application='192.168.1.222:32228|'): Loading >>>> Python script file '/home/carsten/HallCam/web/HallCam/wsgi.py'. >>>> [Mon Feb 06 21:00:02.506063 2023] [wsgi:info] [pid 25977:tid >>>> 140714402760256] mod_wsgi (pid=25977): Create interpreter >>>> '192.168.1.222:32228|'. >>>> [Mon Feb 06 21:00:02.519453 2023] [wsgi:info] [pid 25977:tid >>>> 140714402760256] mod_wsgi (pid=25977): Adding '/home/carsten/HallCam/web' >>>> to path. >>>> [Mon Feb 06 21:00:02.520239 2023] [wsgi:info] [pid 25977:tid >>>> 140714402760256] [remote 88.75.25.178:43650] mod_wsgi (pid=25977, >>>> process='cf-hallcam-site', application='192.168.1.222:32228|'): Loading >>>> Python script file '/home/carsten/HallCam/web/HallCam/wsgi.py'. >>>> [Mon Feb 06 21:00:03.299821 2023] [core:info] [pid 25847:tid >>>> 140714151155264] [client 88.75.25.178:43652] AH00128: File does not exist: >>>> /var/www/HallCam-media/thumbs/camera-2/pic_20230206_205532_4.jpg, referer: >>>> http://vdzuggmrroo5k7e9.myfritz.net:32228/ >>>> [Mon Feb 06 21:05:02.996169 2023] [wsgi:info] [pid 25977:tid >>>> 140714419545664] mod_wsgi (pid=25977): Daemon process deadlock timer >>>> expired, stopping process 'cf-hallcam-site'. >>>> [Mon Feb 06 21:05:02.996308 2023] [wsgi:info] [pid 25977:tid >>>> 140714434877312] mod_wsgi (pid=25977): Shutdown requested >>>> 'cf-hallcam-site'. >>>> [Mon Feb 06 21:05:07.996544 2023] [wsgi:info] [pid 25977:tid >>>> 140714177934912] mod_wsgi (pid=25977): Aborting process 'cf-hallcam-site'. >>>> [Mon Feb 06 21:05:07.996587 2023] [wsgi:info] [pid 25977:tid >>>> 140714177934912] mod_wsgi (pid=25977): Exiting process 'cf-hallcam-site'. >>>> [Mon Feb 06 21:05:08.002106 2023] [wsgi:error] [pid 25848:tid >>>> 140713647920704] [client 37.81.109.237:49944] Truncated or oversized >>>> response headers received from daemon process 'cf-hallcam-site': >>>> /home/carsten/HallCam/web/HallCam/wsgi.py >>>> [Mon Feb 06 21:05:08.002137 2023] [wsgi:error] [pid 25848:tid >>>> 140714025395776] (104)Connection reset by peer: [client >>>> 88.75.25.178:43650] mod_wsgi (pid=25848): Failed to proxy response from >>>> daemon., referer: http://vdzuggmrroo5k7e9.myfritz.net:32228/ >>>> [Mon Feb 06 21:05:09.015715 2023] [wsgi:info] [pid 26028:tid >>>> 140714434877312] mod_wsgi (pid=26028): Attach interpreter ''. >>>> [Mon Feb 06 21:05:09.027591 2023] [wsgi:info] [pid 26028:tid >>>> 140714434877312] mod_wsgi (pid=26028): Adding '/home/carsten/HallCam/web' >>>> to path. >>>> [Mon Feb 06 21:05:29.082057 2023] [wsgi:info] [pid 25846:tid >>>> 140714419545664] mod_wsgi (pid=25846): Daemon process deadlock timer >>>> expired, stopping process 'cf-hallcam-site'. >>>> [Mon Feb 06 21:05:29.082193 2023] [wsgi:info] [pid 25846:tid >>>> 140714434877312] mod_wsgi (pid=25846): Shutdown requested >>>> 'cf-hallcam-site'. >>>> [Mon Feb 06 21:05:34.082447 2023] [wsgi:info] [pid 25846:tid >>>> 140714177934912] mod_wsgi (pid=25846): Aborting process 'cf-hallcam-site'. >>>> [Mon Feb 06 21:05:34.082493 2023] [wsgi:info] [pid 25846:tid >>>> 140714177934912] mod_wsgi (pid=25846): Exiting process 'cf-hallcam-site'. >>>> [Mon Feb 06 21:05:35.038614 2023] [wsgi:info] [pid 26047:tid >>>> 140714434877312] mod_wsgi (pid=26047): Attach interpreter ''. >>>> [Mon Feb 06 21:05:35.050453 2023] [wsgi:info] [pid 26047:tid >>>> 140714434877312] mod_wsgi (pid=26047): Adding '/home/carsten/HallCam/web' >>>> to path. >>>> >>>> >>>> >>>> >>>> >>>> Am 05.02.23 um 23:32 schrieb Graham Dumpleton: >>>>> Two points here to clarify. >>>>> >>>>> In your case the final error is: >>>>> >>>>> Daemon process deadlock timer expired >>>>> >>>>> This means that the full Python interpreter locked up. In this case the >>>>> request timeout may not apply and the feature where by stack traces can >>>>> be dumped might not happen. Depends on what lead up to the issue. >>>>> >>>>> By default the deadlock timeout is 300 seconds. >>>>> >>>>> deadlock-timeout=sss >>>>> Defines the maximum number of seconds allowed to pass before the daemon >>>>> process is shutdown and restarted after a potential deadlock on the >>>>> Python GIL has been detected. The default is 300 seconds. >>>>> >>>>> This option exists to combat the problem of a daemon process freezing as >>>>> the result of a rogue Python C extension module which doesn’t properly >>>>> release the Python GIL when entering into a blocking or long running >>>>> operation. >>>>> >>>>> This can occur when you are using third party Python packages which >>>>> aren't designed to work in Python sub interpreters. More details in: >>>>> >>>>> https://modwsgi.readthedocs.io/en/master/user-guides/application-issues.html#python-simplified-gil-state-api >>>>> The solution as you found is to set: >>>>> >>>>> WSGIApplicationGroup %{GLOBAL} >>>>> >>>>> So likely when you updated Python versions, some third party package you >>>>> are using has shifted to a newer version which breaks in this respect, or >>>>> which runs code different when using newer Python version. >>>>> >>>>> As to request timeout if it were coming into play, the timeout is 30 >>>>> seconds when you have daemon processes which are single threaded. >>>>> >>>>> For multithreaded daemon process the timeout can trigger after 30 seconds >>>>> as uses an average across all active request handler threads. >>>>> >>>>> request-timeout=sss >>>>> Defines the maximum number of seconds that a request is allowed to run >>>>> before the daemon process is restarted. This can be used to recover from >>>>> a scenario where a request blocks indefinitely, and where if all request >>>>> threads were consumed in this way, would result in the whole WSGI >>>>> application process being blocked. >>>>> >>>>> How this option is seen to behave is different depending on whether a >>>>> daemon process uses only one thread, or more than one thread for handling >>>>> requests, as set by the threads option. >>>>> >>>>> If there is only a single thread, and so the process can only handle one >>>>> request at a time, as soon as the timeout has passed, a restart of the >>>>> process will be initiated. >>>>> >>>>> If there is more than one thread, the request timeout is applied to the >>>>> average running time for any requests, across all threads. This means >>>>> that a request can run longer than the request timeout. This is done to >>>>> reduce the possibility of interupting other running requests, and causing >>>>> a user to see a failure. So where there is still capacity to handle more >>>>> requests, restarting of the process will be >>>>> delayed if possible. >>>>> >>>>> >>>>> In your case though because hitting a full lock up of the Python >>>>> interpreter, even though the request timeout may still have triggered, it >>>>> couldn't dump stack traces as doing so requires getting the global Python >>>>> interpreter lock, which wasn't being released and thus threads to dump >>>>> stack traces got stuck. >>>>> >>>>> To understand the cause further and identify which third party package >>>>> may be the problem, I would need to see what extra Python packages you >>>>> are installing. Note that numpy can cause this, so if using anything >>>>> which relies on that, that could be the reason. >>>>> >>>>> Graham >>>>> >>>>>> On 6 Feb 2023, at 3:35 am, Carsten Fuchs <carsten.fu...@cafu.de> >>>>>> <mailto:carsten.fu...@cafu.de> wrote: >>>>>> >>>>>> Adding to my recent post: >>>>>> >>>>>> It seems that the `request-timeout=30` is effective, after all: The >>>>>> error messages have changed, but there is no stack trace. >>>>>> For completeness, here is the site config and error log excerpt: >>>>>> >>>>>> Site config: >>>>>> >>>>>> <VirtualHost *:80> >>>>>> ServerAdmin webmaster@localhost >>>>>> DocumentRoot /var/www/html >>>>>> >>>>>> # Available loglevels: trace8, ..., trace1, debug, info, notice, >>>>>> warn, >>>>>> # error, crit, alert, emerg. >>>>>> # It is also possible to configure the loglevel for particular >>>>>> # modules, e.g. >>>>>> #LogLevel info ssl:warn >>>>>> LogLevel info >>>>>> >>>>>> ErrorLog ${APACHE_LOG_DIR}/error.log >>>>>> CustomLog ${APACHE_LOG_DIR}/access.log combined >>>>>> >>>>>> # Siehe https://bz.apache.org/bugzilla/show_bug.cgi?id=45023 >>>>>> # Hierfür muss mod_headers aktiv sein: sudo a2enmod headers >>>>>> RequestHeader edit "If-None-Match" '^"((.*)-gzip)"$' '"$1", "$2"' >>>>>> >>>>>> Alias /static/ /var/www/HallCam-static/ >>>>>> <Directory /var/www/HallCam-static> >>>>>> Require all granted >>>>>> </Directory> >>>>>> >>>>>> Alias /media/ /var/www/HallCam-media/ >>>>>> <Directory /var/www/HallCam-media> >>>>>> Require all granted >>>>>> </Directory> >>>>>> >>>>>> WSGIDaemonProcess cf-hallcam-site request-timeout=30 user=carsten >>>>>> group=carsten processes=2 display-name=%{GROUP} >>>>>> python-home=/home/carsten/.virtualenvs/HallCam-web >>>>>> python-path=/home/carsten/HallCam/web >>>>>> WSGIProcessGroup cf-hallcam-site >>>>>> # WSGIApplicationGroup %{GLOBAL} >>>>>> >>>>>> WSGIScriptAlias / /home/carsten/HallCam/web/HallCam/wsgi.py >>>>>> <Directory /home/carsten/HallCam/web/HallCam> >>>>>> <Files wsgi.py> >>>>>> Require all granted >>>>>> </Files> >>>>>> </Directory> >>>>>> </VirtualHost> >>>>>> >>>>>> >>>>>> From `/var/log/apache2/error.log`: >>>>>> >>>>>> [Sun Feb 05 17:18:55.470532 2023] [mpm_event:notice] [pid 635:tid >>>>>> 140277991982976] AH00489: Apache/2.4.52 (Ubuntu) mod_wsgi/4.9.0 >>>>>> Python/3.10 configured -- resuming normal operations >>>>>> [Sun Feb 05 17:18:55.690430 2023] [core:notice] [pid 635:tid >>>>>> 140277991982976] AH00094: Command line: '/usr/sbin/apache2' >>>>>> [Sun Feb 05 17:18:56.338443 2023] [wsgi:info] [pid 638:tid >>>>>> 140277991982976] mod_wsgi (pid=638): Attach interpreter ''. >>>>>> [Sun Feb 05 17:18:56.342866 2023] [wsgi:info] [pid 636:tid >>>>>> 140277991982976] mod_wsgi (pid=636): Attach interpreter ''. >>>>>> [Sun Feb 05 17:18:56.366079 2023] [wsgi:info] [pid 636:tid >>>>>> 140277991982976] mod_wsgi (pid=636): Adding '/home/carsten/HallCam/web' >>>>>> to path. >>>>>> [Sun Feb 05 17:18:56.371777 2023] [wsgi:info] [pid 638:tid >>>>>> 140277991982976] mod_wsgi (pid=638): Adding '/home/carsten/HallCam/web' >>>>>> to path. >>>>>> [Sun Feb 05 17:19:24.823860 2023] [wsgi:info] [pid 636:tid >>>>>> 140277959865920] mod_wsgi (pid=636): Create interpreter >>>>>> '192.168.1.222:32228|'. >>>>>> [Sun Feb 05 17:19:24.837012 2023] [wsgi:info] [pid 636:tid >>>>>> 140277959865920] mod_wsgi (pid=636): Adding '/home/carsten/HallCam/web' >>>>>> to path. >>>>>> [Sun Feb 05 17:19:24.837752 2023] [wsgi:info] [pid 636:tid >>>>>> 140277959865920] [remote 88.75.25.178:50558] mod_wsgi (pid=636, >>>>>> process='cf-hallcam-site', application='192.168.1.222:32228|'): Loading >>>>>> Python script file '/home/carsten/HallCam/web/HallCam/wsgi.py'. >>>>>> [Sun Feb 05 17:19:43.356895 2023] [wsgi:info] [pid 638:tid >>>>>> 140277867546176] mod_wsgi (pid=638): Create interpreter >>>>>> '192.168.1.222:32228|'. >>>>>> [Sun Feb 05 17:19:43.369787 2023] [wsgi:info] [pid 638:tid >>>>>> 140277867546176] mod_wsgi (pid=638): Adding '/home/carsten/HallCam/web' >>>>>> to path. >>>>>> [Sun Feb 05 17:19:43.370532 2023] [wsgi:info] [pid 638:tid >>>>>> 140277867546176] [remote 88.75.25.178:48210] mod_wsgi (pid=638, >>>>>> process='cf-hallcam-site', application='192.168.1.222:32228|'): Loading >>>>>> Python script file '/home/carsten/HallCam/web/HallCam/wsgi.py'. >>>>>> [Sun Feb 05 17:23:21.631286 2023] [wsgi:info] [pid 636:tid >>>>>> 140277976651328] mod_wsgi (pid=636): Daemon process request time limit >>>>>> exceeded, stopping process 'cf-hallcam-site'. >>>>>> [Sun Feb 05 17:23:21.631405 2023] [wsgi:info] [pid 636:tid >>>>>> 140277991982976] mod_wsgi (pid=636): Shutdown requested >>>>>> 'cf-hallcam-site'. >>>>>> [Sun Feb 05 17:23:26.631650 2023] [wsgi:info] [pid 636:tid >>>>>> 140277591307840] mod_wsgi (pid=636): Aborting process 'cf-hallcam-site'. >>>>>> [Sun Feb 05 17:23:26.631699 2023] [wsgi:info] [pid 636:tid >>>>>> 140277591307840] mod_wsgi (pid=636): Exiting process 'cf-hallcam-site'. >>>>>> [Sun Feb 05 17:23:26.714616 2023] [wsgi:error] [pid 639:tid >>>>>> 140277725460032] [client 88.75.25.178:48224] Truncated or oversized >>>>>> response headers received from daemon process 'cf-hallcam-site': >>>>>> /home/carsten/HallCam/web/HallCam/wsgi.py, referer: >>>>>> http://vdzuggmrroo5k7e9.myfritz.net:32228/upload/ >>>>>> [Sun Feb 05 17:23:26.714952 2023] [wsgi:error] [pid 639:tid >>>>>> 140277959865920] (104)Connection reset by peer: [client >>>>>> 88.75.25.178:50558] mod_wsgi (pid=639): Failed to proxy response from >>>>>> daemon., referer: http://vdzuggmrroo5k7e9.myfritz.net:32228/upload/ >>>>>> [Sun Feb 05 17:23:26.983714 2023] [wsgi:info] [pid 885:tid >>>>>> 140277991982976] mod_wsgi (pid=885): Attach interpreter ''. >>>>>> [Sun Feb 05 17:23:26.994980 2023] [wsgi:info] [pid 885:tid >>>>>> 140277991982976] mod_wsgi (pid=885): Adding '/home/carsten/HallCam/web' >>>>>> to path. >>>>>> [Sun Feb 05 17:23:37.641476 2023] [wsgi:info] [pid 638:tid >>>>>> 140277976651328] mod_wsgi (pid=638): Daemon process request time limit >>>>>> exceeded, stopping process 'cf-hallcam-site'. >>>>>> [Sun Feb 05 17:23:37.641609 2023] [wsgi:info] [pid 638:tid >>>>>> 140277991982976] mod_wsgi (pid=638): Shutdown requested >>>>>> 'cf-hallcam-site'. >>>>>> [Sun Feb 05 17:23:42.641863 2023] [wsgi:info] [pid 638:tid >>>>>> 140277591307840] mod_wsgi (pid=638): Aborting process 'cf-hallcam-site'. >>>>>> [Sun Feb 05 17:23:42.641910 2023] [wsgi:info] [pid 638:tid >>>>>> 140277591307840] mod_wsgi (pid=638): Exiting process 'cf-hallcam-site'. >>>>>> [Sun Feb 05 17:23:42.648248 2023] [wsgi:error] [pid 639:tid >>>>>> 140277717067328] [client 37.81.109.237:49336] Truncated or oversized >>>>>> response headers received from daemon process 'cf-hallcam-site': >>>>>> /home/carsten/HallCam/web/HallCam/wsgi.py >>>>>> [Sun Feb 05 17:23:42.648624 2023] [wsgi:error] [pid 639:tid >>>>>> 140277733852736] (104)Connection reset by peer: [client >>>>>> 88.75.25.178:48210] mod_wsgi (pid=639): Failed to proxy response from >>>>>> daemon., referer: http://vdzuggmrroo5k7e9.myfritz.net:32228/ >>>>>> [Sun Feb 05 17:23:42.997522 2023] [wsgi:info] [pid 906:tid >>>>>> 140277991982976] mod_wsgi (pid=906): Attach interpreter ''. >>>>>> [Sun Feb 05 17:23:43.039478 2023] [wsgi:info] [pid 906:tid >>>>>> 140277991982976] mod_wsgi (pid=906): Adding '/home/carsten/HallCam/web' >>>>>> to path. >>>>>> [Sun Feb 05 17:25:01.761551 2023] [wsgi:info] [pid 885:tid >>>>>> 140277959865920] mod_wsgi (pid=885): Create interpreter >>>>>> '192.168.1.222:32228|'. >>>>>> [Sun Feb 05 17:25:01.774488 2023] [wsgi:info] [pid 885:tid >>>>>> 140277959865920] mod_wsgi (pid=885): Adding '/home/carsten/HallCam/web' >>>>>> to path. >>>>>> [Sun Feb 05 17:25:01.775225 2023] [wsgi:info] [pid 885:tid >>>>>> 140277959865920] [remote 37.81.109.237:49338] mod_wsgi (pid=885, >>>>>> process='cf-hallcam-site', application='192.168.1.222:32228|'): Loading >>>>>> Python script file '/home/carsten/HallCam/web/HallCam/wsgi.py'. >>>>>> [Sun Feb 05 17:25:03.155261 2023] [wsgi:error] [pid 885:tid >>>>>> 140277959865920] [remote 37.81.109.237:49338] Picture saved to >>>>>> /var/www/HallCam-media/pictures/camera-1/pic_20230205_172500_4.jpg >>>>>> (image/jpeg, 896262 bytes, camera camera-1) >>>>>> [Sun Feb 05 17:30:03.323110 2023] [wsgi:info] [pid 885:tid >>>>>> 140277976651328] mod_wsgi (pid=885): Daemon process deadlock timer >>>>>> expired, stopping process 'cf-hallcam-site'. >>>>>> [Sun Feb 05 17:30:03.323253 2023] [wsgi:info] [pid 885:tid >>>>>> 140277991982976] mod_wsgi (pid=885): Shutdown requested >>>>>> 'cf-hallcam-site'. >>>>>> >>>>>> >>>>>> Carsten Fuchs schrieb am Sonntag, 5. Februar 2023 um 17:23:29 UTC+1: >>>>>>> Dear Graham, >>>>>>> >>>>>>> I experienced the same timeout errors after having upgraded a server >>>>>>> from Ubuntu 20.04 LTS to Ubuntu 22.04 LTS, thereby also upgrading from >>>>>>> Python 3.8 to Python 3.10. The application is a relatively simple >>>>>>> Django project that used to work well until the upgrade. After the >>>>>>> upgrade, I deleted the old virtualenv and built a new one, using `pip >>>>>>> install --no-cache-dir -r requirements.txt` to install it. However, I >>>>>>> experience the same problem. >>>>>>> >>>>>>> Adding `WSGIApplicationGroup %{GLOBAL}` solved the problem, but I am >>>>>>> still concerned because the site worked well with the older Ubuntu >>>>>>> 20.04 LTS and I would prefer to not mask a potential problem and rather >>>>>>> find its root. >>>>>>> >>>>>>> Therefore, I added the `request-timeout=30` option to >>>>>>> `WSGIDaemonProcess` (and temporarily commented `WSGIApplicationGroup` >>>>>>> out again) in order to get a stack trace, however it doesn't seem to >>>>>>> have any effect: Requests time out only much later than 30 seconds. >>>>>>> >>>>>>> Can you please advise what may have caused the problem when upgrading >>>>>>> from Ubuntu 20.04 LTS to Ubuntu 22.04 LTS and why `request-timeout=30` >>>>>>> may not have any effect? >>>>>>> >>>>>>> Best regards, >>>>>>> Carsten >>>>>>> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to modwsgi+unsubscr...@googlegroups.com >> <mailto:modwsgi+unsubscr...@googlegroups.com>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/modwsgi/bb5dc1b7-b183-58d5-195d-716dd96fbedb%40cafu.de >> >> <https://groups.google.com/d/msgid/modwsgi/bb5dc1b7-b183-58d5-195d-716dd96fbedb%40cafu.de?utm_medium=email&utm_source=footer>. > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/054D62E3-BF67-4DC9-BAD1-DCB51AE88564%40gmail.com.
