| Sorry for slow reply. Got distracted with other stuff. See the attached example code for a start. You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout. |
from __future__ import print_function import os import time import threading import atexit import socket
try:
import Queue as queue
except ImportError:
import queue
import mod_wsgi
class Monitor(object):
def __init__(self, interval=1.0):
self.monitor_thread = None
self.monitor_interval = interval
self.monitor_queue = queue.Queue()
self.last_metrics = None
def start(self):
if self.monitor_thread is None:
self.monitor_thread = threading.Thread(target=self.monitor)
self.monitor_thread.setDaemon(True)
self.monitor_thread.start()
atexit.register(self.stop)
def stop(self):
try:
self.monitor_queue.put(True)
except Exception:
pass
self.monitor_thread.join()
def monitor(self, *args):
while True:
current_metrics = mod_wsgi.process_metrics()
if self.last_metrics is not None:
duration = (current_metrics['current_time'] -
self.last_metrics['current_time'])
cpu_user_time = (current_metrics['cpu_user_time'] -
self.last_metrics['cpu_user_time'])
cpu_system_time = (current_metrics['cpu_system_time'] -
self.last_metrics['cpu_system_time'])
request_busy_time = (current_metrics['request_busy_time'] -
self.last_metrics['request_busy_time'])
request_threads = current_metrics['request_threads']
timestamp = current_metrics['current_time']
fields = {}
fields['cpu_user_time'] = cpu_user_time
fields['cpu_system_time'] = cpu_system_time
fields['cpu_usage'] = ((cpu_user_time+cpu_system_time) /
duration)
fields['request_busy_time'] = request_busy_time
fields['request_busy_usage'] = (request_busy_time /
(duration * mod_wsgi.threads_per_process))
fields['threads_per_process'] = mod_wsgi.threads_per_process
fields['request_threads'] = request_threads
print('METRICS', fields)
self.last_metrics = current_metrics
current_time = current_metrics['current_time']
delay = max(0, (current_time + self.monitor_interval) - time.time())
if delay == 0.0:
delay = self.monitor_interval
try:
return self.monitor_queue.get(timeout=delay)
except queue.Empty:
pass
monitor = Monitor(1.0)
monitor.start()
def application(environ, start_response):
status = '200 OK'
output = b'Hello World!'
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)
time.sleep(0.1)
return [output]
If you use that with mod_wsgi, for example as: mod_wsgi-express start-server example.py --log-to-terminal --threads 20 It will output each second for each process something like: METRICS {'cpu_system_time': 0.01000000536441803, 'request_busy_time': 3.3496430000000004, 'cpu_usage': 0.019944821691381887, 'request_busy_usage': 0.1670199913154585, 'request_threads': 16, 'cpu_user_time': 0.01000000536441803, 'threads_per_process': 20L} Key things to look at here are: threads_per_process - This says how many threads the process is configured to use. request_threads - This gives the number of threads in the process that mod_wsgi has activated and used at some point. Effectively it is what is needed at peak times when have lots of requests, or intersection of long running requests. request_busy_usage - This give the percentage capacity used in the last reporting period (default 1 second). So 1.0 is 100% of capacity. If this is always running at a low capacity utilisation, it indicates you have more threads per process than you need. What you might set it to is partly then guide by what request_threads reports, but remember that request_threads represents peak. It may be acceptable to still have less than request_threads if rarely hit the peak and a momentary back log is fine, or if have multiple processes to also handle and peak occurs due to restarts. cpu_usage - This gives percentage of CPU capacity (one core is 1.0) being used. This can technically go over 1.0 indicating using more than one core. Obviously due to the GIL that is hard and would depend on C code which is running without GIL being used. Hammer mod_wsgi and you can see it as mod_wsgi releases GIL for I/O. For normal site load I generally recommend not to let this go over 30-40%. If you are hitting 60-80% all the time, may be better to have more processes. If you can integrate this, adjusting 1 second reporting period to be longer if need be, report the results and we can look at them and explain anything or suggest things. Graham
You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout. |
