[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 64a6824d129d by Senthil Kumaran in branch 'default': Fix Issue6085 - SimpleHTTPServer address_string to return client ip instead of client hostname http://hg.python.org/cpython/rev/64a6824d129d -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Senthil Kumaran sent...@uthcode.com added the comment: The original change was introduced in this issue401197 which seems to use fqdn at *all appropriate places*. In this case, after about a decade, it was realized that using fdqn for client connection may not be appropriate when hostname is not set. In 3.3, this is fixed by making address_string return client ip instead of client hostname. In 2.7 and 3.2, I would like to see it fixed wherein, inside the log_message call, the address_string is not called instead direct client address is used. In this manner, we are not changing any return values of public api (address_string), but we are effectively eliminating the delay cased by fqdn looking while logging to sys.stderr. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset fb1e71c7619a by Senthil Kumaran in branch '2.7': Fix issue6085 - Remove the delay caused by fqdn lookup while logging in BaseHTTPRequestHandler http://hg.python.org/cpython/rev/fb1e71c7619a New changeset 6adad27fcc8c by Senthil Kumaran in branch '3.2': Fix issue6085 - Remove the delay caused by fqdn lookup while logging in BaseHTTPRequestHandler http://hg.python.org/cpython/rev/6adad27fcc8c New changeset f9f11998a20d by Senthil Kumaran in branch 'default': issue6085 - update docs in default branch http://hg.python.org/cpython/rev/f9f11998a20d -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Senthil Kumaran sent...@uthcode.com added the comment: Fixed in all codelines. Closing this. -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Yuval Greenfield ubershme...@gmail.com added the comment: I agree with Antoine on this. Though the suggested patch is wrong. I believe we should leave address_string alone. Simply stop the log_message method from using it. Either way we'd be changing the log format but if we don't have to then we shouldn't completely change the meaning of a method while leaving its name intact. -- nosy: +ubershmekel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Charles-François Natali neolo...@free.fr added the comment: +1 to Antoine’s proposal of removal. Agreed. Here's a patch. Do note, however, that it's a behavior change: the address_string() method is documented to return a resolved hostname (when possible). -- keywords: +needs review stage: test needed - patch review Added file: http://bugs.python.org/file24637/server_resolve.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Changes by wmg wmg...@gmail.com: -- nosy: +wmgaca ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Éric Araujo mer...@netwok.org added the comment: +1 to Antoine’s proposal of removal. -- nosy: +eric.araujo versions: +Python 3.3 -Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Changes by Ralf Schmitt python-b...@systemexit.de: -- nosy: +schmir ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Changes by knickerkicker knicker.kic...@gmail.com: -- nosy: +knicker.kicker ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Changes by Charles-Francois Natali neolo...@free.fr: Removed file: http://bugs.python.org/file17081/base_http_server_fqdn_lag.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Antoine Pitrou pit...@free.fr added the comment: Actually, DNS resolution should be disabled by default, as in most HTTP servers probably. -- nosy: +pitrou stage: - unit test needed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Senthil Kumaran orsent...@gmail.com added the comment: And for this specific request, it fdqn is looked up only for logging to sys.stderr. Either removing the fqdn call or just caching per connection it as the patch does is both fine. I doubt if someone is relying this logging anywhere in the code. This is useful only when BaseHTTPServer is started in the command like as a standalone HTTP server. What shall we do for this? remove the fqdn or cache it once and use it. Either is fine. And there is not testing code present (or required too - as this is a cli invocation scenario). -- assignee: - orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Antoine Pitrou pit...@free.fr added the comment: And for this specific request, it fdqn is looked up only for logging to sys.stderr. Either removing the fqdn call or just caching per connection it as the patch does is both fine. I doubt if someone is relying this logging anywhere in the code. This is useful only when BaseHTTPServer is started in the command like as a standalone HTTP server. I think we should totally remove the call to fqdn. The common practice with HTTP servers is to log numeric IPs and, if desired, let a batch log analysis process (such as awstats) deal with DNS resolution and caching. And there is not testing code present (or required too - as this is a cli invocation scenario). What do you mean? BaseHTTPRequestHandler is tested in test_httpservers. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Senthil Kumaran orsent...@gmail.com added the comment: On Mon, Apr 26, 2010 at 10:45:56AM +, Antoine Pitrou wrote: What do you mean? BaseHTTPRequestHandler is tested in test_httpservers. I meant specifically for that function which is logging to sys.stderr. No return values, state changes and no tests present in test_httpservers. Seems to me the cli invokation scenario of the BaseHTTPServer module. Only other place where fqdn look up happens is HTTPServer.server_bind and in the setting of server_name, if any code is relying on it could break. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Santoso Wijaya santa@me.com added the comment: Please correct my understanding if I am mistaken, but from skimming through the source code, it seems that a new BaseHTTPRequestHandler is instantiated per request. If a caching mechanism is to be adopted as per the patch, wouldn't it be forgotten when a new request from the same client arrives in the future? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Charles-Francois Natali neolo...@free.fr added the comment: @santa4nt: You're correct, the cache is retained only from within the same handler: the other solution would be to keep the cache at the server level, but then you'd have to deal with the cache size (you don't want it to grow forever), and it seems a bit overkill. I just kept the FQDN resolution because the code seemed to do it on purpose, but I agree that if nothing relies on this behaviour, it might be as simple to remove the resolution altogether. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Charles-Francois Natali neolo...@free.fr added the comment: The attached patch caches the result of FQDN lookup. -- keywords: +patch Added file: http://bugs.python.org/file17079/base_http_server_fqdn_lag.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Santoso Wijaya santa@me.com added the comment: Doesn't that only cache the first remote client it encounters, though? Maybe a dictionary of caches? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Changes by Charles-Francois Natali neolo...@free.fr: Removed file: http://bugs.python.org/file17079/base_http_server_fqdn_lag.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Charles-Francois Natali neolo...@free.fr added the comment: Doesn't that only cache the first remote client it encounters, though? Maybe a dictionary of caches? A BaseHTTPRequestHandler is instantiated every time a client connects, so there should be only one client per handler, no (the cache is an attribute of the handler, not the server) ? -- Added file: http://bugs.python.org/file17081/base_http_server_fqdn_lag.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Charles-Francois Natali neolo...@free.fr added the comment: Yes, performing a DNS resolution every time we log something is definitely sub-optimal. We should perform it only once, and cache the result. -- nosy: +neologix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +orsenthil priority: - normal versions: +Python 3.2 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
Santoso Wijaya santoso.wij...@gmail.com added the comment: There is a simple workaround to bypass the hostname resolution mechanism, so that the server can respond faster. # This is a hack to patch slow socket.getfqdn calls that # BaseHTTPServer (and its subclasses) make. # See: http://bugs.python.org/issue6085 # See: http://www.answermysearches.com/xmlrpc-server-slow-in-python-how-to-fix/2140/ import BaseHTTPServer def _bare_address_string(self): host, port = self.client_address[:2] return '%s' % host BaseHTTPServer.BaseHTTPRequestHandler.address_string = \ _bare_address_string # End hack. -- nosy: +Santoso.Wijaya ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6085] Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag
New submission from Wentao Han wentao@gmail.com: The invocation of socket.getfqdn to get FQDN of the host may delay for several seconds on some platforms when no appropriate host name is found. This causes lag when using SimpleXMLRPCServer and other class derived from BaseHTTPServer for every request. -- components: Library (Lib) messages: 88188 nosy: aerodonkey severity: normal status: open title: Logging in BaseHTTPServer.BaseHTTPRequestHandler causes lag type: performance versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6085 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
