Bugs item #1685000, was opened at 2007-03-21 02:15
Message generated for change (Comment added) made by josiahcarlson
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Wont Fix
>Priority: 5
Private: No
Submitted By: billiejoex (billiejoex)
Assigned to: Josiah Carlson (josiahcarlson)
Summary: asyncore DoS vulnerability
Initial Comment:
DoS asyncore vulnerability
asyncore, independently if used with select() or poll(), suffers a DoS-type
vulnerability when a high number of simultaneous connections to handle
simultaneously is reached.
The number of maximum connections is system-dependent as well as the type of
error raised.
I attached two simple Proof of Concept scripts demonstrating such bug.
If you want to try the behaviours listed below run the attached
"asyncore_server.py" and "asyncore_client.py" scripts on your local workstation.
On my Windows XP system (Python 2.5), independently if asyncore has been used
to develop a server or a client, the error is raised by select() inside
asyncore's "poll" function when 512 (socket_map's elements) simultaneous
connections are reached.
Here's the traceback I get:
[...]
connections: 510
connections: 511
connections: 512
Traceback (most recent call last):
File "C:\scripts\asyncore_server.py", line 38, in <module>
asyncore.loop()
File "C:\Python25\lib\asyncore.py", line 191, in loop
poll_fun(timeout, map)
File "C:\Python25\lib\asyncore.py", line 121, in poll
r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()
On my Linux Ubuntu 6.10 (kernel 2.6.17-10, Python 2.5) different type of errors
are raised depending on the application (client or server).
In an asyncore-based client the error is raised by socket module (dispatcher's
"self.socket" attribute) inside 'connect' method of 'dispatcher' class:
[...]
connections: 1018
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
File "asyncore_client.py", line 31, in <module>
File "asyncore.py", line 191, in loop
File "asyncore.py", line 138, in poll
File "asyncore.py", line 80, in write
File "asyncore.py", line 76, in write
File "asyncore.py", line 395, in handle_write_event
File "asyncore_client.py", line 24, in handle_connect
File "asyncore_client.py", line 9, in __init__
File "asyncore.py", line 257, in create_socket
File "socket.py", line 156, in __init__
socket.error: (24, 'Too many open files')
On an asyncore-based server the error is raised by socket module (dispatcher's
"self.socket" attribute) inside 'accept' method of 'dispatcher' class:
[...]
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
File "asyncore_server.py", line 38, in <module>
File "asyncore.py", line 191, in loop
File "asyncore.py", line 132, in poll
File "asyncore.py", line 72, in read
File "asyncore.py", line 68, in read
File "asyncore.py", line 384, in handle_read_event
File "asyncore_server.py", line 16, in handle_accept
File "asyncore.py", line 321, in accept
File "socket.py", line 170, in accept
socket.error: (24, 'Too many open files')
----------------------------------------------------------------------
>Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-13 12:36
Message:
Logged In: YES
user_id=341410
Originator: NO
The OP and I discussed this via email and IM.
There seems to be a few issues that the OP is concerned about. The first
is that the number of allowable sockets/process is platform dependent
(Windows has no limit, linux can be set manually). The second is that some
platforms limit the number of sockets that can be passed to select at a
time (Windows limits this to 512, I don't know about linux). The third is
that the OP wants a solution to handling both a standard denial of service
attack (single client), as well as a distributed denial of service attack
(many clients).
The first issue is annoying, bit it is not within the realm of problems
that should be dealt with by base asyncore. Just like platform floating
point handling is platform-dependent, sockets/file handles per process is
also platform-dependent, and trying to abstract it away is not reasonable.
The second issue is also annoying, but it also isn't within the realm of
problems that should be dealt with by base asyncore. Whether or not an
application should be able to handle more than a few hundred sockets at a
time is dependent on the application, and modifying asyncore to make
assumptions about whether an application should or should not handle that
many sockets is not reasonable.
The third issue is also not reasonable for us to handle. How to respond
to many incoming connections (from a single source, or from many source) is
also application dependent. On a web server, maybe you just don't accept
connections when you are overloaded. Maybe if you follow the advice of
http://www.remote.org/jochen/work/pub/zero-downtime.pdf , you handle them
all. These are all application-specific.
Now, because how to handle the cases are all platform, application,
protocol, etc., dependent, assigning a single set of rules for asyncore to
respond to conditions of high numbers of sockets is outside the range or
problems that asyncore is intended to solve. I would also point out that
Twisted doesn't do anything special to handle these cases. When a Twisted
select reactor on linux comes to the file handle limit, it dies, just like
asyncore. When a Twisted poll reactor on linux comes to the file handle
limit, it dies. On Windows, Twisted uses the Windows messaging API
"WSAEventSelect and MsgWaitForMultipleObjects, or something"
(http://twistedmatrix.com/pipermail/twisted-python/2006-April/012976.html),
but having written a poll method for Windows using this myself, I find that
select uses much less processor (I can provide you with an example).
With all of that said, since a higher-level asynchronous sockets framework
doesn't handle the case that you want asyncore to handle, I can't help but
say, once again, "asyncore wasn't intended to handle the situation that you
describe, and how *you* handle it, as an application developer, is up to
you."
Closing as won't fix.
----------------------------------------------------------------------
Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-09 09:13
Message:
Logged In: YES
user_id=341410
Originator: NO
Assign the "bug" to me, I'm the maintainer for asyncore/asynchat.
With that said, since a user needs to override
asyncore.dispatcher.handle_accept() anyways, which necessarily needs to
call asyncore.dispatcher.accept(), the subclass is free to check the number
of sockets in its socket map before creating a new instance of whatever
subclass of asyncore.dispatcher the user has written.
Also, the number of file handles that select can handle on Windows is a
compile-time constant, and has nothing to do with the actual number of open
file handles. Take and run the following source file on Windows and see
how the total number of open sockets can be significantly larger than the
number of sockets passed to select():
import socket
import asyncore
import random
class new_map(dict):
def items(self):
r = [(i,j) for i,j in dict.items(self) if not random.randrange(4)
and j != h]
r.append((h._fileno, h))
print len(r), len(asyncore.socket_map)
return r
asyncore.socket_map = new_map()
class listener(asyncore.dispatcher):
def handle_accept(self):
x = self.accept()
if x:
conn, addr = x
connection(conn)
class connection(asyncore.dispatcher):
def writable(self):
0
def handle_connect(self):
pass
if __name__ == '__main__':
h = listener()
h.create_socket(socket.AF_INET, socket.SOCK_STREAM)
h.bind(('127.0.0.1', 10001))
h.listen(5)
while len(asyncore.socket_map) < 4096:
a = connection()
a.create_socket(socket.AF_INET, socket.SOCK_STREAM)
a.connect_ex(('127.0.0.1', 10001))
asyncore.poll()
The tail end of a run in Windows:
476 1934
501 1936
516 1938
Traceback (most recent call last):
File "D:\MYDOCS\Projects\python\async_socket.py", line 37, in ?
asyncore.poll()
File "C:\python23\lib\asyncore.py", line 108, in poll
r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()
With a proper definition of new_map, you can handle an unlimited number of
sockets with asyncore by choosing blocks of sockets to return in its
items() call. Note that I used random out of convenience, a proper
implementation could distribute the sockets based on fileno, time inserted,
etc. You can do this on linux, but only if you have used ulimit, as sgala
stated.
I would also mention that a better approach is to create an access time
mapping with blacklisting support to accept/close sockets that have
hammered your server. It would also make sense to handle socket timeouts
(no read/write on a socket for X seconds).
Regardless, none of these things are within the world of problems that
base asyncore is intended to solve. It's a bare-bones async socket
implementation. If you want more features, use twisted or write your own
subclasses and offer it up on the Python Cookbook (activestate.com) or in
the Python Package Index (python.org/pypi). If the community finds that
they are useful and productive features, we may consider it for inclusion
in the standard library.
Until then, suggested close. Will close as "will not fix" if it is
assigned to me.
----------------------------------------------------------------------
Comment By: billiejoex (billiejoex)
Date: 2007-04-06 05:10
Message:
Logged In: YES
user_id=1357589
Originator: YES
> Putting a try/except in place doesn't really help the problem...
> if you fail to create a new socket what action will you take?
Calling a new dispatcher's method (for example:
"handle_exceeded_connection") could be an idea. By default it could close
the current session but it can be overriden if the user want to take some
other action.
> A better approach is to have a configurable limit on the number of
> open connections, and then have a server-specific reaction to
> exceeding that limit.
It doesn't work if I create additional file-objects during the execution
of the loop...
----------------------------------------------------------------------
Comment By: Santiago Gala (sgala)
Date: 2007-04-05 16:09
Message:
Logged In: YES
user_id=178886
Originator: NO
THe limit of resources that an OS can deal with is limited due to
resources, both globally and per user, precisely to avoid DoS attacks by a
uses.
In the case of linux, you can control it with "ulimit -n XXXX" (for the
user that runs the test). The default here is 1024 (and the maximum, unless
it is globally raised).
I imagine windows will have similar limits, and similar (registry, etc.)
ways to handle them.
----------------------------------------------------------------------
Comment By: Sam Rushing (rushing)
Date: 2007-03-30 10:22
Message:
Logged In: YES
user_id=73736
Originator: NO
Turns out medusa doesn't have a socket counter class, that was some other
project I was thinking of.
Putting a try/except in place doesn't really help the problem... if you
fail to create a new socket what action will you take?
A better approach is to have a configurable limit on the number of open
connections, and then have a server-specific reaction to exceeding that
limit. For example, an SMTP server might respond with a 4XX greeting and
close the connection.
An additional problem on Unix is that running out of descriptors affects
more than just sockets. Once you hit the FD limit you can't open files,
or do anything that requires a descriptor.
----------------------------------------------------------------------
Comment By: billiejoex (billiejoex)
Date: 2007-03-29 07:03
Message:
Logged In: YES
user_id=1357589
Originator: YES
> The problem is that there's no portable way to know what the limit
> on file descriptors is.
Why don't put a try/except statement before creating a new socket's
file-descriptor?
I believe that such problem shouldn't be understimated.
Actually asyncore is the only asynchronous module present in Python's
stdlib.
If asyncore suffers a DoS vulnerability it just means that writing a
secure client/server application with Python without using a third-party
library isn't possible.
I wrote a modified version of asyncore that solves the problem with select
(affecting Windows) but still can't find a way to solve the problem with
socket's file-descriptors (affecting Unix).
----------------------------------------------------------------------
Comment By: Sam Rushing (rushing)
Date: 2007-03-23 12:59
Message:
Logged In: YES
user_id=73736
Originator: NO
The problem is that there's no portable way to know what the limit
on file descriptors is. The 'classic' for select/poll is the FD_SETSIZE
macro. But on some operating systems there is no such limit. [e.g.,
win32
does not use the 'lowest-free-int' model common to unix].
I believe that in Medusa there was a derived class or extension that
counted
the number of open sockets, and limited it, using something like a
semaphore.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
