Re: mbuf shortage situations (followup)
I think that what needs to be done is to split the problem in two. First, allow the mbuf routines to return a failure even with M_WAIT. If M_WAIT is used, it simply means 'try harder, sleeping a bit if necessary'. This requires ensuring that all the networking code deal with the failure case - a time consuming but straightforward task. If a failure occurs, one simply drops the packet, not the connection or anything else drastic. just the packet. The second problem that needs to be addressed is resource exhaustion. For example, allocating thousands of connections and socket-opting their buffers as large as possible, or programs such as syslog accepting new connections ad-infinitum. This is a harder problem to fix properly, but a lot of the various issues such as those with syslog can be dealt with in userland rather then the kernel. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: mbuf shortage situations (followup)
:In 4.3, the code was able to deal with cluster allocation failing. We :have a somewhat different situation now, because many network :interface devices have less-flexible DMA mechanisms which don't allow :packet reception into non-contiguous buffers, so we need to have at :least a certain number of clusters available for this purpose. : :-GAWollman : :-- :Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same This is an interrupt level mechanism. The mbuf code is *already* allowed to (and does) return NULL in this case so I don't think it applies to the problem under discussion. The case that is causing the panics is with the non-interrupt mbuf allocation mechanism. Specifically, the case where M_WAIT is used. The second problem under discussion, which really ought to be separated out from the mbuf panic problem, is the potential for a deadlock or denial of service attack when the system is attacked in a manner that eats all available mbufs. :[EMAIL PROTECTED] | O Siem / The fires of freedom :Opinions not those of| Dance in the burning flame -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: mbuf shortage situations (followup)
On Mon, 13 Sep 1999, Matthew Dillon wrote: The case that is causing the panics is with the non-interrupt mbuf allocation mechanism. Specifically, the case where M_WAIT is used. The second problem under discussion, which really ought to be separated out from the mbuf panic problem, is the potential for a deadlock or denial of service attack when the system is attacked in a manner that eats all available mbufs. The traditional way to prevent resource-starvation DoSes from the user populus has been to add administrative limits. Using RLIMIT_SBSIZE does this nicely. Yes, this isn't actually fixing the panics, but it is good preventative medicine. -Matt Matthew Dillon [EMAIL PROTECTED] -- Brian Fundakowski Feldman \ FreeBSD: The Power to Serve! / [EMAIL PROTECTED]`--' To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: mbuf shortage situations (followup)
:In 4.3, the code was able to deal with cluster allocation failing. We :have a somewhat different situation now, because many network :interface devices have less-flexible DMA mechanisms which don't allow :packet reception into non-contiguous buffers, so we need to have at :least a certain number of clusters available for this purpose. : :-GAWollman : :-- :Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same This is an interrupt level mechanism. The mbuf code is *already* allowed to (and does) return NULL in this case so I don't think it applies to the problem under discussion. The case that is causing the panics is with the non-interrupt mbuf allocation mechanism. Specifically, the case where M_WAIT is used. The second problem under discussion, which really ought to be separated out from the mbuf panic problem, is the potential for a deadlock or denial of service attack when the system is attacked in a manner that eats all available mbufs. :woll...@lcs.mit.edu | O Siem / The fires of freedom :Opinions not those of| Dance in the burning flame -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: mbuf shortage situations (followup)
I think that what needs to be done is to split the problem in two. First, allow the mbuf routines to return a failure even with M_WAIT. If M_WAIT is used, it simply means 'try harder, sleeping a bit if necessary'. This requires ensuring that all the networking code deal with the failure case - a time consuming but straightforward task. If a failure occurs, one simply drops the packet, not the connection or anything else drastic. just the packet. The second problem that needs to be addressed is resource exhaustion. For example, allocating thousands of connections and socket-opting their buffers as large as possible, or programs such as syslog accepting new connections ad-infinitum. This is a harder problem to fix properly, but a lot of the various issues such as those with syslog can be dealt with in userland rather then the kernel. -Matt To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: mbuf shortage situations (followup)
On Mon, 13 Sep 1999, Matthew Dillon wrote: The case that is causing the panics is with the non-interrupt mbuf allocation mechanism. Specifically, the case where M_WAIT is used. The second problem under discussion, which really ought to be separated out from the mbuf panic problem, is the potential for a deadlock or denial of service attack when the system is attacked in a manner that eats all available mbufs. The traditional way to prevent resource-starvation DoSes from the user populus has been to add administrative limits. Using RLIMIT_SBSIZE does this nicely. Yes, this isn't actually fixing the panics, but it is good preventative medicine. -Matt Matthew Dillon dil...@backplane.com -- Brian Fundakowski Feldman \ FreeBSD: The Power to Serve! / gr...@freebsd.org`--' To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: mbuf shortage situations (followup)
From woll...@khavrinen.lcs.mit.edu Mon Sep 13 22:39:37 1999 I'm also aware of the possiblity of some people not liking the fact that we tsleep() forever (e.g. tsleep(x,x,x,0)). I don't have any problem with sleeping forever -- but I am concerned about the possibility of deadlock, especially when client-NFS is involved. If the problem just moves around and has harder-to-recover symptoms, the change isn't helping. IMHO merely sleeping can't help if we deal with a DoS. Instead of sleeping, kernel should find out where all kernel memory is wasted and free some memory (and, probably, remove/log reason). And, partially, this mechanism is already implemented in ip_drain() and tp_drain(). -- Stas Kisel. UNIX, security, C, TCP/IP, Web. UNIX - the best adventure game http://www.tekmetrics.com/transcript.shtml?pid=20053 http://www.crimea.edu +380(652)510222,230238 ; s...@crimea.edu s...@sonet.crimea.ua ; 2:460/54.4 To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: mbuf shortage situations (followup)
!I think that what needs to be done is to split the problem in two. First, !allow the mbuf routines to return a failure even with M_WAIT. If M_WAIT !is used, it simply means 'try harder, sleeping a bit if necessary'. This !requires ensuring that all the networking code deal with the failure !case - a time consuming but straightforward task. If a failure occurs, !one simply drops the packet, not the connection or anything else drastic. !just the packet. Yes, these is mainly the part I've been working on recently. The sleeping and what not (as I'm sure you've seen from the patches if you looked at them) has already been completed. Adding a counter that will expire and return a pre-defined error is trivial, in this case. The only real issue here (if we can call it that) is get _all_ the networking code to recognize this. Anyone want to help? :-) ! !The second problem that needs to be addressed is resource exhaustion. !For example, allocating thousands of connections and socket-opting their !buffers as large as possible, or programs such as syslog accepting new !connections ad-infinitum. This is a harder problem to fix properly, !but a lot of the various issues such as those with syslog can be dealt !with in userland rather then the kernel. ! ! -Matt ! I agree. The issue here is somewhat related (if I understand your explanation correctly) to [local] processes attempting to grab a lot of socket buffer space. I was a little less concerned with this issue since, as I previously mentionned, Brian Feldman is working on limiting socket buffer space. Nonetheless, if we do not consider limiting, here's what I believe will need to be done: As explained above, when we run out of mbufs and/or mbuf clusters (and some are needed), if we are M_WAIT (when processes socket opt their buffers as large as possible, the call is usually with M_WAIT), we will end up tsleep()ing for certain periods of time, until our counter expires and we return our pre-defined error (as mentionned above). When we do return this error, however, the caller (for instance, we can consider sosend() the caller -- which, if I remember correctly, is one of the callers to MGET() when we setsockopt a large buffer and consequently write() to this socket), will also have to know how to properly deal with this error (e.g.: kill the process?). Killing the process may seem somewhat sadistic to some ( :-) ), but remember that if we do get to the point where 'normal' local processes eat up so much buffer space that we run out, we should probably be increasing NMBCLUSTERS and/or maxusers anyway. As for script weenies, I hope that Brian (and whomever else may be working on it) gets that sockbuf limiting code done, because, to be quite honest, I don't think that script kids having to comprimise more than one account just so they can DoS a box will be much of an issue (if worse comes to worse, we can limit per gid -as opposed to per uid). With exhaustion attacks such as these, we're better off just limiting. Regards, Bosko Milekic. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: mbuf shortage situations (followup)
On Mon, 13 Sep 1999, Garrett Wollman wrote: !On Sun, 12 Sep 1999 23:19:13 -0400 (EDT), Bosko Milekic [EMAIL PROTECTED] said: ! ! This message is in MIME format. The first part should be readable text, ! while the remaining parts are likely unreadable without MIME-aware tools. ! Send mail to [EMAIL PROTECTED] for more info. ! !It would be preferable if text were sent as text, since MIME-encoded !patches require more effort to read. ! I deffinately agree. This is obviously my mistake, and I was somewhat in a rush, very lagged (modem, eurgh), using pine, and made several [dumb] typos in the 'Attatchement' field. ! I'm also aware of the possiblity of some people not liking the ! fact that we tsleep() forever (e.g. tsleep(x,x,x,0)). ! ! !I don't have any problem with sleeping forever -- but I am concerned !about the possibility of deadlock, especially when client-NFS is !involved. If the problem just moves around and has harder-to-recover !symptoms, the change isn't helping. Well, the main purpose of the code is to basically sleep until something is freed after we've already exhausted the mb_map arena (as I'm sure you've seen if you were able to grab the attachements). This is really a-la-limite stuff. In other words, if 'normal' local programs are having trouble because of mb_map exhaustion, then maxusers nmbclusters would have to be augmented. ! !The 4.3BSD code had two different behaviors: ! ! - For clusters, if M_WAIT was specified and there was no space !left in mb_map, it panicked. However, m_clalloc was never called with !M_WAIT, so that panic was effectively dead code. Hmmm. If m_clalloc was never called with M_WAIT, then all the code calling m_clalloc deffinately checked its return value. It probably had specific ways to deal with m_clalloc returning failures, too? ! ! - For mbufs, if M_WAIT was specified and there were no mbufs !available, it would sleep at PZERO - 1 (which was interruptible). ! !In 4.3, the code was able to deal with cluster allocation failing. We !have a somewhat different situation now, because many network !interface devices have less-flexible DMA mechanisms which don't allow !packet reception into non-contiguous buffers, so we need to have at !least a certain number of clusters available for this purpose. Exactly. This is the next challenge. As for things being interruptable, as I mentionned to a reply to Matt Dillon just a few seconds ago, getting the tsleep to occasionally expire is trivial. As you say above, it's dealing with the failure that is the issue. ! !-GAWollman ! !-- !Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same ![EMAIL PROTECTED] | O Siem / The fires of freedom !Opinions not those of| Dance in the burning flame !MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick ! Cheers, Bosko Milekic. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: mbuf shortage situations (followup)
On Sun, 12 Sep 1999 23:19:13 -0400 (EDT), Bosko Milekic bmile...@dsuper.net said: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to m...@docserver.cac.washington.edu for more info. It would be preferable if text were sent as text, since MIME-encoded patches require more effort to read. I'm also aware of the possiblity of some people not liking the fact that we tsleep() forever (e.g. tsleep(x,x,x,0)). I don't have any problem with sleeping forever -- but I am concerned about the possibility of deadlock, especially when client-NFS is involved. If the problem just moves around and has harder-to-recover symptoms, the change isn't helping. The 4.3BSD code had two different behaviors: - For clusters, if M_WAIT was specified and there was no space left in mb_map, it panicked. However, m_clalloc was never called with M_WAIT, so that panic was effectively dead code. - For mbufs, if M_WAIT was specified and there were no mbufs available, it would sleep at PZERO - 1 (which was interruptible). In 4.3, the code was able to deal with cluster allocation failing. We have a somewhat different situation now, because many network interface devices have less-flexible DMA mechanisms which don't allow packet reception into non-contiguous buffers, so we need to have at least a certain number of clusters available for this purpose. -GAWollman -- Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same woll...@lcs.mit.edu | O Siem / The fires of freedom Opinions not those of| Dance in the burning flame MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: mbuf shortage situations (followup)
!I think that what needs to be done is to split the problem in two. First, !allow the mbuf routines to return a failure even with M_WAIT. If M_WAIT !is used, it simply means 'try harder, sleeping a bit if necessary'. This !requires ensuring that all the networking code deal with the failure !case - a time consuming but straightforward task. If a failure occurs, !one simply drops the packet, not the connection or anything else drastic. !just the packet. Yes, these is mainly the part I've been working on recently. The sleeping and what not (as I'm sure you've seen from the patches if you looked at them) has already been completed. Adding a counter that will expire and return a pre-defined error is trivial, in this case. The only real issue here (if we can call it that) is get _all_ the networking code to recognize this. Anyone want to help? :-) ! !The second problem that needs to be addressed is resource exhaustion. !For example, allocating thousands of connections and socket-opting their !buffers as large as possible, or programs such as syslog accepting new !connections ad-infinitum. This is a harder problem to fix properly, !but a lot of the various issues such as those with syslog can be dealt !with in userland rather then the kernel. ! ! -Matt ! I agree. The issue here is somewhat related (if I understand your explanation correctly) to [local] processes attempting to grab a lot of socket buffer space. I was a little less concerned with this issue since, as I previously mentionned, Brian Feldman is working on limiting socket buffer space. Nonetheless, if we do not consider limiting, here's what I believe will need to be done: As explained above, when we run out of mbufs and/or mbuf clusters (and some are needed), if we are M_WAIT (when processes socket opt their buffers as large as possible, the call is usually with M_WAIT), we will end up tsleep()ing for certain periods of time, until our counter expires and we return our pre-defined error (as mentionned above). When we do return this error, however, the caller (for instance, we can consider sosend() the caller -- which, if I remember correctly, is one of the callers to MGET() when we setsockopt a large buffer and consequently write() to this socket), will also have to know how to properly deal with this error (e.g.: kill the process?). Killing the process may seem somewhat sadistic to some ( :-) ), but remember that if we do get to the point where 'normal' local processes eat up so much buffer space that we run out, we should probably be increasing NMBCLUSTERS and/or maxusers anyway. As for script weenies, I hope that Brian (and whomever else may be working on it) gets that sockbuf limiting code done, because, to be quite honest, I don't think that script kids having to comprimise more than one account just so they can DoS a box will be much of an issue (if worse comes to worse, we can limit per gid -as opposed to per uid). With exhaustion attacks such as these, we're better off just limiting. Regards, Bosko Milekic. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: mbuf shortage situations (followup)
On Mon, 13 Sep 1999, Garrett Wollman wrote: !On Sun, 12 Sep 1999 23:19:13 -0400 (EDT), Bosko Milekic bmile...@dsuper.net said: ! ! This message is in MIME format. The first part should be readable text, ! while the remaining parts are likely unreadable without MIME-aware tools. ! Send mail to m...@docserver.cac.washington.edu for more info. ! !It would be preferable if text were sent as text, since MIME-encoded !patches require more effort to read. ! I deffinately agree. This is obviously my mistake, and I was somewhat in a rush, very lagged (modem, eurgh), using pine, and made several [dumb] typos in the 'Attatchement' field. ! I'm also aware of the possiblity of some people not liking the ! fact that we tsleep() forever (e.g. tsleep(x,x,x,0)). ! ! !I don't have any problem with sleeping forever -- but I am concerned !about the possibility of deadlock, especially when client-NFS is !involved. If the problem just moves around and has harder-to-recover !symptoms, the change isn't helping. Well, the main purpose of the code is to basically sleep until something is freed after we've already exhausted the mb_map arena (as I'm sure you've seen if you were able to grab the attachements). This is really a-la-limite stuff. In other words, if 'normal' local programs are having trouble because of mb_map exhaustion, then maxusers nmbclusters would have to be augmented. ! !The 4.3BSD code had two different behaviors: ! ! - For clusters, if M_WAIT was specified and there was no space !left in mb_map, it panicked. However, m_clalloc was never called with !M_WAIT, so that panic was effectively dead code. Hmmm. If m_clalloc was never called with M_WAIT, then all the code calling m_clalloc deffinately checked its return value. It probably had specific ways to deal with m_clalloc returning failures, too? ! ! - For mbufs, if M_WAIT was specified and there were no mbufs !available, it would sleep at PZERO - 1 (which was interruptible). ! !In 4.3, the code was able to deal with cluster allocation failing. We !have a somewhat different situation now, because many network !interface devices have less-flexible DMA mechanisms which don't allow !packet reception into non-contiguous buffers, so we need to have at !least a certain number of clusters available for this purpose. Exactly. This is the next challenge. As for things being interruptable, as I mentionned to a reply to Matt Dillon just a few seconds ago, getting the tsleep to occasionally expire is trivial. As you say above, it's dealing with the failure that is the issue. ! !-GAWollman ! !-- !Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same !woll...@lcs.mit.edu | O Siem / The fires of freedom !Opinions not those of| Dance in the burning flame !MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick ! Cheers, Bosko Milekic. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: mbuf shortage situations (followup)
Hello (again),
On Thu, 9 Sep 1999, Stas Kisel wrote:
! From [EMAIL PROTECTED] Thu Sep 9 16:17:27 1999
!
! Probably it is not self-evident why we HAVE to drop this connection.
!
! So what if someone manages to crash a program due to a DOS attack ?
! An easy one that comes to mind is syslogd. It's often stuck in disk-wait
! and can easily be targetted with a large number of packets.
!
!1. If ever syslog used (or will use) TCP, it should drop the connection
!which is logging data too quickly.
!OS shouldn't kill process, only drop connection. So no crash.
!More examples?
!
!2. udp_drain() may either drop all packets or intellectually select
!"offending" socket and try to avoid deletion of "right" packets and
!simplifying spoofing. RFC allows 1st way, but 2-nd can improve OS.
!
!3. Another idea. Apart from the *_drain() method. Probably I ever will
!try to implement it somedays (quite low probability, though).
!Set TCP window in a packets according to really available kernel
!memory. Available memory should be distributed non-uniformly
!between maximum number of sockets. So 1-st socket has window=
!=64k-still_not_read_data, and 1024-th has window=MIN_WINDOW-
!-still_not_read_data.
!MIN_WINDOW should be determined for max efficiency. About 2k.
!Distribution can not be linear - it isapproximately like min(NORM*1/x,64k).
!Exactly it can be determined via functional equation. Something like
!\integral_0^maxsockets{dist(x)dx}=kernel_memory and several
!conditions. (sorry for my poor TeX).
!
!In a case of attack new sockets will be created with a very small
!window - about 2k.
!
!Please blame me as much as possible - probably I have missed some significant
!detail.
!Probably all this math suxx and the best is a "stair" function -
!somebody already works on lowering TCP window, if I didn't mistaken.
!
!
!--
!Stas Kisel. UNIX, security, C, TCP/IP, Web. UNIX - the best adventure game
!http://www.tekmetrics.com/transcript.shtml?pid=20053 http://www.crimea.edu
!+380(652)510222,230238 ; [EMAIL PROTECTED] [EMAIL PROTECTED] ; 2:460/54.4
!
These are all interesting ideas.
However, when I initially posted regarding this, I was
disappointed in seeing that we simply handle MGET()s, MGETHDR()s, and
MCLALLOC()s by storing a null _even_ if we are M_WAIT. What basically
ended up happening (and, the last time I checked, it's like this even in
--CURRENT), is m_retry() -- or m_retryhdr() (this is in the case of no
mbufs beings available) would simply panic().
I have produced patches (see attached -- they are seperated into
two different patches, mbuf.patch which patches kern/uipc_mbuf.c and
mbuf2.patch which patches sys/mbuf.h) that will basically tsleep() in the
case of an M_WAIT and mbuf or mbuf cluster shortage. I wanted something
that would make sure that we will get a non-NULL result when we called
with M_WAIT. Obviously, this isn't the definite solution to the DoS
problem that seemed to have become the main idea of discussion in this
thread.
However, I've kept that in mind, and I am now starting work (when
time permits) on some code which will enable us to warn the network
protocol module code that we're out of mbufs (or mbuf clusters) when the
situation occurs. This way, if we can't get anything even with m_reclaim
(which would be called from m_retry if we are M_WAIT), we could have the
protocols figure out what to drop.
I'm also aware of the possiblity of some people not liking the
fact that we tsleep() forever (e.g. tsleep(x,x,x,0)). Thus, I am open to
modifying the diffs to add a counter and have the tsleep expire every once
in a while so that finally, when the counter would expire, we would return
a deffinate null _even_ if we are M_WAIT, but this can only be implemented
if we make sure that ALL the MGET and company callers check for this
(which would be annoying to do).
Cheers,
Bosko Milekic.
--- /usr/src/sys/kern.old/uipc_mbuf.c Wed Sep 8 20:45:50 1999
+++ /usr/src/sys/kern/uipc_mbuf.c Sun Sep 12 22:44:23 1999
@@ -60,6 +60,8 @@
intmax_hdr;
intmax_datalen;
+static int m_mballoc_wid = 0, m_clalloc_wid = 0;
+
SYSCTL_INT(_kern_ipc, KIPC_MAX_LINKHDR, max_linkhdr, CTLFLAG_RW,
max_linkhdr, 0, "");
SYSCTL_INT(_kern_ipc, KIPC_MAX_PROTOHDR, max_protohdr, CTLFLAG_RW,
@@ -153,6 +155,57 @@
return (1);
}
+/*
+ * Function used for waiting on some mbuf to be freed and, upon wakeup,
+ * to go get that mbuf and use it.
+ */
+struct mbuf *
+m_mballoc_wait(caller, type)
+ int caller;
+ int type;
+{
+ struct mbuf *p;
+
+RetryFetch:
+ /* Sleep here until something's available. */
+ m_mballoc_wid++;
+ tsleep(m_mballoc_wid, PVM, "mballc", 0);
+
+ /*
+* Now that we (think) that we've got something, we will redo an
+* MGET, but avoid getting into another instance of m_mballoc_wait()
+* We do this by defining this function as null.
+*/
Re: mbuf shortage situations (followup)
Hello (again),
On Thu, 9 Sep 1999, Stas Kisel wrote:
! From ava...@cheops.anu.edu.au Thu Sep 9 16:17:27 1999
!
! Probably it is not self-evident why we HAVE to drop this connection.
!
! So what if someone manages to crash a program due to a DOS attack ?
! An easy one that comes to mind is syslogd. It's often stuck in disk-wait
! and can easily be targetted with a large number of packets.
!
!1. If ever syslog used (or will use) TCP, it should drop the connection
!which is logging data too quickly.
!OS shouldn't kill process, only drop connection. So no crash.
!More examples?
!
!2. udp_drain() may either drop all packets or intellectually select
!offending socket and try to avoid deletion of right packets and
!simplifying spoofing. RFC allows 1st way, but 2-nd can improve OS.
!
!3. Another idea. Apart from the *_drain() method. Probably I ever will
!try to implement it somedays (quite low probability, though).
!Set TCP window in a packets according to really available kernel
!memory. Available memory should be distributed non-uniformly
!between maximum number of sockets. So 1-st socket has window=
!=64k-still_not_read_data, and 1024-th has window=MIN_WINDOW-
!-still_not_read_data.
!MIN_WINDOW should be determined for max efficiency. About 2k.
!Distribution can not be linear - it isapproximately like min(NORM*1/x,64k).
!Exactly it can be determined via functional equation. Something like
!\integral_0^maxsockets{dist(x)dx}=kernel_memory and several
!conditions. (sorry for my poor TeX).
!
!In a case of attack new sockets will be created with a very small
!window - about 2k.
!
!Please blame me as much as possible - probably I have missed some significant
!detail.
!Probably all this math suxx and the best is a stair function -
!somebody already works on lowering TCP window, if I didn't mistaken.
!
!
!--
!Stas Kisel. UNIX, security, C, TCP/IP, Web. UNIX - the best adventure game
!http://www.tekmetrics.com/transcript.shtml?pid=20053 http://www.crimea.edu
!+380(652)510222,230238 ; s...@crimea.edu s...@sonet.crimea.ua ; 2:460/54.4
!
These are all interesting ideas.
However, when I initially posted regarding this, I was
disappointed in seeing that we simply handle MGET()s, MGETHDR()s, and
MCLALLOC()s by storing a null _even_ if we are M_WAIT. What basically
ended up happening (and, the last time I checked, it's like this even in
--CURRENT), is m_retry() -- or m_retryhdr() (this is in the case of no
mbufs beings available) would simply panic().
I have produced patches (see attached -- they are seperated into
two different patches, mbuf.patch which patches kern/uipc_mbuf.c and
mbuf2.patch which patches sys/mbuf.h) that will basically tsleep() in the
case of an M_WAIT and mbuf or mbuf cluster shortage. I wanted something
that would make sure that we will get a non-NULL result when we called
with M_WAIT. Obviously, this isn't the definite solution to the DoS
problem that seemed to have become the main idea of discussion in this
thread.
However, I've kept that in mind, and I am now starting work (when
time permits) on some code which will enable us to warn the network
protocol module code that we're out of mbufs (or mbuf clusters) when the
situation occurs. This way, if we can't get anything even with m_reclaim
(which would be called from m_retry if we are M_WAIT), we could have the
protocols figure out what to drop.
I'm also aware of the possiblity of some people not liking the
fact that we tsleep() forever (e.g. tsleep(x,x,x,0)). Thus, I am open to
modifying the diffs to add a counter and have the tsleep expire every once
in a while so that finally, when the counter would expire, we would return
a deffinate null _even_ if we are M_WAIT, but this can only be implemented
if we make sure that ALL the MGET and company callers check for this
(which would be annoying to do).
Cheers,
Bosko Milekic.
--- /usr/src/sys/kern.old/uipc_mbuf.c Wed Sep 8 20:45:50 1999
+++ /usr/src/sys/kern/uipc_mbuf.c Sun Sep 12 22:44:23 1999
@@ -60,6 +60,8 @@
intmax_hdr;
intmax_datalen;
+static int m_mballoc_wid = 0, m_clalloc_wid = 0;
+
SYSCTL_INT(_kern_ipc, KIPC_MAX_LINKHDR, max_linkhdr, CTLFLAG_RW,
max_linkhdr, 0, );
SYSCTL_INT(_kern_ipc, KIPC_MAX_PROTOHDR, max_protohdr, CTLFLAG_RW,
@@ -153,6 +155,57 @@
return (1);
}
+/*
+ * Function used for waiting on some mbuf to be freed and, upon wakeup,
+ * to go get that mbuf and use it.
+ */
+struct mbuf *
+m_mballoc_wait(caller, type)
+ int caller;
+ int type;
+{
+ struct mbuf *p;
+
+RetryFetch:
+ /* Sleep here until something's available. */
+ m_mballoc_wid++;
+ tsleep(m_mballoc_wid, PVM, mballc, 0);
+
+ /*
+* Now that we (think) that we've got something, we will redo an
+* MGET, but avoid getting into another instance of m_mballoc_wait()
+* We do this by defining this function as null.
+*/
+#define m_mballoc_wait(caller,type)
