Re: [Opensim-dev] Rate Limit DOS Protection
On 16/10/13 20:46, Toni Alatalo wrote: On Oct 16, 2013, at 8:39 PM, Justin Clark-Casey jjusti...@googlemail.com wrote: In fact, I would say that making OpenSimulator as functional as possible out of the box (within certain bounds such as not bundling a web interface) is a standard OpenSimulator operating procedure. That's why we have SQLite as the default database, for example. I suppose that is certainly the case when you think of it as a distribution also in itself, and not just a part of some larger distros (like Diva's etc). If it would be strictly considered as a lib to be used in some system that fills in the rest, then what makes sense can differ. Like say Bullet or Ogre are. But given that there are a lot of people out there who just use Opensim as is, always new people just grabbing and launching it and starting to use, I figure your statement must have truth in it. I agree that there is a tension between making OpenSimulator usable out-of-the-box and doing different things in downstream distributions. My preference for out-of-the-box is to keep as many people using core OpenSimulator as possible. Multiple distributions are valuable but I feel that they can fragment development resources. And then there is the temptation (which I would have) to focus complex changes on one's own distribution rather than making the effort to negotiate them upstream. -- Justin Clark-Casey (justincc) OSVW Consulting http://justincc.org http://twitter.com/justincc ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
In my opinion, anybody who is sophisticated enough to be using a proxy is sophisticated enough to be aware of what's happening in OpenSimulator and sophisticated enough to to disable these changes if required. Anybody operating a grid must take responsibility for managing their configuration when upgrading. I disagree that no change in behaviour is a standard operating procedure in OpenSimulator. The project changes in behaviour all the time, usually to better match the de-facto standard and features set by Linden Lab or to give a better user/admin experience. In fact, I would say that making OpenSimulator as functional as possible out of the box (within certain bounds such as not bundling a web interface) is a standard OpenSimulator operating procedure. That's why we have SQLite as the default database, for example. Under this principle, one should enable things like core groups support by default when it's ready instead of leaving them forever disabled because some existing users may have configured their system differently. Not following this principle puts the burden of configuration change on the large number of new users who are least able to manage it rather than the smaller number of existing users who are not using an out-of-the-box configuration. Indeed, new users may never know that certain OpenSimulator features exist at all and that the system is less functional than it really is. All this makes it harder to grow the project. On 08/10/13 22:20, Melanie wrote: People like us have the protection in the proxy and need to turn the feature off. Not a problem for me, I'm aware of it, but my concern was for those who didn't listen to me tell them Dont' blindly reuse configs Melanie ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev . -- Justin Clark-Casey (justincc) OSVW Consulting http://justincc.org http://twitter.com/justincc ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
On Oct 16, 2013, at 8:39 PM, Justin Clark-Casey jjusti...@googlemail.com wrote: In fact, I would say that making OpenSimulator as functional as possible out of the box (within certain bounds such as not bundling a web interface) is a standard OpenSimulator operating procedure. That's why we have SQLite as the default database, for example. I suppose that is certainly the case when you think of it as a distribution also in itself, and not just a part of some larger distros (like Diva's etc). If it would be strictly considered as a lib to be used in some system that fills in the rest, then what makes sense can differ. Like say Bullet or Ogre are. But given that there are a lot of people out there who just use Opensim as is, always new people just grabbing and launching it and starting to use, I figure your statement must have truth in it. I do find Melanie's point valid too, I'm sure you agree with that as well: is good practice to keep it stable so that updates don't secretly break setups. But the discussion how it goes in this case you already had. ~Toni ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.I have, both as an example, and good practice, applied the Rate limit protection to the login service.There are new Configuration options in StandaloneCommon.ini and Robust.ini that control how the connections are rate limited and if trusts the X-Forwarded-For header.Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new configuration options in the [LoginService] section of your bin/Robust.ini.example and /bin/config-include/StandaloneCommon.ini.example AND/OR have discussions on what would be more sane default options. There's a chance that this could affect anyone, so don't neglect to take a look at it. You may also notice messages on your console and in your logs like: 21:56:29 - [LOGINDOSPROTECTION]: client: 192.168.1.213 is blocked for 12 milliseconds, X-ForwardedForAllowed status is False, endpoint:192.168.1.213 This is an example of the DOS Protection blocking a connection because the client went beyond the rate limit. The rate limit is defined by X requests in Y period of time and is implemented in a rolling Y fashion. It also has a 'forget' period of time that will unblock the blocked user. At this point, there's one implemented for XMLRPC handlers, one for GenericHTTPHandlers and a base class for StreamHandlers based on BaseStreamHandler. If you are interested in the code changes, you can check the diff: http://opensimulator.org/viewgit/?a=commitdiffp=opensimh=f76cc6036ebf446553ee5201321879538dafe3b2 There's still more to do, and, here's a start to providing some modicum of protection on the services. If you have any questions, feel free to reply and ask.. or send me an e-mail personally. Thanks and Best Regards Teravus ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
Also I might point out that this policy has rarely been adhered in the past, and generally in cases where the practical operations impact is far greater (HG). Cheers On Oct 8, 2013 2:40 AM, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.I have, both as an example, and good practice, applied the Rate limit protection to the login service.There are new Configuration options in StandaloneCommon.ini and Robust.ini that control how the connections are rate limited and if trusts the X-Forwarded-For header.Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new configuration options in the [LoginService] section of your bin/Robust.ini.example and /bin/config-include/StandaloneCommon.ini.example AND/OR have discussions on what would be more sane default options. There's a chance that this could affect anyone, so don't neglect to take a look at it. You may also notice messages on your console and in your logs like: 21:56:29 - [LOGINDOSPROTECTION]: client: 192.168.1.213 is blocked for 12 milliseconds, X-ForwardedForAllowed status is False, endpoint:192.168.1.213 This is an example of the DOS Protection blocking a connection because the client went beyond the rate limit. The rate limit is defined by X requests in Y period of time and is implemented in a rolling Y fashion. It also has a 'forget' period of time that will unblock the blocked user. At this point, there's one implemented for XMLRPC handlers, one for GenericHTTPHandlers and a base class for StreamHandlers based on BaseStreamHandler. If you are interested in the code changes, you can check the diff: http://opensimulator.org/viewgit/?a=commitdiffp=opensimh=f76cc6036ebf446553ee5201321879538dafe3b2 There's still more to do, and, here's a start to providing some modicum of protection on the services. If you have any questions, feel free to reply and ask.. or send me an e-mail personally. Thanks and Best Regards Teravus ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
I'm not a dev and certainly not anybody special, but this has been needed for some time. Thank you Teravus for recognizing the need, and I hope that this can be implemented somehow even if it is not the default configuration. On Tue, Oct 8, 2013 at 6:27 AM, James Stallings II james.stalli...@gmail.com wrote: Also I might point out that this policy has rarely been adhered in the past, and generally in cases where the practical operations impact is far greater (HG). Cheers On Oct 8, 2013 2:40 AM, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.I have, both as an example, and good practice, applied the Rate limit protection to the login service.There are new Configuration options in StandaloneCommon.ini and Robust.ini that control how the connections are rate limited and if trusts the X-Forwarded-For header.Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new configuration options in the [LoginService] section of your bin/Robust.ini.example and /bin/config-include/StandaloneCommon.ini.example AND/OR have discussions on what would be more sane default options. There's a chance that this could affect anyone, so don't neglect to take a look at it. You may also notice messages on your console and in your logs like: 21:56:29 - [LOGINDOSPROTECTION]: client: 192.168.1.213 is blocked for 12 milliseconds, X-ForwardedForAllowed status is False, endpoint:192.168.1.213 This is an example of the DOS Protection blocking a connection because the client went beyond the rate limit. The rate limit is defined by X requests in Y period of time and is implemented in a rolling Y fashion. It also has a 'forget' period of time that will unblock the blocked user. At this point, there's one implemented for XMLRPC handlers, one for GenericHTTPHandlers and a base class for StreamHandlers based on BaseStreamHandler. If you are interested in the code changes, you can check the diff: http://opensimulator.org/viewgit/?a=commitdiffp=opensimh=f76cc6036ebf446553ee5201321879538dafe3b2 There's still more to do, and, here's a start to providing some modicum of protection on the services. If you have any questions, feel free to reply and ask.. or send me an e-mail personally. Thanks and Best Regards Teravus ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
At 11:00 08/10/2013, Teravus Ovares tera...@gmail.com wrote: Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new configuration options in the [LoginService] section of your bin/Robust.ini.example and /bin/config-include/StandaloneCommon.ini.example AND/OR have discussions on what would be more sane default options. Terravus, you need to make the change in Robust.HG.ini.example too. I think some protection should be in place by default. Otherwise grid and server managers/runners have to know and understand these settings to get the protectio we desire. ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
I'm worried that people with larger installations will see service failures because legit traffic is seen as abusive. This could cause issues for the lerger grids out there. I don't believe that whatever tenuous protection this may offer for small grids and standalones outwieghs the potential service impairment it may cause for unsuspecting larger grids. Not every grid operator reads this list, So I'd again suggest that we stick to the way we've always done it and make the default for new features be off. Melanie On 8 Oct 2013, at 09:31, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.I have, both as an example, and good practice, applied the Rate limit protection to the login service.There are new Configuration options in StandaloneCommon.ini and Robust.ini that control how the connections are rate limited and if trusts the X-Forwarded-For header.Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new configuration options in the [LoginService] section of your bin/Robust.ini.example and /bin/config-include/StandaloneCommon.ini.example AND/OR have discussions on what would be more sane default options. There's a chance that this could affect anyone, so don't neglect to take a look at it. You may also notice messages on your console and in your logs like: 21:56:29 - [LOGINDOSPROTECTION]: client: 192.168.1.213 is blocked for 12 milliseconds, X-ForwardedForAllowed status is False, endpoint:192.168.1.213 This is an example of the DOS Protection blocking a connection because the client went beyond the rate limit. The rate limit is defined by X requests in Y period of time and is implemented in a rolling Y fashion. It also has a 'forget' period of time that will unblock the blocked user. At this point, there's one implemented for XMLRPC handlers, one for GenericHTTPHandlers and a base class for StreamHandlers based on BaseStreamHandler. If you are interested in the code changes, you can check the diff: http://opensimulator.org/viewgit/?a=commitdiffp=opensimh=f76cc6036ebf446553ee5201321879538dafe3b2 There's still more to do, and, here's a start to providing some modicum of protection on the services. If you have any questions, feel free to reply and ask.. or send me an e-mail personally. Thanks and Best Regards Teravus ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
As I have often been told by you, Melanie, if I am upgrading and not auditing, adjusting, and testing my configs in accordance with changes, I'm doing it wrong. I think that statement probably applies more to those running larger concerns than anyone else. Honestly, Teravus typically does a better job of coding than to produce work that does not take such matters of scale into consideration. I'm comfortable with whatever he chooses to do, and if it turns out it isn't a case of 'one size fits all', I'll make adjustments accordingly. On Tue, Oct 8, 2013 at 7:52 AM, Melanie mela...@t-data.com wrote: I'm worried that people with larger installations will see service failures because legit traffic is seen as abusive. This could cause issues for the lerger grids out there. I don't believe that whatever tenuous protection this may offer for small grids and standalones outwieghs the potential service impairment it may cause for unsuspecting larger grids. Not every grid operator reads this list, So I'd again suggest that we stick to the way we've always done it and make the default for new features be off. Melanie On 8 Oct 2013, at 09:31, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.I have, both as an example, and good practice, applied the Rate limit protection to the login service.There are new Configuration options in StandaloneCommon.ini and Robust.ini that control how the connections are rate limited and if trusts the X-Forwarded-For header.Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new configuration options in the [LoginService] section of your bin/Robust.ini.example and /bin/config-include/StandaloneCommon.ini.example AND/OR have discussions on what would be more sane default options. There's a chance that this could affect anyone, so don't neglect to take a look at it. You may also notice messages on your console and in your logs like: 21:56:29 - [LOGINDOSPROTECTION]: client: 192.168.1.213 is blocked for 12 milliseconds, X-ForwardedForAllowed status is False, endpoint:192.168.1.213 This is an example of the DOS Protection blocking a connection because the client went beyond the rate limit. The rate limit is defined by X requests in Y period of time and is implemented in a rolling Y fashion. It also has a 'forget' period of time that will unblock the blocked user. At this point, there's one implemented for XMLRPC handlers, one for GenericHTTPHandlers and a base class for StreamHandlers based on BaseStreamHandler. If you are interested in the code changes, you can check the diff: http://opensimulator.org/viewgit/?a=commitdiffp=opensimh=f76cc6036ebf446553ee5201321879538dafe3b2 There's still more to do, and, here's a start to providing some modicum of protection on the services. If you have any questions, feel free to reply and ask.. or send me an e-mail personally. Thanks and Best Regards Teravus ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev -- === http://osgrid.org/ http://simhost.com http://twitter.com/jstallings2 ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de
Re: [Opensim-dev] Rate Limit DOS Protection
Does this class the same as what i ever tried with apache. With apache it worked, but as soon you used groups and the DDOS apaceh filter where set too low it got jammed and stuck. But if you set it to high the use is possible zero. Not saying its useless. But maby good to understand how it works or to what its looking. Helps also for people to tune the setting riht if the want to use it. Especially group chat can be seen as ddos. On 2013-10-08 14:52, Melanie wrote: I'm worried that people with larger installations will see service failures because legit traffic is seen as abusive. This could cause issues for the lerger grids out there. I don't believe that whatever tenuous protection this may offer for small grids and standalones outwieghs the potential service impairment it may cause for unsuspecting larger grids. Not every grid operator reads this list, So I'd again suggest that we stick to the way we've always done it and make the default for new features be off. Melanie On 8 Oct 2013, at 09:31, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.I have, both as an example, and good practice, applied the Rate limit protection to the login service.There are new Configuration options in StandaloneCommon.ini and Robust.ini that control how the connections are rate limited and if trusts the X-Forwarded-For header.Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new configuration options in the [LoginService] section of your bin/Robust.ini.example and /bin/config-include/StandaloneCommon.ini.example AND/OR have discussions on what would be more sane default options. There's a chance that this could affect anyone, so don't neglect to take a look at it. You may also notice messages on your console and in your logs like: 21:56:29 - [LOGINDOSPROTECTION]: client: 192.168.1.213 is blocked for 12 milliseconds, X-ForwardedForAllowed status is False, endpoint:192.168.1.213 This is an example of the DOS Protection blocking a connection because the client went beyond the rate limit. The rate limit is defined by X requests in Y period of time and is implemented in a rolling Y fashion. It also has a 'forget' period of time that will unblock the blocked user. At this point, there's one implemented for XMLRPC handlers, one for GenericHTTPHandlers and a base class for StreamHandlers based on BaseStreamHandler. If you are interested in the code changes, you can check the diff: http://opensimulator.org/viewgit/?a=commitdiffp=opensimh=f76cc6036ebf446553ee5201321879538dafe3b2 There's still more to do, and, here's a start to providing some modicum of protection on the services. If you have any questions, feel free to reply and ask.. or send me an e-mail personally. Thanks and Best Regards Teravus ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
Just to be clear, and comment/answer some questions. I did add the DOS protection variables to the Robust.ini.example in the [LoginService] section. The Rate Limit code does 'per connection' velocity counts. The default rate setting is 5 requests in 10 seconds.If you have a transparent proxy, such as squid, you need to set DOSAllowXForwardedForHeader to true so that the code trusts the X-Forwarded-For header that your proxy adds to the headers.If you want to turn it off, set DOSMaxRequestsInTimeFrame = 0. If you want to allow individual clients to do 20 connections in 5 seconds, set DOSMaxRequestsInTimeFrame = 20 and DOSRequestTimeFrameMS = 5000. If you want to change how long individual connections are blocked when they go over the rate limit, change DOSForgiveClientAfterMS. Hopefully this helps get you started with the config options. One more thing, this DOS protector is configured and implemented 'per service',So, if it's implemented in the login service, it's not going to get triggered by the Group Service. If there's DOS protection on the Group Service and the login service, and a connection gets blocked on the login service, they'll still be able to access the Group Service because they're individually rate limited... This is for flexibility.Choosing what the best rate limit is cannot really be done server wide, it has to be based on the needs of the individual service and that's why it was implemented this way. At this point, the only major service that has rate limiting on by default is the login service. I'll be happy to answer more questions and discuss default settings. Best Regards Teravus On Tue, Oct 8, 2013 at 8:04 AM, James Stallings II james.stalli...@gmail.com wrote: As I have often been told by you, Melanie, if I am upgrading and not auditing, adjusting, and testing my configs in accordance with changes, I'm doing it wrong. I think that statement probably applies more to those running larger concerns than anyone else. Honestly, Teravus typically does a better job of coding than to produce work that does not take such matters of scale into consideration. I'm comfortable with whatever he chooses to do, and if it turns out it isn't a case of 'one size fits all', I'll make adjustments accordingly. On Tue, Oct 8, 2013 at 7:52 AM, Melanie mela...@t-data.com wrote: I'm worried that people with larger installations will see service failures because legit traffic is seen as abusive. This could cause issues for the lerger grids out there. I don't believe that whatever tenuous protection this may offer for small grids and standalones outwieghs the potential service impairment it may cause for unsuspecting larger grids. Not every grid operator reads this list, So I'd again suggest that we stick to the way we've always done it and make the default for new features be off. Melanie On 8 Oct 2013, at 09:31, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.I have, both as an example, and good practice, applied the Rate limit protection to the login service.There are new Configuration options in StandaloneCommon.ini and Robust.ini that control how the connections are rate limited and if trusts the X-Forwarded-For header.Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new configuration options in the [LoginService] section of your bin/Robust.ini.example and /bin/config-include/StandaloneCommon.ini.example AND/OR have discussions on what would be more sane default options. There's a chance that this could affect anyone, so don't neglect to take a look at it. You may also notice messages on your console and in your logs like: 21:56:29 - [LOGINDOSPROTECTION]: client: 192.168.1.213 is blocked for 12 milliseconds, X-ForwardedForAllowed status is False, endpoint:192.168.1.213 This
Re: [Opensim-dev] Rate Limit DOS Protection
I had a look at the Robust.ini.example earlier and I found the instructions to be clear and straight forward, I am glad you did infact allow this to be completely disabled, at OSgrid we use nginx as a reverse proxy and also to do DoS protection and load balancing, without the options to disable this completely it would have been a major problem moving forward. On Tue, Oct 8, 2013 at 12:27 PM, Teravus Ovares tera...@gmail.com wrote: Just to be clear, and comment/answer some questions. I did add the DOS protection variables to the Robust.ini.example in the [LoginService] section. The Rate Limit code does 'per connection' velocity counts. The default rate setting is 5 requests in 10 seconds.If you have a transparent proxy, such as squid, you need to set DOSAllowXForwardedForHeader to true so that the code trusts the X-Forwarded-For header that your proxy adds to the headers.If you want to turn it off, set DOSMaxRequestsInTimeFrame = 0. If you want to allow individual clients to do 20 connections in 5 seconds, set DOSMaxRequestsInTimeFrame = 20 and DOSRequestTimeFrameMS = 5000. If you want to change how long individual connections are blocked when they go over the rate limit, change DOSForgiveClientAfterMS. Hopefully this helps get you started with the config options. One more thing, this DOS protector is configured and implemented 'per service',So, if it's implemented in the login service, it's not going to get triggered by the Group Service. If there's DOS protection on the Group Service and the login service, and a connection gets blocked on the login service, they'll still be able to access the Group Service because they're individually rate limited... This is for flexibility.Choosing what the best rate limit is cannot really be done server wide, it has to be based on the needs of the individual service and that's why it was implemented this way. At this point, the only major service that has rate limiting on by default is the login service. I'll be happy to answer more questions and discuss default settings. Best Regards Teravus On Tue, Oct 8, 2013 at 8:04 AM, James Stallings II james.stalli...@gmail.com wrote: As I have often been told by you, Melanie, if I am upgrading and not auditing, adjusting, and testing my configs in accordance with changes, I'm doing it wrong. I think that statement probably applies more to those running larger concerns than anyone else. Honestly, Teravus typically does a better job of coding than to produce work that does not take such matters of scale into consideration. I'm comfortable with whatever he chooses to do, and if it turns out it isn't a case of 'one size fits all', I'll make adjustments accordingly. On Tue, Oct 8, 2013 at 7:52 AM, Melanie mela...@t-data.com wrote: I'm worried that people with larger installations will see service failures because legit traffic is seen as abusive. This could cause issues for the lerger grids out there. I don't believe that whatever tenuous protection this may offer for small grids and standalones outwieghs the potential service impairment it may cause for unsuspecting larger grids. Not every grid operator reads this list, So I'd again suggest that we stick to the way we've always done it and make the default for new features be off. Melanie On 8 Oct 2013, at 09:31, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.I have, both as an example, and good practice, applied the Rate limit protection to the login service.There are new Configuration options in StandaloneCommon.ini and Robust.ini that control how the connections are rate limited and if trusts the X-Forwarded-For header.Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new
Re: [Opensim-dev] Rate Limit DOS Protection
Yes, just confirmed it: http://www.networkinghowtos.com/howto/set-the-x-forwarded-for-header-on-a-nginx-reverse-proxy-setup/ Documentation on how to set up X-Forwarded-For in nginx Best Regards Teravus On Tue, Oct 8, 2013 at 12:10 PM, Teravus Ovares tera...@gmail.com wrote: I'm sure you could configure nginx to send the X-Forwarded-For header to the OpenSim based service and then enable the DOSAllowXForwardedForHeader in the robust.ini to receive the power of the 'per service' rate limiting while also having the protection at the proxy level.That said, I'm happy you found a configuration that works for you. Best Regards Teravus On Tue, Oct 8, 2013 at 11:43 AM, Michael Emory Cerquoni nebadon2...@gmail.com wrote: I had a look at the Robust.ini.example earlier and I found the instructions to be clear and straight forward, I am glad you did infact allow this to be completely disabled, at OSgrid we use nginx as a reverse proxy and also to do DoS protection and load balancing, without the options to disable this completely it would have been a major problem moving forward. On Tue, Oct 8, 2013 at 12:27 PM, Teravus Ovares tera...@gmail.com wrote: Just to be clear, and comment/answer some questions. I did add the DOS protection variables to the Robust.ini.example in the [LoginService] section. The Rate Limit code does 'per connection' velocity counts. The default rate setting is 5 requests in 10 seconds.If you have a transparent proxy, such as squid, you need to set DOSAllowXForwardedForHeader to true so that the code trusts the X-Forwarded-For header that your proxy adds to the headers.If you want to turn it off, set DOSMaxRequestsInTimeFrame = 0. If you want to allow individual clients to do 20 connections in 5 seconds, set DOSMaxRequestsInTimeFrame = 20 and DOSRequestTimeFrameMS = 5000. If you want to change how long individual connections are blocked when they go over the rate limit, change DOSForgiveClientAfterMS. Hopefully this helps get you started with the config options. One more thing, this DOS protector is configured and implemented 'per service',So, if it's implemented in the login service, it's not going to get triggered by the Group Service. If there's DOS protection on the Group Service and the login service, and a connection gets blocked on the login service, they'll still be able to access the Group Service because they're individually rate limited... This is for flexibility.Choosing what the best rate limit is cannot really be done server wide, it has to be based on the needs of the individual service and that's why it was implemented this way. At this point, the only major service that has rate limiting on by default is the login service. I'll be happy to answer more questions and discuss default settings. Best Regards Teravus On Tue, Oct 8, 2013 at 8:04 AM, James Stallings II james.stalli...@gmail.com wrote: As I have often been told by you, Melanie, if I am upgrading and not auditing, adjusting, and testing my configs in accordance with changes, I'm doing it wrong. I think that statement probably applies more to those running larger concerns than anyone else. Honestly, Teravus typically does a better job of coding than to produce work that does not take such matters of scale into consideration. I'm comfortable with whatever he chooses to do, and if it turns out it isn't a case of 'one size fits all', I'll make adjustments accordingly. On Tue, Oct 8, 2013 at 7:52 AM, Melanie mela...@t-data.com wrote: I'm worried that people with larger installations will see service failures because legit traffic is seen as abusive. This could cause issues for the lerger grids out there. I don't believe that whatever tenuous protection this may offer for small grids and standalones outwieghs the potential service impairment it may cause for unsuspecting larger grids. Not every grid operator reads this list, So I'd again suggest that we stick to the way we've always done it and make the default for new features be off. Melanie On 8 Oct 2013, at 09:31, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On
Re: [Opensim-dev] Rate Limit DOS Protection
I'm sure you could configure nginx to send the X-Forwarded-For header to the OpenSim based service and then enable the DOSAllowXForwardedForHeader in the robust.ini to receive the power of the 'per service' rate limiting while also having the protection at the proxy level.That said, I'm happy you found a configuration that works for you. Best Regards Teravus On Tue, Oct 8, 2013 at 11:43 AM, Michael Emory Cerquoni nebadon2...@gmail.com wrote: I had a look at the Robust.ini.example earlier and I found the instructions to be clear and straight forward, I am glad you did infact allow this to be completely disabled, at OSgrid we use nginx as a reverse proxy and also to do DoS protection and load balancing, without the options to disable this completely it would have been a major problem moving forward. On Tue, Oct 8, 2013 at 12:27 PM, Teravus Ovares tera...@gmail.com wrote: Just to be clear, and comment/answer some questions. I did add the DOS protection variables to the Robust.ini.example in the [LoginService] section. The Rate Limit code does 'per connection' velocity counts. The default rate setting is 5 requests in 10 seconds.If you have a transparent proxy, such as squid, you need to set DOSAllowXForwardedForHeader to true so that the code trusts the X-Forwarded-For header that your proxy adds to the headers.If you want to turn it off, set DOSMaxRequestsInTimeFrame = 0. If you want to allow individual clients to do 20 connections in 5 seconds, set DOSMaxRequestsInTimeFrame = 20 and DOSRequestTimeFrameMS = 5000. If you want to change how long individual connections are blocked when they go over the rate limit, change DOSForgiveClientAfterMS. Hopefully this helps get you started with the config options. One more thing, this DOS protector is configured and implemented 'per service',So, if it's implemented in the login service, it's not going to get triggered by the Group Service. If there's DOS protection on the Group Service and the login service, and a connection gets blocked on the login service, they'll still be able to access the Group Service because they're individually rate limited... This is for flexibility.Choosing what the best rate limit is cannot really be done server wide, it has to be based on the needs of the individual service and that's why it was implemented this way. At this point, the only major service that has rate limiting on by default is the login service. I'll be happy to answer more questions and discuss default settings. Best Regards Teravus On Tue, Oct 8, 2013 at 8:04 AM, James Stallings II james.stalli...@gmail.com wrote: As I have often been told by you, Melanie, if I am upgrading and not auditing, adjusting, and testing my configs in accordance with changes, I'm doing it wrong. I think that statement probably applies more to those running larger concerns than anyone else. Honestly, Teravus typically does a better job of coding than to produce work that does not take such matters of scale into consideration. I'm comfortable with whatever he chooses to do, and if it turns out it isn't a case of 'one size fits all', I'll make adjustments accordingly. On Tue, Oct 8, 2013 at 7:52 AM, Melanie mela...@t-data.com wrote: I'm worried that people with larger installations will see service failures because legit traffic is seen as abusive. This could cause issues for the lerger grids out there. I don't believe that whatever tenuous protection this may offer for small grids and standalones outwieghs the potential service impairment it may cause for unsuspecting larger grids. Not every grid operator reads this list, So I'd again suggest that we stick to the way we've always done it and make the default for new features be off. Melanie On 8 Oct 2013, at 09:31, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.
Re: [Opensim-dev] Rate Limit DOS Protection
The login server should only ever see one connection per client, any more is most likely a DOS. Services that are called by regions are way more critical, as they get all requests from a few entities only. The inventory service, for instance, can't be rate limited because a single login creates a slew of requests. Also, it should not normally be accessible to the world. Melanie On 8 Oct 2013, at 18:27, Teravus Ovares tera...@gmail.com wrote: Just to be clear, and comment/answer some questions. I did add the DOS protection variables to the Robust.ini.example in the [LoginService] section. The Rate Limit code does 'per connection' velocity counts. The default rate setting is 5 requests in 10 seconds.If you have a transparent proxy, such as squid, you need to set DOSAllowXForwardedForHeader to true so that the code trusts the X-Forwarded-For header that your proxy adds to the headers.If you want to turn it off, set DOSMaxRequestsInTimeFrame = 0. If you want to allow individual clients to do 20 connections in 5 seconds, set DOSMaxRequestsInTimeFrame = 20 and DOSRequestTimeFrameMS = 5000. If you want to change how long individual connections are blocked when they go over the rate limit, change DOSForgiveClientAfterMS. Hopefully this helps get you started with the config options. One more thing, this DOS protector is configured and implemented 'per service',So, if it's implemented in the login service, it's not going to get triggered by the Group Service. If there's DOS protection on the Group Service and the login service, and a connection gets blocked on the login service, they'll still be able to access the Group Service because they're individually rate limited... This is for flexibility.Choosing what the best rate limit is cannot really be done server wide, it has to be based on the needs of the individual service and that's why it was implemented this way. At this point, the only major service that has rate limiting on by default is the login service. I'll be happy to answer more questions and discuss default settings. Best Regards Teravus On Tue, Oct 8, 2013 at 8:04 AM, James Stallings II james.stalli...@gmail.com wrote: As I have often been told by you, Melanie, if I am upgrading and not auditing, adjusting, and testing my configs in accordance with changes, I'm doing it wrong. I think that statement probably applies more to those running larger concerns than anyone else. Honestly, Teravus typically does a better job of coding than to produce work that does not take such matters of scale into consideration. I'm comfortable with whatever he chooses to do, and if it turns out it isn't a case of 'one size fits all', I'll make adjustments accordingly. On Tue, Oct 8, 2013 at 7:52 AM, Melanie mela...@t-data.com wrote: I'm worried that people with larger installations will see service failures because legit traffic is seen as abusive. This could cause issues for the lerger grids out there. I don't believe that whatever tenuous protection this may offer for small grids and standalones outwieghs the potential service impairment it may cause for unsuspecting larger grids. Not every grid operator reads this list, So I'd again suggest that we stick to the way we've always done it and make the default for new features be off. Melanie On 8 Oct 2013, at 09:31, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.I have, both as an example, and good practice, applied the Rate limit protection to the login service.There are new Configuration options in StandaloneCommon.ini and Robust.ini that control how the connections are rate limited and if trusts the X-Forwarded-For header.Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new configuration options in the [LoginService] section of your bin/Robust.ini.example and
Re: [Opensim-dev] Rate Limit DOS Protection
LoginService, Very much agreed. Though, I figured I'd give people 5 attempts in 10 seconds before sending them on their way in case they improperly type their password a few times quickly. Inventory Service, Also very much agreed. There is no DOS protection on that service at the moment. That's why I figured that this was pretty sane and pretty closely matches the functionality that people expect while still providing reasonable protection.There are two things that I can think of that may get snagged by the DOS protection.. usage of pcampbot for load testing. Clearly that's a rapid login of many clients and is similar to a DOS attack in many ways. And people with reverse/transparent proxies.That was the reason that I set up the 'AllowXForwardedFor' option. It's off by default now because if it's on by default, the DOS protection is easily gamed with the right header. It can be turned on and the issue goes away for those users. Best Regards Teravus On Tue, Oct 8, 2013 at 3:05 PM, Melanie mela...@t-data.com wrote: The login server should only ever see one connection per client, any more is most likely a DOS. Services that are called by regions are way more critical, as they get all requests from a few entities only. The inventory service, for instance, can't be rate limited because a single login creates a slew of requests. Also, it should not normally be accessible to the world. Melanie On 8 Oct 2013, at 18:27, Teravus Ovares tera...@gmail.com wrote: Just to be clear, and comment/answer some questions. I did add the DOS protection variables to the Robust.ini.example in the [LoginService] section. The Rate Limit code does 'per connection' velocity counts. The default rate setting is 5 requests in 10 seconds.If you have a transparent proxy, such as squid, you need to set DOSAllowXForwardedForHeader to true so that the code trusts the X-Forwarded-For header that your proxy adds to the headers.If you want to turn it off, set DOSMaxRequestsInTimeFrame = 0. If you want to allow individual clients to do 20 connections in 5 seconds, set DOSMaxRequestsInTimeFrame = 20 and DOSRequestTimeFrameMS = 5000. If you want to change how long individual connections are blocked when they go over the rate limit, change DOSForgiveClientAfterMS. Hopefully this helps get you started with the config options. One more thing, this DOS protector is configured and implemented 'per service',So, if it's implemented in the login service, it's not going to get triggered by the Group Service. If there's DOS protection on the Group Service and the login service, and a connection gets blocked on the login service, they'll still be able to access the Group Service because they're individually rate limited... This is for flexibility.Choosing what the best rate limit is cannot really be done server wide, it has to be based on the needs of the individual service and that's why it was implemented this way. At this point, the only major service that has rate limiting on by default is the login service. I'll be happy to answer more questions and discuss default settings. Best Regards Teravus On Tue, Oct 8, 2013 at 8:04 AM, James Stallings II james.stalli...@gmail.com wrote: As I have often been told by you, Melanie, if I am upgrading and not auditing, adjusting, and testing my configs in accordance with changes, I'm doing it wrong. I think that statement probably applies more to those running larger concerns than anyone else. Honestly, Teravus typically does a better job of coding than to produce work that does not take such matters of scale into consideration. I'm comfortable with whatever he chooses to do, and if it turns out it isn't a case of 'one size fits all', I'll make adjustments accordingly. On Tue, Oct 8, 2013 at 7:52 AM, Melanie mela...@t-data.com wrote: I'm worried that people with larger installations will see service failures because legit traffic is seen as abusive. This could cause issues for the lerger grids out there. I don't believe that whatever tenuous protection this may offer for small grids and standalones outwieghs the potential service impairment it may cause for unsuspecting larger grids. Not every grid operator reads this list, So I'd again suggest that we stick to the way we've always done it and make the default for new features be off. Melanie On 8 Oct 2013, at 09:31, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying. It's hard to argue to leave people unprotected from attacks, though.I'm certainly open to making the defaults less protective, and, I'm concerned enough about it that I'd prefer to leave some protection in place there. What are your thoughts on that? Best Regards Teravus On Tue, Oct 8, 2013 at 12:41 AM, Melanie mela...@t-data.com wrote: Hi, in keeping with our SOP, the defaults provided should be emulating the
Re: [Opensim-dev] Rate Limit DOS Protection
Another note, I can probably expand the DOS protection class to identify X sessions and establish a maximum number of sessions pretty easily and with really low overhead.I have not yet.. just X requests in Y seconds and Z seconds to unblock. I may do that later this week. (but won't have it active by default) Best Regards Teravus On Tue, Oct 8, 2013 at 3:23 PM, Teravus Ovares tera...@gmail.com wrote: LoginService, Very much agreed. Though, I figured I'd give people 5 attempts in 10 seconds before sending them on their way in case they improperly type their password a few times quickly. Inventory Service, Also very much agreed. There is no DOS protection on that service at the moment. That's why I figured that this was pretty sane and pretty closely matches the functionality that people expect while still providing reasonable protection.There are two things that I can think of that may get snagged by the DOS protection.. usage of pcampbot for load testing. Clearly that's a rapid login of many clients and is similar to a DOS attack in many ways. And people with reverse/transparent proxies.That was the reason that I set up the 'AllowXForwardedFor' option. It's off by default now because if it's on by default, the DOS protection is easily gamed with the right header. It can be turned on and the issue goes away for those users. Best Regards Teravus On Tue, Oct 8, 2013 at 3:05 PM, Melanie mela...@t-data.com wrote: The login server should only ever see one connection per client, any more is most likely a DOS. Services that are called by regions are way more critical, as they get all requests from a few entities only. The inventory service, for instance, can't be rate limited because a single login creates a slew of requests. Also, it should not normally be accessible to the world. Melanie On 8 Oct 2013, at 18:27, Teravus Ovares tera...@gmail.com wrote: Just to be clear, and comment/answer some questions. I did add the DOS protection variables to the Robust.ini.example in the [LoginService] section. The Rate Limit code does 'per connection' velocity counts. The default rate setting is 5 requests in 10 seconds.If you have a transparent proxy, such as squid, you need to set DOSAllowXForwardedForHeader to true so that the code trusts the X-Forwarded-For header that your proxy adds to the headers.If you want to turn it off, set DOSMaxRequestsInTimeFrame = 0. If you want to allow individual clients to do 20 connections in 5 seconds, set DOSMaxRequestsInTimeFrame = 20 and DOSRequestTimeFrameMS = 5000. If you want to change how long individual connections are blocked when they go over the rate limit, change DOSForgiveClientAfterMS. Hopefully this helps get you started with the config options. One more thing, this DOS protector is configured and implemented 'per service',So, if it's implemented in the login service, it's not going to get triggered by the Group Service. If there's DOS protection on the Group Service and the login service, and a connection gets blocked on the login service, they'll still be able to access the Group Service because they're individually rate limited... This is for flexibility.Choosing what the best rate limit is cannot really be done server wide, it has to be based on the needs of the individual service and that's why it was implemented this way. At this point, the only major service that has rate limiting on by default is the login service. I'll be happy to answer more questions and discuss default settings. Best Regards Teravus On Tue, Oct 8, 2013 at 8:04 AM, James Stallings II james.stalli...@gmail.com wrote: As I have often been told by you, Melanie, if I am upgrading and not auditing, adjusting, and testing my configs in accordance with changes, I'm doing it wrong. I think that statement probably applies more to those running larger concerns than anyone else. Honestly, Teravus typically does a better job of coding than to produce work that does not take such matters of scale into consideration. I'm comfortable with whatever he chooses to do, and if it turns out it isn't a case of 'one size fits all', I'll make adjustments accordingly. On Tue, Oct 8, 2013 at 7:52 AM, Melanie mela...@t-data.com wrote: I'm worried that people with larger installations will see service failures because legit traffic is seen as abusive. This could cause issues for the lerger grids out there. I don't believe that whatever tenuous protection this may offer for small grids and standalones outwieghs the potential service impairment it may cause for unsuspecting larger grids. Not every grid operator reads this list, So I'd again suggest that we stick to the way we've always done it and make the default for new features be off. Melanie On 8 Oct 2013, at 09:31, Teravus Ovares tera...@gmail.com wrote: I understand what you're saying.
Re: [Opensim-dev] Rate Limit DOS Protection
People like us have the protection in the proxy and need to turn the feature off. Not a problem for me, I'm aware of it, but my concern was for those who didn't listen to me tell them Dont' blindly reuse configs Melanie ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
On 10/8/13 5:20 PM, Melanie wrote: People like us have the protection in the proxy and need to turn the feature off. Not a problem for me, I'm aware of it, but my concern was for those who didn't listen to me tell them Dont' blindly reuse configs If they are too stupid or foolish or arrogant ( or some combination thereof), to follow that excellent advice, then they deserve what they get and you shouldn't shed a single tear for them. -ste (smxy) ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Rate Limit DOS Protection
Hi, in keeping with our SOP, the defaults provided should be emulating the previous behavior, e.g. NO rate limiting. I would much appreciate if that procedure would be adhered to, unless we vote to abandon it. Users could suffer because they don't expect the default config to change on them. Cheers, Melanie On 08/10/2013 05:42, Teravus Ovares wrote: Hi there, I just wanted to inform -dev that I added some rate limiting DOS protection classes to use to protect your opensim based services from rapid calling. At the moment, this will be most noticeable in the Login Service.I have, both as an example, and good practice, applied the Rate limit protection to the login service.There are new Configuration options in StandaloneCommon.ini and Robust.ini that control how the connections are rate limited and if trusts the X-Forwarded-For header.Just for the sake of getting something up there, I set the defaults to something sane, however they may not work for everyone, so it may be wise to take a look at the new configuration options in the [LoginService] section of your bin/Robust.ini.example and /bin/config-include/StandaloneCommon.ini.example AND/OR have discussions on what would be more sane default options. There's a chance that this could affect anyone, so don't neglect to take a look at it. You may also notice messages on your console and in your logs like: 21:56:29 - [LOGINDOSPROTECTION]: client: 192.168.1.213 is blocked for 12 milliseconds, X-ForwardedForAllowed status is False, endpoint:192.168.1.213 This is an example of the DOS Protection blocking a connection because the client went beyond the rate limit. The rate limit is defined by X requests in Y period of time and is implemented in a rolling Y fashion. It also has a 'forget' period of time that will unblock the blocked user. At this point, there's one implemented for XMLRPC handlers, one for GenericHTTPHandlers and a base class for StreamHandlers based on BaseStreamHandler. If you are interested in the code changes, you can check the diff: http://opensimulator.org/viewgit/?a=commitdiffp=opensimh=f76cc6036ebf446553ee5201321879538dafe3b2 There's still more to do, and, here's a start to providing some modicum of protection on the services. If you have any questions, feel free to reply and ask.. or send me an e-mail personally. Thanks and Best Regards Teravus ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
