Rate limit the httpd web server for signup requests
I already do some rate limiting with stateful tracking options for PF, which works really great for the stuff I use it for. I also use block lists of known bad IP addresses etc. But what useful methods exists that prevent spamming a HTML signup form from stuffing the database with useless signups? Naturally the accounts that haven't been validated one way or another gets deleted, but the initial signup is a problem as thousands upon thousands of requests are stored before deletion. I have tried blocking by IP, but this is difficult as the IP changes faster than it can be blocked. The User Agent is spoofed with random garbage. Honey pot empty hidden fields gets detected and ignored. Randomly generated form IDs that gets submitted and validated using a session cookie also doesn't work as the cookie is just stored and then send along. A simple CAPTCHA reduces some of the irrelevant noise, but the more sophisticated bots solves the CAPTCHA. Using Cloudflare's or Google's CAPTCHA is frowned upon by the real users, which I fully understand. So I was wondering, if some other clever method can reduce the noise?
Re: Rate limit the httpd web server for signup requests
On 2024-06-11 15:41, Martin wrote: I already do some rate limiting with stateful tracking options for PF, which works really great for the stuff I use it for. I also use block lists of known bad IP addresses etc. But what useful methods exists that prevent spamming a HTML signup form from stuffing the database with useless signups? Naturally the accounts that haven't been validated one way or another gets deleted, but the initial signup is a problem as thousands upon thousands of requests are stored before deletion. I have tried blocking by IP, but this is difficult as the IP changes faster than it can be blocked. The User Agent is spoofed with random garbage. Honey pot empty hidden fields gets detected and ignored. Randomly generated form IDs that gets submitted and validated using a session cookie also doesn't work as the cookie is just stored and then send along. A simple CAPTCHA reduces some of the irrelevant noise, but the more sophisticated bots solves the CAPTCHA. Using Cloudflare's or Google's CAPTCHA is frowned upon by the real users, which I fully understand. So I was wondering, if some other clever method can reduce the noise? I haven't tried it and I'm not sure how useful it is in your case, but mCaptcha uses proof of work. https://mcaptcha.org/ Paul
Re: Rate limit the httpd web server for signup requests
No perfect solution exists, but the following may help. 1) Parse the logs of your web application and ban any IP that attempts to create multiple accounts. Not great because you may have multiple users sharing the same public IP. It only works ok if you automate it via cronjob scripts. 2) Require the user to provide an external means of identification (such as an email address or a phone number) whose existence must be verified before his account is activated. Not great because bots may use disposable addresses/numbers, it delays the activation for legitimate users, and it requires more effort to implement than 1). 3) Alternative capchas. If your site is not that big of a target, you can get away with some naive captcha (such as Captcheck) without annoying your users too much. The problem is (as you have already noticed) naive captchas are not that hard to break for persistent bots; meanwhile complex captchas are bad for users. Maybe try different capctha solutions until you find one that sticks. Martin wrote: But what useful methods exists that prevent spamming a HTML signup form from stuffing the database with useless signups?
Re: Rate limit the httpd web server for signup requests
Jun 12, 2024 00:56:47 Martin : > A simple CAPTCHA reduces some of the irrelevant noise, but the more > sophisticated bots solves the CAPTCHA. > > Using Cloudflare's or Google's CAPTCHA is frowned upon by the real > users, which I fully understand. > > So I was wondering, if some other clever method can reduce the noise? Testing and knowing all the possible solutions to fight "3rd kind intelligence spammers" seems a little overwhelming to me, it appears to me like the story to search for a firewall that solve every security problem. Indeed just consider that a parameter of curl allow you to simulate any post submission. And the problem eventually could be exactly this, the why of the existance of these advanced clients tools. [ ... ] For now, I just implememted my own captcha asking to solve a simple math and that eventually can be enriched for future research.. Some simple php code, easy stuff, happy to share it. -Dan
Re: Rate limit the httpd web server for signup requests
On Tue, Jun 11, 2024 at 10:41:33PM +, Martin wrote: > I already do some rate limiting with stateful tracking options for PF, > which works really great for the stuff I use it for. > > I also use block lists of known bad IP addresses etc. > > But what useful methods exists that prevent spamming a HTML signup form > from stuffing the database with useless signups? > > Naturally the accounts that haven't been validated one way or another > gets deleted, but the initial signup is a problem as thousands upon > thousands of requests are stored before deletion. > > I have tried blocking by IP, but this is difficult as the IP changes > faster than it can be blocked. > > The User Agent is spoofed with random garbage. > > Honey pot empty hidden fields gets detected and ignored. > > Randomly generated form IDs that gets submitted and validated using a > session cookie also doesn't work as the cookie is just stored and then > send along. > > A simple CAPTCHA reduces some of the irrelevant noise, but the more > sophisticated bots solves the CAPTCHA. > > Using Cloudflare's or Google's CAPTCHA is frowned upon by the real > users, which I fully understand. > > So I was wondering, if some other clever method can reduce the noise? > It's not perfect, but I have a long list of regexes that I know are spam that I have my Perl code that processes the form block. Trying to block from a log is not very helpful. It can let through thousands of the same spam attempts before the log catches up to the attempts reaching the log, which is a pretty long time. -- Regards, Chris Bennett "Who controls the past controls the future. Who controls the present controls the past." George Orwell - 1984
Re: Rate limit the httpd web server for signup requests
On 6/12/24 10:32 PM, Chris Bennett wrote: It's not perfect, but I have a long list of regexes that I know are spam that I have my Perl code that processes the form block. Trying to block from a log is not very helpful. It can let through thousands of the same spam attempts before the log catches up to the attempts reaching the log, which is a pretty long time. I was just wondering if you've tried requiring email or SMS link to unique signup URL? Thank you, Paul
Re: Rate limit the httpd web server for signup requests
On Thu, Jun 13, 2024 at 04:30:27AM -0700, Paul Pace wrote: > On 6/12/24 10:32 PM, Chris Bennett wrote: > > It's not perfect, but I have a long list of regexes that I know are spam > > that I have my Perl code that processes the form block. Trying to block > > from a log is not very helpful. It can let through thousands of the same > > spam attempts before the log catches up to the attempts reaching the log, > > which is a pretty long time. > > I was just wondering if you've tried requiring email or SMS link to unique > signup URL? > If it's a form to make a payment, it just gets sent off to another site. If it's a general contact us form, I just try to keep the spam down to a bearable level. Every couple of months I add new regexes. Speaking of which, I really need to do that now. Form spam is now annoying again. I do use an email address for each form and nothing else. That way I can just pick a day to wade through the trash. -- Regards, Chris Bennett "Who controls the past controls the future. Who controls the present controls the past." George Orwell - 1984
Re: Rate limit the httpd web server for signup requests
On Tue, Jun 11, 2024 at 10:41:33PM +, Martin wrote: > But what useful methods exists that prevent spamming a HTML signup form > from stuffing the database with useless signups? > > Naturally the accounts that haven't been validated one way or another > gets deleted, but the initial signup is a problem as thousands upon > thousands of requests are stored before deletion. Out of curiosity, what exactly is the issue here? >From what you've told us it seems as if the genuine signups are being validated by some external means, (such as a confirmation email loop or sms which doesn't require manual intervention by you), and fake signups are stored for a temporary period before being deleted automatically. So the intended result is being achieved. Anything you add to detect fake signups from being submitted is almost guaranteed to have a negative effect on some or all of your genuine users. Why is it a problem to have 'thousands' of requests stored temporarily, if those are later deleted? Is this database using a lot of storage? Is parsing it using a lot of CPU? If that's the issue then presumably either your code is very inefficient or you're using underpowered hardware.
