Re: [anti-abuse-wg] Question about spam to abuse inbox
Cynthia Revström writes: > > It seems to me that if your abuse@ email is being overloaded and > you are unable to keep your network spam free, then you shouldn't be > taking on any more customers until you figure things out. > > As has been noted before in this thread, just because you are getting > 200 abuse emails in a day doesn't necessarily mean you have a huge > issue but it is a lot of emails to deal with. > It might just be one customer who port scanned a /24 somewhere on the > internet. Right. However, as also mentioned, it shouldn't be that hard to group them by IP (which would be a finer granularity than per customer), even for unstructured mails. And, if they are all actually the same issue, they should be very quickly to process, as they all refer to the same incident (quickly per report, I admit it may still take 2-3 hours to clean that inbox). Plus, given the low value of abuse reported, for receiving 200 complaints I expect the actions from the customer account would be of at least an order of magnitude more than that you received complaints about. Probably much more. Also important: how much time passed since first report to customer abuse stopping? how many reports refer to that initial window when you weren't aware of the abuse (by the customer itself, by those that compromised your customer, etc.) until you got notice of that (either from external reports or from your own monitoring) ? Earlier you mentioned taking a week to handle the incident reports. If the abuse continued for so long that would obviously affect more people and cause more complaints. It's true that not everyone reports immediately. Perhaps customer began abuse at t₀, got suspended at t + 2 hours, and yet you receive some complaints next day due to people aggregating their notifications daily (this means less notifications for you, but more delay in receiving them), but if the customer account continued rampant for days, that would obviously make you receive a lot more reports. > > Why do you think that because you tell yourself you are "too big" > that you don't need to monitor your network? > > I don't think anyone is saying that, but if you want a human to read > your emails, you shouldn't automate the sending so you end up with > potential situations like that. No. You should actually love automated reports. If Joe Abusehater automatically reported you every day a number of phishing links on your systems (for example, suppose you are Twitter and these are phishing links using your shortener), there's no problem in automatically processing their emails with e.g. a regex: > "Hello Cynthia,\nIt's Joe again. This time we detected a > (?Pphishing|scam|child pornography|malware|...) link on your > site at (?Phttps?://[^ ]+)) I would like you to take care of > that.\nThanks, Joe" A human read the email, then told the machine what it means and how to handle it. If there's an email the machine doesn't know how to handle, a human goes and takes a look. Now suppose Joe didn't automate sending you the email. He instead hires some sloppy operators. They sometimes use one text, sometimes a different one. From time to time, they forget to include the url, or don't specify the category (which, albeit probably not matching your own categorization, probably is still helpful). Note I'm not covering the quality of the information. In either cases, Joe notifications could generally be either good or bad. If you find Joe to provide reliable information, you may even want to trust their reports automatically. If they have a lot of noise, you probably will want to prioritize them at the bottom of your queue. > Don't assume people are lacking in basic knowledge, rather consider > that some people might have requirements other than yours, and that > it might not be as simple as you suggest. > > This also applies in most cases in this thread, just > because something works for you or might seem easy for you doesn't > mean it works for everyone in all situations. (I feel like this needs > to be said) > > -Cynthia Sadly, problems often lie at the management level, out of the hands of the technicians which suffer them. Still, it would be helpful to know about the requirements that make things so hard for your, as perhaps we could come up with some approach simplifying your processes. Best regards -- INCIBE-CERT - Spanish National CSIRT https://www.incibe-cert.es/ PGP keys: https://www.incibe-cert.es/en/what-is-incibe-cert/pgp-public-keys INCIBE-CERT is the Spanish National CSIRT designated for citizens, private law entities, other entities not included in the subjective scope of application of the "Ley 40/2015, de 1 de octubre, de Régimen Jurídico del Sector Público", as well as digital service providers, operators of essential services and critical operators under the terms of the "Real Decreto-ley 12/2018, de 7 de septiembre, de seguridad
Re: [anti-abuse-wg] Question about spam to abuse inbox
> It seems to me that if your abuse@ email is being overloaded and you are > unable to keep your network spam free, then you shouldn't be taking on any > more customers until you figure things out. great. should be no problem telling the people in management who wear shiny shoes that being socially correct is more important than money. that is what is called a career limiting move. randy
Re: [anti-abuse-wg] Question about spam to abuse inbox
> It seems to me that if your abuse@ email is being overloaded and you are unable to keep your network spam free, then you shouldn't be taking on any more customers until you figure things out. As has been noted before in this thread, just because you are getting 200 abuse emails in a day doesn't necessarily mean you have a huge issue but it is a lot of emails to deal with. It might just be one customer who port scanned a /24 somewhere on the internet. > Why do you think that because you tell yourself you are "too big" that you don't need to monitor your network? I don't think anyone is saying that, but if you want a human to read your emails, you shouldn't automate the sending so you end up with potential situations like that. > The first thing that comes to mind is having your abuse@ email checked via a script to eliminate any actual spam to your server and parse out legitimate emails and requests. As was mentioned in the second email in this thread (from Jordi), just using spam filters on content is not necessarily a good idea for the abuse inbox. > If you need help with something so basic or are asking these types of questions, you shouldn't be running a network in the first place. My ::slams keyboard:: > Yes, if you can't figure newbie stuff out then what the heck are you doing trying to host people on the internet? Don't assume people are lacking in basic knowledge, rather consider that some people might have requirements other than yours, and that it might not be as simple as you suggest. This also applies in most cases in this thread, just because something works for you or might seem easy for you doesn't mean it works for everyone in all situations. (I feel like this needs to be said) -Cynthia On Fri, Feb 26, 2021 at 10:02 PM steve payne wrote: > I don't agree with the "All these responses from people who don't actually > run a network ::slams keyboard::" remarks. > > It seems to me that if your abuse@ email is being overloaded and you are > unable to keep your network spam free, then you shouldn't be taking on any > more customers until you figure things out. > > Why do you think that because you tell yourself you are "too big" that you > don't need to monitor your network? > > If your abuse @ email is being slammed to death, it would seem you need to > optimize this solution. > > The first thing that comes to mind is having your abuse@ email checked > via a script to eliminate any actual spam to your server and parse out > legitimate emails and requests. > > If you need help with something so basic or are asking these types of > questions, you shouldn't be running a network in the first place. My > ::slams keyboard:: > > Yes, if you can't figure newbie stuff out then what the heck are you doing > trying to host people on the internet? > > On Fri, Feb 26, 2021 at 1:19 PM Jacob Slater wrote: > >> If you predicate sending reports via web form, then report forwarding >>> from the ISP to its customer should also be done via web form. >>> >> >> The relationship between an arbitrary internet user and an ISP is >> different from the relationship between an ISP and a customer who is on a >> contract. >> I can (and do) require end users of my infrastructure respond to abuse >> e-mails sent to them in specific ways. If they don't like the terms I've >> set, they are welcome to take their business elsewhere. >> The same relationship does not currently exist with abuse reports. >> >> At times I also try and >>> send fake complaints about my IP, to see if they would forward them to >>> me. All of those messages fall into a black black hole where time is >>> frozen expectations fade. Lazy. >>> >> >> It is also possible your ISP believes the report is fake and does not >> forward it on. Alternatively, perhaps their policy is to not forward >> reports on. They might investigate, deem it incorrect, and delete it. >> >> >> I personally am opposed to banning or discouraging web forms unless we >> standardize some system. If there is an expectation for human review on the >> ISP side, there should be an expectation that the sender is human. If we >> set an expectation for automated sending of abuse reports, limited machine >> review prior to acceptance should be expected. >> >> Solving this is a difficult problem. From my (admittedly limited) >> experience, I'm in agreement with Alex de Joode - a solution cannot impact >> certain operational realities of ISPs. Limited machine review - along with >> automation of abuse reports on the receiving side - is an operational >> reality. False, inaccurate, incomplete, or just plain malicious abuse >> reports are just as real as actual abuse reports. >> >> I would note a further operational reality: any standard we come up with >> outside of the current method of communication (email) is likely to never >> reach large-scale deployment. Even if we make a standard within e-mail (ex. >> ARF), some ISPs will want (or need) details beyond what would be outlined >>
Re: [anti-abuse-wg] Question about spam to abuse inbox
I don't agree with the "All these responses from people who don't actually run a network ::slams keyboard::" remarks. It seems to me that if your abuse@ email is being overloaded and you are unable to keep your network spam free, then you shouldn't be taking on any more customers until you figure things out. Why do you think that because you tell yourself you are "too big" that you don't need to monitor your network? If your abuse @ email is being slammed to death, it would seem you need to optimize this solution. The first thing that comes to mind is having your abuse@ email checked via a script to eliminate any actual spam to your server and parse out legitimate emails and requests. If you need help with something so basic or are asking these types of questions, you shouldn't be running a network in the first place. My ::slams keyboard:: Yes, if you can't figure newbie stuff out then what the heck are you doing trying to host people on the internet? On Fri, Feb 26, 2021 at 1:19 PM Jacob Slater wrote: > If you predicate sending reports via web form, then report forwarding >> from the ISP to its customer should also be done via web form. >> > > The relationship between an arbitrary internet user and an ISP is > different from the relationship between an ISP and a customer who is on a > contract. > I can (and do) require end users of my infrastructure respond to abuse > e-mails sent to them in specific ways. If they don't like the terms I've > set, they are welcome to take their business elsewhere. > The same relationship does not currently exist with abuse reports. > > At times I also try and >> send fake complaints about my IP, to see if they would forward them to >> me. All of those messages fall into a black black hole where time is >> frozen expectations fade. Lazy. >> > > It is also possible your ISP believes the report is fake and does not > forward it on. Alternatively, perhaps their policy is to not forward > reports on. They might investigate, deem it incorrect, and delete it. > > > I personally am opposed to banning or discouraging web forms unless we > standardize some system. If there is an expectation for human review on the > ISP side, there should be an expectation that the sender is human. If we > set an expectation for automated sending of abuse reports, limited machine > review prior to acceptance should be expected. > > Solving this is a difficult problem. From my (admittedly limited) > experience, I'm in agreement with Alex de Joode - a solution cannot impact > certain operational realities of ISPs. Limited machine review - along with > automation of abuse reports on the receiving side - is an operational > reality. False, inaccurate, incomplete, or just plain malicious abuse > reports are just as real as actual abuse reports. > > I would note a further operational reality: any standard we come up with > outside of the current method of communication (email) is likely to never > reach large-scale deployment. Even if we make a standard within e-mail (ex. > ARF), some ISPs will want (or need) details beyond what would be outlined > in the standard. This will inevitably require more non-standard human > interaction. > > Those who do not care to receive abuse reports will fail to respond to > them, regardless of what we decide here. > > - Slater > > > > > On Fri, Feb 26, 2021 at 3:57 PM Alessandro Vesely wrote: > >> On Thu 25/Feb/2021 14:41:00 +0100 Cynthia Revström wrote: >> >> > I think you have misunderstood my point. >> > >> >> Would they send such report using their customer's own web form? >> > >> > No? I don't know what implied that? >> >> >> If you predicate sending reports via web form, then report forwarding >> from the ISP to its customer should also be done via web form. That >> is, the ISP should jump all the required hoops until it finds out >> where and how to fill the appropriate form. However, doing so defeats >> the advantage of having the customer automatically identified. >> >> >> >> Yes, doing so requires some work too, but heck aren't we paying for >> that >> > already? >> > >> > The person sending the abuse report is rarely a paying customer. >> > >> >> The right thing to do would be to arrange for the abuse mailbox address >> > to point (in)directly to the actual user of the IP address. >> > >> > I am assuming you are referring to having a separate abuse contact for >> each >> > customer, so like abuse.cust123@domain and registering it in the RIPE >> > Registry/DB? >> >> >> Yes, exactly. That's the extra work required from the ISP. It is >> paid by cust123. Presumably, abuse.cust123@domain forwards to the >> abuse address chosen by the customer on signing the contract. Keeping >> a copy allows the ISP to monitor how many complaints its customers >> receive. >> >> >> > In some cases with large customers maybe but if you are a hosting >> provider >> > where each customer might only have one or two IPv4 addresses, that can >> get >> > to an insane
Re: [anti-abuse-wg] Question about spam to abuse inbox
> > If you predicate sending reports via web form, then report forwarding > from the ISP to its customer should also be done via web form. > The relationship between an arbitrary internet user and an ISP is different from the relationship between an ISP and a customer who is on a contract. I can (and do) require end users of my infrastructure respond to abuse e-mails sent to them in specific ways. If they don't like the terms I've set, they are welcome to take their business elsewhere. The same relationship does not currently exist with abuse reports. At times I also try and > send fake complaints about my IP, to see if they would forward them to > me. All of those messages fall into a black black hole where time is > frozen expectations fade. Lazy. > It is also possible your ISP believes the report is fake and does not forward it on. Alternatively, perhaps their policy is to not forward reports on. They might investigate, deem it incorrect, and delete it. I personally am opposed to banning or discouraging web forms unless we standardize some system. If there is an expectation for human review on the ISP side, there should be an expectation that the sender is human. If we set an expectation for automated sending of abuse reports, limited machine review prior to acceptance should be expected. Solving this is a difficult problem. From my (admittedly limited) experience, I'm in agreement with Alex de Joode - a solution cannot impact certain operational realities of ISPs. Limited machine review - along with automation of abuse reports on the receiving side - is an operational reality. False, inaccurate, incomplete, or just plain malicious abuse reports are just as real as actual abuse reports. I would note a further operational reality: any standard we come up with outside of the current method of communication (email) is likely to never reach large-scale deployment. Even if we make a standard within e-mail (ex. ARF), some ISPs will want (or need) details beyond what would be outlined in the standard. This will inevitably require more non-standard human interaction. Those who do not care to receive abuse reports will fail to respond to them, regardless of what we decide here. - Slater On Fri, Feb 26, 2021 at 3:57 PM Alessandro Vesely wrote: > On Thu 25/Feb/2021 14:41:00 +0100 Cynthia Revström wrote: > > > I think you have misunderstood my point. > > > >> Would they send such report using their customer's own web form? > > > > No? I don't know what implied that? > > > If you predicate sending reports via web form, then report forwarding > from the ISP to its customer should also be done via web form. That > is, the ISP should jump all the required hoops until it finds out > where and how to fill the appropriate form. However, doing so defeats > the advantage of having the customer automatically identified. > > > >> Yes, doing so requires some work too, but heck aren't we paying for that > > already? > > > > The person sending the abuse report is rarely a paying customer. > > > >> The right thing to do would be to arrange for the abuse mailbox address > > to point (in)directly to the actual user of the IP address. > > > > I am assuming you are referring to having a separate abuse contact for > each > > customer, so like abuse.cust123@domain and registering it in the RIPE > > Registry/DB? > > > Yes, exactly. That's the extra work required from the ISP. It is > paid by cust123. Presumably, abuse.cust123@domain forwards to the > abuse address chosen by the customer on signing the contract. Keeping > a copy allows the ISP to monitor how many complaints its customers > receive. > > > > In some cases with large customers maybe but if you are a hosting > provider > > where each customer might only have one or two IPv4 addresses, that can > get > > to an insane amount of handles and make the database really messy. > > > You can keep a record for each IPv4 address with only a few Terabytes. > > I don't think the reason why ISPs tend to neither assign rfc2317 > reverse delegations nor customer specific abuse-mailbox is because > they or the RIPE cannot afford enough disk space to store that data. > > Every now and then I ask my ISP to assign me an abuse-mailbox (which > my previous ISP did, but then they were acquired by a bigger shark > while the RIPE changed format to abuse-c.) At times I also try and > send fake complaints about my IP, to see if they would forward them to > me. All of those messages fall into a black black hole where time is > frozen expectations fade. Lazy. > > > > Also the customer in question is not the only info that is relevant, like > > is it DoS, spam, or port scanning, etc? > > > > But in general I think there are pros and cons to web forms and email > > templates just as there are pros and cons to arbitrarily structured > emails. > > > For a third alternative, I recently added abuseipdb.com to my > end-of-day abuse reporting script. They provide an http API that >
Re: [anti-abuse-wg] Question about spam to abuse inbox
On Thu 25/Feb/2021 14:41:00 +0100 Cynthia Revström wrote: I think you have misunderstood my point. Would they send such report using their customer's own web form? No? I don't know what implied that? If you predicate sending reports via web form, then report forwarding from the ISP to its customer should also be done via web form. That is, the ISP should jump all the required hoops until it finds out where and how to fill the appropriate form. However, doing so defeats the advantage of having the customer automatically identified. Yes, doing so requires some work too, but heck aren't we paying for that already? The person sending the abuse report is rarely a paying customer. The right thing to do would be to arrange for the abuse mailbox address to point (in)directly to the actual user of the IP address. I am assuming you are referring to having a separate abuse contact for each customer, so like abuse.cust123@domain and registering it in the RIPE Registry/DB? Yes, exactly. That's the extra work required from the ISP. It is paid by cust123. Presumably, abuse.cust123@domain forwards to the abuse address chosen by the customer on signing the contract. Keeping a copy allows the ISP to monitor how many complaints its customers receive. In some cases with large customers maybe but if you are a hosting provider where each customer might only have one or two IPv4 addresses, that can get to an insane amount of handles and make the database really messy. You can keep a record for each IPv4 address with only a few Terabytes. I don't think the reason why ISPs tend to neither assign rfc2317 reverse delegations nor customer specific abuse-mailbox is because they or the RIPE cannot afford enough disk space to store that data. Every now and then I ask my ISP to assign me an abuse-mailbox (which my previous ISP did, but then they were acquired by a bigger shark while the RIPE changed format to abuse-c.) At times I also try and send fake complaints about my IP, to see if they would forward them to me. All of those messages fall into a black black hole where time is frozen expectations fade. Lazy. Also the customer in question is not the only info that is relevant, like is it DoS, spam, or port scanning, etc? But in general I think there are pros and cons to web forms and email templates just as there are pros and cons to arbitrarily structured emails. For a third alternative, I recently added abuseipdb.com to my end-of-day abuse reporting script. They provide an http API that allows to specify the most basic info, IP, time, and kind of abuse. That method has some of the advantages of forms, as it allows semantically bound fields, and some advantages of email messages, as it can be automated. Best Ale --
