[anti-abuse-wg] [db-wg] Fw: NWI reviews: NWI-1 staying on top of abuse contact changes
Hi Lutz Good try but this script does not do it. Firstly, if you start with the allocation object it needs to be -rM in the first query. Then you are only looking for the "org:" attribute. The INET(6)NUM object may have an "abuse-c:" attribute in the object. It may have both in which case the local attribute overrides the "abuse-c:" in the referenced ORGANISATION object. Also to manage this feature it is not just about producing a list of abuse email addresses. It is knowing where these are referenced within a hierarchy and knowing which parts of the hierarchy are covered by which addresses. I agree that most resources will only have a single abuse contact referenced from their single ORGANISATION object. However, we were pushed hard by the community to allow "abuse-c:" to be added to the INET(6)NUM objects. I assume therefore that this feature has been used. So we already have a mix of abuse contacts referenced from both ORGANISATION and resource objects. When this was asked for the emphasis was on 'ease of use' to add these contacts. Probably no one gave much thought to how these contacts will be cleaned up when no longer relevant. It's most likely that someone will ask someone to remove it. Knowing human nature it is more likely some will just be forgotten about. If the contact is still there it is still active, but maybe no one is reading that email address any longer. I am not an operator so I don't know how many end user customers in large networks handle their own abuse reports. Or how many sub allocations handle abuse with their own customers handling abuse. So I don't know how complex this will get and maybe all operators already have good procedures in place to manage all this. That is why I am asking the question...is such a visualisation tool needed, useful or a waste of time? As an analyst, I can only see how complex this 'could' become over time. Unless we go back to the drawing board and redesign the abuse contact system (again) this is not going to be simplified any time soon. cheersdenis co-chair DB-WG On Thursday, 24 September 2020, 15:32:09 CEST, Lutz Donnerhacke via db-wg wrote: > > We need to take some action on these old NWIs. Either we move forward > with them or we cancel them. It is difficult to draw a consensus on 2 > comments. Can you please give us a couple of minutes of your time and > let us know if this NWI-1 is needed, useful or a waste of time. > > I share the view already expressed. If we need a tool, maybe the problem > is somewhere else. I support any proposal, that comes to the same result as the ripe whois database tool. It's not really hard to do correctly. #! /bin/sh getripe="whois -h whois.ripe.net --" $getripe -rL -T inetnum,inet6num "$1" | egrep '^org:' | while read ot ov; do $getripe -r -T organisation $ov | egrep '^abuse-c:' | while read at av; do $getripe -r $av | fgrep abuse-mailbox done done | tail -1 In order to fix the issue on the database side, it's necessary to prevent any occurrence of "abuse-mailbox" in all objects beside an newly created "abuse-contact" type, which is only allowed to referenced in the "abuse-c". See here for a corner case: https://lutz.donnerhacke.de/eng/Blog/Antispammers-and-Abuse-C
[anti-abuse-wg] Fw: NWI reviews: NWI-1 staying on top of abuse contact changes
Colleagues We need to take some action on these old NWIs. Either we move forward with them or we cancel them. It is difficult to draw a consensus on 2 comments. Can you please give us a couple of minutes of your time and let us know if this NWI-1 is needed, useful or a waste of time. cheersdenis co-chair DB-WG - Forwarded message - From: ripedenis--- via anti-abuse-wg To: anti-abuse-wg@ripe.net Sent: Tuesday, 22 September 2020, 22:50:52 CESTSubject: [anti-abuse-wg] NWI reviews: NWI-1 staying on top of abuse contact changes Colleagues This is now a very old NWI that never got out of the starting blocks. I am posting this to the AA-WG as I think the discussion is more relevant to this community given the impact of not properly managing this data. https://www.ripe.net/ripe/mail/archives/db-wg/2016-May/005234.html Since I raised this point the world has moved on and the situation is now even more complex. The "abuse-c:" attribute can be added to:-the resource holders primary ORGANISATION object-any secondary ORGANISATION object referenced from any point in the address hierarchy-any resource object at any point in the address hierarchy-all, or any combination, of the above These (multiple) "abuse-c:" attributes can reference the same or multiple ROLE objects. These (multiple) ROLE objects can include "abuse-mailbox:" attributes referencing the same or multiple email addresses. Over time, with large hierarchies, we could end up with very complex arrangements with objects, attributes and values that have been overlooked and forgotten about. Even though they may have been forgotten about they are still 'active' and queries will return the most appropriate abuse contact details, even if it is the wrong contact. There is no simple query that will return the details of such a complex structure of abuse contact data in the RIPE Database. So there is no easy way for a resource holder to review or manage these abuse contacts. Although most resources will only ever have one abuse contact for the whole hierarchy, over several years we could end up with the kind of data swamp we had before we introduced the "abuse-c:" attribute for the larger networks. Any user could write their own script to do more specific queries, parse the returned objects and work out the abuse contact data structure for their resource. It would require detailed knowledge of the RIPE Database structure, the abuse contact rules and would involve multiple queries just to get the data, which you then have to visualise. My reason for this NWI-1 was to suggest the RIPE NCC creates a tool or RIPEstat widget that would take in a resource and map out the abuse contact details for the whole hierarchy including that resource. Would this be useful, is it necessary or a waste of time? Please discuss cheersdenis co-chair DB-WG
Re: [anti-abuse-wg] NWI reviews: NWI-1 staying on top of abuse contact changes
HI When Tobias and I first developed the "abuse-c:" mechanism we wanted to keep the design simple. But eventually we were overruled by the community who preferred ease of use over simple design. So we are where we are :) cheersdenis co-chair DB-WG On Wednesday, 23 September 2020, 13:14:38 CEST, stephane.dodel...@swisscom.com wrote: Hello Leo, Hello all, > I have no objection to creating useful tools for people to use when > managing their resources. That said, whenever the data structure is so > complex that a special tool is needed to help people understand it, I > wonder if the real problem is the data structure. +1 Such a tool would certainly bring something, but may not be needed if we could simplify the structure in the first place. Best regards Stéphane Dodeller
Re: [anti-abuse-wg] NWI reviews: NWI-1 staying on top of abuse contact changes
Hi Leo I was proposing a tool to help the registrant manage their data. If you want to find the abuse contact you just query the resource and the abuse contact is returned. cheersdenis co-chair DB-WG On Tuesday, 22 September 2020, 23:13:02 CEST, Leo Vegoda wrote: Hi Denis, On Tue, Sep 22, 2020 at 1:51 PM ripedenis--- via anti-abuse-wg wrote: [...] > Over time, with large hierarchies, we could end up with very complex > arrangements with objects, attributes and values that have been overlooked > and forgotten about. Even though they may have been forgotten about they are > still 'active' and queries will return the most appropriate abuse contact > details, even if it is the wrong contact. > > There is no simple query that will return the details of such a complex > structure of abuse contact data in the RIPE Database. So there is no easy way > for a resource holder to review or manage these abuse contacts. Although most > resources will only ever have one abuse contact for the whole hierarchy, over > several years we could end up with the kind of data swamp we had before we > introduced the "abuse-c:" attribute for the larger networks. > > Any user could write their own script to do more specific queries, parse the > returned objects and work out the abuse contact data structure for their > resource. It would require detailed knowledge of the RIPE Database structure, > the abuse contact rules and would involve multiple queries just to get the > data, which you then have to visualise. > > My reason for this NWI-1 was to suggest the RIPE NCC creates a tool or > RIPEstat widget that would take in a resource and map out the abuse contact > details for the whole hierarchy including that resource. > > Would this be useful, is it necessary or a waste of time? Please discuss Should I understand this as a proposal for a tool that is used to keep the registrant informed about changes related to resources for which they are responsible, or a tool to help other network operators and users find the right place to send an abuse report? Many thanks, Leo
[anti-abuse-wg] NWI reviews: NWI-1 staying on top of abuse contact changes
Colleagues This is now a very old NWI that never got out of the starting blocks. I am posting this to the AA-WG as I think the discussion is more relevant to this community given the impact of not properly managing this data. https://www.ripe.net/ripe/mail/archives/db-wg/2016-May/005234.html Since I raised this point the world has moved on and the situation is now even more complex. The "abuse-c:" attribute can be added to:-the resource holders primary ORGANISATION object-any secondary ORGANISATION object referenced from any point in the address hierarchy-any resource object at any point in the address hierarchy-all, or any combination, of the above These (multiple) "abuse-c:" attributes can reference the same or multiple ROLE objects. These (multiple) ROLE objects can include "abuse-mailbox:" attributes referencing the same or multiple email addresses. Over time, with large hierarchies, we could end up with very complex arrangements with objects, attributes and values that have been overlooked and forgotten about. Even though they may have been forgotten about they are still 'active' and queries will return the most appropriate abuse contact details, even if it is the wrong contact. There is no simple query that will return the details of such a complex structure of abuse contact data in the RIPE Database. So there is no easy way for a resource holder to review or manage these abuse contacts. Although most resources will only ever have one abuse contact for the whole hierarchy, over several years we could end up with the kind of data swamp we had before we introduced the "abuse-c:" attribute for the larger networks. Any user could write their own script to do more specific queries, parse the returned objects and work out the abuse contact data structure for their resource. It would require detailed knowledge of the RIPE Database structure, the abuse contact rules and would involve multiple queries just to get the data, which you then have to visualise. My reason for this NWI-1 was to suggest the RIPE NCC creates a tool or RIPEstat widget that would take in a resource and map out the abuse contact details for the whole hierarchy including that resource. Would this be useful, is it necessary or a waste of time? Please discuss cheersdenis co-chair DB-WG
Re: [anti-abuse-wg] [db-wg] RIPE NCC Executive Board election
I would like to second Brian's comments below. The DB-WG has no mandate to concern itself with issues concerning the RIPE NCC Exec Board. Any discussion would be inappropriate on this mailing list. cheersdenis co-chair DB-WG On Thursday, 16 April 2020, 10:34:54 CEST, Brian Nisbet via db-wg wrote: Ronald, While obviously I can only make comments for AA-WG (I note there are many WGs in x-post) I need to point out that this is not a suitable email for this working group. The NCC Exec Board elections are a matter for the NCC members, not this WG nor any other, despite any cross-over in membership. Obviously you may speak to whomever you wish on this matter, but please do not use this mailing list as a vehicle for that. It is not part of the charter nor purpose of the WG. I would also point out that the order of candidates on the website can change, so while I am explicitly not asking you to make any more specific comments, I would point out that mentioning someone's place on a list is not useful and is potentially very harmful. I would, while again asking you not to make any more specific comments about who you are talking about, ask that you acknowledge this. I would please ask you and all members of this list to be extremely careful in regards to mentioning or alluding to any specific people and their activities. Thanks, BrianCo-Chair, RIPE AA-WG Brian Nisbet Service Operations Manager HEAnet CLG, Ireland's National Education and Research Network 1st Floor, 5 George's Dock, IFSC, Dublin D01 X8N7, Ireland +35316609040 brian.nis...@heanet.ie www.heanet.ie Registered in Ireland, No. 275301. CRA No. 20036270 From: anti-abuse-wg on behalf of Ronald F. Guilmette Sent: Thursday 16 April 2020 08:25 To: anti-abuse-wg@ripe.net ; db...@ripe.net ; routing...@ripe.net ; address-policy...@ripe.net Subject: [anti-abuse-wg] RIPE NCC Executive Board election CAUTION[External]: This email originated from outside of the organisation. Do not click on links or open the attachments unless you recognise the sender and know the content is safe. Greetings all, I know that all is not right with the world right now, and that most of you, like me, have much more pressing things on your minds right now, but someone just sent me the following link and I cannot exactly ignore it: https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ripe.net%2Fparticipate%2Fmeetings%2Fgm%2Fmeetings%2Fmay-2020%2Fconfirmed-candidatesdata=02%7C01%7Cbrian.nisbet%40heanet.ie%7Cd7c81f06348b4f53dca108d7e1d75a95%7Ccd9e8269dfb648e082538b7baf8d3391%7C0%7C1%7C637226187374036509sdata=wy76n6BJ0G9TZ9rEmO%2BdoF%2FWFk7Ds3nb5ncvZGPztoo%3Dreserved=0 I would like to call everyone's attention to the last of the three candidates who have, it seems, "qualified" as candidates for open seats of the RIPE NCC Executive Board. As I have already said, I know that things are bad in the world right now, but I must ask this question: Is there really no one other than these three candidates who is willing and/or able to stand for the three open seats on the RIPE NCC Executive board... three open seats that will be voted on at the next general meeting, 13-15 May 2020 ? If not, then it seems that RIPE NCC will soon be following in the new tradition, established first by AFRINIC only last year, of placing well and widely known crooks on it board. I desperately hope it won't come to that, but that is not for me to decide. The decision is in your hands dear friends. Regards, rfg
[anti-abuse-wg] Validation of 'end user' "abuse-mailbox:" (2019-04)
Colleagues The policy proposal 2019-04 makes lots of references to resource holders, RIPE NCC members, LIRs and End Users. Only once does it mention 'customers of resource holders'. I get the feeling that where it refers to 'End Uses' it means PI resource holders. The argument for allowing the "abuse-c:" attribute to be added to resource objects (INET(6)NUM) was to facilitate end user customers, with assignments from member's allocations, to be able to handle their own abuse reports. Over time this could potentially add (tens of) thousands of additional abuse contacts to the database. I would like to see this issue specifically considered and addressed as part of the review of this policy proposal:-How are these contacts going to be found in the RIPE Database?-How much additional work load will this place on the RIPE NCC?-Who is going to be held responsible for any failed End User abuse contact validations?-Does it need a specific follow up procedure for the RIPE NCC as they will have to go through the resource holder? cheers denis co-chair DB-WG
[anti-abuse-wg] Fw: working in new version of 2019-04 (Validation of "abuse-mailbox")
Yes of course it would have to be an automated process. A benefit of encrypting all the data is that it keeps the RIPE NCC out of any legal actions that may follow. They are simply a forwarding service and have no other details. cheers denis co-chair DB-WG On Friday, 17 January 2020, 11:59:51 CET, JORDI PALET MARTINEZ via anti-abuse-wg wrote: I will be fine with this (having RIPE NCC as an intermediator just to send the abuse report), if instead of a web form (or in addition to it), it is possible to automate it, for example RIPE NCC also accepts x-arf via email. RIPE NCC has the obligation to keep the information without disclosing it, so why we need to have a way to encypt it so RIPE NCC can’t read it? Furthermore, this should be an automated process. The staff is not going to handle every report manually. And moreover, in case of a bigger dispute, even if going to the courts, RIPE NCC can provide in a neutral way all the info of what happened. However, I’ve the feeling that in order to get this working, the policy must mandate that all the responser from the operator which customer is producing the abuse, also follow the same path, so: Abuse reporter (Victim or its ISP) -> RIPE NCC -> abuser operator -> RIPE NCC -> abuse reporter Otherwise, there will not be a way for RIPE to have stats of who is responding to abuse cases and who is not, or even simpler than that, what abuse mailboxes get bounced (which will be a policy violation if happens all the time with the same operator). Never mind we decide or not that not-responding is an abuse-c violation. Stats are good, even if not published with operator names. El 17/1/20 1:12, "anti-abuse-wg en nombre de ripedenis--- via anti-abuse-wg" escribió: Hi Sergio As I read through this thread similar ideas came to my mind. The question I would ask is "Is it too late to take a completely different approach to abuse contacts and reporting via the RIPE Database?" Suppose we had a standard form available via the ripe.net website for providing details of abuse. If you are able to find the "abuse-c:" details in the database now then you must know the IP address involved. The RIPE NCC could send the report to the abuse contact taken from the database via the specified IP address. This does not have to be an email interface either. We could look at other options. The RIPE NCC would then at least know if the report was successfully delivered. Using a standard form would make it much easier for the resource holder to interpret the information. Someone said: "Making such a scheme compulsory would be unacceptable to people who wish to interact with network owners without disclosing that in public ..." I have no understanding of the technology involved here, but when I send you a message on WhatsApp it is encrypted end to end. WhatsApp have no idea (they say) of the content of the message. Would it be possible to submit a form on ripe.net in a way that the content of that form is encrypted and sent to the resource holder so the RIPE NCC have no idea of the content of the form? That would satisfy this concern. Regardless of the outcome of the RIPE Database Requirements Task Force, something like this could still be implemented as it is external to the RIPE Database. Food for thought... cheers denis co-chair DB-WG On Wednesday, 15 January 2020, 10:22:28 CET, Sérgio Rocha wrote: Hi, Maybe we can change the approach. If RIPE website had a platform to post abuse report, that send the email for the abuse contact, it will be possible to evaluate the responsiveness of the abuse contact. This way anyone that report an abuse could assess not only the response but also the effectiveness of the actions taken by the network owner. After some time with this evaluations we would easy to realize who manages the reports and even who does not respond at all. Sérgio -Original Message- From: anti-abuse-wg [mailto:anti-abuse-wg-boun...@ripe.net] On Behalf Of Gert Doering Sent: 15 de janeiro de 2020 08:06 To: Carlos Friaças Cc: Gert Doering ; anti-abuse-wg Subject: Re: [anti-abuse-wg] working in new version of 2019-04 (Validation of "abuse-mailbox") Hi, On Wed, Jan 15, 2020 at 07:23:38AM +, Carlos Friaças via anti-abuse-wg wrote: > I obviously don't speak for the incident handling community, but i > think this (making it optional) would be a serious step back. The > current situation is already very bad when in some cases we know from > the start that we are sending (automated) messages/notices to blackholes. So why is it preferrable to send mails which are not acted on, as opposed to "not send mail because you know beforehand that the other network is not interested"? I can see that it is frustrating - but I still cannot support a policy change which will not
Re: [anti-abuse-wg] working in new version of 2019-04 (Validation of "abuse-mailbox")
Hi Sergio As I read through this thread similar ideas came to my mind. The question I would ask is "Is it too late to take a completely different approach to abuse contacts and reporting via the RIPE Database?" Suppose we had a standard form available via the ripe.net website for providing details of abuse. If you are able to find the "abuse-c:" details in the database now then you must know the IP address involved. The RIPE NCC could send the report to the abuse contact taken from the database via the specified IP address. This does not have to be an email interface either. We could look at other options. The RIPE NCC would then at least know if the report was successfully delivered. Using a standard form would make it much easier for the resource holder to interpret the information. Someone said:"Making such a scheme compulsory would be unacceptable to people who wish to interact with network owners without disclosing that in public ..."I have no understanding of the technology involved here, but when I send you a message on WhatsApp it is encrypted end to end. WhatsApp have no idea (they say) of the content of the message. Would it be possible to submit a form on ripe.net in a way that the content of that form is encrypted and sent to the resource holder so the RIPE NCC have no idea of the content of the form? That would satisfy this concern. Regardless of the outcome of the RIPE Database Requirements Task Force, something like this could still be implemented as it is external to the RIPE Database. Food for thought... cheers denis co-chair DB-WG On Wednesday, 15 January 2020, 10:22:28 CET, Sérgio Rocha wrote: Hi, Maybe we can change the approach. If RIPE website had a platform to post abuse report, that send the email for the abuse contact, it will be possible to evaluate the responsiveness of the abuse contact. This way anyone that report an abuse could assess not only the response but also the effectiveness of the actions taken by the network owner. After some time with this evaluations we would easy to realize who manages the reports and even who does not respond at all. Sérgio -Original Message- From: anti-abuse-wg [mailto:anti-abuse-wg-boun...@ripe.net] On Behalf Of Gert Doering Sent: 15 de janeiro de 2020 08:06 To: Carlos Friaças Cc: Gert Doering ; anti-abuse-wg Subject: Re: [anti-abuse-wg] working in new version of 2019-04 (Validation of "abuse-mailbox") Hi, On Wed, Jan 15, 2020 at 07:23:38AM +, Carlos Friaças via anti-abuse-wg wrote: > I obviously don't speak for the incident handling community, but i > think this (making it optional) would be a serious step back. The > current situation is already very bad when in some cases we know from > the start that we are sending (automated) messages/notices to blackholes. So why is it preferrable to send mails which are not acted on, as opposed to "not send mail because you know beforehand that the other network is not interested"? I can see that it is frustrating - but I still cannot support a policy change which will not help dealing with irresponsible networks in any way, but at the same time increases costs and workload for those that do the right thing alrady. > To an extreme, there should always be a known contact responsible for > any network infrastructure. If this is not the case, what's the > purpose of a registry then? "a known contact" and "an *abuse-handling* contact" is not the same thing. Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard, Michael Emmer Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
Re: [anti-abuse-wg] @EXT: RE: working in new version of 2019-04 (Validation of "abuse-mailbox")
Colleagues I have just read this whole thread, it took a while (I should get sick more often and spend a day in bed reading emails). I have a few points to make. Some are similar to points already raised but I will reinforce them. I cut out the bits I want to respond to, but sorry I have not included the authors (you will know if it's you). "If I need to use a web form, which is not standard, for every abuse report that I need to submit, there is no sufficient time in the world to fill all them." So instead each resource holder must interpret randomly written emails and find any relevant information from within lots of junk. "ever since the day that RIPE NCC firstpublished an abuse reporting address in the data base, it has, ineffect, injected itself, even if only to a minimal degree, intothe relationship between a network abuse victim and the relevantresource holders that have clear connections to the abuse source" To be clear, the RIPE NCC is the data controller, not the data content provider. The RIPE NCC does not publish the abuse contacts, they facilitate resource holders to publish them. "make abuse-c: an optional attribute(basically, unrolling the "mandatory" part of the policy proposal thatintroduced it in the first place)" As co-author/designer of "abuse-c:" one of the original aims of the "abuse-c:" attribute was to provide one single point of contact for a resource holder's abuse reports. If it is made optional, abuse reports would simply be sent to the "admin-c:", "tech-c:", "notify:", etc email addresses, as they were before. People will simply search the database for any email address associated with the resource holder and spam them all. It won't stop abuse reports being sent 'somewhere'. And once someone has had to go to the trouble of finding a list of email addresses to use for the resource holder who has no "abuse-c:", then they will probably do the same for all reports they send. So those of you who do respond to abuse complaints will find complaints being sent to a whole host of your email addresses from the RIPE Database. We lose the 'keep it in one well defined location' benefit. "at the very least, RIPE NCC could setup and maintain just a basic review "platform" where the public at largecan at least make it known to all observers which networks are the assholesand which ones aren't." This would be an excellent way for a network operator to 'take out' their competitors. "While I would accept Gert's proposal for making abuse-c an optionalattribute, the reason I offered a counter proposal for publishing "astatement to the effect that the network operator does not act onabuse reports" is to add clarity at a high level." How many operators are going to make such a statement? It would become an invitation to block their traffic. If that was the alternative to any verification then they know if they don't make such a statement there will be no penalty. So just don't make a statement and still ignore the reports. "i'm more worried about someone using real e-mailaddresses of real unrelated people than the /dev/null or unattendedmailboxes." Separately to this discussion we need to have a mechanism to say "Remove my email address from this resource", as Google has when someone uses your gmail address as a recovery address. (A service I use on a weekly basis) "Nice analogy, but when you add the eCommerce Directive into the mix, where a network provider (or hosting provider) is not liable for what their users do, the outcome changes. Only if you have knowledge there might be a possibility for liability, but if you do not accept abuse notices, and therefore do not have knowledge you are not liable. Also note there is no monitoring obligation, but if you do monitor you can gain knowledge and become liable for -everything-." If you hide behind this type of logic, the EU in particular could easily change the law so that refusal to accept notifications renders you liable as if you had received it. 'Ignorance of the law is no excuse' comes to mind. ">It's amazing that nobody cant propose anything without receiving a shower of all sorts of arguments againstIt's called 'democracy'." Many of the countries in the RIPE region are not democracies (including the UK now). Having been on this mailing list for many years, as others have said, this discussion has gone round in circles so many times. It really makes it hard to follow what the general view is. To me there seems to be 2 camps. One camp wants to 'do something' to try to improve the situation. The other camp wants to do nothing for a variety of reasons (not the RIPE NCC's job, gets tangled up with other policies, too much work/time, a burden on those who do the right thing, won't help with those who avoid it, we are engineers not social workers or police). These are the same reasons used against almost every policy proposal on this list. We are in a new decade now. We have to take a
