Re: general badness AS-based reputation system
On Sep 26, 2011, at 02:23 , Manish Karir wrote: We tried to outline some of the challenges of building such a system in our NANOG52 presentation: http://www.merit.edu/networkresearch/papers/pdf/2011/NANOG52_reputation-nanog.pdf In particular see slide 4. where we tried to lay down what we think the requirements are for a socially acceptable reputation system. With a bit of luck we might be able to announce the release of our system before the next NANOG mtg, but in my opinion collating host reputation reports is just a small and the easiest part of the effort. The key is in solving the challenges of allowing (and incentivizing) participation and being robust to false information injection. Hi Manish. As mentioned by Gadi, the maintenance of such tools is not often easy, in particular since some datasources may disappear or become obsolete over time. For example, to have a global view of the BGP landscape the best service I know is RIS from RIPE, but there aren't many alternatives. Although this problem may be reduced through an increase of the total number of datasources, it is something to be considered. Also, since historical data is considered, the fact that some datasources may disappear over time can affect the ranking value. Most importantly, this type of approach is dependent on the level of commitment the network community has, which may be mined by not enough incentives (the problem mentioned in slide 3). Namely (as stated before) the problem of certain customers not being able to reach critical systems just because that ASN was considered evil, is a strong incentive *not* to adhere to the system. This is IMHO THE biggest Problem. Also, if you are a transit AS do you think this to be a viable approach? Although I think this philosophy has strong arguments to move forward, it also has many challenges that must be dealt with and the biggest ones are not technical (what a surprise…). Thanks for your valuable contribution. Regards, S.
Re: general badness AS-based reputation system
On 9/26/11 2:31 AM, Jimmy Hess wrote: Sorry... what makes you think the problem with use of a AS-reputation systems is social and not technical? IP packets are not stamped with the numbers of any of the AS they transitted to reach your network. The IP protocol simply does not expose AS number information, therefore, for filtering purposes, you don't actually have the information Filtering is dangerous, especially when done with ASNs. There are many technical challenges and many levels of filtering, all are technical issues and policy decisions based on how bad it's needed. Let's not forget how dangerous it is to block a network just to find out that your customers no longer get service, that is a much bigger issue that figuring our what is out technically, IMO. I am in agreement with you -- which is why I focus on the cultural aspect. Gadi.
Re: general badness AS-based reputation system
We tried to outline some of the challenges of building such a system in our NANOG52 presentation: http://www.merit.edu/networkresearch/papers/pdf/2011/NANOG52_reputation-nanog.pdf In particular see slide 4. where we tried to lay down what we think the requirements are for a socially acceptable reputation system. With a bit of luck we might be able to announce the release of our system before the next NANOG mtg, but in my opinion collating host reputation reports is just a small and the easiest part of the effort. The key is in solving the challenges of allowing (and incentivizing) participation and being robust to false information injection. I've actually encountered a system that was accepted for operational use once. Bad data was added at some point (bound to happen) and the system was shut down. Thanks for sharing your slides, keep up the good work! Gadi.
general badness AS-based reputation system
Having run one of these in the past, when take-downs of CCs was still semi-useful, my ethos on this is problematic, however, I am as of yet undecided as to this one. An AS-based reputation system for all sorts of badness: http://bgpranking.circl.lu/ In my opinion, third-party security based AS-reputation systems will eventually become de-facto border filtering systems for ISPs, but that day is still not here, as that is still socially unacceptable in our circles, and will remain so until it becomes _necessary_. Regardless of my musings of Operators World cultural future, this systems seems rather interesting, and no doubt you'd want to take a look at your listing. Gadi.
Re: general badness AS-based reputation system
On Sun, Sep 25, 2011 at 10:37 AM, Gadi Evron g...@linuxbox.org wrote: In my opinion, third-party security based AS-reputation systems will eventually become de-facto border filtering systems for ISPs, but that day is still not here, as that is still socially unacceptable in our circles, and will remain so until it becomes _necessary_. Sorry... what makes you think the problem with use of a AS-reputation systems is social and not technical? IP packets are not stamped with the numbers of any of the AS they transitted to reach your network. The IP protocol simply does not expose AS number information, therefore, for filtering purposes, you don't actually have the information It's difficult to justify a complex AS-reputation system that would have limited effectiveness, and really, is little better than other reputation system methods (such as source address blacklisting) -- -JH
Re: general badness AS-based reputation system
On Sep 25, 2011, at 6:31 PM, nanog-requ...@nanog.org wrote: Message: 9 Date: Sun, 25 Sep 2011 18:37:17 +0300 From: Gadi Evron g...@linuxbox.org To: nanog@nanog.org Subject: general badness AS-based reputation system Message-ID: 4e7f4aad.8020...@linuxbox.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Having run one of these in the past, when take-downs of CCs was still semi-useful, my ethos on this is problematic, however, I am as of yet undecided as to this one. An AS-based reputation system for all sorts of badness: http://bgpranking.circl.lu/ In my opinion, third-party security based AS-reputation systems will eventually become de-facto border filtering systems for ISPs, but that day is still not here, as that is still socially unacceptable in our circles, and will remain so until it becomes _necessary_. Regardless of my musings of Operators World cultural future, this systems seems rather interesting, and no doubt you'd want to take a look at your listing. Gadi. We tried to outline some of the challenges of building such a system in our NANOG52 presentation: http://www.merit.edu/networkresearch/papers/pdf/2011/NANOG52_reputation-nanog.pdf In particular see slide 4. where we tried to lay down what we think the requirements are for a socially acceptable reputation system. With a bit of luck we might be able to announce the release of our system before the next NANOG mtg, but in my opinion collating host reputation reports is just a small and the easiest part of the effort. The key is in solving the challenges of allowing (and incentivizing) participation and being robust to false information injection. Comments are welcome. Thanks. -manish
Re: general badness AS-based reputation system
On Sep 25, 2011, at 9:23 PM, Manish Karir wrote: On Sep 25, 2011, at 6:31 PM, nanog-requ...@nanog.org wrote: Message: 9 Date: Sun, 25 Sep 2011 18:37:17 +0300 From: Gadi Evron g...@linuxbox.org To: nanog@nanog.org Subject: general badness AS-based reputation system Message-ID: 4e7f4aad.8020...@linuxbox.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Having run one of these in the past, when take-downs of CCs was still semi-useful, my ethos on this is problematic, however, I am as of yet undecided as to this one. An AS-based reputation system for all sorts of badness: http://bgpranking.circl.lu/ In my opinion, third-party security based AS-reputation systems will eventually become de-facto border filtering systems for ISPs, but that day is still not here, as that is still socially unacceptable in our circles, and will remain so until it becomes _necessary_. Regardless of my musings of Operators World cultural future, this systems seems rather interesting, and no doubt you'd want to take a look at your listing. Gadi. We tried to outline some of the challenges of building such a system in our NANOG52 presentation: http://www.merit.edu/networkresearch/papers/pdf/2011/NANOG52_reputation-nanog.pdf In particular see slide 4. where we tried to lay down what we think the requirements are for a socially acceptable reputation system. With a bit of luck we might be able to announce the release of our system before the next NANOG mtg, but in my opinion collating host reputation reports is just a small and the easiest part of the effort. The key is in solving the challenges of allowing (and incentivizing) participation and being robust to false information injection. Comments are welcome. Thanks. -manish Hi Manish, Looks like very interesting work. Does the system that you'll be announcing provide some means of coming to terms with challenges like the following? 1. Many large operators administer multiple ASNs, but some of the resulting AS sibling relationships may not be identifiable as such based on public-facing whois records, or interconnection relationships, or any other public data sources. Does your system incorporate some means of attributing reputation-related information at the (multi-AS) institutional level -- even when the contours of such institutions are not self-evident? 2. Some members of the ARIN community have recently floated a policy proposal which if approved would make ASNs transferable, and some supporters of that proposal have argued that RIR involvement in such transfers should be strictly limited to the passive recording of whatever information is voluntarily disclosed by the transacting parties, if and when a disclosure is made. Does your system ascribe reputation strictly to specific ASNs, such that declared changes in an ASN's ownership/control would not affect that ASNs accumulated reputation record to-date? Alternately, if declared changes to an ASN's ownership would affect (e.g., restart) an ASN's reputation history, will your system incorporate some mechanism for assessing the material/operational substance of ASN re-registration events (e.g., to filter for possible re-registrations of convenience)? I like to ask these sort of questions (for any/all proposed systems like this) because it seems to me that any system that associates increasing value with a cumulative record of consistent approved behavior over time must take extra care not to provide *asymmetrical* opportunities (i.e., to some but not all participants) to isolate and sanitize their own disapproved behavior, thereby leaving their longstanding (favorable) reputations intact. Note that this problem is *not* reducible to the idea that a reputation system must be absolutely infallible. Obviously it is not reasonable to demand something that is impossible to deliver. However, it is altogether reasonable to demand that any system that is intentionally designed to produce a new, endogenously-driven reputation-based hierarchy of operational entities does something more than just recreate and reinforce pre-existing hierarchies that are completely orthogonal to reputation. Look forward to hearing more! Regards, TV
Re: general badness AS-based reputation system
On Sep 25, 2011, at 11:31 PM, Tom Vest wrote: On Sep 25, 2011, at 9:23 PM, Manish Karir wrote: On Sep 25, 2011, at 6:31 PM, nanog-requ...@nanog.org wrote: Message: 9 Date: Sun, 25 Sep 2011 18:37:17 +0300 From: Gadi Evron g...@linuxbox.org To: nanog@nanog.org Subject: general badness AS-based reputation system Message-ID: 4e7f4aad.8020...@linuxbox.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Having run one of these in the past, when take-downs of CCs was still semi-useful, my ethos on this is problematic, however, I am as of yet undecided as to this one. An AS-based reputation system for all sorts of badness: http://bgpranking.circl.lu/ In my opinion, third-party security based AS-reputation systems will eventually become de-facto border filtering systems for ISPs, but that day is still not here, as that is still socially unacceptable in our circles, and will remain so until it becomes _necessary_. Regardless of my musings of Operators World cultural future, this systems seems rather interesting, and no doubt you'd want to take a look at your listing. Gadi. We tried to outline some of the challenges of building such a system in our NANOG52 presentation: http://www.merit.edu/networkresearch/papers/pdf/2011/NANOG52_reputation-nanog.pdf In particular see slide 4. where we tried to lay down what we think the requirements are for a socially acceptable reputation system. With a bit of luck we might be able to announce the release of our system before the next NANOG mtg, but in my opinion collating host reputation reports is just a small and the easiest part of the effort. The key is in solving the challenges of allowing (and incentivizing) participation and being robust to false information injection. Comments are welcome. Thanks. -manish Hi Manish, Looks like very interesting work. Does the system that you'll be announcing provide some means of coming to terms with challenges like the following? 1. Many large operators administer multiple ASNs, but some of the resulting AS sibling relationships may not be identifiable as such based on public-facing whois records, or interconnection relationships, or any other public data sources. Does your system incorporate some means of attributing reputation-related information at the (multi-AS) institutional level -- even when the contours of such institutions are not self-evident? 2. Some members of the ARIN community have recently floated a policy proposal which if approved would make ASNs transferable, and some supporters of that proposal have argued that RIR involvement in such transfers should be strictly limited to the passive recording of whatever information is voluntarily disclosed by the transacting parties, if and when a disclosure is made. Does your system ascribe reputation strictly to specific ASNs, such that declared changes in an ASN's ownership/control would not affect that ASNs accumulated reputation record to-date? Alternately, if declared changes to an ASN's ownership would affect (e.g., restart) an ASN's reputation history, will your system incorporate some mechanism for assessing the material/operational substance of ASN re-registration events (e.g., to filter for possible re-registrations of convenience)? I like to ask these sort of questions (for any/all proposed systems like this) because it seems to me that any system that associates increasing value with a cumulative record of consistent approved behavior over time must take extra care not to provide *asymmetrical* opportunities (i.e., to some but not all participants) to isolate and sanitize their own disapproved behavior, thereby leaving their longstanding (favorable) reputations intact. Note that this problem is *not* reducible to the idea that a reputation system must be absolutely infallible. Obviously it is not reasonable to demand something that is impossible to deliver. However, it is altogether reasonable to demand that any system that is intentionally designed to produce a new, endogenously-driven reputation-based hierarchy of operational entities does something more than just recreate and reinforce pre-existing hierarchies that are completely orthogonal to reputation. Look forward to hearing more! Regards, TV Hi Tom, At what granularity reputation is useful is an excellent question. Obviously we already have lots of single data source based host reputation sources. Other possible aggregations are prefixes, ASNs, and as you suggest organizations (which might be multi-ASN). Another extreme possible aggregation is country. In my opinion BGP prefix is the right level of aggregation up to be actually useful rather than narrow host reputation lists. You might expect hosts in a prefix to be under the same security policy regime and hence have similar likelihood of future malicious
Re: general badness AS-based reputation system
I would probably limit this to simply identifying rogue prefixes [such as those prefixes - and there are some - owned entirely by criminal spammers, botnet CCs etc] [let us not get into a discussion on listing criteria or what constitutes criminal spam just now, there's a whole lot of such discussion and even a decent RFC draft] http://tools.ietf.org/html/draft-irtf-asrg-bcp-blacklists-07 On Mon, Sep 26, 2011 at 10:41 AM, Manish Karir mka...@merit.edu wrote: On Sep 25, 2011, at 11:31 PM, Tom Vest wrote: On Sep 25, 2011, at 9:23 PM, Manish Karir wrote: On Sep 25, 2011, at 6:31 PM, nanog-requ...@nanog.org wrote: Message: 9 Date: Sun, 25 Sep 2011 18:37:17 +0300 From: Gadi Evron g...@linuxbox.org To: nanog@nanog.org Subject: general badness AS-based reputation system Message-ID: 4e7f4aad.8020...@linuxbox.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Having run one of these in the past, when take-downs of CCs was still semi-useful, my ethos on this is problematic, however, I am as of yet undecided as to this one. An AS-based reputation system for all sorts of badness: http://bgpranking.circl.lu/ In my opinion, third-party security based AS-reputation systems will eventually become de-facto border filtering systems for ISPs, but that day is still not here, as that is still socially unacceptable in our circles, and will remain so until it becomes _necessary_. Regardless of my musings of Operators World cultural future, this systems seems rather interesting, and no doubt you'd want to take a look at your listing. Gadi. We tried to outline some of the challenges of building such a system in our NANOG52 presentation: http://www.merit.edu/networkresearch/papers/pdf/2011/NANOG52_reputation-nanog.pdf In particular see slide 4. where we tried to lay down what we think the requirements are for a socially acceptable reputation system. With a bit of luck we might be able to announce the release of our system before the next NANOG mtg, but in my opinion collating host reputation reports is just a small and the easiest part of the effort. The key is in solving the challenges of allowing (and incentivizing) participation and being robust to false information injection. Comments are welcome. Thanks. -manish Hi Manish, Looks like very interesting work. Does the system that you'll be announcing provide some means of coming to terms with challenges like the following? 1. Many large operators administer multiple ASNs, but some of the resulting AS sibling relationships may not be identifiable as such based on public-facing whois records, or interconnection relationships, or any other public data sources. Does your system incorporate some means of attributing reputation-related information at the (multi-AS) institutional level -- even when the contours of such institutions are not self-evident? 2. Some members of the ARIN community have recently floated a policy proposal which if approved would make ASNs transferable, and some supporters of that proposal have argued that RIR involvement in such transfers should be strictly limited to the passive recording of whatever information is voluntarily disclosed by the transacting parties, if and when a disclosure is made. Does your system ascribe reputation strictly to specific ASNs, such that declared changes in an ASN's ownership/control would not affect that ASNs accumulated reputation record to-date? Alternately, if declared changes to an ASN's ownership would affect (e.g., restart) an ASN's reputation history, will your system incorporate some mechanism for assessing the material/operational substance of ASN re-registration events (e.g., to filter for possible re-registrations of convenience)? I like to ask these sort of questions (for any/all proposed systems like this) because it seems to me that any system that associates increasing value with a cumulative record of consistent approved behavior over time must take extra care not to provide *asymmetrical* opportunities (i.e., to some but not all participants) to isolate and sanitize their own disapproved behavior, thereby leaving their longstanding (favorable) reputations intact. Note that this problem is *not* reducible to the idea that a reputation system must be absolutely infallible. Obviously it is not reasonable to demand something that is impossible to deliver. However, it is altogether reasonable to demand that any system that is intentionally designed to produce a new, endogenously-driven reputation-based hierarchy of operational entities does something more than just recreate and reinforce pre-existing hierarchies that are completely orthogonal to reputation. Look forward to hearing more! Regards, TV Hi Tom, At what granularity reputation is useful is an excellent question. Obviously we already have lots of single
