Re: [db-wg] Role of RIPE NCC in geofeed, abuse-c checks, etc
Hi Ed, On 4/8/2021 3:36 PM, Edward Shryane via db-wg wrote: Hi Jori, On 8 Apr 2021, at 14:42, Tyrasuki wrote: Hi Ed, This seems like a good implementation to me. However, I don't think it's a good idea to limit the values on the "remarks" attribute in this way, as this could cause unwanted side effects with for ex. messages left on objects for other network operators. Given the draft states: " Any particular inetnum: object MAY have, at most, one geofeed reference, whether a remarks: or a proper geofeed: attribute when one is defined." Do we enforce this by validating that there is only one "remarks: Geofeed" value (or "geofeed:") in the object? My apologies, I think I missed this section in the draft, thanks for clarifying the reason to me. Also: "Do not support non-ASCII values in URL domain names or path (these must be converted beforehand)" Do you by this mean not supporting non-ASCII entirely? Or to have for ex. the web-interface convert IDNs to punycode, and have this listed on the object? The RIPE database uses the Latin-1 character set, so IDN domain names or non-ASCII values in the URL path will be substituted with a '?' character, by default. We could support non-ASCII values by automatically converting them (like we do with non-ASCII domains in email addresses). That sounds like a good approach to me, thanks for clarifying. :) I think this is indeed a good starting ground for a minimal NWI, and would like to see where this goes. If the latter, and remarks can remain free-form, I'd say let's implement. Cheers, Jori Vanneste FOD11-RIPE Regards Ed Best regards, Jori Vanneste FOD11-RIPE OpenPGP_0x27A401F86B85D0BC.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature signature.asc Description: OpenPGP digital signature
Re: [db-wg] Role of RIPE NCC in geofeed, abuse-c checks, etc
Hi Jori, > On 8 Apr 2021, at 14:42, Tyrasuki wrote: > > Hi Ed, > > This seems like a good implementation to me. > > However, I don't think it's a good idea to limit the values on the "remarks" > attribute in this way, as this could cause unwanted side effects with for ex. > messages left on objects for other network operators. Given the draft states: " Any particular inetnum: object MAY have, at most, one geofeed reference, whether a remarks: or a proper geofeed: attribute when one is defined." Do we enforce this by validating that there is only one "remarks: Geofeed" value (or "geofeed:") in the object? > > Also: > > "Do not support non-ASCII values in URL domain names or path (these must be > > converted beforehand)" > > Do you by this mean not supporting non-ASCII entirely? Or to have for ex. the > web-interface convert IDNs to punycode, and have this listed on the object? > The RIPE database uses the Latin-1 character set, so IDN domain names or non-ASCII values in the URL path will be substituted with a '?' character, by default. We could support non-ASCII values by automatically converting them (like we do with non-ASCII domains in email addresses). > If the latter, and remarks can remain free-form, I'd say let's implement. > > > Cheers, > Jori Vanneste > FOD11-RIPE > Regards Ed
[db-wg] RIPE Database requirements progress
Colleagues For the benefit of those who don't often check the mail archives of the RIPE Database Task Force there have been 4 meetings in February and March and the TF has just published the minutes of their most recent meeting on their mail archive which can be found here: https://www.ripe.net/ripe/mail/archives/ripe-db-requirements-tf/ Their latest draft requirements document can be found here: https://www.ripe.net/participate/ripe/tf/rdb-requirements-tf/the-ripe-database-requirements-task-force-draft-document.pdf It would be helpful if more people would read this document and comment or discuss publicly on this WG mailing list any issues regarding the TF draft document before they publish their final version of the requirements. They also plan to hold a BOF sometime in May where you can make comments. cheers denis co-chair DB-WG
Re: [db-wg] Role of RIPE NCC in geofeed, abuse-c checks, etc
Hi Denis, On Thu, Apr 08, 2021 at 12:55:32AM +0200, denis walker via db-wg wrote: > I don't see the issue of what, if anything, should be validated as a > show stopper for introducing the "geofeed:" attribute. This is my idea > of utilising the RIRs to improve the value of services with increased > validation. That's why I changed the subject line and started it as a > different thread. We can come back to this later. Apologies for taking > you down a side road, but at least I got some initial feelings from > you on this more general issue. I recognize value in your suggestions on RIR-driven validation, and definitely agree on the desired outcomes, but I'm not convinced its the RIRs that should take on (as permanent task) to do outreach about 'broken geofeeds'. While keeping in mind that 'geofeed:' is a new utility for this industry and we (as collective of producers & consumers) have yet to see how things will work out exactly. One thing stood out to me in what Denis wrote: "This geofeed attribute will delegate this information process out to thousands of organisations." (src: https://www.ripe.net/ripe/mail/archives/db-wg/2021-April/006893.html) While indeed globally thousands of organizations are now being guided and enabled towards populating geofeeds, this in itself is not an indicator that a decentralized approach will 'overtake' the current market for "GeoIP information". It doesn't strike me as unfeasible that the likes of MaxMind, IPinfo.info, or any other GeoIP aggregators will take on the role of 'patrolling' the feeds and providing tools/notifications to those who appear to publish broken information. The (commercial) 'GeoIP market' is far more advanced than a mere IP Address <> Geographical location mapping. Another layer of advancement that exists: customers of GeoIP information oftentimes will correlate the purchased GeoIP information with their own internal records on fraud and other activities of interest, and also acquire GeoIP from multiple (possibly even non-database) sources. Some companies measure latency to help approximate geography. To me it seems unlikely the 'geofeed' mechanism will 'wipe out' the existing market, but rather geofeeds might be a (significant!) enhancement for existing practises. I do think that 'geofeed:' in some ways democratizes the market in the sense that an industry standard publication mechanism and easier access to this type of access means that more people can cheaply acquire GeoIP information which means that existing GeoIP providers will have to step up their game which is a positive! :) I think we should only ask the RIRs 'to do something' when it has become clear the industry itself is unable to organize it themselves. RIPE Atlas probably is a great example of someting only an RIR could've pulled off. Kind regards, Job ps. An example where data quality urgently needed to increase were RPKI ROAs in the 2019-2020 time frame. To help the BGP Default-Free Zone get rid of 'RPKI Invalid' BGP announcements, it was not the RIRs that 'did the cleanup', it was the efforts of individuals such as 'nusenu_', Anurag Bhatia, Massimo Candela's BGPAlerter, and hundreds of network operators actually deploying RPKI ROV that resulted in significant industry-wide cleanup. RIPE NCC indeed does have a 'wrong-ROA' alerting mechanism but it only applied to RIPE managed space, and (imho) is too crude of an alerting mechanism to be useful in most corporate contexts. pps. Another example of global cleanup led by an individuals is Jared Mauch's "Open Resolver Project" for which he was awarded the M3AAWG J.D. Falk Award.
Re: [db-wg] Role of RIPE NCC in geofeed, abuse-c checks, etc
On Thu, Apr 08, 2021 at 02:27:13PM +0200, Edward Shryane via db-wg wrote: > > On 8 Apr 2021, at 13:54, Randy Bush via db-wg wrote: > > > >> Could we consider creating an NWI with a reduced scope? > > > > as an exercise, how minimal can we get? > > Given the draft RFC: > https://datatracker.ietf.org/doc/draft-ietf-opsawg-finding-geofeeds/?include_text=1 > > I suggest the following minimal Solution Definition for an NWI: > > - Implement support for an optional, single "geofeed:" attribute in inetnum > and inet6num object types. > - Validate there is a maximum of either one "remarks: Geofeed" or "geofeed:" > attribute per object. > - Validate the "geofeed:" URL is well-formed and specifies the HTTPS protocol. > - Include the "geofeed:" attribute in database dumps and split files. > > And inversely, what we could leave out (to simplify the implementation): > > - Do not support non-ASCII values in URL domain names or path (these must be > converted beforehand) > - Do not migrate (re-write) "remarks: Geofeed" values as "geofeed:" attributes > - Do not validate that the URL is reachable (available) and do not validate > the content > > Is this enough to satisfy the draft requirements? Is it enough to be > useful for the DB-WG? I think the above is a great start to get a 'minimal viable geofeed' going! The first priority should be to move things over from 'remarks:' to a dedicated attribute. Kind regards, Job
Re: [db-wg] Role of RIPE NCC in geofeed, abuse-c checks, etc
Hi Ed, This seems like a good implementation to me. However, I don't think it's a good idea to limit the values on the "remarks" attribute in this way, as this could cause unwanted side effects with for ex. messages left on objects for other network operators. Also: > "Do not support non-ASCII values in URL domain names or path (these must be > converted beforehand)" Do you by this mean not supporting non-ASCII entirely? Or to have for ex. the web-interface convert IDNs to punycode, and have this listed on the object? If the latter, and remarks can remain free-form, I'd say let's implement. Cheers, Jori Vanneste FOD11-RIPE \ Original Message On Apr 8, 2021, 2:27 PM, Edward Shryane via db-wg < db-wg@ripe.net> wrote: > > > > > > > > > > > Hi Randy, > > > > > On 8 Apr 2021, at 13:54, Randy Bush via db-wg wrote: > > > > > >> Could we consider creating an NWI with a reduced scope? > > > > > > as an exercise, how minimal can we get? > > > > > > randy > > > > > > > Given the draft RFC: > > [https://datatracker.ietf.org/doc/draft-ietf-opsawg-finding-geofeeds/?include\_text=1][https_datatracker.ietf.org_doc_draft-ietf-opsawg-finding-geofeeds_include_text_1] > > > > I suggest the following minimal Solution Definition for an NWI: > > > > \- Implement support for an optional, single "geofeed:" attribute in > > inetnum and inet6num object types. > > \- Validate there is a maximum of either one "remarks: Geofeed" or > > "geofeed:" attribute per object. > > \- Validate the "geofeed:" URL is well-formed and specifies the HTTPS > > protocol. > > \- Include the "geofeed:" attribute in database dumps and split files. > > > > And inversely, what we could leave out (to simplify the implementation): > > > > \- Do not support non-ASCII values in URL domain names or path (these must > > be converted beforehand) > > \- Do not migrate (re-write) "remarks: Geofeed" values as "geofeed:" > > attributes > > \- Do not validate that the URL is reachable (available) and do not > > validate the content > > > > Is this enough to satisfy the draft requirements? Is it enough to be useful > > for the DB-WG? > > > > Regards > > Ed Shryane > > RIPE NCC > > > > [https_datatracker.ietf.org_doc_draft-ietf-opsawg-finding-geofeeds_include_text_1]: https://datatracker.ietf.org/doc/draft-ietf-opsawg-finding-geofeeds/?include_text=1 signature.asc Description: OpenPGP digital signature
Re: [db-wg] Role of RIPE NCC in geofeed, abuse-c checks, etc
Hi Randy, > On 8 Apr 2021, at 13:54, Randy Bush via db-wg wrote: > >> Could we consider creating an NWI with a reduced scope? > > as an exercise, how minimal can we get? > > randy > Given the draft RFC: https://datatracker.ietf.org/doc/draft-ietf-opsawg-finding-geofeeds/?include_text=1 I suggest the following minimal Solution Definition for an NWI: - Implement support for an optional, single "geofeed:" attribute in inetnum and inet6num object types. - Validate there is a maximum of either one "remarks: Geofeed" or "geofeed:" attribute per object. - Validate the "geofeed:" URL is well-formed and specifies the HTTPS protocol. - Include the "geofeed:" attribute in database dumps and split files. And inversely, what we could leave out (to simplify the implementation): - Do not support non-ASCII values in URL domain names or path (these must be converted beforehand) - Do not migrate (re-write) "remarks: Geofeed" values as "geofeed:" attributes - Do not validate that the URL is reachable (available) and do not validate the content Is this enough to satisfy the draft requirements? Is it enough to be useful for the DB-WG? Regards Ed Shryane RIPE NCC
Re: [db-wg] Role of RIPE NCC in geofeed, abuse-c checks, etc
> Could we consider creating an NWI with a reduced scope? as an exercise, how minimal can we get? randy
[db-wg] Whois Release 1.100
Dear Working Group, RIPE Database release 1.100 has been deployed to the Release Candidate environment. We plan to deploy to production on Thursday 22nd April. This release includes the following main changes: * Separate email address and leading/trailing space during Punycode conversion (#782) * Use status index during update validation (#725) * Fixed aut-num attribute managed flag in lookup and search response (#749) * Added client flag to API for proxying requests (#728) * RDAP nameserver not implemented (#711) * Fixed delete object bug (#723) In addition to other small improvements and bug fixes. The release notes are on the website: https://www.ripe.net/manage-ips-and-asns/db/release-notes/ripe-database-release-1.100 More information on the Release Candidate environment can be found on our website: https://www.ripe.net/manage-ips-and-asns/db/release-notes/rc-release-candidate-environment Please let us know if you find any issues with this release in the RC environment. Regards Ed Shryane RIPE NCC
