Re: mail admins?
On Thu, Apr 23, 2020 at 07:47:58PM -0700, Michael Thomas wrote: > On 4/23/20 7:35 PM, Matt Palmer wrote: > > While I do think webauthn is a neat idea, and solves at least one very real > > problem (credential theft via phishing), you do an absolutely terrible job > > of making that case. > > see RFC 4876, it is not about phishing. not even a little bit. Never has > been. Whilst I do *absolutely* agree with you that "A Configuration Profile Schema for Lightweight Directory Access Protocol (LDAP)-Based Agents" is "not about phishing, not even a little bit", I'm not entirely sure how it advances your argument. - Matt
Re: mail admins?
On 2020-04-23 7:31 p.m., Michael Thomas wrote: On 4/23/20 6:20 PM, William Herrin wrote: On Thu, Apr 23, 2020 at 4:57 PM Michael Thomas wrote: Passwords over the wire are the *key* problem of computer security. Nothing else even comes close. One only needs to look at the LinkedIn salting problem to know how trivial it is to exploit password reuse. They are a big company and they still absolutely failed. There are a trillion smaller sites who are just as vulnerable, and all it takes is one. You think sending encrypted passwords over the wire is more of a problem than intentionally allowing untrusted code to run on the same machine that contains personally sensitive information? Really? Do you understand that when malicious code gains a sufficient foothold on your computer, webauthn protects exactly squat? Um, they are not encrypted. The are plain text after TLS unencrypts them. That is their Achilles Heal. The ironic catch 22 is that libsodium.js runs in the browser to encrypt the passwords before being sent over the wire. But happens to be javascript.
Re: mail admins?
On 4/23/20 6:20 PM, William Herrin wrote: On Thu, Apr 23, 2020 at 4:57 PM Michael Thomas wrote: If you want an actual verifiable current day problem which is a clear and present danger, you should be running as fast as you can to retrofit every piece of web technology with webauthn to get rid of over the wire passwords. I think I posted about this before and got a collective ho-hum. Yeah, it came up last week on an ARIN group and I called it "flavor of the month." It does some interesting things on a strictly technical level but it's a solution in search of a problem. You're not at significant risk that your password will be captured from inside an encrypted channel and that's all webauthn adds to other widely deployed technologies that also haven't caught on. PS: you clearly lack imagination. password reuse is the default. $SHINYNEWSITE has only to entice you to enter your reused password which comes out in the clear on the other side of that TLS connection. basically password phishing. you can whine all you like about how stupid they are, but you know what... that is what they average person does. that is reality. js exploits do not hold a candle to that problem. Mike
Re: mail admins?
On 4/23/20 7:35 PM, Matt Palmer wrote: On Thu, Apr 23, 2020 at 06:31:04PM -0700, Michael Thomas wrote: Passwords over the wire are the *key* problem of computer security. Nothing else even comes close. Hmm, a bold claim, but I'm confident the author will have strong support for their position. One only needs to look at the LinkedIn salting problem That was a stored password problem, not a passwords-over-the-wire problem, but OK. I'm sure we'll be back on track shortly. You can't have a stored password problem if you never see them. While I do think webauthn is a neat idea, and solves at least one very real problem (credential theft via phishing), you do an absolutely terrible job of making that case. see RFC 4876, it is not about phishing. not even a little bit. Never has been. Please get a clue. Mike
Re: mail admins?
On Thu, Apr 23, 2020 at 06:31:04PM -0700, Michael Thomas wrote: > Passwords over the wire are the *key* problem of computer security. Nothing > else even comes close. Hmm, a bold claim, but I'm confident the author will have strong support for their position. > One only needs to look at the LinkedIn salting problem That was a stored password problem, not a passwords-over-the-wire problem, but OK. I'm sure we'll be back on track shortly. > to know how trivial it is to exploit password reuse. Not sure how exploiting password reuse causes problems with passwords over the wire. Halfway through the paragraph now, still haven't seen anything talking about passwords over the wire. No doubt the next sentence will address the claim in detail, though. > They are a big company and they still absolutely failed. Starting to think that maybe there isn't going to be the solid justification for the topic sentence that I'd originally assumed. > There are a trillion smaller sites who are just as vulnerable, and all it > takes is one. A trillion smaller sites that are just as vulnerable... to passwords over the wire? Wait, this is the end of the paragraph. How odd, not a single statement in support of the assertion. Perhaps it's not, in fact, true, then, that passwords over the wire are the *key* problem of computer security. While I do think webauthn is a neat idea, and solves at least one very real problem (credential theft via phishing), you do an absolutely terrible job of making that case. - Matt
Re: mail admins?
On 4/23/20 1:47 PM, William Herrin wrote: > Even if mailman saw it, mailman doesn't use VERP. Both 2.1 and 3.0 of mailman support VERP. I've had it running for some time on mailman 2.1. I'm not sure how it will impact delivery time (a consideration). I've found it to be a non issue in my case. I'd be willing to talk off list if anyone wants details on how to configure and test it. -- Bryan Fields 727-409-1194 - Voice http://bryanfields.net
Re: mail admins?
On 4/23/20 6:20 PM, William Herrin wrote: On Thu, Apr 23, 2020 at 4:57 PM Michael Thomas wrote: If you want an actual verifiable current day problem which is a clear and present danger, you should be running as fast as you can to retrofit every piece of web technology with webauthn to get rid of over the wire passwords. I think I posted about this before and got a collective ho-hum. Yeah, it came up last week on an ARIN group and I called it "flavor of the month." It does some interesting things on a strictly technical level but it's a solution in search of a problem. You're not at significant risk that your password will be captured from inside an encrypted channel and that's all webauthn adds to other widely deployed technologies that also haven't caught on. Passwords over the wire are the *key* problem of computer security. Nothing else even comes close. One only needs to look at the LinkedIn salting problem to know how trivial it is to exploit password reuse. They are a big company and they still absolutely failed. There are a trillion smaller sites who are just as vulnerable, and all it takes is one. that is infinitely more serious than some age-old js breaches. and it is especially critical for the equipment that nanog members run every day to configure, monitor, and manage. Ironically, it requires... javascript browser-side. You think sending encrypted passwords over the wire is more of a problem than intentionally allowing untrusted code to run on the same machine that contains personally sensitive information? Really? Do you understand that when malicious code gains a sufficient foothold on your computer, webauthn protects exactly squat? Um, they are not encrypted. The are plain text after TLS unencrypts them. That is their Achilles Heal. Yes, that is way more of a problem than code running in a sandbox. The one -- mischief. The other -- buh-bye retirement savings. Please, get a clue. Mike
Re: mail admins?
On Thu, Apr 23, 2020 at 4:57 PM Michael Thomas wrote: > If you want an actual verifiable current day problem which is a clear > and present danger, you should be running as fast as you can to retrofit > every piece of web technology with webauthn to get rid of over the wire > passwords. > > I think I posted about this before and got a collective ho-hum. Yeah, it came up last week on an ARIN group and I called it "flavor of the month." It does some interesting things on a strictly technical level but it's a solution in search of a problem. You're not at significant risk that your password will be captured from inside an encrypted channel and that's all webauthn adds to other widely deployed technologies that also haven't caught on. > that is infinitely more serious than some age-old js > breaches. and it is especially critical for the equipment that nanog > members run every day to configure, monitor, and manage. Ironically, it > requires... javascript browser-side. You think sending encrypted passwords over the wire is more of a problem than intentionally allowing untrusted code to run on the same machine that contains personally sensitive information? Really? Do you understand that when malicious code gains a sufficient foothold on your computer, webauthn protects exactly squat? Regards, Bill Herrin -- William Herrin b...@herrin.us https://bill.herrin.us/
Re: mail admins?
On 4/23/20 6:07 PM, Matt Palmer wrote: On Thu, Apr 23, 2020 at 09:10:37AM -0700, Michael Thomas wrote: javascript is a hell of a lot safer than downloading native apps on your phone, for example. Because those are, of course, the *only* two possible options for accessing information. I'm sorry that you all need to prove as muchly as possible that you're capable of living in the Stone Age, but us devs give you a collective ::yawn:: you are completely irrelevant. Mike
Re: mail admins?
On Thu, Apr 23, 2020 at 09:10:37AM -0700, Michael Thomas wrote: > javascript is a hell of a lot safer than downloading native apps on your > phone, for example. Because those are, of course, the *only* two possible options for accessing information. - Matt
Re: mail admins?
On Thu, Apr 23, 2020 at 04:30:28PM -0700, Michael Thomas wrote: > Ironically it seems that the way to disable javascript is to install a > browser extension. Nope. chrome://settings/content/javascript for Chromium, about:config -> javascript.enabled in Firefox. - Matt
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
Bottiger, If what you are saying is true and can be backed by documentation, I would start at the abuse contact for the offending 'Amplifier' and then start working your way up the transits of the offending AS# until someone cuts them off. The Squeaky wheel gets the grease! On Thu, Apr 23, 2020 at 3:33 PM Bottiger wrote: > There are many decent options for ddos protection in the US and Europe, > however there are very few in Brazil and Asia that support BGP. Servers and > bandwidth in these areas are much more expensive. > > Even though we are already doing anycast to split up the ddos attack, a > majority of the attack traffic is now ending up in these expensive areas, > and to top it off, these ISPs won't respond to abuse emails. > > It makes me wonder what the point of these abuse email are and if the > regional registries have any power to force them to reply. > > On Thu, Apr 23, 2020 at 3:12 PM Compton, Rich A > wrote: > >> Good luck with that. As Damian Menscher has presented at NANOG, >> even if we do an amazing job and shut down 99% of all DDoS reflectors, >> there will still be enough bandwidth to generate terabit size attacks. >> https://stats.cybergreen.net >> >> I think we need to instead collectively focus on stopping the spoofed >> traffic that allows these attacks to be generated in the first place. >> >> -Rich >> >> >> >> *From: *NANOG Email List on behalf of Bottiger >> >> *Date: *Thursday, April 23, 2020 at 3:32 PM >> *To: *Siyuan Miao >> *Cc: *NANOG list >> *Subject: *Re: Best way to get foreign ISPs to shut down DDoS reflectors? >> >> >> >> We are unable to upgrade our bandwidth in those areas. There are no >> providers within our budget there at the moment. Surely there must be some >> way to get them to respond. >> >> >> >> On Thu, Apr 23, 2020 at 2:23 PM Siyuan Miao wrote: >> >> It won't work. >> >> >> >> Get a good DDoS protection and forget about it. >> >> >> >> On Fri, Apr 24, 2020 at 5:17 AM Bottiger wrote: >> >> Is there a guide on how to get foreign ISPs to shut down reflectors used >> in DDoS attacks? >> >> >> >> I've tried sending emails listed under abuse contacts for their regional >> registries. Either there is none listed, the email is full, email does not >> exist, or they do not reply. Same results when sending to whatever other >> email they have listed. >> >> >> >> Example Networks: >> >> >> >> CLARO S.A. >> >> Telefonica >> >> China Telecom >> >> Korea Telecom >> >> The contents of this e-mail message and >> any attachments are intended solely for the >> addressee(s) and may contain confidential >> and/or legally privileged information. If you >> are not the intended recipient of this message >> or if this message has been addressed to you >> in error, please immediately alert the sender >> by reply e-mail and then delete this message >> and any attachments. If you are not the >> intended recipient, you are notified that >> any use, dissemination, distribution, copying, >> or storage of this message or any attachment >> is strictly prohibited. >> >
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
On Thu, Apr 23, 2020 at 3:26 PM Ca By wrote: > On Thu, Apr 23, 2020 at 3:14 PM Compton, Rich A > wrote: > >> Good luck with that. As Damian Menscher has presented at NANOG, >> even if we do an amazing job and shut down 99% of all DDoS reflectors, >> there will still be enough bandwidth to generate terabit size attacks. >> https://stats.cybergreen.net >> >> I think we need to instead collectively focus on stopping the spoofed >> traffic that allows these attacks to be generated in the first place. >> >> -Rich >> > > The bcp38 religion has failed to deliver the promised land for 20 years. > That's because it's been opt-in for thousands of ASNs. 1 spoofer is all you need to trigger all the reflectors. > A handful of transit providers is all you need to identify and filter all sources of spoofing. I do bcp38, i encourage others to as well, but i do not plan on it > unclogging the pipes in my lifetime. > > You will get more miles from ACL dropping and policing known bad traffic > (most of udp) > Do you have 10 Tbps of spare ingress capacity? If not, you should re-think your strategy (which may simply include a playbook for how to explain the outage to your customers). Damian *From: *NANOG Email List on behalf of Bottiger < >> bottige...@gmail.com> >> *Date: *Thursday, April 23, 2020 at 3:32 PM >> *To: *Siyuan Miao >> >> *Cc: *NANOG list >> *Subject: *Re: Best way to get foreign ISPs to shut down DDoS reflectors? >> >> >> >> We are unable to upgrade our bandwidth in those areas. There are no >> providers within our budget there at the moment. Surely there must be some >> way to get them to respond. >> >> >> >> On Thu, Apr 23, 2020 at 2:23 PM Siyuan Miao wrote: >> >> It won't work. >> >> >> >> Get a good DDoS protection and forget about it. >> >> >> >> On Fri, Apr 24, 2020 at 5:17 AM Bottiger wrote: >> >> Is there a guide on how to get foreign ISPs to shut down reflectors used >> in DDoS attacks? >> >> >> >> I've tried sending emails listed under abuse contacts for their regional >> registries. Either there is none listed, the email is full, email does not >> exist, or they do not reply. Same results when sending to whatever other >> email they have listed. >> >> >> >> Example Networks: >> >> >> >> CLARO S.A. >> >> Telefonica >> >> China Telecom >> >> Korea Telecom >> >> The contents of this e-mail message and >> any attachments are intended solely for the >> addressee(s) and may contain confidential >> and/or legally privileged information. If you >> are not the intended recipient of this message >> or if this message has been addressed to you >> in error, please immediately alert the sender >> by reply e-mail and then delete this message >> and any attachments. If you are not the >> intended recipient, you are notified that >> any use, dissemination, distribution, copying, >> or storage of this message or any attachment >> is strictly prohibited. >> >
Re: mail admins?
On 4/23/20 4:40 PM, William Herrin wrote: On Thu, Apr 23, 2020 at 4:13 PM Scott Weeks wrote: --- m...@mtcc.com wrote: I'm not sure why the admins of nanog's site should particularly care about appeasing the js tinfoil hat set. Not the tin foil hat crowd, security. Can't it be both? Mobile code (javascript) has a long a storied history of security disaster. So yes, I surf with javascript disabled and when I run in to a web site that I can't use without it about 75% of the time I back up to the search engine and pick a different web site because I don't want to let my computer run the horrid crapware the site author thinks I should allow him to run. Does controlling what I allow my computer to run make me a member of the tinfoil hat set? Watching folks around me use their equipment, it's apparent that it does. Is it good security hygiene? Why yes, it's that too. Billions of people and by far the vast majority of users on the planet use js enabled sites. Would that it were that it was even in the top 1% of security problems we face. The fact is, nobody in devland cares whatsoever about this non-issue. that the nanog site ran without the need of js is more of an accident of history more likely than not: if it ain't broke don't fix it. If you want an actual verifiable current day problem which is a clear and present danger, you should be running as fast as you can to retrofit every piece of web technology with webauthn to get rid of over the wire passwords. that is infinitely more serious than some age-old js breaches. and it is especially critical for the equipment that nanog members run every day to configure, monitor, and manage. Ironically, it requires... javascript browser-side. I think I posted about this before and got a collective ho-hum. Mike
Re: mail admins?
On Thu, Apr 23, 2020 at 4:13 PM Scott Weeks wrote: > --- m...@mtcc.com wrote: >> I'm not sure why the admins of nanog's site should >> particularly care about appeasing the js tinfoil hat >> set. > > Not the tin foil hat crowd, security. Can't it be both? Mobile code (javascript) has a long a storied history of security disaster. So yes, I surf with javascript disabled and when I run in to a web site that I can't use without it about 75% of the time I back up to the search engine and pick a different web site because I don't want to let my computer run the horrid crapware the site author thinks I should allow him to run. Does controlling what I allow my computer to run make me a member of the tinfoil hat set? Watching folks around me use their equipment, it's apparent that it does. Is it good security hygiene? Why yes, it's that too. Regards, Bill Herrin -- William Herrin b...@herrin.us https://bill.herrin.us/
Re: mail admins?
On 4/23/20 4:13 PM, Scott Weeks wrote: --- m...@mtcc.com wrote: From: Michael Thomas I'm not sure why the admins of nanog's site should particularly care about appeasing the js tinfoil hat set. i mean, computers computing! who will stop this madness! - Not the tin foil hat crowd, security. Computers be computing with or without security. Many turn off JS. Especially in this crowd. The only time I wanted to use the site anyway was to find a thread as I can't seem to find them well in search engines. For example, what was the thread about SOHO firewalls and pfsense not too long ago? I can't remember what everyone was saying about a pfsense replacement as pfsense is no longer what it was. I am having to greenfield my home network and want to find a nerdable "dual WAN' firewall. That's off topic, though, as it's just a home network question. Unless your $DAYJOB consists solely of not using browsers whatsoever, you cannot function without javascript. Ironically it seems that the way to disable javascript is to install a browser extension. which is both javascript and outside the browser sandbox. *those* i fear. Mike
Re: mail admins?
--- m...@mtcc.com wrote: From: Michael Thomas I'm not sure why the admins of nanog's site should particularly care about appeasing the js tinfoil hat set. i mean, computers computing! who will stop this madness! - Not the tin foil hat crowd, security. Computers be computing with or without security. Many turn off JS. Especially in this crowd. The only time I wanted to use the site anyway was to find a thread as I can't seem to find them well in search engines. For example, what was the thread about SOHO firewalls and pfsense not too long ago? I can't remember what everyone was saying about a pfsense replacement as pfsense is no longer what it was. I am having to greenfield my home network and want to find a nerdable "dual WAN' firewall. That's off topic, though, as it's just a home network question. scott
Re: mail admins?
On 4/23/20 12:15 PM, Scott Weeks wrote: --- m...@mtcc.com wrote So I should just get used to configuring routers with HTTP and Notepad and forget about that nasty, old, 20th century vi crap? :) No, but complaining about javascript on websites - Just to be clear, I was only complaining about NANOG's site. Well, ARIN's, too. I get what you're saying for the internet in general. It seems NANOG could see javascript being blocked and redirect folks to a non-insecure (javascript) site like others (twitter, for example) do. Then, I could use Lynx! (kidding!) :) I'm not sure why the admins of nanog's site should particularly care about appeasing the js tinfoil hat set. i mean, computers computing! who will stop this madness! Mike
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
There are many decent options for ddos protection in the US and Europe, however there are very few in Brazil and Asia that support BGP. Servers and bandwidth in these areas are much more expensive. Even though we are already doing anycast to split up the ddos attack, a majority of the attack traffic is now ending up in these expensive areas, and to top it off, these ISPs won't respond to abuse emails. It makes me wonder what the point of these abuse email are and if the regional registries have any power to force them to reply. On Thu, Apr 23, 2020 at 3:12 PM Compton, Rich A wrote: > Good luck with that. As Damian Menscher has presented at NANOG, even > if we do an amazing job and shut down 99% of all DDoS reflectors, there > will still be enough bandwidth to generate terabit size attacks. > https://stats.cybergreen.net > > I think we need to instead collectively focus on stopping the spoofed > traffic that allows these attacks to be generated in the first place. > > -Rich > > > > *From: *NANOG Email List on behalf of Bottiger < > bottige...@gmail.com> > *Date: *Thursday, April 23, 2020 at 3:32 PM > *To: *Siyuan Miao > *Cc: *NANOG list > *Subject: *Re: Best way to get foreign ISPs to shut down DDoS reflectors? > > > > We are unable to upgrade our bandwidth in those areas. There are no > providers within our budget there at the moment. Surely there must be some > way to get them to respond. > > > > On Thu, Apr 23, 2020 at 2:23 PM Siyuan Miao wrote: > > It won't work. > > > > Get a good DDoS protection and forget about it. > > > > On Fri, Apr 24, 2020 at 5:17 AM Bottiger wrote: > > Is there a guide on how to get foreign ISPs to shut down reflectors used > in DDoS attacks? > > > > I've tried sending emails listed under abuse contacts for their regional > registries. Either there is none listed, the email is full, email does not > exist, or they do not reply. Same results when sending to whatever other > email they have listed. > > > > Example Networks: > > > > CLARO S.A. > > Telefonica > > China Telecom > > Korea Telecom > > The contents of this e-mail message and > any attachments are intended solely for the > addressee(s) and may contain confidential > and/or legally privileged information. If you > are not the intended recipient of this message > or if this message has been addressed to you > in error, please immediately alert the sender > by reply e-mail and then delete this message > and any attachments. If you are not the > intended recipient, you are notified that > any use, dissemination, distribution, copying, > or storage of this message or any attachment > is strictly prohibited. >
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
On Thu, Apr 23, 2020 at 3:14 PM Compton, Rich A wrote: > Good luck with that. As Damian Menscher has presented at NANOG, even > if we do an amazing job and shut down 99% of all DDoS reflectors, there > will still be enough bandwidth to generate terabit size attacks. > https://stats.cybergreen.net > > I think we need to instead collectively focus on stopping the spoofed > traffic that allows these attacks to be generated in the first place. > > -Rich > The bcp38 religion has failed to deliver the promised land for 20 years. 1 spoofer is all you need to trigger all the reflectors. I do bcp38, i encourage others to as well, but i do not plan on it unclogging the pipes in my lifetime. You will get more miles from ACL dropping and policing known bad traffic (most of udp) > > > *From: *NANOG Email List on behalf of Bottiger < > bottige...@gmail.com> > *Date: *Thursday, April 23, 2020 at 3:32 PM > *To: *Siyuan Miao > > *Cc: *NANOG list > *Subject: *Re: Best way to get foreign ISPs to shut down DDoS reflectors? > > > > We are unable to upgrade our bandwidth in those areas. There are no > providers within our budget there at the moment. Surely there must be some > way to get them to respond. > > > > On Thu, Apr 23, 2020 at 2:23 PM Siyuan Miao wrote: > > It won't work. > > > > Get a good DDoS protection and forget about it. > > > > On Fri, Apr 24, 2020 at 5:17 AM Bottiger wrote: > > Is there a guide on how to get foreign ISPs to shut down reflectors used > in DDoS attacks? > > > > I've tried sending emails listed under abuse contacts for their regional > registries. Either there is none listed, the email is full, email does not > exist, or they do not reply. Same results when sending to whatever other > email they have listed. > > > > Example Networks: > > > > CLARO S.A. > > Telefonica > > China Telecom > > Korea Telecom > > The contents of this e-mail message and > any attachments are intended solely for the > addressee(s) and may contain confidential > and/or legally privileged information. If you > are not the intended recipient of this message > or if this message has been addressed to you > in error, please immediately alert the sender > by reply e-mail and then delete this message > and any attachments. If you are not the > intended recipient, you are notified that > any use, dissemination, distribution, copying, > or storage of this message or any attachment > is strictly prohibited. >
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
On Thu, Apr 23, 2020 at 2:38 PM Shawn L via NANOG wrote: > This brings up an interesting question -- what is "good DDoS protection" on > an ISP scale? Apart from having enough bandwidth to weather the attack and > having upstream providers attempt to filter it for you/ Hi Shawn, I believe the normal mechanism is that you use BGP to sink the impacted /24s at many high-bandwidth exchange points worldwide, filter, and pass the traffic which the filter accepts back to your core infrastructure via a tunnel (VPN). Build or buy. If it's practical to sink the bandwidth near the DDOS target, I wouldn't think it was much of a DDOS. A question which interests me: How many attacks do folks find landing in the middle-ground between "annoying but readily handled" and "far beyond my ability?" Regards, Bill Herrin -- William Herrin b...@herrin.us https://bill.herrin.us/
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
Good luck with that. As Damian Menscher has presented at NANOG, even if we do an amazing job and shut down 99% of all DDoS reflectors, there will still be enough bandwidth to generate terabit size attacks. https://stats.cybergreen.net I think we need to instead collectively focus on stopping the spoofed traffic that allows these attacks to be generated in the first place. -Rich From: NANOG Email List on behalf of Bottiger Date: Thursday, April 23, 2020 at 3:32 PM To: Siyuan Miao Cc: NANOG list Subject: Re: Best way to get foreign ISPs to shut down DDoS reflectors? We are unable to upgrade our bandwidth in those areas. There are no providers within our budget there at the moment. Surely there must be some way to get them to respond. On Thu, Apr 23, 2020 at 2:23 PM Siyuan Miao mailto:avel...@misaka.io>> wrote: It won't work. Get a good DDoS protection and forget about it. On Fri, Apr 24, 2020 at 5:17 AM Bottiger mailto:bottige...@gmail.com>> wrote: Is there a guide on how to get foreign ISPs to shut down reflectors used in DDoS attacks? I've tried sending emails listed under abuse contacts for their regional registries. Either there is none listed, the email is full, email does not exist, or they do not reply. Same results when sending to whatever other email they have listed. Example Networks: CLARO S.A. Telefonica China Telecom Korea Telecom E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
The answer is “it depends”. What are you trying to accomplish? Are you trying to detect and surgically mitigate every DDoS attack? If so, you will need a good DDoS attack detection and mitigation solution and a team of people to run it or a 3rd party company that can do this for you. Do you want a cheap solution? There are open source projects that can detect DDoS attacks and generate RTBHs, flowspec rules, and inline filters that can block the traffic (eg. https://fastnetmon.com). Also, RTBHs can usually be advertised upstream (and to UTRS https://www.team-cymru.com/utrs.html) to reduce the amount of attack traffic that the victim network receives. Some ISPs just do the RTBH to the customer’s IP when there’s a DDoS and then force the customer to get another IP via DHCP, etc. -Rich From: NANOG Email List on behalf of NANOG list Reply-To: Shawn L Date: Thursday, April 23, 2020 at 3:39 PM To: NANOG list Subject: Re: Best way to get foreign ISPs to shut down DDoS reflectors? This brings up an interesting question -- what is "good DDoS protection" on an ISP scale? Apart from having enough bandwidth to weather the attack and having upstream providers attempt to filter it for you/ -Original Message- From: "Bottiger" Sent: Thursday, April 23, 2020 5:30pm To: "Siyuan Miao" Cc: "North American Network Operators' Group" Subject: Re: Best way to get foreign ISPs to shut down DDoS reflectors? We are unable to upgrade our bandwidth in those areas. There are no providers within our budget there at the moment. Surely there must be some way to get them to respond. On Thu, Apr 23, 2020 at 2:23 PM Siyuan Miao mailto:avel...@misaka.io>> wrote: It won't work. Get a good DDoS protection and forget about it. On Fri, Apr 24, 2020 at 5:17 AM Bottiger mailto:bottige...@gmail.com>> wrote: Is there a guide on how to get foreign ISPs to shut down reflectors used in DDoS attacks? I've tried sending emails listed under abuse contacts for their regional registries. Either there is none listed, the email is full, email does not exist, or they do not reply. Same results when sending to whatever other email they have listed. Example Networks: CLARO S.A. Telefonica China Telecom Korea Telecom E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
This brings up an interesting question -- what is "good DDoS protection" on an ISP scale? Apart from having enough bandwidth to weather the attack and having upstream providers attempt to filter it for you/ -Original Message- From: "Bottiger" Sent: Thursday, April 23, 2020 5:30pm To: "Siyuan Miao" Cc: "North American Network Operators' Group" Subject: Re: Best way to get foreign ISPs to shut down DDoS reflectors? We are unable to upgrade our bandwidth in those areas. There are no providers within our budget there at the moment. Surely there must be some way to get them to respond. On Thu, Apr 23, 2020 at 2:23 PM Siyuan Miao <[ avel...@misaka.io ]( mailto:avel...@misaka.io )> wrote: It won't work. Get a good DDoS protection and forget about it. On Fri, Apr 24, 2020 at 5:17 AM Bottiger <[ bottige...@gmail.com ]( mailto:bottige...@gmail.com )> wrote: Is there a guide on how to get foreign ISPs to shut down reflectors used in DDoS attacks? I've tried sending emails listed under abuse contacts for their regional registries. Either there is none listed, the email is full, email does not exist, or they do not reply. Same results when sending to whatever other email they have listed. Example Networks: CLARO S.A. Telefonica China Telecom Korea Telecom
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
Sounds like you'll need to talk to your upstreams if they can provide DDOS protection, alternatively look for remote DDOS protection options. Regards, Filip On 23 April 2020 11:30:36 pm GMT+02:00, Bottiger wrote: >We are unable to upgrade our bandwidth in those areas. There are no >providers within our budget there at the moment. Surely there must be >some >way to get them to respond. > >On Thu, Apr 23, 2020 at 2:23 PM Siyuan Miao wrote: > >> It won't work. >> >> Get a good DDoS protection and forget about it. >> >> On Fri, Apr 24, 2020 at 5:17 AM Bottiger >wrote: >> >>> Is there a guide on how to get foreign ISPs to shut down reflectors >used >>> in DDoS attacks? >>> >>> I've tried sending emails listed under abuse contacts for their >regional >>> registries. Either there is none listed, the email is full, email >does not >>> exist, or they do not reply. Same results when sending to whatever >other >>> email they have listed. >>> >>> Example Networks: >>> >>> CLARO S.A. >>> Telefonica >>> China Telecom >>> Korea Telecom >>> >> -- Sent from my mobile device. Please excuse my brevity.
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
We are unable to upgrade our bandwidth in those areas. There are no providers within our budget there at the moment. Surely there must be some way to get them to respond. On Thu, Apr 23, 2020 at 2:23 PM Siyuan Miao wrote: > It won't work. > > Get a good DDoS protection and forget about it. > > On Fri, Apr 24, 2020 at 5:17 AM Bottiger wrote: > >> Is there a guide on how to get foreign ISPs to shut down reflectors used >> in DDoS attacks? >> >> I've tried sending emails listed under abuse contacts for their regional >> registries. Either there is none listed, the email is full, email does not >> exist, or they do not reply. Same results when sending to whatever other >> email they have listed. >> >> Example Networks: >> >> CLARO S.A. >> Telefonica >> China Telecom >> Korea Telecom >> >
Re: Comcast - Significant v4 vs v6 throughput differences, almost stateful.
On Thu, Apr 23, 2020 at 12:45 PM Sabri Berisha wrote: > - On Apr 23, 2020, at 8:06 AM, Nick Zurku > wrote: > > We’re having serious throughput issues with our AS20326 pushing packets to > Comcast over v4. Our transfers are either the full line-speed of the > Comcast customer modem, or they’re seemingly capped at 200-300KB/s. This > behavior appears to be almost stateful, as if the speed is decided when the > connection starts. As long as it starts fast it will remain fast for the > length of the transfer and slow if it starts slow. Traces seem reasonable > and currently we’ve influenced the path onto GTT both ways. If we prepend > and reroute on our side, the same exact issue with happen on another > transit provider. > > Have you tried running a test to see if there may be ECMP issues? I wrote > a rudimentary script once, https://pastebin.com/TTWEj12T, that might help > here. This script is written to detect packet loss on multiple ECMP paths, > but you might be able to modify it for througput. > > The rationale behind my thinking is that if you have certain ECMP links > that are oversubscribed, the TCP sessions following that path will stay > "low" bandwidth. Sessions what win the ECMP lottery and pass through a > non-congested ECMP path may show better performance. > > Thanks, > > Sabri > And for a slightly more formal package to do this, there's UDPing, developed by the amazing networking team at Yahoo; it was written to identify intermittent issues affecting a single link in an ECMP or L2-hashed aggregate link pathway. https://github.com/yahoo/UDPing It does have the disadvantage of being designed for one-way measurement in each direction; that decision was intentional, to ensure each direction was measuring a completely known, deterministic pathway based on the hash values in the packets, without the return trip potentially obscuring or complicating identification of problematic links. But if you have access to both the source and destination ends of the connection, it's a wonderful tool to narrow down exactly where the underlying problem on a hashed ECMP/aggregate link is. Matt
Re: Best way to get foreign ISPs to shut down DDoS reflectors?
It won't work. Get a good DDoS protection and forget about it. On Fri, Apr 24, 2020 at 5:17 AM Bottiger wrote: > Is there a guide on how to get foreign ISPs to shut down reflectors used > in DDoS attacks? > > I've tried sending emails listed under abuse contacts for their regional > registries. Either there is none listed, the email is full, email does not > exist, or they do not reply. Same results when sending to whatever other > email they have listed. > > Example Networks: > > CLARO S.A. > Telefonica > China Telecom > Korea Telecom >
Best way to get foreign ISPs to shut down DDoS reflectors?
Is there a guide on how to get foreign ISPs to shut down reflectors used in DDoS attacks? I've tried sending emails listed under abuse contacts for their regional registries. Either there is none listed, the email is full, email does not exist, or they do not reply. Same results when sending to whatever other email they have listed. Example Networks: CLARO S.A. Telefonica China Telecom Korea Telecom
Re: xplornet contact or any experience with their satellite service?
On Tue, 2020-04-21 at 18:54 +, Mel Beckman wrote: > It’s not really oversold bandwidth. It’s just that the turnaround > time for a bolus of data is too long for two-way video conferencing > to be smooth or reliable. It’s like video conferencing using post > cards :) Except that videoconferencing is just the victim of the problem, and the problem is bursty bandwidth not latency. In fact, the back-and- forth of conversation is actually surprisingly decent for satellite. Not as much "talking over" as I would have suspected. But put the victim application aside, the real data is in the iperf3 results I posted, demonstrating how bursty the throughput is. The problem with that of course is that the "lowest" bandwidth "valleys" becomes the "constant bandwidth" that the codec uses rather than the average -- which of course cannot be used for real-time VC. Cheers, b. signature.asc Description: This is a digitally signed message part
Re: Venmo - Geolocation Challenges
I'm trying to build a list that has anyone of consequence on it for geolocation\VPN issues. http://thebrotherswisp.com/index.php/geo-and-vpn/ If you get anywhere with Venmo, let me know so I can add it to the list. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest-IX http://www.midwest-ix.com - Original Message - From: "Justin Krejci" To: nanog@nanog.org Sent: Thursday, April 23, 2020 1:42:18 PM Subject: Venmo - Geolocation Challenges Hello, I am looking for a Venmo network contact that can assist with a geolocation error in their systems. We have customers on a particular IP prefix who are being flagged by Venmo as outside of the USA but they are not outside of the USA. All standard geolocation systems I can find, as well as ARIN, all show the IP prefix as within the USA. Normal Venmo support channels are not fruitful to resolve the issue, they mostly just indicated users need to use their mobile data connection to get a different IP address for Venmo transactions. That is fine as a temporary work around but that is not a solution. Venmo support has expressed they are not going to do anything more for us in this regard. So if anyone has a relevant contact I might reach out to at Venmo or knows if Venmo uses a particular 3rd party geolocation data set and can share that with me that would be appreciated. I don't mind working with any organization to straighten out any stale data, I just need some assistance getting to someone who has the info or access. Thanks!! Justin Krejci
Re: Comcast - Significant v4 vs v6 throughput differences, almost stateful.
- On Apr 23, 2020, at 8:06 AM, Nick Zurku wrote: > We’re having serious throughput issues with our AS20326 pushing packets to > Comcast over v4. Our transfers are either the full line-speed of the Comcast > customer modem, or they’re seemingly capped at 200-300KB/s. This behavior > appears to be almost stateful, as if the speed is decided when the connection > starts. As long as it starts fast it will remain fast for the length of the > transfer and slow if it starts slow. Traces seem reasonable and currently > we’ve > influenced the path onto GTT both ways. If we prepend and reroute on our side, > the same exact issue with happen on another transit provider. Have you tried running a test to see if there may be ECMP issues? I wrote a rudimentary script once, [ https://pastebin.com/TTWEj12T | https://pastebin.com/TTWEj12T ] , that might help here. This script is written to detect packet loss on multiple ECMP paths, but you might be able to modify it for througput. The rationale behind my thinking is that if you have certain ECMP links that are oversubscribed, the TCP sessions following that path will stay "low" bandwidth. Sessions what win the ECMP lottery and pass through a non-congested ECMP path may show better performance. Thanks, Sabri
Re: mail admins?
--- m...@mtcc.com wrote > So I should just get used to configuring routers with HTTP and > Notepad and forget about that nasty, old, 20th century vi crap? :) No, but complaining about javascript on websites - Just to be clear, I was only complaining about NANOG's site. Well, ARIN's, too. I get what you're saying for the internet in general. It seems NANOG could see javascript being blocked and redirect folks to a non-insecure (javascript) site like others (twitter, for example) do. Then, I could use Lynx! (kidding!) :) scott
Venmo - Geolocation Challenges
Hello, I am looking for a Venmo network contact that can assist with a geolocation error in their systems. We have customers on a particular IP prefix who are being flagged by Venmo as outside of the USA but they are not outside of the USA. All standard geolocation systems I can find, as well as ARIN, all show the IP prefix as within the USA. Normal Venmo support channels are not fruitful to resolve the issue, they mostly just indicated users need to use their mobile data connection to get a different IP address for Venmo transactions. That is fine as a temporary work around but that is not a solution. Venmo support has expressed they are not going to do anything more for us in this regard. So if anyone has a relevant contact I might reach out to at Venmo or knows if Venmo uses a particular 3rd party geolocation data set and can share that with me that would be appreciated. I don't mind working with any organization to straighten out any stale data, I just need some assistance getting to someone who has the info or access. Thanks!! Justin Krejci
Re: Comcast - Significant v4 vs v6 throughput differences, almost stateful.
We started getting the wave of complaints over the last two weeks or so. Perhaps up to a month ago was when the initial few issues that were reported but were chalked up to being “an issues out on the internet.” Did your issues in CT start on a certain date? -- Nick Zurku Systems Engineer TeraSwitch, Inc. Office: 412-945-7048 nzu...@teraswitch.com On April 23, 2020 at 11:24:59 AM, Dovid Bender (do...@telecurve.com) wrote: We have customers in CT with the same issues. When did this start? On Thu, Apr 23, 2020 at 11:07 AM Nick Zurku wrote: > Hello all, > > I would appreciate if someone from Comcast could contact me about this. > > We’re having serious throughput issues with our AS20326 pushing packets to > Comcast over v4. Our transfers are either the full line-speed of the > Comcast customer modem, or they’re seemingly capped at 200-300KB/s. This > behavior appears to be almost stateful, as if the speed is decided when the > connection starts. As long as it starts fast it will remain fast for the > length of the transfer and slow if it starts slow. Traces seem reasonable > and currently we’ve influenced the path onto GTT both ways. If we prepend > and reroute on our side, the same exact issue with happen on another > transit provider. > > This issue does not affect v6 and that is full speed on every attempt. > This may be regionalized to the Comcast Pittsburgh market. > > This is most widely affecting our linux mirror repository server: > http://mirror.pit.teraswitch.com/ > Our colocation customers who are hosting VPN systems are also noticing > bottlenecks have started recently for their Comcast employees. > > -- > Nick Zurku > Systems Engineer > TeraSwitch, Inc. > nzu...@teraswitch.com >
Re: mail admins?
On Thu, Apr 23, 2020 at 1:33 AM Rich Kulawiec wrote: > - I've received erroneous bounces from @email.uscc.net as well. > It should be possible to track down the culprit via Mailman's logs > and the MTA's logs. Hi Rich, One of the annoyances with both those guys and the swedish folks is that they're not sending messages to the return path, they're responding to the header from address. Mailman at NANOG never sees it. It doesn't pass through their servers. Even if mailman saw it, mailman doesn't use VERP. It has to scan the message to match the subscriber and that doesn't consistently work. The subscriber address forwards to another address, the second address bounces and the bounce message doesn't necessarily contain the original subscriber address. To identify these jokers the ops will probably have to send a unique message to each subscriber with crafted headers. That can be folded in to a message that would go to the list anyway but the capability isn't baked in to mailman. > - There is zero point in obfuscating email addresses in archives or > anywhere else on the 'net. None. There hasn't been any point for > most of twenty years. Not with open subscription where any spammer can join the list to harvest the addresses of everybody who sends. Regards, Bill Herrin -- William Herrin b...@herrin.us https://bill.herrin.us/
Re: "Is BGP safe yet?" test
Vincent Bernat писал 2020-04-22 15:26: ❦ 22 avril 2020 12:51 -04, Andrey Kostin: BTW, has anybody yet thought/looked into extending RPKI-RTR protocol for validation of prefixes received from peer-as to make ingress filtering more dynamic and move away prefix filters from the routers? It could be used as is if the client implementations were a bit more flexible. With BIRD, you decide which AS to match. So you can match on the neighbor AS instead of the origin AS. Then, you can use something like GoRTR which accepts using JSON files instead of the RPKI as source. BIRD also allows you to have several ROA tables. So, you can check against the "real" RPKI as well as against your custom IRR-based RPKI. That's what I meant. So I guess IX operators already can use BIRD on route-servers for prefix filtering. I think it could be useful on hw routers as well. Kind regards, Andrey
Re: FlowSpec
On 2020-04-23 19:12, Roland Dobbins wrote: On 23 Apr 2020, at 22:57, Denys Fedoryshchenko wrote: In general operators don't like flowspec Its increasing popularity tens to belie this assertion. Yes, you're right that avoiding overflowing the TCAM is very important. But as Rich notes, a growing number of operators are in fact using flowspec within their own networks, when it's appropriate. One of operators told me why they dont provide flowspec anymore: customers are installing rules by scripts, stacking them, and not removing then when they dont need them anymore. RETN solved that by limiting number of rules customer can install. Smart network operators tend to do quite a bit of lab testing, prototyping, PoCs, et. al. against the very specific combinations of platforms/linecards/ASICs/OSes/trains/revisions before generally deploying new features and functionality; this helps ameliorate many concerns. Definitely, and i know some hosting operators who provide Flowspec functionality different way - over their own web interface/API. This way they can do unit tests, and do additional verifications. But if there is direct BGP, things like https://dyn.com/blog/longer-is-not-better/ might happen, if customer is using some exotic, "nightly-build" BGP implementation.
Re: "Is BGP safe yet?" test
Christopher Morrow писал 2020-04-22 14:05: a question about the data types here... So, a neighbor with no downstream ASN could be filtered directly with ROA == prefixlist-content. A neighbor with a downstream ASN has to be ROA (per asn downstream) == prefixlist-content. So you'd now have to do some calculations which are more complicated than just; "is roa for this prefix here and valid" to construct a prefix-list. correct? Sorry, and this sidesteps the intent of the peer as well. For instance you may have a peer with 2 'downstream' asn, only 1 of which they wish to provide transit to you (from you?) for... how would you know this intent/policy from the peer's perspective? today you know that (most likely) from IRR data. is your answer ASPA / AS-Cone ? ASPA/As-Cone is a concept about whole as-path verification afaik, but I may be mistaken. ROA validation prevents hijacking but doesn't prevent route-leaks. If my transit providers already do ROV, effect of doing it in local network will be limited to direct peers only and expected to be considerably small. For route-leaks prevention we still have to rely on IRR and filters configured directly on routers. Maintaining filters on routers is quite intensive task and they took a lot of space in the configuration. Adopting validation or similar mechanism for it could make it more dynamic and less resources-consuming. Or maybe I'm trying to invent a bicycle? Kind regards, Andrey
Re: Comcast - Significant v4 vs v6 throughput differences, almost stateful.
On Thu, Apr 23, 2020 at 8:06 AM Nick Zurku wrote: > We’re having serious throughput issues with our AS20326 pushing packets to > Comcast over v4. Our transfers are either the full line-speed of the Comcast > customer modem, or they’re seemingly capped at 200-300KB/s. This behavior > appears to be almost stateful, as if the speed is decided when the connection > starts. As long as it starts fast it will remain fast for the length of the > transfer and slow if it starts slow. Hi Nick, That's actually kinda normal for TCP behavior. The two most dominating factors in TCP throughput are the round-trip time (RTT) and how large the congestion window has grown prior to the first lost packet. Other factors (including later mild packet loss) tend to move the needle so slowly you might not notice it moving at all. One of the interesting patterns with TCP is that the sender tends to shove out all the packets it can in the first few percent of the RTT and then sits idle. When the bandwidths are relatively fast, the receiver receives and acks them all in a short time window as well. As a result you get these high-bandwidth spurts where packet loss due to full buffers is likely even though for most of the RTT there are no packets being transmitted at all. It can take several minutes for packets to spread out within the RTT, and by then the congestion window (hence throughput) is firmly established. Regards, Bill Herrin -- William Herrin b...@herrin.us https://bill.herrin.us/
Re: FlowSpec
On 23 Apr 2020, at 22:57, Denys Fedoryshchenko wrote: In general operators don't like flowspec Its increasing popularity tens to belie this assertion. Yes, you're right that avoiding overflowing the TCAM is very important. But as Rich notes, a growing number of operators are in fact using flowspec within their own networks, when it's appropriate. Smart network operators tend to do quite a bit of lab testing, prototyping, PoCs, et. al. against the very specific combinations of platforms/linecards/ASICs/OSes/trains/revisions before generally deploying new features and functionality; this helps ameliorate many concerns. Also, don't forget about S/RTBH. It's generally confined to within an operator's own span of administrative control for some of the same reasons as flowspec (not generally TCAM, per se, but concerns about giving Customer A the ability to interfere with Customer B's traffic, and the difficulty of implementing such constraints). It can be an option worth exploring, in many circumstances. Roland Dobbins
Re: mail admins?
On 4/21/20 7:46 PM, Scott Weeks wrote: --- m...@mtcc.com wrote: From: Michael Thomas To: nanog@nanog.org Subject: Re: mail admins? Date: Tue, 21 Apr 2020 17:34:36 -0700 On 4/21/20 5:19 PM, Scott Weeks wrote: I think you just need to let scripts run in your browser for nanog.org. sad. http://nanog.org used to be the brilliant example of a fully featured web site sans javascript, flash, ... --- I'm not one to plus-one anything, but this should be plus-infinity. I whined about it a year or so ago. Crickets. I gave up on doing anything on the web site because I can't get anything to work unless I make my computer less secure. Sad trend. More flash and trash marketing crap and less network engineering acumen. Like configuring routers from a web browser, rather than a CLI... this ship left port in the 90's. you might as well be an old man yelling at clouds. oh wait, randy does kind of resemble grandpa simpson :) -- So I should just get used to configuring routers with HTTP and Notepad and forget about that nasty, old, 20th century vi crap? :) No, but complaining about javascript on websites is about as relevant today as complaining that the horsepoop pushers union is lacking relevance. javascript is a hell of a lot safer than downloading native apps on your phone, for example. and don't get me started on OAUTH being used for native apps... Mike
Re: FlowSpec
On 2020-04-23 18:13, Colton Conor wrote: Do any of the large transit providers support FlowSpec to transit customers / other carriers, or is that not a thing since they want to sell DDoS protection services? FlowSpec sounds much better than RTBH (remotely triggered blackhole), but I am not sure if FlowSpec is widely implemented. I see the large router manufacturers support it. RETN They have extended blackholing, and FlowSpec, sure its all have costs. I'm using both services from them and quite satisfied. In general operators don't like flowspec, because it is not easy to implement it right, there is bugs and most important its "eating" TCAM. For example: https://blog.cloudflare.com/todays-outage-post-mortem-82515/
Re: FlowSpec
On 2020-04-23 18:13, Colton Conor wrote: Do any of the large transit providers support FlowSpec to transit customers / other carriers, or is that not a thing since they want to sell DDoS protection services? FlowSpec sounds much better than RTBH (remotely triggered blackhole), but I am not sure if FlowSpec is widely implemented. I see the large router manufacturers support it. RETN considered Tier-2? They offer it, but it is more expensive than
Re: Comcast - Significant v4 vs v6 throughput differences, almost stateful.
On Thu, Apr 23, 2020 at 8:27 AM Dovid Bender wrote: > We have customers in CT with the same issues. When did this start? > Seems to have started 5 years ago when we ran out of ipv4 and all comers needed to embrace ipv4 life-support mechanisms https://www.arin.net/vault/announcements/2015/20150924.html The e2e ipv6 internet being faster and more robust than life-supported, bot-ridden, and scarce ipv4 is a feature, not a bug. https://www.internetsociety.org/blog/2015/04/facebook-news-feeds-load-20-40-faster-over-ipv6/ > > On Thu, Apr 23, 2020 at 11:07 AM Nick Zurku wrote: > >> Hello all, >> >> I would appreciate if someone from Comcast could contact me about this. >> >> We’re having serious throughput issues with our AS20326 pushing packets >> to Comcast over v4. Our transfers are either the full line-speed of the >> Comcast customer modem, or they’re seemingly capped at 200-300KB/s. This >> behavior appears to be almost stateful, as if the speed is decided when the >> connection starts. As long as it starts fast it will remain fast for the >> length of the transfer and slow if it starts slow. Traces seem reasonable >> and currently we’ve influenced the path onto GTT both ways. If we prepend >> and reroute on our side, the same exact issue with happen on another >> transit provider. >> >> This issue does not affect v6 and that is full speed on every attempt. >> This may be regionalized to the Comcast Pittsburgh market. >> >> This is most widely affecting our linux mirror repository server: >> http://mirror.pit.teraswitch.com/ >> Our colocation customers who are hosting VPN systems are also noticing >> bottlenecks have started recently for their Comcast employees. >> >> -- >> Nick Zurku >> Systems Engineer >> TeraSwitch, Inc. >> nzu...@teraswitch.com >> >
Re: FlowSpec
Hi Colton, It is fairly common to use flowspec internally at an ISP for mitigation of DDoS attacks. eBGP flowspec is not very common though. I know of only a couple of ISPs that allow flowspec rules to be advertised by their customers. The biggest issue with this is that other providers are very hesitant to allow an external party to reach into their routers and modify the configuration (add a flowspec rule). I (with others at my company) had attempted to work on this to provide a validation mechanism that would be performed on the advertised rules before adding them to the router. We didn’t see much interest at that time on this. https://www.youtube.com/watch?v=rKEz8mXcC7o From conversations I have had with a couple of large ISPs recently it seems like there is an increased interest in this topic. Here is a document on flowspec best practices that I worked on for M3AAWG that may be of interest: https://www.m3aawg.org/sites/default/files/m3aawg-flowspec-bp-2019-02.pdf -Rich From: NANOG Email List on behalf of Colton Conor Date: Thursday, April 23, 2020 at 9:15 AM To: NANOG list Subject: FlowSpec Do any of the large transit providers support FlowSpec to transit customers / other carriers, or is that not a thing since they want to sell DDoS protection services? FlowSpec sounds much better than RTBH (remotely triggered blackhole), but I am not sure if FlowSpec is widely implemented. I see the large router manufacturers support it. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.
Re: Comcast - Significant v4 vs v6 throughput differences, almost stateful.
We have customers in CT with the same issues. When did this start? On Thu, Apr 23, 2020 at 11:07 AM Nick Zurku wrote: > Hello all, > > I would appreciate if someone from Comcast could contact me about this. > > We’re having serious throughput issues with our AS20326 pushing packets to > Comcast over v4. Our transfers are either the full line-speed of the > Comcast customer modem, or they’re seemingly capped at 200-300KB/s. This > behavior appears to be almost stateful, as if the speed is decided when the > connection starts. As long as it starts fast it will remain fast for the > length of the transfer and slow if it starts slow. Traces seem reasonable > and currently we’ve influenced the path onto GTT both ways. If we prepend > and reroute on our side, the same exact issue with happen on another > transit provider. > > This issue does not affect v6 and that is full speed on every attempt. > This may be regionalized to the Comcast Pittsburgh market. > > This is most widely affecting our linux mirror repository server: > http://mirror.pit.teraswitch.com/ > Our colocation customers who are hosting VPN systems are also noticing > bottlenecks have started recently for their Comcast employees. > > -- > Nick Zurku > Systems Engineer > TeraSwitch, Inc. > nzu...@teraswitch.com >
FlowSpec
Do any of the large transit providers support FlowSpec to transit customers / other carriers, or is that not a thing since they want to sell DDoS protection services? FlowSpec sounds much better than RTBH (remotely triggered blackhole), but I am not sure if FlowSpec is widely implemented. I see the large router manufacturers support it.
Comcast - Significant v4 vs v6 throughput differences, almost stateful.
Hello all, I would appreciate if someone from Comcast could contact me about this. We’re having serious throughput issues with our AS20326 pushing packets to Comcast over v4. Our transfers are either the full line-speed of the Comcast customer modem, or they’re seemingly capped at 200-300KB/s. This behavior appears to be almost stateful, as if the speed is decided when the connection starts. As long as it starts fast it will remain fast for the length of the transfer and slow if it starts slow. Traces seem reasonable and currently we’ve influenced the path onto GTT both ways. If we prepend and reroute on our side, the same exact issue with happen on another transit provider. This issue does not affect v6 and that is full speed on every attempt. This may be regionalized to the Comcast Pittsburgh market. This is most widely affecting our linux mirror repository server: http://mirror.pit.teraswitch.com/ Our colocation customers who are hosting VPN systems are also noticing bottlenecks have started recently for their Comcast employees. -- Nick Zurku Systems Engineer TeraSwitch, Inc. nzu...@teraswitch.com
IGMPv3/MLDv2 implementation and deployment survey
[ Apologies if you've seen this already - jtk ] Friends, Those of you with knowledge, interest, or deployment experience with IP multicast in real networks should consider taking the survey linked to below. Forwarded with knowledge and permission of the original email author. The survey deadline has been extended to the end of this month. John From: Stig Venaas Date: Tue, Feb 25, 2020 at 1:24 PM Subject: Re: IGMPv3/MLDv2 implementation and deployment survey To: Cc: The IETF PIM Working Group intends to progress IGMPv3 and MLDv2 from Proposed Standards to Internet Standards. To facilitate that, the working group would like to get input from operators and others deploying multicast, and implementors of these protocols, to learn what the current implementation and deployment status is. If you are using multicast, or have implemented these protocols, please respond to the survey at https://jisc.onlinesurveys.ac.uk/implementation-and-deployment-of-igmpv3-and-mldv2 It is sufficient to get responses from one person from each organization. The survey closes on March 13 2020. To get input from as many organizations as possible, please help forward this email, or distribute the URL, to any contacts you may have that have deployment or implementation experiences, on relevant mailing lists etc. If you have any queries about the survey, please contact the PIM WG chairs at pim-cha...@ietf.org. Regards Stig and Mike
Re: mail admins?
On 4/23/20 6:43 AM, John Osmon wrote: On Wed, Apr 22, 2020 at 08:05:39AM +0300, Töma Gavrichenkov wrote: On Wed, Apr 22, 2020, 12:45 AM Randy Bush wrote: sad. http://nanog.org used to be the brilliant example of a fully featured web site sans javascript, flash, ... That was long ago now. It was using Cvent for everything meeting-related for 3 years already, and Cvent doesn't feel good with JS turned off. Yes -- I think we all understand the technical problems with the site. Thanks for helping Randy with being precise. https://www.youtube.com/watch?v=tJ-LivK4-78 Sorry -- couldn't resist. Believe me, I've been on the receiving end of this myself.
Re: mail admins?
[ Bunch of replies to messages in thread bundled here. ] On Tue, Apr 21, 2020 at 06:28:48PM -0400, Bryan Fields wrote: > It's a mailman list, so nanog-ow...@nanog.org should work. If not reach out > to the communications committee. All mailing lists should support that, regardless of what's running them. Mailman, thankfully, makes it easy for configuring it by default. Other topics: - I've received erroneous bounces from @email.uscc.net as well. It should be possible to track down the culprit via Mailman's logs and the MTA's logs. - There is zero point in obfuscating email addresses in archives or anywhere else on the 'net. None. There hasn't been any point for most of twenty years. It's cargo cult "privacy" and that capability should be excised from Mailman's source code, because its presence unfortunately encourages people to indulge in a worst practice. - I also volunteered (in 2018? not sure, need coffee) to help out with -owner technical issues, but never heard anything back. - One of the queries that I've sent but not had an answer to is whether the entire archives are available in "mbox" format. (Including the older ones.) If they are, then it should be pretty easy to fold the old pre-Mailman archives into the current with-Mailman archives. ---rsk
