dot1q encapsulation overhead?
A while back we had a customer colocated vpn router (2911) come in and we put it on our main vlan for initial set up and testing. Once that was done, I created a separate VLAN for them and a dot1q subinterface on an older, somewhat overloaded 2811. I set up the IPSec Tunnel, a /30 for each end to have an IP and all the static routes needed to make this work and it did. However, a few days later they were complaining of slow speeds...I don't recall, but maybe something like 5mbs when they needed 20 or so. We had no policing on that port. After a lot of testing, we tried putting them back on the main, native vlan and it worked fine...they got the throughput they needed. So my question is: could the dot1q encapsulation be causing throughput issues on a 2811 that's already doing a lot? I regret that I don't recall what sh proc cpu output was, or if I even ran it at all. It was kind of hectic just to get it fixed at the time. Well, a few months later (last week), the chicken came home to roost when their IPSec tunnel started proxy ARP puking stuff to our side that temporarily took out parts of our internal LAN. I have requested a 2911 replacement for the 2811 because I have seen the 2811 cpu load max out a few times when passing lots of traffic. I am hoping it will allow us to go back to this VLAN setup again, but I've never heard whether dot1q adds any overhead.
Re: Bell Canada outage?
Hi, .-- My secret spy satellite informs me that at 12-08-08 11:35 AM Darius Jahandarie wrote: On Wed, Aug 8, 2012 at 2:31 PM, Zachary McGibbon zachary.mcgibbon+na...@gmail.com wrote: Anyone at Bell Canada / Sympatico can tell us what's going on? Our routing table is going nuts with Bell advertising a lot of routes they shouldn't be Bell leaked a full table. To add to the fun, it seems that TATA took the full table and releaked it. A quick analysis leads met to believe AS46618 ( Dery Telecom Inc) is the cause of this. AS46618 is dual homed to VIDEOTRON and Bell. What seems to have happened is that they leaked routes learned from VIDEOTRON to Bell. Based on BGP data I see that at 17:27 UTC AS46618 ( Dery Telecom Inc) started to leak a 'full table', or at least a significant chunk of it to its provider Bell AS577. Bell propagated that to it's peers. Tata was one of the ones that accepted all of that. I can see that Bell propagated at least 74,109 prefixes learned from AS46618 to Tata. Tata selected 70,160 of those routes. Interesting. I have a server hosted on Bell Canada's network and I saw an outage of about 30 minutes today, but it ONLY affected connections from Verizon's network. This includes my own FIOS connection. I still could connect to the server through Comcast, Level 3 and XO with no problems. Traceroutes from my Verizon IP only got 2 hops, stopping at a philly router and traceroutes back to the same IP from that server got as far as NYC.
Re: J.D. Falk has passed on
Somewhere in hell, Spamford Wallace is smiling. A wonderful colleague, friend, and leading purveyor of industry counter-rhetoric solutions. http://www.maawg.org/page/memorial-jd-falk http://www.cauce.org/2011/11/jdfalk.html http://www.facebook.com/jdfalk regards, fh --- Pure J.D. :) Whether you are acting as a Mailbox Provider or a Feedback Consumer, Complaint Feedback processing can be complex and scary -- or, with some intelligence and automation, simple and easy. In either case, it is an important and necessary tool for detecting messaging abuse and ensuring End User satisfaction. http://www.rfc-editor.org/?rfc/pdfrfc/rfc6449.txt.pdf
RE: Outgoing SMTP Servers
On our retail footprint we block outbound traffic from customers with dynamic IPs towards port 25, our support tells them to use their ISP's port 587 server That being said, since all of our home users have 50 mbit/sec or greater upload speeds we are pretty paranoid about the amount of spam that could be originated. We don't block anything on static assignments. Honestly, even as a very geeky user, I probably would not have noticed the block and I can confirm that it is massively important to lowering our spam footprint as a network. I asked our support people, and none of them had ever really had an issue with this policy in terms of keeping customers. I agree with Ricky's current comment on this thread, blocking is unfortunately necessary on the modern consumer portions of the internet. Exactly. Just like not having wide open SMTP relays became unfortunately necessary over a dozen years ago. It's just the way it is and there is a solution for it.
Cisco Ironport and Senderbase...how to get delisted?
We had two users fall for a phishing email recently, and of course the result was that he gave his user/pass to a spammer. We caught one of them in time, but the other got out many thousands of spam the other night before being discovered. I am in the process of cleaning this up. Spamcop and others were good about delisting us promptly. Others will within the next day. However, Senderbase, apparently used in Cisco's Ironport, will let you look up your IP and tell you that your reputation is poor, but offers no way to get delisted. It refers you to Spamcop, which I imagine they rely on for listings, but not delistings. For now, I'm re--routing per domain to a second server, but I'd appreciate any tips if there are any. Seems a lot of .edu's use senderbase.
RE: Cisco Ironport and Senderbase...how to get delisted?
We weren't listed in the PSBL. First thing I did was a few multi-DNSBL lookups and only found a couple of obscure (to me) ones, which I immediately filled out for delisting. Interestingly, comcast.net was BLing us, complete with URL to do apply for delisting. I did, and got a response that we weren't listed. I don't date take down re-routing just yet :-/ Just went through this the other day. In our case once we removed the IP from the PSBL, our senderbase reputation went back to neutral within about an hour or two. Seemed to be pretty directly related. I suspect senderbase checks a handful of reliable BL's and your IP reputation is greatly affected by listings in them. As far as I can tell there is no way to contact them to expedite things. Worse is this one: http://www.commtouch.com/Site/Resources/Check_IP_Reputation.asp Unless it's just me, that page is broken and it's the only way to check listings or request removals. Andrew -Original Message- From: u...@3.am [mailto:u...@3.am] Sent: Wednesday, August 17, 2011 1:58 PM To: nanog@nanog.org Subject: Cisco Ironport and Senderbase...how to get delisted? We had two users fall for a phishing email recently, and of course the result was that he gave his user/pass to a spammer. We caught one of them in time, but the other got out many thousands of spam the other night before being discovered. I am in the process of cleaning this up. Spamcop and others were good about delisting us promptly. Others will within the next day. However, Senderbase, apparently used in Cisco's Ironport, will let you look up your IP and tell you that your reputation is poor, but offers no way to get delisted. It refers you to Spamcop, which I imagine they rely on for listings, but not delistings. For now, I'm re--routing per domain to a second server, but I'd appreciate any tips if there are any. Seems a lot of .edu's use senderbase. - No virus found in this message. Checked by AVG - www.avg.com Version: 10.0.1392 / Virus Database: 1520/3840 - Release Date: 08/17/11
Re: Cisco Ironport and Senderbase...how to get delisted?
Thanks for the tip (BTW, top-post haters, I didn't start it!). I was quickly delisted by SpamCop, but here is their response: -- Once all spam issues have been addressed, **reputation recovery can take anywhere from a few hours to __just over one week__ to improve**, depending on the specifics of the situation, and how much email volume the IP sends. Complaint ratios determine the amount of risk for receiving mail from an IP, so logically, reputation improves as the ratio of legitimate mails increases with respect to the number of complaints. Speeding up the process is not really possible. SenderBase Reputation is an automated system over which we have very little manual influence. Mailflow policy is the sole domain and responsibility of the recipient;SenderBase has no control over how passive or aggressive Cisco-IronPort customers choose to be when implementing SenderBase reputation information. While the reputation is improving, we suggest contacting domains which are rejecting or throttling mail from the IP, and request they whitelist the IP temporarily. Regards, -SenderBase Support - In sort, wait... Once you're de-listed from SpamCop (which is owned by IronPort and plays a non-trivial part in their SenderBase scoring) you should find that your reputation increases fairly quickly - normally within 24 hours presuming that the spam has actually stopped. Scott. On Wed, Aug 17, 2011 at 1:57 PM, u...@3.am wrote: We had two users fall for a phishing email recently, and of course the result was that he gave his user/pass to a spammer. We caught one of them in time, but the other got out many thousands of spam the other night before being discovered. I am in the process of cleaning this up. Spamcop and others were good about delisting us promptly. Others will within the next day. However, Senderbase, apparently used in Cisco's Ironport, will let you look up your IP and tell you that your reputation is poor, but offers no way to get delisted. It refers you to Spamcop, which I imagine they rely on for listings, but not delistings. For now, I'm re--routing per domain to a second server, but I'd appreciate any tips if there are any. Seems a lot of .edu's use senderbase.
Stupid Cisco ACL question
Ok, I've done a lot of Cisco standard and extended ACLs, but I do not understand why the following does not work the way I think it should. Near the end of this extended named ACL, I have the following: permit tcp any eq 443 any permit tcp any eq 80 any deny ip any host 2.2.3.4 permit ip any any This is applied to an inbound interface(s). We want anybody outside to be able to reach ports 80 and 443 of any host on our network, no matter what, then block ALL other access to select hosts, such as 2.2.3.4, even ICMP. However, as soon as I apply this rule to the interface, ports 80 and 443 of that host become unreachable. A telnet to 2.2.3.4 443 gets Connection refused until I tear out the deny ACL above. I even tried adding udp for both ports, to no avail. I had always thought that these ACLs were processed in order, so that the explicit permit statement, though limited to a specific protocol but for all hosts, gets considered before the explicit deny statement for all IP to a particular host. What did I forget to consider? TIA,
Re: Stupid Cisco ACL question
Thanks everyone, of course this is what I wanted. Like I said, a stupid ACL question...I'm blaming heavy medication, sorry for the noise! On Thu, 21 Apr 2011, u...@3.am wrote: permit tcp any eq 443 any permit tcp any eq 80 any deny ip any host 2.2.3.4 permit ip any any This is applied to an inbound interface(s). We want anybody outside to be able to reach ports 80 and 443 of any host on our network, no matter what, then block ALL other access to select hosts, such as 2.2.3.4, even ICMP. However, as soon as I apply this rule to the interface, ports 80 and 443 of that host become unreachable. A telnet to 2.2.3.4 443 gets Connection refused until I tear out the deny ACL above. I even tried adding udp for both ports, to no avail. Your ACL is apply the 80 443 as source ports, not destination ports. You probably want: permit tcp any any eq 443 permit tcp any any eq 80 deny ip any host 2.2.3.4 permit ip any any Jay Ford, Network Engineering Group, Information Technology Services University of Iowa, Iowa City, IA 52242 email: jay-f...@uiowa.edu, phone: 319-335-, fax: 319-335-2951
POP3 DoS attacks and mailanyone.net?
For the first time since I can remember, my POP3 server was effectively shut down by too many simultaneous connections today. The first fix I tried was to raise the number of connections from the default 40 to 100, but the problem soon returned. I finally ipfw'd off the offending IP (98.190.204.2 for anyone interested), then went to look for other possible offenders in the log. I noticed several thousand connections today to a few dozen former users from 4 IPs from 208.70.128.0/21. One of the users was actually legitimate. These IPs belong to mailanyone.net. The tech contact in their ARIN record is listed as: OrgTechHandle: BHE57-ARIN OrgTechName: Heitman, Bryan OrgTechPhone: +1-816-587-4700 OrgTechEmail: hostmas...@mailanyone.net However, that phone number goes to a UPS store that has no idea what I'm talking about. I then dialed their suppseod NOC number: Comment:FuseMail, LLC Network Operations Center contact Comment:877.888.3873 x3 I am on hold with that number right now with some very loud and annoying music. Can anyone offer any insight as to these people and how/who to deal with there? Would a provider be amiss to just block their entire /21? TIA, James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am =
Re: Issues with Gmail
pop.gmail.com is answering on port 995 (pop3 ssl) as well, so I think it's safe to assume this is probably a httpd-side problem. On Tue, 1 Sep 2009, Jeff Kell wrote: m...@sabbota.com wrote: I think it just may be front end services that are impacted. I'm able to send/receive mail through my BB BIS gmail account. IMAP seems to still be up. Jeff James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am =
Data Center QoS equipment breaking http 1.1?
Sorry if this is a little OT, but we're seeing a serious problem and was wondering if it is what I think it is. In short: I have been moving services off of our servers in a data center onto a server at eSecuredata, who rents dedicated servers. The idea is to lower costs and eliminate having to deal with hardware. The advertise unmetered bandwidth, but mention QoS measure to control bandwidth hogs. One of my customers, whose site I just moved from a unique IP virtual host on my old server onto an Apache NameVirtualHost on the new one, worked fine at first. Then today, they started complaining about getting one of our home pages. I figured DNS or web caching issues, until I started seeing it for myself. It was no caching issue, it was NameVirtualHost breaking. I poured over my configs (I've done this config countless times), and saw this in the apache docs: http://httpd.apache.org/docs/2.2/vhosts/name-based.html Some operating systems and network equipment implement bandwidth management techniques that cannot differentiate between hosts unless they are on separate IP addresses. So, I installed lynx on the server, and sure enough, it worked perfectly fine there, just not from anywhere outside eSecuredata's network that I could see. Can anyone shed any light on this particular practice, of this company in particular? thanks James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am =
Re: Data Center QoS equipment breaking http 1.1?
Please disregard this idiocy of mine...it appears that the Apache UseCanonicalName directive selectively breaks some NameVirtualHosts, while leaving others unscathed, but turning it off fixed it anyway. On Fri, 31 Jul 2009, u...@3.am wrote: Sorry if this is a little OT, but we're seeing a serious problem and was wondering if it is what I think it is. In short: I have been moving services off of our servers in a data center onto a server at eSecuredata, who rents dedicated servers. The idea is to lower costs and eliminate having to deal with hardware. The advertise unmetered bandwidth, but mention QoS measure to control bandwidth hogs. One of my customers, whose site I just moved from a unique IP virtual host on my old server onto an Apache NameVirtualHost on the new one, worked fine at first. Then today, they started complaining about getting one of our home pages. I figured DNS or web caching issues, until I started seeing it for myself. It was no caching issue, it was NameVirtualHost breaking. I poured over my configs (I've done this config countless times), and saw this in the apache docs: http://httpd.apache.org/docs/2.2/vhosts/name-based.html Some operating systems and network equipment implement bandwidth management techniques that cannot differentiate between hosts unless they are on separate IP addresses. So, I installed lynx on the server, and sure enough, it worked perfectly fine there, just not from anywhere outside eSecuredata's network that I could see. Can anyone shed any light on this particular practice, of this company in particular? thanks James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am = James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am =
Verizon transparent web caching issue? WASRe: Data Center QoS equipment breaking http 1.1?
Disregard my disregard. The problem resurfaced with no changes on my part. I purged browser caches and tried them from 3 browsers and each time: http://www.countytheater.org redirected to: http://webmail.ns3.pil.net/ which is another NameVhost on that server sharing that IP. This is incorrect. However, I then switch from a Verizon connection to an ATT 3g connection on the IPhone and the problem goes away. Has anyone heard of upstream transparent caching issues causing this kind of problem? Does anyone else here get the redirect instead of the correct page? TIA On Fri, 31 Jul 2009, u...@3.am wrote: Please disregard this idiocy of mine...it appears that the Apache UseCanonicalName directive selectively breaks some NameVirtualHosts, while leaving others unscathed, but turning it off fixed it anyway. On Fri, 31 Jul 2009, u...@3.am wrote: Sorry if this is a little OT, but we're seeing a serious problem and was wondering if it is what I think it is. In short: I have been moving services off of our servers in a data center onto a server at eSecuredata, who rents dedicated servers. The idea is to lower costs and eliminate having to deal with hardware. The advertise unmetered bandwidth, but mention QoS measure to control bandwidth hogs. One of my customers, whose site I just moved from a unique IP virtual host on my old server onto an Apache NameVirtualHost on the new one, worked fine at first. Then today, they started complaining about getting one of our home pages. I figured DNS or web caching issues, until I started seeing it for myself. It was no caching issue, it was NameVirtualHost breaking. I poured over my configs (I've done this config countless times), and saw this in the apache docs: http://httpd.apache.org/docs/2.2/vhosts/name-based.html Some operating systems and network equipment implement bandwidth management techniques that cannot differentiate between hosts unless they are on separate IP addresses. So, I installed lynx on the server, and sure enough, it worked perfectly fine there, just not from anywhere outside eSecuredata's network that I could see. Can anyone shed any light on this particular practice, of this company in particular? thanks James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am = James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am = James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am =
Re: Verizon transparent web caching issue? WASRe: Data Center QoS equipment breaking http 1.1?
Again, turned out to be my own stupidity. It was just DNS on a secondary DNS server, which was pointing to the old IP, which was redirecting to the new IP, but at that point, the headers are lost. I would have thought that on MacOSX (my client; the server is FreeBSD 7.2-STABLE), if I tell the /etc/resolv.conf to look at the primary name server only, which has the correct info, plus doing a dnscacheutil -flushcache, that this wouldn't be an issue. Apparently, I was wrong, or perhaps it doesn't override what Verizon does with my browser's queries, despite what nslookup shows in a terminal window. On Fri, 31 Jul 2009, u...@3.am wrote: Disregard my disregard. The problem resurfaced with no changes on my part. I purged browser caches and tried them from 3 browsers and each time: http://www.countytheater.org redirected to: http://webmail.ns3.pil.net/ which is another NameVhost on that server sharing that IP. This is incorrect. However, I then switch from a Verizon connection to an ATT 3g connection on the IPhone and the problem goes away. Has anyone heard of upstream transparent caching issues causing this kind of problem? Does anyone else here get the redirect instead of the correct page? TIA On Fri, 31 Jul 2009, u...@3.am wrote: Please disregard this idiocy of mine...it appears that the Apache UseCanonicalName directive selectively breaks some NameVirtualHosts, while leaving others unscathed, but turning it off fixed it anyway. On Fri, 31 Jul 2009, u...@3.am wrote: Sorry if this is a little OT, but we're seeing a serious problem and was wondering if it is what I think it is. In short: I have been moving services off of our servers in a data center onto a server at eSecuredata, who rents dedicated servers. The idea is to lower costs and eliminate having to deal with hardware. The advertise unmetered bandwidth, but mention QoS measure to control bandwidth hogs. One of my customers, whose site I just moved from a unique IP virtual host on my old server onto an Apache NameVirtualHost on the new one, worked fine at first. Then today, they started complaining about getting one of our home pages. I figured DNS or web caching issues, until I started seeing it for myself. It was no caching issue, it was NameVirtualHost breaking. I poured over my configs (I've done this config countless times), and saw this in the apache docs: http://httpd.apache.org/docs/2.2/vhosts/name-based.html Some operating systems and network equipment implement bandwidth management techniques that cannot differentiate between hosts unless they are on separate IP addresses. So, I installed lynx on the server, and sure enough, it worked perfectly fine there, just not from anywhere outside eSecuredata's network that I could see. Can anyone shed any light on this particular practice, of this company in particular? thanks James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am = James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am = James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am = James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am =
Re: Level 3 - legacy Wiltel/Looking Glass bandwidth
On Wed, 1 Jul 2009, Scott Howard wrote: We're looking at getting connectivity via Level 3 in a particular datacenter, but we're being told that it's legacy Wiltel/Looking Glass rather than true Level 3. Given that both of these acquisitions occurred years ago should I be worried, or is this legacy connectivity the same as L3 at any other datacenter? While I cannot speak directly to their treatment of former Wiltel customers, I can tell you that once they acquired Broadwing, service in their Norristown, PA data center went from not-so-great to completely unacceptable. IIRC, we've had about 6 multi-hour outages in the past year. Apparently, that data center is connected to their Philly POP via a Foundry Big Iron switch that suffers from broadcast storms periodically, which can only be fixed by their dispatching a tech to Philly to power-cycle it, which for some reason takes from 1 to 4 hours. Why they're not familiar with remote-power cycling equipment is beyond me, let alone why they haven't resolved the issue properly, despite having supposedly replaced hardware at one point. My 3 year contract is up next month, after which I am so out of there. The fact that L3 tried to double their price on me in the middle of that contract, only backing down after getting two lawyers involved, didn't help my opinion of them as a company, either. James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am =
Level 3 Philly Major outage?
I have a cabinet at Broadwing's (now Level 3) Norristown data center, for over 2 years now. It has always seemed something of a backwater in terms of Broadwing's network, and even more so with Level 3. An outage started yesterday morning, reportedly caused by a broadcast storm on their Philly Big Iron switch that connects to Norristown (this happened before last August). In a couple of hours, they had it fixed, only for it to go into up-and-down mode a couple of hours later, for the rest of the day. I escalated the ticket at around 6:20pm, but saw no lasting improvment. This morning, I started getting customer calls that it was down again. I called Level 3 once more, after seeing the same Analysis in Progress message on their portal from the previous evening. The rep described that a major outage was happening in Philly, even though there's nothing on their Network Outages list about it. I would think that if this was a Major Outage in Philadelphia for Level 3, there would be some NANOG chatter on it, which I don't see. It's back up for now, but does anybody have any knowledge of this? As an aside, would it be ok for me to solicit colocation services in the Philadelphia area on this list? A change has to be made at some point. Thanks, James Smallacombe PlantageNet, Inc. CEO and Janitor u...@3.am http://3.am =
[NANOG] Level3 not honoring Broadwing contracts?
In 2006, I signed a 3 year contract with Broadwing for a 1 cabinet colocation with 6Mbs dedicated for under $1,000/mo. A few weeks ago, about halfway through this contract, I get a letter from Level 3's Director of Colocation that they are going to raise my price by several hundred dollars a month. I spoke with my new Level 3 rep, and he just notified me that their legal deparment confirms that all they have to do is give me 30 days notice to increase their price. This does not make sense to me. I am bound to a 3 year contract, where I have to pay them the rest of the term if I were to leave early, but they can jack up the price by 40-50% during that time, arbitrarily? I do not see that provision in my contract, and would rather avoid legal expenses if possible. Has anyone else had to deal with this sort of thing from Level 3? TIA, James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am = ___ NANOG mailing list NANOG@nanog.org http://mailman.nanog.org/mailman/listinfo/nanog
Port 1080 probes from AOL
One of my virtual web host servers have been getting multiple probes to TCP port 1080 (socks) every day for months from AOL IP addresses. Is AOL known to be doing something relatively innocuous on that port? I ask because I have portsentry null routing IP addresses that make probes like this. TIA, James Smallacombe PlantageNet, Inc. CEO and Janitor [EMAIL PROTECTED] http://3.am =
