Re: What's going on with AS147028?

[noc]

I scanned all LLIX members<https://ixp.ll-ix.com/api/v4/member-export/ixf/1.0> 
again, no LLIX members feeds junk routes to RIS now. I think we can remove 
AS141011/AS140731/AS141237/AS147028 from the filter list now.


From: JK-Net NOC 
Sent: Saturday, July 16, 2022 3:50 PM
To: nanog@nanog.org 
Cc: n...@he.net 
Subject: Re: What's going on with AS147028?

I think one of the reason is LL-IX strips their own ASN (59947) from the path 
and feed it to the route server. I see it as junk routes, and lots of people 
forgot(or intentionally not) to add 59947 back.
I scanned all LLIX members<https://ixp.ll-ix.com/api/v4/member-export/ixf/1.0> 
this morning(https://i.imgur.com/CsB8I1M.png), only AS140731 is still feeding 
junk routes generated by LL-IX to the RIS and bgp.tools, I think it's fine to 
remove AS141011/AS141237/AS147028 from the filter list.


On Wed, Jul 13, 2022 at 6:22 AM Mike Leber wrote:



This kind of thing is a problem from time to time with the data we get
from route collectors.

When we see it we have to add the culprit ASN to a filter list we keep
in bgp.he.net.

It tends to be a repeat problem with some collectors and some ASNs.

We haven't really figured out why people send junk routes to route
collectors.

The things we've seen aren't just route leaks.  We've seen a variety of
AS path spoofing.

We've already added this specific ASN to the filter list and pushed an
update for bgp.he.net.

Note, this email is specifically talking about routes received from
route collectors and not routes operationally received by he.net via BGP
sessions with actual networks.

Mike.

On 7/12/22 12:49 PM, Eric Dugas via NANOG wrote:


A friend of mine mentioned that both our Canadian ASNs were listed in
AS147028's peer list on https://bgp.he.net/AS147028 but we have no
adjacency to this network.

Their peer count jumped from 1 in May 2022 to 1,800 and just a few
days ago jumped to 8,800. Beside NL-IX, all the IX they are listed on
are virtual IX with a few dozen "hobby networks".

The only lead I have is they use HE as transit and they're pumping
back BGP feed to route collectors like RIPE RIS or Route Views with
routes stripped of HE's ASN.

Eric

Re: What's going on with AS147028?

[noc]

I think one of the reason is LL-IX strips their own ASN (59947) from the path 
and feed it to the route server. I see it as junk routes, and lots of people 
forgot(or intentionally not) to add 59947 back.
I scanned all LLIX members 
this morning(https://i.imgur.com/CsB8I1M.png), only AS140731 is still feeding 
junk routes generated by LL-IX to the RIS and bgp.tools, I think it's fine to 
remove AS141011/AS141237/AS147028 from the filter list.


On Wed, Jul 13, 2022 at 6:22 AM Mike Leber wrote:



This kind of thing is a problem from time to time with the data we get
from route collectors.

When we see it we have to add the culprit ASN to a filter list we keep
in bgp.he.net.

It tends to be a repeat problem with some collectors and some ASNs.

We haven't really figured out why people send junk routes to route
collectors.

The things we've seen aren't just route leaks.  We've seen a variety of
AS path spoofing.

We've already added this specific ASN to the filter list and pushed an
update for bgp.he.net.

Note, this email is specifically talking about routes received from
route collectors and not routes operationally received by he.net via BGP
sessions with actual networks.

Mike.

On 7/12/22 12:49 PM, Eric Dugas via NANOG wrote:


A friend of mine mentioned that both our Canadian ASNs were listed in
AS147028's peer list on https://bgp.he.net/AS147028 but we have no
adjacency to this network.

Their peer count jumped from 1 in May 2022 to 1,800 and just a few
days ago jumped to 8,800. Beside NL-IX, all the IX they are listed on
are virtual IX with a few dozen "hobby networks".

The only lead I have is they use HE as transit and they're pumping
back BGP feed to route collectors like RIPE RIS or Route Views with
routes stripped of HE's ASN.

Eric

Time Warner contact?

[noc]

We're seeing 90% packet loss between Time Warner in socal, starting at 
24.30.169.141 (tge0-9-0-23.lsapcawv02h.socal.rr.com), and anything in 74.125 
(google space).



Has been in ticket hell for the last week.  Could you please contact me off 
list for the ticket number and additional information?



This is a major impairment, at least for those TW customers it's affecting.



Thanks,

Spencer

PBRC NOC

RE: Route leak in Bangladesh

[Adam Datacenter - NOC]

Yes, we have seen one of our prefixes hikacked. We contacted to Fiberathome and 
they told us the issue has been solved.

Greetings.
Ferran.

-Mensaje original-
De: NANOG [mailto:nanog-boun...@nanog.org] En nombre de Grzegorz Janoszka
Enviado el: martes, 30 de junio de 2015 10:27
Para: nanog@nanog.org
Asunto: Route leak in Bangladesh


We have just received alert from bgpmon that AS58587 Fiber @ Home 
Limited has hijacked most of our (AS43996) prefixes and Hurricane 
Electric gladly accepted them.

Anybody see their prefixes hijacked as well?

-- 
Grzegorz Janoszka

BGP failure analysis and recommendations

[JRC NOC]

Hello Nanog -

On Saturday, October 19th at about 13:00 UTC we experienced an IP failure 
at one of our sites in the New York area.
It was apparently a widespread outage on the East coast, but I haven't seen 
it discussed here.


We are multihomed, using EBGP to three (diverse) upstream providers. One 
provider experienced a hardware failure in a core component at one POP.
Regrettably, during the outage our BGP session remained active and we 
continued receiving full routes from the affected AS.  And our prefixes 
continued to be advertised at their border. However basically none of the 
traffic between those prefixes over that provider was delivered. The bogus 
routes stayed up for hours. We shutdown the BGP peering session when the 
nature of the problem became clear. This was effective. I believe that all 
customer BGP routes were similarly affected, including those belonging to 
some large regional networks and corporations. I have raised the questions 
below with the provider but haven't received any information or advice.


My question is why did our BGP configuration fail?  I'm guessing the basic 
answer is that the IGP and route reflectors within that provider were still 
connected, but the forwarding paths were unavailable.  My BGP session 
basically acted like a bunch of static routes, with no awareness of the 
failure(s) and no dynamic reconfiguration of the RIB.


Is this just an unavoidable issue with scaling large networks?
Is it perhaps a known side effect of MPLS?
Have we/they lost something important in the changeover to converged 
mutiprotocol networks?
Is there a better way for us edge networks to achieve IP resiliency in the 
current environment?


This is an operational issue. Thanks in advance for any hints about what 
happened or better practices to reduce the impact of a routine hardware 
fault in an upstream network.


- Eric Jensen












Date: Wed, 23 Oct 2013 21:26:43 -0400
To: c...@chrisjensen.org
From: JRC NetOps 
Subject: Fwd: BGP failure analysis and recommendations



Date: Mon, 21 Oct 2013 23:19:28 -0400
To: christopher.sm...@level3.com
From: Eric Jensen 
Subject: BGP failure analysis and recommendations
Cc: "Joe Budelis Fast-E.com" 
Bcc: n...@jensenresearch.com

Hello Christopher Smith -

I left you a voicemail message today. The Customer Service folks also 
gave me your email address.


We have a small, but high-value multi-homed corporate network.
We operate using our AS number 17103.

We have BGP transit circuits with Level 3, Lightpath, and at our colo 
center (AS8001)

The Level 3 circuit ID is BBPM9946

On Saturday, October 19 2013 we had a large IP outage. I tracked it back 
to our Level 3 circuit and opened a ticket (7126634).
I have copied (below) an email I sent our channel salesman with more 
details about our BGP problems during your outage.
Briefly, I am very concerned that Level 3 presented routes to us that 
were not actually reachable through your network, and even worse Level 3 
kept advertising our prefixes even though your network couldn't deliver 
the traffic to us for those prefixes.


I believe that the BGP NLRI data should follow the same IP path as the 
forwarded data itself. Apparently this isn't the case at Level 3.
I also believe that your MPLS backbone should have recovered 
automatically from the forwarding failure, but this didn't happen either.

My only fix was to manually shutdown the BGP peering session with Level 3.

Can you explain to me how Level 3 black-holed my routes?
Can you suggest some change to our or your BGP configuration to eliminate 
this BGP failure mode?


Just to be clear, I don't expect our circuit, or your network, to be up 
all the time. But I do expect that the routes you advertise to us and to 
your BGP peers actually be reachable through your network. On Saturday 
this didn't happen. The routes stayed up while the data transport was down.


Our IPv4 BGP peering session with Level 3 remains down in the interim.
Please get back to me as soon as possible.

- Eric Jensen
AS17103
201-741-9509




Date: Mon, 21 Oct 2013 22:55:35 -0400
To: "Joe Budelis Fast-E.com" 
From: Eric Jensen 
Subject: Re:  Fwd: Level3 Interim Response
Bcc: n...@jensenresearch.com

Hi Joe-

Thanks for making the new inquiry.
This was a big outage. Apparently Time Warner Cable and  Cablevision 
were affected greatly. Plus many large corporate networks. And of course 
all the single-homed Level 3 customers worldwide. My little network was 
just one more casualty.


See:

http://www.dslreports.com/forum/r28749556-Internet-Level3-Outage-

http://online.wsj.com/news/articles/SB10001424052702304864504579145813698584246

For our site, the massive outage started at about 9:00 am Saturday and 
lasted until after 2:00PM. I opened a ticket about 9:30 am but only 
realized the routing issues and took down our BGP session about 12:00 to 
try to minimize the problems for our traffic caused by their 
misconfigured BGP.


There can always be equipment failures

Re: Metering power in data center

[NOC]

Hi,

For our need, we use : http://www.lem.com/ They have a lot of products 
to do that. We use a magnetic meter. You don't need to break the circuit 
to implement it.


Regards,

Bastien


Wallace Keith a écrit :

-Original Message-
From: Jay Nakamura [mailto:zeusda...@gmail.com] 
Sent: Thursday, April 08, 2010 2:10 PM

To: NANOG
Subject: Metering power in data center

I am looking for suggestions on devices that can
monitor(A)/meter(kw/h) power usage in a data center.  Getting a
metered PDU everywhere seems a little expensive and cumbersome.

Are there devices you can wire into breaker box to meter each AC
circuit?

Thanks in advance for any suggestions.

-Jay

We have a few of these running: http://www.emon.com/products_webmon.html
-Keith

Re: Interesting Point of view - Russian police and RIPE accused of aiding RBN

[noc acrino]

Greetings!

By the way, Jeffrey, by the 24th of October, when you posted the information
that the RBN is located in our networks we couldn't even know about any
malware redirectors on our clients resources -
http://www.stopbadware.org/reports/asn/44571. I'm trying to solve the Google
SB issue (still under investigation both by our team and the resource owner,
but NB - it's only 1 ip from 345 sites tested by Google ) but one little
question - how did you get to know about the malware abuse _before_ the
actual report on stopbadware.org or on google? What were your conclusions
based on? Why didn't you write to the abuse email the way it's traditionally
done in the network operators' sphere?

Kanak

Akrino Abuse Team

Re: Interesting Point of view - Russian police and RIPE accused of aiding RBN

[noc acrino]

2009/11/6 Jeffrey Lyon 

>  The primary issue is that we receive a fair
> deal of customers who end up with wide scale DDoS attacks followed by
> an offer for "protection" to move to your network. In almost every
> case the attacks cease once the customer has agreed to pay this
> "protection" fee. Every one of these attacks was nearly identical in
> signature.
>

By the way, Jeffrey, we can provide reports on HTTP-flood because our system
builds it's signatures on http traffic dumps like

=== IP: 88.246.76.65, last receiving time: 2009-10-25T23:07:37+03:00, many
identical requests (length 198):
GET / HTTP/1.1
Accept: */*
Accept-language: en-us
User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.1.1)
Gecko/20061204 Firefox/2.0.0.1
Host: [censored]
Connection: Keep-Alive

So using this info we can map botnets, learn different attacks and in
collaboration with ISPs - find CCs of new botnets. And what are your
accusations of the identical signatures based on when simple Staminus
resellers (like you are) do not have access to their signatures database?

Kanak

Akrino Abuse Team

Re: Interesting Point of view - Russian police and RIPE accused of aiding RBN

[noc acrino]

Hello, Jeffery and other NANOC members.

Sorry for making another thread - I'm not too experienced in mailgroups.

The problem is in structure of new generation advert or banner networks -
they allow to return other subject traffic  to the partner's URL. And this
could also be used to redirect the traffic to different exploits (a simple
way to compromise a banner network or hosting provider). This is extremely
hard to monitor or to take preventive measures in case of a large banner or
advert network. Unfortunately Google doesn't provide a detailed report on
their check results: this could allow the resource's owner easily block
their partners in that case.

Anyway I'll contact the owner of this resource (91.202.63.96) now in order
to perform a check of their partners. I suppose, just having a few domains
would be enough.

The other resource is situated on the public ip of our reseller - I'll ask
him to check this domain, too.

Thank you for that information, I'll report on that issue later.

Kanak

Akrino Support Team


2009/11/7 Jeffrey Lyon 

> Kanak,
>
> Can you please detail your plans to correct the malware issues on your
> network? (reference:
> http://google.com/safebrowsing/diagnostic?site=AS:44571 ).
>
> Best regards, Jeff
>
>
>
> [offlist communication snipped for privacy]
>
> >
> > Kanak
> >
> > Akrino Abuse Team
> >
>
>
>
> --
> Jeffrey Lyon, Leadership Team
> jeffrey.l...@blacklotus.net | http://www.blacklotus.net
> Black Lotus Communications of The IRC Company, Inc.
>
> Platinum sponsor of HostingCon 2010. Come to Austin, TX on July 19 -
> 21 to find out how to "protect your booty."
>