Re: rfc 1812 third party address on traceroute

[Randy Bush]

> I'm assuming you'd like this behavior on EdgeOS changed?

no, the opposite.  j & c got it right.  microtik did not.  vyatta seems
to have.

randy

Re: rfc 1812 third party address on traceroute

[Josh Reynolds]

I'm assuming you'd like this behavior on EdgeOS changed? I know a guy...
On Jun 5, 2016 8:41 PM, "Randy Bush"  wrote:

> > is anyone seeing the dreaded rfc1812 behavior in a citable fashion?  how
> > common is it?
>
> we verified that the juniper and cisco platforms we tested replied with
> the source address being the ingress interface.  this is, imiho, good.
>
> a kind soul actually sent citable tests
>
> > At least my MikroTik RB850Gx2, running 'latest stable' (RouterOS
> > v6.32.2) replies with the outbound interface, not the inbound.
> >
> > I'd assume this is because by default, icmp_errors_use_inbound_ifaddr in
> > linux is disabled, and they haven't changed the default.
> >
> > No idea if that can be tweaked in the weird maze of mikrotik config
> options.
>
> and from the same kind engineer
>
> > And just to add even more inconsistency, I checked on my Ubiquiti
> > EdgeMax (a VyOS fork) which does let me check the state of sysctls:
> >
> > router:/etc/sysctl.d$ cat 30-vyatta-router.conf
> > 
> > # Send ICMP responses with primary address of exiting interface
> > net.ipv4.icmp_errors_use_inbound_ifaddr=1
> > 
> >
> > So someone in Vyatta decided to explictly set this to be enabled.
>
> so one win and one loss
>
> randy
>

Re: rfc 1812 third party address on traceroute

[Randy Bush]

> is anyone seeing the dreaded rfc1812 behavior in a citable fashion?  how
> common is it?

we verified that the juniper and cisco platforms we tested replied with
the source address being the ingress interface.  this is, imiho, good.

a kind soul actually sent citable tests

> At least my MikroTik RB850Gx2, running 'latest stable' (RouterOS
> v6.32.2) replies with the outbound interface, not the inbound.
> 
> I'd assume this is because by default, icmp_errors_use_inbound_ifaddr in
> linux is disabled, and they haven't changed the default.
> 
> No idea if that can be tweaked in the weird maze of mikrotik config options.

and from the same kind engineer

> And just to add even more inconsistency, I checked on my Ubiquiti
> EdgeMax (a VyOS fork) which does let me check the state of sysctls:
> 
> router:/etc/sysctl.d$ cat 30-vyatta-router.conf
> 
> # Send ICMP responses with primary address of exiting interface
> net.ipv4.icmp_errors_use_inbound_ifaddr=1
> 
> 
> So someone in Vyatta decided to explictly set this to be enabled.

so one win and one loss

randy

Re: rfc 1812 third party address on traceroute

[Randy Bush]

>>.-.
>>| |
>>|   B |- D
>> S -| A  R|
>>|   C |- (toward S)
>>| |
>>`-'
>> 
>> of course, simpletons such as i would desire the source of the time
>> exceeded message to be A.  after all, this is the interface to which i
>> sent the icmp with the TTL to expire.
> 
> Do you mean the source address or the source interface?

the source address

> I'm not sure if you mean that, if sent through C it should have the
> source addres of A, or that it should actually be sent through A
> regardless of the routing table (which sounds better to me).

not to me.  i have kinda grown used to fibs

randy

Re: rfc 1812 third party address on traceroute

[Marc Storck]

I'm not saying anyone is wrong here. I merely want to point out eventual 
incompatabilities.

So please don't get me wrong.

Regards,

Marc

> On 1 juin 2016, at 23:46, William Herrin  wrote:
> 
> On Wed, Jun 1, 2016 at 3:16 PM, Marc Storck  wrote:
>>>  .-.
>>>  | |
>>>  |   B |- D
>>>   S -| A  R|
>>>  |   C |- (toward S)
>>>  | |
>>>  `-'
>> With BCP38 in mind, could there be situations
>> where Router R is not allowed to source packets
>> with address A out of interface C?
> 
> Hi Marc,
> 
> I think you're right. Address A in a /30 from ISP A. ISP C accepts
> source addresses from your /24 but not the A /30.  So if the router
> does not follow the RFC (sends an ICMP packet out C with a source
> address from A), typical asynchronous routing can result in
> black-holding the ICMP error message.
> 
> You've hit on a good reason to follow the RFC by default instead of
> doing what Randy wants. ;)
> 
> -Bill
> 
> 
> 
> 
> -- 
> William Herrin  her...@dirtside.com  b...@herrin.us
> Owner, Dirtside Systems . Web: 

Re: rfc 1812 third party address on traceroute

[William Herrin]

On Wed, Jun 1, 2016 at 3:16 PM, Marc Storck  wrote:
>>   .-.
>>   | |
>>   |   B |- D
>>S -| A  R|
>>   |   C |- (toward S)
>>   | |
>>   `-'
>>
> With BCP38 in mind, could there be situations
> where Router R is not allowed to source packets
> with address A out of interface C?

Hi Marc,

I think you're right. Address A in a /30 from ISP A. ISP C accepts
source addresses from your /24 but not the A /30.  So if the router
does not follow the RFC (sends an ICMP packet out C with a source
address from A), typical asynchronous routing can result in
black-holding the ICMP error message.

You've hit on a good reason to follow the RFC by default instead of
doing what Randy wants. ;)

-Bill




-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: rfc 1812 third party address on traceroute

[William Herrin]

On Wed, Jun 1, 2016 at 5:03 PM, Octavio Alvarez  wrote:
> On 05/31/2016 11:22 AM, William Herrin wrote:
>>> I'm not sure if you mean that, if sent through C it should have the
>>> source addres of A, or that it should actually be sent through A
>>> regardless of the routing table (which sounds better to me).
>>
>> That doesn't make sense. There may be multiple next hops out A. If the
>> next hop in the FIB is out C, how would the router pick the next hop
>> to send to out A?
>
> Back to the physical address that sent the TTL-offending packet.

Howdy,

That would be an example of a layer violation. The only guarantee that
layer 2 makes to layer 3 is that if you tell the layer 2 stack the
layer 3 next hop address on that lan segment, it can figure out where
to deliver your packet (via arp on ethernet, but this is not
necessarily true of other layer 2s).

Long story short, layer violations break things. Indeed, many of BGP's
thornier problems and the mess that is mobile routing can all be
traced to a single layer violation that TCP commits on IP.

Regards,
Bill Herrin


-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: rfc 1812 third party address on traceroute

[Octavio Alvarez]

On 05/31/2016 09:52 AM, Hugo Slabbert wrote:
>> I'm not sure if you mean that, if sent through C it should have the
>> source addres of A, or that it should actually be sent through A
>> regardless of the routing table (which sounds better to me).
> 
> How is the latter better?  What guarantees are there that the 
> adjacent L3 device on R's interface A has a route for S [?]

Consider this scenario:

 .---.  ISP1ADDR/30{
D---|B   R  A|---[ ISP 1 ] {
 `---C---' {
 |(towards S)  { S is someplace
 | { over this side
.F---. {
 ---|G  R2  H|--*[ ISP 2 ] {
`'  ISP2ADDR/30{

In the asterisk there is BCP38 filtering which won't allow ISPADDR/30.
The packet expired on R incoming from ISP 1. Under Randy's scenario, the
TTL-exceeded packet would get dropped by ISP2.

The only way for the packet to get through is to follow RFC 1812, or to
send it back through A using A's address (this follows RFC 1812 4.3.2.4).

> and if such a route exists that it doesn't simply point at R?

If the route points back to R, then R just forwards it using the routing
table as with any packet.


Best regards.

Re: rfc 1812 third party address on traceroute

[Hugo Slabbert]

On Wed 2016-Jun-01 14:03:41 -0700, Octavio Alvarez  
wrote:


On 05/31/2016 11:22 AM, William Herrin wrote:

I'm not sure if you mean that, if sent through C it should have the
source addres of A, or that it should actually be sent through A
regardless of the routing table (which sounds better to me).


That doesn't make sense. There may be multiple next hops out A. If the
next hop in the FIB is out C, how would the router pick the next hop
to send to out A?


Back to the physical address that sent the TTL-offending packet.


Which comes back to my question:
What guarantees do you have that the device at that physical address (so, 
adjacent off of R's interface A) has a valid route for S, and that the 
route does not simply point back to R?



Anyway, Randy's comment was about source address selection, not
routing. With the packet coming from S into interface A, he'd prefer
that the ICMP error message be sourced from the IP address assigned to
A, not the IP address assigned to C or R.


Thanks.


--
Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
pgp key: B178313E   | also on Signal


signature.asc
Description: Digital signature

Re: rfc 1812 third party address on traceroute

[Octavio Alvarez]

On 05/31/2016 11:22 AM, William Herrin wrote:
>> I'm not sure if you mean that, if sent through C it should have the
>> source addres of A, or that it should actually be sent through A
>> regardless of the routing table (which sounds better to me).
> 
> That doesn't make sense. There may be multiple next hops out A. If the
> next hop in the FIB is out C, how would the router pick the next hop
> to send to out A?

Back to the physical address that sent the TTL-offending packet.

> Anyway, Randy's comment was about source address selection, not
> routing. With the packet coming from S into interface A, he'd prefer
> that the ICMP error message be sourced from the IP address assigned to
> A, not the IP address assigned to C or R.

Thanks.

Re: rfc 1812 third party address on traceroute

[Marc Storck]

With BCP38 in mind, could therre be situations where Router R is not allowed to 
source packets with address A out of intergace C?

I think that the possibility does exist.

E.g. If interface A and C are upstream interfaces, router R may use an IP 
address from ISP A on interface A and an address from ISP C on interface C.

Obviously BCP38 is not widely deployed but yet...

Regards,

Marc

> On 31 mai 2016, at 07:05, Randy Bush  wrote:
> 
> rfc1812 says
> 
>   4.3.2.4 ICMP Message Source Address
> 
>   Except where this document specifies otherwise, the IP source address
>   in an ICMP message originated by the router MUST be one of the IP
>   addresses associated with the physical interface over which the ICMP
>   message is transmitted.  If the interface has no IP addresses
>   associated with it, the router's router-id (see Section [5.2.5]) is
>   used instead.
> 
> some folk have interpreted this to mean that, if a router R has three
> interfaces
> 
>   .-.
>   | |
>   |   B |- D
>S -| A  R|
>   |   C |- (toward S)
>   | |
>   `-'
> 
> if the source of a traceroute from S toward D with TTL to expire on R,
> and R's FIB wants to exit via C to get back to S (yes, virginia, the
> internet is highly asymmetric), the source address of the time exceeded
> message should be C.
> 
> of course, simpletons such as i would desire the source of the time
> exceeded message to be A.  after all, this is the interface to which i
> sent the icmp with the TTL to expire.
> 
> ras's preso,
> https://www.nanog.org/meetings/nanog47/presentations/Sunday/RAS_Traceroute_N47_Sun.pdf
> page 10 illustrates this issue with rfc1812
> 
> cursory research and talking with C & J seem to indicate that they do
> what i want not what some folk have interpreted 1812 to mean.  at least
> on some models.
> 
> is anyone seeing the dreaded rfc1812 behavior in a citable fashion?  how
> common is it?
> 
> randy

Re: rfc 1812 third party address on traceroute

[William Herrin]

On Tue, May 31, 2016 at 12:08 PM, Octavio Alvarez
 wrote:
>>.-.
>>| |
>>|   B |- D
>> S -| A  R|
>>|   C |- (toward S)
>>| |
>>`-'

> I'm not sure if you mean that, if sent through C it should have the
> source addres of A, or that it should actually be sent through A
> regardless of the routing table (which sounds better to me).

Howdy,

That doesn't make sense. There may be multiple next hops out A. If the
next hop in the FIB is out C, how would the router pick the next hop
to send to out A?

Anyway, Randy's comment was about source address selection, not
routing. With the packet coming from S into interface A, he'd prefer
that the ICMP error message be sourced from the IP address assigned to
A, not the IP address assigned to C or R.

Regards,
Bill

-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: rfc 1812 third party address on traceroute

[Hugo Slabbert]

On Tue 2016-May-31 09:08:42 -0700, Octavio Alvarez  
wrote:


On 05/30/2016 10:03 PM, Randy Bush wrote:

rfc1812 says

   4.3.2.4 ICMP Message Source Address

   Except where this document specifies otherwise, the IP source address
   in an ICMP message originated by the router MUST be one of the IP
   addresses associated with the physical interface over which the ICMP
   message is transmitted.  If the interface has no IP addresses
   associated with it, the router's router-id (see Section [5.2.5]) is
   used instead.

some folk have interpreted this to mean that, if a router R has three
interfaces

   .-.
   | |
   |   B |- D
S -| A  R|
   |   C |- (toward S)
   | |
   `-'

of course, simpletons such as i would desire the source of the time
exceeded message to be A.  after all, this is the interface to which i
sent the icmp with the TTL to expire.


Do you mean the source address or the source interface?

I'm not sure if you mean that, if sent through C it should have the
source addres of A, or that it should actually be sent through A
regardless of the routing table (which sounds better to me).


How is the latter better?  What guarantees are there that the adjacent L3 
device on R's interface A has a route for S and if such a route exists that 
it doesn't simply point at R?  As Randy so eloquently put it:



(yes, virginia, the internet is highly asymmetric)




Octavio.


--
Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
pgp key: B178313E   | also on Signal


signature.asc
Description: Digital signature

Re: rfc 1812 third party address on traceroute

[Octavio Alvarez]

On 05/30/2016 10:03 PM, Randy Bush wrote:
> rfc1812 says
> 
>4.3.2.4 ICMP Message Source Address
> 
>Except where this document specifies otherwise, the IP source address
>in an ICMP message originated by the router MUST be one of the IP
>addresses associated with the physical interface over which the ICMP
>message is transmitted.  If the interface has no IP addresses
>associated with it, the router's router-id (see Section [5.2.5]) is
>used instead.
> 
> some folk have interpreted this to mean that, if a router R has three
> interfaces
> 
>.-.
>| |
>|   B |- D
> S -| A  R|
>|   C |- (toward S)
>| |
>`-'
> 
> of course, simpletons such as i would desire the source of the time
> exceeded message to be A.  after all, this is the interface to which i
> sent the icmp with the TTL to expire.

Do you mean the source address or the source interface?

I'm not sure if you mean that, if sent through C it should have the
source addres of A, or that it should actually be sent through A
regardless of the routing table (which sounds better to me).

Octavio.

Re: rfc 1812 third party address on traceroute

[William Herrin]

On Tue, May 31, 2016 at 1:03 AM, Randy Bush  wrote:
>.-.
>| |
>|   B |- D
> S -| A  R|
>|   C |- (toward S)
>| |
>`-'
>
> i would desire the source of the time
> exceeded message to be A.  after all, this is the interface to which i
> sent the icmp with the TTL to expire.

Hi Randy,

I've thought for a number of years that routers should have an "ip
icmp-error-from" interface directive which allows the operator to
specify the source address for ICMP errors messages generated due to
packets received on that interface.

The behavior you describe where the time-exceeded message comes from C
instead of A is a nuisance. The RDNS gives you clues which point in
the wrong direction. Darn. Guess you'll have to rely on the preceding
router to tell you where the packet came from before it reached R.

The behavior Mikael notes is more deadly. Bogon filters drop packets
from RFC1918 sources. They aren't subtle enough to allow ICMP errors
through while dropping other IP packets. With bogon filters in place,
ICMP errors originated from RFC1918 space don't reach S. PMTUD dies
and your TCP connections die along with it. It's really important that
an Internet router not originate ICMP from 192.168.1.1!

It would also have been nice if ICMP error messages had defined a text
comment field where ops could place diagnostic information such as the
received interface. Overloading the functionality of the layer-3
address for any purpose (such as hanging an RDNS entry with textual
diagnostic information) is bad bad bad. Probably too late to shoehorn
that in.

Regards,
Bill Herrin



-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: rfc 1812 third party address on traceroute

[Owen DeLong]

It seems to me that a plain text reading of RFC-1812 is as Randy describes 
undesirable. It also seems that the violation of this text is commonplace in 
actual implementations because of yet another time where operators have made it 
clear to developers that the IETF is silly. 

I like the Linux solution... Comply with the RFC by default and provide a knob 
to do the "right thing" if desired. 

Best of all would be to put forth an errata against RFC1813 to change the text 
to specify the inbound interface of the packet triggering the ICMP message when 
applicable. The behavior currently described should be preserved for ICMP 
packets which are not triggered by inbound packets. 

Owen


> On May 31, 2016, at 01:27, Job Snijders  wrote:
> 
>> On Mon, May 30, 2016 at 10:03:33PM -0700, Randy Bush wrote:
>>   .-.
>>   | |
>>   |   B |- D
>>S -| A  R|
>>   |   C |- (toward S)
>>   | |
>>   `-'
>> 
>> if the source of a traceroute from S toward D with TTL to expire on R,
>> and R's FIB wants to exit via C to get back to S (yes, virginia, the
>> internet is highly asymmetric), the source address of the time exceeded
>> message should be C.
>> 
>> of course, simpletons such as i would desire the source of the time
>> exceeded message to be A.  after all, this is the interface to which i
>> sent the icmp with the TTL to expire.
>> 
>> is anyone seeing the dreaded rfc1812 behavior in a citable fashion?  how
>> common is it?
> 
> On most Linux the default behaviour is using source address "C", but
> this can be corrected by setting the following somewhere in your
> /etc/sysctl.d/ files:
> 
># make traceroute nice
>net.ipv4.icmp_errors_use_inbound_ifaddr=1
> 
> Kind regards,
> 
> Job

Re: rfc 1812 third party address on traceroute

[Job Snijders]

On Mon, May 30, 2016 at 10:03:33PM -0700, Randy Bush wrote:
>.-.
>| |
>|   B |- D
> S -| A  R|
>|   C |- (toward S)
>| |
>`-'
> 
> if the source of a traceroute from S toward D with TTL to expire on R,
> and R's FIB wants to exit via C to get back to S (yes, virginia, the
> internet is highly asymmetric), the source address of the time exceeded
> message should be C.
> 
> of course, simpletons such as i would desire the source of the time
> exceeded message to be A.  after all, this is the interface to which i
> sent the icmp with the TTL to expire.
> 
> is anyone seeing the dreaded rfc1812 behavior in a citable fashion?  how
> common is it?

On most Linux the default behaviour is using source address "C", but
this can be corrected by setting the following somewhere in your
/etc/sysctl.d/ files:

# make traceroute nice
net.ipv4.icmp_errors_use_inbound_ifaddr=1

Kind regards,

Job

Re: rfc 1812 third party address on traceroute

[Larry Sheldon]

I am completely innocent of rfc1812, and have been out of the game for a 
long time, but I am pretty sure I was taught (and in turn taught) that a 
router would reply using the address of the interface that originated 
the reply unless that interface was unnumbered, in which case it would 
reply from the loop-back address.


Never too old to learn something.


--
"Everybody is a genius.  But if you judge a fish by
its ability to climb a tree, it will live its whole
life believing that it is stupid."

--Albert Einstein

From Larry's Cox account.

Re: rfc 1812 third party address on traceroute

[Mikael Abrahamsson]

On Mon, 30 May 2016, Randy Bush wrote:


of course, simpletons such as i would desire the source of the time
exceeded message to be A.  after all, this is the interface to which i
sent the icmp with the TTL to expire.


I agree 100%, and I'd venture to guess that most of the people running 
networks expect it to work like you describe.


cursory research and talking with C & J seem to indicate that they do 
what i want not what some folk have interpreted 1812 to mean.  at least 
on some models.


is anyone seeing the dreaded rfc1812 behavior in a citable fashion?  how
common is it?


I have been told that there were versions of IOS XR that stopped doing 
what people wanted, people screamed, and then it's now back to the 
behaviour that you describe.


In RFC1812 2.2.7 there is talk about router-id. When reading that I think 
it is generic enough to work for IPv6 as well?


Another thing I've seen: People number their links with ULAs. ICMP error 
messages (including PTBs) are then sent from the router using the ULA 
address. This is obviously a disaster since that PTB sourced from ULA 
address is going to be BCP38:ed (hopefully). What's the interaction here 
with choosing a source address for the ICMP error message from something 
with the same RFC6724 label as the ICMP error message is being sent to?


--
Mikael Abrahamssonemail: swm...@swm.pp.se