Re: [Lsr] Dynamic flow control for flooding

[tony . li]

Les,


> Very true.  It would not be unreasonable for an implementation to report free 
> space in the FIFO (in number of PDUs) divided by the number of active 
> adjacencies.  Everyone gets their fair share.
>  
> [If dynamic flooding is enabled, this could be based on the number of 
> adjacencies that should be actively flooding. That should be a much smaller 
> number.]
> [Les:] So you are agreeing that when a receiver wants to “dial back” it will 
> need to do so on all interfaces enabled for flooding?


Certainly.  There’s one CPU, one input queue.   If there’s a flooding event in 
progress, then it’s likely to arrive from all sides.  Dialing back on all 
interfaces makes sense as self-preservation.


> LSPs may be dropped at lower layers – IS-IS receiver may be unaware that the 
> overload condition exists
>  
> That’s an implementation problem. The implementation NEEDS to be able to see 
> its input queue plus input drops.
>  
> [Les:] And you want to ship this feature when…? 😊
> I think this is a difficult ask.
> Before we decide this is what is required we should explore the path of 
> monitoring the unacknowledged Tx queue.


That’s the tail wagging the dog.  

I’m less concerned about when.  The people who want this feature are going to 
pay for it.  They will set the agenda and schedule, not us.  We need to figure 
out how to do it properly.

 
> Updating hellos dynamically to alter flooding transmission rate is an OOB 
> signaling mechanism consuming  resources at a time when routers are the most 
> busy
> Consistent flooding rates will require updated hellos be sent to all 
> neighbors – exacerbating the cost on both sender and receiver
>  
> This is why I suggest sending the feedback in PSNPs as well as in IIHs.  
> Regardless of the details, we need to consider sending PSNPs back more 
> frequently.  I concur that optimizing the rate and triggers for sending more 
> PSNPs is an open issue.
>  
> Strictly speaking, sending a TLV inside of our protocol PDUs is an in-band 
> signaling mechanism.
> [Les:] I agree – PSNP would be better since we need to send it anyway in 
> order to ACK. Still does not convince me this is the preferred approach – but 
> I agree it is better than hellos.


Please note that I prefer BOTH.  Sending it in IIHs is useful because it allows 
adjacencies that are not expecting a PSNP to still get some feedback. It would 
also prevent problems that we’ve seen with TCP and connections getting stuck 
with a zero window.


> The resources consumed by maintaining a running count of a queue in silicon 
> or in process space is effectively zero.
>  
> [Les:] It is not about counting – it is about how a given queue might be 
> used. It isn’t reasonable to mandate that a dataplane-to-forwarding plane 
> queue be dedicated to IS-IS. What other control plane entities are using the 
> queue and how they empty it will introduce new variables. And the 
> implementation cost comes in providing “real time updates” on the current 
> queue space to clients that need it.


Well, that’s up to your implementation.  Even if it is a shared queue, it has a 
finite depth and the transmitter should not overrun it. Scale your numbers 
accordingly.


> I really think monitoring the unacknowledged TX queue will give us what we 
> need and make the solution completely contained within the IS-IS 
> implementation.
> Guess I need to work on more details on that approach.


Please focus on how the transmitter distinguishes between packets lost and 
packets queued at the receiver.

Both imply that the TX rate was too high.  One suggests that retransmissions 
should be expedited.  The other suggests that they should not.

Tony


___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Les Ginsberg (ginsberg)]

Tony -

From: Tony Li  On Behalf Of tony...@tony.li
Sent: Wednesday, July 24, 2019 3:37 PM
To: Les Ginsberg (ginsberg) 
Cc: stephane.litkow...@orange.com; lsr@ietf.org
Subject: Re: [Lsr] Dynamic flow control for flooding


Les,


If you disagree please take things bullet-by-bullet:


  *   LSP input queue implementations are typically interface independent FIFOs


Very true.  It would not be unreasonable for an implementation to report free 
space in the FIFO (in number of PDUs) divided by the number of active 
adjacencies.  Everyone gets their fair share.

[If dynamic flooding is enabled, this could be based on the number of 
adjacencies that should be actively flooding. That should be a much smaller 
number.]
[Les:] So you are agreeing that when a receiver wants to “dial back” it will 
need to do so on all interfaces enabled for flooding?



  *   Overloaded Receiver does not know which senders are disproportionately 
causing the overflow


This doesn’t matter.  The receiver needs them all to slow down.



  *   LSPs may be dropped at lower layers – IS-IS receiver may be unaware that 
the overload condition exists


That’s an implementation problem. The implementation NEEDS to be able to see 
its input queue plus input drops.

[Les:] And you want to ship this feature when…? 😊
I think this is a difficult ask.
Before we decide this is what is required we should explore the path of 
monitoring the unacknowledged Tx queue.


  *   Updating hellos dynamically to alter flooding transmission rate is an OOB 
signaling mechanism consuming  resources at a time when routers are the most 
busy
  *   Consistent flooding rates will require updated hellos be sent to all 
neighbors – exacerbating the cost on both sender and receiver


This is why I suggest sending the feedback in PSNPs as well as in IIHs.  
Regardless of the details, we need to consider sending PSNPs back more 
frequently.  I concur that optimizing the rate and triggers for sending more 
PSNPs is an open issue.

Strictly speaking, sending a TLV inside of our protocol PDUs is an in-band 
signaling mechanism.
[Les:] I agree – PSNP would be better since we need to send it anyway in order 
to ACK. Still does not convince me this is the preferred approach – but I agree 
it is better than hellos.

The resources consumed by maintaining a running count of a queue in silicon or 
in process space is effectively zero.

[Les:] It is not about counting – it is about how a given queue might be used. 
It isn’t reasonable to mandate that a dataplane-to-forwarding plane queue be 
dedicated to IS-IS. What other control plane entities are using the queue and 
how they empty it will introduce new variables. And the implementation cost 
comes in providing “real time updates” on the current queue space to clients 
that need it.

I really think monitoring the unacknowledged TX queue will give us what we need 
and make the solution completely contained within the IS-IS implementation.
Guess I need to work on more details on that approach.

   Les


Tony


___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[tony . li]

Les,

> If you disagree please take things bullet-by-bullet:
>  
> LSP input queue implementations are typically interface independent FIFOs


Very true.  It would not be unreasonable for an implementation to report free 
space in the FIFO (in number of PDUs) divided by the number of active 
adjacencies.  Everyone gets their fair share.

[If dynamic flooding is enabled, this could be based on the number of 
adjacencies that should be actively flooding. That should be a much smaller 
number.]


> Overloaded Receiver does not know which senders are disproportionately 
> causing the overflow


This doesn’t matter.  The receiver needs them all to slow down.


> LSPs may be dropped at lower layers – IS-IS receiver may be unaware that the 
> overload condition exists


That’s an implementation problem. The implementation NEEDS to be able to see 
its input queue plus input drops.


> Updating hellos dynamically to alter flooding transmission rate is an OOB 
> signaling mechanism consuming  resources at a time when routers are the most 
> busy
> Consistent flooding rates will require updated hellos be sent to all 
> neighbors – exacerbating the cost on both sender and receiver


This is why I suggest sending the feedback in PSNPs as well as in IIHs.  
Regardless of the details, we need to consider sending PSNPs back more 
frequently.  I concur that optimizing the rate and triggers for sending more 
PSNPs is an open issue.

Strictly speaking, sending a TLV inside of our protocol PDUs is an in-band 
signaling mechanism.

The resources consumed by maintaining a running count of a queue in silicon or 
in process space is effectively zero.

Tony


___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[tony . li]

> Whether a BCP draft is desired is something I am open to considering.


Process nit: what we’re talking about doing, regardless of how we do it, is 
overriding 10589.  As that’s a normative reference, overriding that requires 
that we have a standards track document.

I don’t believe that even a BCP is sufficient. 

Well, that’s my understanding, at least.

Tony


___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[tony . li]

Les,


> This thread reminds me of how easy it is to miscommunicate – and I bear some 
> of the responsibility for that.


Communications takes two.  The receiver must also be focused on the input. My 
bad too.


> I don’t see anything in there about changing the PSNP signaling rate.  From 
> your comments to Henk, I infer that you’re open to changing that rate.
> [Les:] The proposal in the slides is simply an example/straw man. I did not 
> spend a lot of time on it – in fact in the first draft of the slides I did 
> not even provide a proposal. It certainly needs more refinement.
> It is meant only to illustrate how we can do things w/o requiring the receive 
> side to do calculations for which the raw data may be difficult and w/o 
> requiring new TLVs.


Understood.  The difficulty in implementation is not our primary concern and 
something of a chicken and egg situation: if we do not ask to get the data 
about the input critical path, we will not get it.

We’ve had a very similar situation with IP option handling: IP header options 
used to be very, very rare, so the first generation of forwarding ASICs passed 
off options for software-based forwarding. This made options slow.  IPv6 
designers then decided that options were slow, so they weren’t going to use 
options.  ;-)

If we need data from the platform silicon, then we should ask for it.  We’re 
not going to get it if we don’t ask.  And without specifics about what’s going 
on, we’re not going to make good approximations to the optimal rate.


> As soon as you do that, you’re now providing receiver based feedback and 
> creating flow control.  You’re accepting that rates will vary per interface.
>  
> [Les:] Yes – but only when we know that continuing to send at a high rate 
> isn’t useful. It isn’t meant to fix things (as I keep emphasizing) and in a 
> network that works as intended it should never be necessary.


Ok, below you say that you’re willing to be aggressive.  But being aggressive 
means that we WILL push things into flow control.


> What you’re NOT doing is providing information about the receiver’s input 
> queue and requested input rate.  With less information, the transmitter can 
> only approximate the optimal rate and your proposal seems like a Newton’s 
> method approach to determining that rate.  
>  
> [Les:] For all of the implementations I have worked on (5 now – across 3 
> different vendors – not all still available 😊 ) such information is not 
> easily determined. Buffer pools are shared among many components, input 
> queues may have multiple stages not all of which are visible to the routing 
> protocol. Plus, since once flow control is needed there is already a problem, 
> this isn’t fixing things – it is just trying to get by.
>  
> A solution which depends on current receiver state “all the time” is hard – 
> and hard to optimize. And I think we don’t need that degree of precision for 
> optimal operation.



No one is suggesting “all the time”.   The point is to provide more information 
than what’s currently in the PSNP.  The more information the transmitter has, 
the more accurately it can approximate the optimal transmit rate and maximize 
goodput.
 

> Your proposal depends on two constants: Usafe and Umax.  How do you know what 
> those are?
>  
> [Les:] Not yet.


That seems like a core problem.


> That’s information about the receiver. 
>  
> [Les:] Happy to agree to that.


Yay!  One more thing that we agree on.  :-)


> I infer that you propose to hard code some conservative values for these.  In 
> my mind, that implies that you will be going more slowly than you could if 
> you had more accurate data.  And pretty much what we’re proposing is that the 
> receiver advertise this type of information so that we don’t have to assume 
> the worst case.  This also is nice because an implementation only has to know 
> about it’s own capabilities.
>  
> [Les:] I expect the values to be aggressive – because the downside of 
> flooding LSPs too fast for (say) a few seconds is small.


Hmmm…. I’m not yet convinced.  Elsewhere on the thread, you said that a good 
implementation should prioritize receiving IIHs over LSPs and *SNPs.  I concur 
with that, but in my experience (5 vendors) I only know of one that implemented 
that in silicon, and that one’s not available (sob!).

Thus, my concern is that without a good approximation for the goodput rate, we 
will either chronically underestimate, penalizing convergence, or chronically 
overestimate, risking the stability of the network.

Tony

___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Les Ginsberg (ginsberg)]

Tony –

This thread reminds me of how easy it is to miscommunicate – and I bear some of 
the responsibility for that.
Inline.

From: Tony Li  On Behalf Of tony...@tony.li
Sent: Wednesday, July 24, 2019 1:07 PM
To: Les Ginsberg (ginsberg) 
Cc: lsr@ietf.org
Subject: Re: [Lsr] Dynamic flow control for flooding

Les,

Ok, let me reset.  I’ve re-read your slides.

I don’t see anything in there about changing the PSNP signaling rate.  From 
your comments to Henk, I infer that you’re open to changing that rate.
[Les:] The proposal in the slides is simply an example/straw man. I did not 
spend a lot of time on it – in fact in the first draft of the slides I did not 
even provide a proposal. It certainly needs more refinement.
It is meant only to illustrate how we can do things w/o requiring the receive 
side to do calculations for which the raw data may be difficult and w/o 
requiring new TLVs.


As soon as you do that, you’re now providing receiver based feedback and 
creating flow control.  You’re accepting that rates will vary per interface.

[Les:] Yes – but only when we know that continuing to send at a high rate isn’t 
useful. It isn’t meant to fix things (as I keep emphasizing) and in a network 
that works as intended it should never be necessary.

What you’re NOT doing is providing information about the receiver’s input queue 
and requested input rate.  With less information, the transmitter can only 
approximate the optimal rate and your proposal seems like a Newton’s method 
approach to determining that rate.

[Les:] For all of the implementations I have worked on (5 now – across 3 
different vendors – not all still available 😊 ) such information is not easily 
determined. Buffer pools are shared among many components, input queues may 
have multiple stages not all of which are visible to the routing protocol. 
Plus, since once flow control is needed there is already a problem, this isn’t 
fixing things – it is just trying to get by.

A solution which depends on current receiver state “all the time” is hard – and 
hard to optimize. And I think we don’t need that degree of precision for 
optimal operation.


Your proposal depends on two constants: Usafe and Umax.  How do you know what 
those are?

[Les:] Not yet.

That’s information about the receiver.

[Les:] Happy to agree to that.

I infer that you propose to hard code some conservative values for these.  In 
my mind, that implies that you will be going more slowly than you could if you 
had more accurate data.  And pretty much what we’re proposing is that the 
receiver advertise this type of information so that we don’t have to assume the 
worst case.  This also is nice because an implementation only has to know about 
it’s own capabilities.

[Les:] I expect the values to be aggressive – because the downside of flooding 
LSPs too fast for (say) a few seconds is small.


Tony



On Jul 24, 2019, at 12:31 PM, Les Ginsberg (ginsberg) 
mailto:ginsb...@cisco.com>> wrote:

Tony –

I have NEVER proposed that the flooding rate be determined by the slowest node.
Quite the opposite.

Flooding rate should be based on the target convergence time and should be 
aggressive because most topology changes involve much fewer than 1000 LSPs 
(arbitrary number). So even w a slow node fast flooding won’t be an issue for 
the vast majority of changes.

When we get a topology change with enough LSPs to expose the slowest node 
limitations we (in decreasing order of importance):

1)Continue to flood fast to those nodes/links which can handle it
2)Report the slow node to the operator (so they can address the limitation)
3)Do what we can to limit the overload on the slow node/link

Hope this helps.

   Les


From: Tony Li mailto:tony1ath...@gmail.com>> On Behalf 
Of tony...@tony.li
Sent: Wednesday, July 24, 2019 12:04 PM
To: Les Ginsberg (ginsberg) mailto:ginsb...@cisco.com>>
Cc: lsr@ietf.org
Subject: Re: [Lsr] Dynamic flow control for flooding


Les,


Optimizing the throughput through a slow receiver is pretty low on my list 
because the ROI is low.


Ok, I disagree. The slow receiver is the critical path to convergence.  Only 
when the slow receiver has absorbed all changes and SPFed do we have 
convergence.


First, the rate that you select might be too fast for one neighbor and not for 
the others.  Real flow control would help address this.

[Les:] At the cost of convergence. Not a good tradeoff.
I am arguing that we do want to flood at the same rate on all interfaces used 
for flooding. When we cannot, flow control does not help with convergence. It 
may decrease some wasted bandwidth – but as we all agree that bandwidth isn’t a 
significant limitation this isn’t a great concern.


Rate limiting flooding delays convergence.  Please consider the following 
topology:


1 —— 2 —— 3
|||
|||
4 —— 5 —— 6
|||
|||
7 —— 8 —— 9


Suppose that we hav

Re: [Lsr] Dynamic flow control for flooding

[tony . li]

Les,

Ok, let me reset.  I’ve re-read your slides.
 
I don’t see anything in there about changing the PSNP signaling rate.  From 
your comments to Henk, I infer that you’re open to changing that rate.

As soon as you do that, you’re now providing receiver based feedback and 
creating flow control.  You’re accepting that rates will vary per interface.

What you’re NOT doing is providing information about the receiver’s input queue 
and requested input rate.  With less information, the transmitter can only 
approximate the optimal rate and your proposal seems like a Newton’s method 
approach to determining that rate.  

Your proposal depends on two constants: Usafe and Umax.  How do you know what 
those are?

That’s information about the receiver. 

I infer that you propose to hard code some conservative values for these.  In 
my mind, that implies that you will be going more slowly than you could if you 
had more accurate data.  And pretty much what we’re proposing is that the 
receiver advertise this type of information so that we don’t have to assume the 
worst case.  This also is nice because an implementation only has to know about 
it’s own capabilities.

Tony


> On Jul 24, 2019, at 12:31 PM, Les Ginsberg (ginsberg)  
> wrote:
> 
> Tony –
>  
> I have NEVER proposed that the flooding rate be determined by the slowest 
> node.
> Quite the opposite.
>  
> Flooding rate should be based on the target convergence time and should be 
> aggressive because most topology changes involve much fewer than 1000 LSPs 
> (arbitrary number). So even w a slow node fast flooding won’t be an issue for 
> the vast majority of changes.
>  
> When we get a topology change with enough LSPs to expose the slowest node 
> limitations we (in decreasing order of importance):
>  
> 1)Continue to flood fast to those nodes/links which can handle it
> 2)Report the slow node to the operator (so they can address the limitation)
> 3)Do what we can to limit the overload on the slow node/link
>  
> Hope this helps.
>  
>Les
>  
>  
> From: Tony Li mailto:tony1ath...@gmail.com>> On 
> Behalf Of tony...@tony.li 
> Sent: Wednesday, July 24, 2019 12:04 PM
> To: Les Ginsberg (ginsberg) mailto:ginsb...@cisco.com>>
> Cc: lsr@ietf.org 
> Subject: Re: [Lsr] Dynamic flow control for flooding
>  
>  
> Les,
>  
>  
> Optimizing the throughput through a slow receiver is pretty low on my list 
> because the ROI is low.
>  
>  
> Ok, I disagree. The slow receiver is the critical path to convergence.  Only 
> when the slow receiver has absorbed all changes and SPFed do we have 
> convergence.
>  
>  
> First, the rate that you select might be too fast for one neighbor and not 
> for the others.  Real flow control would help address this.
>  
> [Les:] At the cost of convergence. Not a good tradeoff.
> I am arguing that we do want to flood at the same rate on all interfaces used 
> for flooding. When we cannot, flow control does not help with convergence. It 
> may decrease some wasted bandwidth – but as we all agree that bandwidth isn’t 
> a significant limitation this isn’t a great concern.
>  
>  
> Rate limiting flooding delays convergence.  Please consider the following 
> topology:
>  
>  
> 1 —— 2 —— 3
> |||
> |||
> 4 —— 5 —— 6
> |||
> |||
> 7 —— 8 —— 9
>  
>  
> Suppose that we have 1000 LSPs injected at router 1.  Suppose further that 
> router 2 runs at half the rate of router 4.  [How router 1 knows this 
> requires $DEITY and is out of scope for the moment.]
>  
> Router 1 now floods at the optimal rate for router 2.  Router 1 uses that 
> same rate to flood to router 4.  Suppose that it takes time T for this to 
> complete.
>  
> When does the network converge?
>  
> Option 1: All nodes use the same flooding rate.
>  
> Router 2 will flood to router 3 concurrent with receiving updates from router 
> 1. Thus, router 3 will receive all updates in time T + delta, where delta is 
> router 2’s processing time.  For now, let’s approximate delta as zero.
>  
> Similarly, all routers will use the same rate, so router 4 will flood to 7 in 
> time T + delta, and so on, with router 9 receiving everything in time T + 3 * 
> delta.
>  
> Assuming no nodes SPF during the process, the network converges nearly 
> simultaneously in about time T.
>  
> Option 2: We flood a bit faster where we can.
>  
> Suppose that router 1 now floods at the full rate to router 4.  The full 
> update now takes time T/2.  Because all of the other nodes in the network are 
> fast, router 4 floods in time T/2 + delta to nodes 5 and 7.  Carrying this 
> forward, router 9 gets a full update in time T/2 + 3 * delta.  Even router 3 
> has full updates in T/2 + 3 * delta.
>  
> With the exception of node 2, the network has converged in half the time.  
> Even node 2 converges in time T.
>  
> Key points: 
>  
> 1) Yes, the slow node delays converge

Re: [Lsr] Dynamic flow control for flooding

[Les Ginsberg (ginsberg)]

Tony –

I have NEVER proposed that the flooding rate be determined by the slowest node.
Quite the opposite.

Flooding rate should be based on the target convergence time and should be 
aggressive because most topology changes involve much fewer than 1000 LSPs 
(arbitrary number). So even w a slow node fast flooding won’t be an issue for 
the vast majority of changes.

When we get a topology change with enough LSPs to expose the slowest node 
limitations we (in decreasing order of importance):

1)Continue to flood fast to those nodes/links which can handle it
2)Report the slow node to the operator (so they can address the limitation)
3)Do what we can to limit the overload on the slow node/link

Hope this helps.

   Les


From: Tony Li  On Behalf Of tony...@tony.li
Sent: Wednesday, July 24, 2019 12:04 PM
To: Les Ginsberg (ginsberg) 
Cc: lsr@ietf.org
Subject: Re: [Lsr] Dynamic flow control for flooding


Les,


Optimizing the throughput through a slow receiver is pretty low on my list 
because the ROI is low.


Ok, I disagree. The slow receiver is the critical path to convergence.  Only 
when the slow receiver has absorbed all changes and SPFed do we have 
convergence.


First, the rate that you select might be too fast for one neighbor and not for 
the others.  Real flow control would help address this.

[Les:] At the cost of convergence. Not a good tradeoff.
I am arguing that we do want to flood at the same rate on all interfaces used 
for flooding. When we cannot, flow control does not help with convergence. It 
may decrease some wasted bandwidth – but as we all agree that bandwidth isn’t a 
significant limitation this isn’t a great concern.


Rate limiting flooding delays convergence.  Please consider the following 
topology:


1 —— 2 —— 3
|||
|||
4 —— 5 —— 6
|||
|||
7 —— 8 —— 9


Suppose that we have 1000 LSPs injected at router 1.  Suppose further that 
router 2 runs at half the rate of router 4.  [How router 1 knows this requires 
$DEITY and is out of scope for the moment.]

Router 1 now floods at the optimal rate for router 2.  Router 1 uses that same 
rate to flood to router 4.  Suppose that it takes time T for this to complete.

When does the network converge?

Option 1: All nodes use the same flooding rate.

Router 2 will flood to router 3 concurrent with receiving updates from router 
1. Thus, router 3 will receive all updates in time T + delta, where delta is 
router 2’s processing time.  For now, let’s approximate delta as zero.

Similarly, all routers will use the same rate, so router 4 will flood to 7 in 
time T + delta, and so on, with router 9 receiving everything in time T + 3 * 
delta.

Assuming no nodes SPF during the process, the network converges nearly 
simultaneously in about time T.

Option 2: We flood a bit faster where we can.

Suppose that router 1 now floods at the full rate to router 4.  The full update 
now takes time T/2.  Because all of the other nodes in the network are fast, 
router 4 floods in time T/2 + delta to nodes 5 and 7.  Carrying this forward, 
router 9 gets a full update in time T/2 + 3 * delta.  Even router 3 has full 
updates in T/2 + 3 * delta.

With the exception of node 2, the network has converged in half the time.  Even 
node 2 converges in time T.

Key points:

1) Yes, the slow node delays convergence and causes micro-loops as everyone 
around it SPFs.  The point here (and I think you agree) is that slow nodes need 
to be upgraded.

2) There is no way for us to know how fast a node can go without some form of 
flow control, other than to go absurdly slowly.

3) There are many folks who want to converge quickly.  It is mission critical 
for them.  They will address slow nodes. They will not accept pessimal timing 
to avoid micro-loops.



[Les:] I do not see how flow control improves things.


Flow control allows the transmitter to transmit at the optimal rate for the 
receiver.



Dropping down to the least common denominator CPU speed in the entire network 
is going to be undoable without an oracle, and absurdly slow even with that.

[Les:] Never advocated that – please do not put those words in my mouth.


How is that different than what you’ve proposed?  Router 1 can only flood at 
the rate that it gets PSNPs from router 2.  That paces its flooding to router 
4.  Following that logic, you somehow want router 4 to run at the same rate, 
forcing a uniformly slow rate.

Tony



___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[stephane.litkowski]

Les,

Can’t we use from a transmitter point of view, the lack of ACKs from the 
receiver as a signal that the transmitter should slow down ?
I agree that depending on the exact bottleneck/issue, the IS-IS stack of the 
receiver may not be aware that something bad is happening and can’t provide 
feedback to the transmitter. However if the transmitter sees that the receiver 
is acking slowly or quickly, wouldn’t it be able to adjust its flooding speed. 
Can we have a receiver intentionally postponing a PSNP to aggregate multiple 
ACKs in a single message ?

From: Les Ginsberg (ginsberg) [mailto:ginsb...@cisco.com]
Sent: Wednesday, July 24, 2019 14:48
To: tony...@tony.li
Cc: LITKOWSKI Stephane OBS/OINIS; lsr@ietf.org
Subject: RE: [Lsr] Dynamic flow control for flooding

More inline…

From: Tony Li  On Behalf Of tony...@tony.li
Sent: Tuesday, July 23, 2019 10:56 PM
To: Les Ginsberg (ginsberg) 
Cc: stephane.litkow...@orange.com; lsr@ietf.org
Subject: Re: [Lsr] Dynamic flow control for flooding


Les,

There is something to be said for simply “flooding fast” and not worrying about 
flow control at all (regardless of whether TX or RX mechanisms would be used).


The only thing I would say to that is: really bad idea.

[Les:] I have to watch my words more closely. 😊
I am not arguing for this – but I do think that “most of the time” this 
strategy would actually be optimal.
We are discussing the extreme cases – as we should – where we have a large # of 
LSPs to flood. But let’s not lose sight of the fact that the simple approach 
works most of the time. For the times when the simple approach doesn’t work 
well, I am then arguing we should not overcomplicate the solution – 
particularly because the strategies we might use don’t help convergence.

If you supersaturate the receiver, you waste transmitter in the transmission, 
you swamp the receiver causing packet loss, you potentially trigger the loss of 
IIH’s, you risk causing a cascade failure, and until we come up with a better 
retransmission mechanism, you probably actually delay network convergence, as 
nothing is going to happen until you’ve completed retransmissions.
[Les:] Prioritization of hellos over LSPs/SNPs is a longstanding best practice 
(both on Tx and Rx) and this must not change. No one is advocating that – 
certainly not me.

The way to maximize goodput is NOT to transmit blindly.


[Les:] Not arguing for blindness, but I am arguing for simplicity.

But most important to me is to recognize that flow control (however done) is 
not fixing anything – nor making the flooding work better. The network is 
compromised and flow control won’t fix it.


 The network is not compromised.

[Les:] If the SLA the customer expects is convergence in less than N, then a 
slow link jeopardizes our ability to achieve that.

If you accept that, then it makes sense to look for the simplest way to do flow 
control and that is decidedly not from the RX side. (I expect Tony Li to 
disagree with that 😊– but I have already outlined why it is more complex to do 
it from the Rx side.)



Flow control cannot be done without involvement of the RX side.  That’s why 
it’s called flow _control_.  The only thing that can be done purely from the TX 
side is a unilateral (and pessimal) transmit rate cap that will have to allow 
for the worst case CPU in the worst case situation (e.g., BGP impacting the 
CPU).

Flow control is the creation of a control loop and that requires feedback from 
the receiver.  This is true in every form of true flow control: XON/XOFF, 
RTS/CTS, sliding window protocols, credit based fabric mechanisms, etc.

I’ll go so far as to quote Wikipedia:

"In data communications, 
flow control is the process of managing the rate of data transmission between 
two nodes to prevent a fast sender from overwhelming a slow receiver. It 
provides a mechanism for the receiver to control the transmission speed, so 
that the receiving node is not overwhelmed with data from transmitting node.”

[Les:] I will not argue about the definition.
In this specific case, there are difficulties in controlling the flooding rate 
based on advertisements from the RX side. The difficulties are outlined in my 
slides and largely have to do with the difficulties/costs of dynamically 
calculating what number to advertise. (A static advertisement is also difficult 
to calculate w/o being overly conservative.)

If you disagree please take things bullet-by-bullet:


  *   LSP input queue implementations are typically interface independent FIFOs
  *   Overloaded Receiver does not know which senders are disproportionately 
causing the overflow
  *   LSPs may be dropped at lower layers – IS-IS receiver may be unaware that 
the overload condition exists
  *   Updating hellos dynamically to alter flooding transmission rate is an OOB 
signaling mechanism consuming  resources at a time when routers are the most 
busy
  *   Consistent flooding rates w

Re: [Lsr] Dynamic flow control for flooding

[Les Ginsberg (ginsberg)]

Henk -

Inline.

> -Original Message-
> From: Henk Smit 
> Sent: Wednesday, July 24, 2019 6:18 AM
> To: Les Ginsberg (ginsberg) 
> Cc: stephane.litkow...@orange.com; Tony Li ; lsr@ietf.org
> Subject: Re: [Lsr] Dynamic flow control for flooding
> 
> 
> Hello Les,
> 
> Les Ginsberg (ginsberg) wrote on 2019-07-24 07:17:
> 
> > If you accept that, then it makes sense to look for the simplest way
> > to do flow control and that is decidedly not from the RX side. (I
> > expect Tony Li to disagree with that 😊 – but I have already
> > outlined why it is more complex to do it from the Rx side.)
> 
> In your talk on Monday you called the idea in
> draft-decraene-lsr-isis-flooding-speed-01 "receiver driven flow
> control".
> You don't like that. You want "transmit based flow control".
> You argued that you can do "transmit based flow control" on the sender
> only.
> Therefor your algorithm is merely a "local trick".
> And "local tricks" don't need RFCs. I agree with that.
> But I don't agree that your algorithm is just a "local trick".
> 
> 
> In your algorithm, a "sender" sends a number of LSPs to a receiver.
> Without waiting for acks (PNSPs). Like in any sliding window protocol.
> The sending router keeps an eye on the number of unacked LSPs.
> And it determines how fast it can send more LSPs based on the current
> number of unacked LSPs. Every time the sender receives a PSNP, it
> knows the receiver got a number of LSPs, so it can increase its
> send-window again, and then send more LSPs.
> Correct ?
> 
> I agree that the core idea of this algorithm makes sense.
> After all, it looks a lot like TCP.
> I believe the authors of draft-decraene-lsr-isis-flooding-speed were
> planning something like that for the next version of their draft.
> 
> 
> However, I do not agree with the name "tx driven flow control".
> I also do not agree that this algorithm is "a local trick".
> Therefor I also do not think this algorithm doesn't need to be
> documented (in an RFC).
> 
> In your "tx based flow control", the sender (tx) sends LSPs at a rate
> that is derived from the rate at which it receives PSNPs. Therefor
> it is the sender of the PSNPs that sets the speed of transmission !
> So it is still the receiver (of LSPs) that controls the flow control.
> The name "tx based flow control" is a little misleading, imho.
> 
> 
> It is important to realize that the success of your algorithm actually
> depends on the behaviour of the receiver. How does it send PSNPs ?
> Does it send one PSNP per received LSP ? Or does it pack multiple acks
> in one PSNP ? Does it send a PSNP immediatly, or does it wait a short
> time ? Does it try to fill a PSNP to the max (putting ~90 acks in one
> PSNP) ? Does the receiver does something in between ? I don't think
> the behaviour is specified exactly anywhere.
> 
> I know about an IS-IS implementation from the nineties. When a router
> would receive an LSP, it would a) set the SSN bit (for that
> LSP/interface),
> and b) start the psnp-timer for that interface (if not already running).
> The psnp-timer would expire 2 seconds later. The router would then walk
> the LSPDB, find all LSPs with the SSN-bit set for that interface. And
> then build a PSNP with acks for all those LSPs. The result would be
> that: a) the first PSNP would be send 2 seconds (+/- jitter) after
> receiving the first LSP, and b) the PSNP would include ~66 acks. (As
> a router receiving at full speed would have received 66 LSPs in 2
> seconds).
> 
> For your "tx based flow control" algorithm to work properly, this has
> to change. The receiving router must send PSNPs more quickly and more
> aggressively. The result would be that there will be less acks in each
> PSNP. And thus more PSNPs will be sent.
> 
> This makes us realize: in the current situation, if a router receives
> a 1000 LSPs, and sends those LSPs to 64 neighbors, it would receive:
> - the 1000 LSPs from an upstream neighbor, plus
> - 1000/66 = 16 PSNPs from each downstream neighbor = 64 * 16 = 1024
> PSNPs.
> This makes a total of ~2000K PDUs received.
> 
> If routers would send one PSNP per LSP (to have faster flow control),
> then the router in this example would receive:
> - the 1000 LSPs from an upstream neighbor, plus
> - 1000 PSNPs from each downstream neighbor * 16 = 1600 PSNPs.
> This makes a total of ~17000 PDUs received.
> 
> The total number of PDUs received on this router would go from 2K PDUs
> to 17K PDUs.
> 
> Remember that the problem we're trying to solve here is to make sure
> that routers do not get overrun on the receipt side with too many
> packets too quickly. It seems an aggressive PSNP-scheme, to achieve
> faster flow-control, is actually very counter-productive.
> 
> Of course the algorithm can be tweaked. E.g. TCP sends one ack per
> every 2 received segments (if I'm not mistaken). If we do that here,
> the number of PDUs would go down from 17K to 9K PDUs. What do you
> propose ? How do you want the feedback of PSNPs to be quick, whil

Re: [Lsr] Dynamic flow control for flooding

[tony . li]

Les,


> Optimizing the throughput through a slow receiver is pretty low on my list 
> because the ROI is low.


Ok, I disagree. The slow receiver is the critical path to convergence.  Only 
when the slow receiver has absorbed all changes and SPFed do we have 
convergence.


> First, the rate that you select might be too fast for one neighbor and not 
> for the others.  Real flow control would help address this.
>  
> [Les:] At the cost of convergence. Not a good tradeoff.
> I am arguing that we do want to flood at the same rate on all interfaces used 
> for flooding. When we cannot, flow control does not help with convergence. It 
> may decrease some wasted bandwidth – but as we all agree that bandwidth isn’t 
> a significant limitation this isn’t a great concern.


Rate limiting flooding delays convergence.  Please consider the following 
topology:


1 —— 2 —— 3
|||
|||
4 —— 5 —— 6
|||
|||
7 —— 8 —— 9


Suppose that we have 1000 LSPs injected at router 1.  Suppose further that 
router 2 runs at half the rate of router 4.  [How router 1 knows this requires 
$DEITY and is out of scope for the moment.]

Router 1 now floods at the optimal rate for router 2.  Router 1 uses that same 
rate to flood to router 4.  Suppose that it takes time T for this to complete.

When does the network converge?

Option 1: All nodes use the same flooding rate.

Router 2 will flood to router 3 concurrent with receiving updates from router 
1. Thus, router 3 will receive all updates in time T + delta, where delta is 
router 2’s processing time.  For now, let’s approximate delta as zero.

Similarly, all routers will use the same rate, so router 4 will flood to 7 in 
time T + delta, and so on, with router 9 receiving everything in time T + 3 * 
delta.

Assuming no nodes SPF during the process, the network converges nearly 
simultaneously in about time T.

Option 2: We flood a bit faster where we can.

Suppose that router 1 now floods at the full rate to router 4.  The full update 
now takes time T/2.  Because all of the other nodes in the network are fast, 
router 4 floods in time T/2 + delta to nodes 5 and 7.  Carrying this forward, 
router 9 gets a full update in time T/2 + 3 * delta.  Even router 3 has full 
updates in T/2 + 3 * delta.

With the exception of node 2, the network has converged in half the time.  Even 
node 2 converges in time T.

Key points: 

1) Yes, the slow node delays convergence and causes micro-loops as everyone 
around it SPFs.  The point here (and I think you agree) is that slow nodes need 
to be upgraded.

2) There is no way for us to know how fast a node can go without some form of 
flow control, other than to go absurdly slowly.

3) There are many folks who want to converge quickly.  It is mission critical 
for them.  They will address slow nodes. They will not accept pessimal timing 
to avoid micro-loops.


> [Les:] I do not see how flow control improves things.


Flow control allows the transmitter to transmit at the optimal rate for the 
receiver.


> Dropping down to the least common denominator CPU speed in the entire network 
> is going to be undoable without an oracle, and absurdly slow even with that.
>  
> [Les:] Never advocated that – please do not put those words in my mouth.


How is that different than what you’ve proposed?  Router 1 can only flood at 
the rate that it gets PSNPs from router 2.  That paces its flooding to router 
4.  Following that logic, you somehow want router 4 to run at the same rate, 
forcing a uniformly slow rate.

Tony



___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Les Ginsberg (ginsberg)]

Henk -

Welcome to the discussion.

Inline.

> -Original Message-
> From: henk.i...@xs4all.nl 
> Sent: Wednesday, July 24, 2019 5:34 AM
> To: Les Ginsberg (ginsberg) 
> Cc: stephane.litkow...@orange.com; Tony Li ; lsr@ietf.org
> Subject: Re: [Lsr] Dynamic flow control for flooding
> 
> 
> Hello Les,
> 
> Les Ginsberg (ginsberg) schreef op 2019-07-24 07:17:
> >
> > There is something to be said for simply “flooding fast” and not
> > worrying about flow control at all (regardless of whether TX or RX
> > mechanisms would be used). Some packets would be dropped, but
> > retransmission timers will insure that the flooding eventually
> > succeeds and retransmit timers are long (5 seconds by default). (I am
> > not the only one mentioning this BTW…)
> 
> Why do we have initial waits and exponential backoffs for LSP-generation
> and SPF-computations ? Why not react immediately ? Why not react
> constantly ?
> 
[Les:] I think you know what I am about to say.. 😊

Initial wait exists in order to minimize premature SPF calculation. Topology 
changes always entail multiple LSP updates (often 2 for a single link failure - 
more for a node failure or line card failure).
We wait a short while to increase the likelihood that we have received all of 
the LSP updates associated w the topology event.

Backoff is used so that if the network has an extended period of instability we 
do not invest cycles w diminishing returns i.e., the updates we are making to 
forwarding are likely stale before we even complete the update.

> We have a lot of bandwidth and cpu-power now. Isn't simple always better
> than "overly complex stuff" like exponential backoffs ? If you have more
> cpu-power, more memory and more bandwidth, why invent new algorithms ?

[Les:] Base spec never defined an algorithm for dynamically changing flooding 
speed (unless you consider retransmit timers). So this is "inventing" - not 
"reinventing".

   Les
> 
> henk.
> 
> 
> I hope it is clear to everyone that these are not serious questions. I'm
> just
> saying: "sometimes fast is slow". I am sure that if we ask the "old
> guys", they
> can come up with many stories how these problems are sometimes
> counter-intuitive.
> And how networks have melted because "fast is slow". I could tell at
> least 2
> of those stories myself.
___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Les Ginsberg (ginsberg)]

More inline…

From: Tony Li  On Behalf Of tony...@tony.li
Sent: Tuesday, July 23, 2019 10:56 PM
To: Les Ginsberg (ginsberg) 
Cc: stephane.litkow...@orange.com; lsr@ietf.org
Subject: Re: [Lsr] Dynamic flow control for flooding


Les,

There is something to be said for simply “flooding fast” and not worrying about 
flow control at all (regardless of whether TX or RX mechanisms would be used).


The only thing I would say to that is: really bad idea.

[Les:] I have to watch my words more closely. 😊
I am not arguing for this – but I do think that “most of the time” this 
strategy would actually be optimal.
We are discussing the extreme cases – as we should – where we have a large # of 
LSPs to flood. But let’s not lose sight of the fact that the simple approach 
works most of the time. For the times when the simple approach doesn’t work 
well, I am then arguing we should not overcomplicate the solution – 
particularly because the strategies we might use don’t help convergence.

If you supersaturate the receiver, you waste transmitter in the transmission, 
you swamp the receiver causing packet loss, you potentially trigger the loss of 
IIH’s, you risk causing a cascade failure, and until we come up with a better 
retransmission mechanism, you probably actually delay network convergence, as 
nothing is going to happen until you’ve completed retransmissions.
[Les:] Prioritization of hellos over LSPs/SNPs is a longstanding best practice 
(both on Tx and Rx) and this must not change. No one is advocating that – 
certainly not me.

The way to maximize goodput is NOT to transmit blindly.


[Les:] Not arguing for blindness, but I am arguing for simplicity.


But most important to me is to recognize that flow control (however done) is 
not fixing anything – nor making the flooding work better. The network is 
compromised and flow control won’t fix it.


 The network is not compromised.

[Les:] If the SLA the customer expects is convergence in less than N, then a 
slow link jeopardizes our ability to achieve that.


If you accept that, then it makes sense to look for the simplest way to do flow 
control and that is decidedly not from the RX side. (I expect Tony Li to 
disagree with that 😊– but I have already outlined why it is more complex to do 
it from the Rx side.)



Flow control cannot be done without involvement of the RX side.  That’s why 
it’s called flow _control_.  The only thing that can be done purely from the TX 
side is a unilateral (and pessimal) transmit rate cap that will have to allow 
for the worst case CPU in the worst case situation (e.g., BGP impacting the 
CPU).

Flow control is the creation of a control loop and that requires feedback from 
the receiver.  This is true in every form of true flow control: XON/XOFF, 
RTS/CTS, sliding window protocols, credit based fabric mechanisms, etc.

I’ll go so far as to quote Wikipedia:

"In data communications, 
flow control is the process of managing the rate of data transmission between 
two nodes to prevent a fast sender from overwhelming a slow receiver. It 
provides a mechanism for the receiver to control the transmission speed, so 
that the receiving node is not overwhelmed with data from transmitting node.”

[Les:] I will not argue about the definition.
In this specific case, there are difficulties in controlling the flooding rate 
based on advertisements from the RX side. The difficulties are outlined in my 
slides and largely have to do with the difficulties/costs of dynamically 
calculating what number to advertise. (A static advertisement is also difficult 
to calculate w/o being overly conservative.)

If you disagree please take things bullet-by-bullet:


  *   LSP input queue implementations are typically interface independent FIFOs
  *   Overloaded Receiver does not know which senders are disproportionately 
causing the overflow
  *   LSPs may be dropped at lower layers – IS-IS receiver may be unaware that 
the overload condition exists
  *   Updating hellos dynamically to alter flooding transmission rate is an OOB 
signaling mechanism consuming  resources at a time when routers are the most 
busy
  *   Consistent flooding rates will require updated hellos be sent to all 
neighbors – exacerbating the cost on both sender and receiver

   Les


Tony


___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Robert Raszuk]

*[Les:] At the cost of convergence. Not a good tradeoff.*

Hi Les - why at the cost of convergence ? No one as I see it is proposing
the same rate to every peer. Quite contrary -- per peer rate of flooding.
So if you keep flooding high speed on fast links the convergence will not
be any slower with flow control.

Thx,
R.

On Wed, Jul 24, 2019 at 8:28 PM Les Ginsberg (ginsberg) 
wrote:

> Tony –
>
>
>
> Inline.
>
>
>
> *From:* Tony Li  *On Behalf Of *tony...@tony.li
> *Sent:* Tuesday, July 23, 2019 10:39 PM
> *To:* Les Ginsberg (ginsberg) 
> *Cc:* lsr@ietf.org
> *Subject:* Re: [Lsr] Dynamic flow control for flooding
>
>
>
>
>
>
>
> Les,
>
>
>
> I also think we all agree on the goal - which is to flood significantly
> faster than many implementations do today to handle deployments like the
> case you mention below.
>
>
>
>
>
> I agree with that, but you haven’t responded to the goal that I proposed:
> keep the receiver processing PDUs.
>
>
>
> *[Les:] My goals are:*
>
>
>
> *Optimize convergence.*
>
> *Minimize complexity.*
>
>
>
> *Optimizing the throughput through a slow receiver is pretty low on my
> list because the ROI is low.*
>
>
>
> Beyond this point, I have a different perspective.
>
>
>
> As network-wide convergence depends upon fast propagation of LSP changes –
> which in turn requires consistent flooding rates on all interfaces enabled
> for flooding – a properly provisioned network MUST be able to sustain a
> consistent flooding rate or the operation of the network will suffer. We
> therefore need to view flow control issues as indicative of a problem.
>
>
>
> It is a mistake to equate LSP flooding with a set of independent P2P
> “connections” – each of which can operate at a rate independent of the
> other.
>
>
>
> If we can agree on this, then I believe we will have placed the flow
> control problem in its proper perspective – in which case it will become
> easier to agree on the best way to implement flow control.
>
>
>
>
>
> I agree that we want network wide convergence.  However, I disagree that
> the right thing to do is to uniformly flood at the same rate on all
> interfaces.
>
>
>
> First, the rate that you select might be too fast for one neighbor and not
> for the others.  Real flow control would help address this.
>
>
>
> *[Les:] At the cost of convergence. Not a good tradeoff.*
>
> *I am arguing that we do want to flood at the same rate on all interfaces
> used for flooding. When we cannot, flow control does not help with
> convergence. It may decrease some wasted bandwidth – but as we all agree
> that bandwidth isn’t a significant limitation this isn’t a great concern.*
>
>
>
> *The same reasons that drive us to use same SPF delay and LSP Generation
> timers on all routers also apply to flooding rate.*
>
>
>
> Second, all flooding paths are not created equal.  You do not know, a
> priori, how to flood to get uniform network wide propagation.  The variance
> in CPU loading alone is going to cause different routers to flood at
> different rates, and so it may actaully be optimal to flood quickly down
> one path, knowing that the data will reach the other end of the network and
> flood back before the slow CPU can absorb and flood.
>
>
>
> *[Les:] I do not see how flow control improves things.*
>
> *If you have alternate flooding paths around the slow link(s) this argues
> more that you should not have the slow link in the flooding topology in the
> first place. But this heads off topic – I think we agree that dynamic
> flooding is a separate discussion which we don’t want to bring into this
> thread.*
>
>
>
> Dropping down to the least common denominator CPU speed in the entire
> network is going to be undoable without an oracle, and absurdly slow even
> with that.
>
>
>
> *[Les:] Never advocated that – please do not put those words in my mouth.*
>
>
>
> *   Les*
>
>
>
>
>
> Tony
>
>
> ___
> Lsr mailing list
> Lsr@ietf.org
> https://www.ietf.org/mailman/listinfo/lsr
>
___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Les Ginsberg (ginsberg)]

Tony –

Inline.

From: Tony Li  On Behalf Of tony...@tony.li
Sent: Tuesday, July 23, 2019 10:39 PM
To: Les Ginsberg (ginsberg) 
Cc: lsr@ietf.org
Subject: Re: [Lsr] Dynamic flow control for flooding



Les,

I also think we all agree on the goal - which is to flood significantly faster 
than many implementations do today to handle deployments like the case you 
mention below.


I agree with that, but you haven’t responded to the goal that I proposed: keep 
the receiver processing PDUs.

[Les:] My goals are:

Optimize convergence.
Minimize complexity.

Optimizing the throughput through a slow receiver is pretty low on my list 
because the ROI is low.


Beyond this point, I have a different perspective.

As network-wide convergence depends upon fast propagation of LSP changes – 
which in turn requires consistent flooding rates on all interfaces enabled for 
flooding – a properly provisioned network MUST be able to sustain a consistent 
flooding rate or the operation of the network will suffer. We therefore need to 
view flow control issues as indicative of a problem.

It is a mistake to equate LSP flooding with a set of independent P2P 
“connections” – each of which can operate at a rate independent of the other.

If we can agree on this, then I believe we will have placed the flow control 
problem in its proper perspective – in which case it will become easier to 
agree on the best way to implement flow control.


I agree that we want network wide convergence.  However, I disagree that the 
right thing to do is to uniformly flood at the same rate on all interfaces.

First, the rate that you select might be too fast for one neighbor and not for 
the others.  Real flow control would help address this.

[Les:] At the cost of convergence. Not a good tradeoff.
I am arguing that we do want to flood at the same rate on all interfaces used 
for flooding. When we cannot, flow control does not help with convergence. It 
may decrease some wasted bandwidth – but as we all agree that bandwidth isn’t a 
significant limitation this isn’t a great concern.

The same reasons that drive us to use same SPF delay and LSP Generation timers 
on all routers also apply to flooding rate.

Second, all flooding paths are not created equal.  You do not know, a priori, 
how to flood to get uniform network wide propagation.  The variance in CPU 
loading alone is going to cause different routers to flood at different rates, 
and so it may actaully be optimal to flood quickly down one path, knowing that 
the data will reach the other end of the network and flood back before the slow 
CPU can absorb and flood.

[Les:] I do not see how flow control improves things.
If you have alternate flooding paths around the slow link(s) this argues more 
that you should not have the slow link in the flooding topology in the first 
place. But this heads off topic – I think we agree that dynamic flooding is a 
separate discussion which we don’t want to bring into this thread.

Dropping down to the least common denominator CPU speed in the entire network 
is going to be undoable without an oracle, and absurdly slow even with that.

[Les:] Never advocated that – please do not put those words in my mouth.

   Les


Tony

___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[tony . li]

Robert,

> The second part of the question was really about at what layer it makes most 
> sense to provide this control loop. 


To me, the obvious thing to do is to make minor revisions to the protocol. We 
need to:

- Add a TLV so that the receiver can provide feedback. IMHO, this should be in 
IIH’s and PSNPs.

- Add text to modify the transmitters behavior.  In the presence of this TLV, 
the transmitter is released from 10589 compliance and may transmit (details 
TBD).

- Add text to modify the receivers behavior.  If you support this feature, then 
add the TLV, send PSNPs more frequently (rate & trigger TBD).


> Options seems to be: 
> 
> * Invent new or use existing link layer flow control (IEEE)
> * Reuse existing transport layer (TCP) 
> * App layer (QUIC or QUIC like)


All of these seem like massive overkill.


> I guess it would be useful to up front list on what type of media this must 
> be supported as it may change the game drastically: 
> 
> * directly connected fiber p2p 
> * p2mp (via switch)
> * p2p over encapsulation 
> etc…


All of the above, plus legacy media too.  No reason why this doesn’t apply to 
100BaseT.  Bandwidth is not the constraint.

Tony


___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[tony . li]

Robert,

Nothing has changed about the probability of network partitioning. That was 
simply a use case selected to motivate the discussion about flooding speed.

The entire discussion is almost orthogonal to dynamic flooding.  Let’s please 
take that out of the discussion.

Tony


> On Jul 24, 2019, at 7:38 AM, Robert Raszuk  wrote:
> 
> Hi,
> 
> Yes indeed while I was reading your richly connected node restart problem use 
> of overload-bit should be explored, proposed, implemented. 
> 
> For the partition problem I have two general comments: 
> 
> a) If network partitions is likely to happen more often in the case of 
> dynamic flooding perhaps as already said before we should increase the max 
> number of occurrences given LSP is to arrive at flooding optimized node. Two 
> may not be enough.
> 
> b)  If protocol extensions will help to mitigate effects of network partition 
> via much faster repair some folks may treat network partitions as normal 
> operational model and instead of re-architecting the network to make sure 
> network partition events are as rear as possible. 
> 
> Thx,
> R.
> 
> On Wed, Jul 24, 2019 at 4:12 PM Henk Smit  > wrote:
> 
> Hello Robert,
> 
> Tony brought up the example of a partioned network.
> But there are more examples.
> 
> E.g. in a network there is a router with a 1000 neighbors.
> (When discussing distributed vs centralized flooding-topology
>   reduction algorithms, I've been told these network designs exist).
> When such a router reboots/crashes/comes back up, all 1000 neighbors
> will create a new version of their own LSP. This causes a 1000 different
> LSPs to be flooded through the network at the same time. Impacting every
> router in the network.
> 
> The case I was thinking of myself, was when a router in a large network
> boots. When it brings up a number of adjacencies, each neighbor will
> try to synchronize its LSPDB with the newly booted router. As the newly
> booted router will send emtpy CSNPs to each of its neighbors, each
> neighbor will start sending the full LSPDB. If such a network has 10k
> LSPs, and such a router has 100 neighbors, that router will receive 100 
> * 10k
> is 1 million LSPs. Having a faster and more efficient flooding 
> transport,
> with flow-control, will make a reboot in such a topology less painful.
> 
> (In that last case, creative use of the overload-bit could prevent 
> black-holing
> or microloops while ISIS synchronizes its LSPDB after a reboot. Just 
> like we
> used the overload-bit to solve the problem of slow convergence of BGP 
> after
> a reboot, 22 years ago. I have no idea if there are any implementations 
> that
> use the overload-bit to alleviate slow convergence of IS-IS after a 
> reboot).
> 
> henk.
> 
> 
> Robert Raszuk schreef op 2019-07-24 15:33:
> > Hey Henk & all,
> > 
> > If acks for 1000 LSPs take 16 PSNPs (max 66 per PSNP) or even as long
> > as Tony mentioned the full flooding as Tony said may take 33 sec - is
> > this really a problem ?
> > 
> > Remember we are not talking about protocol convergence after link flap
> > or node going down. We are talking about serious network partitioning
> > which itself may have lasted for minutes, hours or days. While just
> > considering absolute numbers yelds desire to go faster and faster, if
> > we put things in the overall perspective is there really a problem to
> > be solved in the first place ?
> > 
> > Would there still be a problem if LSR WG recommends faster acking
> > maybe not for each LSP but for say 20 or 30 max ?
> > 
> > Thx,
> > R.
> 
> ___
> Lsr mailing list
> Lsr@ietf.org 
> https://www.ietf.org/mailman/listinfo/lsr 
> 

___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Henk Smit]

Les Ginsberg (ginsberg) schreef op 2019-07-23 22:29:


It is a mistake to equate LSP flooding with a set of independent P2P
“connections” – each of which can operate at a rate independent
of the other.


Of course, if some routers are slow, convergence in parts of the network
might be slow. But as Stephane has already suggested, it is up to the
operators to determine if slower convergence in parts of their network
is acceptable. E.g. they can chose to put fast/expensive/new routers
in the center of their network, and move older routers to, or buy 
cheaper

routers for, the edges of their network.


But I have a question for you, Les:

During your talk, or maybe in older emails, I was under the impression
that you wanted to warn for another problem. Namely microloops.
I am not sure I understand correctly. So let me try to explain what
I understood. And please correct me if I am wrong.


Between the time a link breaks, and the first router(s) start to repair
the network, some traffic is dropped. Bad for that traffic of course. 
But

the network keeps functioning. Once all routers have re-converged and
adjusted their FIBs, everything is fine again.

In the time in between, between the first router adjusting its FIB and
the last router adjusting its FIB, you can have microloops. Microloops
multiply traffic. Which can cause the whole network to suffer of 
congestion.

Impacting traffic that did not (originally) go over the broken link.

So you want the transition from "wrong FIBs", that point over the broken
path, to "the final FIBs", where all traffic flows correctly, to have
that transition happen on all routers at once. That would make the 
network
go from "drop some traffic" to "forward over the new path" without a 
stage

of "some microloops" in between.

Am I correct ? Is this what you try to prevent ?
Is this why you want all flooding between routers go at the same speed ?

Thanks in advance,

henk.

___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Tony Przygienda]

I'm under the impression the whole discussion got triggered by my rather
loose remark during the dinner based on, admittedly, my quite dated
implementation experience (with 2 disjoint distributed SPTs to reduce
flooding). I realized it seems not clearly spelled out on the thread. So to
hopefully clarify a bit: the scenario was not partition really, the
scenario was where I ended up in reduced graphs where rather large
"cliques" ended up being connected by either long'ish think chains (2 links
minimal cuts) which ended up "bottle-necking" the flooding on large changes
like redistribution/restart scenarios causing significant amount of LSPs
needed refresh from "left" to "right" through the bottlenecks. Transients
on the network became longer to the point we ended up (then) scrapping the
attempt after considering how we would bulild two trees _per flooding
source_ to really deal with it which seemed like overkill (and RIFT does
BTW but only because of known topology properties) and then setting
overload for which we couldn't figure out a good dynamic way (too short
dangerous, too long as bad). All that with standard flooding rates (beauty
of having mix-of-implementaitons requirements posed). Controlling the span
in distributed fashion is of course much harder than centralized fashion so
one has to be careful to compare apples-with-oranges here ...

that said pretty much out this thread

--- tony



On Wed, Jul 24, 2019 at 9:34 AM Robert Raszuk  wrote:

> Hey Henk & all,
>
> If acks for 1000 LSPs take 16 PSNPs (max 66 per PSNP) or even as long as
> Tony mentioned the full flooding as Tony said may take 33 sec - is this
> really a problem ?
>
> Remember we are not talking about protocol convergence after link flap or
> node going down. We are talking about serious network partitioning which
> itself may have lasted for minutes, hours or days. While just considering
> absolute numbers yelds desire to go faster and faster, if we put things in
> the overall perspective is there really a problem to be solved in the first
> place ?
>
> Would there still be a problem if LSR WG recommends faster acking maybe
> not for each LSP but for say 20 or 30 max ?
>
> Thx,
> R.
>
>
>
>
>
>
>
>
> On Wed, Jul 24, 2019 at 3:18 PM Henk Smit  wrote:
>
>>
>> Hello Les,
>>
>> Les Ginsberg (ginsberg) wrote on 2019-07-24 07:17:
>>
>> > If you accept that, then it makes sense to look for the simplest way
>> > to do flow control and that is decidedly not from the RX side. (I
>> > expect Tony Li to disagree with that 😊 – but I have already
>> > outlined why it is more complex to do it from the Rx side.)
>>
>> In your talk on Monday you called the idea in
>> draft-decraene-lsr-isis-flooding-speed-01 "receiver driven flow
>> control".
>> You don't like that. You want "transmit based flow control".
>> You argued that you can do "transmit based flow control" on the sender
>> only.
>> Therefor your algorithm is merely a "local trick".
>> And "local tricks" don't need RFCs. I agree with that.
>> But I don't agree that your algorithm is just a "local trick"..
>>
>>
>> In your algorithm, a "sender" sends a number of LSPs to a receiver.
>> Without waiting for acks (PNSPs). Like in any sliding window protocol.
>> The sending router keeps an eye on the number of unacked LSPs.
>> And it determines how fast it can send more LSPs based on the current
>> number of unacked LSPs. Every time the sender receives a PSNP, it
>> knows the receiver got a number of LSPs, so it can increase its
>> send-window again, and then send more LSPs.
>> Correct ?
>>
>> I agree that the core idea of this algorithm makes sense.
>> After all, it looks a lot like TCP.
>> I believe the authors of draft-decraene-lsr-isis-flooding-speed were
>> planning something like that for the next version of their draft.
>>
>>
>> However, I do not agree with the name "tx driven flow control".
>> I also do not agree that this algorithm is "a local trick".
>> Therefor I also do not think this algorithm doesn't need to be
>> documented (in an RFC).
>>
>> In your "tx based flow control", the sender (tx) sends LSPs at a rate
>> that is derived from the rate at which it receives PSNPs. Therefor
>> it is the sender of the PSNPs that sets the speed of transmission !
>> So it is still the receiver (of LSPs) that controls the flow control.
>> The name "tx based flow control" is a little misleading, imho.
>>
>>
>> It is important to realize that the success of your algorithm actually
>> depends on the behaviour of the receiver. How does it send PSNPs ?
>> Does it send one PSNP per received LSP ? Or does it pack multiple acks
>> in one PSNP ? Does it send a PSNP immediatly, or does it wait a short
>> time ? Does it try to fill a PSNP to the max (putting ~90 acks in one
>> PSNP) ? Does the receiver does something in between ? I don't think
>> the behaviour is specified exactly anywhere.
>>
>> I know about an IS-IS implementation from the nineties. When a router
>> would receive an LSP, it would a) set the 

Re: [Lsr] Dynamic flow control for flooding

[Robert Raszuk]

Hi,

Yes indeed while I was reading your richly connected node restart problem
use of overload-bit should be explored, proposed, implemented.

For the partition problem I have two general comments:

a) If network partitions is likely to happen more often in the case of
dynamic flooding perhaps as already said before we should increase the max
number of occurrences given LSP is to arrive at flooding optimized node.
Two may not be enough.

b)  If protocol extensions will help to mitigate effects of network
partition via much faster repair some folks may treat network partitions as
normal operational model and instead of re-architecting the network to make
sure network partition events are as rear as possible.

Thx,
R.

On Wed, Jul 24, 2019 at 4:12 PM Henk Smit  wrote:

>
> Hello Robert,
>
> Tony brought up the example of a partioned network.
> But there are more examples.
>
> E.g. in a network there is a router with a 1000 neighbors.
> (When discussing distributed vs centralized flooding-topology
>   reduction algorithms, I've been told these network designs exist).
> When such a router reboots/crashes/comes back up, all 1000 neighbors
> will create a new version of their own LSP. This causes a 1000 different
> LSPs to be flooded through the network at the same time. Impacting every
> router in the network.
>
> The case I was thinking of myself, was when a router in a large network
> boots. When it brings up a number of adjacencies, each neighbor will
> try to synchronize its LSPDB with the newly booted router. As the newly
> booted router will send emtpy CSNPs to each of its neighbors, each
> neighbor will start sending the full LSPDB. If such a network has 10k
> LSPs, and such a router has 100 neighbors, that router will receive 100
> * 10k
> is 1 million LSPs. Having a faster and more efficient flooding
> transport,
> with flow-control, will make a reboot in such a topology less painful.
>
> (In that last case, creative use of the overload-bit could prevent
> black-holing
> or microloops while ISIS synchronizes its LSPDB after a reboot. Just
> like we
> used the overload-bit to solve the problem of slow convergence of BGP
> after
> a reboot, 22 years ago. I have no idea if there are any implementations
> that
> use the overload-bit to alleviate slow convergence of IS-IS after a
> reboot).
>
> henk.
>
>
> Robert Raszuk schreef op 2019-07-24 15:33:
> > Hey Henk & all,
> >
> > If acks for 1000 LSPs take 16 PSNPs (max 66 per PSNP) or even as long
> > as Tony mentioned the full flooding as Tony said may take 33 sec - is
> > this really a problem ?
> >
> > Remember we are not talking about protocol convergence after link flap
> > or node going down. We are talking about serious network partitioning
> > which itself may have lasted for minutes, hours or days. While just
> > considering absolute numbers yelds desire to go faster and faster, if
> > we put things in the overall perspective is there really a problem to
> > be solved in the first place ?
> >
> > Would there still be a problem if LSR WG recommends faster acking
> > maybe not for each LSP but for say 20 or 30 max ?
> >
> > Thx,
> > R.
>
> ___
> Lsr mailing list
> Lsr@ietf.org
> https://www.ietf.org/mailman/listinfo/lsr
>
___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Henk Smit]

Hello Robert,

Tony brought up the example of a partioned network.
But there are more examples.

E.g. in a network there is a router with a 1000 neighbors.
(When discussing distributed vs centralized flooding-topology
 reduction algorithms, I've been told these network designs exist).
When such a router reboots/crashes/comes back up, all 1000 neighbors
will create a new version of their own LSP. This causes a 1000 different
LSPs to be flooded through the network at the same time. Impacting every
router in the network.

The case I was thinking of myself, was when a router in a large network
boots. When it brings up a number of adjacencies, each neighbor will
try to synchronize its LSPDB with the newly booted router. As the newly
booted router will send emtpy CSNPs to each of its neighbors, each
neighbor will start sending the full LSPDB. If such a network has 10k
LSPs, and such a router has 100 neighbors, that router will receive 100 
* 10k
is 1 million LSPs. Having a faster and more efficient flooding 
transport,

with flow-control, will make a reboot in such a topology less painful.

(In that last case, creative use of the overload-bit could prevent 
black-holing
or microloops while ISIS synchronizes its LSPDB after a reboot. Just 
like we
used the overload-bit to solve the problem of slow convergence of BGP 
after
a reboot, 22 years ago. I have no idea if there are any implementations 
that
use the overload-bit to alleviate slow convergence of IS-IS after a 
reboot).


henk.


Robert Raszuk schreef op 2019-07-24 15:33:

Hey Henk & all,

If acks for 1000 LSPs take 16 PSNPs (max 66 per PSNP) or even as long
as Tony mentioned the full flooding as Tony said may take 33 sec - is
this really a problem ?

Remember we are not talking about protocol convergence after link flap
or node going down. We are talking about serious network partitioning
which itself may have lasted for minutes, hours or days. While just
considering absolute numbers yelds desire to go faster and faster, if
we put things in the overall perspective is there really a problem to
be solved in the first place ?

Would there still be a problem if LSR WG recommends faster acking
maybe not for each LSP but for say 20 or 30 max ?

Thx,
R.


___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Robert Raszuk]

Hey Henk & all,

If acks for 1000 LSPs take 16 PSNPs (max 66 per PSNP) or even as long as
Tony mentioned the full flooding as Tony said may take 33 sec - is this
really a problem ?

Remember we are not talking about protocol convergence after link flap or
node going down. We are talking about serious network partitioning which
itself may have lasted for minutes, hours or days. While just considering
absolute numbers yelds desire to go faster and faster, if we put things in
the overall perspective is there really a problem to be solved in the first
place ?

Would there still be a problem if LSR WG recommends faster acking maybe not
for each LSP but for say 20 or 30 max ?

Thx,
R.








On Wed, Jul 24, 2019 at 3:18 PM Henk Smit  wrote:

>
> Hello Les,
>
> Les Ginsberg (ginsberg) wrote on 2019-07-24 07:17:
>
> > If you accept that, then it makes sense to look for the simplest way
> > to do flow control and that is decidedly not from the RX side. (I
> > expect Tony Li to disagree with that 😊 – but I have already
> > outlined why it is more complex to do it from the Rx side.)
>
> In your talk on Monday you called the idea in
> draft-decraene-lsr-isis-flooding-speed-01 "receiver driven flow
> control".
> You don't like that. You want "transmit based flow control".
> You argued that you can do "transmit based flow control" on the sender
> only.
> Therefor your algorithm is merely a "local trick".
> And "local tricks" don't need RFCs. I agree with that.
> But I don't agree that your algorithm is just a "local trick".
>
>
> In your algorithm, a "sender" sends a number of LSPs to a receiver.
> Without waiting for acks (PNSPs). Like in any sliding window protocol.
> The sending router keeps an eye on the number of unacked LSPs.
> And it determines how fast it can send more LSPs based on the current
> number of unacked LSPs. Every time the sender receives a PSNP, it
> knows the receiver got a number of LSPs, so it can increase its
> send-window again, and then send more LSPs.
> Correct ?
>
> I agree that the core idea of this algorithm makes sense.
> After all, it looks a lot like TCP.
> I believe the authors of draft-decraene-lsr-isis-flooding-speed were
> planning something like that for the next version of their draft.
>
>
> However, I do not agree with the name "tx driven flow control".
> I also do not agree that this algorithm is "a local trick".
> Therefor I also do not think this algorithm doesn't need to be
> documented (in an RFC).
>
> In your "tx based flow control", the sender (tx) sends LSPs at a rate
> that is derived from the rate at which it receives PSNPs. Therefor
> it is the sender of the PSNPs that sets the speed of transmission !
> So it is still the receiver (of LSPs) that controls the flow control.
> The name "tx based flow control" is a little misleading, imho.
>
>
> It is important to realize that the success of your algorithm actually
> depends on the behaviour of the receiver. How does it send PSNPs ?
> Does it send one PSNP per received LSP ? Or does it pack multiple acks
> in one PSNP ? Does it send a PSNP immediatly, or does it wait a short
> time ? Does it try to fill a PSNP to the max (putting ~90 acks in one
> PSNP) ? Does the receiver does something in between ? I don't think
> the behaviour is specified exactly anywhere.
>
> I know about an IS-IS implementation from the nineties. When a router
> would receive an LSP, it would a) set the SSN bit (for that
> LSP/interface),
> and b) start the psnp-timer for that interface (if not already running).
> The psnp-timer would expire 2 seconds later. The router would then walk
> the LSPDB, find all LSPs with the SSN-bit set for that interface. And
> then build a PSNP with acks for all those LSPs. The result would be
> that: a) the first PSNP would be send 2 seconds (+/- jitter) after
> receiving the first LSP, and b) the PSNP would include ~66 acks. (As
> a router receiving at full speed would have received 66 LSPs in 2
> seconds).
>
> For your "tx based flow control" algorithm to work properly, this has
> to change. The receiving router must send PSNPs more quickly and more
> aggressively. The result would be that there will be less acks in each
> PSNP. And thus more PSNPs will be sent.
>
> This makes us realize: in the current situation, if a router receives
> a 1000 LSPs, and sends those LSPs to 64 neighbors, it would receive:
> - the 1000 LSPs from an upstream neighbor, plus
> - 1000/66 = 16 PSNPs from each downstream neighbor = 64 * 16 = 1024
> PSNPs.
> This makes a total of ~2000K PDUs received.
>
> If routers would send one PSNP per LSP (to have faster flow control),
> then the router in this example would receive:
> - the 1000 LSPs from an upstream neighbor, plus
> - 1000 PSNPs from each downstream neighbor * 16 = 1600 PSNPs.
> This makes a total of ~17000 PDUs received.
>
> The total number of PDUs received on this router would go from 2K PDUs
> to 17K PDUs.
>
> Remember that the problem we're trying to solve here is to make sure
> 

Re: [Lsr] Dynamic flow control for flooding

[Henk Smit]

Hello Les,

Les Ginsberg (ginsberg) wrote on 2019-07-24 07:17:


If you accept that, then it makes sense to look for the simplest way
to do flow control and that is decidedly not from the RX side. (I
expect Tony Li to disagree with that 😊 – but I have already
outlined why it is more complex to do it from the Rx side.)


In your talk on Monday you called the idea in
draft-decraene-lsr-isis-flooding-speed-01 "receiver driven flow 
control".

You don't like that. You want "transmit based flow control".
You argued that you can do "transmit based flow control" on the sender 
only.

Therefor your algorithm is merely a "local trick".
And "local tricks" don't need RFCs. I agree with that.
But I don't agree that your algorithm is just a "local trick".


In your algorithm, a "sender" sends a number of LSPs to a receiver.
Without waiting for acks (PNSPs). Like in any sliding window protocol.
The sending router keeps an eye on the number of unacked LSPs.
And it determines how fast it can send more LSPs based on the current
number of unacked LSPs. Every time the sender receives a PSNP, it
knows the receiver got a number of LSPs, so it can increase its
send-window again, and then send more LSPs.
Correct ?

I agree that the core idea of this algorithm makes sense.
After all, it looks a lot like TCP.
I believe the authors of draft-decraene-lsr-isis-flooding-speed were
planning something like that for the next version of their draft.


However, I do not agree with the name "tx driven flow control".
I also do not agree that this algorithm is "a local trick".
Therefor I also do not think this algorithm doesn't need to be
documented (in an RFC).

In your "tx based flow control", the sender (tx) sends LSPs at a rate
that is derived from the rate at which it receives PSNPs. Therefor
it is the sender of the PSNPs that sets the speed of transmission !
So it is still the receiver (of LSPs) that controls the flow control.
The name "tx based flow control" is a little misleading, imho.


It is important to realize that the success of your algorithm actually
depends on the behaviour of the receiver. How does it send PSNPs ?
Does it send one PSNP per received LSP ? Or does it pack multiple acks
in one PSNP ? Does it send a PSNP immediatly, or does it wait a short
time ? Does it try to fill a PSNP to the max (putting ~90 acks in one
PSNP) ? Does the receiver does something in between ? I don't think
the behaviour is specified exactly anywhere.

I know about an IS-IS implementation from the nineties. When a router
would receive an LSP, it would a) set the SSN bit (for that 
LSP/interface),

and b) start the psnp-timer for that interface (if not already running).
The psnp-timer would expire 2 seconds later. The router would then walk
the LSPDB, find all LSPs with the SSN-bit set for that interface. And
then build a PSNP with acks for all those LSPs. The result would be
that: a) the first PSNP would be send 2 seconds (+/- jitter) after
receiving the first LSP, and b) the PSNP would include ~66 acks. (As
a router receiving at full speed would have received 66 LSPs in 2 
seconds).


For your "tx based flow control" algorithm to work properly, this has
to change. The receiving router must send PSNPs more quickly and more
aggressively. The result would be that there will be less acks in each
PSNP. And thus more PSNPs will be sent.

This makes us realize: in the current situation, if a router receives
a 1000 LSPs, and sends those LSPs to 64 neighbors, it would receive:
- the 1000 LSPs from an upstream neighbor, plus
- 1000/66 = 16 PSNPs from each downstream neighbor = 64 * 16 = 1024 
PSNPs.

This makes a total of ~2000K PDUs received.

If routers would send one PSNP per LSP (to have faster flow control),
then the router in this example would receive:
- the 1000 LSPs from an upstream neighbor, plus
- 1000 PSNPs from each downstream neighbor * 16 = 1600 PSNPs.
This makes a total of ~17000 PDUs received.

The total number of PDUs received on this router would go from 2K PDUs
to 17K PDUs.

Remember that the problem we're trying to solve here is to make sure
that routers do not get overrun on the receipt side with too many
packets too quickly. It seems an aggressive PSNP-scheme, to achieve
faster flow-control, is actually very counter-productive.

Of course the algorithm can be tweaked. E.g. TCP sends one ack per
every 2 received segments (if I'm not mistaken). If we do that here,
the number of PDUs would go down from 17K to 9K PDUs. What do you
propose ? How do you want the feedback of PSNPs to be quick, while
maintaining an efficient packing of multiple acks per PSNP ?


In any case, the points I'm trying to make here:
*) Your algorithm is not sender-driven, but still receiver-driven.
*) Your algorithm changes/dictates behaviour both on sender and 
receiver.
*) Interaction between a sender and a receiver is what we call a 
protocol.
   If you want to make this work, especially in multi-vendor 
environments,

   we need to document these algorith

Re: [Lsr] Dynamic flow control for flooding

[henk . ietf]

Hello Les,

Les Ginsberg (ginsberg) schreef op 2019-07-24 07:17:


There is something to be said for simply “flooding fast” and not
worrying about flow control at all (regardless of whether TX or RX
mechanisms would be used). Some packets would be dropped, but
retransmission timers will insure that the flooding eventually
succeeds and retransmit timers are long (5 seconds by default). (I am
not the only one mentioning this BTW…)


Why do we have initial waits and exponential backoffs for LSP-generation
and SPF-computations ? Why not react immediately ? Why not react 
constantly ?


We have a lot of bandwidth and cpu-power now. Isn't simple always better
than "overly complex stuff" like exponential backoffs ? If you have more
cpu-power, more memory and more bandwidth, why invent new algorithms ?

henk.


I hope it is clear to everyone that these are not serious questions. I'm 
just
saying: "sometimes fast is slow". I am sure that if we ask the "old 
guys", they
can come up with many stories how these problems are sometimes 
counter-intuitive.
And how networks have melted because "fast is slow". I could tell at 
least 2

of those stories myself.

___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr

Re: [Lsr] Dynamic flow control for flooding

[Robert Raszuk]

Hi Tony,

> I am assuming that there is no link layer flow control.  I can’t recall
> working on a system that supports X.25 since about 1995, so I don’t think
> that’s a common use case today.
>

I was more thinking along the lines of leveraging IEEE 802.3x or 802.1Qbb
standard not necessarily suggest fancy X.25 or Frame Relay :)

 Henk proposed that we simply pick up TCP for this, but my concern with
> that is really about introducing a whole new dependency to the protocol.
> That’s a lot to chew.  Do we really need it all? I hope not.  Thus, Bruno’s
> original suggestion sparked my interest in doing something dynamic and
> simple.
>

The second part of the question was really about at what layer it makes
most sense to provide this control loop.

Options seems to be:

* Invent new or use existing link layer flow control (IEEE)
* Reuse existing transport layer (TCP)
* App layer (QUIC or QUIC like)

I guess it would be useful to up front list on what type of media this must
be supported as it may change the game drastically:

* directly connected fiber p2p
* p2mp (via switch)
* p2p over encapsulation
etc...

Thx,
R.
___
Lsr mailing list
Lsr@ietf.org
https://www.ietf.org/mailman/listinfo/lsr