Re: [Pce] new draft on segment routing approach to TSN

[Yaakov Stein]

So was I.

Y(J)S

From: Jakob Heitz (jheitz) 
Sent: 26/02/2021 09:24
To: Yaakov Stein 
Cc: Pascal Thubert (pthubert) ; 
det...@ietf.org; Tianran Zhou ; pce@ietf.org; 
spr...@ietf.org
Subject: Re: new draft on segment routing approach to TSN

i'm talking about point to point
Regards,
Jakob.



On Feb 25, 2021, at 10:08 PM, Yaakov Stein 
mailto:yaako...@rad.com>> wrote:

And I forgot to mention the real way this is handled in sophisticated wireless 
systems.

There shouldn’t be any idle time!

If there are less bits to be transmitted then the modulation order is reduced
increasing the margin and reducing the probability of error!

Even for NRZ Ethernet transmitting random frames that happen to the next in a 
cyclic buffer
is in general not the best option.
Not only does it not guarantee retransmission of a bad packet (unless is really 
very little to transmit),
it wastes a lot of energy on transmitting packets that have already been 
received.
If there is so little to transmit you should be powering down your transmitter 
(802.3az).

Y(J)S

From: Jakob Heitz (jheitz) mailto:jhe...@cisco.com>>
Sent: 26/02/2021 07:56
To: Yaakov Stein mailto:yaako...@rad.com>>; Pascal Thubert 
(pthubert) 
mailto:pthubert=40cisco@dmarc.ietf.org>>
Cc: det...@ietf.org; Tianran Zhou 
mailto:zhoutian...@huawei.com>>; 
pce@ietf.org; spr...@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Lots of ways. Another:
Use a circular buffer. As you transmit a packet put it into the buffer.
As it is acknowledged, remove it.
Just keep transmitting the buffer circularly.
If one is NACKed, retransmit it immediately.
The receiver can keep track of received sequence numbers.
They should be in order. Ack the latest sequence whenever there is a chance.
If the receiver sees a hole in the received sequence, NACK it.

Regards,
Jakob.

From: Yaakov Stein mailto:yaako...@rad.com>>
Sent: Thursday, February 25, 2021 9:48 PM
To: Jakob Heitz (jheitz) mailto:jhe...@cisco.com>>; Pascal 
Thubert (pthubert) 
mailto:pthubert=40cisco@dmarc.ietf.org>>
Cc: det...@ietf.org; Tianran Zhou 
mailto:zhoutian...@huawei.com>>; 
pce@ietf.org; spr...@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Jakob

I forgot to address your first sentence.

Retransmission without NACK is an interesting idea, but I am not sure it really 
helps.

If there is nothing to be transmitted then you could go into a loop and 
retransmit packets over and over,
but I assume that this usually won’t happen.
So assume that you have a short idle window – which packet are you going to 
retransmit?
Without knowing which packet was not received correctly you have no idea.

So let’s assume that you reserve a full 50% of your BW and retransmit every 
packet.
Some packets received twice will differ – which one is correct? You can’t know 
without a FEC.
And if you are using a FEC it is more efficient to use a strong one then 
retransmit the whole packet!

So, let’s assume that you don’t use a FEC (maybe you don’t want the hassle of 
implementing one in hardware).
Then you need to send each packet at least 3 times to be able to do a majority 
decision.
So, you have reduced your BW efficiency to 33%.

I’m not saying that this is never justified. Lots of messages are sent 3 times 
for redundancy.
Going back to cellular, triple redundancy is used for critical control messages 
(in addition to other error correction means)/
But only for them. It is really suboptimal for everything else.

Y(J)S




From: Yaakov Stein
Sent: 26/02/2021 07:38
To: Jakob Heitz (jheitz) mailto:jhe...@cisco.com>>; Pascal 
Thubert (pthubert) 
mailto:pthubert=40cisco@dmarc.ietf.org>>
Cc: det...@ietf.org; Tianran Zhou 
mailto:zhoutian...@huawei.com>>; 
pce@ietf.org; spr...@ietf.org
Subject: RE: new draft on segment routing approach to TSN

My point regarding HARQ was that this kind of retransmission can be more 
efficiently handled
at layers lower than where you see packets.
4G/5G has actually two mechanisms precisely of the type you suggest at two 
different layers,
but the magic happens on the physical signals (HARQ) or the SDUs (PDCP 
retransmission)
and is completely transparent to the user packets.

True, high speed Ethernet links are no longer physical broadcast domains
and one could insert sequence numbers into the frames and perform local 
retransmission.
This has been done in the past (I personally have a patent on such a method, 
although it also gives an alternative to numbering).
But if you are already numbering it might be even better to go all the way and 
use RFER.
(Or maybe even combine ACK/NACK retransmission and replication.)

Y(J)S


From: Jakob Heitz (jheitz) mailto:jhe...@cisco.com>>
Sent: 26/02/2021 07:00
To: Yaakov Stein mailto:yaako...@rad.com>>; Pascal Thubert 
(pt

Re: [Pce] new draft on segment routing approach to TSN

[Tianran Zhou]

Hi Yaakov,

Thanks for your clarification.
Again I think this is an interesting work.
Please see inline.

Cheers,
Tianran

From: Yaakov Stein [mailto:yaako...@rad.com]
Sent: Friday, February 26, 2021 1:09 PM
To: Tianran Zhou 
Cc: det...@ietf.org; spr...@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Tianran,

For some reason I didn’t see your email and thus never responded to it. My 
apologies.


1.   I believe that I address this question at the beginning of my draft.

Using a single deadline per packet at is suboptimal at any particular switch.

For example, say two packets arrive at a switch, one with only 20 microseconds 
left on its deadline

and one with 50. The switch would naturally schedule the 10 microsecond one 
first.

But what the switch doesn’t know is that it is the last hop for the 10 
microsecond one

and there are only 2 microseconds from it to destination,

while the 50 microsecond packet still has 45 microseconds of physical time 
ahead of it!



ZTR> Yes, you provided a valid use case in some sense.

But I have to say, IMO, the real-time system cares more about whether the 
deadline could meet.

So in practice, the task set is fixed, and all the latencies are calculated and 
guaranteed to meet the dead line. I do not care whether the packet could arrive 
earlier or not.

So, an deterministic and easy schedule make the evaluation/plan easier. And 
It’s better to see all the tasks could be accommodated in the plan.



ZTR> I can provide an use case, say a task could meet the 20 end to end 
deadline within 2 hops, one hop for 14, and the other for 6.

If we give split hop by hop deadlines, 10 for each.  That means the first hop 
with 14 delay cannot meet the 10 local deadline.

So for the first schedule, this task could be accommodated. But for the second 
one, this task cannot be accommodated.


2.   I gave a specific algorithm in the draft and pointed to Andrews and 
Zhang who give another one.

ZTR> Thanks for the pointer, I would like to look into them.

3.   I don’t think a service needs to support EDF, but perhaps a switch 
does.

I haven’t seen EDF support since the ATM days, but am proposing that perhaps 
the time has come to re-evaluate it.

However, I am not advocating for EDF in this draft, just for the stack.

I mention a variant of EDF which I believe is better, and am working on another 
variant which is even better.

They can all be built into hardware, although admitted are more complex than a 
simple queue.

But they may be simpler than a set of queues with time schedules like Qbv

and are much much simpler than monstrosities like MEF 10.3 token bucket with 
cross-color/cross-CoS sharing and coupling.



ZTR> Not sure if my thoughts are common sense or not. ☺

IMO, the merit of Qbv, or TAS, is to isolate hard real-time, soft real-time, 
best effort tasks. So that they can accommodate in one switch.

This is one critical requirement for IT/OT integration.

EDF does not conflict with TAS. I worked on some hierarchical scheduling 
mechanisms many years ago, EDF is used in the time slot for hard real-time 
tasks.


Y(J)S



From: detnet mailto:detnet-boun...@ietf.org>> On 
Behalf Of Tianran Zhou
Sent: jeudi 25 février 2021 9:14
To: Yaakov Stein mailto:yaako...@rad.com>>; 
det...@ietf.org; 
spr...@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Hi Yaakov,

This is an interesting topic.
After a quick review, there are several questions as follows:
1. It’s clear to me to have a deadline for each packet. So that router can 
schedule the packet based on the urgency. But what’s the motivation to split 
the end to end deadline to several local ones?
2. How to divide an end to end deadline into several local deadlines? Is there 
any example algorithm that could be used by the controller?
3. As far as I know, most devices do not support edf. I am not sure whether 
your proposal based on edf could really be useful.

Cheers,
Tianran


From: Pce [mailto:pce-boun...@ietf.org] On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 9:14 PM
To: det...@ietf.org; 
spr...@ietf.org; pce@ietf.org
Subject: [Pce] new draft on segment routing approach to TSN

All,

I would like to call your attention to a new ID 
https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt
which describes using a stack-based approach (similar to segment routing) to 
time sensitive 

Re: [Pce] new draft on segment routing approach to TSN

[Pascal Thubert (pthubert)]

Hello Yaakov

If it was me I’d split the draft to take the deeper explanation of the why in a 
framework informational piece and leave the minimum in the standard track 
document.

Think that the implementer is a person of the art, and that the IESG will 
challenge every word in the standard track specification...

I d welcome text that explains that the PSE has to look at not only where to 
forward but also when to forward, in the RAW architecture/ framework draft.

And yes I often use train station and bus lines and connection to explain 
deterministic...

Take care,

Pascal

Le 26 févr. 2021 à 05:32, Yaakov Stein  a écrit :


Pascal,

Please accept my apologies for having misunderstood your intention regarding 
“green waves”.

I originally read it as two separate statements, namely

  1.  people won’t understand the “green wave” analogy
  2.  people in general think that TSN is magic and no packet ever waits
while I now understand that you intended one statement

  1.  people will misunderstand the analogy to green waves and believe that I 
am implying that no packet ever waits.

My reply to your true intention is that even in those places where traffic 
lights are coordinated
it doesn’t mean that every car is first to go through every light or that you 
don’t have to slow down before lights.
It merely implies that there is high correlation between car arrival times and 
green light periods.
This is what Qbv and calendaring are supposed to accomplish,
and if people will be misled then perhaps I need to find another analogy.

Perhaps something with bus arrival times is more appropriate where people queue 
up to enter the bus.
The reason I wanted to avoid that one is that buses don’t stick to schedule.
Since they wait longer when there happen to be more passengers getting on,
they then meet even more passengers at the next stop, while the following bus 
meets fewer, resulting in bunching.

Y(J)S

From: Pascal Thubert (pthubert) 
Sent: 25/02/2021 23:27
To: Yaakov Stein 
Cc: Tianran Zhou ; det...@ietf.org; spr...@ietf.org; 
pce@ietf.org
Subject: Re: new draft on segment routing approach to TSN

Hello Yaakov

Please see below:



Le 25 févr. 2021 à 21:33, Yaakov Stein 
mailto:yaako...@rad.com>> a écrit :

Pascal,

Wow, there is a lot to respond to.

Thanks for the pointers. I was not aware of this work.
However, I believe this proposal is both different and can be complementary to 
the ones you mention.

Indeed; the questions that this raises are:

- which WG, so I added RAW to the candidate list in this thread

- how to package the complementary proposals in how many draft

Your excellent work focuses on SR signaling in the forwarding plane with strict 
intermediate latencies at each stage. It would be grand to neatly insert this 
in a SRv6 together with other information that the RAW PSE processes in its 
forwarding/scheduling decision.

There are multiple dimensions in which the scope of your work could be 
enlarged; one of them would be to use a similar signaling but with a best 
effort objective, which is what I discussed. This could easily be combined if 
we decide to, so the same signaling serves both use cases. And maybe more, to 
be discussed.



I was going to add a section on how to reproduce Qbv behavior using a stack of 
deadlines (the argument is a bit tricky, but straightforward).
I don’t necessarily think that is a good idea, but it should be there to 
counter claims that SRTSN only reduces the expectation
but can’t provide an absolute upper bound on latency (when based on EDF).

Very cool;



I like to use the term latency, but sometimes use delay to avoid overusing it.

Your decision, at least as long as it is personal submission ; but having a 
clear terminology is useful. Literature value is of lesser importance here I 
guess. Latency is what we use in DetNet and RAW. Delay often refers to extra, 
possibly unwanted time, like a queueing delay. At least you’d make my reader 
life easier by defining your terminology and sticking to it quite strictly : )



I thought that “green wave” would be understood by those who drive cars in 
cities where traffic light coordination is common.

Yes, and I think you were clear enough; but then only a few deterministic flows 
really operate like that. Another flow that use the same outgoing port will 
have to suffer some delay in a time-triggered queue till the port is free, the 
delay being taken from the end to end latency budget. My suggestion was to 
reconsider the text to avoid giving the impression that all flows benefit from 
that green wave.



I attempted to describe a data structure for EDF to be used instead of a queue,

Neat


but never implied that “queueing” (meaning packets waiting to be transmitted) 
doesn’t occur.
If that were the case there would not be a need for any data structure at all.

Cool


(Another thing I didn’t get into was the trade-offs among the different options,
 from worst case complexity and difficulty to implement in

Re: [Pce] new draft on segment routing approach to TSN

[Pascal Thubert (pthubert)]

Hello Jakob

There’s a lot about that in the RAW architecture; please start there and then 
visit the DetNet architecture as well if you need to dig deeper.

 The bottom lines are:

- there will be multiple copies of a packet using all sorts of diversity 
including path diversity;
- the packets may be fragmented/ network coded to reduce the influence of very 
short term bit error events;
- there is no ARQ as typically thought of, though there can be more than a 
single TXOP scheduled for that frame over that wireless hop
- the tail end will typically receive more then one copy of a packet, think 2.x 
in average so as to almost never completely miss a packet.

Take care,

Pascal

Le 26 févr. 2021 à 06:00, Jakob Heitz (jheitz) 
 a écrit :


Suppose you're sending packets out an interface and all the queues are empty. 
You got nothing to send.
What are you going to do with the dead air? Send idle patterns?
No.
Retransmit your previous queues. Just in case something got lost the preemptive 
retransmission will fix it quicker.
Then put a link layer sequence number and ACK and NACK numbers so you can 
manage what you retransmit.
This will only work on point-to-point. Ethernet's not broadcast on the physical 
layer anymore, is it?

Regards,
Jakob.

From: Yaakov Stein 
Sent: Thursday, February 25, 2021 8:56 PM
To: Jakob Heitz (jheitz) ; Pascal Thubert (pthubert) 

Cc: det...@ietf.org; Tianran Zhou ; pce@ietf.org; 
spr...@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Of course they are common in wireless.
Cellular HARQ works just like that – if the FEC fixes the problem then fine, if 
not then the information is retransmitted.
And HARQ not only causes timing problems when there are errors, the HARQ SAW 
(Stop and Wait) process adds delay to all information transmitted.

However, I was mostly thinking about wireline cases.
Pascal pointed to the work being done in RAW, which is where this question 
needs to be addressed.

Y(J)S

From: Jakob Heitz (jheitz) mailto:jhe...@cisco.com>>
Sent: 26/02/2021 06:47
To: Yaakov Stein mailto:yaako...@rad.com>>; Pascal Thubert 
(pthubert) 
mailto:pthubert=40cisco@dmarc.ietf.org>>
Cc: det...@ietf.org; Tianran Zhou 
mailto:zhoutian...@huawei.com>>; 
pce@ietf.org; spr...@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Are they rare in wireless transmission? I believe some wireless protocols have 
lower layers that retransmit to provide reliability, but I don't know much 
about them.
My point is if you are going to provide a guarantee of time for delivery, you 
need to consider all factors that influence that time.

Also, what happens if you can't meet the deadline? Do you drop the packet or 
deliver with best effort?

Regards,
Jakob.

From: Yaakov Stein mailto:yaako...@rad.com>>
Sent: Thursday, February 25, 2021 8:40 PM
To: Jakob Heitz (jheitz) mailto:jhe...@cisco.com>>; Pascal 
Thubert (pthubert) 
mailto:pthubert=40cisco@dmarc.ietf.org>>
Cc: det...@ietf.org; Tianran Zhou 
mailto:zhoutian...@huawei.com>>; 
pce@ietf.org; spr...@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Jakob

Yes, one always needs to consider bit errors (even if they are very rare in 
wireline nowadays).

I’m just not sure where you want to hang this on the present discussion.

A local retransmission mechanism such as the one you suggest
would obviously thwart any calendaring mechanism that attempts to coordinate 
scheduling based on traffic timing
(unless the detection, signaling, and retransmission takes place much faster 
than an opening time).
LIS could be made to work by retransmitting with the original timestamp, and 
EDF would work just fine.

Y(J)S

From: Jakob Heitz (jheitz) mailto:jhe...@cisco.com>>
Sent: 26/02/2021 01:36
To: Pascal Thubert (pthubert) 
mailto:pthubert=40cisco@dmarc.ietf.org>>;
 Yaakov Stein mailto:yaako...@rad.com>>
Cc: det...@ietf.org; Tianran Zhou 
mailto:zhoutian...@huawei.com>>; 
pce@ietf.org; spr...@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Another thing to consider is packets dropped due to bit errors.
Possibilities: Link level acknowledgement and preemptive retransmission
if the link would otherwise be idle. I.e. "just in case" retransmission.

Regards,
Jakob.

From: spring mailto:spring-boun...@ietf.org>> On 
Behalf Of Pascal Thubert (pthubert)
Sent: Thursday, February 25, 2021 1:27 PM
To: Yaakov Stein mailto:yaako...@rad.com>>
Cc: det...@ietf.org; Tianran Zhou 
mailto:zhoutian...@huawei.com>>; 
pce@ietf.org; spr...@ietf.org
Subject: Re: [spring] new draft on segment routing approach to TSN

Hello Yaakov

Please see below:


Le 25 févr. 2021 à 21:33, Yaakov Stein 
mailto:yaako...@rad.com>> a écrit :

Pascal,

Re: [Pce] Alvaro Retana's No Objection on draft-ietf-pce-pcep-extension-for-pce-controller-12: (with COMMENT)

[Pengshuping (Peng Shuping)]

Hi Alvaro,

Thank you for your comments! Please find the diff and the responses in line 
below. Thank you!

Diff: 
https://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=draft-ietf-pce-pcep-extension-for-pce-controller-12&url2=https://raw.githubusercontent.com/dhruvdhody/ietf/master/draft-ietf-pce-pcep-extension-for-pce-controller-13.txt
 


> -Original Message-
> From: Alvaro Retana via Datatracker [mailto:nore...@ietf.org]
> Sent: Tuesday, February 23, 2021 2:28 AM
> To: The IESG 
> Cc: draft-ietf-pce-pcep-extension-for-pce-control...@ietf.org;
> pce-cha...@ietf.org; pce@ietf.org; Julien Meuric
> 
> Subject: Alvaro Retana's No Objection on
> draft-ietf-pce-pcep-extension-for-pce-controller-12: (with COMMENT)
> 
> Alvaro Retana has entered the following ballot position for
> draft-ietf-pce-pcep-extension-for-pce-controller-12: No Objection
> 
> When responding, please keep the subject line intact and reply to all email
> addresses included in the To and CC lines. (Feel free to cut this introductory
> paragraph, however.)
> 
> 
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
> 
> 
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-pce-pcep-extension-for-pce-contr
> oller/
> 
> 
> 
> --
> COMMENT:
> --
> 
> (0) The fact that the procedures (§5) are presented before introducing the
> messages/objects (§6-7) makes reading this document harder and more
> complex than it has to be.  Consider changing the order or at least adding
> forward references in §5.

Forward references are added.

> 
> (1) §5.2:  Is there a reason for the messages from rfc8231 to be in
> parenthesis?

No, removed parenthesis!

> 
> (2) §5.4:
> 
>The PCECC-CAPABILITY sub-TLV SHOULD NOT be used without the
>corresponding Path Setup Type being listed in the PATH-SETUP-TYPE-
>CAPABILITY TLV.  If it is present without the corresponding Path
>Setup Type listed in the PATH-SETUP-TYPE-CAPABILITY TLV, it MUST be
>ignored.
> 
> When is it ok to use the PCECC-CAPABILITY sub-TLV without the
> corresponding PST?  If the result is that it will be ignored, then I don't
> understand why the use of both is not required.

Changed to MUST NOT

> 
> (3) §5.5.1/§5.5.4: "ingress MAY further choose to deploy a data plane check
> mechanism and report the status back to the PCE"  Is this (checking and
> reporting) specified somewhere?  Because you're using normative language,
> please add a reference.
> 
> A similar statement is made in §5.5.7: "ingress PCC MAY choose to apply any
> OAM mechanism to check the status of LSP in the Data plane and MAY
> further send its status in a PCRpt message to the PCE".

Removed the normative language and added "The exact mechanism is out of scope 
of this document."!

> 
> (4) §5.5.3: s/central controller instructions...is done/central controller
> instructions...are done

Updated

> 
> (5) §5.5.8: "The PCC SHOULD allocate the Label and SHOULD report to the
> PCE
> using the PCRpt message."   When is it ok for the PCC to not allocate
> and/or
> report?  IOW, why are these actions only recommended and not required?
> Note that the very next paragraph requires the behavior.

Updated to "The PCC MUST try to allocate the Label and MUST report to the PCE 
via PCRpt or PCErr message."

> 
> (6) §7.3/§7.3.1:  In the out-label case, "it is mandatory to encode the
> next-hop information".  Should this information point at a directly
> connected IP address/interface, or can it point at a remote next-hop (which
> may be resolved through a routing protocol)?  What if the expected
> conditions are not met?


I propose to add this text - 

"The next-hop information encoded in the Address TLVs needs to be a directly 
connected IP address/interface information. If the PCC is not able to resolve 
the next-hop information, it MUST reject the CCI and respond with a PCErr 
message with Error-Type = TBD5 ("PCECC failure") and Error
Value = TBD18 ("Invalid next-hop information")."

> 
> 
Best regards, 
Shuping 
___
Pce mailing list
Pce@ietf.org
https://www.ietf.org/mailman/listinfo/pce

Re: [Pce] new draft on segment routing approach to TSN

[Yaakov Stein]

Tianran,

All valid points.

I do not doubt that there are RT cases with drop-dead deadlines,
but even they must live with rare losses (which is what a failure to meet 
deadline becomes).

As I said earlier the stack mechanism can replicate time gating behavior.
But time gating loses bandwidth efficiency and is really hard to scale up.

EDF in general will beat calendaring (i.e., has a lower expected latency),
but has a longer tail (which will be the aforementioned losses).
Given some statistics of packet transmission times one can calculate the 
percentage of packets
in that tail for a simple-to-setup approach with EDF.
It might be the case that some loss is better than not being able to calculate 
the schedule
and having to devote a lambda to each flow.

Y(J)S


From: Tianran Zhou 
Sent: 26/02/2021 10:20
To: Yaakov Stein 
Cc: det...@ietf.org; spr...@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN


CAUTION: External sender. Do not click links or open attachments unless you 
know the content is safe.
Hi Yaakov,

Thanks for your clarification.
Again I think this is an interesting work.
Please see inline.

Cheers,
Tianran

From: Yaakov Stein [mailto:yaako...@rad.com]
Sent: Friday, February 26, 2021 1:09 PM
To: Tianran Zhou mailto:zhoutian...@huawei.com>>
Cc: det...@ietf.org; 
spr...@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Tianran,

For some reason I didn't see your email and thus never responded to it. My 
apologies.


1.   I believe that I address this question at the beginning of my draft.

Using a single deadline per packet at is suboptimal at any particular switch.

For example, say two packets arrive at a switch, one with only 20 microseconds 
left on its deadline

and one with 50. The switch would naturally schedule the 10 microsecond one 
first.

But what the switch doesn't know is that it is the last hop for the 10 
microsecond one

and there are only 2 microseconds from it to destination,

while the 50 microsecond packet still has 45 microseconds of physical time 
ahead of it!



ZTR> Yes, you provided a valid use case in some sense.

But I have to say, IMO, the real-time system cares more about whether the 
deadline could meet.

So in practice, the task set is fixed, and all the latencies are calculated and 
guaranteed to meet the dead line. I do not care whether the packet could arrive 
earlier or not.

So, an deterministic and easy schedule make the evaluation/plan easier. And 
It's better to see all the tasks could be accommodated in the plan.



ZTR> I can provide an use case, say a task could meet the 20 end to end 
deadline within 2 hops, one hop for 14, and the other for 6.

If we give split hop by hop deadlines, 10 for each.  That means the first hop 
with 14 delay cannot meet the 10 local deadline.

So for the first schedule, this task could be accommodated. But for the second 
one, this task cannot be accommodated.



  1.  I gave a specific algorithm in the draft and pointed to Andrews and Zhang 
who give another one.

ZTR> Thanks for the pointer, I would like to look into them.


  1.  I don't think a service needs to support EDF, but perhaps a switch does.

I haven't seen EDF support since the ATM days, but am proposing that perhaps 
the time has come to re-evaluate it.

However, I am not advocating for EDF in this draft, just for the stack.

I mention a variant of EDF which I believe is better, and am working on another 
variant which is even better.

They can all be built into hardware, although admitted are more complex than a 
simple queue.

But they may be simpler than a set of queues with time schedules like Qbv

and are much much simpler than monstrosities like MEF 10.3 token bucket with 
cross-color/cross-CoS sharing and coupling.



ZTR> Not sure if my thoughts are common sense or not. :)

IMO, the merit of Qbv, or TAS, is to isolate hard real-time, soft real-time, 
best effort tasks. So that they can accommodate in one switch.

This is one critical requirement for IT/OT integration.

EDF does not conflict with TAS. I worked on some hierarchical scheduling 
mechanisms many years ago, EDF is used in the time slot for hard real-time 
tasks.


Y(J)S



From: detnet mailto:detnet-boun...@ietf.org>> On 
Behalf Of Tianran Zhou
Sent: jeudi 25 février 2021 9:14
To: Yaakov Stein mailto:yaako...@rad.com>>; 
det...@ietf.org; 
spr...@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Hi Yaakov,

This is an interesting topic.
After a quick review, there are several questions as follows:
1. It's clear to me to have a deadline for each packet. So that router can 
schedule the packet based on the urgency. But what's the motivation to split 
the end to end deadline to several local ones?
2. How to divide an end to end deadline into several 

Re: [Pce] [Detnet] new draft on segment routing approach to TSN

[Pascal Thubert (pthubert)]

Your RFC should be able to signal drop dead but I believe that it would not be 
the decision of an intermediate hop. Missing a deadline between b and c may be 
recoverable between c and d...


Regards,

Pascal

Le 26 févr. 2021 à 11:14, Yaakov Stein  a écrit :


Tianran,

All valid points.

I do not doubt that there are RT cases with drop-dead deadlines,
but even they must live with rare losses (which is what a failure to meet 
deadline becomes).

As I said earlier the stack mechanism can replicate time gating behavior.
But time gating loses bandwidth efficiency and is really hard to scale up.

EDF in general will beat calendaring (i.e., has a lower expected latency),
but has a longer tail (which will be the aforementioned losses).
Given some statistics of packet transmission times one can calculate the 
percentage of packets
in that tail for a simple-to-setup approach with EDF.
It might be the case that some loss is better than not being able to calculate 
the schedule
and having to devote a lambda to each flow.

Y(J)S


From: Tianran Zhou 
Sent: 26/02/2021 10:20
To: Yaakov Stein 
Cc: det...@ietf.org; spr...@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN


CAUTION: External sender. Do not click links or open attachments unless you 
know the content is safe.
Hi Yaakov,

Thanks for your clarification.
Again I think this is an interesting work.
Please see inline.

Cheers,
Tianran

From: Yaakov Stein [mailto:yaako...@rad.com]
Sent: Friday, February 26, 2021 1:09 PM
To: Tianran Zhou mailto:zhoutian...@huawei.com>>
Cc: det...@ietf.org; 
spr...@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Tianran,

For some reason I didn’t see your email and thus never responded to it. My 
apologies.


1.   I believe that I address this question at the beginning of my draft.

Using a single deadline per packet at is suboptimal at any particular switch.

For example, say two packets arrive at a switch, one with only 20 microseconds 
left on its deadline

and one with 50. The switch would naturally schedule the 10 microsecond one 
first.

But what the switch doesn’t know is that it is the last hop for the 10 
microsecond one

and there are only 2 microseconds from it to destination,

while the 50 microsecond packet still has 45 microseconds of physical time 
ahead of it!



ZTR> Yes, you provided a valid use case in some sense.

But I have to say, IMO, the real-time system cares more about whether the 
deadline could meet.

So in practice, the task set is fixed, and all the latencies are calculated and 
guaranteed to meet the dead line. I do not care whether the packet could arrive 
earlier or not.

So, an deterministic and easy schedule make the evaluation/plan easier. And 
It’s better to see all the tasks could be accommodated in the plan.



ZTR> I can provide an use case, say a task could meet the 20 end to end 
deadline within 2 hops, one hop for 14, and the other for 6.

If we give split hop by hop deadlines, 10 for each.  That means the first hop 
with 14 delay cannot meet the 10 local deadline.

So for the first schedule, this task could be accommodated. But for the second 
one, this task cannot be accommodated.



  1.  I gave a specific algorithm in the draft and pointed to Andrews and Zhang 
who give another one.

ZTR> Thanks for the pointer, I would like to look into them.


  1.  I don’t think a service needs to support EDF, but perhaps a switch does.

I haven’t seen EDF support since the ATM days, but am proposing that perhaps 
the time has come to re-evaluate it.

However, I am not advocating for EDF in this draft, just for the stack.

I mention a variant of EDF which I believe is better, and am working on another 
variant which is even better.

They can all be built into hardware, although admitted are more complex than a 
simple queue.

But they may be simpler than a set of queues with time schedules like Qbv

and are much much simpler than monstrosities like MEF 10.3 token bucket with 
cross-color/cross-CoS sharing and coupling.



ZTR> Not sure if my thoughts are common sense or not. ☺

IMO, the merit of Qbv, or TAS, is to isolate hard real-time, soft real-time, 
best effort tasks. So that they can accommodate in one switch.

This is one critical requirement for IT/OT integration.

EDF does not conflict with TAS. I worked on some hierarchical scheduling 
mechanisms many years ago, EDF is used in the time slot for hard real-time 
tasks.


Y(J)S



From: detnet mailto:detnet-boun...@ietf.org>> On 
Behalf Of Tianran Zhou
Sent: jeudi 25 février 2021 9:14
To: Yaakov Stein mailto:yaako...@rad.com>>; 
det...@ietf.org; 
spr...@ietf.org; pce@ietf.org
Subject: Re: [Detnet] new draft on segment routing approach to TSN

Hi Yaakov,

This is an interesting topic.
After a quick review, there are several questions as 

Re: [Pce] [Detnet] new draft on segment routing approach to TSN

[Tianran Zhou]

Yes, Pascal, that is exactly the case I mentioned.

Best,
Tianran






Sent from WeLink
发件人： Pascal Thubert (pthubert)mailto:pthub...@cisco.com>>
收件人： Yaakov Steinmailto:yaako...@rad.com>>
抄送： Tianran 
Zhoumailto:zhoutian...@huawei.com>>;springmailto:spr...@ietf.org>>;detnetmailto:det...@ietf.org>>;pcemailto:pce@ietf.org>>
主题： Re: [Detnet] new draft on segment routing approach to TSN
时间： 2021-02-26 18:58:40

Your RFC should be able to signal drop dead but I believe that it would not be 
the decision of an intermediate hop. Missing a deadline between b and c may be 
recoverable between c and d...


Regards,

Pascal

Le 26 févr. 2021 à 11:14, Yaakov Stein  a écrit :


Tianran,

All valid points.

I do not doubt that there are RT cases with drop-dead deadlines,
but even they must live with rare losses (which is what a failure to meet 
deadline becomes).

As I said earlier the stack mechanism can replicate time gating behavior.
But time gating loses bandwidth efficiency and is really hard to scale up.

EDF in general will beat calendaring (i.e., has a lower expected latency),
but has a longer tail (which will be the aforementioned losses).
Given some statistics of packet transmission times one can calculate the 
percentage of packets
in that tail for a simple-to-setup approach with EDF.
It might be the case that some loss is better than not being able to calculate 
the schedule
and having to devote a lambda to each flow.

Y(J)S


From: Tianran Zhou 
Sent: 26/02/2021 10:20
To: Yaakov Stein 
Cc: det...@ietf.org; spr...@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN



CAUTION: External sender. Do not click links or open attachments unless you 
know the content is safe.

Hi Yaakov,

Thanks for your clarification.
Again I think this is an interesting work.
Please see inline.

Cheers,
Tianran

From: Yaakov Stein [mailto:yaako...@rad.com]
Sent: Friday, February 26, 2021 1:09 PM
To: Tianran Zhou mailto:zhoutian...@huawei.com>>
Cc: det...@ietf.org; 
spr...@ietf.org; pce@ietf.org
Subject: RE: new draft on segment routing approach to TSN

Tianran,

For some reason I didn’t see your email and thus never responded to it. My 
apologies.


1.   I believe that I address this question at the beginning of my draft.

Using a single deadline per packet at is suboptimal at any particular switch.

For example, say two packets arrive at a switch, one with only 20 microseconds 
left on its deadline

and one with 50. The switch would naturally schedule the 10 microsecond one 
first.

But what the switch doesn’t know is that it is the last hop for the 10 
microsecond one

and there are only 2 microseconds from it to destination,

while the 50 microsecond packet still has 45 microseconds of physical time 
ahead of it!



ZTR> Yes, you provided a valid use case in some sense.

But I have to say, IMO, the real-time system cares more about whether the 
deadline could meet.

So in practice, the task set is fixed, and all the latencies are calculated and 
guaranteed to meet the dead line. I do not care whether the packet could arrive 
earlier or not.

So, an deterministic and easy schedule make the evaluation/plan easier. And 
It’s better to see all the tasks could be accommodated in the plan.



ZTR> I can provide an use case, say a task could meet the 20 end to end 
deadline within 2 hops, one hop for 14, and the other for 6.

If we give split hop by hop deadlines, 10 for each.  That means the first hop 
with 14 delay cannot meet the 10 local deadline.

So for the first schedule, this task could be accommodated. But for the second 
one, this task cannot be accommodated.



  1.  I gave a specific algorithm in the draft and pointed to Andrews and Zhang 
who give another one.

ZTR> Thanks for the pointer, I would like to look into them.


  1.  I don’t think a service needs to support EDF, but perhaps a switch does.

I haven’t seen EDF support since the ATM days, but am proposing that perhaps 
the time has come to re-evaluate it.

However, I am not advocating for EDF in this draft, just for the stack.

I mention a variant of EDF which I believe is better, and am working on another 
variant which is even better.

They can all be built into hardware, although admitted are more complex than a 
simple queue.

But they may be simpler than a set of queues with time schedules like Qbv

and are much much simpler than monstrosities like MEF 10.3 token bucket with 
cross-color/cross-CoS sharing and coupling.



ZTR> Not sure if my thoughts are common sense or not. ☺

IMO, the merit of Qbv, or TAS, is to isolate hard real-time, soft real-time, 
best effort tasks. So that they can accommodate in one switch.

This is one critical requirement for IT/OT integration.

EDF does not conflict with TAS. I worked on some hierarchical scheduling 
mechanisms many years ago, EDF is used in the time slot for hard real-time 
ta

Re: [Pce] new draft on segment routing approach to TSN

[Balázs Varga A]

Hi Yaakov,

many thanks for this well-written and information-rich draft.
I have enjoyed to read it (and also the discussions on the lists). :--)))


Conceptual (clarification) questions:
1, What is the node behavior if deadtime expires? Drop? That requires a quite 
good design/allocation of the latency budget. Otherwise it may result in 
unnecessary loss of a packet being relative late at a hop, however that might 
be compensated by the remaining hops. (E.g., see your Figure 2, getting 35 
microsecond at R1 instead of the designed 25, but R2,R3,R4 adds only 10-10-10 
extra microseconds). You may consider to add a "late flag" to the packet 
instead of a drop ...

2, Do I understand it correctly that the described solution is practically a 
new per-hop-behavior? Does it have no tools to "influence" the arrival time at 
the next hops? In end-to-end latency design the DetNet/TSN functions are often 
used to "adapt" the arrival of packets belonging to a DetNet flow/TSN stream at 
given network points. If "latency design" requires such a "time shift" then the 
described method has to be combined with other DetNet/TSN tools.

3, Design of a guaranteed upper bound usually requires deterministic arrival at 
each node along the path. In this solution the flow becomes less-an-less 
deterministic as it reaches the destination. I mean e.g., as per Figure 2, 
arrival time range at R1 is 2-2 usec, at R2 is 26-51 usec, at R3 is 70-120 usec 
and at R4 is 92-167 usec. So, the flow is getting less-and-less deterministic 
at each hop. That can make deterministic design of other flows passing R4 more 
difficult or even impossible in some scenarios. Have I missed something here?

4, Wire-speed timestamping and calculation: I would be interested in your view 
regarding how realistic are (1) the wire speed timestamping capability and (2) 
adding multiple calculated deadline to packets. Both are required for this 
solution. DetNet flows can have high bandwidth, not like 1588 packets.


Specific comments (Section 2-3):
5, Time gated Queuing [802.1Qbv] allows multiple gates to be open 
simultaneously. Furthermore preemption [802.1Qbu] can be combined with Qbv. So 
I cannot fully agree with your evaluation of time-aware scheduling. (Section 2, 
"However, time-aware scheduling suffers from two major disadvantages.") Of 
course with a bad design one may shoot in his/her own leg, but in well-designed 
scenarios efficiency loss can be eliminated without expensive computation.

6, Deadtime calculation is also complex if impact of other DetNet flows must be 
considered. The pre-calculation of individual "local" deadlines may need to be 
re-calculated when flows added to the network and they are using some common 
links. So I cannot agree with your statement. (Section 3, "... Since the 
ingress router inserts the deadline stack into the packet headers, no other 
router needs to be aware of the requirements of the time sensitive flows.  
Hence admitting a new flow only requires updating the information base of the 
ingress router."). Adding a flow may result in updating many ingress routers' 
configuration.


Some topics for further discussions/considerations:
7, Impact of some DetNet functions (e.g., PREOF [RFC8655]) needs further 
discussions. The usage of replication/elimination impacts the deadline 
calculation. Disjoint paths used by member flows requires different label stack 
and deadlines. Furthermore, these path specific labels+deadlines must be added 
by the replication node (PRF). So at least, a combination of PRF with B-SID and 
computing/adding deadlines (ingress SRTSN function) seems to be necessary in 
your solution.

8, The ever-green topic ( and not the green-wave :--))) ): label stack size. 
You need multiple labels per hop and the label stack contains them for each 
hop. Could result in a quite big label stack to be pushed at the ingress (nx10s 
of labels).


I am happy to have further discussions on this interesting idea

Thanks & Cheers
Bala'zs

From: detnet  On Behalf Of Yaakov Stein
Sent: Tuesday, February 23, 2021 2:14 PM
To: det...@ietf.org; spr...@ietf.org; pce@ietf.org
Subject: [Detnet] new draft on segment routing approach to TSN

All,

I would like to call your attention to a new ID 
https://www.ietf.org/archive/id/draft-stein-srtsn-00.txt
which describes using a stack-based approach (similar to segment routing) to 
time sensitive networking.
It furthermore proposes combining segment routing with this approach to TSN
resulting in a unified approach to forwarding and scheduling.

The draft is information at this point, since it discusses the concepts and 
does not yet pin down the precise formats.

Apologies for simultaneously sending to 3 lists,
but I am not sure which WG is the most appropriate for discussions of this 
topic.

  *   DetNet is most relevant since the whole point is to control end-to-end 
latency of a time-sensitive flow.
  *   Spring is also directly relevant due to the use of a stack in the hea

Re: [Pce] Alvaro Retana's No Objection on draft-ietf-pce-pcep-extension-for-pce-controller-12: (with COMMENT)

[Alvaro Retana]

Shuping:

Hi!

The changes look good to me.

Thanks!

Alvaro.

On February 26, 2021 at 5:10:41 AM, Pengshuping (Peng Shuping) (
pengshup...@huawei.com) wrote:

Thank you for your comments! Please find the diff and the responses in line
below. Thank you!
___
Pce mailing list
Pce@ietf.org
https://www.ietf.org/mailman/listinfo/pce