In addition to skip order function I think we need to handle the case when a packet gets fragmented (e.g. IP fragmentation). In this case the fragments should share the original sequence number and the order restoration logic should be aware when the last fragment occurs.
Alex On 3 November 2014 13:45, Bill Fischofer <bill.fischo...@linaro.org> wrote: > OK, thanks. That certainly matches the step-by-step semantics Petri > proposed. We'll have to discuss what, if anything, more we need regarding > ORDERED queues post-v1.0 next year. > > Bill > > On Mon, Nov 3, 2014 at 3:03 AM, Gilad Ben Yossef <gil...@ezchip.com> > wrote: > >> >> >> >> >> What NPS does in HW is that each buffer you dequeue actually stays on the >> queue (but is marked as consumed by the application) and you are given a >> "tag" for it. >> >> >> >> When I want to queue the buffer to the next, I give the HW the tag I got, >> not the buffer handle and what the HW does is only than fully dequeue the >> buffer from the original queue before queuing it into the new queue – but >> only after all preceding (older) buffers on the original queue have been >> fully dequeued, >> >> >> >> Free of a buffer causes a dequeue from the original queue as well. >> >> >> >> And I can also give release a buffer from the original queue (give the HW >> the tag back and say "you don't have to wait for this buffer any longer). >> >> >> >> I hope this helps, >> >> Gilad >> >> >> >> *Gilad Ben-Yossef* >> >> Software Architect >> >> EZchip Technologies Ltd. >> >> 37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel >> >> Tel: +972-4-959-6666 ext. 576, Fax: +972-8-681-1483 >> Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388 >> >> Email: gil...@ezchip.com, Web: http://www.ezchip.com >> >> >> >> *"Ethernet always wins."* >> >> — Andy Bechtolsheim >> >> >> >> *From:* Bill Fischofer [mailto:bill.fischo...@linaro.org] >> *Sent:* Sunday, November 02, 2014 5:44 PM >> *To:* Gilad Ben Yossef >> *Cc:* Savolainen, Petri (NSN - FI/Espoo); lng-odp@lists.linaro.org >> >> *Subject:* Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions >> >> >> >> Thanks Gilad. Can you elaborate on that a bit more? I can understand >> how step-by-step would be potentially easier to implement, but does it also >> capture the majority of expected application use cases? >> >> >> >> A good confirmation that this is the correct approach for v1.0 though. >> >> >> >> On Sun, Nov 2, 2014 at 9:26 AM, Gilad Ben Yossef <gil...@ezchip.com> >> wrote: >> >> >> >> For what it's worth, as a SoC vendor rep. that has ordered queue in HW, >> Petri's definition is actually preferred for us even going forward J >> >> >> >> >> >> Thanks, >> >> Gilad >> >> >> >> *Gilad Ben-Yossef* >> >> Software Architect >> >> EZchip Technologies Ltd. >> >> 37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel >> >> Tel: +972-4-959-6666 ext. 576, Fax: +972-8-681-1483 >> Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388 >> >> Email: gil...@ezchip.com, Web: http://www.ezchip.com >> >> >> >> *"Ethernet always wins."* >> >> — Andy Bechtolsheim >> >> >> >> *From:* lng-odp-boun...@lists.linaro.org [mailto: >> lng-odp-boun...@lists.linaro.org] *On Behalf Of *Bill Fischofer >> *Sent:* Friday, October 31, 2014 2:59 PM >> *To:* Savolainen, Petri (NSN - FI/Espoo) >> *Cc:* lng-odp@lists.linaro.org >> >> >> *Subject:* Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions >> >> >> >> This may well be a reasonable restriction for ODP v1.0 but I believe it's >> something we need to put on the list for "production grade" improvements >> for 2015. >> >> >> >> Bill >> >> >> >> On Fri, Oct 31, 2014 at 7:57 AM, Savolainen, Petri (NSN - FI/Espoo) < >> petri.savolai...@nsn.com> wrote: >> >> Yes, it’s step-by-step and I think it’s the level of ordering we need for >> v1.0. Most SoCs can implement it, even when the HW scheduler would not >> have order support but only atomic/parallel. This way defined atomic >> scheduling can be used to implement functionality correct ordered queues, >> the throughput is not improved but it functions correctly. >> >> >> >> -Petri >> >> >> >> >> >> *From:* ext Bill Fischofer [mailto:bill.fischo...@linaro.org] >> *Sent:* Friday, October 31, 2014 2:48 PM >> *To:* Alexandru Badicioiu >> *Cc:* Petri Savolainen; lng-odp@lists.linaro.org >> *Subject:* Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions >> >> >> >> I can well imagine the step-by-step order preservation to be simpler to >> implement (in SW) but it would also seem to limit performance since the >> only way to ensure end-to-end order preservation would be if each >> intermediate queue from ingress to egress were an ordered queue. If there >> is a parallel queue anywhere in the chain that would break things. >> >> >> >> The question is: Is this restriction needed and/or sufficient for ODP >> v1.0? >> >> >> >> On Fri, Oct 31, 2014 at 7:42 AM, Alexandru Badicioiu < >> alexandru.badici...@linaro.org> wrote: >> >> "+ * The original enqueue order of the source queue is maintained when >> buffers are >> + * enqueued to their destination queue(s) before another schedule call" >> >> >> >> Is this assuming that the order will be restored always at the next >> enqueue? I think there should be an option to explicitly indicate if the >> next enqueue is supposed to restore the order or not, especially when >> packets move from queue to queue. Ordered queues are costly compared with >> the ordinary ones. >> >> >> >> Alex >> >> >> >> On 31 October 2014 14:25, Petri Savolainen <petri.savolai...@linaro.org> >> wrote: >> >> Improved atomic and ordered synchronisation definitions. Added >> order skip function prototype. >> >> Signed-off-by: Petri Savolainen <petri.savolai...@linaro.org> >> >> --- >> This is the ordered queue definition (in patch format) promised >> in the call yesterday. >> --- >> platform/linux-generic/include/api/odp_queue.h | 31 +++++++++++++++- >> platform/linux-generic/include/api/odp_schedule.h | 45 >> ++++++++++++++++++----- >> 2 files changed, 64 insertions(+), 12 deletions(-) >> >> diff --git a/platform/linux-generic/include/api/odp_queue.h >> b/platform/linux-generic/include/api/odp_queue.h >> index b8ac4bb..c0c3969 100644 >> --- a/platform/linux-generic/include/api/odp_queue.h >> +++ b/platform/linux-generic/include/api/odp_queue.h >> @@ -78,8 +78,35 @@ typedef int odp_schedule_prio_t; >> typedef int odp_schedule_sync_t; >> >> #define ODP_SCHED_SYNC_NONE 0 /**< Queue not synchronised */ >> -#define ODP_SCHED_SYNC_ATOMIC 1 /**< Atomic queue */ >> -#define ODP_SCHED_SYNC_ORDERED 2 /**< Ordered queue */ >> + >> +/** >> + * Atomic queue synchronisation >> + * >> + * The scheduler gives buffers from a queue to a single core at a time. >> This >> + * serialises processing of the buffers from the source queue and helps >> user to >> + * avoid SW locking. Another schedule call will implicitely release the >> atomic >> + * synchronisation of the source queue and free the scheduler to give >> buffers >> + * from the queue to other cores. >> + * >> + * User can hint the scheduler to release the atomic synchronisation >> early with >> + * odp_schedule_release_atomic(). >> + */ >> +#define ODP_SCHED_SYNC_ATOMIC 1 >> + >> +/** >> + * Ordered queue synchronisation >> + * >> + * The scheduler may give out buffers to multiple cores for parallel >> processing. >> + * The original enqueue order of the source queue is maintained when >> buffers are >> + * enqueued to their destination queue(s) before another schedule call. >> Buffers >> + * from the same ordered (source) queue appear in their original order >> when >> + * dequeued from a destination queue. The destination queue type >> (POLL/SCHED) or >> + * synchronisation (NONE/ATOMIC/ORDERED) is not limited. >> + * >> + * User can command the scheduler to skip ordering of a buffer with >> + * odp_schedule_skip_order(). >> + */ >> +#define ODP_SCHED_SYNC_ORDERED 2 >> >> /** Default queue synchronisation */ >> #define ODP_SCHED_SYNC_DEFAULT ODP_SCHED_SYNC_ATOMIC >> diff --git a/platform/linux-generic/include/api/odp_schedule.h >> b/platform/linux-generic/include/api/odp_schedule.h >> index 91fec10..2a1a642 100644 >> --- a/platform/linux-generic/include/api/odp_schedule.h >> +++ b/platform/linux-generic/include/api/odp_schedule.h >> @@ -52,8 +52,8 @@ uint64_t odp_schedule_wait_time(uint64_t ns); >> * for a buffer according to the wait parameter setting. Returns >> * ODP_BUFFER_INVALID if reaches end of the wait period. >> * >> - * @param from Output parameter for the source queue (where the >> buffer was >> - * dequeued from). Ignored if NULL. >> + * @param src The source queue (output). Indicates from which queue >> the >> + * buffer was dequeued. Ignored if NULL. >> * @param wait Minimum time to wait for a buffer. Waits infinitely, >> if set to >> * ODP_SCHED_WAIT. Does not wait, if set to >> ODP_SCHED_NO_WAIT. >> * Use odp_schedule_wait_time() to convert time to other >> wait >> @@ -61,7 +61,7 @@ uint64_t odp_schedule_wait_time(uint64_t ns); >> * >> * @return Next highest priority buffer, or ODP_BUFFER_INVALID >> */ >> -odp_buffer_t odp_schedule(odp_queue_t *from, uint64_t wait); >> +odp_buffer_t odp_schedule(odp_queue_t *src, uint64_t wait); >> >> /** >> * Schedule one buffer >> @@ -76,8 +76,8 @@ odp_buffer_t odp_schedule(odp_queue_t *from, uint64_t >> wait); >> * >> * User can exit the schedule loop without first calling >> odp_schedule_pause(). >> * >> - * @param from Output parameter for the source queue (where the >> buffer was >> - * dequeued from). Ignored if NULL. >> + * @param src The source queue (output). Indicates from which queue >> the >> + * buffer was dequeued. Ignored if NULL. >> * @param wait Minimum time to wait for a buffer. Waits infinitely, >> if set to >> * ODP_SCHED_WAIT. Does not wait, if set to >> ODP_SCHED_NO_WAIT. >> * Use odp_schedule_wait_time() to convert time to other >> wait >> @@ -85,7 +85,7 @@ odp_buffer_t odp_schedule(odp_queue_t *from, uint64_t >> wait); >> * >> * @return Next highest priority buffer, or ODP_BUFFER_INVALID >> */ >> -odp_buffer_t odp_schedule_one(odp_queue_t *from, uint64_t wait); >> +odp_buffer_t odp_schedule_one(odp_queue_t *src, uint64_t wait); >> >> >> /** >> @@ -93,8 +93,8 @@ odp_buffer_t odp_schedule_one(odp_queue_t *from, >> uint64_t wait); >> * >> * Like odp_schedule(), but returns multiple buffers from a queue. >> * >> - * @param from Output parameter for the source queue (where the >> buffer was >> - * dequeued from). Ignored if NULL. >> + * @param src The source queue (output). Indicates from which queue >> the >> + * buffer was dequeued. Ignored if NULL. >> * @param wait Minimum time to wait for a buffer. Waits infinitely, >> if set to >> * ODP_SCHED_WAIT. Does not wait, if set to >> ODP_SCHED_NO_WAIT. >> * Use odp_schedule_wait_time() to convert time to other >> wait >> @@ -104,7 +104,7 @@ odp_buffer_t odp_schedule_one(odp_queue_t *from, >> uint64_t wait); >> * >> * @return Number of buffers outputed (0 ... num) >> */ >> -int odp_schedule_multi(odp_queue_t *from, uint64_t wait, odp_buffer_t >> out_buf[], >> +int odp_schedule_multi(odp_queue_t *src, uint64_t wait, odp_buffer_t >> out_buf[], >> unsigned int num); >> >> /** >> @@ -129,11 +129,36 @@ void odp_schedule_pause(void); >> void odp_schedule_resume(void); >> >> /** >> - * Release currently hold atomic context >> + * Release the current atomic context >> + * >> + * This calls is valid only when the source queue has ATOMIC >> synchronisation. It >> + * hints the scheduler that the user has completed processing the >> critical >> + * section that needs atomic synchronisation. After the call, the >> scheduler is >> + * allowed to give next buffer from the same queue to another core. >> + * >> + * Usage of the call may increase parallelism and thus system >> performance, but >> + * carefulness is needed when splitting processing into critical vs. >> + * non-critical sections. >> */ >> void odp_schedule_release_atomic(void); >> >> /** >> + * Skip ordering of the buffer >> + * >> + * This calls is valid only when source queue has ORDERED >> synchronisation. It >> + * commands the scheduler to skip ordering of the buffer. Scheduler >> maintains >> + * ordering between two queues: source and destination. The dest >> parameter >> + * identifies the destination queue of that pair. After the call, >> ordering is >> + * not maintained for the buffer anymore but user still owns the buffer. >> User >> + * can e.g. store it, free it or enqueue it (to the same or another >> queue). >> + * >> + * @param dest Destination queue >> + * @param buf Buffer >> + */ >> +void odp_schedule_skip_order(odp_queue_t dest, odp_buffer_t buf); >> + >> + >> +/** >> * Number of scheduling priorities >> * >> * @return Number of scheduling priorities >> -- >> 2.1.1 >> >> >> _______________________________________________ >> lng-odp mailing list >> lng-odp@lists.linaro.org >> http://lists.linaro.org/mailman/listinfo/lng-odp >> >> >> >> >> _______________________________________________ >> lng-odp mailing list >> lng-odp@lists.linaro.org >> http://lists.linaro.org/mailman/listinfo/lng-odp >> >> >> >> >> >> >> > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp > >
_______________________________________________ lng-odp mailing list lng-odp@lists.linaro.org http://lists.linaro.org/mailman/listinfo/lng-odp
