Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify use of 0 len on odp_packet_alloc()

[Savolainen, Petri (Nokia - FI/Espoo)]

The API does not specify "default length". Pool pkt parameters guide pool size 
(number of packet with min len) and segmentation (min bytes in the first 
segment). Implementation decides on segmentation and actual  segment sizes. 

pkt = odp_packet_alloc(pool, default_len);
odp_packet_reset(pkt, 0);

So, the example above does not actually specify anything. Any value of 
"default_len" is handled the same way. Implementation uses 'len' to allocate 
enough segments and initialize data pointers/lengths accordingly. 

// Allocated packet can hold at least 0 data bytes
// odp_packet_data() points to the first byte of data. Valid start pointer, 
although no data follows.
// odp_packet_len() returns 0
pkt1 = odp_packet_alloc(pool, 0);


// Allocated packet can hold at least 1 data bytes
// odp_packet_data() points to the first byte of data
// odp_packet_len() returns 1
pkt2 = odp_packet_alloc(pool, 1);

...


Which API definition was source of the confusion? It should be enough to 
highlight that either zero is legal or illegal value of 'len'. 


-Petri


> -Original Message-
> From: ext Zoltan Kiss [mailto:zoltan.k...@linaro.org]
> Sent: Thursday, May 28, 2015 4:59 PM
> To: Savolainen, Petri (Nokia - FI/Espoo); ext Bill Fischofer
> Cc: lng-odp@lists.linaro.org
> Subject: Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify use
> of 0 len on odp_packet_alloc()
> 
> 
> 
> On 28/05/15 13:58, Savolainen, Petri (Nokia - FI/Espoo) wrote:
> > How people think 0 is special or different from e.g. value 1?
> 
> Well, 0 is a special number in many ways, not just in mathematics and
> computer science:
> 
> http://en.wikipedia.org/wiki/0_%28number%29
> 
> It is special in this case as well, because it's defined to be equal to:
> 
> pkt = odp_packet_alloc(pool, default_len);
> odp_packet_reset(pkt, 0);
> 
> Which is not necessarily what you would expect. Most people had
> different ideas what 0 should mean.
> 
> >
> > We can change documentation wording, but not add lengthy documentation
> for value 0 as a special case (which is not).
> >
> > -Petri
> >
> >
> >> -Original Message-
> >> From: ext Zoltan Kiss [mailto:zoltan.k...@linaro.org]
> >> Sent: Thursday, May 28, 2015 3:51 PM
> >> To: Savolainen, Petri (Nokia - FI/Espoo); ext Bill Fischofer
> >> Cc: lng-odp@lists.linaro.org
> >> Subject: Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify
> use
> >> of 0 len on odp_packet_alloc()
> >>
> >> I don't think so. There was a lengthy discussion on the arch meeting
> how
> >> should you interpret that, I think that proves it's worth to clarify
> >> what's the expected behaviour.
> >>
> >> On 28/05/15 13:45, Savolainen, Petri (Nokia - FI/Espoo) wrote:
> >>> I think it should be enough to note that len can be 0. There’s no
> >>> special handling for value zero.
> >>>
> >>> -Petri
> >>>
> >>> *From:*ext Bill Fischofer [mailto:bill.fischo...@linaro.org]
> >>> *Sent:* Thursday, May 28, 2015 2:48 PM
> >>> *To:* Savolainen, Petri (Nokia - FI/Espoo)
> >>> *Cc:* lng-odp@lists.linaro.org
> >>> *Subject:* Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet:
> clarify
> >>> use of 0 len on odp_packet_alloc()
> >>>
> >>> The purpose of the note was to expand on the case of len = 0.  Are you
> >>> saying this is not needed?  The consensus on the Wednesday arch call
> was
> >>> that it was, hence this patch.
> >>>
> >>> On Thu, May 28, 2015 at 5:40 AM, Savolainen, Petri (Nokia - FI/Espoo)
> >>> mailto:petri.savolai...@nokia.com>>
> wrote:
> >>>
> >>>
> >>>* ... The
> >>>* packet is initialized with data pointers and lengths set
> according
> >>> to the
> >>>* specified len, ...
> >>>
> >>> The current documentation covers functionality also when len is 0. We
> >>> should minimize @note content over all the APIs, otherwise the actual
> >>> API spec gets fragmented. API documentation and functionality is the
> >>> same for len == 0, len == 1, ... In all the cases, implementation
> >>> decides on packet segmentation in limits of pool parameters.
> >>>
> >>> If needed, we could add:
> >>>
> >>> @note Zero is a valid 'len' value
> >>>
> >>>
> >>> -Petri
> >>>
> >>>
> >>>
> >>>   > -Original Message-
> >>>   > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org
> >>> ] On Behalf Of ext
> >>>   > Bill Fischofer
> >>>   > Sent: Wednesday, May 27, 2015 6:51 PM
> >>>   > To: lng-odp@lists.linaro.org 
> >>>   > Subject: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify
> use
> >>> of 0
> >>>   > len on odp_packet_alloc()
> >>>   >
> >>>   > Signed-off-by: Bill Fischofer  >>> >
> >>>   > ---
> >>>   >  include/odp/api/packet.h | 10 ++
> >>>   >  1 file changed, 10 insertions(+)
> >>>   >
> >>>   > diff --git a/include/odp/api/packet.h b/include/odp/api/packet.h
> >>>   > index 3a454b5..ea124df 100644
> >>>   > --- a/include/odp/api/packet.h
> >>>   > +++ b/include/odp/api/packet.h

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Savolainen, Petri (Nokia - FI/Espoo)]

Agree with Ola. Completed TX ring resources can be freed in the next 
odp_pktio_send/odp_pktio_recv/odp_schedule/etc calls. Application should not 
care how and when those resources are freed. It would be awkward if 
implementation requires application to call the flush function e.g. every 1ms, 
or otherwise some send calls may start failing, etc.

-Petri


From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of ext Ola 
Liljedahl
Sent: Thursday, May 28, 2015 7:41 PM
To: Zoltan Kiss
Cc: LNG ODP Mailman List
Subject: Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add 
odp_pktio_send_complete() definition

On 28 May 2015 at 17:23, Zoltan Kiss 
mailto:zoltan.k...@linaro.org>> wrote:


On 28/05/15 16:00, Ola Liljedahl wrote:
I disprove of this solution. TX completion processing (cleaning TX
descriptor rings after transmission complete) is an implementation
(hardware) aspect and should be hidden from the application.

Unfortunately you can't, if you want your pktio application work with poll mode 
drivers. In that case TX completion interrupt (can be) disabled and the 
application has to control that as well. In case of DPDK you just call the send 
function (with 0 packets, if you don't have anything to send at the time)
Why do you have to retire transmitted packet if you are not transmitting new 
packets (and need those descriptors in the TX ring)? Does the application have 
too few packets in the pool so that reception will suffer?



 There isn't
any corresponding call that refills the RX descriptor rings with fresh
buffers.
You can do that in the receive function, I think that's how the drivers are 
doing it generally.

The completion processing can be performed from any ODP call, not
necessary odp_pktio_send().

I think "any" is not specific enough. Which one?
odp_pktio_recv, odp_schedule. Wherever the application blocks or busy waits 
waiting for more packets.



Can you provide a vague draft how would you fix the l2fwd example below?
I don't think anything needs fixing on the application level.


-- Ola


On 28 May 2015 at 16:38, Zoltan Kiss 
mailto:zoltan.k...@linaro.org>
>> wrote:

A pktio interface can be used with poll mode drivers, where TX
completion often
has to be done manually. This turned up as a problem with ODP-DPDK and
odp_l2fwd:

while (!exit_threads) {
 pkts = odp_pktio_recv(pktio_src,...);
 if (pkts <= 0)
 continue;
...
 if (pkts_ok > 0)
 odp_pktio_send(pktio_dst, pkt_tbl, pkts_ok);
...
}

In this example we never call odp_pktio_send() on pktio_dst if there
wasn't
any new packets received on pktio_src. DPDK needs manual TX
completion. The
above example should have an odp_pktio_send_completion(pktio_dst)
right at the
beginning of the loop.

Signed-off-by: Zoltan Kiss 
mailto:zoltan.k...@linaro.org>
>>

---
  include/odp/api/packet_io.h | 16 
  1 file changed, 16 insertions(+)

diff --git a/include/odp/api/packet_io.h b/include/odp/api/packet_io.h
index b97b2b8..3a4054c 100644
--- a/include/odp/api/packet_io.h
+++ b/include/odp/api/packet_io.h
@@ -119,6 +119,22 @@ int odp_pktio_recv(odp_pktio_t pktio,
odp_packet_t pkt_table[], int len);
  int odp_pktio_send(odp_pktio_t pktio, odp_packet_t pkt_table[],
int len);

  /**
+ * Release sent packets
+ *
+ * This function should be called after sending on a pktio. If the
platform
+ * doesn't implement send completion in other ways, this function
should call
+ * odp_packet_free() on packets where transmission is already
completed. It can
+ * be a no-op if the platform guarantees that the packets will be
released upon
+ * completion, but the application must call it periodically after
send to make
+ * sure packets are released.
+ *
+ * @param pktioODP packet IO handle
+ *
+ * @retval <0 on failure
+ */
+int odp_pktio_send_complete(odp_pktio_t pktio);
+
+/**
   * Set the default input queue to be associated with a pktio handle
   *
   * @param pktioODP packet IO handle
--
1.9.1

___
lng-odp mailing list
lng-odp@lists.linaro.org 
>
https://lists.linaro.org/mailman/listinfo/lng-odp


___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv2] apply-and-build.sh: fixing cover-letter issue

[Christophe Milard]

When an empty patch (such as a cover letter) was given to
apply-and-build, the latter would fail applying the patch and all
following patches (saying OK when it could be wrong)

Signed-off-by: Christophe Milard 
---
 apply-and-build.sh | 10 --
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/apply-and-build.sh b/apply-and-build.sh
index 7195f40..d4cc0d9 100755
--- a/apply-and-build.sh
+++ b/apply-and-build.sh
@@ -147,18 +147,16 @@ apply_patch() {
 
 echo "  Trying to apply patch"
 git am ${patch} >> ${logfile_basename}-am.log 2>&1
-egrep "^Patch failed at" ${logfile_basename}-am.log > /dev/null
-if [ $? -eq 0 ]; then
+if [ $? -ne 0 ]; then
 cat ${logfile_basename}-am.log
-git am --abort || exit 1
+git am --abort
 git checkout . || exit 1
 git clean -xdf || exit 1
 git am --3way ${patch} >> ${logfile_basename}-am-3way.log 2>&1
-egrep "^Patch failed at" ${logfile_basename}-am-3way.log > /dev/null
-if [ $? -eq 0 ]; then
+if [ $? -ne 0 ]; then
 cat ${logfile_basename}-am-3way.log
 echo "  Error: Patch failed to apply"
-git am --abort || exit 1
+git am --abort
 git checkout . || exit 1
 git clean -xdf || exit 1
 popd > /dev/null
-- 
1.9.1

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv2] apply-and-build.sh: cosmetic fix

[Christophe Milard]

function apply_patch() now returns a status rather than directely breaking the
loop of its caller. (for loop in main, continue statment in apply_patch())

Signed-off-by: Christophe Milard 
---
 apply-and-build.sh | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/apply-and-build.sh b/apply-and-build.sh
index d4cc0d9..d81eff5 100755
--- a/apply-and-build.sh
+++ b/apply-and-build.sh
@@ -160,7 +160,7 @@ apply_patch() {
 git checkout . || exit 1
 git clean -xdf || exit 1
 popd > /dev/null
-continue
+return 1
 else
 echo "  Warning: git am --3way, applied"
 fi
@@ -173,6 +173,7 @@ apply_patch() {
 fi
 
 popd > /dev/null
+return 0
 }
 
 format_patch() {
@@ -238,8 +239,12 @@ FILE_EXT_REGEX=".*\.\("$(echo ${FILE_EXT}|sed -e 's/[ 
,]/\\|/g')"\)$"
 for patch in $(find $PATCH_DIR -type f -iregex "${FILE_EXT_REGEX}" |sort -h); 
do
 BUILD_FAILURE=0
 CURRENT_LOG="${ODP_LOGDIR}/$(echo $(basename ${patch}))"
+
 echo -e "\n\nUsing patch: $(basename ${patch})"
 apply_patch ${ODP_BUILDDIR} ${CURRENT_LOG}
+if [ $? -ne 0 ]; then
+continue#go directely to next pacth if patch application failed
+fi
 
 format_patch ${ODP_BUILDDIR} ${CURRENT_LOG}
 
-- 
1.9.1

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCH v2 0/4] Init API and thread type

[Savolainen, Petri (Nokia - FI/Espoo)]

Ping.

> -Original Message-
> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of ext
> Petri Savolainen
> Sent: Friday, May 08, 2015 10:07 AM
> To: lng-odp@lists.linaro.org
> Subject: [lng-odp] [API-NEXT PATCH v2 0/4] Init API and thread type
> 
> Introduce thread types: worker and control. Implementations may be HW
> limited
> how many worker threads can be supported. Number of control threads may be
> only
> SW limited.
> 
> Note: This series depends on "example: classifier: remove extra local
> init"
> patch.
> 
> v2:
>  * Rebased
>  * Clean up doxygen documentation in init.h
>  * Changed odp_init_global params to const pointer types
> 
> 
> Petri Savolainen (4):
>   style: init api: documentation clean up
>   api: init: use const pointer types
>   api: thread: added thread type
>   api: init: added thread type to local init
> 
>  example/classifier/odp_classifier.c   |  2 +-
>  example/generator/odp_generator.c |  2 +-
>  example/ipsec/odp_ipsec.c |  2 +-
>  example/packet/odp_pktio.c|  2 +-
>  example/timer/odp_timer_test.c|  2 +-
>  helper/linux.c|  4 +-
>  include/odp/api/init.h| 98 --
> -
>  include/odp/api/thread.h  | 38 
>  platform/linux-generic/odp_init.c |  6 +-
>  test/performance/odp_l2fwd.c  |  2 +-
>  test/performance/odp_pktio_perf.c |  2 +-
>  test/performance/odp_scheduling.c |  2 +-
>  test/validation/common/odp_cunit_common.c |  2 +-
>  13 files changed, 104 insertions(+), 60 deletions(-)
> 
> --
> 2.4.0
> 
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Zoltan Kiss]

On 28/05/15 17:40, Ola Liljedahl wrote:

On 28 May 2015 at 17:23, Zoltan Kiss mailto:zoltan.k...@linaro.org>> wrote:



On 28/05/15 16:00, Ola Liljedahl wrote:

I disprove of this solution. TX completion processing (cleaning TX
descriptor rings after transmission complete) is an implementation
(hardware) aspect and should be hidden from the application.


Unfortunately you can't, if you want your pktio application work
with poll mode drivers. In that case TX completion interrupt (can
be) disabled and the application has to control that as well. In
case of DPDK you just call the send function (with 0 packets, if you
don't have anything to send at the time)

Why do you have to retire transmitted packet if you are not transmitting
new packets (and need those descriptors in the TX ring)?
Because otherwise they are a memory leak. Those buffers might be needed 
somewhere else. If they are only released when you send/receive packets 
out next time, you are in trouble, because that might never happen. 
Especially when that event is blocked because your TX ring is full of 
unreleased packets.


 Does the

application have too few packets in the pool so that reception will suffer?
Let me approach the problem from a different angle: the current 
workaround is that you have to allocate a pool with _loooads_ of 
buffers, so you have a good chance you never run out of free buffers. 
Probably. Because it still doesn't guarantee that there will be a next 
send/receive event on that interface to release the packets.







  There isn't

any corresponding call that refills the RX descriptor rings with
fresh
buffers.

You can do that in the receive function, I think that's how the
drivers are doing it generally.


The completion processing can be performed from any ODP call, not
necessary odp_pktio_send().


I think "any" is not specific enough. Which one?

odp_pktio_recv, odp_schedule. Wherever the application blocks or busy
waits waiting for more packets.
We do that already on odp_pktio_recv. It doesn't help, because you can 
only release the buffers held in the current interface's TX ring. You 
can't do anything about other interfaces.
I mean, you could trigger TX completion on every interface every time 
you receive on one, but that would be a scalability nightmare.






Can you provide a vague draft how would you fix the l2fwd example below?

I don't think anything needs fixing on the application level.


Wrong. odp_l2fwd uses one packet pool, receives from pktio_src and then 
if there is anything received, it sends it out on pktio_dst. Let's say 
the pool has 576 elements, and the interfaces uses 256 RX and 256 TX 
descriptors. You start with 2*256 buffers kept in the two RX ring. Let's 
say you receive the first 64 packets, you refill the RX ring 
immediately, so now you're out of buffers. You can send out that 64, but 
in the next iteration odp_pktio_recv() will return 0 because it can't 
refill the RX descriptors. (and the driver won't give you back any 
buffer unless you can refill it). And now you are in an infinite loop, 
recv will always return 0, because you never release the packets.

There are several ways to fix this:
- tell the application writer that if you see deadlocks, increase the 
element size of the buffer. I doubt anyone would ever use ODP to 
anything serious when seeing such thing.
- you can't really give anything more specific than in the previous 
point, because such details as RX/TX descriptor numbers are abstracted 
away, intentionally. And your platform can't autotune them, because it 
doesn't know how many elements you have in the pool used for TX. In 
fact, it could be more than just one pool.
- make sure that you run odp_pktio_send even if pkts == 0. In case of 
ODP-DPDK it can help because that actually triggers TX completion. 
Actually, we can make odp_pktio_send_complete() == 
odp_pktio_send(len=0), so we don't have to introduce a new function. But 
that doesn't change the fact that we have to call TX completion 
periodically to make sure nothing is blocked.
- or we can just do what I proposed in the patch, which is very similar 
to the previous point, but articulate the importance of TX completion more.






-- Ola


On 28 May 2015 at 16:38, Zoltan Kiss mailto:zoltan.k...@linaro.org>
>>
wrote:

 A pktio interface can be used with poll mode drivers, where TX
 completion often
 has to be done manually. This turned up as a problem with
ODP-DPDK and
 odp_l2fwd:

 while (!exit_threads) {
  pkts = odp_pktio_recv(pktio_src,...);
  if (pkts <= 0)
  continue;
 ...
  if (pkts_ok > 0)
  odp_pktio_sen

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Zoltan Kiss]

On 29/05/15 11:27, Savolainen, Petri (Nokia - FI/Espoo) wrote:

Agree with Ola. Completed TX ring resources can be freed in the next
odp_pktio_send/odp_pktio_recv/odp_schedule/etc calls.
I've explained it in my previous reply to Ola, but that wouldn't be good 
enough: receive and schedule can't do anything with packets on sent out 
on other interfaces. odp_pktio_send could be a solution, but then you 
have to make sure that you call it in a finite time, even if there is 
nothing to send. That wouldn't differ too much from what I proposed, 
only that we won't have a separate function call for TX completion, but 
odp_pktio_send(len=0)



 Application should

not care how and when those resources are freed.
ODP API is defined in C, so we don't have a garbage collector, and the 
whole point of poll mode applications is to avoid interrupts, so we 
won't have TX completion interrupts as well. That leaves the application 
to make sure TX completion happens in a finite time.



 It would be awkward if

implementation requires application to call the flush function e.g.
every 1ms, or otherwise some send calls may start failing, etc.
It's not about send fails, it's that without making sure TX completion 
happens after a finite period of time, no matter what, you are in the 
danger that your application blocks somewhere else because of those 
buffers are missing. You can mitigate that problem by providing plenty 
of buffers, and making sure different users will limit their usage so 
the pool is never exhausted, but this second part is probably much 
harder than you think: the app don't know how many RX/TX descriptors are 
used in the platform, and the platform doesn't know how many packets are 
in the pools where the packets coming from, and when is the time to give 
them back because that pool is exhausted.

Again, see that reply to Ola's mail for more details.



-Petri

*From:*lng-odp [mailto:lng-odp-boun...@lists.linaro.org] *On Behalf Of
*ext Ola Liljedahl
*Sent:* Thursday, May 28, 2015 7:41 PM
*To:* Zoltan Kiss
*Cc:* LNG ODP Mailman List
*Subject:* Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add
odp_pktio_send_complete() definition

On 28 May 2015 at 17:23, Zoltan Kiss mailto:zoltan.k...@linaro.org>> wrote:



On 28/05/15 16:00, Ola Liljedahl wrote:

I disprove of this solution. TX completion processing (cleaning TX
descriptor rings after transmission complete) is an implementation
(hardware) aspect and should be hidden from the application.


Unfortunately you can't, if you want your pktio application work with
poll mode drivers. In that case TX completion interrupt (can be)
disabled and the application has to control that as well. In case of
DPDK you just call the send function (with 0 packets, if you don't have
anything to send at the time)

Why do you have to retire transmitted packet if you are not transmitting
new packets (and need those descriptors in the TX ring)? Does the
application have too few packets in the pool so that reception will suffer?



  There isn't

any corresponding call that refills the RX descriptor rings with fresh
buffers.

You can do that in the receive function, I think that's how the
drivers are doing it generally.


The completion processing can be performed from any ODP call, not
necessary odp_pktio_send().


I think "any" is not specific enough. Which one?

odp_pktio_recv, odp_schedule. Wherever the application blocks or busy
waits waiting for more packets.


Can you provide a vague draft how would you fix the l2fwd example below?

I don't think anything needs fixing on the application level.


-- Ola


On 28 May 2015 at 16:38, Zoltan Kiss mailto:zoltan.k...@linaro.org>
>>
wrote:

 A pktio interface can be used with poll mode drivers, where TX
 completion often
 has to be done manually. This turned up as a problem with
ODP-DPDK and
 odp_l2fwd:

 while (!exit_threads) {
  pkts = odp_pktio_recv(pktio_src,...);
  if (pkts <= 0)
  continue;
 ...
  if (pkts_ok > 0)
  odp_pktio_send(pktio_dst, pkt_tbl, pkts_ok);
 ...
 }

 In this example we never call odp_pktio_send() on pktio_dst
if there
 wasn't
 any new packets received on pktio_src. DPDK needs manual TX
 completion. The
 above example should have an
odp_pktio_send_completion(pktio_dst)
 right at the
 beginning of the loop.

 Signed-off-by: Zoltan Kiss mailto:zoltan.k...@linaro.org>
 >>


 ---
   include/odp/api/packet_io.h | 16 
   1 fil

Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify use of 0 len on odp_packet_alloc()

[Bill Fischofer]

The issue has to do with odp_packet_reset().  If I say:

pkt  = odp_packet_alloc(pool, 1500);

I can then subsequently call:

odp_packet_reset(pkt, n);

for any value of N <= 1500.  If I try to reset a packet to a length greater
than was initially allocated for it, that is an error.  If we followed that
definition then
allocating a packet of length 0 would be pointless since it could never be
grown to anything useful.

The purpose of the note regarding the use of 0 is to have a convenient
means of allocating a packet of implementation-default useful but unknown
length and then be able to build packets from there via
odp_packet_push_tail() calls.

On Fri, May 29, 2015 at 5:06 AM, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia.com> wrote:

> The API does not specify "default length". Pool pkt parameters guide pool
> size (number of packet with min len) and segmentation (min bytes in the
> first segment). Implementation decides on segmentation and actual  segment
> sizes.
>
> pkt = odp_packet_alloc(pool, default_len);
> odp_packet_reset(pkt, 0);
>
> So, the example above does not actually specify anything. Any value of
> "default_len" is handled the same way. Implementation uses 'len' to
> allocate enough segments and initialize data pointers/lengths accordingly.
>
> // Allocated packet can hold at least 0 data bytes
> // odp_packet_data() points to the first byte of data. Valid start
> pointer, although no data follows.
> // odp_packet_len() returns 0
> pkt1 = odp_packet_alloc(pool, 0);
>
>
> // Allocated packet can hold at least 1 data bytes
> // odp_packet_data() points to the first byte of data
> // odp_packet_len() returns 1
> pkt2 = odp_packet_alloc(pool, 1);
>
> ...
>
>
> Which API definition was source of the confusion? It should be enough to
> highlight that either zero is legal or illegal value of 'len'.
>
>
> -Petri
>
>
> > -Original Message-
> > From: ext Zoltan Kiss [mailto:zoltan.k...@linaro.org]
> > Sent: Thursday, May 28, 2015 4:59 PM
> > To: Savolainen, Petri (Nokia - FI/Espoo); ext Bill Fischofer
> > Cc: lng-odp@lists.linaro.org
> > Subject: Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify use
> > of 0 len on odp_packet_alloc()
> >
> >
> >
> > On 28/05/15 13:58, Savolainen, Petri (Nokia - FI/Espoo) wrote:
> > > How people think 0 is special or different from e.g. value 1?
> >
> > Well, 0 is a special number in many ways, not just in mathematics and
> > computer science:
> >
> > http://en.wikipedia.org/wiki/0_%28number%29
> >
> > It is special in this case as well, because it's defined to be equal to:
> >
> > pkt = odp_packet_alloc(pool, default_len);
> > odp_packet_reset(pkt, 0);
> >
> > Which is not necessarily what you would expect. Most people had
> > different ideas what 0 should mean.
> >
> > >
> > > We can change documentation wording, but not add lengthy documentation
> > for value 0 as a special case (which is not).
> > >
> > > -Petri
> > >
> > >
> > >> -Original Message-
> > >> From: ext Zoltan Kiss [mailto:zoltan.k...@linaro.org]
> > >> Sent: Thursday, May 28, 2015 3:51 PM
> > >> To: Savolainen, Petri (Nokia - FI/Espoo); ext Bill Fischofer
> > >> Cc: lng-odp@lists.linaro.org
> > >> Subject: Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify
> > use
> > >> of 0 len on odp_packet_alloc()
> > >>
> > >> I don't think so. There was a lengthy discussion on the arch meeting
> > how
> > >> should you interpret that, I think that proves it's worth to clarify
> > >> what's the expected behaviour.
> > >>
> > >> On 28/05/15 13:45, Savolainen, Petri (Nokia - FI/Espoo) wrote:
> > >>> I think it should be enough to note that len can be 0. There’s no
> > >>> special handling for value zero.
> > >>>
> > >>> -Petri
> > >>>
> > >>> *From:*ext Bill Fischofer [mailto:bill.fischo...@linaro.org]
> > >>> *Sent:* Thursday, May 28, 2015 2:48 PM
> > >>> *To:* Savolainen, Petri (Nokia - FI/Espoo)
> > >>> *Cc:* lng-odp@lists.linaro.org
> > >>> *Subject:* Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet:
> > clarify
> > >>> use of 0 len on odp_packet_alloc()
> > >>>
> > >>> The purpose of the note was to expand on the case of len = 0.  Are
> you
> > >>> saying this is not needed?  The consensus on the Wednesday arch call
> > was
> > >>> that it was, hence this patch.
> > >>>
> > >>> On Thu, May 28, 2015 at 5:40 AM, Savolainen, Petri (Nokia - FI/Espoo)
> > >>> mailto:petri.savolai...@nokia.com>>
> > wrote:
> > >>>
> > >>>
> > >>>* ... The
> > >>>* packet is initialized with data pointers and lengths set
> > according
> > >>> to the
> > >>>* specified len, ...
> > >>>
> > >>> The current documentation covers functionality also when len is 0. We
> > >>> should minimize @note content over all the APIs, otherwise the actual
> > >>> API spec gets fragmented. API documentation and functionality is the
> > >>> same for len == 0, len == 1, ... In all the cases, implementation
> > >>> decides on packet segmentation in limits of pool parameters.
> > >>>
>

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Savolainen, Petri (Nokia - FI/Espoo)]

> -Original Message-
> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of ext
> Zoltan Kiss
> Sent: Friday, May 29, 2015 2:56 PM
> To: Ola Liljedahl
> Cc: LNG ODP Mailman List
> Subject: Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add
> odp_pktio_send_complete() definition
> 
> 
> 
> On 28/05/15 17:40, Ola Liljedahl wrote:
> > On 28 May 2015 at 17:23, Zoltan Kiss  > > wrote:
> >
> >
> >
> > On 28/05/15 16:00, Ola Liljedahl wrote:
> >
> > I disprove of this solution. TX completion processing (cleaning
> TX
> > descriptor rings after transmission complete) is an
> implementation
> > (hardware) aspect and should be hidden from the application.
> >
> >
> > Unfortunately you can't, if you want your pktio application work
> > with poll mode drivers. In that case TX completion interrupt (can
> > be) disabled and the application has to control that as well. In
> > case of DPDK you just call the send function (with 0 packets, if you
> > don't have anything to send at the time)
> >
> > Why do you have to retire transmitted packet if you are not transmitting
> > new packets (and need those descriptors in the TX ring)?
> Because otherwise they are a memory leak. Those buffers might be needed
> somewhere else. If they are only released when you send/receive packets
> out next time, you are in trouble, because that might never happen.
> Especially when that event is blocked because your TX ring is full of
> unreleased packets.
> 
>   Does the
> > application have too few packets in the pool so that reception will
> suffer?
> Let me approach the problem from a different angle: the current
> workaround is that you have to allocate a pool with _loooads_ of
> buffers, so you have a good chance you never run out of free buffers.
> Probably. Because it still doesn't guarantee that there will be a next
> send/receive event on that interface to release the packets.

I guess CPUs can always burst packets so fast that the TX ring gets full. So, 
you should design the pool/ring configuration/init so that "full ring" is part 
of normal operation. What is the benefit of configuring so large ring that it 
cannot be filled to the max? The pools size needs to be RX + TX ring size + 
number of in-flight packets.

-Petri





___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify use of 0 len on odp_packet_alloc()

[Savolainen, Petri (Nokia - FI/Espoo)]

/**
* Reset packet
*
* Resets all packet metadata to their default values. Packet length is used
* to initialize pointers and lengths. It must be less than the total buffer
* length of the packet minus the default headroom length. Packet is not
* modified on failure.
*
* @param pkt   Packet handle
* @param len   Packet data length
*
* @retval 0 on success
* @retval <0 on failure
*
* @see odp_packet_buf_len()
*/
int odp_packet_reset(odp_packet_t pkt, uint32_t len);


/**
* Total packet buffer length
*
* Returns sum of buffer lengths over all packet segments.
*
* @param pkt  Packet handle
*
* @return  Total packet buffer length in bytes
*
* @see odp_packet_reset()
*/
uint32_t odp_packet_buf_len(odp_packet_t pkt);


pkt  = odp_packet_alloc(pool, 0);

…

buf_len = odp_packet_buf_len(pkt);
odp_packet_reset(pkt, buf_len - ODP_CONFIG_PACKET_HEADROOM - x);

Reset does not refer to initial packet data length, but the current maximum 
data length (total buffer length). The idea is to reuse a packet, if current 
segmentation allows it.

-Petri



From: ext Bill Fischofer [mailto:bill.fischo...@linaro.org]
Sent: Friday, May 29, 2015 3:32 PM
To: Savolainen, Petri (Nokia - FI/Espoo)
Cc: ext Zoltan Kiss; lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify use of 0 
len on odp_packet_alloc()

The issue has to do with odp_packet_reset().  If I say:

pkt  = odp_packet_alloc(pool, 1500);

I can then subsequently call:

odp_packet_reset(pkt, n);

for any value of N <= 1500.  If I try to reset a packet to a length greater 
than was initially allocated for it, that is an error.  If we followed that 
definition then
allocating a packet of length 0 would be pointless since it could never be 
grown to anything useful.

The purpose of the note regarding the use of 0 is to have a convenient means of 
allocating a packet of implementation-default useful but unknown length and 
then be able to build packets from there via odp_packet_push_tail() calls.

On Fri, May 29, 2015 at 5:06 AM, Savolainen, Petri (Nokia - FI/Espoo) 
mailto:petri.savolai...@nokia.com>> wrote:
The API does not specify "default length". Pool pkt parameters guide pool size 
(number of packet with min len) and segmentation (min bytes in the first 
segment). Implementation decides on segmentation and actual  segment sizes.

pkt = odp_packet_alloc(pool, default_len);
odp_packet_reset(pkt, 0);

So, the example above does not actually specify anything. Any value of 
"default_len" is handled the same way. Implementation uses 'len' to allocate 
enough segments and initialize data pointers/lengths accordingly.

// Allocated packet can hold at least 0 data bytes
// odp_packet_data() points to the first byte of data. Valid start pointer, 
although no data follows.
// odp_packet_len() returns 0
pkt1 = odp_packet_alloc(pool, 0);


// Allocated packet can hold at least 1 data bytes
// odp_packet_data() points to the first byte of data
// odp_packet_len() returns 1
pkt2 = odp_packet_alloc(pool, 1);

...


Which API definition was source of the confusion? It should be enough to 
highlight that either zero is legal or illegal value of 'len'.


-Petri


> -Original Message-
> From: ext Zoltan Kiss 
> [mailto:zoltan.k...@linaro.org]
> Sent: Thursday, May 28, 2015 4:59 PM
> To: Savolainen, Petri (Nokia - FI/Espoo); ext Bill Fischofer
> Cc: lng-odp@lists.linaro.org
> Subject: Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify use
> of 0 len on odp_packet_alloc()
>
>
>
> On 28/05/15 13:58, Savolainen, Petri (Nokia - FI/Espoo) wrote:
> > How people think 0 is special or different from e.g. value 1?
>
> Well, 0 is a special number in many ways, not just in mathematics and
> computer science:
>
> http://en.wikipedia.org/wiki/0_%28number%29
>
> It is special in this case as well, because it's defined to be equal to:
>
> pkt = odp_packet_alloc(pool, default_len);
> odp_packet_reset(pkt, 0);
>
> Which is not necessarily what you would expect. Most people had
> different ideas what 0 should mean.
>
> >
> > We can change documentation wording, but not add lengthy documentation
> for value 0 as a special case (which is not).
> >
> > -Petri
> >
> >
> >> -Original Message-
> >> From: ext Zoltan Kiss 
> >> [mailto:zoltan.k...@linaro.org]
> >> Sent: Thursday, May 28, 2015 3:51 PM
> >> To: Savolainen, Petri (Nokia - FI/Espoo); ext Bill Fischofer
> >> Cc: lng-odp@lists.linaro.org
> >> Subject: Re: [lng-odp] [API-NEXT PATCH 2/2] api-next: packet: clarify
> use
> >> of 0 len on odp_packet_alloc()
> >>
> >> I don't think so. There was a lengthy discussion on the arch meeting
> how
> >> should you interpret that, I think that proves it's worth to clarify
> >> what's the expected behaviour.
> >>
> >> On 28/05/15 13:45, Savolainen, Petri (Nokia - FI/Espoo) wrote:
> >>> I think it should be enough 

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Ola Liljedahl]

On 29 May 2015 at 13:55, Zoltan Kiss  wrote:

>
>
> On 28/05/15 17:40, Ola Liljedahl wrote:
>
>> On 28 May 2015 at 17:23, Zoltan Kiss > > wrote:
>>
>>
>>
>> On 28/05/15 16:00, Ola Liljedahl wrote:
>>
>> I disprove of this solution. TX completion processing (cleaning TX
>> descriptor rings after transmission complete) is an implementation
>> (hardware) aspect and should be hidden from the application.
>>
>>
>> Unfortunately you can't, if you want your pktio application work
>> with poll mode drivers. In that case TX completion interrupt (can
>> be) disabled and the application has to control that as well. In
>> case of DPDK you just call the send function (with 0 packets, if you
>> don't have anything to send at the time)
>>
>> Why do you have to retire transmitted packet if you are not transmitting
>> new packets (and need those descriptors in the TX ring)?
>>
> Because otherwise they are a memory leak.

They are not leaked! They are still in the TX ring, just waiting to get
retired.


> Those buffers might be needed somewhere else. If they are only released
> when you send/receive packets out next time, you are in trouble, because
> that might never happen. Especially when that event is blocked because your
> TX ring is full of unreleased packets.

Having to few buffers is always a problem. You don't want to have too large
RX/TX rings because that just increases buffering and latency ("buffer
bloat" problem).


>
>  Does the
>
>> application have too few packets in the pool so that reception will
>> suffer?
>>
> Let me approach the problem from a different angle: the current workaround
> is that you have to allocate a pool with _loooads_ of buffers, so you have
> a good chance you never run out of free buffers. Probably. Because it still
> doesn't guarantee that there will be a next send/receive event on that
> interface to release the packets.



>
>
>
>
>>
>>
>>   There isn't
>>
>> any corresponding call that refills the RX descriptor rings with
>> fresh
>> buffers.
>>
>> You can do that in the receive function, I think that's how the
>> drivers are doing it generally.
>>
>>
>> The completion processing can be performed from any ODP call, not
>> necessary odp_pktio_send().
>>
>>
>> I think "any" is not specific enough. Which one?
>>
>> odp_pktio_recv, odp_schedule. Wherever the application blocks or busy
>> waits waiting for more packets.
>>
> We do that already on odp_pktio_recv. It doesn't help, because you can
> only release the buffers held in the current interface's TX ring. You can't
> do anything about other interfaces.
>
Why not?

There is no guarantee that the application thread calling odp_pktio_recv()
on one interface is the only one transmitting on that specific egress
interface. In the general case, all threads may be using all pktio
interfaces for both reception and transmission.

I mean, you could trigger TX completion on every interface every time you
> receive on one, but that would be a scalability nightmare.

Maybe not every time. I expect a more creative solution than this. Perhaps
when you run out of buffers in the pool?



>
>
>
>>
>>
>> Can you provide a vague draft how would you fix the l2fwd example
>> below?
>>
>> I don't think anything needs fixing on the application level.
>>
>
> Wrong. odp_l2fwd uses one packet pool, receives from pktio_src and then if
> there is anything received, it sends it out on pktio_dst.

This specific application has this specific behavior. Are you sure this is
a general solution? I am not.


> Let's say the pool has 576 elements, and the interfaces uses 256 RX and
> 256 TX descriptors. You start with 2*256 buffers kept in the two RX ring.
> Let's say you receive the first 64 packets, you refill the RX ring
> immediately, so now you're out of buffers. You can send out that 64, but in
> the next iteration odp_pktio_recv() will return 0 because it can't refill
> the RX descriptors. (and the driver won't give you back any buffer unless
> you can refill it). And now you are in an infinite loop, recv will always
> return 0, because you never release the packets.
>
The size of the pool should somehow be correlated with the size of the RX
and TX rings for "best performance" (whatever this means). But I also think
that the system should function regardless of RX/TX ring sizes and pool
size, "function" meaning not deadlock.

There are several ways to fix this:
> - tell the application writer that if you see deadlocks, increase the
> element size of the buffer. I doubt anyone would ever use ODP to anything
> serious when seeing such thing.
> - you can't really give anything more specific than in the previous point,
> because such details as RX/TX descriptor numbers are abstracted away,
> intentionally. And your platform can't autotune them, because it doesn't
> know how many elements you have in the pool used for TX. In fac

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Zoltan Kiss]

On 29/05/15 13:33, Savolainen, Petri (Nokia - FI/Espoo) wrote:




-Original Message-
From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of ext
Zoltan Kiss
Sent: Friday, May 29, 2015 2:56 PM
To: Ola Liljedahl
Cc: LNG ODP Mailman List
Subject: Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add
odp_pktio_send_complete() definition



On 28/05/15 17:40, Ola Liljedahl wrote:

On 28 May 2015 at 17:23, Zoltan Kiss mailto:zoltan.k...@linaro.org>> wrote:



 On 28/05/15 16:00, Ola Liljedahl wrote:

 I disprove of this solution. TX completion processing (cleaning

TX

 descriptor rings after transmission complete) is an

implementation

 (hardware) aspect and should be hidden from the application.


 Unfortunately you can't, if you want your pktio application work
 with poll mode drivers. In that case TX completion interrupt (can
 be) disabled and the application has to control that as well. In
 case of DPDK you just call the send function (with 0 packets, if you
 don't have anything to send at the time)

Why do you have to retire transmitted packet if you are not transmitting
new packets (and need those descriptors in the TX ring)?

Because otherwise they are a memory leak. Those buffers might be needed
somewhere else. If they are only released when you send/receive packets
out next time, you are in trouble, because that might never happen.
Especially when that event is blocked because your TX ring is full of
unreleased packets.

   Does the

application have too few packets in the pool so that reception will

suffer?
Let me approach the problem from a different angle: the current
workaround is that you have to allocate a pool with _loooads_ of
buffers, so you have a good chance you never run out of free buffers.
Probably. Because it still doesn't guarantee that there will be a next
send/receive event on that interface to release the packets.


I guess CPUs can always burst packets so fast that the TX ring gets full. So, you should 
design the pool/ring configuration/init so that "full ring" is part of normal 
operation. What is the benefit of configuring so large ring that it cannot be filled to 
the max? The pools size needs to be RX + TX ring size + number of in-flight packets.


In case of l2fwd that calculation is: src RX ring size * 2 (so you can 
always refill) + dst RX ring size (because the RX queue holds the 
buffers even when not used) + dst TX ring size. That's for 
unidirectional traffic, both direction looks like: 2 * (if1 RX ring size 
+ if2 RX ring size + max(if1,if2) ring size)
You only need to know the ring sizes in this case (which we doesn't 
expose now), but there could be more complicated scenarios.
In case of OVS you need 2 * RX ring size + TX ring size, for each port. 
You need to create a separate pool for each port, currently we have one 
big pool for each port created at startup.
But I guess there could be more applications than a simple store and 
forward scenario, and they would need to make very careful assumptions 
about the theoretical highest pool usage with the actual platform they 
use, and reserve memory accordingly. I th

- we have to expose RX/TX ring sizes through pktio
- it's very easy to make a mistake in those assumptions
- you have to scale your application for extreme buffer usage in order 
to make sure you never fail




-Petri






___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH] linux-generic: timer: use plain buffers for timeouts

[Ola Liljedahl]

On 13 May 2015 at 17:31, Ola Liljedahl  wrote:

> On 13 May 2015 at 12:19, Maxim Uvarov  wrote:
>
>> Patch looks good. Validation test passed. Need one more review.
>>
> Thanks. I did this primarily for KS2 being able to use the linux-generic
> timer implementation without any changes. I am waiting for Taras to verify
> this.
>
Well it seems now that Taras is not going to verify that this patch solves
his problems on KS2. I like the patch anyway and would like to see it
merged. Any reviewers?

-- Ola


>
> -- Ola
>
>
>>
>> Thanks,
>> Maxim.
>>
>>
>> On 04/24/2015 17:10, Ola Liljedahl wrote:
>>
>>> Use plain buffers for timeouts. Store the timeout header in the buffer
>>> data area. This simplifies re-use on other platforms where the event
>>> header cannot be arbitrarily defined.
>>>
>>> Signed-off-by: Ola Liljedahl 
>>> ---
>>> (This document/code contribution attached is provided under the terms of
>>> agreement LES-LTM-21309)
>>>
>>>   platform/linux-generic/include/odp_timer_internal.h | 15
>>> ---
>>>   platform/linux-generic/odp_pool.c   |  7 ---
>>>   platform/linux-generic/odp_timer.c  | 15
>>> +--
>>>   3 files changed, 25 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/platform/linux-generic/include/odp_timer_internal.h
>>> b/platform/linux-generic/include/odp_timer_internal.h
>>> index 90af62c..a631bd0 100644
>>> --- a/platform/linux-generic/include/odp_timer_internal.h
>>> +++ b/platform/linux-generic/include/odp_timer_internal.h
>>> @@ -16,36 +16,45 @@
>>> #include 
>>>   #include 
>>> +#include 
>>>   #include 
>>>   #include 
>>>   #include 
>>> /**
>>>* Internal Timeout header
>>> + * For compatibility with buffers, we use the buffer_hdr here and
>>> nothing else
>>>*/
>>>   typedef struct {
>>> /* common buffer header */
>>> odp_buffer_hdr_t buf_hdr;
>>> +} odp_timeout_fakehdr_t;
>>>   +/* The real timeout header is in a separate struct in a separate
>>> location */
>>> +typedef struct {
>>> /* Requested expiration time */
>>> uint64_t expiration;
>>> /* User ptr inherited from parent timer */
>>> void *user_ptr;
>>> +   /* Handle of buffer we are located in */
>>> +   odp_buffer_t buf;
>>> /* Parent timer */
>>> odp_timer_t timer;
>>>   } odp_timeout_hdr_t;
>>> typedef struct odp_timeout_hdr_stride {
>>> -   uint8_t
>>> pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(odp_timeout_hdr_t))];
>>> +   uint8_t
>>> pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(odp_timeout_fakehdr_t))];
>>>   } odp_timeout_hdr_stride;
>>>   /**
>>>* Return the timeout header
>>>*/
>>> -static inline odp_timeout_hdr_t *odp_timeout_hdr(odp_buffer_t buf)
>>> +static inline odp_timeout_hdr_t *odp_timeout_hdr_from_buf(odp_buffer_t
>>> buf)
>>>   {
>>> -   return (odp_timeout_hdr_t *)odp_buf_to_hdr(buf);
>>> +   /* The real timeout header is stored in the buffer data */
>>> +   ODP_ASSERT(odp_buffer_size(buf) == sizeof(odp_timeout_hdr_t));
>>> +   return (odp_timeout_hdr_t *)odp_buffer_addr(buf);
>>>   }
>>> #endif
>>> diff --git a/platform/linux-generic/odp_pool.c
>>> b/platform/linux-generic/odp_pool.c
>>> index bf49623..fc10aa4 100644
>>> --- a/platform/linux-generic/odp_pool.c
>>> +++ b/platform/linux-generic/odp_pool.c
>>> @@ -32,7 +32,7 @@
>>>   typedef union buffer_type_any_u {
>>> odp_buffer_hdr_t  buf;
>>> odp_packet_hdr_t  pkt;
>>> -   odp_timeout_hdr_t tmo;
>>> +   odp_timeout_fakehdr_t tmo;
>>>   } odp_anybuf_t;
>>> _ODP_STATIC_ASSERT((sizeof(union buffer_type_any_u) % 8) == 0,
>>> @@ -148,7 +148,8 @@ odp_pool_t odp_pool_create(const char *name,
>>> /* Default size and align for timeouts */
>>> if (params->type == ODP_POOL_TIMEOUT) {
>>> -   params->buf.size  = 0; /* tmo.__res1 */
>>> +   /* The real timeout header is stored in the buffer */
>>> +   params->buf.size  = sizeof(odp_timeout_hdr_t); /*
>>> tmo.__res1 */
>>> params->buf.align = 0; /* tmo.__res2 */
>>> }
>>>   @@ -226,7 +227,7 @@ odp_pool_t odp_pool_create(const char *name,
>>> break;
>>> case ODP_POOL_TIMEOUT:
>>> -   blk_size = 0;
>>> +   blk_size = params->buf.size;
>>> buf_num = params->tmo.num;
>>> buf_stride = sizeof(odp_timeout_hdr_stride);
>>> break;
>>> diff --git a/platform/linux-generic/odp_timer.c
>>> b/platform/linux-generic/odp_timer.c
>>> index e5391dc..1922f94 100644
>>> --- a/platform/linux-generic/odp_timer.c
>>> +++ b/platform/linux-generic/odp_timer.c
>>> @@ -78,7 +78,7 @@ static _odp_atomic_flag_t locks[NUM_LOCKS]; /*
>>> Multiple locks per cache line! */
>>> static odp_timeout_hdr_t *timeout_hdr_from_buf(odp_buffer_t buf)
>>>   {
>>> -   return (odp_timeout_hdr_t *)odp_buf_to_hdr(buf);
>>> +   return odp_timeout_hdr_from_buf(buf);
>>>   }
>

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Ola Liljedahl]

On 29 May 2015 at 16:26, Zoltan Kiss  wrote:

>
>
> On 29/05/15 13:33, Savolainen, Petri (Nokia - FI/Espoo) wrote:
>
>>
>>
>>  -Original Message-
>>> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of ext
>>> Zoltan Kiss
>>> Sent: Friday, May 29, 2015 2:56 PM
>>> To: Ola Liljedahl
>>> Cc: LNG ODP Mailman List
>>> Subject: Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add
>>> odp_pktio_send_complete() definition
>>>
>>>
>>>
>>> On 28/05/15 17:40, Ola Liljedahl wrote:
>>>
 On 28 May 2015 at 17:23, Zoltan Kiss >>> > wrote:



  On 28/05/15 16:00, Ola Liljedahl wrote:

  I disprove of this solution. TX completion processing (cleaning

>>> TX
>>>
  descriptor rings after transmission complete) is an

>>> implementation
>>>
  (hardware) aspect and should be hidden from the application.


  Unfortunately you can't, if you want your pktio application work
  with poll mode drivers. In that case TX completion interrupt (can
  be) disabled and the application has to control that as well. In
  case of DPDK you just call the send function (with 0 packets, if
 you
  don't have anything to send at the time)

 Why do you have to retire transmitted packet if you are not transmitting
 new packets (and need those descriptors in the TX ring)?

>>> Because otherwise they are a memory leak. Those buffers might be needed
>>> somewhere else. If they are only released when you send/receive packets
>>> out next time, you are in trouble, because that might never happen.
>>> Especially when that event is blocked because your TX ring is full of
>>> unreleased packets.
>>>
>>>Does the
>>>
 application have too few packets in the pool so that reception will

>>> suffer?
>>> Let me approach the problem from a different angle: the current
>>> workaround is that you have to allocate a pool with _loooads_ of
>>> buffers, so you have a good chance you never run out of free buffers.
>>> Probably. Because it still doesn't guarantee that there will be a next
>>> send/receive event on that interface to release the packets.
>>>
>>
>> I guess CPUs can always burst packets so fast that the TX ring gets full.
>> So, you should design the pool/ring configuration/init so that "full ring"
>> is part of normal operation. What is the benefit of configuring so large
>> ring that it cannot be filled to the max? The pools size needs to be RX +
>> TX ring size + number of in-flight packets.
>>
>
> In case of l2fwd that calculation is: src RX ring size * 2 (so you can
> always refill) + dst RX ring size (because the RX queue holds the buffers
> even when not used) + dst TX ring size. That's for unidirectional traffic,
> both direction looks like: 2 * (if1 RX ring size + if2 RX ring size +
> max(if1,if2) ring size)
> You only need to know the ring sizes in this case (which we doesn't expose
> now), but there could be more complicated scenarios.
> In case of OVS you need 2 * RX ring size + TX ring size, for each port.
> You need to create a separate pool for each port, currently we have one big
> pool for each port created at startup.
> But I guess there could be more applications than a simple store and
> forward scenario, and they would need to make very careful assumptions
> about the theoretical highest pool usage with the actual platform they use,
> and reserve memory accordingly. I th
> - we have to expose RX/TX ring sizes through pktio
> - it's very easy to make a mistake in those assumptions
> - you have to scale your application for extreme buffer usage in order to
> make sure you never fail
>
If you are not wire-speed for the worst packet rate (minimum packet size),
there is no guarantee that you will "never fail". Increasing buffer sizes
(e.g. RX/TX ring sizes and pool size) doesn't help and is actually a bad
solution anyway. Overload situations should be expected and the design
should handle them gracefully (maintain peak packet rate and drop excess
packets according to some chosen QoS policy).

Does it matter if unused packets are located in a TX ring or in the pool
proper? If odp_packet_alloc() encounters a pool exhausted situation,
attempt to reclaim transmitted packets from TX rings.


>
>> -Petri
>>
>>
>>
>>
>>
>>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Zoltan Kiss]

On 29/05/15 15:21, Ola Liljedahl wrote:

On 29 May 2015 at 13:55, Zoltan Kiss mailto:zoltan.k...@linaro.org>> wrote:



On 28/05/15 17:40, Ola Liljedahl wrote:

On 28 May 2015 at 17:23, Zoltan Kiss mailto:zoltan.k...@linaro.org>
>>
wrote:



 On 28/05/15 16:00, Ola Liljedahl wrote:

 I disprove of this solution. TX completion processing
(cleaning TX
 descriptor rings after transmission complete) is an
implementation
 (hardware) aspect and should be hidden from the
application.


 Unfortunately you can't, if you want your pktio application
work
 with poll mode drivers. In that case TX completion
interrupt (can
 be) disabled and the application has to control that as
well. In
 case of DPDK you just call the send function (with 0
packets, if you
 don't have anything to send at the time)

Why do you have to retire transmitted packet if you are not
transmitting
new packets (and need those descriptors in the TX ring)?

Because otherwise they are a memory leak.

They are not leaked! They are still in the TX ring, just waiting to get
retired.


Indeed, leak is not the right word because they are still referenced. 
But they still can't be released currently, only as a side effect of 
something else which might not happen.


Those buffers might be needed somewhere else. If they are only
released when you send/receive packets out next time, you are in
trouble, because that might never happen. Especially when that event
is blocked because your TX ring is full of unreleased packets.

Having to few buffers is always a problem. You don't want to have too
large RX/TX rings because that just increases buffering and latency
("buffer bloat" problem).


That's the generic principle, yes, but we need a bulletproof way here 
that applications can't get blocked because they forget to call the 
magic function which otherwise release the TX buffers.






  Does the

application have too few packets in the pool so that reception
will suffer?

Let me approach the problem from a different angle: the current
workaround is that you have to allocate a pool with _loooads_ of
buffers, so you have a good chance you never run out of free
buffers. Probably. Because it still doesn't guarantee that there
will be a next send/receive event on that interface to release the
packets.







   There isn't

 any corresponding call that refills the RX descriptor
rings with
 fresh
 buffers.

 You can do that in the receive function, I think that's how the
 drivers are doing it generally.


 The completion processing can be performed from any ODP
call, not
 necessary odp_pktio_send().


 I think "any" is not specific enough. Which one?

odp_pktio_recv, odp_schedule. Wherever the application blocks or
busy
waits waiting for more packets.

We do that already on odp_pktio_recv. It doesn't help, because you
can only release the buffers held in the current interface's TX
ring. You can't do anything about other interfaces.

Why not?

There is no guarantee that the application thread calling
odp_pktio_recv() on one interface is the only one transmitting on that
specific egress interface. In the general case, all threads may be using
all pktio interfaces for both reception and transmission.

I mean, you could trigger TX completion on every interface every
time you receive on one, but that would be a scalability nightmare.

Maybe not every time. I expect a more creative solution than this.
Perhaps when you run out of buffers in the pool?


That might be better, I will think about it.









 Can you provide a vague draft how would you fix the l2fwd
example below?

I don't think anything needs fixing on the application level.


Wrong. odp_l2fwd uses one packet pool, receives from pktio_src and
then if there is anything received, it sends it out on pktio_dst.

This specific application has this specific behavior. Are you sure this
is a general solution? I am not.

Let's say the pool has 576 elements, and the interfaces uses 256 RX
and 256 TX descriptors. You start with 2*256 buffers kept in the two
RX ring. Let's say you receive the first 64 packets, you refill the
RX ring immediately, so now you're out of buffers. You can send out
that 64, but in the next iteration odp_pktio_recv() will return 0
because it can't refill the RX descriptors. (and the driver won't
give you back any buffer unless you can refill it). And now you are
in an infinite loop, recv will 

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Zoltan Kiss]

On 29/05/15 15:36, Ola Liljedahl wrote:

On 29 May 2015 at 16:26, Zoltan Kiss mailto:zoltan.k...@linaro.org>> wrote:



On 29/05/15 13:33, Savolainen, Petri (Nokia - FI/Espoo) wrote:



-Original Message-
From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org
] On Behalf Of ext
Zoltan Kiss
Sent: Friday, May 29, 2015 2:56 PM
To: Ola Liljedahl
Cc: LNG ODP Mailman List
Subject: Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add
odp_pktio_send_complete() definition



On 28/05/15 17:40, Ola Liljedahl wrote:

On 28 May 2015 at 17:23, Zoltan Kiss
mailto:zoltan.k...@linaro.org>
>> wrote:



  On 28/05/15 16:00, Ola Liljedahl wrote:

  I disprove of this solution. TX completion
processing (cleaning

TX

  descriptor rings after transmission complete)
is an

implementation

  (hardware) aspect and should be hidden from
the application.


  Unfortunately you can't, if you want your pktio
application work
  with poll mode drivers. In that case TX completion
interrupt (can
  be) disabled and the application has to control
that as well. In
  case of DPDK you just call the send function (with
0 packets, if you
  don't have anything to send at the time)

Why do you have to retire transmitted packet if you are
not transmitting
new packets (and need those descriptors in the TX ring)?

Because otherwise they are a memory leak. Those buffers
might be needed
somewhere else. If they are only released when you
send/receive packets
out next time, you are in trouble, because that might never
happen.
Especially when that event is blocked because your TX ring
is full of
unreleased packets.

Does the

application have too few packets in the pool so that
reception will

suffer?
Let me approach the problem from a different angle: the current
workaround is that you have to allocate a pool with _loooads_ of
buffers, so you have a good chance you never run out of free
buffers.
Probably. Because it still doesn't guarantee that there will
be a next
send/receive event on that interface to release the packets.


I guess CPUs can always burst packets so fast that the TX ring
gets full. So, you should design the pool/ring
configuration/init so that "full ring" is part of normal
operation. What is the benefit of configuring so large ring that
it cannot be filled to the max? The pools size needs to be RX +
TX ring size + number of in-flight packets.


In case of l2fwd that calculation is: src RX ring size * 2 (so you
can always refill) + dst RX ring size (because the RX queue holds
the buffers even when not used) + dst TX ring size. That's for
unidirectional traffic, both direction looks like: 2 * (if1 RX ring
size + if2 RX ring size + max(if1,if2) ring size)
You only need to know the ring sizes in this case (which we doesn't
expose now), but there could be more complicated scenarios.
In case of OVS you need 2 * RX ring size + TX ring size, for each
port. You need to create a separate pool for each port, currently we
have one big pool for each port created at startup.
But I guess there could be more applications than a simple store and
forward scenario, and they would need to make very careful
assumptions about the theoretical highest pool usage with the actual
platform they use, and reserve memory accordingly. I th
- we have to expose RX/TX ring sizes through pktio
- it's very easy to make a mistake in those assumptions
- you have to scale your application for extreme buffer usage in
order to make sure you never fail

If you are not wire-speed for the worst packet rate (minimum packet
size), there is no guarantee that you will "never fail".


When I say "fail" here, I mean you are deadlocked, not just overloaded.

 Increasing

buffer sizes (e.g. RX/TX ring sizes and pool size) doesn't help and is
actually a bad solution anyway. Overload situations should be expected
and the design should handle them gracefully (maintain peak packet rate
and drop excess packets according to some chosen QoS policy).

Does it matter 

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Ola Liljedahl]

On 29 May 2015 at 17:03, Zoltan Kiss  wrote:

>
>
> On 29/05/15 15:36, Ola Liljedahl wrote:
>
>> On 29 May 2015 at 16:26, Zoltan Kiss > > wrote:
>>
>>
>>
>> On 29/05/15 13:33, Savolainen, Petri (Nokia - FI/Espoo) wrote:
>>
>>
>>
>> -Original Message-
>> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org
>> ] On Behalf Of ext
>> Zoltan Kiss
>> Sent: Friday, May 29, 2015 2:56 PM
>> To: Ola Liljedahl
>> Cc: LNG ODP Mailman List
>> Subject: Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add
>> odp_pktio_send_complete() definition
>>
>>
>>
>> On 28/05/15 17:40, Ola Liljedahl wrote:
>>
>> On 28 May 2015 at 17:23, Zoltan Kiss
>> mailto:zoltan.k...@linaro.org>
>> >
>> >> wrote:
>>
>>
>>
>>   On 28/05/15 16:00, Ola Liljedahl wrote:
>>
>>   I disprove of this solution. TX completion
>> processing (cleaning
>>
>> TX
>>
>>   descriptor rings after transmission complete)
>> is an
>>
>> implementation
>>
>>   (hardware) aspect and should be hidden from
>> the application.
>>
>>
>>   Unfortunately you can't, if you want your pktio
>> application work
>>   with poll mode drivers. In that case TX completion
>> interrupt (can
>>   be) disabled and the application has to control
>> that as well. In
>>   case of DPDK you just call the send function (with
>> 0 packets, if you
>>   don't have anything to send at the time)
>>
>> Why do you have to retire transmitted packet if you are
>> not transmitting
>> new packets (and need those descriptors in the TX ring)?
>>
>> Because otherwise they are a memory leak. Those buffers
>> might be needed
>> somewhere else. If they are only released when you
>> send/receive packets
>> out next time, you are in trouble, because that might never
>> happen.
>> Especially when that event is blocked because your TX ring
>> is full of
>> unreleased packets.
>>
>> Does the
>>
>> application have too few packets in the pool so that
>> reception will
>>
>> suffer?
>> Let me approach the problem from a different angle: the
>> current
>> workaround is that you have to allocate a pool with _loooads_
>> of
>> buffers, so you have a good chance you never run out of free
>> buffers.
>> Probably. Because it still doesn't guarantee that there will
>> be a next
>> send/receive event on that interface to release the packets.
>>
>>
>> I guess CPUs can always burst packets so fast that the TX ring
>> gets full. So, you should design the pool/ring
>> configuration/init so that "full ring" is part of normal
>> operation. What is the benefit of configuring so large ring that
>> it cannot be filled to the max? The pools size needs to be RX +
>> TX ring size + number of in-flight packets.
>>
>>
>> In case of l2fwd that calculation is: src RX ring size * 2 (so you
>> can always refill) + dst RX ring size (because the RX queue holds
>> the buffers even when not used) + dst TX ring size. That's for
>> unidirectional traffic, both direction looks like: 2 * (if1 RX ring
>> size + if2 RX ring size + max(if1,if2) ring size)
>> You only need to know the ring sizes in this case (which we doesn't
>> expose now), but there could be more complicated scenarios.
>> In case of OVS you need 2 * RX ring size + TX ring size, for each
>> port. You need to create a separate pool for each port, currently we
>> have one big pool for each port created at startup.
>> But I guess there could be more applications than a simple store and
>> forward scenario, and they would need to make very careful
>> assumptions about the theoretical highest pool usage with the actual
>> platform they use, and reserve memory accordingly. I th
>> - we have to expose RX/TX ring sizes through pktio
>> - it's very easy to make a mistake in those assumptions
>> - you have to scale your application for extreme buffer usage in
>> order to make sure you never fail
>>
>> If you are not wire-speed for the worst packet rate (minimum packet
>> size), there is no guarantee that you will "never fail".
>>
>
> Wh

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Jerin Jacob]

On Fri, May 29, 2015 at 04:21:03PM +0200, Ola Liljedahl wrote:
> On 29 May 2015 at 13:55, Zoltan Kiss  wrote:
> 
> >
> >
> > On 28/05/15 17:40, Ola Liljedahl wrote:
> >
> >> On 28 May 2015 at 17:23, Zoltan Kiss  >> > wrote:
> >>
> >>
> >>
> >> On 28/05/15 16:00, Ola Liljedahl wrote:
> >>
> >> I disprove of this solution. TX completion processing (cleaning TX
> >> descriptor rings after transmission complete) is an implementation
> >> (hardware) aspect and should be hidden from the application.
> >>
> >>
> >> Unfortunately you can't, if you want your pktio application work
> >> with poll mode drivers. In that case TX completion interrupt (can
> >> be) disabled and the application has to control that as well. In
> >> case of DPDK you just call the send function (with 0 packets, if you
> >> don't have anything to send at the time)
> >>
> >> Why do you have to retire transmitted packet if you are not transmitting
> >> new packets (and need those descriptors in the TX ring)?
> >>
> > Because otherwise they are a memory leak.
> 
> They are not leaked! They are still in the TX ring, just waiting to get
> retired.
> 
> 
> > Those buffers might be needed somewhere else. If they are only released
> > when you send/receive packets out next time, you are in trouble, because
> > that might never happen. Especially when that event is blocked because your
> > TX ring is full of unreleased packets.
> 
> Having to few buffers is always a problem. You don't want to have too large
> RX/TX rings because that just increases buffering and latency ("buffer
> bloat" problem).
> 
> 
> >
> >  Does the
> >
> >> application have too few packets in the pool so that reception will
> >> suffer?
> >>
> > Let me approach the problem from a different angle: the current workaround
> > is that you have to allocate a pool with _loooads_ of buffers, so you have
> > a good chance you never run out of free buffers. Probably. Because it still
> > doesn't guarantee that there will be a next send/receive event on that
> > interface to release the packets.
> 
> 
> 
> >
> >
> >
> >
> >>
> >>
> >>   There isn't
> >>
> >> any corresponding call that refills the RX descriptor rings with
> >> fresh
> >> buffers.
> >>
> >> You can do that in the receive function, I think that's how the
> >> drivers are doing it generally.
> >>
> >>
> >> The completion processing can be performed from any ODP call, not
> >> necessary odp_pktio_send().
> >>
> >>
> >> I think "any" is not specific enough. Which one?
> >>
> >> odp_pktio_recv, odp_schedule. Wherever the application blocks or busy
> >> waits waiting for more packets.
> >>
> > We do that already on odp_pktio_recv. It doesn't help, because you can
> > only release the buffers held in the current interface's TX ring. You can't
> > do anything about other interfaces.
> >
> Why not?
> 
> There is no guarantee that the application thread calling odp_pktio_recv()
> on one interface is the only one transmitting on that specific egress
> interface. In the general case, all threads may be using all pktio
> interfaces for both reception and transmission.
> 
> I mean, you could trigger TX completion on every interface every time you
> > receive on one, but that would be a scalability nightmare.
> 
> Maybe not every time. I expect a more creative solution than this. Perhaps
> when you run out of buffers in the pool?
> 
> 
> 
> >
> >
> >
> >>
> >>
> >> Can you provide a vague draft how would you fix the l2fwd example
> >> below?
> >>
> >> I don't think anything needs fixing on the application level.
> >>
> >
> > Wrong. odp_l2fwd uses one packet pool, receives from pktio_src and then if
> > there is anything received, it sends it out on pktio_dst.
> 
> This specific application has this specific behavior. Are you sure this is
> a general solution? I am not.
> 
> 
> > Let's say the pool has 576 elements, and the interfaces uses 256 RX and
> > 256 TX descriptors. You start with 2*256 buffers kept in the two RX ring.
> > Let's say you receive the first 64 packets, you refill the RX ring
> > immediately, so now you're out of buffers. You can send out that 64, but in
> > the next iteration odp_pktio_recv() will return 0 because it can't refill
> > the RX descriptors. (and the driver won't give you back any buffer unless
> > you can refill it). And now you are in an infinite loop, recv will always
> > return 0, because you never release the packets.
> >
> The size of the pool should somehow be correlated with the size of the RX
> and TX rings for "best performance" (whatever this means). But I also think
> that the system should function regardless of RX/TX ring sizes and pool
> size, "function" meaning not deadlock.
> 
> There are several ways to fix this:
> > - tell the application writer that if you see deadlocks, increase the
> > element size of the buffer. I doubt anyone would ever use ODP 

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Zoltan Kiss]

Hi,

On 29/05/15 16:58, Jerin Jacob wrote:

I agree. Is it possbile to dedicate "core 0"/"any core" in ODP-DPDK 
implementation
to do the house keeping job ? If we are planning for ODP-DPDK
implementation as just wrapper over DPDK API then there will not be any
value addition to use the ODP API. At least from my experience, We have
changed our  SDK "a lot" to fit into ODP model. IMO that kind of effort will
be required for useful ODP-DPDK port.


It would be good to have some input from other implementations as well: 
when do you release the sent packets in the Cavium implementation?

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Ola Liljedahl]

On 29 May 2015 at 18:03, Zoltan Kiss  wrote:

> Hi,
>
> On 29/05/15 16:58, Jerin Jacob wrote:
>
>> I agree. Is it possbile to dedicate "core 0"/"any core" in ODP-DPDK
>> implementation
>> to do the house keeping job ? If we are planning for ODP-DPDK
>> implementation as just wrapper over DPDK API then there will not be any
>> value addition to use the ODP API. At least from my experience, We have
>> changed our  SDK "a lot" to fit into ODP model. IMO that kind of effort
>> will
>> be required for useful ODP-DPDK port.
>>
>
> It would be good to have some input from other implementations as well:
> when do you release the sent packets in the Cavium implementation?
>
Most networking SoC's have HW buffer management. Buffers are automatically
freed after transmission. On reception, free buffers are automatically
allocated (potentially after classification so the HW knows which buffer
pool to use). HW buffer management actually saves a measurable amount of
cycles, on an old Freescale PowerQUICC device (with SW-managed RX/TX
rings), I estimated the overhead of SW buffer management (not thread-safe)
to be around 90 cycles which is a lot when the total per-packet cost was
around 600 cycles.
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCH] api-next: pktio: add odp_pktio_send_complete() definition

[Bill Fischofer]

The intent is that ODP APIs abstract the functionality that the application
needs so that implementations have flexibility to migrate the low-level
"how tos" into HW.  This means that you DO NOT want the application to have
responsibility for making housekeeping calls--these should be implicit in
the implementation of ODP APIs.  The goal is that SW is only involved for
higher-level decision making that HW cannot do with sufficient
flexibility.  This means that by the time SW is called, everything is
predigested and prestaged so that SW simply does its decision logic and
returns the result back to HW, which takes it from there.

Not every implementation will be able to do this immediately, but if we
keep that target model in mind it helps to guide the API design.

On Fri, May 29, 2015 at 12:22 PM, Ola Liljedahl 
wrote:

> On 29 May 2015 at 18:03, Zoltan Kiss  wrote:
>
>> Hi,
>>
>> On 29/05/15 16:58, Jerin Jacob wrote:
>>
>>> I agree. Is it possbile to dedicate "core 0"/"any core" in ODP-DPDK
>>> implementation
>>> to do the house keeping job ? If we are planning for ODP-DPDK
>>> implementation as just wrapper over DPDK API then there will not be any
>>> value addition to use the ODP API. At least from my experience, We have
>>> changed our  SDK "a lot" to fit into ODP model. IMO that kind of effort
>>> will
>>> be required for useful ODP-DPDK port.
>>>
>>
>> It would be good to have some input from other implementations as well:
>> when do you release the sent packets in the Cavium implementation?
>>
> Most networking SoC's have HW buffer management. Buffers are automatically
> freed after transmission. On reception, free buffers are automatically
> allocated (potentially after classification so the HW knows which buffer
> pool to use). HW buffer management actually saves a measurable amount of
> cycles, on an old Freescale PowerQUICC device (with SW-managed RX/TX
> rings), I estimated the overhead of SW buffer management (not thread-safe)
> to be around 90 cycles which is a lot when the total per-packet cost was
> around 600 cycles.
>
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp