Oriol: I've added this and the TM issue you raised to the agenda for
Tuesday's ODP public call. Please join us if you're able to discuss this
further. Thanks.
Regards,
Bill
On Mon, May 1, 2017 at 7:04 AM, Bill Fischofer <bill.fischo...@linaro.org>
wrote:
>
>
> On Thu, Apr 27, 2017 at 10:34 AM, Oriol Arcas <or...@starflownetworks.com>
> wrote:
>
>> Hello,
>>
>> I have some questions regarding how scheduled PKTIOs are closed. Please
>> correct me if I am wrong...
>>
>> In ODP 1.11 Monarch (I think 1.14 as well) scheduled PKTIOs cannot be
>> closed before they are stopped and unscheduled. This is why there are the
>> transitional states PKTIO_STATE_STOP_PENDING and
>> PKTIO_STATE_CLOSE_PENDING.
>>
>> When a PKTIO is odp_pktio_closed(), if it is scheduled it becomes
>> PKTIO_STATE_CLOSE_PENDING. From this state, it can only become
>> PKTIO_STATE_FREE through sched_cb_pktio_stop_finalize(), a callback that
>> is
>> only called from do_schedule().
>>
>> Now my concern appears. In do_schedule(), if a PKTIO is in
>> PKTIO_STATE_CLOSE_PENDING, it is unscheduled and the callback is called.
>> So, a program must call odp_schedule() to make sure all the PKTIOs are
>> correctly closed.
>>
>> However, do_schedule() has a number of command queues (4 by default),
>> among
>> which the PKTIOs schedule tokens are distributed. A single odp_schedule()
>> call may unschedule and correctly close 4 PKTIOs:
>>
>> for (i = 0; i < POLL_CMD_QUEUES; i++, id++) {
>> ...
>> cmd_queue = sched->poll_cmd[id].queue;
>> ...
>> if (schedule_pktio_stop(sched_cmd) == 0) {
>> sched_cb_pktio_stop_finalize(pktio_index);
>> }
>> ...
>> }
>>
>> Subsequent odp_schedule() calls may be needed to make sure all the PKTIOs
>> are closed. The number of required odp_schedule() calls depends on how
>> many
>> scheduled PKTIOs there are, and how many schedule command queues (roughly,
>> # scheduled PKTIOs / # schedule command queues). It is a workaround, but
>> certainly not a nice one, especially because the app code doesn't know the
>> number of command queues.
>>
>> The reason of all this is that if PKTIOs are not closed properly (they
>> remain in a *_PENDING state), odp_term_global() may end with an abort
>> signal!
>>
>> ret = _pktio_close(pktio_entry);
>> if (ret) {
>> ODP_ABORT("unable to close pktio %s\n",
>> pktio_entry->s.name);
>> }
>>
>> So beyond 4 PKTIOs, I am getting ungraceful terminations, unless I do 2
>> odp_schedule() after closing the PKTIOs.
>>
>> Am I missing something, is there a proper way of doing this?
>>
>
> Sounds like a bug. Pktio termination processing should be doing any needed
> scheduling calls to flush pending closes under the covers since the details
> of particular scheduler implementations are not part of the API.
>
>
>>
>> --
>> Oriol Arcas
>> Software Engineer
>> Starflow Networks
>>
>
>
