On Thu, Apr 6, 2017 at 8:22 AM, Radosław Biernacki <r...@semihalf.com> wrote: > Hi Bill, > > Thank you for reply and sorry for my long reply. > > IMHO the description which you give could be copied to > doc/users-guide/users-guide.adoc as there are not many information how this > should work across all implementations. > I don't fully understand following sentence "since each individual index > follows the same order, which is fixed for both Thread A and B" > > Could you please describe what should happen in below example? > Each line specify the time of event: > T1: WorkerA calls odp_schedule() -> gets packet 1 from queueA > T2: WorkerB calls odp_schedule() -> gets packet 2 from queueA (which came > into interface after packet1) > > T3: WorkerB calls odp_schedule_order_lock(1)
WorkerB blocks waiting for sequence 2's turn to use lock(1) > T3: WorkerA calls odp_schedule_order_lock(0) Since WorkerA has the current sequence number (1) for lock(0), WorkerA enters the critical section protected by lock(0). > > than WorkerA calls odp_schedule_order_unlock(0) and calls This means that lock(0) is now available to sequence 2. Any callers with higher sequence numbers will wait. It is a programming error for WorkerA to attempt to re-acquire lock(0) under its current context since it's already used this lock. > odp_schedule_order_lock(1) (and unlock(1)) > while WorkerB only unlock the 1 > > Will WorkerB be suspended till WorkerA unlocks order lock 1? Yes. WorkerB has sequence 2 and lock 1 is not available to it until after sequence 1 (WorkerA) has used this lock (or released it's context). > We implementing this API now and our world seems totally different than > linux-generic so it would be best if you could specify the remaining part of > the sequence (as it would be a sequence requirement for us). > > Topic 2: I guess that nested order locks are forbidden? If that's the case, > than we should also mention that in docs. No, ordered locks may be nested without restriction. For example, suppose WorkerA acquires lock(0) and then tries to acquire lock(1) before it releases lock(0). This is fine since it holds the proper sequence for both locks. The difference between ordered locks and spinlocks is that they can only be acquired by a thread that's in sequence, so the thread holding the current sequence always gets the lock when it asks for it and all others wait. So lock inversions are impossible with ordered locks because inversions can only happen when it's unpredictable who gets which lock. In the case of ordered locks that sequence is fixed by the ordered context itself, not by the individual locks. > > 2017-03-29 19:38 GMT+02:00 Bill Fischofer <bill.fischo...@linaro.org>: >> >> On Wed, Mar 29, 2017 at 7:18 AM, Radosław Biernacki <r...@semihalf.com> >> wrote: >> > Hi all, >> > >> > The documentation for odp_schedule_order_lock(unsigned lock_index) does >> > not >> > specify the sequence in which the lock_index need to be given. >> >> That's because there is no such required sequence. Each lock index is >> a distinct ordered synchronization point and the only requirement is >> that each thread may only use each index at most once per ordered >> context. Threads may skip any or all indexes as there is no obligation >> for threads to use ordered locks when they are present in the context, >> and the index order doesn't matter to ODP. >> >> When threads enter an ordered context they hold a sequence that is >> determined by the source ordered queue. For a thread to be able to >> acquire ordered lock index i, all that is required is that all threads >> holding lower sequences have either acquired and released that index >> or else definitively passed on using it by exiting the ordered >> context. So if thread A tries to acquire index 1 first and then index >> 2 while Thread B tries to acquire index 2 first and then index 1 it >> doesn't matter since each individual index follows the same order, >> which is fixed for both Thread A and B. >> >> Note that there may be a loss of parallelism if indexes are permuted, >> however there is no possibility of deadlock. Best parallelism will be >> achieved when all threads use indexes in the same sequence, but ODP >> doesn't care what sequence of indexes the application chooses to use. >> >> > >> > Shouldn't the following statements be included in description of this >> > function? >> > 1) All code paths calling this function (in the same synchronization >> > context) need to use the same lock_index sequence, eg: 1, 3, 2, 4 or the >> > results are undefined (like the sequence of events will not be >> > preserved) >> > if this rule is not followed >> > 2) The doc should emphasize a bit more to what the synchronization >> > context >> > is bound to (source queue). For eg. it should say that lock_index >> > sequence >> > can be different for different source queues (synchronization contexts). >> > 3) It is possible to skip some lock_index in sequence. But skipped >> > lock_indexes cannot be used outside of the sequence (since this will >> > alter >> > the sequence which is violation of rule 1). > >
