> >> > >> === Addressing Scope > >> -Unless specifically noted in the API, all ODP resources are global to > >> the ODP > >> -application, whether it runs as a single process or multiple > processes. > >> ODP > >> -handles therefore have common meaning within an ODP application but > have > >> no > >> -meaning outside the scope of the application. > >> +ODP objects and their associated handles are intended to be global to > an > >> ODP > >> +instance, _i.e.,_ to the thread that calls `odp_init_global()` and its > >> +descendants. > > > > > > This is saying that all ODP threads of a given ODP instance are > descendant > > of the ODP instantiation process. I already tried to define ODP threads > > this way, but it was rejected, and the ODP instance number was added to > the > > API so that any thread/process could join any ODP instance. > > > > We should revisit that because without this assumption I don't think we > can > define a consistent set of semantics for ODP object visibility. For > example, if I create an ODP instance and then pass the instance ID > privately to a completely separate program, do we expect it to be able to > use that instance ID to join the (remote) ODP instance by calling > odp_init_local()? Clearly not. >
Yes, if the implementation supports that. E.g. there's no processes or pthreads in when an SDK runs in bare metal mode. Each cpu starts from C lib / SDK startup code and stops at some point to wait others. There's some shared memory set up at init phase. You could start up e.g. two separate ODP instances of ODP, so that half of the cpus joins the first instance and the other half joins the other instance. odp-linux would not support (at least at first) non-related processes to form an instance. But API would not limit, if support for it would be added later. > > > I still get the feeling that, despite this add-on, we are still saying > > that a odpthread belongs to the ODP instance it is a descendand of, > rather > > than the one specified in the "instance" parameter passed at > > odp_local_init() time... > > The only possible usage of the odp_instance parameter passed at > > odp_local_init() time would then be on OSes which do not support process > > hierarchy... Is any such OS known to suport ODP? > > > > I concur. I think it's there more for potential future flexibility than > anything else since it's effectively a dummy parameter with a single > hard-coded value in odp-linux. I see no harm in it being there, but it's > presence doesn't change the fact that ODP instances only cover the > odp_global_init() caller and its descendants. This is the point. Instance ID enables various thread setups (for linux, RTOS, bare metal, ...), odp-linux implements some of those. > > > > If not, I thing the odp_instance number should either be suppressed, or > > clearly stated as meaningless on OS which have process hierarchy, as the > > process hierarchy seems to take precedence over this parameter in all > our > > discussions. An implementation may use the instance ID any way it likes. It may carry a file name, file descriptor, index to shared memory, magic number, ... etc information to find the instance or just error check that global_init was indeed called before local_init was called. No need to state it meaningless for any implementation. > > > > Confused: are talking monarch or some later version? BTW is monarch the > > name of the API version only or does the "monarch" name also include the > > linux-generic platform example? > > > > Both. Monarch is the current API level and odp-linux implements that level > of the ODP API. These statements will still be true of Monarch even after > we move on to Tiger Moth, which may or may not introduce additional > capabilities. > Monarch is API level and various implementations will be maintained for it for "a long time". E.g. during that time, odp-linux will be optimized further, new pktios may be added, process mode support will be added, new implementation specific configuration file / parameters may be introduced, ... API remains the same but implementations may evolve. > > > > > > >> + > >> +If a platform uses process that follow a Unix-style `fork()` model of > >> process > >> +creation, then for best portability ODP applications should be > designed > >> so > >> +that any ODP resource creation (_e.g.,_ pools, shared memory objects, > >> pktios) > >> > > > > "any ODP resource": That includes everything, e.g. atomics, barrier, > > timers... as well...right...? > > > > The key word here is creation, not declaration or allocation. Atomics, > barriers, and locks are not created, they are declared and initialized, so > the caller is controlling the space that they occupy. Similarly packets > and timers are allocated from pools that are created, so these objects > inherit their visibility from the pools they are drawn from. The > requirement is that the pools be created at a point of common > addressability for all descendants that reference the pool or similar > created object if the objects drawn from them are to be sharable across > process boundaries. > > Queues are created, however they are created from the ODP instance itself, > so they automatically have ODP instance-wide visibility. It may be simpler > to just enumerate the APIs for which this caveat applies. They are > odp_pool_create(), odp_timer_pool_create(), and odp_shm_reserve(). > odp_pktio_open() is another case due to the (implied) use of file > descriptors. The traffic manager as currently implemented in odp-linux > assumes a single address space so we'll have to see what impacts there are > there for adding any process mode support, but my preference would be for > it to be treated more like queues than pools so that it is implicitly tied > directly to the ODP instance. > > Note that the whole preallocation discussion was to root all of these > things to the ODP instance rather than the create() call itself to avoid > these subtleties. > Any allocated object may consume lots of shared memory. E.g. 1B queues (or timers/timeouts or tm queues or classification rules or ...) would consume gigabytes of ODP internal memory, and in that perspective a 100MB pool creation would be in significant addition to the total shared memory consumption - and memory for that pool could be very well included into ODP instance itself. So, there's no obvious preference order for odp-linux to state which objects are created at init time and which dynamically. For simplicity all memory should be reserved from the system at global init time. For application all handles should be sharable, no matter in which order the objects were created. Only restriction would be on if pointers to data can be shared. For example, two processes could * always share a packet handle * but in odp-linux, could share pointer to packet data (return value of odp_packet_data()) only if the packet pool was created before both of those processes were forked (from a common ancestor) -Petri _______________________________________________ lng-odp mailing list lng-odp@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lng-odp
