On 20 May 2015 at 13:21, Ola Liljedahl <ola.liljed...@linaro.org> wrote:
> On 20 May 2015 at 10:25, Maxim Uvarov <maxim.uva...@linaro.org> wrote: > > Hello Ola, > > > > please find some replays bellow. > > > > On 19 May 2015 at 01:03, Ola Liljedahl <ola.liljed...@linaro.org> wrote: > >> > >> As promised, here is my first attempt at a standalone API for IPC - > inter > >> process communication in a shared nothing architecture (message passing > >> between processes which do not share memory). > >> > >> Currently all definitions are in the file ipc.h but it is possible to > >> break out some message/event related definitions (everything from > >> odp_ipc_sender) in a separate file message.h. This would mimic the > >> packet_io.h/packet.h separation. > >> > >> The semantics of message passing is that sending a message to an > endpoint > >> will always look like it succeeds. The appearance of endpoints is > >> explicitly > >> notified through user-defined messages specified in the > odp_ipc_resolve() > >> call. Similarly, the disappearance (e.g. death or otherwise lost > >> connection) > >> is also explicitly notified through user-defined messages specified in > the > >> odp_ipc_monitor() call. The send call does not fail because the > addressed > >> endpoints has disappeared. > >> > >> Messages (from endpoint A to endpoint B) are delivered in order. If > >> message > >> N sent to an endpoint is delivered, then all messages <N have also been > >> delivered. Message delivery does not guarantee actual processing by the > >> recipient. End-to-end acknowledgements (using messages) should be used > if > >> this guarantee is important to the user. > >> > >> IPC endpoints can be seen as interfaces (taps) to an internal reliable > >> multidrop network where each endpoint has a unique address which is only > >> valid for the lifetime of the endpoint. I.e. if an endpoint is destroyed > >> and then recreated (with the same name), the new endpoint will have a > >> new address (eventually endpoints addresses will have to be recycled but > >> not for a very long time). Endpoints names do not necessarily have to be > >> unique. > >> > >> Signed-off-by: Ola Liljedahl <ola.liljed...@linaro.org> > >> --- > >> (This document/code contribution attached is provided under the terms of > >> agreement LES-LTM-21309) > >> > >> include/odp/api/ipc.h | 261 > >> ++++++++++++++++++++++++++++++++++++++++++++++++++ > >> 1 file changed, 261 insertions(+) > >> create mode 100644 include/odp/api/ipc.h > >> > >> diff --git a/include/odp/api/ipc.h b/include/odp/api/ipc.h > >> new file mode 100644 > >> index 0000000..3395a34 > >> --- /dev/null > >> +++ b/include/odp/api/ipc.h > >> @@ -0,0 +1,261 @@ > >> +/* Copyright (c) 2015, Linaro Limited > >> + * All rights reserved. > >> + * > >> + * SPDX-License-Identifier: BSD-3-Clause > >> + */ > >> + > >> + > >> +/** > >> + * @file > >> + * > >> + * ODP IPC API > >> + */ > >> + > >> +#ifndef ODP_API_IPC_H_ > >> +#define ODP_API_IPC_H_ > >> + > >> +#ifdef __cplusplus > >> +extern "C" { > >> +#endif > >> + > >> +/** @defgroup odp_ipc ODP IPC > >> + * @{ > >> + */ > >> + > >> +/** > >> + * @typedef odp_ipc_t > >> + * ODP IPC handle > >> + */ > >> + > >> +/** > >> + * @typedef odp_ipc_msg_t > >> + * ODP IPC message handle > >> + */ > >> + > >> + > >> +/** > >> + * @def ODP_IPC_ADDR_SIZE > >> + * Size of the address of an IPC endpoint > >> + */ > >> + > >> +/** > >> + * Create IPC endpoint > >> + * > >> + * @param name Name of local IPC endpoint > >> + * @param pool Pool for incoming messages > >> + * > >> + * @return IPC handle on success > >> + * @retval ODP_IPC_INVALID on failure and errno set > >> + */ > >> +odp_ipc_t odp_ipc_create(const char *name, odp_pool_t pool); > >> + > > > > > > Will it be separate pool or you want to attach existing pool to your ipc? > It should be a pool with events of type messages (ODP_EVENT_MESSAGE) > so in practice it will likely be a separate pool. > > > > > > >> > >> +/** > >> + * Destroy IPC endpoint > >> + * > >> + * @param ipc IPC handle > >> + * > >> + * @retval 0 on success > >> + * @retval <0 on failure > >> + */ > >> +int odp_ipc_destroy(odp_ipc_t ipc); > >> + > >> +/** > >> + * Set the default input queue for an IPC endpoint > >> + * > >> + * @param ipc IPC handle > >> + * @param queue Queue handle > >> + * > >> + * @retval 0 on success > >> + * @retval <0 on failure > >> + */ > >> +int odp_ipc_inq_setdef(odp_ipc_t ipc, odp_queue_t queue); > >> + > >> +/** > >> + * Remove the default input queue > >> + * > >> + * Remove (disassociate) the default input queue from an IPC endpoint. > >> + * The queue itself is not touched. > >> + * > >> + * @param ipc IPC handle > >> + * > >> + * @retval 0 on success > >> + * @retval <0 on failure > >> + */ > >> +int odp_ipc_inq_remdef(odp_ipc_t ipc); > >> + > >> +/** > >> + * Resolve endpoint by name > >> + * > >> + * Look up an existing or future endpoint by name. > >> + * When the endpoint exists, return the specified message with the > >> endpoint > >> + * as the sender. > >> + * > >> + * @param ipc IPC handle > >> + * @param name Name to resolve > >> + * @param msg Message to return > >> + */ > >> +void odp_ipc_resolve(odp_ipc_t ipc, > >> + const char *name, > >> + odp_ipc_msg_t msg); > >> + > >> +/** > >> + * Monitor endpoint > >> + * > >> + * Monitor an existing (potentially already dead) endpoint. > >> + * When the endpoint is dead, return the specified message with the > >> endpoint > >> + * as the sender. > >> + * > >> + * Unrecognized or invalid endpoint addresses are treated as dead > >> endpoints. > >> + * > >> + * @param ipc IPC handle > >> + * @param addr Address of monitored endpoint > >> + * @param msg Message to return > >> + */ > >> +void odp_ipc_monitor(odp_ipc_t ipc, > >> + const uint8_t addr[ODP_IPC_ADDR_SIZE], > >> + odp_ipc_msg_t msg); > >> + > >> +/** > >> + * Send message > >> + * > >> + * Send a message to an endpoint (which may already be dead). > >> + * Message delivery is ordered and reliable. All (accepted) messages > will > >> be > >> + * delivered up to the point of endpoint death or lost connection. > >> + * Actual reception and processing is not guaranteed (use end-to-end > >> + * acknowledgements for that). > >> + * Monitor the remote endpoint to detect death or lost connection. > >> + * > >> + * @param ipc IPC handle > >> + * @param msg Message to send > >> + * @param addr Address of remote endpoint > >> + * > >> + * @retval 0 on success > >> + * @retval <0 on error > >> + */ > >> +int odp_ipc_send(odp_ipc_t ipc, > >> + odp_ipc_msg_t msg, > >> + const uint8_t addr[ODP_IPC_ADDR_SIZE]); > >> + > >> +/** > >> + * Get address of sender (source) of message > >> + * > >> + * @param msg Message handle > >> + * @param addr Address of sender endpoint > >> + */ > >> +void odp_ipc_sender(odp_ipc_msg_t msg, > >> + uint8_t addr[ODP_IPC_ADDR_SIZE]); > >> + > >> +/** > >> + * Message data pointer > >> + * > >> + * Return a pointer to the message data > >> + * > >> + * @param msg Message handle > >> + * > >> + * @return Pointer to the message data > >> + */ > >> +void *odp_ipc_data(odp_ipc_msg_t msg); > >> + > >> +/** > >> + * Message data length > >> + * > >> + * Return length of the message data. > >> + * > >> + * @param msg Message handle > >> + * > >> + * @return Message length > >> + */ > >> +uint32_t odp_ipc_length(const odp_ipc_msg_t msg); > >> + > >> +/** > >> + * Set message length > >> + * > >> + * Set length of the message data. > >> + * > >> + * @param msg Message handle > >> + * @param len New length > >> + * > >> + * @retval 0 on success > >> + * @retval <0 on error > >> + */ > >> +int odp_ipc_reset(const odp_ipc_msg_t msg, uint32_t len); > > > > > > reset usually stand to reset to some default value. In that case it's > better > > to be odp_ipc_len_set(msg, len). > Well I copied this from packet.h where odp_packet_reset(pkt, len) > seemed to be equivalent to what I want to achieve but I agree that > odp_ipc_len_set() is more direct. I can change. > > > And reset is just void inline function to reset to default value > > odp_ipc_reset(msg). > > And len might be is not needed here because you transfer events. So just > get > The IPC mechanism transfers messages. And different messages have > different lengths. > > size from event. > The message size specified when creating a message pool that is > associated with an IPC endpoint is basically an MTU (the largest > message size that can be sent or received). > > > > > Might be something like: > > > > msg.event = ev; // len will be get from event > > msg.remote = addr; > I don't understand what you are trying to describe here. > > > > > > > > >> > >> + > >> +/** > >> + * Allocate message > >> + * > >> + * Allocate a message of a specific size. > >> + * > >> + * @param pool Message pool to allocate message from > >> + * @param len Length of the allocated message > >> + * > >> + * @return IPC message handle on success > >> + * @retval ODP_IPC_MSG_INVALID on failure and errno set > >> + */ > >> +odp_ipc_msg_t odp_ipc_alloc(odp_pool_t pool, uint32_t len); > >> + > > > > > > is len sizeof(odp_ipc_msg_t) here? > No. odp_ipc_msg_t is a handle to a message, just like odp_packet_t is > a handle to a packet. > The implementation of odp_ipc_msg_t is platform specific and thus > sizeof(odp_ipc_msg_t) as well (but 32 or 64 bits is a good guess). > > The len parameter in odp_ipc_alloc() specifies the current/actual > length of the message which depends on what the application wants to > use that specific message for. > > > > > > >> > >> +/** > >> + * Free message > >> + * > >> + * Free message back to the message pool it was allocated from. > >> + * > >> + * @param msg Handle of message to free > >> + */ > >> +void odp_ipc_free(odp_ipc_msg_t msg); > >> + > >> +/** > >> + * Get message handle from event > >> + * > >> + * Converts an ODP_EVENT_MESSAGE type event to a message. > >> + * > >> + * @param ev Event handle > >> + * > >> + * @return Message handle > >> + * > >> + * @see odp_event_type() > >> + */ > >> +odp_ipc_msg_t odp_message_from_event(odp_event_t ev); > >> + > >> +/** > >> + * Convert message handle to event > >> + * > >> + * @param msg Message handle > >> + * > >> + * @return Event handle > >> + */ > >> +odp_event_t odp_message_to_event(odp_ipc_msg_t msg); > >> + > > > > > > Most of functions look like pktio. But that version of IPC looks like > fully > > software implementation. > I think some SoC's could use HW mechanism to transfer messages (which > are basically buffers with individual lengths) between endpoints (e.g. > different address spaces). > > > Idea is good - send and receive odp events. > Send and receive ODP *messages*, not any type of ODP events. > > > It looks like odp_ipc_t ipc > > object also connected to > > some odp_pktio_t where actual packets will be sent. And it will be > > interesting to make queues interface > > work with that ipc. > > > > Might be: > > odp_pktio_t pktio can have ipc atribute like > > > > odp_ipc_pktio_set(odp_ipc_t ipc, odp_pktio_t pktio); > Now you are down to implementation details. Why should this be part of > the public API? > > > > > > then queue created for that pktio can send/recv packets from outside. > I think you can already do this with the current packet_io API. No > need to conflate this with IPC/message passing. > > > > > If I understand right, that functions use queue only for ipc messages: > > int odp_ipc_inq_setdef(odp_ipc_t ipc, odp_queue_t queue); > > int odp_ipc_inq_remdef(odp_ipc_t ipc); > You want to be able to receive and dispatch IPC messages just like any > other type of events so incoming messages should be put on queues. The > queue could be specified when the IPC endpoint is created but I copied > the packet_io API where it was possible. > > > and they are not linked with outside pktio. Earlier you said that > propose > > to send some packets to some internal tap device. > No not packets, messages. And the reference to "tap" was in a general > sense, not related to Linux tap interfaces. > > > And in that case tap > > device can be pktio. And then this pktio-tap connected to ipc object to > > dispatch events from it. > > > > > And probably we need better names for our ipc implementation. Mine can be > > odp_pktio_ipc and your can be odp_ipcmsg_ because the propose is > different. > Actually I don't understand why your packet pipes need to change the > ODP API. They are just pktio interfaces with different semantics. By > using different pktio device names (which anyway are platform > specific), you create packet pipes instead of opening physical network > interfaces. But they behave the same from the user perspective. Or? > > Yes in my case is only pipes and it's just pktio with different name. No difference to application how to use this pktio. > > > > > > > > > >> > >> +/** > >> + * Get printable value for an odp_ipc_t > >> + * > >> + * @param ipc IPC handle to be printed > >> + * @return uint64_t value that can be used to print/display this > >> + * handle > >> + * > >> + * @note This routine is intended to be used for diagnostic purposes > >> + * to enable applications to generate a printable value that represents > >> + * an odp_ipc_t handle. > >> + */ > >> +uint64_t odp_ipc_to_u64(odp_ipc_t ipc); > >> + > >> +/** > >> + * Get printable value for an odp_ipc_msg_t > >> + * > >> + * @param msg Message handle to be printed > >> + * @return uint64_t value that can be used to print/display this > >> + * handle > >> + * > >> + * @note This routine is intended to be used for diagnostic purposes > >> + * to enable applications to generate a printable value that represents > >> + * an odp_ipc_msg_t handle. > >> + */ > >> +uint64_t odp_ipc_msg_to_u64(odp_ipc_msg_t msg); > >> + > >> +/** > >> + * @} > >> + */ > >> + > >> +#ifdef __cplusplus > >> +} > >> +#endif > >> + > >> +#endif > >> -- > >> 1.9.1 > > > > > > > > After writing replay I came with idea why not build one pktio based on > other > > pktio? > > In that case eth0_pktio = ocp_create_pktio("eth0") can be used for > external > > packets exchange and > > ipc_eth0_pktio = odp_create_pktio(eth0_pktio, ..). > > So read from eth0_pktio will get only outside packets and read from > > ipc_eth0_pktio can deliver only > > ipc packets. > There are no IPC packets, just IPC messages. > > > In that case you do not need to implement transport layer and > > just do functionality above > > on current pktio. > IPC/message passing has different semantics (e.g. reliable/in-order) > from network interfaces. > IPC/message passing is for communication between local entities (e.g. > applications on the same host), not for communicating with external > entities over some physical network. > So I see several serious incompatibilities and I see no reason to > build IPC/message passing on top of some physical network interface > and its representation in ODP. > > Ok, I understand that it's exchange with some messages on one machine. I just not sure why conrtol plane and data plane be different host. Like ODL where OVS on one machine and controller on other. As I understand you don't what not transfer any odp events. Just ipc messages. Btw what is inside that messages? How that message looks like? Maxim. > > > > Thanks, > > Maxim. > > > > > > > > > > > >> > >> > >> _______________________________________________ > >> lng-odp mailing list > >> lng-odp@lists.linaro.org > >> https://lists.linaro.org/mailman/listinfo/lng-odp > > > > >
_______________________________________________ lng-odp mailing list lng-odp@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lng-odp