>>> The relay function is called by 'ovs_keepalive' thread. I didn't >>> completely understand your concerns here. I would be happy to verify >>> if you have any scenarios in mind that you think pose problems. >> >>My concern stems from this: >> "is relayed to monitoring framework by unlocking the semaphore." >> >>Assume the OvS vswitchd crashes while another process is pended on this >>locked semaphore. Since you do a shm_unlink, I think the pended >>process waiting on that semaphore will be stuck. Even worse, since we >>do have a process left with a handle, the name will be unlinked, and >>the memory and reading process (which may be ceilometer, but may be >>something else) will be left orphaned waiting for an update so that they can >destroy it. >> >>There's a lot of coordination that needs to be added here, and it's >>likely not portable. I'm not even sure if there are examples where it >>has been shown to work 100% on specific systems. >> >>A much better design is to use a socket to share state. Since it is >>tied to the lifetime of a process, and cleanup is guaranteed by POSIX >>(and all sane OSes, >>anyway) you can rely on it. It is a much better mechanism for >>signaling changes, and a message based interface means you can make >>this dump information to _any_ interested subscriber - even those who >>are off system if you choose. >>
Hello Aaron, I got a chance to talk to collectd team and understood that the 'dpdkevents' plugin of collectd implementation changed a bit which I wasn't aware of when I posted v1 of this patch. In the new implementation semaphore isn't needed. How it works is: - DPDK primary process(OvS-DPDK in our case) initializes the shared memory during startup and keeps writing the events. - dpdkevents plugin of collectd would read the events from SHM block and in case of change in 'core state' notify the ceilometer. - when the OvS crashes, it would automatically be restarted(ovs-vswitchd --monitor ) and as part of initialization the shared memory would be reinitialized(flags=O_CREAT | O_TRUNC | O_RDWR, 0666). - dpdkevents plugin will call "dpdk_events_read()" at every predefined interval and shall open the SHM block and get the file descriptor. It goes on to check if there is any change in 'inode'. If there is no change in 'inode' it means the OvS is intact and healthy. In case of change in inode(OvS has restarted), it would immediately unmap the memory, close the stale file descriptor and memory map the shared memory using the new file descriptor and read the updated events. In this way we can avoid semaphore and also make sure the collectd would detect the primary application crashes. I tested this and found to be working in different scenarios. Please note that I was sending SIGABRT to OvS and started OvS using 'ovs-vswitch --monitor' option to restart the OvS automatically. Sorry for taking so long to reply, some of the above mentioned changes in collectd are still WIP and are in the process of getting upstreamed. Regards, Bhanuprakash. > >Thanks Aaron for clarifying this and appreciate all your feedback on this patch >series. >I shall try out the scenarios leading to unexpected crashes and see how well >the applications recover on a restart and check for any further shortcomings. > >- Bhanuprakash. _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
