Hi, as a developer of a product that uses DPDK I inherited a custom changeset towards DPDK 2.0 to allow better integration into our application stack. I am currently underway of updating the used DPDK release and would like to get rid of these local changes in the future by influencing the upstream development to some degree.
Concerning the integration we try to cope with the circumstance that DPDK is more comparable to an application framework and less to a library. The main part of the changes deal with EAL and specifically with rte_eal_init() and prune or replace functionality. The pruned features are in the responsibility of other proprietary libraries and modules within our application. For illustration purposes, I have included an old changeset towards v2.0.0. for reference, but due to the nature of the whole changeset it does not make sense to apply it directly, as there is a total lack of generality in the specific implementation (e.g. only the linuxapp part is customized, incomplete implemented concept in termination behavior ...). Nonetheless I am underway of researching concepts and approaches that are general enough for upstream inclusion. The main reason for attaching this patchset is to be able to discuss possible approaches of further development, as I would like to get feedback on possible development approaches to address the named problems. In detail the changes address the following areas: Logging behavior Opening a connection to the system logger for our program is within the responsibility of our specific logging module that is also used in the proprietary parts of the application. For this reason openlog() should not be called in our use case. In addition, using rte_openlog_stream() is not usable before rte_eal_init() as logging init will overwrite the used log stream. For this reason a large part of the init logs can not be handled by a custom log stream. Btw, after removal of the log caching, is there a fundamental difference between early log and normal logging? A possible approach for the future could be to initialize rte_logs.file to NULL (in fact it is, as it is static) and only set it during initialization if it is NULL with the goal to be able to use rte_openlog_stream() before rte_eal_init(). As the openlog() call is only relevant when the default log stream is used (and architecture dependent!) I introduced a specific entry point for calling openlog. The main complexity to this changeset is added due to two reasons. First reason is the circumstance that rte_eal_common_log_init() is used several times during early logging and real logging initialization. Second aspect is that a user might revert to the use of the default_log_stream after a custom log stream has be used straight from the beginning (even before rte_eal_init()). A dedicated changeset towards v16.07 for this improvement is attached for review, feedback and possible inclusion. Process termination behavior As the DPDK functionality is only a small part of the whole application ownership of the running process is not with DPDK and it is not allowed to cancel the running process as a result of an error. For this reason, the current changeset instead of calling rte_panic or rte_exit returns an error code and handles it in the calling function. Unfortunately this change was only implemented in rte_eal_init() and not in other places (drivers, libs ...), so there is currently no safety that an unknown code part terminates the whole application on the spot. Future changes and patches could change the error handling in a more thorough approach, but I would like to have some feedback and opinions before I start the heavy-lifting work in over 700 code locations. Even some interfaces might be affected, as lots of functions return void now, Thread creation In our application thread creation and setting the affinity is already handled in a proprietary module. Frames are polled and processed in non-EAL pthreads. For this reason the lcore thread creation in rte_eal_init() is completely removed as we do not want additional threads to be assigned to processing CPUs. Unfortunately setting the coremask to 0 is not feasible directly. For the non-EAL threads a custom function within the application sets the per_thread _lcore variable to the right value to try to enable mbuf cacheing. The changeset also disables the interrupt handling thread as in our application it currently seems not necessary (maybe make this optional?) In an experiment I removed the error check for the non-zero number of lcores and quick-fixed certain locations where rte_lcore_count() is used for memory size computations and at least got a running application. A possible approach could be to enhance enum rte_lcore_role_t with an auxiliary thread type and introduce additional eal arguments to assign lcores with these auxiliary role. I would like to receive feedback on possible development directions. Thank you, Steffen Bauch -- Steffen Bauch Senior Software Engineer Platform & Technology R&S Cybersecurity ipoque GmbH Augustusplatz 9, D-04109 Leipzig Phone: + 49 341 59403 0 Email: steffen.bauch at rohde-schwarz.com Internet: www.ipoque.com Trade register Amtsgericht Leipzig HRB21462 Gesellschaft mit beschr?nkter Haftung (GmbH) Dirk Czepluch - Managing Director