The man page says: If O_APPEND is set for iocb->aio_fildes, write operations append to the file in the same order as the calls were made.
https://man.freebsd.org/cgi/man.cgi?query=aio_write But in practice the kernel threads seem to pick up operations from the queue regardless of the flag and cause operation reordering. This leads to logs reordering, or even interleaving if a single log requires more than one aio_write. And that leads to test failures in FreeBSD CI for tests that rely on specific log output. It seems a little excessive to just disable POSIX AIO support on FreeBSD entirely, as the reordering is typically not too severe under normal circumstances, while logging synchronously would cause delays in processing. So, let's just add a knob to disable it and use this knob in our CI. Not mentioning this knob in the NEWS file, as we don't really want people to use it outside of the CI environment at this time. Signed-off-by: Ilya Maximets <[email protected]> --- The intention here is to apply this to all active branches to make CI more stable. There are a few flaky tests for reasons other than the log reordering, but those pass on rerun in most cases, though I'm looking into making them stable. .ci/freebsd-build.sh | 6 +++++- m4/openvswitch.m4 | 23 +++++++++++++++++++++-- 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/.ci/freebsd-build.sh b/.ci/freebsd-build.sh index 25ec93c9b..dc924c05d 100755 --- a/.ci/freebsd-build.sh +++ b/.ci/freebsd-build.sh @@ -55,7 +55,11 @@ freebsd_wait_firstboot 30 5 freebsd_ssh "mkdir -p /root/ovs" freebsd_rsync_to "$(pwd)/" /root/ovs/ +# POSIX async I/O is disabled because FreeBSD doesn't honor the order of +# aio_write operations on the same O_APPEND file descriptor causing log +# reordering, even though the man page says otherwise. freebsd_ssh "cd /root/ovs && ./boot.sh && \ - ./configure CC=${CC} CFLAGS='-g -O2 -Wall' MAKE=gmake --enable-Werror" + ./configure CC=${CC} CFLAGS='-g -O2 -Wall' MAKE=gmake \ + --disable-posix-aio --enable-Werror" freebsd_ssh "cd /root/ovs && gmake -j8 check TESTSUITEFLAGS=-j8 RECHECK=yes" diff --git a/m4/openvswitch.m4 b/m4/openvswitch.m4 index f9b73945b..92daf8dfd 100644 --- a/m4/openvswitch.m4 +++ b/m4/openvswitch.m4 @@ -448,8 +448,27 @@ AC_DEFUN([OVS_CHECK_ATOMIC_ALWAYS_LOCK_FREE], dnl OVS_CHECK_POSIX_AIO AC_DEFUN([OVS_CHECK_POSIX_AIO], - [AC_SEARCH_LIBS([aio_write], [rt]) - AM_CONDITIONAL([HAVE_POSIX_AIO], [test "$ac_cv_search_aio_write" != no])]) + [AC_ARG_ENABLE( + [posix-aio], + [AS_HELP_STRING([--disable-posix-aio], + [Disable POSIX asynchronous I/O for logging])], + [case "${enableval}" in + (yes) posix_aio=true ;; + (no) posix_aio=false ;; + (*) AC_MSG_ERROR([bad value ${enableval} for --enable-posix-aio]) ;; + esac], + [posix_aio=check]) + + if test "$posix_aio" != false; then + AC_SEARCH_LIBS([aio_write], [rt]) + fi + + if test "$posix_aio" = true && test "$ac_cv_search_aio_write" = no; then + AC_MSG_ERROR([POSIX AIO support requested, but aio_write not found]) + fi + + AM_CONDITIONAL([HAVE_POSIX_AIO], + [test "$posix_aio" != false && test "$ac_cv_search_aio_write" != no])]) dnl OVS_CHECK_INCLUDE_NEXT AC_DEFUN([OVS_CHECK_INCLUDE_NEXT], -- 2.54.0 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
