On Dienstag, 8. Februar 2022 19:28:21 CET Christian Schoenebeck wrote:
> On Dienstag, 8. Februar 2022 19:04:31 CET Will Cohen wrote:
> > On Tue, Feb 8, 2022 at 11:19 AM Will Cohen <wwco...@gmail.com> wrote:
> > > On Tue, Feb 8, 2022 at 11:11 AM Christian Schoenebeck <
> > >
> > > qemu_...@crudebyte.com> wrote:
> > >> On Dienstag, 8. Februar 2022 16:57:55 CET Will Cohen wrote:
> > >> > My inclination is to go with the __builtin_available(macOS 10.12, *)
> > >>
> > >> path,
> > >>
> > >> > if acceptable, since it partially mirrors the API_AVAILABLE macro
> > >> > idea.
> > >>
> > >> I
> > >>
> > >> OTOH that's duplication of the ">= macOS 10.12" info, plus
> > >> __builtin_available
> > >> is direct use of a clang-only extension, whereas API_AVAILABLE() works
> > >> (or
> > >> more precisely: doesn't error out at least) with other compilers like
> > >> GCC
> > >> as
> > >> well. GCC is sometimes used for cross-compilation.
> > >>
> > >> Moreover, I would also add an error message in this case, e.g.:
> > >> if (!pthread_fchdir_np) {
> > >>
> > >> error_report_once("pthread_fchdir_np() is not available on this
> > >>
> > >> macOS version");
> > >>
> > >> return -ENOTSUPP;
> > >>
> > >> }
> > >>
> > >> I should elaborate why I think this is needed: you are already doing a
> > >> Meson
> > >> check for the existence of pthread_fchdir_np(), but the system where
> > >> QEMU
> > >> is
> > >> compiled and the systems where the compiled binary will be running,
> > >> might
> > >> be
> > >> different ones (i.e. different macOS versions).
> > >>
> > >> Best regards,
> > >> Christian Schoenebeck
> > >
> > > Agreed, that way actually closes the edge case. Something along these
> > > lines briefly crossed my mind during a previous version, but I quickly
> > > got
> > > passed it by assuming that the compiling entity would always be the
> > > bottleneck, which makes no sense in hindsight, so I very much appreciate
> > > that you caught this.
> >
> > Ah, rebuilding leads to a compiler error:
> >
> > ../os-posix.c:348:10: warning: address of function 'pthread_fchdir_np'
> > will
> > always evaluate to 'true' [-Wpointer-bool-conversion]
> >
> > if (!pthread_fchdir_np) {
> >
> > ~^~~~~~~~~~~~~~~~~
> >
> > I don't have a machine that's pre-10.12 so I can't see what the result is
> > there, but this might be why the __builtin_available approach got taken.
>
> I guess that's because you are compiling QEMU with minimum deployment target
> being macOS >= 10.12 already. In this case the compiler won't make
> pthread_fchdir_np a weak link, it only does emit a weak link if you are
> targeting macOS versions prior than the defined availablity attribute,
> hence the address would never be NULL here and hence the compiler warning.
>
> So I guess it is okay if you just omit checking presence of
> pthread_fchdir_np at runtime and just assume it exists.
>
> Added Akihiko on CC, just in case he would have something to add on this
> macOS issue here. :)
On a second thought: this case a bit special. Are we worried that
pthread_fchdir_np() is "not yet" available on macOS, or "no longer" available.
Probably both, right?
So maybe it would make sense to replace the API_AVAILABLE() attribute directly
with a __attribute__((weak)) attribute. Then the runtime check with the
proposed error message would also trigger if a bleeding edge macOS version no
longer has pthread_fchdir_np().
Also keep in mind: there are always also the MAC_OS_X_VERSION_MIN_REQUIRED and
MAC_OS_X_VERSION_MAX_ALLOWED macros to query the deployment target at compile
time to wrap deployment target dependent code accordingly.
On doubt you could just make some tests there by simply "inventing" a non-
existent function.
Best regards,
Christian Schoenebeck
> > >> > guess it's perhaps a tradeoff between predicting the future unknown
> > >> > availability of functions versus just ensuring a minimum macOS
> > >> > version
> > >>
> > >> and
> > >>
> > >> > hoping for the best. With any luck, the distinction between the two
> > >> > approaches will be moot, if we try to assume that a future macOS
> > >> > version
> > >> > that removes this also provides mknodat.
> > >> >
> > >> > On Tue, Feb 8, 2022 at 10:03 AM Christian Schoenebeck <
> > >> >
> > >> > qemu_...@crudebyte.com> wrote:
> > >> > > On Dienstag, 8. Februar 2022 14:36:42 CET Will Cohen wrote:
> > >> > > > On Mon, Feb 7, 2022 at 5:56 PM Christian Schoenebeck
> > >> > > > <qemu_...@crudebyte.com>
> > >> > > >
> > >> > > > wrote:
> > >> > > > > On Montag, 7. Februar 2022 23:40:22 CET Will Cohen wrote:
> > >> > > > > > From: Keno Fischer <k...@juliacomputing.com>
> > >> > > > > >
> > >> > > > > > Darwin does not support mknodat. However, to avoid race
> > >>
> > >> conditions
> > >>
> > >> > > > > > with later setting the permissions, we must avoid using mknod
> > >> > > > > > on
> > >> > > > > > the full path instead. We could try to fchdir, but that would
> > >>
> > >> cause
> > >>
> > >> > > > > > problems if multiple threads try to call mknodat at the same
> > >>
> > >> time.
> > >>
> > >> > > > > > However, luckily there is a solution: Darwin includes a
> > >> > > > > > function
> > >> > > > > > that sets the cwd for the current thread only.
> > >> > > > > > This should suffice to use mknod safely.
> > >> > > > > >
> > >> > > > > > This function (pthread_fchdir_np) is protected by a check in
> > >> > > > > > meson in a patch later in tihs series.
> > >> > > > > >
> > >> > > > > > Signed-off-by: Keno Fischer <k...@juliacomputing.com>
> > >> > > > > > Signed-off-by: Michael Roitzsch <reactorcont...@icloud.com>
> > >> > > > > > [Will Cohen: - Adjust coding style
> > >> > > > > >
> > >> > > > > > - Replace clang references with gcc
> > >> > > > > > - Note radar filed with Apple for missing
> > >> > > > > > syscall
> > >> > > > > > - Replace direct syscall with pthread_fchdir_np
> > >> > > > > > and
> > >> > > > > >
> > >> > > > > > adjust patch notes accordingly
> > >> > > > > >
> > >> > > > > > - Move qemu_mknodat from 9p-util to osdep and
> > >>
> > >> os-posix]
> > >>
> > >> > > > > > Signed-off-by: Will Cohen <wwco...@gmail.com>
> > >> > > > > > ---
> > >> > > > >
> > >> > > > > Like already mentioned by me moments ago on previous v4 (just
> > >>
> > >> echoing)
> > >>
> > >> > > ...
> > >> > >
> > >> > > > > > hw/9pfs/9p-local.c | 4 ++--
> > >> > > > > > include/qemu/osdep.h | 10 ++++++++++
> > >> > > > > > os-posix.c | 34 ++++++++++++++++++++++++++++++++++
> > >> > > > > > 3 files changed, 46 insertions(+), 2 deletions(-)
> > >> > > > > >
> > >> > > > > > diff --git a/hw/9pfs/9p-local.c b/hw/9pfs/9p-local.c
> > >> > > > > > index a0d08e5216..d42ce6d8b8 100644
> > >> > > > > > --- a/hw/9pfs/9p-local.c
> > >> > > > > > +++ b/hw/9pfs/9p-local.c
> > >> > > > > > @@ -682,7 +682,7 @@ static int local_mknod(FsContext *fs_ctx,
> > >> > >
> > >> > > V9fsPath
> > >> > >
> > >> > > > > > *dir_path,
> > >> > > > > >
> > >> > > > > > if (fs_ctx->export_flags & V9FS_SM_MAPPED ||
> > >> > > > > >
> > >> > > > > > fs_ctx->export_flags & V9FS_SM_MAPPED_FILE) {
> > >> > > > > >
> > >> > > > > > - err = mknodat(dirfd, name, fs_ctx->fmode | S_IFREG,
> > >> > > > > > 0);
> > >> > > > > > + err = qemu_mknodat(dirfd, name, fs_ctx->fmode |
> > >>
> > >> S_IFREG,
> > >>
> > >> > > > > > 0);
> > >> > > > > >
> > >> > > > > > if (err == -1) {
> > >> > > > > >
> > >> > > > > > goto out;
> > >> > > > > >
> > >> > > > > > }
> > >> > > > > >
> > >> > > > > > @@ -697,7 +697,7 @@ static int local_mknod(FsContext *fs_ctx,
> > >> > >
> > >> > > V9fsPath
> > >> > >
> > >> > > > > > *dir_path, }
> > >> > > > > >
> > >> > > > > > } else if (fs_ctx->export_flags & V9FS_SM_PASSTHROUGH ||
> > >> > > > > >
> > >> > > > > > fs_ctx->export_flags & V9FS_SM_NONE) {
> > >> > > > > >
> > >> > > > > > - err = mknodat(dirfd, name, credp->fc_mode,
> > >>
> > >> credp->fc_rdev);
> > >>
> > >> > > > > > + err = qemu_mknodat(dirfd, name, credp->fc_mode,
> > >> > > > > > credp->fc_rdev);
> > >> > > > > >
> > >> > > > > > if (err == -1) {
> > >> > > > > >
> > >> > > > > > goto out;
> > >> > > > > >
> > >> > > > > > }
> > >> > > > > >
> > >> > > > > > diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
> > >> > > > > > index d1660d67fa..f3a8367ece 100644
> > >> > > > > > --- a/include/qemu/osdep.h
> > >> > > > > > +++ b/include/qemu/osdep.h
> > >> > > > > > @@ -810,3 +810,13 @@ static inline int
> > >> > > > > > platform_does_not_support_system(const char *command) #endif
> > >> > > > > >
> > >> > > > > > #endif
> > >> > > > > >
> > >> > > > > > +
> > >> > > > > > +/*
> > >> > > > > > + * As long as mknodat is not available on macOS, this
> > >>
> > >> workaround
> > >>
> > >> > > > > > + * using pthread_fchdir_np is needed. qemu_mknodat is
> > >> > > > > > defined
> > >>
> > >> in
> > >>
> > >> > > > > > + * os-posix.c
> > >> > > > > > + */
> > >> > > > > > +#ifdef CONFIG_DARWIN
> > >> > > > > > +int pthread_fchdir_np(int fd);
> > >> > > > > > +#endif
> > >> > > > >
> > >> > > > > I would make that:
> > >> > > > >
> > >> > > > > #ifdef CONFIG_DARWIN
> > >> > > > > int pthread_fchdir_np(int fd) API_AVAILABLE(macosx(10.12));
> > >> > > > > #endif
> > >> > > > >
> > >> > > > > here and ...
> > >> > > > >
> > >> > > > > > +int qemu_mknodat(int dirfd, const char *filename, mode_t
> > >> > > > > > mode,
> > >> > > > > > dev_t
> > >> > > > >
> > >> > > > > dev);
> > >> > > > >
> > >> > > > > > diff --git a/os-posix.c b/os-posix.c
> > >> > > > > > index ae6c9f2a5e..95c1607065 100644
> > >> > > > > > --- a/os-posix.c
> > >> > > > > > +++ b/os-posix.c
> > >> > > > > > @@ -24,6 +24,7 @@
> > >> > > > > >
> > >> > > > > > */
> > >> > > > > >
> > >> > > > > > #include "qemu/osdep.h"
> > >> > > > > >
> > >> > > > > > +#include <os/availability.h>
> > >> > > > > >
> > >> > > > > > #include <sys/wait.h>
> > >> > > > > > #include <pwd.h>
> > >> > > > > > #include <grp.h>
> > >> > > > > >
> > >> > > > > > @@ -332,3 +333,36 @@ int os_mlock(void)
> > >> > > > > >
> > >> > > > > > return -ENOSYS;
> > >> > > > > >
> > >> > > > > > #endif
> > >> > > > > > }
> > >> > > > > >
> > >> > > > > > +
> > >> > > > > > +/*
> > >> > > > > > + * As long as mknodat is not available on macOS, this
> > >>
> > >> workaround
> > >>
> > >> > > > > > + * using pthread_fchdir_np is needed.
> > >> > > > > > + *
> > >> > > > > > + * Radar filed with Apple for implementing mknodat:
> > >> > > > > > + * rdar://FB9862426
> > >> > > > > > (https://openradar.appspot.com/FB9862426)
> > >> > > > > > + */
> > >> > > > > > +#ifdef CONFIG_DARWIN
> > >> > > > > > +
> > >> > > > > > +int pthread_fchdir_np(int fd) API_AVAILABLE(macosx(10.12));
> > >> > > > >
> > >> > > > > ... drop the duplicate declaration of pthread_fchdir_np() here.
> > >> > > >
> > >> > > > Trying this out, it reminds me that this use of API_AVAILABLE in
> > >> > >
> > >> > > os-posix.c
> > >> > >
> > >> > > > relies on the added #include <os/availability.h>.
> > >> > > >
> > >> > > > Leaving the include out leads to:
> > >> > > > .../include/qemu/osdep.h:820:31: error: expected function body
> > >> > > > after
> > >> > > > function declarator
> > >> > > > int pthread_fchdir_np(int fd) API_AVAILABLE(macosx(10.12));
> > >> > > >
> > >> > > > ^
> > >> > > >
> > >> > > > 1 error generated.
> > >> > > > ninja: build stopped: subcommand failed.
> > >> > > > make[1]: *** [run-ninja] Error 1
> > >> > > > make: *** [all] Error 2
> > >> > > >
> > >> > > > The admonition against modifying osdep.h's includes too much led
> > >> > > > me
> > >>
> > >> to
> > >>
> > >> > > > steer away from putting it all in there. If there's no issue with
> > >>
> > >> adding
> > >>
> > >> > > > +#include <os/availability.h> to osdep.h, then this change makes
> > >>
> > >> sense
> > >>
> > >> > > > to
> > >> > > > me.
> > >> > >
> > >> > > If you embed that include into ifdefs, sure!
> > >> > >
> > >> > > #ifdef CONFIG_DARWIN
> > >> > > /* defines API_AVAILABLE(...) */
> > >> > > #include <os/availability.h>
> > >> > > #endif
> > >> > >
> > >> > > One more thing though ...
> > >> > >
> > >> > > > > > +
> > >> > > > > > +int qemu_mknodat(int dirfd, const char *filename, mode_t
> > >> > > > > > mode,
> > >> > > > > > dev_t
> > >> > > > >
> > >> > > > > dev)
> > >> > > > >
> > >> > > > > > +{
> > >> > > > > > + int preserved_errno, err;
> > >> > >
> > >> > > pthread_fchdir_np() is weakly linked. So I guess here should be a
> > >>
> > >> check
> > >>
> > >> > > like:
> > >> > > if (!pthread_fchdir_np) {
> > >> > >
> > >> > > return -ENOTSUPP;
> > >> > >
> > >> > > }
> > >> > >
> > >> > > Before trying to call pthread_fchdir_np() below. As already
> > >> > > discussed
> > >>
> > >> with
> > >>
> > >> > > the
> > >> > > Chromium [1] example, some do that a bit differently by using
> > >> > >
> > >> > > __builtin_available():
> > >> > > if (__builtin_available(macOS 10.12, *)) {
> > >> > >
> > >> > > return -ENOTSUPP;
> > >> > >
> > >> > > }
> > >> > >
> > >> > > Which makes me wonder why they are not doing a simple NULL check?
> > >> > >
> > >> > > [1]
> > >>
> > >> https://chromium.googlesource.com/chromium/src/+/lkgr/base/process/laun
> > >> ch
> > >> _
> > >>
> > >> > > mac.cc#110>
> > >> > >
> > >> > > > > > + if (pthread_fchdir_np(dirfd) < 0) {
> > >> > > > > > + return -1;
> > >> > > > > > + }
> > >> > > > > > + err = mknod(filename, mode, dev);
> > >> > > > > > + preserved_errno = errno;
> > >> > > > > > + /* Stop using the thread-local cwd */
> > >> > > > > > + pthread_fchdir_np(-1);
> > >> > > > > > + if (err < 0) {
> > >> > > > > > + errno = preserved_errno;
> > >> > > > > > + }
> > >> > > > > > + return err;
> > >> > > > > > +}
> > >> > > > > > +#else
> > >> > > > > > +int qemu_mknodat(int dirfd, const char *filename, mode_t
> > >> > > > > > mode,
> > >> > > > > > dev_t
> > >> > > > >
> > >> > > > > dev)
> > >> > > > >
> > >> > > > > > +{
> > >> > > > > > + return mknodat(dirfd, filename, mode, dev);
> > >> > > > > > +}
> > >> > > > > > +#endif
>
> Best regards,
> Christian Schoenebeck
Best regards,
Christian Schoenebeck
