On Thu, May 16, 2019 at 10:58:04AM +0200, Roman Penyaev wrote: > Each ep_poll_callback() is called when fd calls wakeup() on epfd. > So account new event in user ring. > > The tricky part here is EPOLLONESHOT. Since we are lockless we > have to be deal with ep_poll_callbacks() called in paralle, thus > use cmpxchg to clear public event bits and filter out concurrent > call from another cpu. > > Signed-off-by: Roman Penyaev <[email protected]> > Cc: Andrew Morton <[email protected]> > Cc: Al Viro <[email protected]> > Cc: Linus Torvalds <[email protected]> > Cc: [email protected] > Cc: [email protected] > > diff --git a/fs/eventpoll.c b/fs/eventpoll.c > index 2f551c005640..55612da9651e 100644 > --- a/fs/eventpoll.c > +++ b/fs/eventpoll.c > @@ -1407,6 +1407,29 @@ struct file *get_epoll_tfile_raw_ptr(struct file > *file, int tfd, > } > #endif /* CONFIG_CHECKPOINT_RESTORE */ > > +/** > + * Atomically clear public event bits and return %true if the old value has > + * public event bits set. > + */ > +static inline bool ep_clear_public_event_bits(struct epitem *epi) > +{ > + __poll_t old, flags; > + > + /* > + * Here we race with ourselves and with ep_modify(), which can > + * change the event bits. In order not to override events updated > + * by ep_modify() we have to do cmpxchg. > + */ > + > + old = epi->event.events; > + do { > + flags = old; > + } while ((old = cmpxchg(&epi->event.events, flags, > + flags & EP_PRIVATE_BITS)) != flags); > + > + return flags & ~EP_PRIVATE_BITS; > +}
AFAICT epi->event.events also has normal writes to it, eg. in ep_modify(). A number of architectures cannot handle concurrent normal writes and cmpxchg() to the same variable.
