----- On Nov 17, 2016, at 8:40 AM, Paul E. McKenney paul...@linux.vnet.ibm.com
wrote:
> On Thu, Nov 17, 2016 at 11:46:34AM +0000, Mathieu Desnoyers wrote:
>> ----- On Nov 17, 2016, at 1:51 AM, Lai Jiangshan jiangshan...@gmail.com
>> wrote:
>>
>> > On Fri, Nov 4, 2016 at 12:29 AM, Mathieu Desnoyers
>> > <mathieu.desnoy...@efficios.com> wrote:
>> >> Userspace applications should be allowed to expect the membarrier system
>> >> call with MEMBARRIER_CMD_SHARED command to issue memory barriers on
>> >> nohz_full CPUs, but synchronize_sched() does not take those into
>> >> account.
>> >>
>> >> Given that we do not want unrelated processes to be able to affect
>> >> real-time sensitive nohz_full CPUs, simply return ENOSYS when membarrier
>> >> is invoked on a kernel with enabled nohz_full CPUs.
>> >>
>> >> Signed-off-by: Mathieu Desnoyers <mathieu.desnoy...@efficios.com>
>> >> CC: "Paul E. McKenney" <paul...@linux.vnet.ibm.com>
>> >> CC: Josh Triplett <j...@joshtriplett.org>
>> >> CC: Steven Rostedt <rost...@goodmis.org>
>> >> CC: Lai Jiangshan <jiangshan...@gmail.com>
>> >> CC: <sta...@vger.kernel.org> [3.10+]
>> >> ---
>> >> kernel/membarrier.c | 4 ++++
>> >> 1 file changed, 4 insertions(+)
>> >>
>> >> diff --git a/kernel/membarrier.c b/kernel/membarrier.c
>> >> index 536c727..9f9284f 100644
>> >> --- a/kernel/membarrier.c
>> >> +++ b/kernel/membarrier.c
>> >> @@ -16,6 +16,7 @@
>> >>
>> >> #include <linux/syscalls.h>
>> >> #include <linux/membarrier.h>
>> >> +#include <linux/tick.h>
>> >>
>> >> /*
>> >> * Bitmask made from a "or" of all commands within enum membarrier_cmd,
>> >> @@ -51,6 +52,9 @@
>> >> */
>> >> SYSCALL_DEFINE2(membarrier, int, cmd, int, flags)
>> >> {
>> >> + /* MEMBARRIER_CMD_SHARED is not compatible with nohz_full. */
>> >> + if (tick_nohz_full_enabled())
>> >> + return -ENOSYS;
>> >
>> > I guess this code needs to be moved down into the branch of
>> > "case MEMBARRIER_CMD_SHARED" to match its comment.
>>
>> No, that would be unexpected from user-space. Either a system
>> call is implemented or not, not "implemented for some parameters".
>>
>> We also want MEMBARRIER_CMD_QUERY to return -ENOSYS in this case,
>> and all other parameter values to also return -ENOSYS (rather than
>> -EINVAL).
>>
>> If a system call that returns successfully on CMD_QUERY or EINVAL,
>> user-space may assume it will not have to handle ENOSYS in the
>> next calls.
>>
>>
>> >
>> > Acked-by: Lai Jiangshan <jiangshan...@gmail.com>
>> >
>> > But I'm afraid, in the future, tick_nohz_full will become a default y
>> > feature. thus it makes sys_membarrier() always disabled. we might
>> > need a new MEMBARRIER_CMD_XXX to handle it?
>>
>> This may require that we send an IPI to nohz_full CPUs, which will
>> disturb them real-time wise. Any better ideas ?
>
> Restrict the IPIs to CPUs running the process executing the
> sys_membarrier() system call. This would mean that CPUs only
> are interrupted by their own application's request.
This would break use-cases of cross-process shared memory. :-(
Mathieu
>
> Thanx, Paul
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
