Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
Hi On Mon, Sep 15, 2014 at 4:22 PM, John Haxby wrote: > I really appreciate proper explanations, thank you. Quite simple: We want to avoid calling into udev and back into the kernel if we there's a way to skip this step. That is, the default should work without udev. Reasons: * We want init=/bin/bash or anything else to work. Or more precisely: I don't care for init=/bin/bash, but others do. Lets not make their life miserable, but make defaults work with init=/bin/bash if possible * You can use your new shiny kernel with CPU hotplugging together with an old udev user-space and the default will just work. Same is true for any other new bus type. But if we require udev rules for the _default_ to work, you need to install udev rules (or update udev respectively) just to make the default work. That already sounds backwards, doesn't it? Furthermore, this seems cumbersome and kinda makes it no longer the "default", right? * It is wasted time and wasted resources to call into user-space to trigger the default action. If something is the default, it should just happen and not require a bunch of messages between different services. Instead, the opposite should be true: If you don't want the default actions, tell the kernel about it and only then you should get hooked into the decision. Imagine all decisions made by the kernel go through udev. It would slow down systems considerably. So lets keep it down unless there's strong reason to send it through udev. * We want people to chose *sane* defaults. Everywhere. So if the kernel default is *sane*, why do we need an udev rule? Obviously, there're exceptions. Mostly backwards compatibility and interactions with totally backwards subsystems. But if there's a chance to reduce the amount of default udev rules, I bet you get praised for each and every properly removed rule. Does that make sense? Maybe Kay has additions/corrections, but I think this explanation should be fairly straightforward. Thanks David ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On 15/09/14 15:02, David Herrmann wrote:
> Hi
>
> On Mon, Sep 15, 2014 at 3:19 PM, John Haxby wrote:
>> On 12/09/14 16:03, Kay Sievers wrote:
>>> On Fri, Sep 12, 2014 at 3:04 PM, John Haxby wrote:
When I think of changing the
behaviour of any removable hardware, udev is automatically where I look
first. So if I'm missing something here, I would really like some more
input.
>>>
>>> Udev does not decide which device show up, it just classifies stuff
>>> that processes events.
>>
>> That's what I don't understand.
>>
>> Let's say I plug in a USB printer. Looking through the udev rules on my
>> machine I see
>>
>> SUBSYSTEM=="printer", TAG+="systemd"
>>ENV{SYSTEMD_WANTS}+="printer.target"
>>
>> (line split for readability).A whileback I proposed
>>
>>ACTION==“add” KERNEL==“cpu[0-9]*” TAG+="systemd"
>>ENV{SYSTEMD_WANTS}+=“cpuadd.service”
>>
>> Aside from the obvious syntactic differences, what's the difference
>> between these?
>
> One is a target, the other is a service. There is a fundamental
> difference between both. A target does not cause an immediate action,
> a service does. The target just hooks up the udev event into systemd,
> it does not associate any default action. The cpuadd.service, on the
> other hand, does add a default action.
>
>> Does it make any difference if the default is not to bring online more
>> CPUs that the system booted with? Or some other clever default?
>>
>> Why does the kernel not bring CPUs online automatically? This must have
>> come up before, and what you have been saying seems to suggest that
>> there is something going on here that I am just missing.
>
> The idea is to add a kernel runtime option (like sysctl) which is
> named "cpu_auto_on" or something like that. Or make it a sysfs file in
> /sys/bus/cpu/ just like "drivers_autoprobe". This should be set to "1"
> by default, thus, all CPUs get activated without any udev interaction
> (if you want to avoid races during bootup, use the kernel
> command-line.. or a static kernel config option..).
> Everybody else is free to set this option to "0" and write their own
> udev rules. This way, udev does not have to bother with default
> actions.
>
> Is there anything wrong with this proposed solution? Or why do you
> insist on your solution?
Forgive me, I'm not insisting on anything. It was me that that said
"hang on, we need to push that fix upstream" a while ago precisely to
ensure that we got the right solution for the right reasons.
I really appreciate proper explanations, thank you.
>
> Thanks
> David
>
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
Hi
On Mon, Sep 15, 2014 at 3:19 PM, John Haxby wrote:
> On 12/09/14 16:03, Kay Sievers wrote:
>> On Fri, Sep 12, 2014 at 3:04 PM, John Haxby wrote:
>>> When I think of changing the
>>> behaviour of any removable hardware, udev is automatically where I look
>>> first. So if I'm missing something here, I would really like some more
>>> input.
>>
>> Udev does not decide which device show up, it just classifies stuff
>> that processes events.
>
> That's what I don't understand.
>
> Let's say I plug in a USB printer. Looking through the udev rules on my
> machine I see
>
> SUBSYSTEM=="printer", TAG+="systemd"
>ENV{SYSTEMD_WANTS}+="printer.target"
>
> (line split for readability).A whileback I proposed
>
>ACTION==“add” KERNEL==“cpu[0-9]*” TAG+="systemd"
>ENV{SYSTEMD_WANTS}+=“cpuadd.service”
>
> Aside from the obvious syntactic differences, what's the difference
> between these?
One is a target, the other is a service. There is a fundamental
difference between both. A target does not cause an immediate action,
a service does. The target just hooks up the udev event into systemd,
it does not associate any default action. The cpuadd.service, on the
other hand, does add a default action.
> Does it make any difference if the default is not to bring online more
> CPUs that the system booted with? Or some other clever default?
>
> Why does the kernel not bring CPUs online automatically? This must have
> come up before, and what you have been saying seems to suggest that
> there is something going on here that I am just missing.
The idea is to add a kernel runtime option (like sysctl) which is
named "cpu_auto_on" or something like that. Or make it a sysfs file in
/sys/bus/cpu/ just like "drivers_autoprobe". This should be set to "1"
by default, thus, all CPUs get activated without any udev interaction
(if you want to avoid races during bootup, use the kernel
command-line.. or a static kernel config option..).
Everybody else is free to set this option to "0" and write their own
udev rules. This way, udev does not have to bother with default
actions.
Is there anything wrong with this proposed solution? Or why do you
insist on your solution?
Thanks
David
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On 12/09/14 16:03, Kay Sievers wrote:
> On Fri, Sep 12, 2014 at 3:04 PM, John Haxby wrote:
>> On 02/09/14 16:42, Kay Sievers wrote:
> Either the kernel has to provide a mechanism for the userspace to
> control onlining, or do it itself and provide a mechanism to prevent
> automatic onlining. I think that the first option is actually
> cleaner. So yeah, let's add the original rule which does it
> unconditionally,
>>> No, as outlined already, we are not doing this. It is just wrong.
>>>
> and people who have too many cpus and have special
> needs can provide a custom udev rule which does something different.
>
> cpuadd.service or cpuadd@.service seem overkill, since the one we
> would provide would still do the exact same thing: unconditionally
> enable the CPU.
>>> Nothing wrong with that, if people need that. But this can for sure
>>> not live in systemd/udev, it is not the right place.
>>
>> Kay, I've worked on a case with Xen domUs which have only some of their
>> virtual CPUs online at any given time. This deployment has scripts which
>> only turn up extra CPUs when one of the applications has to do a
>> failover from one virtual machine to another. The point is that needed
>> CPU hotplug behaviour is not necessarily uniform to everyone.
>>
>> Even if the default, out of the box, behaviour is to turn up the CPUs
>> unconditionally, udev seems the best place to change this behaviour at
>> runtime. The fact that CPUs can be hotplugged at all should imply that
>> some users will need different behaviour than others. Here, the default
>> action is almost a trivial configuration... but not the only possible
>> desired configuration.
>>
>> Can I ask your reasoning for CPU hotplug behaviour not being the role of
>> udev to fulfill? If that's not the right place, where do you believe
>> would be the appropriate alternative?
>
> As explained several times, there is no point in mis-using udev to
> unconditionally react to kernel events to trigger things in the kernel
> again. That is not how driver/device binding works for all the other
> subsystems on Linux.
>
I can't find an explanation, just "this is not going to happen". Also,
we are proposing a mechanism by which a policy for deciding whether CPUs
should be placed on line can be implemented. Possible choices range
from "no, not yet" (more-or-less the above) through licensing
constraints, topological considerations all the way to "just give me
everything". We have to pick something for the default and probably the
least surprising for the end user is just to bring the CPU on line. The
present behaviour is perhaps the most surprising: completely ignore
hot-plugged CPUs.
>> I'm hoping you have another place in mind.
>
> The kernel itself or any other custom facility. This stuff has no
> place in default/upstream udev, it is the wrong way to do things in
> default setups.
So we need to introduce yet another mechanism for one particular class
of hotplug device? I must admit I'm at a loss to see why hotplug CPUs
are so different for everything else that can be hotplugged.
Could you explain why udev shouldn't do this and what makes hotplug CPUs
so different?I must be missing something really obvious here.
>
>> When I think of changing the
>> behaviour of any removable hardware, udev is automatically where I look
>> first. So if I'm missing something here, I would really like some more
>> input.
>
> Udev does not decide which device show up, it just classifies stuff
> that processes events.
That's what I don't understand.
Let's say I plug in a USB printer. Looking through the udev rules on my
machine I see
SUBSYSTEM=="printer", TAG+="systemd"
ENV{SYSTEMD_WANTS}+="printer.target"
(line split for readability).A whileback I proposed
ACTION==“add” KERNEL==“cpu[0-9]*” TAG+="systemd"
ENV{SYSTEMD_WANTS}+=“cpuadd.service”
Aside from the obvious syntactic differences, what's the difference
between these?
Does it make any difference if the default is not to bring online more
CPUs that the system booted with? Or some other clever default?
Why does the kernel not bring CPUs online automatically? This must have
come up before, and what you have been saying seems to suggest that
there is something going on here that I am just missing.
jch
PS I'm not (yet) subscribed to systemd-devel, please make sure you keep
the cc's.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
Hi On Fri, Sep 12, 2014 at 5:48 PM, Todd Vierling wrote: > What we do with those events is udev's rule configuration. Some of > those actions do involve telling the kernel to do something in > addition to the original event; see the two examples above (though > these are not the only ones in common use, obviously). If there were > no need to configure differing reactions to kernel events, rules.d > wouldn't exist. > > All that is being proposed here is to offer a default reaction to a > CPU add event -- the configuration for which can be changed by a > system administrator, distro, or tool just by changing a text file, > rather than recompiling a kernel. We don't oppose cpu-hotplug rules, we just want the default to work _without_ rules. Nobody tells you to recompile your kernel to change the behavior. We just want the kernel to have sane defaults and then people can add custom udev scripts on top. We don't have udev rules to probe drivers on devices. It this is the _default_ behavior (auto-probing). However, you can use udev to prevent that and add custom rules to bind custom drivers. Why can't you do the same for CPUs? It is ridiculous to call into user-space just to call back into the kernel without doing _anything_. If you want configuration options, make the kernel call into user-space *if the configuration is set*. Otherwise, just skip it. Thanks David ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On Fri, Sep 12, 2014 at 11:03 AM, Kay Sievers wrote: >> Here, the default >> action is almost a trivial configuration... but not the only possible >> desired configuration. >> >> Can I ask your reasoning for CPU hotplug behaviour not being the role of >> udev to fulfill? If that's not the right place, where do you believe >> would be the appropriate alternative? > > As explained several times, there is no point in mis-using udev to > unconditionally react to kernel events to trigger things in the kernel > again. That is not how driver/device binding works for all the other > subsystems on Linux. We can now rename network interfaces based on their MACs upon attach without involving the kernel? And reconfiguring a loaded kdump image upon memory change no longer requires doing anything in the kernel? I must have missed a piece of documentation somewhere. I also didn't realize that having a configurable action made something "unconditional". I thought that "unconditional" was in direct opposition to "configurable". I need a new technical dictionary to help me understand these new terms. If it isn't clear, those statements have a liberal coating of sarcasm to drive the points home. >> I'm hoping you have another place in mind. > > The kernel itself or any other custom facility. This stuff has no > place in default/upstream udev, it is the wrong way to do things in > default setups. udev processes kernel events in a programmatic fashion. That has always been its purpose, stated in a single sentence, without exceptions. There has never been some sort of holistic mission for udev with a spiritual direction on the "way to do things". udev processes events and allows configurable actions to handle them, full stop. What we do with those events is udev's rule configuration. Some of those actions do involve telling the kernel to do something in addition to the original event; see the two examples above (though these are not the only ones in common use, obviously). If there were no need to configure differing reactions to kernel events, rules.d wouldn't exist. All that is being proposed here is to offer a default reaction to a CPU add event -- the configuration for which can be changed by a system administrator, distro, or tool just by changing a text file, rather than recompiling a kernel. Making the action configurable, by definition, means that it is *not* unconditional. -- -- Todd Vierling ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On Fri, Sep 12, 2014 at 3:04 PM, John Haxby wrote: > On 02/09/14 16:42, Kay Sievers wrote: >>> > Either the kernel has to provide a mechanism for the userspace to >>> > control onlining, or do it itself and provide a mechanism to prevent >>> > automatic onlining. I think that the first option is actually >>> > cleaner. So yeah, let's add the original rule which does it >>> > unconditionally, >> No, as outlined already, we are not doing this. It is just wrong. >> >>> > and people who have too many cpus and have special >>> > needs can provide a custom udev rule which does something different. >>> > >>> > cpuadd.service or cpuadd@.service seem overkill, since the one we >>> > would provide would still do the exact same thing: unconditionally >>> > enable the CPU. >> Nothing wrong with that, if people need that. But this can for sure >> not live in systemd/udev, it is not the right place. > > Kay, I've worked on a case with Xen domUs which have only some of their > virtual CPUs online at any given time. This deployment has scripts which > only turn up extra CPUs when one of the applications has to do a > failover from one virtual machine to another. The point is that needed > CPU hotplug behaviour is not necessarily uniform to everyone. > > Even if the default, out of the box, behaviour is to turn up the CPUs > unconditionally, udev seems the best place to change this behaviour at > runtime. The fact that CPUs can be hotplugged at all should imply that > some users will need different behaviour than others. Here, the default > action is almost a trivial configuration... but not the only possible > desired configuration. > > Can I ask your reasoning for CPU hotplug behaviour not being the role of > udev to fulfill? If that's not the right place, where do you believe > would be the appropriate alternative? As explained several times, there is no point in mis-using udev to unconditionally react to kernel events to trigger things in the kernel again. That is not how driver/device binding works for all the other subsystems on Linux. > I'm hoping you have another place in mind. The kernel itself or any other custom facility. This stuff has no place in default/upstream udev, it is the wrong way to do things in default setups. > When I think of changing the > behaviour of any removable hardware, udev is automatically where I look > first. So if I'm missing something here, I would really like some more > input. Udev does not decide which device show up, it just classifies stuff that processes events. Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On 02/09/14 16:42, Kay Sievers wrote: >> > Either the kernel has to provide a mechanism for the userspace to >> > control onlining, or do it itself and provide a mechanism to prevent >> > automatic onlining. I think that the first option is actually >> > cleaner. So yeah, let's add the original rule which does it >> > unconditionally, > No, as outlined already, we are not doing this. It is just wrong. > >> > and people who have too many cpus and have special >> > needs can provide a custom udev rule which does something different. >> > >> > cpuadd.service or cpuadd@.service seem overkill, since the one we >> > would provide would still do the exact same thing: unconditionally >> > enable the CPU. > Nothing wrong with that, if people need that. But this can for sure > not live in systemd/udev, it is not the right place. Kay, I've worked on a case with Xen domUs which have only some of their virtual CPUs online at any given time. This deployment has scripts which only turn up extra CPUs when one of the applications has to do a failover from one virtual machine to another. The point is that needed CPU hotplug behaviour is not necessarily uniform to everyone. Even if the default, out of the box, behaviour is to turn up the CPUs unconditionally, udev seems the best place to change this behaviour at runtime. The fact that CPUs can be hotplugged at all should imply that some users will need different behaviour than others. Here, the default action is almost a trivial configuration... but not the only possible desired configuration. Can I ask your reasoning for CPU hotplug behaviour not being the role of udev to fulfill? If that's not the right place, where do you believe would be the appropriate alternative? I'm hoping you have another place in mind. When I think of changing the behaviour of any removable hardware, udev is automatically where I look first. So if I'm missing something here, I would really like some more input. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On 02/09/14 16:40, Kay Sievers wrote: >>> >> But just in case someone thinks of any per-CPU onlining policy >>> >> machinery now: We will not ship anything in that area in systemd/udev >>> >> upstream. This stuff just belongs into the kernel, like it works for >>> >> any other device. >> > >> > Do you mean that the kernel should not be generating udev events for cpu >> > hotplug? > Sure, nothing wrong with these events. > >> > Or that there is some other mechanism that the kernel should >> > be using to allow cpu hotplug policy to be decided by userland? > It should not be decided by userland. And as the proposed rules show, > nobody needed that so far, and we are not making up theoretical use > cases here now, just as an argument for the userspace even echo. > > If things should be onlined when they show up, the kernel should just > do that itself. There is no point in looping anything like that by > default through userspace. It is the same as automatic driver-binding, > it can (nobody does that) be disabled and taken-over by userspace, but > the default is just in the kernel where it belongs. Hmmm. I wonder why the kernel policy is not to put the cpu online? I know it's been that way for some time now which is why we had the old "bring it online" udev rule. I'll go hunting. jch ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On Tue, Sep 2, 2014 at 5:29 PM, Zbigniew Jędrzejewski-Szmek
wrote:
> On Tue, Sep 02, 2014 at 10:27:37AM +0200, Kay Sievers wrote:
>> On Tue, Sep 2, 2014 at 10:01 AM, John Haxby wrote:
>> >
>> > On 2 Sep 2014, at 06:55, Kay Sievers wrote:
>> >
>> >> On Tue, Sep 2, 2014 at 5:22 AM, Zhenzhong Duan
>> >> wrote:
>> >>> Cpu doesn't get online automaticly after hotplug when we test guest cpu
>> >>> add/remove in xen env.
>> >>>
>> >>> I don't have an baremetal env to test this, but I think it's same.
>> >>>
>> >>> The rule is missed in systemd but exist in legacy udev.
>> >>
>> >> Udev is not a mechanism to establish an unconditional loop from the
>> >> kernel back to the kernel. Such rule makes no sense and we never
>> >> shipped that and will not ship it upstream now.
>> >>
>> >> If a device should be unconditionally change its state, the kernel
>> >> should just do that on its own, and not rely on userspace to do that.
>> >
>> >
>> > Kay,
>> >
>> > I’m curious. Are you thinking of 40-redhat.rules in the RHEL6 udev as
>> > not being part of the legacy distribution?
>> >
>> > Should there be a rule that uses a systemd target to hot-add CPUs to
>> > provide the conditional aspect that’s needed? Am I right in thinking that
>> > this is as simple as turning the old redhat rule into something like
>> >
>> > ACTION==“add” KERNEL==“cpu[0-9]*” TAG+=“systemd”
>> > ENV{SYSTEMD_WANTS}+=“cpuadd.service”
>> >
>> > And the corresponding service’s ExecStart is what actually puts the CPU on
>> > line?
>> >
>> > Sorry for the questions, I’m still trying to find the time to get to grips
>> > with systemd — that’s a poor excuse for something that’s been around for
>> > some considerable time I know, but be kind to me :)
>>
>> No, that is the same, just even more complicated. Establishing
>> unconditional loops from the kernel trough userspace back to the
>> kernel are wrong and pointless. If the CPU should always be onlined
>> with that event, please change the kernel to do that on its own, and
>> do not hook up userspace to do that.
> Either the kernel has to provide a mechanism for the userspace to
> control onlining, or do it itself and provide a mechanism to prevent
> automatic onlining. I think that the first option is actually
> cleaner. So yeah, let's add the original rule which does it
> unconditionally,
No, as outlined already, we are not doing this. It is just wrong.
> and people who have too many cpus and have special
> needs can provide a custom udev rule which does something different.
>
> cpuadd.service or cpuadd@.service seem overkill, since the one we
> would provide would still do the exact same thing: unconditionally
> enable the CPU.
Nothing wrong with that, if people need that. But this can for sure
not live in systemd/udev, it is not the right place.
Kay
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On Tue, Sep 2, 2014 at 5:30 PM, John Haxby wrote: > On 02/09/14 16:23, Kay Sievers wrote: >> On Tue, Sep 2, 2014 at 5:11 PM, Colin Guthrie wrote: >>> >>> John Haxby wrote on 02/09/14 10:31: Col, forgive my ignorance, but cpuadd@$name.service seems to imply that you'd have one file or symlink per CPU. That's going to be unwieldy when you have hundreds of CPUs isn't it? >>> >>> Not quite. systemd units with an @ in them are a bit special. You have >>> one unit file on disk (cpuadd@.service) and you can instantiate >>> instances of them via cpuadd@xxx.service where the value xxx is >>> available inside the unit and can be passed to scripts or binaries in >>> the ExecStart lines via the %i and %I modifiers. >>> >>> In this case you wouldn't ever enable them individually via explicit >>> symlinks (which is valid in some use cases) but udev would defer to >>> systemd to start the instance as needed. This is very similar to how the >>> built in systemd-rfkill@.service works. See >>> /usr/lib/systemd/system/systemd-rfkill@.service. It's instantiated as >>> systemd-rfkill@rfkill1.service for example. >>> >>> >>> It's basically the way to pass in the the CPU number or name to the >>> service so you can implement that mission critical "only allowing prime >>> numbered CPUs on Wednesday" policy ;) >> >> Nice description. :) >> >> But just in case someone thinks of any per-CPU onlining policy >> machinery now: We will not ship anything in that area in systemd/udev >> upstream. This stuff just belongs into the kernel, like it works for >> any other device. > > Do you mean that the kernel should not be generating udev events for cpu > hotplug? Sure, nothing wrong with these events. > Or that there is some other mechanism that the kernel should > be using to allow cpu hotplug policy to be decided by userland? It should not be decided by userland. And as the proposed rules show, nobody needed that so far, and we are not making up theoretical use cases here now, just as an argument for the userspace even echo. If things should be onlined when they show up, the kernel should just do that itself. There is no point in looping anything like that by default through userspace. It is the same as automatic driver-binding, it can (nobody does that) be disabled and taken-over by userspace, but the default is just in the kernel where it belongs. Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On 02/09/14 16:23, Kay Sievers wrote: > On Tue, Sep 2, 2014 at 5:11 PM, Colin Guthrie wrote: >> >> John Haxby wrote on 02/09/14 10:31: >>> Col, forgive my ignorance, but cpuadd@$name.service seems to imply that >>> you'd have one file or symlink per CPU. That's going to be unwieldy >>> when you have hundreds of CPUs isn't it? >> >> Not quite. systemd units with an @ in them are a bit special. You have >> one unit file on disk (cpuadd@.service) and you can instantiate >> instances of them via cpuadd@xxx.service where the value xxx is >> available inside the unit and can be passed to scripts or binaries in >> the ExecStart lines via the %i and %I modifiers. >> >> In this case you wouldn't ever enable them individually via explicit >> symlinks (which is valid in some use cases) but udev would defer to >> systemd to start the instance as needed. This is very similar to how the >> built in systemd-rfkill@.service works. See >> /usr/lib/systemd/system/systemd-rfkill@.service. It's instantiated as >> systemd-rfkill@rfkill1.service for example. >> >> >> It's basically the way to pass in the the CPU number or name to the >> service so you can implement that mission critical "only allowing prime >> numbered CPUs on Wednesday" policy ;) > > Nice description. :) > > But just in case someone thinks of any per-CPU onlining policy > machinery now: We will not ship anything in that area in systemd/udev > upstream. This stuff just belongs into the kernel, like it works for > any other device. > Do you mean that the kernel should not be generating udev events for cpu hotplug? Or that there is some other mechanism that the kernel should be using to allow cpu hotplug policy to be decided by userland? > Kay > ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On Tue, Sep 02, 2014 at 10:27:37AM +0200, Kay Sievers wrote:
> On Tue, Sep 2, 2014 at 10:01 AM, John Haxby wrote:
> >
> > On 2 Sep 2014, at 06:55, Kay Sievers wrote:
> >
> >> On Tue, Sep 2, 2014 at 5:22 AM, Zhenzhong Duan
> >> wrote:
> >>> Cpu doesn't get online automaticly after hotplug when we test guest cpu
> >>> add/remove in xen env.
> >>>
> >>> I don't have an baremetal env to test this, but I think it's same.
> >>>
> >>> The rule is missed in systemd but exist in legacy udev.
> >>
> >> Udev is not a mechanism to establish an unconditional loop from the
> >> kernel back to the kernel. Such rule makes no sense and we never
> >> shipped that and will not ship it upstream now.
> >>
> >> If a device should be unconditionally change its state, the kernel
> >> should just do that on its own, and not rely on userspace to do that.
> >
> >
> > Kay,
> >
> > I’m curious. Are you thinking of 40-redhat.rules in the RHEL6 udev as not
> > being part of the legacy distribution?
> >
> > Should there be a rule that uses a systemd target to hot-add CPUs to
> > provide the conditional aspect that’s needed? Am I right in thinking that
> > this is as simple as turning the old redhat rule into something like
> >
> > ACTION==“add” KERNEL==“cpu[0-9]*” TAG+=“systemd”
> > ENV{SYSTEMD_WANTS}+=“cpuadd.service”
> >
> > And the corresponding service’s ExecStart is what actually puts the CPU on
> > line?
> >
> > Sorry for the questions, I’m still trying to find the time to get to grips
> > with systemd — that’s a poor excuse for something that’s been around for
> > some considerable time I know, but be kind to me :)
>
> No, that is the same, just even more complicated. Establishing
> unconditional loops from the kernel trough userspace back to the
> kernel are wrong and pointless. If the CPU should always be onlined
> with that event, please change the kernel to do that on its own, and
> do not hook up userspace to do that.
Either the kernel has to provide a mechanism for the userspace to
control onlining, or do it itself and provide a mechanism to prevent
automatic onlining. I think that the first option is actually
cleaner. So yeah, let's add the original rule which does it
unconditionally, and people who have too many cpus and have special
needs can provide a custom udev rule which does something different.
cpuadd.service or cpuadd@.service seem overkill, since the one we
would provide would still do the exact same thing: unconditionally
enable the CPU.
Zbyszek
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On Tue, Sep 2, 2014 at 5:11 PM, Colin Guthrie wrote: > > John Haxby wrote on 02/09/14 10:31: >> Col, forgive my ignorance, but cpuadd@$name.service seems to imply that >> you'd have one file or symlink per CPU. That's going to be unwieldy >> when you have hundreds of CPUs isn't it? > > Not quite. systemd units with an @ in them are a bit special. You have > one unit file on disk (cpuadd@.service) and you can instantiate > instances of them via cpuadd@xxx.service where the value xxx is > available inside the unit and can be passed to scripts or binaries in > the ExecStart lines via the %i and %I modifiers. > > In this case you wouldn't ever enable them individually via explicit > symlinks (which is valid in some use cases) but udev would defer to > systemd to start the instance as needed. This is very similar to how the > built in systemd-rfkill@.service works. See > /usr/lib/systemd/system/systemd-rfkill@.service. It's instantiated as > systemd-rfkill@rfkill1.service for example. > > > It's basically the way to pass in the the CPU number or name to the > service so you can implement that mission critical "only allowing prime > numbered CPUs on Wednesday" policy ;) Nice description. :) But just in case someone thinks of any per-CPU onlining policy machinery now: We will not ship anything in that area in systemd/udev upstream. This stuff just belongs into the kernel, like it works for any other device. Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
John Haxby wrote on 02/09/14 10:31: > Col, forgive my ignorance, but cpuadd@$name.service seems to imply that > you'd have one file or symlink per CPU. That's going to be unwieldy > when you have hundreds of CPUs isn't it? Not quite. systemd units with an @ in them are a bit special. You have one unit file on disk (cpuadd@.service) and you can instantiate instances of them via cpuadd@xxx.service where the value xxx is available inside the unit and can be passed to scripts or binaries in the ExecStart lines via the %i and %I modifiers. In this case you wouldn't ever enable them individually via explicit symlinks (which is valid in some use cases) but udev would defer to systemd to start the instance as needed. This is very similar to how the built in systemd-rfkill@.service works. See /usr/lib/systemd/system/systemd-rfkill@.service. It's instantiated as systemd-rfkill@rfkill1.service for example. It's basically the way to pass in the the CPU number or name to the service so you can implement that mission critical "only allowing prime numbered CPUs on Wednesday" policy ;) Col -- Colin Guthrie http://colin.guthr.ie/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On 02/09/14 09:42, Colin Guthrie wrote:
> Kay Sievers wrote on 02/09/14 09:27:
>> On Tue, Sep 2, 2014 at 10:01 AM, John Haxby wrote:
>>>
>>> On 2 Sep 2014, at 06:55, Kay Sievers wrote:
>>>
On Tue, Sep 2, 2014 at 5:22 AM, Zhenzhong Duan
wrote:
> Cpu doesn't get online automaticly after hotplug when we test guest cpu
> add/remove in xen env.
>
> I don't have an baremetal env to test this, but I think it's same.
>
> The rule is missed in systemd but exist in legacy udev.
Udev is not a mechanism to establish an unconditional loop from the
kernel back to the kernel. Such rule makes no sense and we never
shipped that and will not ship it upstream now.
If a device should be unconditionally change its state, the kernel
should just do that on its own, and not rely on userspace to do that.
>>>
>>>
>>> Kay,
>>>
>>> I’m curious. Are you thinking of 40-redhat.rules in the RHEL6 udev as not
>>> being part of the legacy distribution?
>>>
>>> Should there be a rule that uses a systemd target to hot-add CPUs to
>>> provide the conditional aspect that’s needed? Am I right in thinking that
>>> this is as simple as turning the old redhat rule into something like
>>>
>>> ACTION==“add” KERNEL==“cpu[0-9]*” TAG+=“systemd”
>>> ENV{SYSTEMD_WANTS}+=“cpuadd.service”
>>>
>>> And the corresponding service’s ExecStart is what actually puts the CPU on
>>> line?
>>>
>>> Sorry for the questions, I’m still trying to find the time to get to grips
>>> with systemd — that’s a poor excuse for something that’s been around for
>>> some considerable time I know, but be kind to me :)
>>
>> No, that is the same, just even more complicated. Establishing
>> unconditional loops from the kernel trough userspace back to the
>> kernel are wrong and pointless. If the CPU should always be onlined
>> with that event, please change the kernel to do that on its own, and
>> do not hook up userspace to do that.
>
> Does that not depend on what cpuadd.service actually does? (and I'm
> guessing it should probably be cpuadd@$name.service" or something
> appropriate to identify the single CPU in question).
>
> If there is some kind of conditional logic in the service (perhaps
> checking current system load before deciding if it's needed) then I
> presume this is something legitimate and valid?
>
> It's just the fact that there is currently no logic at all involved
> currently that makes this pointless and something the kernel could do on
> it's own.
That's what I thought. I have no idea what the policy for deciding
whether a cpu should be brought on line might be: it could be as simple
as limiting the maximum number of CPUs or it could be as complicated as
only allowing prime numbered CPUs on Wednesday(*). The default
/lib/systemd/system/cpuadd*.service would implement the policy "if it's
there we want it" and someone needing something more complex can
override it with /etc/systemd/system/cpuadd*.service.
Col, forgive my ignorance, but cpuadd@$name.service seems to imply that
you'd have one file or symlink per CPU. That's going to be unwieldy
when you have hundreds of CPUs isn't it?
jch
>
> Col
>
>
(*) My employer is not alone in licensing by CPU so only allowing a
specified number of CPUs or only allowing CPUs with some specific
topographical relationship is not beyond the bounds of reason.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On 2014-9-2 16:25, Kay Sievers wrote: On Tue, Sep 2, 2014 at 8:12 AM, Zhenzhong Duan wrote: On 2014-9-2 13:55, Kay Sievers wrote: On Tue, Sep 2, 2014 at 5:22 AM, Zhenzhong Duan wrote: Cpu doesn't get online automaticly after hotplug when we test guest cpu add/remove in xen env. I don't have an baremetal env to test this, but I think it's same. The rule is missed in systemd but exist in legacy udev. Udev is not a mechanism to establish an unconditional loop from the kernel back to the kernel. Such rule makes no sense and we never shipped that and will not ship it upstream now. If a device should be unconditionally change its state, the kernel should just do that on its own, and not rely on userspace to do that. Thanks for your suggestion. Maybe we should add this patch in our internal branch. In fact, kernel does expect user level to control online/offline of cpus. There are two step to make a cpu online, 1. plug it, 2. online it. Plug doesn't have to be followed by online op. But you rule does that unconditionally. Shipping such things in udev rules makes no sense. Kay Hi I treat this as a default rule for most of people. They want cpu online right after plug it. And in kernel side, it expects which cpus getting online/offline is controlable from user side. thanks zduan ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
Kay Sievers wrote on 02/09/14 09:27:
> On Tue, Sep 2, 2014 at 10:01 AM, John Haxby wrote:
>>
>> On 2 Sep 2014, at 06:55, Kay Sievers wrote:
>>
>>> On Tue, Sep 2, 2014 at 5:22 AM, Zhenzhong Duan
>>> wrote:
Cpu doesn't get online automaticly after hotplug when we test guest cpu
add/remove in xen env.
I don't have an baremetal env to test this, but I think it's same.
The rule is missed in systemd but exist in legacy udev.
>>>
>>> Udev is not a mechanism to establish an unconditional loop from the
>>> kernel back to the kernel. Such rule makes no sense and we never
>>> shipped that and will not ship it upstream now.
>>>
>>> If a device should be unconditionally change its state, the kernel
>>> should just do that on its own, and not rely on userspace to do that.
>>
>>
>> Kay,
>>
>> I’m curious. Are you thinking of 40-redhat.rules in the RHEL6 udev as not
>> being part of the legacy distribution?
>>
>> Should there be a rule that uses a systemd target to hot-add CPUs to provide
>> the conditional aspect that’s needed? Am I right in thinking that this is
>> as simple as turning the old redhat rule into something like
>>
>> ACTION==“add” KERNEL==“cpu[0-9]*” TAG+=“systemd”
>> ENV{SYSTEMD_WANTS}+=“cpuadd.service”
>>
>> And the corresponding service’s ExecStart is what actually puts the CPU on
>> line?
>>
>> Sorry for the questions, I’m still trying to find the time to get to grips
>> with systemd — that’s a poor excuse for something that’s been around for
>> some considerable time I know, but be kind to me :)
>
> No, that is the same, just even more complicated. Establishing
> unconditional loops from the kernel trough userspace back to the
> kernel are wrong and pointless. If the CPU should always be onlined
> with that event, please change the kernel to do that on its own, and
> do not hook up userspace to do that.
Does that not depend on what cpuadd.service actually does? (and I'm
guessing it should probably be cpuadd@$name.service" or something
appropriate to identify the single CPU in question).
If there is some kind of conditional logic in the service (perhaps
checking current system load before deciding if it's needed) then I
presume this is something legitimate and valid?
It's just the fact that there is currently no logic at all involved
currently that makes this pointless and something the kernel could do on
it's own.
Col
--
Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/
Day Job:
Tribalogic Limited http://www.tribalogic.net/
Open Source:
Mageia Contributor http://www.mageia.org/
PulseAudio Hacker http://www.pulseaudio.org/
Trac Hacker http://trac.edgewall.org/
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On Tue, Sep 2, 2014 at 10:01 AM, John Haxby wrote:
>
> On 2 Sep 2014, at 06:55, Kay Sievers wrote:
>
>> On Tue, Sep 2, 2014 at 5:22 AM, Zhenzhong Duan
>> wrote:
>>> Cpu doesn't get online automaticly after hotplug when we test guest cpu
>>> add/remove in xen env.
>>>
>>> I don't have an baremetal env to test this, but I think it's same.
>>>
>>> The rule is missed in systemd but exist in legacy udev.
>>
>> Udev is not a mechanism to establish an unconditional loop from the
>> kernel back to the kernel. Such rule makes no sense and we never
>> shipped that and will not ship it upstream now.
>>
>> If a device should be unconditionally change its state, the kernel
>> should just do that on its own, and not rely on userspace to do that.
>
>
> Kay,
>
> I’m curious. Are you thinking of 40-redhat.rules in the RHEL6 udev as not
> being part of the legacy distribution?
>
> Should there be a rule that uses a systemd target to hot-add CPUs to provide
> the conditional aspect that’s needed? Am I right in thinking that this is as
> simple as turning the old redhat rule into something like
>
> ACTION==“add” KERNEL==“cpu[0-9]*” TAG+=“systemd”
> ENV{SYSTEMD_WANTS}+=“cpuadd.service”
>
> And the corresponding service’s ExecStart is what actually puts the CPU on
> line?
>
> Sorry for the questions, I’m still trying to find the time to get to grips
> with systemd — that’s a poor excuse for something that’s been around for some
> considerable time I know, but be kind to me :)
No, that is the same, just even more complicated. Establishing
unconditional loops from the kernel trough userspace back to the
kernel are wrong and pointless. If the CPU should always be onlined
with that event, please change the kernel to do that on its own, and
do not hook up userspace to do that.
Kay
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On Tue, Sep 2, 2014 at 8:12 AM, Zhenzhong Duan wrote: > On 2014-9-2 13:55, Kay Sievers wrote: >> >> On Tue, Sep 2, 2014 at 5:22 AM, Zhenzhong Duan >> wrote: >>> >>> Cpu doesn't get online automaticly after hotplug when we test guest cpu >>> add/remove in xen env. >>> >>> I don't have an baremetal env to test this, but I think it's same. >>> >>> The rule is missed in systemd but exist in legacy udev. >> >> Udev is not a mechanism to establish an unconditional loop from the >> kernel back to the kernel. Such rule makes no sense and we never >> shipped that and will not ship it upstream now. >> >> If a device should be unconditionally change its state, the kernel >> should just do that on its own, and not rely on userspace to do that. > Thanks for your suggestion. Maybe we should add this patch in our internal > branch. > > In fact, kernel does expect user level to control online/offline of cpus. > There are two step to make a cpu online, 1. plug it, 2. online it. > > Plug doesn't have to be followed by online op. But you rule does that unconditionally. Shipping such things in udev rules makes no sense. Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On 2 Sep 2014, at 06:55, Kay Sievers wrote:
> On Tue, Sep 2, 2014 at 5:22 AM, Zhenzhong Duan
> wrote:
>> Cpu doesn't get online automaticly after hotplug when we test guest cpu
>> add/remove in xen env.
>>
>> I don't have an baremetal env to test this, but I think it's same.
>>
>> The rule is missed in systemd but exist in legacy udev.
>
> Udev is not a mechanism to establish an unconditional loop from the
> kernel back to the kernel. Such rule makes no sense and we never
> shipped that and will not ship it upstream now.
>
> If a device should be unconditionally change its state, the kernel
> should just do that on its own, and not rely on userspace to do that.
Kay,
I’m curious. Are you thinking of 40-redhat.rules in the RHEL6 udev as not
being part of the legacy distribution?
Should there be a rule that uses a systemd target to hot-add CPUs to provide
the conditional aspect that’s needed? Am I right in thinking that this is as
simple as turning the old redhat rule into something like
ACTION==“add” KERNEL==“cpu[0-9]*” TAG+=“systemd”
ENV{SYSTEMD_WANTS}+=“cpuadd.service”
And the corresponding service’s ExecStart is what actually puts the CPU on line?
Sorry for the questions, I’m still trying to find the time to get to grips with
systemd — that’s a poor excuse for something that’s been around for some
considerable time I know, but be kind to me :)
jch
>
> Kay
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On 2014-9-2 13:55, Kay Sievers wrote: On Tue, Sep 2, 2014 at 5:22 AM, Zhenzhong Duan wrote: Cpu doesn't get online automaticly after hotplug when we test guest cpu add/remove in xen env. I don't have an baremetal env to test this, but I think it's same. The rule is missed in systemd but exist in legacy udev. Udev is not a mechanism to establish an unconditional loop from the kernel back to the kernel. Such rule makes no sense and we never shipped that and will not ship it upstream now. If a device should be unconditionally change its state, the kernel should just do that on its own, and not rely on userspace to do that. Kay Hi Kay Thanks for your suggestion. Maybe we should add this patch in our internal branch. In fact, kernel does expect user level to control online/offline of cpus. There are two step to make a cpu online, 1. plug it, 2. online it. Plug doesn't have to be followed by online op. thanks zduan ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH] Add the CPU hotplug rule
On Tue, Sep 2, 2014 at 5:22 AM, Zhenzhong Duan wrote: > Cpu doesn't get online automaticly after hotplug when we test guest cpu > add/remove in xen env. > > I don't have an baremetal env to test this, but I think it's same. > > The rule is missed in systemd but exist in legacy udev. Udev is not a mechanism to establish an unconditional loop from the kernel back to the kernel. Such rule makes no sense and we never shipped that and will not ship it upstream now. If a device should be unconditionally change its state, the kernel should just do that on its own, and not rely on userspace to do that. Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
