Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, 12 Nov 2013 19:01:49 +0400 Andrey Borzenkov
wrote:
> В Tue, 12 Nov 2013 21:17:19 +1100
> NeilBrown пишет:
>
> > On Tue, 12 Nov 2013 18:16:24 +0900 Greg KH
> > wrote:
> >
> > > On Tue, Nov 12, 2013 at 07:54:42PM +1100, NeilBrown wrote:
> > > > On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH
> > > > wrote:
> > > >
> > > > > On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote:
> > > > > > Alternately, is there some "all devices have been probed, nothing
> > > > > > new will
> > > > > > appear unless it is hot-plugged" event. That would be equally
> > > > > > useful (and
> > > > > > probably mirrors what hardware-RAID cards do).
> > > > >
> > > > > No, there's no way to ever know this in a hotplug world, sorry.
> > > > > Especially with USB devices, they show up when they show up, there's
> > > > > no
> > > > > "oh look, the bus is all scanned now and all devices currently plugged
> > > > > in are found" type knowledge at all.
> > > > >
> > > > > Then there are hotplug PCI systems where people slam in PCI cards
> > > > > whenever they feel like it (remember, thunderbolt is PCI express...)
> > > > >
> > > > > Sorry,
> > > > >
> > > > > greg k-h
> > > >
> > > > Surely something must be possible.
> > >
> > > For USB, nope, there isn't, sorry.
> > >
> > > > Clearly a physical hot-plug event will cause more devices to appear, but
> > > > there must come a point at which no more (non-virtual) devices will
> > > > appear
> > > > unless a physical event happens?
> > >
> > > Not for USB, sorry.
> > >
> > > The USB bus just announces devices when it finds them, there is no "all
> > > is quiet" type signal or detection.
> > >
> > > Same for PCI hotplug, devices can show up at any point in time, you
> > > never know when, and you don't know when all devices are "found".
> > >
> > > sorry,
> > >
> > > greg k-h
> >
> >
> > Hmmm... OK. USB doesn't bother me a lot, but PCI is important.
> >
> > I guess I'll just have to settle for a timeout much like the current
> > device-discovery timeout that systemd has.
> > Still hoping someone can tell me how to plug into that though...
> >
>
> If information about array name or other identification is available in
> udev rule (I see reference to device node only) what you can do is to
> start timer with "now+5second" (pick your timeout) that simply fires off
> mdadm -IRs for specific array. Something like
>
> mdadm-last-resort@.timer
>
> [Timer]
> OnCalendar=+5s
>
> mdadm-last-resort@.service
>
> [Service]
> Type=oneshot
> ExecStart=/sbin/mdadm -IRs %n
>
> udev rule
>
> ... SYSTEMD_WANTS=mdadm-last-resort@$ENV{SOMETHING_UNIQUE}.timer
>
Just wanted to let you know that I've gone ahead with this approach and it
seems to work OK. It took me a while to find "DefaultDependencies=no" which
I need to make it work, but otherwise it appears to work as advertise.
The relevant files will be in the next release of mdadm.
Thanks again,
NeilBrown
signature.asc
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Thu, 14 Nov 2013 11:23:30 +0600 "Alexander E. Patrakov"
wrote:
> NeilBrown пишет:
> > On Wed, 13 Nov 2013 22:11:27 +0600 "Alexander E. Patrakov"
> > wrote:
> >
> >> 2013/11/13 NeilBrown :
> >>> On Tue, 12 Nov 2013 19:01:49 +0400 Andrey Borzenkov
> >>> wrote:
> >>>
> Something like
>
> mdadm-last-resort@.timer
>
> [Timer]
> OnCalendar=+5s
>
> mdadm-last-resort@.service
>
> [Service]
> Type=oneshot
> ExecStart=/sbin/mdadm -IRs %n
>
> udev rule
>
> ... SYSTEMD_WANTS=mdadm-last-resort@$ENV{SOMETHING_UNIQUE}.timer
>
> >>> Thanks. This certainly looks interesting and might be part of a solution.
> >>> However it gets the timeout test backwards.
> >>>
> >>> I don't want to set the timeout when the array starts to appear. I want
> >>> to
> >>> set the time out when someone wants to use the array.
> >>> If no-one is waiting for the array device, then there is no point forcing
> >>> it.
> >>>
> >>> That's why I want to plug into the timeout that systemd already has.
> >>>
> >>> Maybe that requirement isn't really necessary though. I'll experiment
> >>> with
> >>> your approach.
> >> It is useless to even try to plug into the existing systemd timeout,
> >> for a very simple reason. in the setups where your RAID array is not
> >> on the top of the storage device hierarchy, systemd does not know that
> >> it wants your RAID array to appear.
> >>
> >> So the statement "If no-one is waiting for the array device, then
> >> there is no point forcing it" is false, because there is no way to
> >> know that no-one is waiting.
> >>
> > "useless" seems a bit harsh. "not-optimal" may be true.
> >
> > If systemd was waiting for a device, then it is clear that something was
> > waiting for something. In this case it might be justified to activate as
> > much as possible in the hope that the important things will get activated.
> > This is what "mdadm -IRs" does. It activates all arrays that are still
> > inactive but have enough devices to become active (though degraded). It
> > isn't selective.
> > If they are deep in some hierarchy, then udev will pull it all together and
> > the root device will appear.
> >
> > If systemd is not waiting for a device, then there is no justification for
> > prematurely starting degraded arrays.
> >
> >
> > Maybe I could get emergency.service to run "mdadm -IRs" and if that actually
> > started anything, then to somehow restart local-fs.target. Might that be
> > possible?
>
> If this is the way forward, then we, obviously, should think about a
> general mechanism that is useful not only for mdadm, but also to other
> layered storage implementations such as dm-raid, or maybe multi-device
> btrfs, and that is useful if more than one of these technologies are
> used on top of each other. This by necessity leads to multiple emergency
> missing-device handlers. And then a question immediately appears, in
> which order the emergency handlers should be tried, because all that is
> known at the time of emergency is that some device listed in /etc/fstab
> is missing. I suspect that the answer is "in arbitrary order" or even
> "in parallel", but then there is a chance that one run of all of them
> will not be enough.
>
> This is not a criticism, just something to be fully thought out before
> starting an implementation.
>
Good points.
However dmraid doesn't have very good support for RAID arrays that actually
have redundancy. It is fine for RAID0 but only really supports RAID1 in ddf
and Intel's IMSM and mdadm has better support for those. So dmraid doesn't
really figure much here.
And I suspect that btrfs would end up being a very different situation
because it vertically integrates the filesystem with the RAID layer.
My ideal solution would be for mdadm to assemble a degraded array as soon as
enough devices are available, but mark it soft-read-only. When all the
expected disks arrive it would switch to read-write.
systemd would see this soft-read-only state and wait a bit longer for it to
become read-write. If that didn't happen in time, it would force it to be
read-write.
Every stacked device would need to understand this soft-read-only state and
would set itself to soft-read-only if any component was soft-read-only, or
was missing. It would notice changes in component devices and propagate them
up.
When the top device was forced to read-write, this would propagate all the
way down.
This would avoid the guess work an "emergency catch-all" ugliness and yield a
fairly clean and predictable result.
If anything tried to write to a soft-read-only device, that would have the
same effect as forcing to read-write.
This way the whole storage stack would be plumbed at the earliest possible
moment, but would not generate any writes. When systemd discovers the
device, if it notices that it is "soft-read-only", it could then wait a
timeout before forcing the device to read-write. That
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
]] Dax Kelson > On Nov 13, 2013 8:03 AM, "Lennart Poettering" > wrote: > > > > I also have the suspicion that the best strategy for handling degraded > > arrays is to timeout and not assemble them but rather put the system in > > a state where the admin has to become active. Auto-assembling degraded > > arrays has the feel of taping over issues. If the admin chooses to > > boot-up with degraded disks then that's ok, but I am pretty sure this is > > something the admin should explicitly decide. > > As an experienced admin, I disagree with this. If I've gone to the effort > to setup a RAID volume obviously I value high availability. That's not obvious. You might value data integrity. You might value the ability to have file systems larger than a single disk which has some resilence against single- or double disk failures. The problem with just continuing with a degraded RAID is for those who run without monitoring turned on: How is the admin to know that the system's RAID is degraded? (Think simple home server without working outgoing email for mdadm, no monitoring, no desktop style login where you can pop up a notification.) Sadly, this means that us experienced admins have to flip the defaults because we have working email from mdadm and monitoring and alerts and we would rather the volume be available and degraded than not available at all. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
NeilBrown пишет:
On Wed, 13 Nov 2013 22:11:27 +0600 "Alexander E. Patrakov"
wrote:
2013/11/13 NeilBrown :
On Tue, 12 Nov 2013 19:01:49 +0400 Andrey Borzenkov
wrote:
Something like
mdadm-last-resort@.timer
[Timer]
OnCalendar=+5s
mdadm-last-resort@.service
[Service]
Type=oneshot
ExecStart=/sbin/mdadm -IRs %n
udev rule
... SYSTEMD_WANTS=mdadm-last-resort@$ENV{SOMETHING_UNIQUE}.timer
Thanks. This certainly looks interesting and might be part of a solution.
However it gets the timeout test backwards.
I don't want to set the timeout when the array starts to appear. I want to
set the time out when someone wants to use the array.
If no-one is waiting for the array device, then there is no point forcing it.
That's why I want to plug into the timeout that systemd already has.
Maybe that requirement isn't really necessary though. I'll experiment with
your approach.
It is useless to even try to plug into the existing systemd timeout,
for a very simple reason. in the setups where your RAID array is not
on the top of the storage device hierarchy, systemd does not know that
it wants your RAID array to appear.
So the statement "If no-one is waiting for the array device, then
there is no point forcing it" is false, because there is no way to
know that no-one is waiting.
"useless" seems a bit harsh. "not-optimal" may be true.
If systemd was waiting for a device, then it is clear that something was
waiting for something. In this case it might be justified to activate as
much as possible in the hope that the important things will get activated.
This is what "mdadm -IRs" does. It activates all arrays that are still
inactive but have enough devices to become active (though degraded). It
isn't selective.
If they are deep in some hierarchy, then udev will pull it all together and
the root device will appear.
If systemd is not waiting for a device, then there is no justification for
prematurely starting degraded arrays.
Maybe I could get emergency.service to run "mdadm -IRs" and if that actually
started anything, then to somehow restart local-fs.target. Might that be
possible?
If this is the way forward, then we, obviously, should think about a
general mechanism that is useful not only for mdadm, but also to other
layered storage implementations such as dm-raid, or maybe multi-device
btrfs, and that is useful if more than one of these technologies are
used on top of each other. This by necessity leads to multiple emergency
missing-device handlers. And then a question immediately appears, in
which order the emergency handlers should be tried, because all that is
known at the time of emergency is that some device listed in /etc/fstab
is missing. I suspect that the answer is "in arbitrary order" or even
"in parallel", but then there is a chance that one run of all of them
will not be enough.
This is not a criticism, just something to be fully thought out before
starting an implementation.
--
Alexander E. Patrakov
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Wed, 13 Nov 2013 22:11:27 +0600 "Alexander E. Patrakov"
wrote:
> 2013/11/13 NeilBrown :
> > On Tue, 12 Nov 2013 19:01:49 +0400 Andrey Borzenkov
> > wrote:
> >
> >> В Tue, 12 Nov 2013 21:17:19 +1100
> >> NeilBrown пишет:
> >>
> >> > On Tue, 12 Nov 2013 18:16:24 +0900 Greg KH
> >> > wrote:
> >> >
> >> > > On Tue, Nov 12, 2013 at 07:54:42PM +1100, NeilBrown wrote:
> >> > > > On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH
> >> > > > wrote:
> >> > > >
> >> > > > > On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote:
> >> > > > > > Alternately, is there some "all devices have been probed,
> >> > > > > > nothing new will
> >> > > > > > appear unless it is hot-plugged" event. That would be equally
> >> > > > > > useful (and
> >> > > > > > probably mirrors what hardware-RAID cards do).
> >> > > > >
> >> > > > > No, there's no way to ever know this in a hotplug world, sorry.
> >> > > > > Especially with USB devices, they show up when they show up,
> >> > > > > there's no
> >> > > > > "oh look, the bus is all scanned now and all devices currently
> >> > > > > plugged
> >> > > > > in are found" type knowledge at all.
> >> > > > >
> >> > > > > Then there are hotplug PCI systems where people slam in PCI cards
> >> > > > > whenever they feel like it (remember, thunderbolt is PCI
> >> > > > > express...)
> >> > > > >
> >> > > > > Sorry,
> >> > > > >
> >> > > > > greg k-h
> >> > > >
> >> > > > Surely something must be possible.
> >> > >
> >> > > For USB, nope, there isn't, sorry.
> >> > >
> >> > > > Clearly a physical hot-plug event will cause more devices to appear,
> >> > > > but
> >> > > > there must come a point at which no more (non-virtual) devices will
> >> > > > appear
> >> > > > unless a physical event happens?
> >> > >
> >> > > Not for USB, sorry.
> >> > >
> >> > > The USB bus just announces devices when it finds them, there is no "all
> >> > > is quiet" type signal or detection.
> >> > >
> >> > > Same for PCI hotplug, devices can show up at any point in time, you
> >> > > never know when, and you don't know when all devices are "found".
> >> > >
> >> > > sorry,
> >> > >
> >> > > greg k-h
> >> >
> >> >
> >> > Hmmm... OK. USB doesn't bother me a lot, but PCI is important.
> >> >
> >> > I guess I'll just have to settle for a timeout much like the current
> >> > device-discovery timeout that systemd has.
> >> > Still hoping someone can tell me how to plug into that though...
> >> >
> >>
> >> If information about array name or other identification is available in
> >> udev rule (I see reference to device node only) what you can do is to
> >> start timer with "now+5second" (pick your timeout) that simply fires off
> >> mdadm -IRs for specific array. Something like
> >>
> >> mdadm-last-resort@.timer
> >>
> >> [Timer]
> >> OnCalendar=+5s
> >>
> >> mdadm-last-resort@.service
> >>
> >> [Service]
> >> Type=oneshot
> >> ExecStart=/sbin/mdadm -IRs %n
> >>
> >> udev rule
> >>
> >> ... SYSTEMD_WANTS=mdadm-last-resort@$ENV{SOMETHING_UNIQUE}.timer
> >>
> >
> > Thanks. This certainly looks interesting and might be part of a solution.
> > However it gets the timeout test backwards.
> >
> > I don't want to set the timeout when the array starts to appear. I want to
> > set the time out when someone wants to use the array.
> > If no-one is waiting for the array device, then there is no point forcing
> > it.
> >
> > That's why I want to plug into the timeout that systemd already has.
> >
> > Maybe that requirement isn't really necessary though. I'll experiment with
> > your approach.
>
> It is useless to even try to plug into the existing systemd timeout,
> for a very simple reason. in the setups where your RAID array is not
> on the top of the storage device hierarchy, systemd does not know that
> it wants your RAID array to appear.
>
> So the statement "If no-one is waiting for the array device, then
> there is no point forcing it" is false, because there is no way to
> know that no-one is waiting.
>
"useless" seems a bit harsh. "not-optimal" may be true.
If systemd was waiting for a device, then it is clear that something was
waiting for something. In this case it might be justified to activate as
much as possible in the hope that the important things will get activated.
This is what "mdadm -IRs" does. It activates all arrays that are still
inactive but have enough devices to become active (though degraded). It
isn't selective.
If they are deep in some hierarchy, then udev will pull it all together and
the root device will appear.
If systemd is not waiting for a device, then there is no justification for
prematurely starting degraded arrays.
Maybe I could get emergency.service to run "mdadm -IRs" and if that actually
started anything, then to somehow restart local-fs.target. Might that be
possible?
Thanks,
NeilBrown
signature.asc
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Nov 13, 2013 8:03 AM, "Lennart Poettering" wrote: > > I also have the suspicion that the best strategy for handling degraded > arrays is to timeout and not assemble them but rather put the system in > a state where the admin has to become active. Auto-assembling degraded > arrays has the feel of taping over issues. If the admin chooses to > boot-up with degraded disks then that's ok, but I am pretty sure this is > something the admin should explicitly decide. As an experienced admin, I disagree with this. If I've gone to the effort to setup a RAID volume obviously I value high availability. The box should boot without manual intervention even with a degraded RAID volume. With above proposal, the box automatically continue to will run properly with a degraded RAID volume (like normal) but won't reboot properly? Rebooting/booting is a normal part of operating. Also. There are many scenarios where a box can reboot on its own (kernel panic, power failure, etc). It should come back automatically if it can. I'd be livid if it didn't. Make it an option if you insist, but have it default to booting without manual intervention if possible. Dax Kelson ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
2013/11/13 NeilBrown :
> On Tue, 12 Nov 2013 19:01:49 +0400 Andrey Borzenkov
> wrote:
>
>> В Tue, 12 Nov 2013 21:17:19 +1100
>> NeilBrown пишет:
>>
>> > On Tue, 12 Nov 2013 18:16:24 +0900 Greg KH
>> > wrote:
>> >
>> > > On Tue, Nov 12, 2013 at 07:54:42PM +1100, NeilBrown wrote:
>> > > > On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH
>> > > > wrote:
>> > > >
>> > > > > On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote:
>> > > > > > Alternately, is there some "all devices have been probed, nothing
>> > > > > > new will
>> > > > > > appear unless it is hot-plugged" event. That would be equally
>> > > > > > useful (and
>> > > > > > probably mirrors what hardware-RAID cards do).
>> > > > >
>> > > > > No, there's no way to ever know this in a hotplug world, sorry.
>> > > > > Especially with USB devices, they show up when they show up, there's
>> > > > > no
>> > > > > "oh look, the bus is all scanned now and all devices currently
>> > > > > plugged
>> > > > > in are found" type knowledge at all.
>> > > > >
>> > > > > Then there are hotplug PCI systems where people slam in PCI cards
>> > > > > whenever they feel like it (remember, thunderbolt is PCI express...)
>> > > > >
>> > > > > Sorry,
>> > > > >
>> > > > > greg k-h
>> > > >
>> > > > Surely something must be possible.
>> > >
>> > > For USB, nope, there isn't, sorry.
>> > >
>> > > > Clearly a physical hot-plug event will cause more devices to appear,
>> > > > but
>> > > > there must come a point at which no more (non-virtual) devices will
>> > > > appear
>> > > > unless a physical event happens?
>> > >
>> > > Not for USB, sorry.
>> > >
>> > > The USB bus just announces devices when it finds them, there is no "all
>> > > is quiet" type signal or detection.
>> > >
>> > > Same for PCI hotplug, devices can show up at any point in time, you
>> > > never know when, and you don't know when all devices are "found".
>> > >
>> > > sorry,
>> > >
>> > > greg k-h
>> >
>> >
>> > Hmmm... OK. USB doesn't bother me a lot, but PCI is important.
>> >
>> > I guess I'll just have to settle for a timeout much like the current
>> > device-discovery timeout that systemd has.
>> > Still hoping someone can tell me how to plug into that though...
>> >
>>
>> If information about array name or other identification is available in
>> udev rule (I see reference to device node only) what you can do is to
>> start timer with "now+5second" (pick your timeout) that simply fires off
>> mdadm -IRs for specific array. Something like
>>
>> mdadm-last-resort@.timer
>>
>> [Timer]
>> OnCalendar=+5s
>>
>> mdadm-last-resort@.service
>>
>> [Service]
>> Type=oneshot
>> ExecStart=/sbin/mdadm -IRs %n
>>
>> udev rule
>>
>> ... SYSTEMD_WANTS=mdadm-last-resort@$ENV{SOMETHING_UNIQUE}.timer
>>
>
> Thanks. This certainly looks interesting and might be part of a solution.
> However it gets the timeout test backwards.
>
> I don't want to set the timeout when the array starts to appear. I want to
> set the time out when someone wants to use the array.
> If no-one is waiting for the array device, then there is no point forcing it.
>
> That's why I want to plug into the timeout that systemd already has.
>
> Maybe that requirement isn't really necessary though. I'll experiment with
> your approach.
It is useless to even try to plug into the existing systemd timeout,
for a very simple reason. in the setups where your RAID array is not
on the top of the storage device hierarchy, systemd does not know that
it wants your RAID array to appear.
So the statement "If no-one is waiting for the array device, then
there is no point forcing it" is false, because there is no way to
know that no-one is waiting.
--
Alexander E. Patrakov
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, 12.11.13 11:31, NeilBrown (ne...@suse.de) wrote: > > Hi, > I wonder if I could get some advice > > mdadm is quite good at assembling arrays incrementally. "udev" runs > "mdadm -I" for each new device and mdadm gathers them into arrays and > activates the array once all the expected devices have been gathered. > > This simple rule falls down if some expected device doesn't appear. > > If an array is marked as degraded then mdadm doesn't expect the missing > device and will happily start the degaded array. However if the system shuts > down with the md array fully functional, and a device is removed before the > system is restarted, then mdadm does not know that the device is missing and > will effectively wait for it forever. > > mdadm can handle this, but it needs to be told. The command: > mdadm -IRs > will find any arrays which have enough devices to start degraded but haven't > been started yet, and will start them. > I used this quite effectively in out initrd. There is a loop that count up > to N waiting for the root device to appear and when we get to "N/2" I run > "mdadm -IRs" which makes any newly-degraded arrays appear. > > I'm wondering how to integrate this with systemd. Systemd has its own > mechanism to wait for devices to appear, but I cannot see anyway to run > "mdadm -IRs" at the half-way mark. > > It would of course be sufficient to wait for the complete timeout, then run > "mdadm -IRs", then check if the device has appeared, but I can't see how to > fit this into systemd either. > > So: how can I fit this need for "run some command on device timeout which > might be successful in activating the device"? > > Alternately, is there some "all devices have been probed, nothing new will > appear unless it is hot-plugged" event. That would be equally useful (and > probably mirrors what hardware-RAID cards do). As pointed out by the others in the thread: there is no point in time where all devices have shown. Whether USB, PCI Express or iSCSI or even other stuff: assuming there was a point in time where everything has shown up is wrong. There's no such point in time. I also have the suspicion that the best strategy for handling degraded arrays is to timeout and not assemble them but rather put the system in a state where the admin has to become active. Auto-assembling degraded arrays has the feel of taping over issues. If the admin chooses to boot-up with degraded disks then that's ok, but I am pretty sure this is something the admin should explicitly decide. Note that systemd will timeout already when we look for devices to mount. Hence, you could just decide to do nothing in mdadm: when the stuff doesn't appear then systemd will already now put the system into emergency mode and give the admin a shell. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, 12 Nov 2013 18:49:34 +0100 Colin Guthrie wrote: > 'Twas brillig, and NeilBrown at 12/11/13 11:17 did gyre and gimble: > > On Tue, 12 Nov 2013 18:16:24 +0900 Greg KH > > wrote: > > > >> On Tue, Nov 12, 2013 at 07:54:42PM +1100, NeilBrown wrote: > >>> On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH > >>> wrote: > >>> > On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote: > > Alternately, is there some "all devices have been probed, nothing new > > will > > appear unless it is hot-plugged" event. That would be equally useful > > (and > > probably mirrors what hardware-RAID cards do). > > No, there's no way to ever know this in a hotplug world, sorry. > Especially with USB devices, they show up when they show up, there's no > "oh look, the bus is all scanned now and all devices currently plugged > in are found" type knowledge at all. > > Then there are hotplug PCI systems where people slam in PCI cards > whenever they feel like it (remember, thunderbolt is PCI express...) > > Sorry, > > greg k-h > >>> > >>> Surely something must be possible. > >> > >> For USB, nope, there isn't, sorry. > >> > >>> Clearly a physical hot-plug event will cause more devices to appear, but > >>> there must come a point at which no more (non-virtual) devices will appear > >>> unless a physical event happens? > >> > >> Not for USB, sorry. > >> > >> The USB bus just announces devices when it finds them, there is no "all > >> is quiet" type signal or detection. > >> > >> Same for PCI hotplug, devices can show up at any point in time, you > >> never know when, and you don't know when all devices are "found". > >> > >> sorry, > >> > >> greg k-h > > > > > > Hmmm... OK. USB doesn't bother me a lot, but PCI is important. > > > > I guess I'll just have to settle for a timeout much like the current > > device-discovery timeout that systemd has. > > Still hoping someone can tell me how to plug into that though... > > Wouldn't it be nicer to work on a nice text-UI/wizard type thing that > would allow an admin to manually say "yes, I appreciate that the raid > array is degraded and I would like to start it anyway"? > > It seems to me that doing this automatically is a bad idea if someone > simply forgot to plug in a drive... or, and this has happened to me > (tho' I stress not *by* me!), removes the wrong drive. I guess I'm in > two minds on this one as I can see the usefulness of just coping and > carrying on but the prospect of a multi-day resync with modern large > disks for a simple mistake isn't too nice either! > > There doesn't appear to be any generic way to get user approval for such > interactive questions, but perhaps this is the kind of infrastructure > that should be provided - perhaps similar to the whole password agent thing? > > Maybe this is just all a bit too much tho'... Thanks for the suggestion. However lots of machines are expected to boot unattended and that is the case that I really want to work. I think boot should only fail if it cannot possibly succeed (without risking serious corruption). Thanks, NeilBrown signature.asc Description: PGP signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, 12 Nov 2013 19:01:49 +0400 Andrey Borzenkov
wrote:
> В Tue, 12 Nov 2013 21:17:19 +1100
> NeilBrown пишет:
>
> > On Tue, 12 Nov 2013 18:16:24 +0900 Greg KH
> > wrote:
> >
> > > On Tue, Nov 12, 2013 at 07:54:42PM +1100, NeilBrown wrote:
> > > > On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH
> > > > wrote:
> > > >
> > > > > On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote:
> > > > > > Alternately, is there some "all devices have been probed, nothing
> > > > > > new will
> > > > > > appear unless it is hot-plugged" event. That would be equally
> > > > > > useful (and
> > > > > > probably mirrors what hardware-RAID cards do).
> > > > >
> > > > > No, there's no way to ever know this in a hotplug world, sorry.
> > > > > Especially with USB devices, they show up when they show up, there's
> > > > > no
> > > > > "oh look, the bus is all scanned now and all devices currently plugged
> > > > > in are found" type knowledge at all.
> > > > >
> > > > > Then there are hotplug PCI systems where people slam in PCI cards
> > > > > whenever they feel like it (remember, thunderbolt is PCI express...)
> > > > >
> > > > > Sorry,
> > > > >
> > > > > greg k-h
> > > >
> > > > Surely something must be possible.
> > >
> > > For USB, nope, there isn't, sorry.
> > >
> > > > Clearly a physical hot-plug event will cause more devices to appear, but
> > > > there must come a point at which no more (non-virtual) devices will
> > > > appear
> > > > unless a physical event happens?
> > >
> > > Not for USB, sorry.
> > >
> > > The USB bus just announces devices when it finds them, there is no "all
> > > is quiet" type signal or detection.
> > >
> > > Same for PCI hotplug, devices can show up at any point in time, you
> > > never know when, and you don't know when all devices are "found".
> > >
> > > sorry,
> > >
> > > greg k-h
> >
> >
> > Hmmm... OK. USB doesn't bother me a lot, but PCI is important.
> >
> > I guess I'll just have to settle for a timeout much like the current
> > device-discovery timeout that systemd has.
> > Still hoping someone can tell me how to plug into that though...
> >
>
> If information about array name or other identification is available in
> udev rule (I see reference to device node only) what you can do is to
> start timer with "now+5second" (pick your timeout) that simply fires off
> mdadm -IRs for specific array. Something like
>
> mdadm-last-resort@.timer
>
> [Timer]
> OnCalendar=+5s
>
> mdadm-last-resort@.service
>
> [Service]
> Type=oneshot
> ExecStart=/sbin/mdadm -IRs %n
>
> udev rule
>
> ... SYSTEMD_WANTS=mdadm-last-resort@$ENV{SOMETHING_UNIQUE}.timer
>
Thanks. This certainly looks interesting and might be part of a solution.
However it gets the timeout test backwards.
I don't want to set the timeout when the array starts to appear. I want to
set the time out when someone wants to use the array.
If no-one is waiting for the array device, then there is no point forcing it.
That's why I want to plug into the timeout that systemd already has.
Maybe that requirement isn't really necessary though. I'll experiment with
your approach.
Thanks,
NeilBrown
signature.asc
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, 12 Nov 2013 14:52:43 +0100 Zbigniew Jędrzejewski-Szmek wrote: > On Tue, Nov 12, 2013 at 09:17:19PM +1100, NeilBrown wrote: > > On Tue, 12 Nov 2013 18:16:24 +0900 Greg KH > > wrote: > > > Not for USB, sorry. > > > > > > The USB bus just announces devices when it finds them, there is no "all > > > is quiet" type signal or detection. > > > > > > Same for PCI hotplug, devices can show up at any point in time, you > > > never know when, and you don't know when all devices are "found". > > > > Hmmm... OK. USB doesn't bother me a lot, but PCI is important. > > > > I guess I'll just have to settle for a timeout much like the current > > device-discovery timeout that systemd has. > > Still hoping someone can tell me how to plug into that though... > I think you'll have to replicate the waiting logic internally. udev > will tell you about new devices, but systemd by itself doesn't know > what devices are part of the raid set, so can't help with knowing > when enough physical volumes have been found, either for degraded more > or for normal mode. Just make sure to set your timeout to something > lower than systemd, i.e. have timeout-waiting-for-normal-set < > timeout-waiting-for-degraded-set < systemd-timeout-for-device. > > Zbyszek Internally to what? Systemd knows that it wants to mount a particular device. It waits for it and eventually times out. At that point, when systemd has timed out, I want it to tell me so I can try to make the device appear. Systemd would then need to check if the device actually has appeared and mount it if it has, or fall back on the current failure handing if it hasn't. Thanks, NeilBrown signature.asc Description: PGP signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, 12 Nov 2013 03:03:44 +0100 (CET) Jan Engelhardt wrote: > > On Tuesday 2013-11-12 01:31, NeilBrown wrote: > > > >mdadm is quite good at assembling arrays incrementally. "udev" runs > >"mdadm -I" for each new device and mdadm gathers them into arrays and > >activates the array once all the expected devices have been gathered. > >[mdadm might] wait [...] forever. > >mdadm can handle this, but it needs to be told. The command: > > mdadm -IRs > >will find any arrays which have enough devices to start degraded but haven't > >been started yet, and will start them. > > Using mdadm -R in udev would be a really bad idea, because it would > make for a race, I think. Consider a system that has finished booting > and is as ready as can be, and you plug in USB disks that form a RAID > array. If you use -R, the array may start before you had the chance > to plug them all in, causing the array to start in degraded mode, > potentially causing a long resync when all missing devices have been > plugged into their sockets. > > >Alternately, is there some "all devices have been probed, nothing new will > >appear unless it is hot-plugged" event. That would be equally useful (and > >probably mirrors what hardware-RAID cards do). > > IIRC, openSUSE had a /etc/init.d/boot.coldplug once upon a time > > (seems to be 9.3) that acted like what you describe. > The order was like udevadm trigger, load static module list, > udevadm settle, run mdadm, chroot /realroot, repeat for real root > once more. Would that still work? It is easy to do what I describe using ad hoc scripts - that's what I do in the initrd. But for no-root filesystems I don't really want to use some ad-hoc script. I want to integrate into systemd to benefit from all the good bits of systemd. Thanks, NeilBrown signature.asc Description: PGP signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On 11/12/2013 11:49 AM, Colin Guthrie wrote: > It seems to me that doing this automatically is a bad idea if someone > simply forgot to plug in a drive... or, and this has happened to me > (tho' I stress not *by* me!), removes the wrong drive. I guess I'm in > two minds on this one as I can see the usefulness of just coping and > carrying on but the prospect of a multi-day resync with modern large > disks for a simple mistake isn't too nice either! It can be hard to get user input during early boot on a headless server. ;-) I think that there are actually several related issues here: 1) A way to express per-array policy, maybe in mdadm.conf. - Wait XX seconds for all devices to show up, then start degraded - Wait for some other sort of event before starting degraded - Never start degraded - Start as soon as possible, even if degraded (not sure what the use case for this one would be) 2) Implementation of those policies -- udev rules, dracut, systemd, mdadm? 3) Sensible "hardcoded" defaults 4) Support for creating appropriate policies in tools like anaconda Only after 1 and 2 are in place can 3 and 4 be meaningfully addressed. -- Ian Pilcher arequip...@gmail.com Sent from the cloud -- where it's already tomorrow ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
'Twas brillig, and NeilBrown at 12/11/13 11:17 did gyre and gimble: > On Tue, 12 Nov 2013 18:16:24 +0900 Greg KH wrote: > >> On Tue, Nov 12, 2013 at 07:54:42PM +1100, NeilBrown wrote: >>> On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH >>> wrote: >>> On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote: > Alternately, is there some "all devices have been probed, nothing new will > appear unless it is hot-plugged" event. That would be equally useful (and > probably mirrors what hardware-RAID cards do). No, there's no way to ever know this in a hotplug world, sorry. Especially with USB devices, they show up when they show up, there's no "oh look, the bus is all scanned now and all devices currently plugged in are found" type knowledge at all. Then there are hotplug PCI systems where people slam in PCI cards whenever they feel like it (remember, thunderbolt is PCI express...) Sorry, greg k-h >>> >>> Surely something must be possible. >> >> For USB, nope, there isn't, sorry. >> >>> Clearly a physical hot-plug event will cause more devices to appear, but >>> there must come a point at which no more (non-virtual) devices will appear >>> unless a physical event happens? >> >> Not for USB, sorry. >> >> The USB bus just announces devices when it finds them, there is no "all >> is quiet" type signal or detection. >> >> Same for PCI hotplug, devices can show up at any point in time, you >> never know when, and you don't know when all devices are "found". >> >> sorry, >> >> greg k-h > > > Hmmm... OK. USB doesn't bother me a lot, but PCI is important. > > I guess I'll just have to settle for a timeout much like the current > device-discovery timeout that systemd has. > Still hoping someone can tell me how to plug into that though... Wouldn't it be nicer to work on a nice text-UI/wizard type thing that would allow an admin to manually say "yes, I appreciate that the raid array is degraded and I would like to start it anyway"? It seems to me that doing this automatically is a bad idea if someone simply forgot to plug in a drive... or, and this has happened to me (tho' I stress not *by* me!), removes the wrong drive. I guess I'm in two minds on this one as I can see the usefulness of just coping and carrying on but the prospect of a multi-day resync with modern large disks for a simple mistake isn't too nice either! There doesn't appear to be any generic way to get user approval for such interactive questions, but perhaps this is the kind of infrastructure that should be provided - perhaps similar to the whole password agent thing? Maybe this is just all a bit too much tho'... Col -- Colin Guthrie gmane(at)colin.guthr.ie http://colin.guthr.ie/ Day Job: Tribalogic Limited http://www.tribalogic.net/ Open Source: Mageia Contributor http://www.mageia.org/ PulseAudio Hacker http://www.pulseaudio.org/ Trac Hacker http://trac.edgewall.org/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
В Tue, 12 Nov 2013 21:17:19 +1100
NeilBrown пишет:
> On Tue, 12 Nov 2013 18:16:24 +0900 Greg KH wrote:
>
> > On Tue, Nov 12, 2013 at 07:54:42PM +1100, NeilBrown wrote:
> > > On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH
> > > wrote:
> > >
> > > > On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote:
> > > > > Alternately, is there some "all devices have been probed, nothing new
> > > > > will
> > > > > appear unless it is hot-plugged" event. That would be equally useful
> > > > > (and
> > > > > probably mirrors what hardware-RAID cards do).
> > > >
> > > > No, there's no way to ever know this in a hotplug world, sorry.
> > > > Especially with USB devices, they show up when they show up, there's no
> > > > "oh look, the bus is all scanned now and all devices currently plugged
> > > > in are found" type knowledge at all.
> > > >
> > > > Then there are hotplug PCI systems where people slam in PCI cards
> > > > whenever they feel like it (remember, thunderbolt is PCI express...)
> > > >
> > > > Sorry,
> > > >
> > > > greg k-h
> > >
> > > Surely something must be possible.
> >
> > For USB, nope, there isn't, sorry.
> >
> > > Clearly a physical hot-plug event will cause more devices to appear, but
> > > there must come a point at which no more (non-virtual) devices will appear
> > > unless a physical event happens?
> >
> > Not for USB, sorry.
> >
> > The USB bus just announces devices when it finds them, there is no "all
> > is quiet" type signal or detection.
> >
> > Same for PCI hotplug, devices can show up at any point in time, you
> > never know when, and you don't know when all devices are "found".
> >
> > sorry,
> >
> > greg k-h
>
>
> Hmmm... OK. USB doesn't bother me a lot, but PCI is important.
>
> I guess I'll just have to settle for a timeout much like the current
> device-discovery timeout that systemd has.
> Still hoping someone can tell me how to plug into that though...
>
If information about array name or other identification is available in
udev rule (I see reference to device node only) what you can do is to
start timer with "now+5second" (pick your timeout) that simply fires off
mdadm -IRs for specific array. Something like
mdadm-last-resort@.timer
[Timer]
OnCalendar=+5s
mdadm-last-resort@.service
[Service]
Type=oneshot
ExecStart=/sbin/mdadm -IRs %n
udev rule
... SYSTEMD_WANTS=mdadm-last-resort@$ENV{SOMETHING_UNIQUE}.timer
signature.asc
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, Nov 12, 2013 at 09:17:19PM +1100, NeilBrown wrote: > On Tue, 12 Nov 2013 18:16:24 +0900 Greg KH wrote: > > Not for USB, sorry. > > > > The USB bus just announces devices when it finds them, there is no "all > > is quiet" type signal or detection. > > > > Same for PCI hotplug, devices can show up at any point in time, you > > never know when, and you don't know when all devices are "found". > > Hmmm... OK. USB doesn't bother me a lot, but PCI is important. > > I guess I'll just have to settle for a timeout much like the current > device-discovery timeout that systemd has. > Still hoping someone can tell me how to plug into that though... I think you'll have to replicate the waiting logic internally. udev will tell you about new devices, but systemd by itself doesn't know what devices are part of the raid set, so can't help with knowing when enough physical volumes have been found, either for degraded more or for normal mode. Just make sure to set your timeout to something lower than systemd, i.e. have timeout-waiting-for-normal-set < timeout-waiting-for-degraded-set < systemd-timeout-for-device. Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, 12 Nov 2013 18:16:24 +0900 Greg KH wrote: > On Tue, Nov 12, 2013 at 07:54:42PM +1100, NeilBrown wrote: > > On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH > > wrote: > > > > > On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote: > > > > Alternately, is there some "all devices have been probed, nothing new > > > > will > > > > appear unless it is hot-plugged" event. That would be equally useful > > > > (and > > > > probably mirrors what hardware-RAID cards do). > > > > > > No, there's no way to ever know this in a hotplug world, sorry. > > > Especially with USB devices, they show up when they show up, there's no > > > "oh look, the bus is all scanned now and all devices currently plugged > > > in are found" type knowledge at all. > > > > > > Then there are hotplug PCI systems where people slam in PCI cards > > > whenever they feel like it (remember, thunderbolt is PCI express...) > > > > > > Sorry, > > > > > > greg k-h > > > > Surely something must be possible. > > For USB, nope, there isn't, sorry. > > > Clearly a physical hot-plug event will cause more devices to appear, but > > there must come a point at which no more (non-virtual) devices will appear > > unless a physical event happens? > > Not for USB, sorry. > > The USB bus just announces devices when it finds them, there is no "all > is quiet" type signal or detection. > > Same for PCI hotplug, devices can show up at any point in time, you > never know when, and you don't know when all devices are "found". > > sorry, > > greg k-h Hmmm... OK. USB doesn't bother me a lot, but PCI is important. I guess I'll just have to settle for a timeout much like the current device-discovery timeout that systemd has. Still hoping someone can tell me how to plug into that though... Thanks, NeilBrown signature.asc Description: PGP signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, Nov 12, 2013 at 07:54:42PM +1100, NeilBrown wrote: > On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH wrote: > > > On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote: > > > Alternately, is there some "all devices have been probed, nothing new will > > > appear unless it is hot-plugged" event. That would be equally useful (and > > > probably mirrors what hardware-RAID cards do). > > > > No, there's no way to ever know this in a hotplug world, sorry. > > Especially with USB devices, they show up when they show up, there's no > > "oh look, the bus is all scanned now and all devices currently plugged > > in are found" type knowledge at all. > > > > Then there are hotplug PCI systems where people slam in PCI cards > > whenever they feel like it (remember, thunderbolt is PCI express...) > > > > Sorry, > > > > greg k-h > > Surely something must be possible. For USB, nope, there isn't, sorry. > Clearly a physical hot-plug event will cause more devices to appear, but > there must come a point at which no more (non-virtual) devices will appear > unless a physical event happens? Not for USB, sorry. The USB bus just announces devices when it finds them, there is no "all is quiet" type signal or detection. Same for PCI hotplug, devices can show up at any point in time, you never know when, and you don't know when all devices are "found". sorry, greg k-h ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, Nov 12, 2013 at 06:16:24PM +0900, Greg KH wrote: > On Tue, Nov 12, 2013 at 07:54:42PM +1100, NeilBrown wrote: > > On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH > > wrote: > > > > > On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote: > > > > Alternately, is there some "all devices have been probed, nothing new > > > > will > > > > appear unless it is hot-plugged" event. That would be equally useful > > > > (and > > > > probably mirrors what hardware-RAID cards do). > > > > > > No, there's no way to ever know this in a hotplug world, sorry. > > > Especially with USB devices, they show up when they show up, there's no > > > "oh look, the bus is all scanned now and all devices currently plugged > > > in are found" type knowledge at all. > > > > > > Then there are hotplug PCI systems where people slam in PCI cards > > > whenever they feel like it (remember, thunderbolt is PCI express...) > > > > > > Sorry, > > > > > > greg k-h > > > > Surely something must be possible. > > For USB, nope, there isn't, sorry. > > > Clearly a physical hot-plug event will cause more devices to appear, but > > there must come a point at which no more (non-virtual) devices will appear > > unless a physical event happens? > > Not for USB, sorry. > > The USB bus just announces devices when it finds them, there is no "all > is quiet" type signal or detection. > > Same for PCI hotplug, devices can show up at any point in time, you > never know when, and you don't know when all devices are "found". Oh, and even if there was such a "flag", it wouldn't really matter, because a device could be plugged in right after you think everything is 'quiet'. Which is probably the reason why this isn't possible, even if you had it, you couldn't rely on it. greg k-h ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, 12 Nov 2013 00:10:28 -0800 Greg KH wrote: > On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote: > > Alternately, is there some "all devices have been probed, nothing new will > > appear unless it is hot-plugged" event. That would be equally useful (and > > probably mirrors what hardware-RAID cards do). > > No, there's no way to ever know this in a hotplug world, sorry. > Especially with USB devices, they show up when they show up, there's no > "oh look, the bus is all scanned now and all devices currently plugged > in are found" type knowledge at all. > > Then there are hotplug PCI systems where people slam in PCI cards > whenever they feel like it (remember, thunderbolt is PCI express...) > > Sorry, > > greg k-h Surely something must be possible. Clearly a physical hot-plug event will cause more devices to appear, but there must come a point at which no more (non-virtual) devices will appear unless a physical event happens? Or may unless "a physical event or a deliberate configuration change" happens. Maybe it is hard. Maybe it is too hard. But surely it is at least theoretically possible. NeilBrown signature.asc Description: PGP signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tue, Nov 12, 2013 at 11:31:45AM +1100, NeilBrown wrote: > Alternately, is there some "all devices have been probed, nothing new will > appear unless it is hot-plugged" event. That would be equally useful (and > probably mirrors what hardware-RAID cards do). No, there's no way to ever know this in a hotplug world, sorry. Especially with USB devices, they show up when they show up, there's no "oh look, the bus is all scanned now and all devices currently plugged in are found" type knowledge at all. Then there are hotplug PCI systems where people slam in PCI cards whenever they feel like it (remember, thunderbolt is PCI express...) Sorry, greg k-h ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Need help with a systemd/mdadm interaction.
On Tuesday 2013-11-12 01:31, NeilBrown wrote: > >mdadm is quite good at assembling arrays incrementally. "udev" runs >"mdadm -I" for each new device and mdadm gathers them into arrays and >activates the array once all the expected devices have been gathered. >[mdadm might] wait [...] forever. >mdadm can handle this, but it needs to be told. The command: > mdadm -IRs >will find any arrays which have enough devices to start degraded but haven't >been started yet, and will start them. Using mdadm -R in udev would be a really bad idea, because it would make for a race, I think. Consider a system that has finished booting and is as ready as can be, and you plug in USB disks that form a RAID array. If you use -R, the array may start before you had the chance to plug them all in, causing the array to start in degraded mode, potentially causing a long resync when all missing devices have been plugged into their sockets. >Alternately, is there some "all devices have been probed, nothing new will >appear unless it is hot-plugged" event. That would be equally useful (and >probably mirrors what hardware-RAID cards do). IIRC, openSUSE had a /etc/init.d/boot.coldplug once upon a time (seems to be 9.3) that acted like what you describe. The order was like udevadm trigger, load static module list, udevadm settle, run mdadm, chroot /realroot, repeat for real root once more. Would that still work? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Need help with a systemd/mdadm interaction.
Hi, I wonder if I could get some advice mdadm is quite good at assembling arrays incrementally. "udev" runs "mdadm -I" for each new device and mdadm gathers them into arrays and activates the array once all the expected devices have been gathered. This simple rule falls down if some expected device doesn't appear. If an array is marked as degraded then mdadm doesn't expect the missing device and will happily start the degaded array. However if the system shuts down with the md array fully functional, and a device is removed before the system is restarted, then mdadm does not know that the device is missing and will effectively wait for it forever. mdadm can handle this, but it needs to be told. The command: mdadm -IRs will find any arrays which have enough devices to start degraded but haven't been started yet, and will start them. I used this quite effectively in out initrd. There is a loop that count up to N waiting for the root device to appear and when we get to "N/2" I run "mdadm -IRs" which makes any newly-degraded arrays appear. I'm wondering how to integrate this with systemd. Systemd has its own mechanism to wait for devices to appear, but I cannot see anyway to run "mdadm -IRs" at the half-way mark. It would of course be sufficient to wait for the complete timeout, then run "mdadm -IRs", then check if the device has appeared, but I can't see how to fit this into systemd either. So: how can I fit this need for "run some command on device timeout which might be successful in activating the device"? Alternately, is there some "all devices have been probed, nothing new will appear unless it is hot-plugged" event. That would be equally useful (and probably mirrors what hardware-RAID cards do). Thanks, NeilBrown signature.asc Description: PGP signature ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
