Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup
On Thu, 2018-06-14 at 06:42 -0700, Christoph Hellwig wrote: > On Thu, Jun 14, 2018 at 01:39:50PM +, Bart Van Assche wrote: > > On Thu, 2018-06-14 at 10:01 +, Damien Le Moal wrote: > > > Applied. Thanks Luis ! > > > > Hello Damien, > > > > Can this still be undone? I agree with Mike that it's wrong to > > invoke > > "/sbin/dmsetup create ... zoned ..." from a udev rule. > > Yes. We'll really need to verfify the device has dm-zoned metadata > first. Preferably including a uuid for stable device naming. libblkid would be the central hub for metadata discovery, so perhaps a patch should be made to make libblkid dm-zoned-aware. Anyway, as Damien explained, dmzoned bails out if it doesn't find matching meta data, so AFAICS, little harm is done by calling it for a SMR device in host-managed mode. I fail to get the point why this would be wrong in general - what's the difference to e.g. calling "mdadm -I"? Regards Martin -- Dr. Martin Wilck , Tel. +49 (0)911 74053 2107 SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)
Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup
On Thu, Jun 14, 2018 at 04:19:23PM +, Bart Van Assche wrote:
> On Wed, 2018-06-13 at 17:11 -0700, Luis R. Rodriguez wrote:
> > This tries to put a bit of this tribal knowledge into an initial udev
> > rule for development with the hopes Linux distributions can later
> > deploy. Three rule are added. One rule is optional for now, it should be
> > extended later to be more distribution-friendly and then I think this
> > may be ready for consideration for integration on distributions.
> >
> > 1) scheduler setup
> > 2) backlist f2fs devices
> > 3) run dmsetup for the rest of devices
>
> Hello Luis,
>
> I think it is wrong to package the zoned block device scheduler rule in the
> dm-zoned-tools package. That udev rule should be activated whether or not the
> dm-zoned-tools package has been installed. Have you considered to submit the
> zoned block device scheduler rule to the systemd project since today that
> project includes all base udev rules?
Nope, this is a udev rule intended for developers wishing to brush up on our
state of affairs. All I wanted to do is to get my own drive to work at home in
a reasonably reliable and generic form, which also allowed me to hop around
kernels without a fatal issue.
Clearly there is much to discuss still and a roadmap to illustrate. We should
update the documentation to reflect this first, and based on that enable then
distributions based on that discussion. We should have short term and long term
plans. That discussion may have taken place already so forgive me for not
knowing about it, I did try to read as much from the code and existing tools
and just didn't find anything else to be clear on it.
> > +# Zoned disks can only work with the deadline or mq-deadline scheduler.
> > This is
> > +# mandated for all SMR drives since v4.16. It has been determined this
> > must be
> > +# done through a udev rule, and the kernel should not set this up for
> > disks.
> > +# This magic will have to live for *all* zoned disks.
> > +# XXX: what about distributions that want mq-deadline ? Probably easy for
> > now
> > +# to assume deadline and later have a mapping file to enable
> > +# mq-deadline for specific serial devices?
> > +ACTION=="add|change", KERNEL=="sd*[!0-9]",
> > ATTRS{queue/zoned}=="host-managed", \
> > + ATTR{queue/scheduler}="deadline"
>
> I think it is wrong to limit this rule to SCSI disks only. Work is ongoing to
> add zoned block device support to the null_blk driver. That is a block driver
> and not a SCSI driver. I think the above udev rule should apply to that block
> driver too.
Sure patches welcomed :)
> Regarding blk-mq, from the mq-deadline source code:
> .elevator_alias = "deadline",
>
> In other words, the name "deadline" should work both for legacy and for blk-mq
> block devices.
Groovy that helps.
Luis
Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup
On Thu, Jun 14, 2018 at 01:39:50PM +, Bart Van Assche wrote: > On Thu, 2018-06-14 at 10:01 +, Damien Le Moal wrote: > > Applied. Thanks Luis ! > > Hello Damien, > > Can this still be undone? I agree with Mike that it's wrong to invoke > "/sbin/dmsetup create ... zoned ..." from a udev rule. Yes. We'll really need to verfify the device has dm-zoned metadata first. Preferably including a uuid for stable device naming.
Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup
On Thu, 2018-06-14 at 10:01 +, Damien Le Moal wrote: > Applied. Thanks Luis ! Hello Damien, Can this still be undone? I agree with Mike that it's wrong to invoke "/sbin/dmsetup create ... zoned ..." from a udev rule. Thanks, Bart.
Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup
On 6/14/18 09:11, Luis R. Rodriguez wrote:
> Setting up a zoned disks in a generic form is not so trivial. There
> is also quite a bit of tribal knowledge with these devices which is not
> easy to find.
>
> The currently supplied demo script works but it is not generic enough to be
> practical for Linux distributions or even developers which often move
> from one kernel to another.
>
> This tries to put a bit of this tribal knowledge into an initial udev
> rule for development with the hopes Linux distributions can later
> deploy. Three rule are added. One rule is optional for now, it should be
> extended later to be more distribution-friendly and then I think this
> may be ready for consideration for integration on distributions.
>
> 1) scheduler setup
> 2) backlist f2fs devices
> 3) run dmsetup for the rest of devices
>
> Note that this udev rule will not work well if you want to use a disk
> with f2fs on part of the disk and another filesystem on another part of
> the disk. That setup will require manual love so these setups can use
> the same backlist on rule 2).
>
> Its not widely known for instance that as of v4.16 it is mandated to use
> either deadline or the mq-deadline scheduler for *all* SMR drivers. Its
> also been determined that the Linux kernel is not the place to set this up,
> so a udev rule *is required* as per latest discussions. This is the
> first rule we add.
>
> Furthermore if you are *not* using f2fs you always have to run dmsetup.
> dmsetups do not persist, so you currently *always* have to run a custom
> sort of script, which is not ideal for Linux distributions. We can invert
> this logic into a udev rule to enable users to blacklist disks they know they
> want to use f2fs for. This the second optional rule. This blacklisting
> can be generalized further in the future with an exception list file, for
> instance using INPUT{db} or the like.
>
> The third and final rule added then runs dmsetup for the rest of the disks
> using the disk serial number for the new device mapper name.
>
> Note that it is currently easy for users to make a mistake and run mkfs
> on the the original disk, not the /dev/mapper/ device for non f2fs
> arrangements. If that is done experience shows things can easily fall
> apart with alignment *eventually*. We have no generic way today to
> error out on this condition and proactively prevent this.
>
> Signed-off-by: Luis R. Rodriguez
> ---
> README| 10 +-
> udev/99-zoned-disks.rules | 78
> +++
> 2 files changed, 87 insertions(+), 1 deletion(-)
> create mode 100644 udev/99-zoned-disks.rules
>
> diff --git a/README b/README
> index 65e96c34fd04..f49541eaabc8 100644
> --- a/README
> +++ b/README
> @@ -168,7 +168,15 @@ Options:
> reclaiming random zones if the percentage of
> free random data zones falls below .
>
> -V. Example scripts
> +V. Udev zone disk deployment
> +
> +
> +A udev rule is provided which enables you to set the IO scheduler, blacklist
> +driver to run dmsetup, and runs dmsetup for the rest of the zone drivers.
> +If you use this udev rule the below script is not needed. Be sure to mkfs
> only
> +on the resulting /dev/mapper/zone-$serial device you end up with.
> +
> +VI. Example scripts
> ==
>
> [[
> diff --git a/udev/99-zoned-disks.rules b/udev/99-zoned-disks.rules
> new file mode 100644
> index ..e19b738dcc0e
> --- /dev/null
> +++ b/udev/99-zoned-disks.rules
> @@ -0,0 +1,78 @@
> +# To use a zone disks first thing you need to:
> +#
> +# 1) Enable zone disk support in your kernel
> +# 2) Use the deadline or mq-deadline scheduler for it - mandated as of v4.16
> +# 3) Blacklist devices dedicated for f2fs as of v4.10
> +# 4) Run dmsetup other disks
> +# 5) Create the filesystem -- NOTE: use mkfs /dev/mapper/zone-serial if
> +#you enabled use dmsetup on the disk.
> +# 6) Consider using nofail mount option in case you run an supported kernel
> +#
> +# You can use this udev rules file for 2) 3) and 4). Further details below.
> +#
> +# 1) Enable zone disk support in your kernel
> +#
> +#o CONFIG_BLK_DEV_ZONED
> +#o CONFIG_DM_ZONED
> +#
> +# This will let the kernel actually see these devices, ie, via fdisk /dev/sda
> +# for instance. Run:
> +#
> +#dmzadm --format /dev/sda
> +
> +# 2) Set deadline or mq-deadline for all disks which are zoned
> +#
> +# Zoned disks can only work with the deadline or mq-deadline scheduler. This
> is
> +# mandated for all SMR drives since v4.16. It has been determined this must
> be
> +# done through a udev rule, and the kernel should not set this up for disks.
> +# This magic will have to live for *all* zoned disks.
> +# XXX: what about distributions that want mq-deadline ? Probably easy for now
> +# to assume deadline and later have a mapping file to enable
> +# mq-deadline for specific serial devices?
[PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup
Setting up a zoned disks in a generic form is not so trivial. There
is also quite a bit of tribal knowledge with these devices which is not
easy to find.
The currently supplied demo script works but it is not generic enough to be
practical for Linux distributions or even developers which often move
from one kernel to another.
This tries to put a bit of this tribal knowledge into an initial udev
rule for development with the hopes Linux distributions can later
deploy. Three rule are added. One rule is optional for now, it should be
extended later to be more distribution-friendly and then I think this
may be ready for consideration for integration on distributions.
1) scheduler setup
2) backlist f2fs devices
3) run dmsetup for the rest of devices
Note that this udev rule will not work well if you want to use a disk
with f2fs on part of the disk and another filesystem on another part of
the disk. That setup will require manual love so these setups can use
the same backlist on rule 2).
Its not widely known for instance that as of v4.16 it is mandated to use
either deadline or the mq-deadline scheduler for *all* SMR drivers. Its
also been determined that the Linux kernel is not the place to set this up,
so a udev rule *is required* as per latest discussions. This is the
first rule we add.
Furthermore if you are *not* using f2fs you always have to run dmsetup.
dmsetups do not persist, so you currently *always* have to run a custom
sort of script, which is not ideal for Linux distributions. We can invert
this logic into a udev rule to enable users to blacklist disks they know they
want to use f2fs for. This the second optional rule. This blacklisting
can be generalized further in the future with an exception list file, for
instance using INPUT{db} or the like.
The third and final rule added then runs dmsetup for the rest of the disks
using the disk serial number for the new device mapper name.
Note that it is currently easy for users to make a mistake and run mkfs
on the the original disk, not the /dev/mapper/ device for non f2fs
arrangements. If that is done experience shows things can easily fall
apart with alignment *eventually*. We have no generic way today to
error out on this condition and proactively prevent this.
Signed-off-by: Luis R. Rodriguez
---
README| 10 +-
udev/99-zoned-disks.rules | 78 +++
2 files changed, 87 insertions(+), 1 deletion(-)
create mode 100644 udev/99-zoned-disks.rules
diff --git a/README b/README
index 65e96c34fd04..f49541eaabc8 100644
--- a/README
+++ b/README
@@ -168,7 +168,15 @@ Options:
reclaiming random zones if the percentage of
free random data zones falls below .
-V. Example scripts
+V. Udev zone disk deployment
+
+
+A udev rule is provided which enables you to set the IO scheduler, blacklist
+driver to run dmsetup, and runs dmsetup for the rest of the zone drivers.
+If you use this udev rule the below script is not needed. Be sure to mkfs only
+on the resulting /dev/mapper/zone-$serial device you end up with.
+
+VI. Example scripts
==
[[
diff --git a/udev/99-zoned-disks.rules b/udev/99-zoned-disks.rules
new file mode 100644
index ..e19b738dcc0e
--- /dev/null
+++ b/udev/99-zoned-disks.rules
@@ -0,0 +1,78 @@
+# To use a zone disks first thing you need to:
+#
+# 1) Enable zone disk support in your kernel
+# 2) Use the deadline or mq-deadline scheduler for it - mandated as of v4.16
+# 3) Blacklist devices dedicated for f2fs as of v4.10
+# 4) Run dmsetup other disks
+# 5) Create the filesystem -- NOTE: use mkfs /dev/mapper/zone-serial if
+#you enabled use dmsetup on the disk.
+# 6) Consider using nofail mount option in case you run an supported kernel
+#
+# You can use this udev rules file for 2) 3) and 4). Further details below.
+#
+# 1) Enable zone disk support in your kernel
+#
+#o CONFIG_BLK_DEV_ZONED
+#o CONFIG_DM_ZONED
+#
+# This will let the kernel actually see these devices, ie, via fdisk /dev/sda
+# for instance. Run:
+#
+# dmzadm --format /dev/sda
+
+# 2) Set deadline or mq-deadline for all disks which are zoned
+#
+# Zoned disks can only work with the deadline or mq-deadline scheduler. This is
+# mandated for all SMR drives since v4.16. It has been determined this must be
+# done through a udev rule, and the kernel should not set this up for disks.
+# This magic will have to live for *all* zoned disks.
+# XXX: what about distributions that want mq-deadline ? Probably easy for now
+# to assume deadline and later have a mapping file to enable
+# mq-deadline for specific serial devices?
+ACTION=="add|change", KERNEL=="sd*[!0-9]", ATTRS{queue/zoned}=="host-managed",
\
+ ATTR{queue/scheduler}="deadline"
+
+# 3) Blacklist f2fs devices as of v4.10
+# We don't have to run dmsetup on on disks where you want to use f2fs, so you
+# can use this rule to skip
