Re: [PATCH v2 5/8] cxl/mem: Add a "RAW" send command
On Thu, 11 Feb 2021 08:01:48 -0800 Ben Widawsky wrote: > On 21-02-11 11:19:24, Jonathan Cameron wrote: > > On Tue, 9 Feb 2021 16:02:56 -0800 > > Ben Widawsky wrote: > > > > > The CXL memory device send interface will have a number of supported > > > commands. The raw command is not such a command. Raw commands allow > > > userspace to send a specified opcode to the underlying hardware and > > > bypass all driver checks on the command. This is useful for a couple of > > > usecases, mainly: > > > 1. Undocumented vendor specific hardware commands > > > > This one I get. There are things we'd love to standardize but often they > > need proving in a generation of hardware before the data is available to > > justify taking it to a standards body. Stuff like performance stats. > > This stuff will all sit in the vendor defined range. Maybe there is an > > argument for in driver hooks to allow proper support even for these > > (Ben mentioned this in the other branch of the thread). > > > > > 2. Prototyping new hardware commands not yet supported by the driver > > > > For 2, could just have a convenient place to enable this by one line patch. > > Some subsystems (SPI comes to mind) do this for their equivalent of raw > > commands. The code is all there to enable it but you need to hook it > > up if you want to use it. Avoids chance of a distro shipping it. > > > > I'm fine to drop #2 as a justification point, or maybe reword the commit > message > to say, "you could also just do... but since we have it for #1 already..." > > > > > > > While this all sounds very powerful it comes with a couple of caveats: > > > 1. Bug reports using raw commands will not get the same level of > > >attention as bug reports using supported commands (via taint). > > > 2. Supported commands will be rejected by the RAW command. > > > > Perhaps I'm missing reading this point 2 (not sure the code actually does > > it!) > > > > As stated what worries me as it means when we add support for a new > > bit of the spec we just broke the userspace ABI. > > > > It does not break ABI. The agreement is userspace must always use the QUERY > command to find out what commands are supported. If it tries to use a RAW > command that is a supported command, it will be rejected. In the case you > mention, that's an application bug. If there is a way to document that better > than what's already in the UAPI kdocs, I'm open to suggestions. > > Unlike perhaps other UAPI, this one only promises to give you a way to > determine > what commands you can use, not the list of what commands you can use. *crossed fingers* on this. Users may have a different view when their application just stops working. It might print a nice error message telling them why but it still doesn't work and that way lies grumpy Linus and reverts... Mostly we'll get away with it because no one will notice, but it's unfortunately still risky. Personal preference is toplay safer and not allow direct userspace access to commands in the spec (unless we've decided they will always be available directly to userspace). This includes anything in the ranges reserved for future spec usage. Jonathan > > > > > > > With this comes new debugfs knob to allow full access to your toes with > > > your weapon of choice. > > > > A few trivial things inline, > > > > Jonathan > > > > > > > > Cc: Ariel Sibley > > > Signed-off-by: Ben Widawsky > > > Reviewed-by: Dan Williams > > > --- > > > drivers/cxl/Kconfig | 18 + > > > drivers/cxl/mem.c| 125 ++- > > > include/uapi/linux/cxl_mem.h | 12 +++- > > > 3 files changed, 152 insertions(+), 3 deletions(-) > > > > > > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig > > > index c4ba3aa0a05d..08eaa8e52083 100644 > > > --- a/drivers/cxl/Kconfig > > > +++ b/drivers/cxl/Kconfig > > > @@ -33,6 +33,24 @@ config CXL_MEM > > > > > > If unsure say 'm'. > > > > > > +config CXL_MEM_RAW_COMMANDS > > > + bool "RAW Command Interface for Memory Devices" > > > + depends on CXL_MEM > > > + help > > > + Enable CXL RAW command interface. > > > + > > > + The CXL driver ioctl interface may assign a kernel ioctl command > > > + number for each specification defined opcode. At any given point in > > > + time the number of opcodes that the specification defines and a device > > > + may implement may exceed the kernel's set of associated ioctl function > > > + numbers. The mismatch is either by omission, specification is too new, > > > + or by design. When prototyping new hardware, or developing / debugging > > > + the driver it is useful to be able to submit any possible command to > > > + the hardware, even commands that may crash the kernel due to their > > > + potential impact to memory currently in use by the kernel. > > > + > > > + If developing CXL hardware or the driver say Y, otherwise say N. > > > + > > > config
Re: [PATCH v2 5/8] cxl/mem: Add a "RAW" send command
On Wed, Feb 10, 2021 at 7:27 AM wrote: > > > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig > > index c4ba3aa0a05d..08eaa8e52083 100644 > > --- a/drivers/cxl/Kconfig > > +++ b/drivers/cxl/Kconfig > > @@ -33,6 +33,24 @@ config CXL_MEM > > > > If unsure say 'm'. > > > > +config CXL_MEM_RAW_COMMANDS > > + bool "RAW Command Interface for Memory Devices" > > + depends on CXL_MEM > > + help > > + Enable CXL RAW command interface. > > + > > + The CXL driver ioctl interface may assign a kernel ioctl command > > + number for each specification defined opcode. At any given point > > in > > + time the number of opcodes that the specification defines and a > > device > > + may implement may exceed the kernel's set of associated ioctl > > function > > + numbers. The mismatch is either by omission, specification is too > > new, > > + or by design. When prototyping new hardware, or developing / > > debugging > > + the driver it is useful to be able to submit any possible command > > to > > + the hardware, even commands that may crash the kernel due to their > > + potential impact to memory currently in use by the kernel. > > + > > + If developing CXL hardware or the driver say Y, otherwise say N. > > Blocking RAW commands by default will prevent vendors from developing user > space tools that utilize vendor specific commands. Vendors of CXL.mem devices > should take ownership of ensuring any vendor defined commands that could > cause user data to be exposed or corrupted are disabled at the device level > for shipping configurations. What follows is my personal opinion as a Linux kernel developer, not necessarily the opinion of my employer... Aside from the convention that new functionality is always default N it is the Linux distributor that decides the configuration. In an environment where the kernel is developing features like CONFIG_SECURITY_LOCKDOWN_LSM that limit the ability of the kernel to subvert platform features like secure boot, it is incumbent upon drivers to evaluate what they must do to protect platform integrity. See the ongoing tightening of /dev/mem like interfaces for an example of the shrinking ability of root to have unfettered access to all platform/hardware capabilities. CXL is unique in that it impacts "System RAM" resources and that it interleaves multiple devices. Compare this to NVME where the blast radius of misbehavior is contained to an endpoint and is behind an IOMMU. The larger impact to me increases the responsibility of CXL enabling to review system impacts and vendor specific functionality is typically unreviewable. There are 2 proposals I can see to improve the unreviewable problem. First, of course, get commands into the standard proper. One strawman proposal is to take the "Code First" process that seems to be working well for the ACPI and UEFI working groups and apply it to CXL command definitions. That vastly shortens the time between proposal and Linux enabling. The second proposal is to define a mechanism for de-facto standards to develop. That need I believe was the motivation for "designated vendor-specific" in the first instance? I.e. to share implementations across vendors pre-standardization. So, allocate a public id for the command space, publish a public specification, and then send kernel patches. This was the process for accepting command sets outside of ACPI into the LIBNVDIMM subsystem. See drivers/acpi/nfit/nfit.h for the reference to the public command sets. ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
Re: [PATCH v2 5/8] cxl/mem: Add a "RAW" send command
On 21-02-11 11:19:24, Jonathan Cameron wrote:
> On Tue, 9 Feb 2021 16:02:56 -0800
> Ben Widawsky wrote:
>
> > The CXL memory device send interface will have a number of supported
> > commands. The raw command is not such a command. Raw commands allow
> > userspace to send a specified opcode to the underlying hardware and
> > bypass all driver checks on the command. This is useful for a couple of
> > usecases, mainly:
> > 1. Undocumented vendor specific hardware commands
>
> This one I get. There are things we'd love to standardize but often they
> need proving in a generation of hardware before the data is available to
> justify taking it to a standards body. Stuff like performance stats.
> This stuff will all sit in the vendor defined range. Maybe there is an
> argument for in driver hooks to allow proper support even for these
> (Ben mentioned this in the other branch of the thread).
>
> > 2. Prototyping new hardware commands not yet supported by the driver
>
> For 2, could just have a convenient place to enable this by one line patch.
> Some subsystems (SPI comes to mind) do this for their equivalent of raw
> commands. The code is all there to enable it but you need to hook it
> up if you want to use it. Avoids chance of a distro shipping it.
>
I'm fine to drop #2 as a justification point, or maybe reword the commit message
to say, "you could also just do... but since we have it for #1 already..."
> >
> > While this all sounds very powerful it comes with a couple of caveats:
> > 1. Bug reports using raw commands will not get the same level of
> >attention as bug reports using supported commands (via taint).
> > 2. Supported commands will be rejected by the RAW command.
>
> Perhaps I'm missing reading this point 2 (not sure the code actually does it!)
>
> As stated what worries me as it means when we add support for a new
> bit of the spec we just broke the userspace ABI.
>
It does not break ABI. The agreement is userspace must always use the QUERY
command to find out what commands are supported. If it tries to use a RAW
command that is a supported command, it will be rejected. In the case you
mention, that's an application bug. If there is a way to document that better
than what's already in the UAPI kdocs, I'm open to suggestions.
Unlike perhaps other UAPI, this one only promises to give you a way to determine
what commands you can use, not the list of what commands you can use.
> >
> > With this comes new debugfs knob to allow full access to your toes with
> > your weapon of choice.
>
> A few trivial things inline,
>
> Jonathan
>
> >
> > Cc: Ariel Sibley
> > Signed-off-by: Ben Widawsky
> > Reviewed-by: Dan Williams
> > ---
> > drivers/cxl/Kconfig | 18 +
> > drivers/cxl/mem.c| 125 ++-
> > include/uapi/linux/cxl_mem.h | 12 +++-
> > 3 files changed, 152 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
> > index c4ba3aa0a05d..08eaa8e52083 100644
> > --- a/drivers/cxl/Kconfig
> > +++ b/drivers/cxl/Kconfig
> > @@ -33,6 +33,24 @@ config CXL_MEM
> >
> > If unsure say 'm'.
> >
> > +config CXL_MEM_RAW_COMMANDS
> > + bool "RAW Command Interface for Memory Devices"
> > + depends on CXL_MEM
> > + help
> > + Enable CXL RAW command interface.
> > +
> > + The CXL driver ioctl interface may assign a kernel ioctl command
> > + number for each specification defined opcode. At any given point in
> > + time the number of opcodes that the specification defines and a device
> > + may implement may exceed the kernel's set of associated ioctl function
> > + numbers. The mismatch is either by omission, specification is too new,
> > + or by design. When prototyping new hardware, or developing / debugging
> > + the driver it is useful to be able to submit any possible command to
> > + the hardware, even commands that may crash the kernel due to their
> > + potential impact to memory currently in use by the kernel.
> > +
> > + If developing CXL hardware or the driver say Y, otherwise say N.
> > +
> > config CXL_MEM_INSECURE_DEBUG
> > bool "CXL.mem debugging"
> > depends on CXL_MEM
> > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> > index ce65630bb75e..6d766a994dce 100644
> > --- a/drivers/cxl/mem.c
> > +++ b/drivers/cxl/mem.c
> > @@ -1,6 +1,8 @@
> > // SPDX-License-Identifier: GPL-2.0-only
> > /* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> > #include
> > +#include
> > +#include
> > #include
> > #include
> > #include
> > @@ -41,7 +43,14 @@
> >
> > enum opcode {
> > CXL_MBOX_OP_INVALID = 0x,
> > + CXL_MBOX_OP_RAW = CXL_MBOX_OP_INVALID,
> > + CXL_MBOX_OP_ACTIVATE_FW = 0x0202,
> > CXL_MBOX_OP_IDENTIFY= 0x4000,
> > + CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101,
> > + CXL_MBOX_OP_SET_LSA = 0x4103,
> > +
Re: [PATCH v2 5/8] cxl/mem: Add a "RAW" send command
On Tue, 9 Feb 2021 16:02:56 -0800
Ben Widawsky wrote:
> The CXL memory device send interface will have a number of supported
> commands. The raw command is not such a command. Raw commands allow
> userspace to send a specified opcode to the underlying hardware and
> bypass all driver checks on the command. This is useful for a couple of
> usecases, mainly:
> 1. Undocumented vendor specific hardware commands
This one I get. There are things we'd love to standardize but often they
need proving in a generation of hardware before the data is available to
justify taking it to a standards body. Stuff like performance stats.
This stuff will all sit in the vendor defined range. Maybe there is an
argument for in driver hooks to allow proper support even for these
(Ben mentioned this in the other branch of the thread).
> 2. Prototyping new hardware commands not yet supported by the driver
For 2, could just have a convenient place to enable this by one line patch.
Some subsystems (SPI comes to mind) do this for their equivalent of raw
commands. The code is all there to enable it but you need to hook it
up if you want to use it. Avoids chance of a distro shipping it.
>
> While this all sounds very powerful it comes with a couple of caveats:
> 1. Bug reports using raw commands will not get the same level of
>attention as bug reports using supported commands (via taint).
> 2. Supported commands will be rejected by the RAW command.
Perhaps I'm missing reading this point 2 (not sure the code actually does it!)
As stated what worries me as it means when we add support for a new
bit of the spec we just broke the userspace ABI.
>
> With this comes new debugfs knob to allow full access to your toes with
> your weapon of choice.
A few trivial things inline,
Jonathan
>
> Cc: Ariel Sibley
> Signed-off-by: Ben Widawsky
> Reviewed-by: Dan Williams
> ---
> drivers/cxl/Kconfig | 18 +
> drivers/cxl/mem.c| 125 ++-
> include/uapi/linux/cxl_mem.h | 12 +++-
> 3 files changed, 152 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
> index c4ba3aa0a05d..08eaa8e52083 100644
> --- a/drivers/cxl/Kconfig
> +++ b/drivers/cxl/Kconfig
> @@ -33,6 +33,24 @@ config CXL_MEM
>
> If unsure say 'm'.
>
> +config CXL_MEM_RAW_COMMANDS
> + bool "RAW Command Interface for Memory Devices"
> + depends on CXL_MEM
> + help
> + Enable CXL RAW command interface.
> +
> + The CXL driver ioctl interface may assign a kernel ioctl command
> + number for each specification defined opcode. At any given point in
> + time the number of opcodes that the specification defines and a device
> + may implement may exceed the kernel's set of associated ioctl function
> + numbers. The mismatch is either by omission, specification is too new,
> + or by design. When prototyping new hardware, or developing / debugging
> + the driver it is useful to be able to submit any possible command to
> + the hardware, even commands that may crash the kernel due to their
> + potential impact to memory currently in use by the kernel.
> +
> + If developing CXL hardware or the driver say Y, otherwise say N.
> +
> config CXL_MEM_INSECURE_DEBUG
> bool "CXL.mem debugging"
> depends on CXL_MEM
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index ce65630bb75e..6d766a994dce 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -1,6 +1,8 @@
> // SPDX-License-Identifier: GPL-2.0-only
> /* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> #include
> +#include
> +#include
> #include
> #include
> #include
> @@ -41,7 +43,14 @@
>
> enum opcode {
> CXL_MBOX_OP_INVALID = 0x,
> + CXL_MBOX_OP_RAW = CXL_MBOX_OP_INVALID,
> + CXL_MBOX_OP_ACTIVATE_FW = 0x0202,
> CXL_MBOX_OP_IDENTIFY= 0x4000,
> + CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101,
> + CXL_MBOX_OP_SET_LSA = 0x4103,
> + CXL_MBOX_OP_SET_SHUTDOWN_STATE = 0x4204,
> + CXL_MBOX_OP_SCAN_MEDIA = 0x4304,
> + CXL_MBOX_OP_GET_SCAN_MEDIA = 0x4305,
> CXL_MBOX_OP_MAX = 0x1
> };
>
> @@ -91,6 +100,8 @@ struct cxl_memdev {
>
> static int cxl_mem_major;
> static DEFINE_IDA(cxl_memdev_ida);
> +static struct dentry *cxl_debugfs;
> +static bool raw_allow_all;
>
> /**
> * struct cxl_mem_command - Driver representation of a memory device command
> @@ -132,6 +143,49 @@ struct cxl_mem_command {
> */
> static struct cxl_mem_command mem_commands[] = {
> CXL_CMD(IDENTIFY, NONE, 0, 0x43),
> +#ifdef CONFIG_CXL_MEM_RAW_COMMANDS
> + CXL_CMD(RAW, NONE, ~0, ~0),
> +#endif
> +};
> +
> +/*
> + * Commands that RAW doesn't permit. The rationale for each:
> + *
> + * CXL_MBOX_OP_ACTIVATE_FW: Firmware activation requires adjustment /
> + * coordination of
RE: [PATCH v2 5/8] cxl/mem: Add a "RAW" send command
> > > > > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig > > > > > index c4ba3aa0a05d..08eaa8e52083 100644 > > > > > --- a/drivers/cxl/Kconfig > > > > > +++ b/drivers/cxl/Kconfig > > > > > @@ -33,6 +33,24 @@ config CXL_MEM > > > > > > > > > > If unsure say 'm'. > > > > > > > > > > +config CXL_MEM_RAW_COMMANDS > > > > > + bool "RAW Command Interface for Memory Devices" > > > > > + depends on CXL_MEM > > > > > + help > > > > > + Enable CXL RAW command interface. > > > > > + > > > > > + The CXL driver ioctl interface may assign a kernel ioctl > > > > > command > > > > > + number for each specification defined opcode. At any given > > > > > point in > > > > > + time the number of opcodes that the specification defines > > > > > and a device > > > > > + may implement may exceed the kernel's set of associated > > > > > ioctl function > > > > > + numbers. The mismatch is either by omission, specification > > > > > is too new, > > > > > + or by design. When prototyping new hardware, or developing / > > > > > debugging > > > > > + the driver it is useful to be able to submit any possible > > > > > command to > > > > > + the hardware, even commands that may crash the kernel due > > > > > to their > > > > > + potential impact to memory currently in use by the kernel. > > > > > + > > > > > + If developing CXL hardware or the driver say Y, otherwise > > > > > say N. > > > > > > > > Blocking RAW commands by default will prevent vendors from developing > > > > user > > > > space tools that utilize vendor specific commands. Vendors of CXL.mem > > > > devices > > > > should take ownership of ensuring any vendor defined commands that > > > > could cause > > > > user data to be exposed or corrupted are disabled at the device level > > > > for > > > > shipping configurations. > > > > > > Thanks for brining this up Ariel. If there is a recommendation on how to > > > codify > > > this, I would certainly like to know because the explanation will be long. > > > > > > --- > > > > > > The background: > > > > > > The enabling/disabling of the Kconfig option is driven by the distribution > > > and/or system integrator. Even if we made the default 'y', nothing stops > > > them > > > from changing that. if you are using this driver in production and insist > > > on > > > using RAW commands, you are free to carry around a small patch to get rid > > > of the > > > WARN (it is a one-liner). > > > > > > To recap why this is in place - the driver owns the sanctity of the > > > device and > > > therefore a [large] part of the whole system. What we can do as driver > > > writers > > > is figure out the set of commands that are "safe" and allow those. Aside > > > from > > > being able to validate them, we're able to mediate them with other > > > parallel > > > operations that might conflict. We gain the ability to squint extra hard > > > at bug > > > reports. We provide a reason to try to use a well defined part of the > > > spec. > > > Realizing that only allowing that small set of commands in a rapidly > > > growing > > > ecosystem is not a welcoming API; we decided on RAW. > > > > > > Vendor commands can be one of two types: > > > 1. Some functionality probably most vendors want. > > > 2. Functionality that is really single vendor specific. > > > > > > Hopefully we can agree that the path for case #1 is to work with the > > > consortium > > > to standardize a command that does what is needed and that can eventually > > > become > > > part of UAPI. The situation is unfortunate, but temporary. If you won't > > > be able > > > to upgrade your kernel, patch out the WARN as above. > > > > > > The second situation is interesting and does need some more thought and > > > discussion. > > > > > > --- > > > > > > I see 3 realistic options for truly vendor specific commands. > > > 1. Tough noogies. Vendors aren't special and they shouldn't do that. > > > 2. modparam to disable the WARN for specific devices (let the sysadmin > > > decide) > > > 3. Try to make them part of UAPI. > > > > > > The right answer to me is #1, but I also realize I live in the real world. > > > > > > #2 provides too much flexibility. Vendors will just do what they please > > > and > > > distros and/or integrators will be seen as hostile if they don't > > > accommodate. > > > > > > I like #3, but I have a feeling not everyone will agree. My proposal for > > > vendor > > > specific commands is, if it's clear it's truly a unique command, allow > > > adding it > > > as part of UAPI (moving it out of RAW). I expect like 5 of these, ever. > > > If we > > > start getting multiple per vendor, we've failed. The infrastructure is > > > already > > > in place to allow doing this pretty easily. I think we'd have to draw up > > > some > > > guidelines (like adding test cases for the command) to allow these to > > > come in. > > >
RE: [PATCH v2 5/8] cxl/mem: Add a "RAW" send command
> > > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig > > > index c4ba3aa0a05d..08eaa8e52083 100644 > > > --- a/drivers/cxl/Kconfig > > > +++ b/drivers/cxl/Kconfig > > > @@ -33,6 +33,24 @@ config CXL_MEM > > > > > > If unsure say 'm'. > > > > > > +config CXL_MEM_RAW_COMMANDS > > > + bool "RAW Command Interface for Memory Devices" > > > + depends on CXL_MEM > > > + help > > > + Enable CXL RAW command interface. > > > + > > > + The CXL driver ioctl interface may assign a kernel ioctl command > > > + number for each specification defined opcode. At any given > > > point in > > > + time the number of opcodes that the specification defines and a > > > device > > > + may implement may exceed the kernel's set of associated ioctl > > > function > > > + numbers. The mismatch is either by omission, specification is > > > too new, > > > + or by design. When prototyping new hardware, or developing / > > > debugging > > > + the driver it is useful to be able to submit any possible > > > command to > > > + the hardware, even commands that may crash the kernel due to > > > their > > > + potential impact to memory currently in use by the kernel. > > > + > > > + If developing CXL hardware or the driver say Y, otherwise say N. > > > > Blocking RAW commands by default will prevent vendors from developing user > > space tools that utilize vendor specific commands. Vendors of CXL.mem > > devices > > should take ownership of ensuring any vendor defined commands that could > > cause > > user data to be exposed or corrupted are disabled at the device level for > > shipping configurations. > > Thanks for brining this up Ariel. If there is a recommendation on how to > codify > this, I would certainly like to know because the explanation will be long. > > --- > > The background: > > The enabling/disabling of the Kconfig option is driven by the distribution > and/or system integrator. Even if we made the default 'y', nothing stops them > from changing that. if you are using this driver in production and insist on > using RAW commands, you are free to carry around a small patch to get rid of > the > WARN (it is a one-liner). > > To recap why this is in place - the driver owns the sanctity of the device and > therefore a [large] part of the whole system. What we can do as driver writers > is figure out the set of commands that are "safe" and allow those. Aside from > being able to validate them, we're able to mediate them with other parallel > operations that might conflict. We gain the ability to squint extra hard at > bug > reports. We provide a reason to try to use a well defined part of the spec. > Realizing that only allowing that small set of commands in a rapidly growing > ecosystem is not a welcoming API; we decided on RAW. > > Vendor commands can be one of two types: > 1. Some functionality probably most vendors want. > 2. Functionality that is really single vendor specific. > > Hopefully we can agree that the path for case #1 is to work with the > consortium > to standardize a command that does what is needed and that can eventually > become > part of UAPI. The situation is unfortunate, but temporary. If you won't be > able > to upgrade your kernel, patch out the WARN as above. > > The second situation is interesting and does need some more thought and > discussion. > > --- > > I see 3 realistic options for truly vendor specific commands. > 1. Tough noogies. Vendors aren't special and they shouldn't do that. > 2. modparam to disable the WARN for specific devices (let the sysadmin decide) > 3. Try to make them part of UAPI. > > The right answer to me is #1, but I also realize I live in the real world. > > #2 provides too much flexibility. Vendors will just do what they please and > distros and/or integrators will be seen as hostile if they don't accommodate. > > I like #3, but I have a feeling not everyone will agree. My proposal for > vendor > specific commands is, if it's clear it's truly a unique command, allow adding > it > as part of UAPI (moving it out of RAW). I expect like 5 of these, ever. If we > start getting multiple per vendor, we've failed. The infrastructure is already > in place to allow doing this pretty easily. I think we'd have to draw up some > guidelines (like adding test cases for the command) to allow these to come in. > Anything with command effects is going to need extra scrutiny. This would necessitate adding specific opcode values in the range C000h-h to UAPI, and those would then be allowed for all CXL.mem devices, correct? If so, I do not think this is the right approach, as opcodes in this range are by definition vendor defined. A given opcode value will have totally different effects depending on the vendor. I think you may be on to something with the command effects. But rather than "extra scrutiny" for opcodes that have command
RE: [PATCH v2 5/8] cxl/mem: Add a "RAW" send command
> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig > index c4ba3aa0a05d..08eaa8e52083 100644 > --- a/drivers/cxl/Kconfig > +++ b/drivers/cxl/Kconfig > @@ -33,6 +33,24 @@ config CXL_MEM > > If unsure say 'm'. > > +config CXL_MEM_RAW_COMMANDS > + bool "RAW Command Interface for Memory Devices" > + depends on CXL_MEM > + help > + Enable CXL RAW command interface. > + > + The CXL driver ioctl interface may assign a kernel ioctl command > + number for each specification defined opcode. At any given point in > + time the number of opcodes that the specification defines and a > device > + may implement may exceed the kernel's set of associated ioctl > function > + numbers. The mismatch is either by omission, specification is too > new, > + or by design. When prototyping new hardware, or developing / > debugging > + the driver it is useful to be able to submit any possible command to > + the hardware, even commands that may crash the kernel due to their > + potential impact to memory currently in use by the kernel. > + > + If developing CXL hardware or the driver say Y, otherwise say N. Blocking RAW commands by default will prevent vendors from developing user space tools that utilize vendor specific commands. Vendors of CXL.mem devices should take ownership of ensuring any vendor defined commands that could cause user data to be exposed or corrupted are disabled at the device level for shipping configurations. ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
Re: [PATCH v2 5/8] cxl/mem: Add a "RAW" send command
On 21-02-10 18:46:04, ariel.sib...@microchip.com wrote: > > > > > > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig > > > > > > index c4ba3aa0a05d..08eaa8e52083 100644 > > > > > > --- a/drivers/cxl/Kconfig > > > > > > +++ b/drivers/cxl/Kconfig > > > > > > @@ -33,6 +33,24 @@ config CXL_MEM > > > > > > > > > > > > If unsure say 'm'. > > > > > > > > > > > > +config CXL_MEM_RAW_COMMANDS > > > > > > + bool "RAW Command Interface for Memory Devices" > > > > > > + depends on CXL_MEM > > > > > > + help > > > > > > + Enable CXL RAW command interface. > > > > > > + > > > > > > + The CXL driver ioctl interface may assign a kernel ioctl > > > > > > command > > > > > > + number for each specification defined opcode. At any > > > > > > given point in > > > > > > + time the number of opcodes that the specification defines > > > > > > and a device > > > > > > + may implement may exceed the kernel's set of associated > > > > > > ioctl function > > > > > > + numbers. The mismatch is either by omission, > > > > > > specification is too new, > > > > > > + or by design. When prototyping new hardware, or > > > > > > developing / > > > > > > debugging > > > > > > + the driver it is useful to be able to submit any possible > > > > > > command to > > > > > > + the hardware, even commands that may crash the kernel due > > > > > > to their > > > > > > + potential impact to memory currently in use by the kernel. > > > > > > + > > > > > > + If developing CXL hardware or the driver say Y, otherwise > > > > > > say N. > > > > > > > > > > Blocking RAW commands by default will prevent vendors from developing > > > > > user > > > > > space tools that utilize vendor specific commands. Vendors of CXL.mem > > > > > devices > > > > > should take ownership of ensuring any vendor defined commands that > > > > > could cause > > > > > user data to be exposed or corrupted are disabled at the device level > > > > > for > > > > > shipping configurations. > > > > > > > > Thanks for brining this up Ariel. If there is a recommendation on how > > > > to codify > > > > this, I would certainly like to know because the explanation will be > > > > long. > > > > > > > > --- > > > > > > > > The background: > > > > > > > > The enabling/disabling of the Kconfig option is driven by the > > > > distribution > > > > and/or system integrator. Even if we made the default 'y', nothing > > > > stops them > > > > from changing that. if you are using this driver in production and > > > > insist on > > > > using RAW commands, you are free to carry around a small patch to get > > > > rid of the > > > > WARN (it is a one-liner). > > > > > > > > To recap why this is in place - the driver owns the sanctity of the > > > > device and > > > > therefore a [large] part of the whole system. What we can do as driver > > > > writers > > > > is figure out the set of commands that are "safe" and allow those. > > > > Aside from > > > > being able to validate them, we're able to mediate them with other > > > > parallel > > > > operations that might conflict. We gain the ability to squint extra > > > > hard at bug > > > > reports. We provide a reason to try to use a well defined part of the > > > > spec. > > > > Realizing that only allowing that small set of commands in a rapidly > > > > growing > > > > ecosystem is not a welcoming API; we decided on RAW. > > > > > > > > Vendor commands can be one of two types: > > > > 1. Some functionality probably most vendors want. > > > > 2. Functionality that is really single vendor specific. > > > > > > > > Hopefully we can agree that the path for case #1 is to work with the > > > > consortium > > > > to standardize a command that does what is needed and that can > > > > eventually become > > > > part of UAPI. The situation is unfortunate, but temporary. If you won't > > > > be able > > > > to upgrade your kernel, patch out the WARN as above. > > > > > > > > The second situation is interesting and does need some more thought and > > > > discussion. > > > > > > > > --- > > > > > > > > I see 3 realistic options for truly vendor specific commands. > > > > 1. Tough noogies. Vendors aren't special and they shouldn't do that. > > > > 2. modparam to disable the WARN for specific devices (let the sysadmin > > > > decide) > > > > 3. Try to make them part of UAPI. > > > > > > > > The right answer to me is #1, but I also realize I live in the real > > > > world. > > > > > > > > #2 provides too much flexibility. Vendors will just do what they please > > > > and > > > > distros and/or integrators will be seen as hostile if they don't > > > > accommodate. > > > > > > > > I like #3, but I have a feeling not everyone will agree. My proposal > > > > for vendor > > > > specific commands is, if it's clear it's truly a unique command, allow > > > > adding it > > > > as part of UAPI (moving it out of RAW). I
Re: [PATCH v2 5/8] cxl/mem: Add a "RAW" send command
On 21-02-10 18:03:35, ariel.sib...@microchip.com wrote: > > > > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig > > > > index c4ba3aa0a05d..08eaa8e52083 100644 > > > > --- a/drivers/cxl/Kconfig > > > > +++ b/drivers/cxl/Kconfig > > > > @@ -33,6 +33,24 @@ config CXL_MEM > > > > > > > > If unsure say 'm'. > > > > > > > > +config CXL_MEM_RAW_COMMANDS > > > > + bool "RAW Command Interface for Memory Devices" > > > > + depends on CXL_MEM > > > > + help > > > > + Enable CXL RAW command interface. > > > > + > > > > + The CXL driver ioctl interface may assign a kernel ioctl > > > > command > > > > + number for each specification defined opcode. At any given > > > > point in > > > > + time the number of opcodes that the specification defines and > > > > a device > > > > + may implement may exceed the kernel's set of associated ioctl > > > > function > > > > + numbers. The mismatch is either by omission, specification is > > > > too new, > > > > + or by design. When prototyping new hardware, or developing / > > > > debugging > > > > + the driver it is useful to be able to submit any possible > > > > command to > > > > + the hardware, even commands that may crash the kernel due to > > > > their > > > > + potential impact to memory currently in use by the kernel. > > > > + > > > > + If developing CXL hardware or the driver say Y, otherwise say > > > > N. > > > > > > Blocking RAW commands by default will prevent vendors from developing user > > > space tools that utilize vendor specific commands. Vendors of CXL.mem > > > devices > > > should take ownership of ensuring any vendor defined commands that could > > > cause > > > user data to be exposed or corrupted are disabled at the device level for > > > shipping configurations. > > > > Thanks for brining this up Ariel. If there is a recommendation on how to > > codify > > this, I would certainly like to know because the explanation will be long. > > > > --- > > > > The background: > > > > The enabling/disabling of the Kconfig option is driven by the distribution > > and/or system integrator. Even if we made the default 'y', nothing stops > > them > > from changing that. if you are using this driver in production and insist on > > using RAW commands, you are free to carry around a small patch to get rid > > of the > > WARN (it is a one-liner). > > > > To recap why this is in place - the driver owns the sanctity of the device > > and > > therefore a [large] part of the whole system. What we can do as driver > > writers > > is figure out the set of commands that are "safe" and allow those. Aside > > from > > being able to validate them, we're able to mediate them with other parallel > > operations that might conflict. We gain the ability to squint extra hard at > > bug > > reports. We provide a reason to try to use a well defined part of the spec. > > Realizing that only allowing that small set of commands in a rapidly growing > > ecosystem is not a welcoming API; we decided on RAW. > > > > Vendor commands can be one of two types: > > 1. Some functionality probably most vendors want. > > 2. Functionality that is really single vendor specific. > > > > Hopefully we can agree that the path for case #1 is to work with the > > consortium > > to standardize a command that does what is needed and that can eventually > > become > > part of UAPI. The situation is unfortunate, but temporary. If you won't be > > able > > to upgrade your kernel, patch out the WARN as above. > > > > The second situation is interesting and does need some more thought and > > discussion. > > > > --- > > > > I see 3 realistic options for truly vendor specific commands. > > 1. Tough noogies. Vendors aren't special and they shouldn't do that. > > 2. modparam to disable the WARN for specific devices (let the sysadmin > > decide) > > 3. Try to make them part of UAPI. > > > > The right answer to me is #1, but I also realize I live in the real world. > > > > #2 provides too much flexibility. Vendors will just do what they please and > > distros and/or integrators will be seen as hostile if they don't > > accommodate. > > > > I like #3, but I have a feeling not everyone will agree. My proposal for > > vendor > > specific commands is, if it's clear it's truly a unique command, allow > > adding it > > as part of UAPI (moving it out of RAW). I expect like 5 of these, ever. If > > we > > start getting multiple per vendor, we've failed. The infrastructure is > > already > > in place to allow doing this pretty easily. I think we'd have to draw up > > some > > guidelines (like adding test cases for the command) to allow these to come > > in. > > Anything with command effects is going to need extra scrutiny. > > This would necessitate adding specific opcode values in the range C000h-h > to UAPI, and those would then be allowed for all CXL.mem devices,
Re: [PATCH v2 5/8] cxl/mem: Add a "RAW" send command
On 21-02-10 15:26:27, ariel.sib...@microchip.com wrote: > > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig > > index c4ba3aa0a05d..08eaa8e52083 100644 > > --- a/drivers/cxl/Kconfig > > +++ b/drivers/cxl/Kconfig > > @@ -33,6 +33,24 @@ config CXL_MEM > > > > If unsure say 'm'. > > > > +config CXL_MEM_RAW_COMMANDS > > + bool "RAW Command Interface for Memory Devices" > > + depends on CXL_MEM > > + help > > + Enable CXL RAW command interface. > > + > > + The CXL driver ioctl interface may assign a kernel ioctl command > > + number for each specification defined opcode. At any given point > > in > > + time the number of opcodes that the specification defines and a > > device > > + may implement may exceed the kernel's set of associated ioctl > > function > > + numbers. The mismatch is either by omission, specification is too > > new, > > + or by design. When prototyping new hardware, or developing / > > debugging > > + the driver it is useful to be able to submit any possible command > > to > > + the hardware, even commands that may crash the kernel due to their > > + potential impact to memory currently in use by the kernel. > > + > > + If developing CXL hardware or the driver say Y, otherwise say N. > > Blocking RAW commands by default will prevent vendors from developing user > space tools that utilize vendor specific commands. Vendors of CXL.mem devices > should take ownership of ensuring any vendor defined commands that could cause > user data to be exposed or corrupted are disabled at the device level for > shipping configurations. Thanks for brining this up Ariel. If there is a recommendation on how to codify this, I would certainly like to know because the explanation will be long. --- The background: The enabling/disabling of the Kconfig option is driven by the distribution and/or system integrator. Even if we made the default 'y', nothing stops them from changing that. if you are using this driver in production and insist on using RAW commands, you are free to carry around a small patch to get rid of the WARN (it is a one-liner). To recap why this is in place - the driver owns the sanctity of the device and therefore a [large] part of the whole system. What we can do as driver writers is figure out the set of commands that are "safe" and allow those. Aside from being able to validate them, we're able to mediate them with other parallel operations that might conflict. We gain the ability to squint extra hard at bug reports. We provide a reason to try to use a well defined part of the spec. Realizing that only allowing that small set of commands in a rapidly growing ecosystem is not a welcoming API; we decided on RAW. Vendor commands can be one of two types: 1. Some functionality probably most vendors want. 2. Functionality that is really single vendor specific. Hopefully we can agree that the path for case #1 is to work with the consortium to standardize a command that does what is needed and that can eventually become part of UAPI. The situation is unfortunate, but temporary. If you won't be able to upgrade your kernel, patch out the WARN as above. The second situation is interesting and does need some more thought and discussion. --- I see 3 realistic options for truly vendor specific commands. 1. Tough noogies. Vendors aren't special and they shouldn't do that. 2. modparam to disable the WARN for specific devices (let the sysadmin decide) 3. Try to make them part of UAPI. The right answer to me is #1, but I also realize I live in the real world. #2 provides too much flexibility. Vendors will just do what they please and distros and/or integrators will be seen as hostile if they don't accommodate. I like #3, but I have a feeling not everyone will agree. My proposal for vendor specific commands is, if it's clear it's truly a unique command, allow adding it as part of UAPI (moving it out of RAW). I expect like 5 of these, ever. If we start getting multiple per vendor, we've failed. The infrastructure is already in place to allow doing this pretty easily. I think we'd have to draw up some guidelines (like adding test cases for the command) to allow these to come in. Anything with command effects is going to need extra scrutiny. In my opinion, as maintainers of the driver, we do owe the community an answer as to our direction for this. Dan, what is your thought? ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
