Re: [PATCH v2 2/3] firmware: google: memconsole: Escape unprintable characters

[Julius Werner]

> Binary sysfs files are supposed to be "pass through" only, the kernel
> should not be touching the data at all, it's up to userspace to do what
> it wants to do with things.  So don't escape anything at all, that's not
> the kernel's job here.

Okay, I'll drop this patch.

Re: [PATCH v2 2/3] firmware: google: memconsole: Escape unprintable characters

[Julius Werner]

> Binary sysfs files are supposed to be "pass through" only, the kernel
> should not be touching the data at all, it's up to userspace to do what
> it wants to do with things.  So don't escape anything at all, that's not
> the kernel's job here.

Okay, I'll drop this patch.

Re: [PATCH v2 2/3] firmware: google: memconsole: Escape unprintable characters

[Greg Kroah-Hartman]

On Mon, May 01, 2017 at 04:44:15PM -0700, Julius Werner wrote:
> On Fri, Apr 28, 2017 at 10:37 PM, Greg Kroah-Hartman
>  wrote:
> > On Fri, Apr 28, 2017 at 01:42:24PM -0700, Julius Werner wrote:
> >> Recent improvements in coreboot's memory console allow it to contain
> >> logs from more than one boot as long as the information persists in
> >> memory. Since trying to persist a memory buffer across reboots often
> >> doesn't quite work perfectly it is now more likely for random bit flips
> >> to occur in the console. With the current implementation this can lead
> >> to stray control characters that cause weird effects for the most common
> >> use cases (such as just dumping the console with 'cat').
> >>
> >> This patch changes the memconsole driver to replace unprintable
> >> characters with '?' by default. It also adds a new /sys/firmware/rawlog
> >> node next to the existing /sys/firmware/log for use cases where it's
> >> desired to read the raw characters.
> >
> > Again, you are doing multiple things here.  Break it up.
> 
> Sorry, I'm not sure what else you want me to break up here? If I add
> the escaping in one patch and then add the raw node in a different
> patch, there's a gap between the two patches where we're losing
> functionality. Isn't that undesirable?

Raw node first and then escape the old one?

> > And, can userspace handle this change?  You are now changing the format
> > of the existing file.
> 
> I'm escaping characters that previously hadn't been escaped, but
> really, I'm just trying to make sure that people can continue to use
> this node as before. The real change has already happened in coreboot,
> outside of Linux. This node could previously be relied upon to only
> contain printable characters, and with newer versions of coreboot
> that's not always the case anymore. I chose to do it this way because
> I thought it would be the least disruptive for most users.
> 
> It can be handled in userspace, yes. It depends on what you use to
> read the file, mostly... people using less will be fine, but people
> using cat may get weird behavior. You're right that it doesn't really
> *need* to be escaped in the kernel, and honestly I don't care too
> much... I can leave it out if you want. But I think this is the most
> convenient way to do it for existing users.

Binary sysfs files are supposed to be "pass through" only, the kernel
should not be touching the data at all, it's up to userspace to do what
it wants to do with things.  So don't escape anything at all, that's not
the kernel's job here.

thanks,

greg k-h

Re: [PATCH v2 2/3] firmware: google: memconsole: Escape unprintable characters

[Greg Kroah-Hartman]

On Mon, May 01, 2017 at 04:44:15PM -0700, Julius Werner wrote:
> On Fri, Apr 28, 2017 at 10:37 PM, Greg Kroah-Hartman
>  wrote:
> > On Fri, Apr 28, 2017 at 01:42:24PM -0700, Julius Werner wrote:
> >> Recent improvements in coreboot's memory console allow it to contain
> >> logs from more than one boot as long as the information persists in
> >> memory. Since trying to persist a memory buffer across reboots often
> >> doesn't quite work perfectly it is now more likely for random bit flips
> >> to occur in the console. With the current implementation this can lead
> >> to stray control characters that cause weird effects for the most common
> >> use cases (such as just dumping the console with 'cat').
> >>
> >> This patch changes the memconsole driver to replace unprintable
> >> characters with '?' by default. It also adds a new /sys/firmware/rawlog
> >> node next to the existing /sys/firmware/log for use cases where it's
> >> desired to read the raw characters.
> >
> > Again, you are doing multiple things here.  Break it up.
> 
> Sorry, I'm not sure what else you want me to break up here? If I add
> the escaping in one patch and then add the raw node in a different
> patch, there's a gap between the two patches where we're losing
> functionality. Isn't that undesirable?

Raw node first and then escape the old one?

> > And, can userspace handle this change?  You are now changing the format
> > of the existing file.
> 
> I'm escaping characters that previously hadn't been escaped, but
> really, I'm just trying to make sure that people can continue to use
> this node as before. The real change has already happened in coreboot,
> outside of Linux. This node could previously be relied upon to only
> contain printable characters, and with newer versions of coreboot
> that's not always the case anymore. I chose to do it this way because
> I thought it would be the least disruptive for most users.
> 
> It can be handled in userspace, yes. It depends on what you use to
> read the file, mostly... people using less will be fine, but people
> using cat may get weird behavior. You're right that it doesn't really
> *need* to be escaped in the kernel, and honestly I don't care too
> much... I can leave it out if you want. But I think this is the most
> convenient way to do it for existing users.

Binary sysfs files are supposed to be "pass through" only, the kernel
should not be touching the data at all, it's up to userspace to do what
it wants to do with things.  So don't escape anything at all, that's not
the kernel's job here.

thanks,

greg k-h

Re: [PATCH v2 2/3] firmware: google: memconsole: Escape unprintable characters

[Julius Werner]

On Fri, Apr 28, 2017 at 10:37 PM, Greg Kroah-Hartman
 wrote:
> On Fri, Apr 28, 2017 at 01:42:24PM -0700, Julius Werner wrote:
>> Recent improvements in coreboot's memory console allow it to contain
>> logs from more than one boot as long as the information persists in
>> memory. Since trying to persist a memory buffer across reboots often
>> doesn't quite work perfectly it is now more likely for random bit flips
>> to occur in the console. With the current implementation this can lead
>> to stray control characters that cause weird effects for the most common
>> use cases (such as just dumping the console with 'cat').
>>
>> This patch changes the memconsole driver to replace unprintable
>> characters with '?' by default. It also adds a new /sys/firmware/rawlog
>> node next to the existing /sys/firmware/log for use cases where it's
>> desired to read the raw characters.
>
> Again, you are doing multiple things here.  Break it up.

Sorry, I'm not sure what else you want me to break up here? If I add
the escaping in one patch and then add the raw node in a different
patch, there's a gap between the two patches where we're losing
functionality. Isn't that undesirable?

> And, can userspace handle this change?  You are now changing the format
> of the existing file.

I'm escaping characters that previously hadn't been escaped, but
really, I'm just trying to make sure that people can continue to use
this node as before. The real change has already happened in coreboot,
outside of Linux. This node could previously be relied upon to only
contain printable characters, and with newer versions of coreboot
that's not always the case anymore. I chose to do it this way because
I thought it would be the least disruptive for most users.

It can be handled in userspace, yes. It depends on what you use to
read the file, mostly... people using less will be fine, but people
using cat may get weird behavior. You're right that it doesn't really
*need* to be escaped in the kernel, and honestly I don't care too
much... I can leave it out if you want. But I think this is the most
convenient way to do it for existing users.

>> - return sysfs_create_bin_file(firmware_kobj, _bin_attr);
>> + return sysfs_create_bin_file(firmware_kobj, _log_attr) ||
>> + sysfs_create_bin_file(firmware_kobj, _raw_attr);
>
> While it is really rare, please do this properly and create one file,
> and then the other, and provide proper error handling here (i.e. if the
> second fails, remove the first).

Okay, will send that with the next set once we agree on the other stuff.

Re: [PATCH v2 2/3] firmware: google: memconsole: Escape unprintable characters

[Julius Werner]

On Fri, Apr 28, 2017 at 10:37 PM, Greg Kroah-Hartman
 wrote:
> On Fri, Apr 28, 2017 at 01:42:24PM -0700, Julius Werner wrote:
>> Recent improvements in coreboot's memory console allow it to contain
>> logs from more than one boot as long as the information persists in
>> memory. Since trying to persist a memory buffer across reboots often
>> doesn't quite work perfectly it is now more likely for random bit flips
>> to occur in the console. With the current implementation this can lead
>> to stray control characters that cause weird effects for the most common
>> use cases (such as just dumping the console with 'cat').
>>
>> This patch changes the memconsole driver to replace unprintable
>> characters with '?' by default. It also adds a new /sys/firmware/rawlog
>> node next to the existing /sys/firmware/log for use cases where it's
>> desired to read the raw characters.
>
> Again, you are doing multiple things here.  Break it up.

Sorry, I'm not sure what else you want me to break up here? If I add
the escaping in one patch and then add the raw node in a different
patch, there's a gap between the two patches where we're losing
functionality. Isn't that undesirable?

> And, can userspace handle this change?  You are now changing the format
> of the existing file.

I'm escaping characters that previously hadn't been escaped, but
really, I'm just trying to make sure that people can continue to use
this node as before. The real change has already happened in coreboot,
outside of Linux. This node could previously be relied upon to only
contain printable characters, and with newer versions of coreboot
that's not always the case anymore. I chose to do it this way because
I thought it would be the least disruptive for most users.

It can be handled in userspace, yes. It depends on what you use to
read the file, mostly... people using less will be fine, but people
using cat may get weird behavior. You're right that it doesn't really
*need* to be escaped in the kernel, and honestly I don't care too
much... I can leave it out if you want. But I think this is the most
convenient way to do it for existing users.

>> - return sysfs_create_bin_file(firmware_kobj, _bin_attr);
>> + return sysfs_create_bin_file(firmware_kobj, _log_attr) ||
>> + sysfs_create_bin_file(firmware_kobj, _raw_attr);
>
> While it is really rare, please do this properly and create one file,
> and then the other, and provide proper error handling here (i.e. if the
> second fails, remove the first).

Okay, will send that with the next set once we agree on the other stuff.

Re: [PATCH v2 2/3] firmware: google: memconsole: Escape unprintable characters

[Greg Kroah-Hartman]

On Fri, Apr 28, 2017 at 01:42:24PM -0700, Julius Werner wrote:
> Recent improvements in coreboot's memory console allow it to contain
> logs from more than one boot as long as the information persists in
> memory. Since trying to persist a memory buffer across reboots often
> doesn't quite work perfectly it is now more likely for random bit flips
> to occur in the console. With the current implementation this can lead
> to stray control characters that cause weird effects for the most common
> use cases (such as just dumping the console with 'cat').
> 
> This patch changes the memconsole driver to replace unprintable
> characters with '?' by default. It also adds a new /sys/firmware/rawlog
> node next to the existing /sys/firmware/log for use cases where it's
> desired to read the raw characters.

Again, you are doing multiple things here.  Break it up.

And, can userspace handle this change?  You are now changing the format
of the existing file.

> Signed-off-by: Julius Werner 
> ---
>  drivers/firmware/google/memconsole.c | 32 +++-
>  1 file changed, 27 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/firmware/google/memconsole.c 
> b/drivers/firmware/google/memconsole.c
> index 166f07c68c02..807fcd4f7f85 100644
> --- a/drivers/firmware/google/memconsole.c
> +++ b/drivers/firmware/google/memconsole.c
> @@ -15,6 +15,7 @@
>   * GNU General Public License for more details.
>   */
>  
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -24,7 +25,7 @@
>  
>  static ssize_t (*memconsole_read_func)(char *, loff_t, size_t);
>  
> -static ssize_t memconsole_read(struct file *filp, struct kobject *kobp,
> +static ssize_t memconsole_read_raw(struct file *filp, struct kobject *kobp,
>  struct bin_attribute *bin_attr, char *buf,
>  loff_t pos, size_t count)
>  {
> @@ -33,9 +34,28 @@ static ssize_t memconsole_read(struct file *filp, struct 
> kobject *kobp,
>   return memconsole_read_func(buf, pos, count);
>  }
>  
> -static struct bin_attribute memconsole_bin_attr = {
> +static ssize_t memconsole_read_log(struct file *filp, struct kobject *kobp,
> +struct bin_attribute *bin_attr, char *buf,
> +loff_t pos, size_t count)
> +{
> + ssize_t i;
> + ssize_t rv = memconsole_read_raw(filp, kobp, bin_attr, buf, pos, count);
> +
> + if (rv > 0)
> + for (i = 0; i < rv; i++)
> + if (!isprint(buf[i]) && !isspace(buf[i]))
> + buf[i] = '?';
> + return rv;
> +}
> +
> +/* Memconsoles may be much longer than 4K, so need to use binary attributes. 
> */
> +static struct bin_attribute memconsole_log_attr = {
>   .attr = {.name = "log", .mode = 0444},
> - .read = memconsole_read,
> + .read = memconsole_read_log,
> +};
> +static struct bin_attribute memconsole_raw_attr = {
> + .attr = {.name = "rawlog", .mode = 0444},
> + .read = memconsole_read_raw,
>  };
>  
>  void memconsole_setup(ssize_t (*read_func)(char *, loff_t, size_t))
> @@ -46,13 +66,15 @@ EXPORT_SYMBOL(memconsole_setup);
>  
>  int memconsole_sysfs_init(void)
>  {
> - return sysfs_create_bin_file(firmware_kobj, _bin_attr);
> + return sysfs_create_bin_file(firmware_kobj, _log_attr) ||
> + sysfs_create_bin_file(firmware_kobj, _raw_attr);

While it is really rare, please do this properly and create one file,
and then the other, and provide proper error handling here (i.e. if the
second fails, remove the first).

thanks,

greg k-h

Re: [PATCH v2 2/3] firmware: google: memconsole: Escape unprintable characters

[Greg Kroah-Hartman]

On Fri, Apr 28, 2017 at 01:42:24PM -0700, Julius Werner wrote:
> Recent improvements in coreboot's memory console allow it to contain
> logs from more than one boot as long as the information persists in
> memory. Since trying to persist a memory buffer across reboots often
> doesn't quite work perfectly it is now more likely for random bit flips
> to occur in the console. With the current implementation this can lead
> to stray control characters that cause weird effects for the most common
> use cases (such as just dumping the console with 'cat').
> 
> This patch changes the memconsole driver to replace unprintable
> characters with '?' by default. It also adds a new /sys/firmware/rawlog
> node next to the existing /sys/firmware/log for use cases where it's
> desired to read the raw characters.

Again, you are doing multiple things here.  Break it up.

And, can userspace handle this change?  You are now changing the format
of the existing file.

> Signed-off-by: Julius Werner 
> ---
>  drivers/firmware/google/memconsole.c | 32 +++-
>  1 file changed, 27 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/firmware/google/memconsole.c 
> b/drivers/firmware/google/memconsole.c
> index 166f07c68c02..807fcd4f7f85 100644
> --- a/drivers/firmware/google/memconsole.c
> +++ b/drivers/firmware/google/memconsole.c
> @@ -15,6 +15,7 @@
>   * GNU General Public License for more details.
>   */
>  
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -24,7 +25,7 @@
>  
>  static ssize_t (*memconsole_read_func)(char *, loff_t, size_t);
>  
> -static ssize_t memconsole_read(struct file *filp, struct kobject *kobp,
> +static ssize_t memconsole_read_raw(struct file *filp, struct kobject *kobp,
>  struct bin_attribute *bin_attr, char *buf,
>  loff_t pos, size_t count)
>  {
> @@ -33,9 +34,28 @@ static ssize_t memconsole_read(struct file *filp, struct 
> kobject *kobp,
>   return memconsole_read_func(buf, pos, count);
>  }
>  
> -static struct bin_attribute memconsole_bin_attr = {
> +static ssize_t memconsole_read_log(struct file *filp, struct kobject *kobp,
> +struct bin_attribute *bin_attr, char *buf,
> +loff_t pos, size_t count)
> +{
> + ssize_t i;
> + ssize_t rv = memconsole_read_raw(filp, kobp, bin_attr, buf, pos, count);
> +
> + if (rv > 0)
> + for (i = 0; i < rv; i++)
> + if (!isprint(buf[i]) && !isspace(buf[i]))
> + buf[i] = '?';
> + return rv;
> +}
> +
> +/* Memconsoles may be much longer than 4K, so need to use binary attributes. 
> */
> +static struct bin_attribute memconsole_log_attr = {
>   .attr = {.name = "log", .mode = 0444},
> - .read = memconsole_read,
> + .read = memconsole_read_log,
> +};
> +static struct bin_attribute memconsole_raw_attr = {
> + .attr = {.name = "rawlog", .mode = 0444},
> + .read = memconsole_read_raw,
>  };
>  
>  void memconsole_setup(ssize_t (*read_func)(char *, loff_t, size_t))
> @@ -46,13 +66,15 @@ EXPORT_SYMBOL(memconsole_setup);
>  
>  int memconsole_sysfs_init(void)
>  {
> - return sysfs_create_bin_file(firmware_kobj, _bin_attr);
> + return sysfs_create_bin_file(firmware_kobj, _log_attr) ||
> + sysfs_create_bin_file(firmware_kobj, _raw_attr);

While it is really rare, please do this properly and create one file,
and then the other, and provide proper error handling here (i.e. if the
second fails, remove the first).

thanks,

greg k-h

[PATCH v2 2/3] firmware: google: memconsole: Escape unprintable characters

[Julius Werner]

Recent improvements in coreboot's memory console allow it to contain
logs from more than one boot as long as the information persists in
memory. Since trying to persist a memory buffer across reboots often
doesn't quite work perfectly it is now more likely for random bit flips
to occur in the console. With the current implementation this can lead
to stray control characters that cause weird effects for the most common
use cases (such as just dumping the console with 'cat').

This patch changes the memconsole driver to replace unprintable
characters with '?' by default. It also adds a new /sys/firmware/rawlog
node next to the existing /sys/firmware/log for use cases where it's
desired to read the raw characters.

Signed-off-by: Julius Werner 
---
 drivers/firmware/google/memconsole.c | 32 +++-
 1 file changed, 27 insertions(+), 5 deletions(-)

diff --git a/drivers/firmware/google/memconsole.c 
b/drivers/firmware/google/memconsole.c
index 166f07c68c02..807fcd4f7f85 100644
--- a/drivers/firmware/google/memconsole.c
+++ b/drivers/firmware/google/memconsole.c
@@ -15,6 +15,7 @@
  * GNU General Public License for more details.
  */
 
+#include 
 #include 
 #include 
 #include 
@@ -24,7 +25,7 @@
 
 static ssize_t (*memconsole_read_func)(char *, loff_t, size_t);
 
-static ssize_t memconsole_read(struct file *filp, struct kobject *kobp,
+static ssize_t memconsole_read_raw(struct file *filp, struct kobject *kobp,
   struct bin_attribute *bin_attr, char *buf,
   loff_t pos, size_t count)
 {
@@ -33,9 +34,28 @@ static ssize_t memconsole_read(struct file *filp, struct 
kobject *kobp,
return memconsole_read_func(buf, pos, count);
 }
 
-static struct bin_attribute memconsole_bin_attr = {
+static ssize_t memconsole_read_log(struct file *filp, struct kobject *kobp,
+  struct bin_attribute *bin_attr, char *buf,
+  loff_t pos, size_t count)
+{
+   ssize_t i;
+   ssize_t rv = memconsole_read_raw(filp, kobp, bin_attr, buf, pos, count);
+
+   if (rv > 0)
+   for (i = 0; i < rv; i++)
+   if (!isprint(buf[i]) && !isspace(buf[i]))
+   buf[i] = '?';
+   return rv;
+}
+
+/* Memconsoles may be much longer than 4K, so need to use binary attributes. */
+static struct bin_attribute memconsole_log_attr = {
.attr = {.name = "log", .mode = 0444},
-   .read = memconsole_read,
+   .read = memconsole_read_log,
+};
+static struct bin_attribute memconsole_raw_attr = {
+   .attr = {.name = "rawlog", .mode = 0444},
+   .read = memconsole_read_raw,
 };
 
 void memconsole_setup(ssize_t (*read_func)(char *, loff_t, size_t))
@@ -46,13 +66,15 @@ EXPORT_SYMBOL(memconsole_setup);
 
 int memconsole_sysfs_init(void)
 {
-   return sysfs_create_bin_file(firmware_kobj, _bin_attr);
+   return sysfs_create_bin_file(firmware_kobj, _log_attr) ||
+   sysfs_create_bin_file(firmware_kobj, _raw_attr);
 }
 EXPORT_SYMBOL(memconsole_sysfs_init);
 
 void memconsole_exit(void)
 {
-   sysfs_remove_bin_file(firmware_kobj, _bin_attr);
+   sysfs_remove_bin_file(firmware_kobj, _log_attr);
+   sysfs_remove_bin_file(firmware_kobj, _raw_attr);
 }
 EXPORT_SYMBOL(memconsole_exit);
 
-- 
2.12.2

[PATCH v2 2/3] firmware: google: memconsole: Escape unprintable characters

[Julius Werner]

Recent improvements in coreboot's memory console allow it to contain
logs from more than one boot as long as the information persists in
memory. Since trying to persist a memory buffer across reboots often
doesn't quite work perfectly it is now more likely for random bit flips
to occur in the console. With the current implementation this can lead
to stray control characters that cause weird effects for the most common
use cases (such as just dumping the console with 'cat').

This patch changes the memconsole driver to replace unprintable
characters with '?' by default. It also adds a new /sys/firmware/rawlog
node next to the existing /sys/firmware/log for use cases where it's
desired to read the raw characters.

Signed-off-by: Julius Werner 
---
 drivers/firmware/google/memconsole.c | 32 +++-
 1 file changed, 27 insertions(+), 5 deletions(-)

diff --git a/drivers/firmware/google/memconsole.c 
b/drivers/firmware/google/memconsole.c
index 166f07c68c02..807fcd4f7f85 100644
--- a/drivers/firmware/google/memconsole.c
+++ b/drivers/firmware/google/memconsole.c
@@ -15,6 +15,7 @@
  * GNU General Public License for more details.
  */
 
+#include 
 #include 
 #include 
 #include 
@@ -24,7 +25,7 @@
 
 static ssize_t (*memconsole_read_func)(char *, loff_t, size_t);
 
-static ssize_t memconsole_read(struct file *filp, struct kobject *kobp,
+static ssize_t memconsole_read_raw(struct file *filp, struct kobject *kobp,
   struct bin_attribute *bin_attr, char *buf,
   loff_t pos, size_t count)
 {
@@ -33,9 +34,28 @@ static ssize_t memconsole_read(struct file *filp, struct 
kobject *kobp,
return memconsole_read_func(buf, pos, count);
 }
 
-static struct bin_attribute memconsole_bin_attr = {
+static ssize_t memconsole_read_log(struct file *filp, struct kobject *kobp,
+  struct bin_attribute *bin_attr, char *buf,
+  loff_t pos, size_t count)
+{
+   ssize_t i;
+   ssize_t rv = memconsole_read_raw(filp, kobp, bin_attr, buf, pos, count);
+
+   if (rv > 0)
+   for (i = 0; i < rv; i++)
+   if (!isprint(buf[i]) && !isspace(buf[i]))
+   buf[i] = '?';
+   return rv;
+}
+
+/* Memconsoles may be much longer than 4K, so need to use binary attributes. */
+static struct bin_attribute memconsole_log_attr = {
.attr = {.name = "log", .mode = 0444},
-   .read = memconsole_read,
+   .read = memconsole_read_log,
+};
+static struct bin_attribute memconsole_raw_attr = {
+   .attr = {.name = "rawlog", .mode = 0444},
+   .read = memconsole_read_raw,
 };
 
 void memconsole_setup(ssize_t (*read_func)(char *, loff_t, size_t))
@@ -46,13 +66,15 @@ EXPORT_SYMBOL(memconsole_setup);
 
 int memconsole_sysfs_init(void)
 {
-   return sysfs_create_bin_file(firmware_kobj, _bin_attr);
+   return sysfs_create_bin_file(firmware_kobj, _log_attr) ||
+   sysfs_create_bin_file(firmware_kobj, _raw_attr);
 }
 EXPORT_SYMBOL(memconsole_sysfs_init);
 
 void memconsole_exit(void)
 {
-   sysfs_remove_bin_file(firmware_kobj, _bin_attr);
+   sysfs_remove_bin_file(firmware_kobj, _log_attr);
+   sysfs_remove_bin_file(firmware_kobj, _raw_attr);
 }
 EXPORT_SYMBOL(memconsole_exit);
 
-- 
2.12.2