Re: [PATCH] firmware: google: memconsole: Adapt to new coreboot ring buffer format

[Greg Kroah-Hartman]

On Fri, Apr 28, 2017 at 01:44:45PM -0700, Julius Werner wrote:
> > -8<
> 
> That needs to go below the --- line, not above it.
> 
> 
> Really? git am --scissors seems to do the right thing for me this way...

Ah, never used that option before, it's not part of my normal workflow.

> Anyway, it was only an FYI, I'll just leave it out.
>  
> 
> When you say things like "in addition" and "finally", that implies that
> the code needs to be broken up into multiple patches.  Rememeber, each
> patch only should ever do 1 thing.
> 
> 
> Well... all these things are required to make the new format work. But I can
> split them up further if you like. 

Please do, makes it easier to review, right?

thanks,

greg k-h

Re: [PATCH] firmware: google: memconsole: Adapt to new coreboot ring buffer format

[Greg Kroah-Hartman]

On Fri, Apr 28, 2017 at 01:44:45PM -0700, Julius Werner wrote:
> > -8<
> 
> That needs to go below the --- line, not above it.
> 
> 
> Really? git am --scissors seems to do the right thing for me this way...

Ah, never used that option before, it's not part of my normal workflow.

> Anyway, it was only an FYI, I'll just leave it out.
>  
> 
> When you say things like "in addition" and "finally", that implies that
> the code needs to be broken up into multiple patches.  Rememeber, each
> patch only should ever do 1 thing.
> 
> 
> Well... all these things are required to make the new format work. But I can
> split them up further if you like. 

Please do, makes it easier to review, right?

thanks,

greg k-h

Re: [PATCH] firmware: google: memconsole: Adapt to new coreboot ring buffer format

[Greg Kroah-Hartman]

On Thu, Apr 27, 2017 at 01:59:17PM -0700, Julius Werner wrote:
> This patch needs to be applied on top of the other memconsole patches
> currently queued up in char-misc-next (ending in 049a59db34e).
> 
> -8<

That needs to go below the --- line, not above it.

> 
> The upstream coreboot implementation of memconsole was enhanced from a
> single-boot console to a persistent ring buffer
> (https://review.coreboot.org/#/c/18301). This patch changes the kernel
> memconsole driver to be able to read the new format.
> 
> In addition, it is desirable for the driver to always return the current
> state of the console, so that it may be used by runtime firmware to log
> further messages after the kernel is already initialized. The existing
> implementation would already re-read the buffer on every access, but
> only measured the total size of the console once during initialization.
> Thus, the generic console interface was redesigned to allow the
> implementation full control over how to process an access, and the
> coreboot implementation will re-read the console size every time.
> 
> Finally, add a new /sys/firmware/rawlog node and change the existing
> /sys/firmware/log to sanitize non-printable characters from the output.
> With the new persistent console it becomes more likely that bytes might
> get slightly corrupted (due to DRAM degradation during a reboot), and
> it's usually undesirable to get stray control characters in the console
> dump because a bit in a letter flipped.

When you say things like "in addition" and "finally", that implies that
the code needs to be broken up into multiple patches.  Rememeber, each
patch only should ever do 1 thing.

Please do that here and make a patch series.

thanks,

greg k-h

Re: [PATCH] firmware: google: memconsole: Adapt to new coreboot ring buffer format

[Greg Kroah-Hartman]

On Thu, Apr 27, 2017 at 01:59:17PM -0700, Julius Werner wrote:
> This patch needs to be applied on top of the other memconsole patches
> currently queued up in char-misc-next (ending in 049a59db34e).
> 
> -8<

That needs to go below the --- line, not above it.

> 
> The upstream coreboot implementation of memconsole was enhanced from a
> single-boot console to a persistent ring buffer
> (https://review.coreboot.org/#/c/18301). This patch changes the kernel
> memconsole driver to be able to read the new format.
> 
> In addition, it is desirable for the driver to always return the current
> state of the console, so that it may be used by runtime firmware to log
> further messages after the kernel is already initialized. The existing
> implementation would already re-read the buffer on every access, but
> only measured the total size of the console once during initialization.
> Thus, the generic console interface was redesigned to allow the
> implementation full control over how to process an access, and the
> coreboot implementation will re-read the console size every time.
> 
> Finally, add a new /sys/firmware/rawlog node and change the existing
> /sys/firmware/log to sanitize non-printable characters from the output.
> With the new persistent console it becomes more likely that bytes might
> get slightly corrupted (due to DRAM degradation during a reboot), and
> it's usually undesirable to get stray control characters in the console
> dump because a bit in a letter flipped.

When you say things like "in addition" and "finally", that implies that
the code needs to be broken up into multiple patches.  Rememeber, each
patch only should ever do 1 thing.

Please do that here and make a patch series.

thanks,

greg k-h

[PATCH] firmware: google: memconsole: Adapt to new coreboot ring buffer format

[Julius Werner]

This patch needs to be applied on top of the other memconsole patches
currently queued up in char-misc-next (ending in 049a59db34e).

-8<

The upstream coreboot implementation of memconsole was enhanced from a
single-boot console to a persistent ring buffer
(https://review.coreboot.org/#/c/18301). This patch changes the kernel
memconsole driver to be able to read the new format.

In addition, it is desirable for the driver to always return the current
state of the console, so that it may be used by runtime firmware to log
further messages after the kernel is already initialized. The existing
implementation would already re-read the buffer on every access, but
only measured the total size of the console once during initialization.
Thus, the generic console interface was redesigned to allow the
implementation full control over how to process an access, and the
coreboot implementation will re-read the console size every time.

Finally, add a new /sys/firmware/rawlog node and change the existing
/sys/firmware/log to sanitize non-printable characters from the output.
With the new persistent console it becomes more likely that bytes might
get slightly corrupted (due to DRAM degradation during a reboot), and
it's usually undesirable to get stray control characters in the console
dump because a bit in a letter flipped.

Signed-off-by: Julius Werner 
---
 drivers/firmware/google/memconsole-coreboot.c   | 51 +
 drivers/firmware/google/memconsole-x86-legacy.c | 18 +++--
 drivers/firmware/google/memconsole.c| 46 +++---
 drivers/firmware/google/memconsole.h|  7 ++--
 4 files changed, 96 insertions(+), 26 deletions(-)

diff --git a/drivers/firmware/google/memconsole-coreboot.c 
b/drivers/firmware/google/memconsole-coreboot.c
index 21210144def7..6e9b07697f70 100644
--- a/drivers/firmware/google/memconsole-coreboot.c
+++ b/drivers/firmware/google/memconsole-coreboot.c
@@ -26,13 +26,52 @@
 
 /* CBMEM firmware console log descriptor. */
 struct cbmem_cons {
-   u32 buffer_size;
-   u32 buffer_cursor;
-   u8  buffer_body[0];
+   u32 size;
+   u32 cursor;
+   u8  body[0];
 } __packed;
 
+#define CURSOR_MASK ((1 << 28) - 1)
+#define OVERFLOW (1 << 31)
+
 static struct cbmem_cons __iomem *cbmem_console;
 
+/*
+ * The cbmem_console structure is read again on every access because it may
+ * change at any time if runtime firmware logs new messages. This may rarely
+ * lead to race conditions where the firmware overwrites the beginning of the
+ * ring buffer with more lines after we have already read |cursor|. It should 
be
+ * rare and harmless enough that we don't spend extra effort working around it.
+ */
+static ssize_t memconsole_coreboot_read(char *buf, loff_t pos, size_t count)
+{
+   u32 cursor = cbmem_console->cursor & CURSOR_MASK;
+   u32 flags = cbmem_console->cursor & ~CURSOR_MASK;
+   u32 size = cbmem_console->size;
+   struct seg {/* describes ring buffer segments in logical order */
+   u32 phys;   /* physical offset from start of mem buffer */
+   u32 len;/* length of segment */
+   } seg[2] = { {0}, {0} };
+   size_t done = 0;
+   int i;
+
+   if (flags & OVERFLOW) {
+   if (cursor > size)  /* Shouldn't really happen, but... */
+   cursor = 0;
+   seg[0] = (struct seg){.phys = cursor, .len = size - cursor};
+   seg[1] = (struct seg){.phys = 0, .len = cursor};
+   } else {
+   seg[0] = (struct seg){.phys = 0, .len = min(cursor, size)};
+   }
+
+   for (i = 0; i < ARRAY_SIZE(seg) && count > done; i++) {
+   done += memory_read_from_buffer(buf + done, count - done, ,
+   cbmem_console->body + seg[i].phys, seg[i].len);
+   pos -= seg[i].len;
+   }
+   return done;
+}
+
 static int memconsole_coreboot_init(phys_addr_t physaddr)
 {
struct cbmem_cons __iomem *tmp_cbmc;
@@ -43,16 +82,14 @@ static int memconsole_coreboot_init(phys_addr_t physaddr)
return -ENOMEM;
 
cbmem_console = memremap(physaddr,
-tmp_cbmc->buffer_size + sizeof(*cbmem_console),
+tmp_cbmc->size + sizeof(*cbmem_console),
 MEMREMAP_WB);
memunmap(tmp_cbmc);
 
if (!cbmem_console)
return -ENOMEM;
 
-   memconsole_setup(cbmem_console->buffer_body,
-   min(cbmem_console->buffer_cursor, cbmem_console->buffer_size));
-
+   memconsole_setup(memconsole_coreboot_read);
return 0;
 }
 
diff --git a/drivers/firmware/google/memconsole-x86-legacy.c 
b/drivers/firmware/google/memconsole-x86-legacy.c
index 529078c62488..1b556e944599 100644
--- a/drivers/firmware/google/memconsole-x86-legacy.c
+++ 

[PATCH] firmware: google: memconsole: Adapt to new coreboot ring buffer format

[Julius Werner]

This patch needs to be applied on top of the other memconsole patches
currently queued up in char-misc-next (ending in 049a59db34e).

-8<

The upstream coreboot implementation of memconsole was enhanced from a
single-boot console to a persistent ring buffer
(https://review.coreboot.org/#/c/18301). This patch changes the kernel
memconsole driver to be able to read the new format.

In addition, it is desirable for the driver to always return the current
state of the console, so that it may be used by runtime firmware to log
further messages after the kernel is already initialized. The existing
implementation would already re-read the buffer on every access, but
only measured the total size of the console once during initialization.
Thus, the generic console interface was redesigned to allow the
implementation full control over how to process an access, and the
coreboot implementation will re-read the console size every time.

Finally, add a new /sys/firmware/rawlog node and change the existing
/sys/firmware/log to sanitize non-printable characters from the output.
With the new persistent console it becomes more likely that bytes might
get slightly corrupted (due to DRAM degradation during a reboot), and
it's usually undesirable to get stray control characters in the console
dump because a bit in a letter flipped.

Signed-off-by: Julius Werner 
---
 drivers/firmware/google/memconsole-coreboot.c   | 51 +
 drivers/firmware/google/memconsole-x86-legacy.c | 18 +++--
 drivers/firmware/google/memconsole.c| 46 +++---
 drivers/firmware/google/memconsole.h|  7 ++--
 4 files changed, 96 insertions(+), 26 deletions(-)

diff --git a/drivers/firmware/google/memconsole-coreboot.c 
b/drivers/firmware/google/memconsole-coreboot.c
index 21210144def7..6e9b07697f70 100644
--- a/drivers/firmware/google/memconsole-coreboot.c
+++ b/drivers/firmware/google/memconsole-coreboot.c
@@ -26,13 +26,52 @@
 
 /* CBMEM firmware console log descriptor. */
 struct cbmem_cons {
-   u32 buffer_size;
-   u32 buffer_cursor;
-   u8  buffer_body[0];
+   u32 size;
+   u32 cursor;
+   u8  body[0];
 } __packed;
 
+#define CURSOR_MASK ((1 << 28) - 1)
+#define OVERFLOW (1 << 31)
+
 static struct cbmem_cons __iomem *cbmem_console;
 
+/*
+ * The cbmem_console structure is read again on every access because it may
+ * change at any time if runtime firmware logs new messages. This may rarely
+ * lead to race conditions where the firmware overwrites the beginning of the
+ * ring buffer with more lines after we have already read |cursor|. It should 
be
+ * rare and harmless enough that we don't spend extra effort working around it.
+ */
+static ssize_t memconsole_coreboot_read(char *buf, loff_t pos, size_t count)
+{
+   u32 cursor = cbmem_console->cursor & CURSOR_MASK;
+   u32 flags = cbmem_console->cursor & ~CURSOR_MASK;
+   u32 size = cbmem_console->size;
+   struct seg {/* describes ring buffer segments in logical order */
+   u32 phys;   /* physical offset from start of mem buffer */
+   u32 len;/* length of segment */
+   } seg[2] = { {0}, {0} };
+   size_t done = 0;
+   int i;
+
+   if (flags & OVERFLOW) {
+   if (cursor > size)  /* Shouldn't really happen, but... */
+   cursor = 0;
+   seg[0] = (struct seg){.phys = cursor, .len = size - cursor};
+   seg[1] = (struct seg){.phys = 0, .len = cursor};
+   } else {
+   seg[0] = (struct seg){.phys = 0, .len = min(cursor, size)};
+   }
+
+   for (i = 0; i < ARRAY_SIZE(seg) && count > done; i++) {
+   done += memory_read_from_buffer(buf + done, count - done, ,
+   cbmem_console->body + seg[i].phys, seg[i].len);
+   pos -= seg[i].len;
+   }
+   return done;
+}
+
 static int memconsole_coreboot_init(phys_addr_t physaddr)
 {
struct cbmem_cons __iomem *tmp_cbmc;
@@ -43,16 +82,14 @@ static int memconsole_coreboot_init(phys_addr_t physaddr)
return -ENOMEM;
 
cbmem_console = memremap(physaddr,
-tmp_cbmc->buffer_size + sizeof(*cbmem_console),
+tmp_cbmc->size + sizeof(*cbmem_console),
 MEMREMAP_WB);
memunmap(tmp_cbmc);
 
if (!cbmem_console)
return -ENOMEM;
 
-   memconsole_setup(cbmem_console->buffer_body,
-   min(cbmem_console->buffer_cursor, cbmem_console->buffer_size));
-
+   memconsole_setup(memconsole_coreboot_read);
return 0;
 }
 
diff --git a/drivers/firmware/google/memconsole-x86-legacy.c 
b/drivers/firmware/google/memconsole-x86-legacy.c
index 529078c62488..1b556e944599 100644
--- a/drivers/firmware/google/memconsole-x86-legacy.c
+++