Re: [PATCH v5 01/15] ring-buffer: Refactor ring buffer implementation

2023-12-20 Thread Steven Rostedt
On Wed, 20 Dec 2023 18:48:43 +0900 Masami Hiramatsu (Google) wrote: > > From: "Tzvetomir Stoyanov (VMware)" > > > > In order to introduce sub-buffer size per ring buffer, some internal > > refactoring is needed. As ring_buffer_print_page_header() will depend on > > the trace_buffer structure,

[PATCH v5 16/15] ring-buffer: Use subbuf_order for buffer page masking

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" The comparisons to PAGE_SIZE were all converted to use the buffer->subbuf_order, but the use of PAGE_MASK was missed. Convert all the PAGE_MASK usages over to: (PAGE_SIZE << cpu_buffer->buffer->subbuf_order) - 1 Fixes: TBD ("r

Re: [PATCH v8 0/2] Introducing trace buffer mapping by user-space

2023-12-19 Thread Steven Rostedt
On Tue, 19 Dec 2023 18:45:54 + Vincent Donnefort wrote: > The tracing ring-buffers can be stored on disk or sent to network > without any copy via splice. However the later doesn't allow real time > processing of the traces. A solution is to give userspace direct access > to the ring-buffer

[PATCH v5 15/15] tracing: Update subbuffer with kilobytes not page order

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Using page order for deciding what the size of the ring buffer sub buffers are is exposing a bit too much of the implementation. Although the sub buffers are only allocated in orders of pages, allow the user to specify the minimum size of each sub-

[PATCH v5 14/15] ringbuffer/selftest: Add basic selftest to test changing subbuf order

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Add a self test that will write into the trace buffer with differ trace sub buffer order sizes. Signed-off-by: Steven Rostedt (Google) --- .../ftrace/test.d/00basic/ringbuffer_order.tc | 95 +++ 1 file changed, 95 insertions(+) c

[PATCH v5 13/15] ring-buffer: Add documentation on the buffer_subbuf_order file

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Add to the documentation how to use the buffer_subbuf_order file to change the size and how it affects what events can be added to the ring buffer. Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/ftrace.rst | 27

[PATCH v5 12/15] ring-buffer: Just update the subbuffers when changing their allocation order

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" The ring_buffer_subbuf_order_set() was creating ring_buffer_per_cpu cpu_buffers with the new subbuffers with the updated order, and if they all successfully were created, then they the ring_buffer's per_cpu buffers would be freed and replaced by them. T

[PATCH v5 09/15] tracing: Update snapshot order along with main buffer order

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" When updating the order of the sub buffers for the main buffer, make sure that if the snapshot buffer exists, that it gets its order updated as well. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/tr

[PATCH v5 11/15] ring-buffer: Keep the same size when updating the order

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" The function ring_buffer_subbuf_order_set() just updated the sub-buffers to the new size, but this also changes the size of the buffer in doing so. As the size is determined by nr_pages * subbuf_size. If the subbuf_size is increased without decreasing th

[PATCH v5 10/15] tracing: Stop the tracing while changing the ring buffer subbuf size

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Because the main buffer and the snapshot buffer need to be the same for some tracers, otherwise it will fail and disable all tracing, the tracers need to be stopped while updating the sub buffer sizes so that the tracers see the main and snapsh

[PATCH v5 08/15] ring-buffer: Make sure the spare sub buffer used for reads has same size

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Now that the ring buffer specifies the size of its sub buffers, they all need to be the same size. When doing a read, a swap is done with a spare page. Make sure they are the same size before doing the swap, otherwise the read will fail. Signed-off-

[PATCH v5 07/15] ring-buffer: Do no swap cpu buffers if order is different

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" As all the subbuffer order (subbuffer sizes) must be the same throughout the ring buffer, check the order of the buffers that are doing a CPU buffer swap in ring_buffer_swap_cpu() to make sure they are the same. If the are not the same, then fail to d

[PATCH v5 06/15] ring-buffer: Clear pages on error in ring_buffer_subbuf_order_set() failure

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" On failure to allocate ring buffer pages, the pointer to the CPU buffer pages is freed, but the pages that were allocated previously were not. Make sure they are freed too. Fixes: TBD ("tracing: Set new size of the ring buffer sub page") S

[PATCH v5 04/15] ring-buffer: Set new size of the ring buffer sub page

2023-12-19 Thread Steven Rostedt
race-devel/20211213094825.61876-5-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 80 ++ 1 file changed, 73 insertions(+), 7 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/ke

[PATCH v5 05/15] ring-buffer: Read and write to ring buffers with custom sub buffer size

2023-12-19 Thread Steven Rostedt
_read_page() ] Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 11 ++-- kernel/trace/ring_buffer.c | 75 kernel/trace/ring_buffer_benchmark.c | 10 ++-- kernel/trace/trace.c | 34 +++-- 4 files changed, 89

[PATCH v5 03/15] ring-buffer: Add interface for configuring trace sub buffer size

2023-12-19 Thread Steven Rostedt
mir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 4 ++ kernel/trace/ring_buffer.c | 73 + kernel/trace/trace.c| 48 3 files changed, 125 insertions(+) diff --git a/include/linux/ring_buff

[PATCH v5 02/15] ring-buffer: Page size per ring buffer

2023-12-19 Thread Steven Rostedt
fer. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-3-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 2 +- kernel/trace/ring_buffer.c | 68 +--

[PATCH v5 01/15] ring-buffer: Refactor ring buffer implementation

2023-12-19 Thread Steven Rostedt
inux-trace-devel/20211213094825.61876-2-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 60 +++--- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/ke

[PATCH v5 00/15] ring-buffer/tracing: Allow ring buffer to have bigger sub buffers

2023-12-19 Thread Steven Rostedt
Note, this has been on my todo list since the ring buffer was created back in 2008. Tzvetomir last worked on this in 2021 and I need to finally get it in. His last series was: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-1-tz.stoya...@gmail.com/ With the description of:

Re: [PATCH] ring-buffer: Fix slowpath of interrupted event

2023-12-19 Thread Steven Rostedt
On Tue, 19 Dec 2023 10:36:13 -0500 Steven Rostedt wrote: > |-- interrupt event --|-- normal context event --|-- interrupt event --| > > ^^ ^ > || | > ts is befo

Re: [PATCH] ring-buffer: Fix slowpath of interrupted event

2023-12-19 Thread Steven Rostedt
On Tue, 19 Dec 2023 10:10:27 -0500 Steven Rostedt wrote: > 1000 - interrupt event > 2000 - normal context event > 2100 - next normal context event > > Where we see the delta between the interrupt event and the normal context > event was 1000. But if we just had it be d

Re: [PATCH] ring-buffer: Fix slowpath of interrupted event

2023-12-19 Thread Steven Rostedt
On Tue, 19 Dec 2023 23:37:10 +0900 Masami Hiramatsu (Google) wrote: > Yeah the above works, but my question is, do we really need this > really slow path? I mean; > > > if (w == write - event length) { > > /* Nothing interrupted between A and C */ > > /*E*/

[PATCH] ring-buffer: Check if absolute timestamp goes backwards

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" The check_buffer() which checks the timestamps of the ring buffer sub-buffer page, when enabled, only checks if the adding of deltas of the events from the last absolute timestamp or the timestamp of the sub-buffer page adds up to the current event. Wh

[PATCH] ring-buffer: Add interrupt information to dump of data sub-buffer

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" When the ring buffer timestamp verifier triggers, it dumps the content of the sub-buffer. But currently it only dumps the timestamps and the offset of the data as well as the deltas. It would be even more informative if the event data also showed the

[PATCH v3] ring-buffer: Remove 32bit timestamp logic

2023-12-19 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Each event has a 27 bit timestamp delta that is used to hold the delta from the last event. If the time between events is greater than 2^27, then a timestamp is added that holds a 59 bit absolute timestamp. Until a389d86f7fd09 ("ring-buffer: Hav

[PATCH] ring-buffer: Fix slowpath of interrupted event

2023-12-18 Thread Steven Rostedt
From: "Steven Rostedt (Google)" To synchronize the timestamps with the ring buffer reservation, there are two timestamps that are saved in the buffer meta data. 1. before_stamp 2. write_stamp When the two are equal, the write_stamp is considered valid, as in, it may be used to

[PATCH] ring-buffer: Check if absolute timestamp goes backwards

2023-12-18 Thread Steven Rostedt
From: "Steven Rostedt (Google)" The check_buffer() which checks the timestamps of the ring buffer sub-buffer page, when enabled, only checks if the adding of deltas of the events from the last absolute timestamp or the timestamp of the sub-buffer page adds up to the current event. Wh

Re: [PATCH v2] ring-buffer: Add interrupt information to dump of data sub-buffer

2023-12-18 Thread Steven Rostedt
On Mon, 18 Dec 2023 17:01:06 -0500 Steven Rostedt wrote: > @@ -3347,7 +3418,8 @@ static void check_buffer(struct ring_buffer_per_cpu > *cpu_buffer, > } > } > if ((full && ts > info->ts) || > - (!full && ts + info->del

Re: [PATCH 1/2] ring-buffer: Replace rb_time_cmpxchg() with rb_time_cmp_and_update()

2023-12-18 Thread Steven Rostedt
On Mon, 18 Dec 2023 13:42:40 -0500 Steven Rostedt wrote: > > > > > > > static bool rb_time_cmp_and_update(rb_time_t *t, u64 expect, u64 set) > > > > { > > > > - return rb_time_cmpxchg(t, expect, set); > > > > +#ifdef RB_T

[PATCH v2] ring-buffer: Add interrupt information to dump of data sub-buffer

2023-12-18 Thread Steven Rostedt
From: "Steven Rostedt (Google)" When the ring buffer timestamp verifier triggers, it dumps the content of the sub-buffer. But currently it only dumps the timestamps and the offset of the data as well as the deltas. It would be even more informative if the event data also showed the

[PATCH] ring-buffer: Add interrupt information to dump of data sub-buffer

2023-12-18 Thread Steven Rostedt
From: "Steven Rostedt (Google)" When the ring buffer timestamp verifier triggers, it dumps the content of the sub-buffer. But currently it only dumps the timestamps and the offset of the data as well as the deltas. It would be even more informative if the event data also showed the

[PATCH v2 0/2] tracing: Replace final 64-bit cmpxchg with compare and update if available

2023-12-18 Thread Steven Rostedt
me_cmpxchg() to work the same for both 64-bit and 32-bit. - Fixed reading t->time to use local64_read() and not READ_ONCE(). Steven Rostedt (Google) (2): ring-buffer: Replace rb_time_cmpxchg() with rb_time_cmp_and_update() ring-buffer: Remove 32bit timestamp logic kernel/tr

[PATCH v2 1/2] ring-buffer: Replace rb_time_cmpxchg() with rb_time_cmp_and_update()

2023-12-18 Thread Steven Rostedt
From: "Steven Rostedt (Google)" There's only one place that performs a 64-bit cmpxchg for the timestamp processing. The cmpxchg is only to set the write_stamp equal to the before_stamp, and if it doesn't get set, then the next event will simply be forced to add an absolute timesta

[PATCH v2 2/2] ring-buffer: Remove 32bit timestamp logic

2023-12-18 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Each event has a 27 bit timestamp delta that is used to hold the delta from the last event. If the time between events is greater than 2^27, then a timestamp is added that holds a 59 bit absolute timestamp. Until a389d86f7fd09 ("ring-buffer: Hav

Re: [PATCH 1/2] ring-buffer: Replace rb_time_cmpxchg() with rb_time_cmp_and_update()

2023-12-18 Thread Steven Rostedt
On Mon, 18 Dec 2023 10:15:31 -0500 Steven Rostedt wrote: > Basically I broke it into: > > 1. Remove workaround exposure from the main logic. (this patch) > 2. Remove the workaround. (next patch). > > > > > Isn't this part actual change? > > This part is a

Re: [PATCH v7 0/2] ring-buffer: Rename sub-buffer into buffer page

2023-12-18 Thread Steven Rostedt
On Mon, 18 Dec 2023 11:28:17 -0500 Steven Rostedt wrote: > > Remove all references to sub-buffer and replace them with either bpage > > or ring_buffer_page. The user interface should not be changed. But what I would like to have changed (and this will come after all o

Re: [PATCH v7 0/2] ring-buffer: Rename sub-buffer into buffer page

2023-12-18 Thread Steven Rostedt
On Mon, 18 Dec 2023 15:46:18 + Vincent Donnefort wrote: > Previously was introduced the ability to change the ring-buffer page > size. It also introduced the concept of sub-buffer that is, a contiguous > virtual memory space which can now be bigger than the system page size > (4K on most

[PATCH v3] tracing: Add filter-direct option

2023-12-18 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Normally, when the filter is enabled, a temporary buffer is created to copy the event data into it to perform the filtering logic. If the filter passes and the event should be recorded, then the event is copied from the temporary buffer into the r

Re: [PATCH 1/2] ring-buffer: Replace rb_time_cmpxchg() with rb_time_cmp_and_update()

2023-12-18 Thread Steven Rostedt
On Mon, 18 Dec 2023 23:24:55 +0900 Masami Hiramatsu (Google) wrote: > On Fri, 15 Dec 2023 11:55:13 -0500 > Steven Rostedt wrote: > > > From: "Steven Rostedt (Google)" > > > > There's only one place that performs a 64-bit cmpxchg for the timestamp >

Re: [PATCH] tracing: Add disable-filter-buf option

2023-12-17 Thread Steven Rostedt
On Sun, 17 Dec 2023 17:10:45 +0900 Masami Hiramatsu (Google) wrote: > > >> It exposes the following details which IMHO should be hidden or > > >> configurable in a way that allows moving to a whole new mechanism > > >> which will have significantly different characteristics in the > > >> future:

Re: [PATCH] tracing: Add disable-filter-buf option

2023-12-15 Thread Steven Rostedt
On Fri, 15 Dec 2023 13:25:07 -0500 Mathieu Desnoyers wrote: > > I am not against exposing an ABI that allows userspace to alter the > filter behavior. I disagree on the way you plan to expose the ABI. These are no different than the knobs for sched_debug > > Exposing this option as an ABI in

[PATCH v2] tracing: Add filter-buffer option

2023-12-15 Thread Steven Rostedt
From 62a1de0f0f9d942934565e625a7880fd85ae216a Mon Sep 17 00:00:00 2001 From: "Steven Rostedt (Google)" Date: Fri, 15 Dec 2023 10:26:33 -0500 Subject: [PATCH] tracing: Add filter-buffer option Normally, when the filter is enabled, a temporary buffer is created to copy the

[PATCH v4 15/15] tracing: Update subbuffer with kilobytes not page order

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Using page order for deciding what the size of the ring buffer sub buffers are is exposing a bit too much of the implementation. Although the sub buffers are only allocated in orders of pages, allow the user to specify the minimum size of each sub-

[PATCH v4 14/15] ringbuffer/selftest: Add basic selftest to test changing subbuf order

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Add a self test that will write into the trace buffer with differ trace sub buffer order sizes. Signed-off-by: Steven Rostedt (Google) --- .../ftrace/test.d/00basic/ringbuffer_order.tc | 95 +++ 1 file changed, 95 insertions(+) c

[PATCH v4 13/15] ring-buffer: Add documentation on the buffer_subbuf_order file

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Add to the documentation how to use the buffer_subbuf_order file to change the size and how it affects what events can be added to the ring buffer. Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/ftrace.rst | 27

[PATCH v4 12/15] ring-buffer: Just update the subbuffers when changing their allocation order

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" The ring_buffer_subbuf_order_set() was creating ring_buffer_per_cpu cpu_buffers with the new subbuffers with the updated order, and if they all successfully were created, then they the ring_buffer's per_cpu buffers would be freed and replaced by them. T

[PATCH v4 11/15] ring-buffer: Keep the same size when updating the order

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" The function ring_buffer_subbuf_order_set() just updated the sub-buffers to the new size, but this also changes the size of the buffer in doing so. As the size is determined by nr_pages * subbuf_size. If the subbuf_size is increased without decreasing th

[PATCH v4 09/15] tracing: Update snapshot order along with main buffer order

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" When updating the order of the sub buffers for the main buffer, make sure that if the snapshot buffer exists, that it gets its order updated as well. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/tr

[PATCH v4 10/15] tracing: Stop the tracing while changing the ring buffer subbuf size

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Because the main buffer and the snapshot buffer need to be the same for some tracers, otherwise it will fail and disable all tracing, the tracers need to be stopped while updating the sub buffer sizes so that the tracers see the main and snapsh

[PATCH v4 08/15] ring-buffer: Make sure the spare sub buffer used for reads has same size

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Now that the ring buffer specifies the size of its sub buffers, they all need to be the same size. When doing a read, a swap is done with a spare page. Make sure they are the same size before doing the swap, otherwise the read will fail. Signed-off-

[PATCH v4 06/15] ring-buffer: Clear pages on error in ring_buffer_subbuf_order_set() failure

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" On failure to allocate ring buffer pages, the pointer to the CPU buffer pages is freed, but the pages that were allocated previously were not. Make sure they are freed too. Fixes: TBD ("tracing: Set new size of the ring buffer sub page") S

[PATCH v4 07/15] ring-buffer: Do no swap cpu buffers if order is different

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" As all the subbuffer order (subbuffer sizes) must be the same throughout the ring buffer, check the order of the buffers that are doing a CPU buffer swap in ring_buffer_swap_cpu() to make sure they are the same. If the are not the same, then fail to d

[PATCH v4 05/15] ring-buffer: Read and write to ring buffers with custom sub buffer size

2023-12-15 Thread Steven Rostedt
_read_page() ring_buffer_read_page() A new API is introduced: ring_buffer_read_page_data() Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-6-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/rin

[PATCH v4 04/15] ring-buffer: Set new size of the ring buffer sub page

2023-12-15 Thread Steven Rostedt
race-devel/20211213094825.61876-5-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 80 ++ 1 file changed, 73 insertions(+), 7 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/ke

[PATCH v4 03/15] ring-buffer: Add interface for configuring trace sub buffer size

2023-12-15 Thread Steven Rostedt
mir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 4 ++ kernel/trace/ring_buffer.c | 73 + kernel/trace/trace.c| 48 3 files changed, 125 insertions(+) diff --git a/include/linux/ring_buff

[PATCH v4 02/15] ring-buffer: Page size per ring buffer

2023-12-15 Thread Steven Rostedt
fer. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-3-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 2 +- kernel/trace/ring_buffer.c | 68 +--

[PATCH v4 01/15] ring-buffer: Refactor ring buffer implementation

2023-12-15 Thread Steven Rostedt
inux-trace-devel/20211213094825.61876-2-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 60 +++--- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/ke

[PATCH v4 00/15] ring-buffer/tracing: Allow ring buffer to have bigger sub buffers

2023-12-15 Thread Steven Rostedt
/linux-trace.git - I made the tests a bit more advanced. Still a smoke test, but it now checks if the string written is the same as the string read. Steven Rostedt (Google) (10): ring-buffer: Clear pages on error in ring_buffer_subbuf_order_set() failure ring-buffer: Do no swap cpu

Re: [PATCH] tracing: Add disable-filter-buf option

2023-12-15 Thread Steven Rostedt
On Fri, 15 Dec 2023 12:24:14 -0500 Mathieu Desnoyers wrote: > On 2023-12-15 12:04, Steven Rostedt wrote: > > On Fri, 15 Dec 2023 10:53:39 -0500 > > Mathieu Desnoyers wrote: > [...] > >> > >> So rather than stacking tons of "on/off" switches for

Re: [PATCH] tracing: Add disable-filter-buf option

2023-12-15 Thread Steven Rostedt
On Fri, 15 Dec 2023 10:53:39 -0500 Mathieu Desnoyers wrote: > > > I'm not convinced that a boolean state is what you need here. I will admit the biggest motivation for this was to allow for debugging ;-) > > Yes, today you are in a position where you have two options: > > a) use the filter

[PATCH 1/2] ring-buffer: Replace rb_time_cmpxchg() with rb_time_cmp_and_update()

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" There's only one place that performs a 64-bit cmpxchg for the timestamp processing. The cmpxchg is only to set the write_stamp equal to the before_stamp, and if it doesn't get set, then the next event will simply be forced to add an absolute timesta

[PATCH 2/2] ring-buffer: Remove 32bit timestamp logic

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Each event has a 27 bit timestamp delta that is used to hold the delta from the last event. If the time between events is greater than 2^27, then a timestamp is added that holds a 59 bit absolute timestamp. Until a389d86f7fd09 ("ring-buffer: Hav

[PATCH 0/2] tracing: Replace final 64-bit cmpxchg with compare and update if available

2023-12-15 Thread Steven Rostedt
es the complex logic of that code. The difference now is that 32-bit architectures will have to add an absolute timestamp to an event that follows an event that suffered the race condition. Steven Rostedt (Google) (2): ring-buffer: Replace rb_time_cmpxchg() with rb_time_cmp_and_update() ring-

[PATCH] tracing: Add disable-filter-buf option

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Normally, when the filter is enabled, a temporary buffer is created to copy the event data into it to perform the filtering logic. If the filter passes and the event should be recorded, then the event is copied from the temporary buffer into the r

[PATCH] ring-buffer: Have rb_time_cmpxchg() set the msb counter too

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" The rb_time_cmpxchg() on 32-bit architectures requires setting three 32-bit words to represent the 64-bit timestamp, with some salt for synchronization. Those are: msb, top, and bottom The issue is, the rb_time_cmpxchg() did not properly salt the m

[PATCH v2] ring-buffer: Remove useless update to write_stamp in rb_try_to_discard()

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" When filtering is enabled, a temporary buffer is created to place the content of the trace event output so that the filter logic can decide from the trace event output if the trace event should be filtered out or not. If it is to be filtered out, t

Re: [PATCH] ring-buffer: Remove useless update to write_stamp in rb_try_to_discard()

2023-12-15 Thread Steven Rostedt
On Fri, 15 Dec 2023 07:41:51 -0500 Steven Rostedt wrote: > From: "Steven Rostedt (Google)" > > When filtering is enabled, a temporary buffer is created to place the > content of the trace event output so that the filter logic can decide > from the trace event output

[PATCH] ring-buffer: Remove useless update to write_stamp in rb_try_to_discard()

2023-12-15 Thread Steven Rostedt
From: "Steven Rostedt (Google)" When filtering is enabled, a temporary buffer is created to place the content of the trace event output so that the filter logic can decide from the trace event output if the trace event should be filtered out or not. If it is to be filtered out, t

Re: [PATCH] ring-buffer: Do not try to put back write_stamp

2023-12-14 Thread Steven Rostedt
On Thu, 14 Dec 2023 22:18:03 -0500 Steven Rostedt wrote: > From: "Steven Rostedt (Google)" > > If an update to an event is interrupted by another event between the time > the initial event allocated its buffer and where it wrote to the > write_stamp, the code try to r

[PATCH v2] ring-buffer: Do not try to put back write_stamp

2023-12-14 Thread Steven Rostedt
From: "Steven Rostedt (Google)" If an update to an event is interrupted by another event between the time the initial event allocated its buffer and where it wrote to the write_stamp, the code try to reset the write stamp back to the what it had just overwritten. It knows that it was o

[PATCH] ring-buffer: Do not try to put back write_stamp

2023-12-14 Thread Steven Rostedt
From: "Steven Rostedt (Google)" If an update to an event is interrupted by another event between the time the initial event allocated its buffer and where it wrote to the write_stamp, the code try to reset the write stamp back to the what it had just overwritten. It knows that it was o

Re: [PATCH] ring-buffer: Remove 32bit timestamp logic

2023-12-14 Thread Steven Rostedt
On Thu, 14 Dec 2023 12:50:29 -0800 Linus Torvalds wrote: > > But do all archs have an implementation of cmpxchg64, even if it requires > > disabling interrupts? If not, then I definitely cannot remove this code. > > We have a generic header file, so anybody who uses that would get the >

Re: [PATCH v3] ring-buffer: Remove 32bit timestamp logic

2023-12-14 Thread Steven Rostedt
On Thu, 14 Dec 2023 12:32:38 -0800 Linus Torvalds wrote: > On Thu, 14 Dec 2023 at 12:30, Linus Torvalds > wrote: > > > > Read my email. Don't do random x86-centric things. We have that > > > > #ifndef system_has_cmpxchg64 > > #define system_has_cmpxchg64() false > > #endif > > > >

Re: [PATCH] ring-buffer: Remove 32bit timestamp logic

2023-12-14 Thread Steven Rostedt
On Thu, 14 Dec 2023 11:44:55 -0800 Linus Torvalds wrote: > On Thu, 14 Dec 2023 at 08:55, Steven Rostedt wrote: > > > > And yes, this does get called in NMI context. > > Not on an i486-class machine they won't. You don't have a local apic > on those, and they won't ha

Re: [PATCH v3] ring-buffer: Remove 32bit timestamp logic

2023-12-14 Thread Steven Rostedt
On Thu, 14 Dec 2023 15:19:11 -0500 Steven Rostedt wrote: > For this issue of the 64bit cmpxchg, is there any config that works for any > arch that do not have a safe 64-bit cmpxchg? At least for 486, is the > second half of the if condition reasonable? > > if (IS_ENABLE

Re: [PATCH v3] ring-buffer: Remove 32bit timestamp logic

2023-12-14 Thread Steven Rostedt
On Thu, 14 Dec 2023 11:46:55 -0800 Linus Torvalds wrote: > On Thu, 14 Dec 2023 at 09:53, Steven Rostedt wrote: > > > > + /* > > +* For architectures that can not do cmpxchg() in NMI, or require > > +* disabling interrupts to do 64-bit c

[PATCH v3] ring-buffer: Remove 32bit timestamp logic

2023-12-14 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Each event has a 27 bit timestamp delta that is used to hold the delta from the last event. If the time between events is greater than 2^27, then a timestamp is added that holds a 59 bit absolute timestamp. Until a389d86f7fd09 ("ring-buffer: Hav

Re: [PATCH] ring-buffer: Remove 32bit timestamp logic

2023-12-14 Thread Steven Rostedt
On Wed, 13 Dec 2023 22:53:19 -0800 Linus Torvalds wrote: > On Wed, 13 Dec 2023 at 18:45, Steven Rostedt wrote: > > > > tl;dr; The ring-buffer timestamp requires a 64-bit cmpxchg to keep the > > timestamps in sync (only in the slow paths). I was told that 64-bit cmpxchg

[PATCH v2] ring-buffer: Remove 32bit timestamp logic

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Each event has a 27 bit timestamp delta that is used to hold the delta from the last event. If the time between events is greater than 2^27, then a timestamp is added that holds a 59 bit absolute timestamp. Until a389d86f7fd09 ("ring-buffer: Hav

Re: [PATCH] ring-buffer: Remove 32bit timestamp logic

2023-12-13 Thread Steven Rostedt
On Wed, 13 Dec 2023 21:11:26 -0500 Steven Rostedt wrote: > @@ -3720,6 +3517,7 @@ rb_reserve_next_event(struct trace_buffer *buffer, > struct rb_event_info info; > int nr_loops = 0; > int add_ts_default; > + static int once; > > /* ring buffer do

Re: [PATCH] ring-buffer: Remove 32bit timestamp logic

2023-12-13 Thread Steven Rostedt
hould just say "screw it" and nuke it now? Or do you think it's worth keeping for some other architecture that 3 32-bit cmpxchgs are still faster than a single 64-bit one? Thanks, -- Steve On Wed, 13 Dec 2023 21:11:26 -0500 Steven Rostedt wrote: > From: "Steven Rostedt (G

[PATCH] ring-buffer: Remove 32bit timestamp logic

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Each event has a 27 bit timestamp delta that is used to hold the delta from the last event. If the time between events is greater than 2^27, then a timestamp is added that holds a 59 bit absolute timestamp. Until a389d86f7fd09 ("ring-buffer: Hav

[PATCH] ring-buffer: Do not record in NMI if the arch does not support cmpxchg in NMI

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" As the ring buffer recording requires cmpxchg() to work, if the architecture does not support cmpxchg in NMI, then do not do any recording within an NMI. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 6 ++ 1 file

[PATCH v3 00/15] ring-buffer/tracing: Allow ring buffer to have bigger sub buffers

2023-12-13 Thread Steven Rostedt
the implemantion is. - I rebased on top of trace/core in the: git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git - I made the tests a bit more advanced. Still a smoke test, but it now checks if the string written is the same as the string read. Steven Rostedt (Google)

[PATCH v3 04/15] ring-buffer: Set new size of the ring buffer sub page

2023-12-13 Thread Steven Rostedt
race-devel/20211213094825.61876-5-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 80 ++ 1 file changed, 73 insertions(+), 7 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/ke

[PATCH v3 14/15] ringbuffer/selftest: Add basic selftest to test changing subbuf order

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Add a self test that will write into the trace buffer with differ trace sub buffer order sizes. Signed-off-by: Steven Rostedt (Google) --- .../ftrace/test.d/00basic/ringbuffer_order.tc | 95 +++ 1 file changed, 95 insertions(+) c

[PATCH v3 05/15] ring-buffer: Read and write to ring buffers with custom sub buffer size

2023-12-13 Thread Steven Rostedt
_read_page() ring_buffer_read_page() A new API is introduced: ring_buffer_read_page_data() Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-6-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/rin

[PATCH v3 15/15] tracing: Update subbuffer with kilobytes not page order

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Using page order for deciding what the size of the ring buffer sub buffers are is exposing a bit too much of the implementation. Although the sub buffers are only allocated in orders of pages, allow the user to specify the minimum size of each sub-

[PATCH v3 09/15] tracing: Update snapshot order along with main buffer order

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" When updating the order of the sub buffers for the main buffer, make sure that if the snapshot buffer exists, that it gets its order updated as well. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/tr

[PATCH v3 03/15] ring-buffer: Add interface for configuring trace sub buffer size

2023-12-13 Thread Steven Rostedt
mir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 4 ++ kernel/trace/ring_buffer.c | 73 + kernel/trace/trace.c| 48 3 files changed, 125 insertions(+) diff --git a/include/linux/ring_buff

[PATCH v3 08/15] ring-buffer: Make sure the spare sub buffer used for reads has same size

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Now that the ring buffer specifies the size of its sub buffers, they all need to be the same size. When doing a read, a swap is done with a spare page. Make sure they are the same size before doing the swap, otherwise the read will fail. Signed-off-

[PATCH v3 10/15] tracing: Stop the tracing while changing the ring buffer subbuf size

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Because the main buffer and the snapshot buffer need to be the same for some tracers, otherwise it will fail and disable all tracing, the tracers need to be stopped while updating the sub buffer sizes so that the tracers see the main and snapsh

[PATCH v3 12/15] ring-buffer: Just update the subbuffers when changing their allocation order

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" The ring_buffer_subbuf_order_set() was creating ring_buffer_per_cpu cpu_buffers with the new subbuffers with the updated order, and if they all successfully were created, then they the ring_buffer's per_cpu buffers would be freed and replaced by them. T

[PATCH v3 11/15] ring-buffer: Keep the same size when updating the order

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" The function ring_buffer_subbuf_order_set() just updated the sub-buffers to the new size, but this also changes the size of the buffer in doing so. As the size is determined by nr_pages * subbuf_size. If the subbuf_size is increased without decreasing th

[PATCH v3 02/15] ring-buffer: Page size per ring buffer

2023-12-13 Thread Steven Rostedt
fer. Link: https://lore.kernel.org/linux-trace-devel/20211213094825.61876-3-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- include/linux/ring_buffer.h | 2 +- kernel/trace/ring_buffer.c | 68 +--

[PATCH v3 13/15] ring-buffer: Add documentation on the buffer_subbuf_order file

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" Add to the documentation how to use the buffer_subbuf_order file to change the size and how it affects what events can be added to the ring buffer. Signed-off-by: Steven Rostedt (Google) --- Documentation/trace/ftrace.rst | 27

[PATCH v3 07/15] ring-buffer: Do no swap cpu buffers if order is different

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" As all the subbuffer order (subbuffer sizes) must be the same throughout the ring buffer, check the order of the buffers that are doing a CPU buffer swap in ring_buffer_swap_cpu() to make sure they are the same. If the are not the same, then fail to d

[PATCH v3 06/15] ring-buffer: Clear pages on error in ring_buffer_subbuf_order_set() failure

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" On failure to allocate ring buffer pages, the pointer to the CPU buffer pages is freed, but the pages that were allocated previously were not. Make sure they are freed too. Fixes: TBD ("tracing: Set new size of the ring buffer sub page") S

[PATCH v3 01/15] ring-buffer: Refactor ring buffer implementation

2023-12-13 Thread Steven Rostedt
inux-trace-devel/20211213094825.61876-2-tz.stoya...@gmail.com Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ring_buffer.c | 60 +++--- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/ke

[PATCH v2] tracing: Increase size of trace_marker_raw to max ring buffer entry

2023-12-13 Thread Steven Rostedt
From: "Steven Rostedt (Google)" There's no reason to give an arbitrary limit to the size of a raw trace marker. Just let it be as big as the size that is allowed by the ring buffer itself. And there's also no reason to artificially break up the write to TRACE_BUF_SIZE, as that's not

<    3   4   5   6   7   8   9   10   11   12   >