On 2024/6/1 08:31, Jeff Johnson wrote:
make allmodconfig && make W=1 C=1 reports:
WARNING: modpost: missing MODULE_DESCRIPTION() in lib/test_objpool.o
Add the missing invocation of the MODULE_DESCRIPTION() macro.
Signed-off-by: Jeff Johnson
---
lib/test_objpool.c | 3 ++-
1 file changed, 2
On 2024/5/10 16:20, Vlastimil Babka wrote:
On 5/10/24 9:59 AM, wuqiang.matt wrote:
On 2024/5/7 21:55, Vlastimil Babka wrote:
>>
+ } while (!try_cmpxchg_acquire(>tail, , tail + 1));
+
+ /* now the tail position is reserved for the given obj */
+ WRITE_ONCE(slot
On 2024/5/7 21:55, Vlastimil Babka wrote:
On 4/24/24 11:52 PM, Andrii Nakryiko wrote:
objpool_push() and objpool_pop() are very performance-critical functions
and can be called very frequently in kretprobe triggering path.
As such, it makes sense to allow compiler to inline them completely to
Hello Andi,
On 2023/11/23 06:17, Andi Shyti wrote:
Hi Wuqiang,
On Tue, Nov 21, 2023 at 10:23:43PM +0800, wuqiang.matt wrote:
arch_cmpxchg() should check data size rather than pointer size in case
CONFIG_ARC_HAS_LLSC is defined. So rename __cmpxchg to __cmpxchg_32 to
emphasize it's explicit
The xtensa architecture already has __cmpxchg_local defined but not used.
The purpose of __cmpxchg_local() is solely for arch_cmpxchg_local(), just
as the definition of arch_cmpxchg_local() for other architectures like x86,
arm and powerpc.
Signed-off-by: wuqiang.matt
---
arch/xtensa/include
generci_cmpxchg[64]_local is to be used.
Reported-by: kernel test robot
Closes:
https://lore.kernel.org/oe-kbuild-all/202310272207.tlpflya4-...@intel.com/
Signed-off-by: wuqiang.matt
Reviewed-by: Masami Hiramatsu (Google)
---
arch/hexagon/include/asm/cmpxchg.h | 51 +-
1 file
generci_cmpxchg[64]_local is to be used.
Reported-by: kernel test robot
Closes:
https://lore.kernel.org/oe-kbuild-all/202310272207.tlpflya4-...@intel.com/
Signed-off-by: wuqiang.matt
Reviewed-by: Masami Hiramatsu (Google)
---
arch/openrisc/include/asm/cmpxchg.h | 6 ++
1 file changed, 6 insertions
generci_cmpxchg[64]_local is to be used.
Reported-by: kernel test robot
Closes:
https://lore.kernel.org/oe-kbuild-all/202310272207.tlpflya4-...@intel.com/
Signed-off-by: wuqiang.matt
Reviewed-by: Masami Hiramatsu (Google)
---
arch/arc/include/asm/cmpxchg.h | 28
1 file
mpxchg variants if avaialble, as Arnd advised
Signed-off-by: wuqiang.matt
Reviewed-by: Masami Hiramatsu (Google)
---
arch/arc/include/asm/cmpxchg.h | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/arc/include/asm/cmpxchg.h b/arch/arc/include/asm/cmpxchg.h
/openrisc/hexagon. Patch 5
defines arch_cmpxchg_local as existing __cmpxchg_local rather the
generic variant.
wuqiang.matt (5):
arch,locking/atomic: arc: arch_cmpxchg should check data size
arch,locking/atomic: arc: add arch_cmpxchg[64]_local
arch,locking/atomic: openrisc: add arch_cmpxchg[64
quot;lib: objpool added: ring-array based lockless MPMC")
v1 -> v2:
- Title updated since it's a common issue of objpool for big.LITTLE
systems, verified on Rockchip RK3588 and Amlogic A311D
- Fixes tag added, as reminded by Masami Hiramatsu
Signed-off-by: wuqiang.matt
Ack
On 2023/11/20 13:18, Masami Hiramatsu (Google) wrote:
On Tue, 14 Nov 2023 19:51:48 +0800
"wuqiang.matt" wrote:
objpool overrun stress with test_objpool on OrangePi5+ SBC triggered the
following kernel warnings:
WARNING: CPU: 6 PID: 3115 at lib/objpool.c:168 objpool_push+
3730493 232232850
24T32T48T64T96T
native:746406039 1000174750 1493236240 1998318364 2942911180
objpool: 349164852 467284332 702296756 934459713 1387898285
objpool+: 348388180 462750976 696606096 927865887 1368402195
Signed-off-by: wuqi
-...@intel.com/
Signed-off-by: wuqiang.matt
---
lib/test_objpool.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/lib/test_objpool.c b/lib/test_objpool.c
index a94078402138..bfdb81599832 100644
--- a/lib/test_objpool.c
+++ b/lib/test_objpool.c
@@ -311,7 +311,7 @@ static
For architectures that support native cmpxchg, we'd like to
implement arch_cmpxchg[64]_local with the native variants of
supported data size. If not, the generci_cmpxchg[64]_local
will be used.
Signed-off-by: wuqiang.matt
---
arch/openrisc/include/asm/cmpxchg.h | 6 ++
1 file changed, 6
-...@intel.com/
Signed-off-by: wuqiang.matt
---
arch/hexagon/include/asm/cmpxchg.h | 51 +-
1 file changed, 50 insertions(+), 1 deletion(-)
diff --git a/arch/hexagon/include/asm/cmpxchg.h
b/arch/hexagon/include/asm/cmpxchg.h
index bf6cf5579cf4..2b5e5bbaf807 100644
For architectures that support native cmpxchg, we'd like to
implement arch_cmpxchg[64]_local with the native variants of
supported data size. If not, the generci_cmpxchg[64]_local
will be used.
Signed-off-by: wuqiang.matt
---
arch/arc/include/asm/cmpxchg.h | 28
1
checking is uncecessary.
Signed-off-by: wuqiang.matt
---
arch/arc/include/asm/cmpxchg.h | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/arc/include/asm/cmpxchg.h b/arch/arc/include/asm/cmpxchg.h
index e138fde067de..bf46514f6f12 100644
--- a/arch/arc/include
This patch series implement arch_cmpxchg[64]_local for arc,
openrisc and hexagon.
For architectures that support native cmpxchg, we'd like to
implement arch_cmpxchg[64]_local with the native variants of
supported data size. If not, the generci_cmpxchg[64]_local
will be used.
wuqiang.matt (4
On 2023/10/30 01:05, Guenter Roeck wrote:
On Mon, Oct 23, 2023 at 07:24:52PM +0800, wuqiang.matt wrote:
The objpool_push can only happen on local cpu node, so only the local
cpu can touch slot->tail and slot->last, which ensures the correctness
of using cmpxchg without lock prefix
On 2023/10/26 16:46, Arnd Bergmann wrote:
On Thu, Oct 26, 2023, at 09:39, wuqiang.matt wrote:
arch_cmpxchg[64]_local() are not defined for openrisc. So implement
them with generci_cmpxchg[64]_local, advised by Masami Hiramatsu.
Closes:
https://lore.kernel.org/linux-trace-kernel
/202310241310.ir5uukog-...@intel.com
Signed-off-by: wuqiang.matt
---
arch/openrisc/include/asm/cmpxchg.h | 14 ++
1 file changed, 14 insertions(+)
diff --git a/arch/openrisc/include/asm/cmpxchg.h
b/arch/openrisc/include/asm/cmpxchg.h
index 8ee151c072e4..7d0555257389 100644
--- a/arch/openrisc
/202310241310.ir5uukog-...@intel.com
Signed-off-by: wuqiang.matt
---
arch/hexagon/include/asm/cmpxchg.h | 14 ++
1 file changed, 14 insertions(+)
diff --git a/arch/hexagon/include/asm/cmpxchg.h
b/arch/hexagon/include/asm/cmpxchg.h
index bf6cf5579cf4..c14520730194 100644
--- a/arch/hexagon
-kbuild-all/202310241310.ir5uukog-...@intel.com/
Signed-off-by: wuqiang.matt
---
arch/arc/include/asm/cmpxchg.h | 14 ++
1 file changed, 14 insertions(+)
diff --git a/arch/arc/include/asm/cmpxchg.h b/arch/arc/include/asm/cmpxchg.h
index e138fde067de..ef644cdbe956 100644
--- a/arch/arc
On 2023/10/24 22:52, Masami Hiramatsu (Google) wrote:
From: Masami Hiramatsu (Google)
Use generic_cmpxchg_local() for arch_cmpxchg_local() implementation
in SH architecture because it does not implement arch_cmpxchg_local().
Reported-by: kernel test robot
Closes:
On 2023/10/25 09:51, wuqiang.matt wrote:
On 2023/10/25 07:42, Masami Hiramatsu (Google) wrote:
On Tue, 24 Oct 2023 16:08:12 +0100
Mark Rutland wrote:
On Tue, Oct 24, 2023 at 11:52:54PM +0900, Masami Hiramatsu (Google) wrote:
From: Masami Hiramatsu (Google)
Use generic_cmpxchg_local
,
with the original loop of all cpu nodes removed
Signed-off-by: wuqiang.matt
---
lib/objpool.c | 26 +-
1 file changed, 9 insertions(+), 17 deletions(-)
diff --git a/lib/objpool.c b/lib/objpool.c
index a032701beccb..7474a3a60cad 100644
--- a/lib/objpool.c
+++ b/lib/objpool.c
On 2023/10/25 07:42, Masami Hiramatsu (Google) wrote:
On Tue, 24 Oct 2023 16:08:12 +0100
Mark Rutland wrote:
On Tue, Oct 24, 2023 at 11:52:54PM +0900, Masami Hiramatsu (Google) wrote:
From: Masami Hiramatsu (Google)
Use generic_cmpxchg_local() for arch_cmpxchg_local() implementation
in SH
On 2023/10/24 09:01, Masami Hiramatsu (Google) wrote:
On Mon, 23 Oct 2023 11:43:04 -0400
Steven Rostedt wrote:
On Mon, 23 Oct 2023 19:24:52 +0800
"wuqiang.matt" wrote:
The objpool_push can only happen on local cpu node, so only the local
cpu can touch slot->tail and slot
526317 1454991286 1861053557 2059530343 2171732306
Signed-off-by: wuqiang.matt
---
lib/objpool.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/objpool.c b/lib/objpool.c
index ce0087f64400..a032701beccb 100644
--- a/lib/objpool.c
+++ b/lib/objpool.c
@@ -166,7
://lore.kernel.org/oe-kbuild-all/202310191512.vvypku5z-...@intel.com/
Signed-off-by: wuqiang.matt
---
kernel/trace/rethook.c | 2 --
lib/test_objpool.c | 6 --
2 files changed, 8 deletions(-)
diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c
index 13c8e6773892..6fd7d4ecbbc6 100644
On 2023/10/18 10:18, Chengming Zhou wrote:
On 2023/10/17 21:56, wuqiang.matt wrote:
objpool is a scalable implementation of high performance queue for
object allocation and reclamation, such as kretprobe instances.
With leveraging percpu ring-array to mitigate hot spots of memory
contention
This patch will remove freelist.h from kernel source tree, since the
only use cases (kretprobe and rethook) are converted to objpool.
Signed-off-by: wuqiang.matt
---
include/linux/freelist.h | 129 ---
1 file changed, 129 deletions(-)
delete mode 100644
objpool, a scalable and lockless ring-array based object pool, was
introduced to replace the original freelist (a LIFO queue based on
singly linked list) to improve kretprobe scalability.
Signed-off-by: wuqiang.matt
---
MAINTAINERS | 7 +++
1 file changed, 7 insertions(+)
diff --git
33824782945292
objpool: 328192025 439439564 668534502 887401381 1319972072
Signed-off-by: wuqiang.matt
---
include/linux/kprobes.h | 11 ++---
include/linux/rethook.h | 16 ++-
kernel/kprobes.c| 93 +
kernel/trace/fprobe.c | 32
to control the objpool lifecycle
5) group 5: asynchronous mode with hrtimer
hrtimer softirq is introduced to stress async objpool operations
Signed-off-by: wuqiang.matt
---
lib/Kconfig.debug | 11 +
lib/Makefile | 2 +
lib/test_objpool.c | 689
array,
which consumes more memory than linked lists
Signed-off-by: wuqiang.matt
---
include/linux/objpool.h | 176 +
lib/Makefile| 2 +-
lib/objpool.c | 286
3 files changed, 463 insertions(+), 1 deletion
releasing of objpool now covered
wuqiang.matt (5):
lib: objpool added: ring-array based lockless MPMC
lib: objpool test module added
kprobes: kretprobe scalability improvement with objpool
kprobes: freelist.h removed
MAINTAINERS: objpool added
MAINTAINERS | 7 +
include
/* drop all remained objects from objpool */
+ while (objpool_pop(pool))
+ count++;
+
+ if (refcount_sub_and_test(count, >ref))
+ objpool_free(pool);
+}
+EXPORT_SYMBOL_GPL(objpool_fini);
--
Regards,
Wuqiang
On 2023/10/16 20:18, Masami Hiramatsu (
On 2023/10/16 21:21, Masami Hiramatsu (Google) wrote:
On Sun, 15 Oct 2023 13:32:49 +0800
"wuqiang.matt" wrote:
kretprobe is using freelist to manage return-instances, but freelist,
as LIFO queue based on singly linked list, scales badly and reduces
the overall throughput of
On 2023/10/16 20:18, Masami Hiramatsu (Google) wrote:
Hi Wuqiang,
On Mon, 16 Oct 2023 10:45:30 +0800
"wuqiang.matt" wrote:
On 2023/10/16 07:26, Masami Hiramatsu (Google) wrote:
On Mon, 16 Oct 2023 00:06:11 +0800
"wuqiang.matt" wrote:
On 2023/10/15 23:43, Masami Hira
On 2023/10/16 20:18, Masami Hiramatsu (Google) wrote:
Hi Wuqiang,
On Mon, 16 Oct 2023 10:45:30 +0800
"wuqiang.matt" wrote:
On 2023/10/16 07:26, Masami Hiramatsu (Google) wrote:
On Mon, 16 Oct 2023 00:06:11 +0800
"wuqiang.matt" wrote:
On 2023/10/15 23:43, Masami Hira
On 2023/10/16 07:26, Masami Hiramatsu (Google) wrote:
On Mon, 16 Oct 2023 00:06:11 +0800
"wuqiang.matt" wrote:
On 2023/10/15 23:43, Masami Hiramatsu (Google) wrote:
On Sun, 15 Oct 2023 13:32:47 +0800
"wuqiang.matt" wrote:
objpool is a scalable implementation of hi
On 2023/10/15 23:43, Masami Hiramatsu (Google) wrote:
On Sun, 15 Oct 2023 13:32:47 +0800
"wuqiang.matt" wrote:
objpool is a scalable implementation of high performance queue for
object allocation and reclamation, such as kretprobe instances.
With leveraging percpu ring-array to mi
objpool, a scalable and lockless ring-array based object pool, was
introduced to replace the original freelist (a LIFO queue based on
singly linked list) to improve kretprobe scalability.
Signed-off-by: wuqiang.matt
---
MAINTAINERS | 7 +++
1 file changed, 7 insertions(+)
diff --git
This patch will remove freelist.h from kernel source tree, since the
only use cases (kretprobe and rethook) are converted to objpool.
Signed-off-by: wuqiang.matt
---
include/linux/freelist.h | 129 ---
1 file changed, 129 deletions(-)
delete mode 100644
33824782945292
objpool: 328192025 439439564 668534502 887401381 990067903
Signed-off-by: wuqiang.matt
---
include/linux/kprobes.h | 11 ++---
include/linux/rethook.h | 16 ++-
kernel/kprobes.c| 93 +
kernel/trace/fprobe.c | 32
to control the objpool lifecycle
5) group 5: asynchronous mode with hrtimer
hrtimer softirq is introduced to stress async objpool operations
Signed-off-by: wuqiang.matt
---
lib/Kconfig.debug | 11 +
lib/Makefile | 2 +
lib/test_objpool.c | 689
array,
which consumes more memory than linked lists
Signed-off-by: wuqiang.matt
---
include/linux/objpool.h | 176 +
lib/Makefile| 2 +-
lib/objpool.c | 286
3 files changed, 463 insertions(+), 1 deletion
This patch series introduces a scalable and lockless ring-array based
object pool to improve scalability of kretprobed routines.
v10:
*) objpool simplified and improved according to Masami's advices
wuqiang.matt (5):
lib: objpool added: ring-array based lockless MPMC
lib: objpool test
On 2023/10/13 09:59, Masami Hiramatsu (Google) wrote:
On Fri, 13 Oct 2023 01:36:05 +0800
"wuqiang.matt" wrote:
On 2023/10/12 22:02, Masami Hiramatsu (Google) wrote:
Hi Wuqiang,
On Mon, 9 Oct 2023 17:23:34 +0800
wuqiang wrote:
Hello Masami,
Just got time for the new pat
1)
try_cmpxchg_acquire(>tail, , next)
WRITE_ONCE(slot->entries[tail & slot->mask], obj)
return slot->entries[head & slot->mask]
The pre-condition should be: CPU 1 tries to push to a full slot, in this case
tail = head + capacity but tail
))
+ objpool_free(pool);
+ } while (obj);
+}
+EXPORT_SYMBOL_GPL(objpool_fini);
On 2023/10/9 22:19, Masami Hiramatsu (Google) wrote:
Hi,
On Mon, 9 Oct 2023 02:40:53 +0800
wuqiang wrote:
On 2023/9/23 17:48, Masami Hiramatsu (Google) wrote:
Hi Wuqiang,
Sorry for
53 matches
Mail list logo
