[PATCH v2] MAINTAINERS: Enlarge coverage of TRACING inside architectures
Most architectures have ftrace related stuff in arch/*/kernel/ftrace.c but powerpc has it spread in multiple files located in arch/powerpc/kernel/trace/ In several architectures, there are also additional files containing 'ftrace' as part of the name but with some prefix or suffix. Use wildcards to enlarge coverage. With arch/*/*/*/*ftrace*: arch/alpha/include/asm/ftrace.h arch/arm64/include/asm/ftrace.h arch/arm/include/asm/ftrace.h arch/csky/include/asm/ftrace.h arch/csky/kernel/probes/ftrace.c arch/ia64/include/asm/ftrace.h arch/m68k/include/asm/ftrace.h arch/microblaze/include/asm/ftrace.h arch/mips/include/asm/ftrace.h arch/nds32/include/asm/ftrace.h arch/parisc/include/asm/ftrace.h arch/powerpc/include/asm/ftrace.h arch/powerpc/kernel/trace/ftrace_64_pg.S arch/powerpc/kernel/trace/ftrace.c arch/powerpc/kernel/trace/ftrace_low.S arch/powerpc/kernel/trace/ftrace_mprofile.S arch/riscv/include/asm/ftrace.h arch/riscv/kernel/probes/ftrace.c arch/s390/include/asm/ftrace.h arch/s390/include/asm/ftrace.lds.h arch/sh/include/asm/ftrace.h arch/sparc/include/asm/ftrace.h arch/x86/include/asm/ftrace.h arch/x86/kernel/kprobes/ftrace.c arch/xtensa/include/asm/ftrace.h With arch/*/*/*ftrace*: arch/arm64/kernel/entry-ftrace.S arch/arm64/kernel/ftrace.c arch/arm/kernel/entry-ftrace.S arch/arm/kernel/ftrace.c arch/csky/kernel/ftrace.c arch/ia64/kernel/ftrace.c arch/microblaze/kernel/ftrace.c arch/mips/kernel/ftrace.c arch/nds32/kernel/ftrace.c arch/parisc/kernel/ftrace.c arch/powerpc/kernel/kprobes-ftrace.c arch/riscv/kernel/ftrace.c arch/s390/kernel/ftrace.c arch/s390/kernel/ftrace.h arch/sh/kernel/ftrace.c arch/sparc/kernel/ftrace.c arch/x86/kernel/ftrace_32.S arch/x86/kernel/ftrace_64.S arch/x86/kernel/ftrace.c Signed-off-by: Christophe Leroy --- MAINTAINERS | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/MAINTAINERS b/MAINTAINERS index e127c2fb08a7..e03c471fcd73 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -19590,8 +19590,8 @@ M: Ingo Molnar S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git F: Documentation/trace/ftrace.rst -F: arch/*/*/*/ftrace.h -F: arch/*/kernel/ftrace.c +F: arch/*/*/*/*ftrace* +F: arch/*/*/*ftrace* F: fs/tracefs/ F: include/*/ftrace.h F: include/linux/trace*.h -- 2.35.1
Re: [PATCH 1/2] powerpc: Reject probes on instructions that can't be single stepped
Murilo Opsfelder Araújo writes:
> On 3/23/22 08:51, Naveen N. Rao wrote:
...
>> +case 31:
>> +switch ((inst >> 1) & 0x3ff) {
>> +case 4: /* tw */
>> +return false;
>> +case 68:/* td */
>> +return false;
>> +case 146: /* mtmsr */
>> +return false;
>> +case 178: /* mtmsrd */
>> +return false;
>> +}
>> +break;
>> +}
>> +return true;
>> +}
>> +
>
> Can't OP_* definitions from ppc-opcode.h be used for all of these switch-case
> statements?
Yes please. And add any that are missing.
cheers
Re: [PATCH v1] ftrace: Make ftrace_graph_is_dead() static inline
On Thu, 24 Mar 2022 11:01:45 +0100
Christophe Leroy wrote:
> @@ -1006,7 +1006,20 @@ unsigned long ftrace_graph_ret_addr(struct task_struct
> *task, int *idx,
> extern int register_ftrace_graph(struct fgraph_ops *ops);
> extern void unregister_ftrace_graph(struct fgraph_ops *ops);
>
> -extern bool ftrace_graph_is_dead(void);
> +/**
> + * ftrace_graph_is_dead - returns true if ftrace_graph_stop() was called
> + *
> + * ftrace_graph_stop() is called when a severe error is detected in
> + * the function graph tracing. This function is called by the critical
> + * paths of function graph to keep those paths from doing any more harm.
> + */
> +extern bool kill_ftrace_graph;
> +
> +static inline bool ftrace_graph_is_dead(void)
> +{
> + return kill_ftrace_graph;
> +}
> +
> extern void ftrace_graph_stop(void);
The reason I did not expose that variable, is because I didn't want it to
be touched outside of the kernel/trace directory. Or the ftrace.c file for
that matter (although, I could put it in fgraph.c :-/)
What would be better, is to make it a static branch.
extern struct static_key fgraph_dead;
static inline bool ftrace_graph_is_dead(void)
{
if (static_key_false(&fgraph_dead))
return true;
return false;
}
That way we even get rid of the conditional branch.
Yeah, the fgraph_dead is still exposed for anyone to touch, but it still
requires a function to modify it, so I'm not as worried it will be touched
as easily.
-- Steve
Re: [PATCH] MAINTAINERS: Add arch/powerpc/kernel/trace to TRACING
On Thu, 24 Mar 2022 15:38:28 +0100 Christophe Leroy wrote: > diff --git a/MAINTAINERS b/MAINTAINERS > index e127c2fb08a7..3b5943f34568 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -19592,6 +19592,7 @@ T:git > git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git > F: Documentation/trace/ftrace.rst > F: arch/*/*/*/ftrace.h > F: arch/*/kernel/ftrace.c > +F: arch/powerpc/kernel/trace/ Perhaps replace the above two with: F: arch/*/*/ftrace* F: arch/*/*/*/ftrace* As we also have things like ftrace_32.S, ftrace_64.S Maybe even add: F: arch/*/*/*_ftrace.* F: arch/*/*/*/*_ftrace.* to catch the "entry_ftrace.S" and friends (needing the leading "_" to not get confused by any "bpftrace" files). ? But in reality, I'm only a reviewer for ftrace in the archs. It really is maintained by the arch maintainers. But it is still good that I get Cc'd on any ftrace arch related changes. -- Steve > F: fs/tracefs/ > F: include/*/ftrace.h > F: include/linux/trace*.h
Re: [PATCH 2/2] powerpc/eeh: Use pcie_reset_state_t type in function arguments
On Thu, Mar 10, 2022 at 09:51:13AM +0100, Christophe Leroy wrote:
> Le 13/07/2021 à 02:25, Krzysztof Wilczyński a écrit :
> > The pcie_reset_state_t type has been introduced in the commit
> > f7bdd12d234d ("pci: New PCI-E reset API") along with the enum
> > pcie_reset_state, but it has never been used for anything else
> > other than to define the members of the enumeration set in the
> > enum pcie_reset_state.
> >
> > Thus, replace the direct use of enum pcie_reset_state in function
> > arguments and replace it with pcie_reset_state_t type so that the
> > argument type matches the type used in enum pcie_reset_state.
> >
> > Signed-off-by: Krzysztof Wilczyński
>
> I don't understand the purpose of this change. Does any tool like
> sparse of so reports an error here ?
>
> My feeling is that by doing this you loose the added value of using
> an enumerate.
>
> state is used in a switch/case, that's exactly what we expect from
> an enum.
I think this is true: in the patch below, we remove use of "enum
pcie_reset_state", so the compiler no longer knows that "state" is an
enum, and it cannot verify that "state" has a legal value in the
switch statement. And at least with "gcc -Wall", it looks like it
*does* complain in that case.
Whether that value is worthwhile, I don't know. AFAICT this is the
only place that uses "state", so there's not *much* value.
If we did apply the patch below, I think we could probably make "enum
pcie_reset_state" an anonymous enum instead, like the enum for
pci_channel_state_t.
But let's back up for a minute. This is only used in the
pci_set_pcie_reset_state() path, and that's only used by three
drivers: cxl, genwqe, and ipr, and obviously only on powerpc, since
that's the only arch that implements pcibios_set_pcie_reset_state().
What's special about them? Why do they need this and no other drivers
do? And why only on powerpc?
I wonder if that powerpc functionality could be implemented in some
way that's more integrated into the PCI core reset and error handling
framework.
> > ---
> > arch/powerpc/kernel/eeh.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
> > index 3bbdcc86d01b..15485abb89ff 100644
> > --- a/arch/powerpc/kernel/eeh.c
> > +++ b/arch/powerpc/kernel/eeh.c
> > @@ -714,7 +714,7 @@ static void eeh_restore_dev_state(struct eeh_dev *edev,
> > void *userdata)
> >* Return value:
> >*0 if success
> >*/
> > -int pcibios_set_pcie_reset_state(struct pci_dev *dev, enum
> > pcie_reset_state state)
> > +int pcibios_set_pcie_reset_state(struct pci_dev *dev, pcie_reset_state_t
> > state)
> > {
> > struct eeh_dev *edev = pci_dev_to_eeh_dev(dev);
> > struct eeh_pe *pe = eeh_dev_to_pe(edev);
[PATCH] MAINTAINERS: Add arch/powerpc/kernel/trace to TRACING
Most architectures have ftrace related stuff in arch/*/kernel/ftrace.c but powerpc has spread in multiple files located in arch/powerpc/kernel/trace/ Signed-off-by: Christophe Leroy --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index e127c2fb08a7..3b5943f34568 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -19592,6 +19592,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git F: Documentation/trace/ftrace.rst F: arch/*/*/*/ftrace.h F: arch/*/kernel/ftrace.c +F: arch/powerpc/kernel/trace/ F: fs/tracefs/ F: include/*/ftrace.h F: include/linux/trace*.h -- 2.35.1
[PATCH v1 02/22] powerpc/ftrace: Remove redundant create_branch() calls
Since commit d5937db114e4 ("powerpc/code-patching: Fix patch_branch()
return on out-of-range failure") patch_branch() fails with -ERANGE
when trying to branch out of range.
No need to perform the test twice. Remove redundant create_branch()
calls.
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/trace/ftrace.c | 20
1 file changed, 20 deletions(-)
diff --git a/arch/powerpc/kernel/trace/ftrace.c
b/arch/powerpc/kernel/trace/ftrace.c
index 7a266fd469b7..3ce3697e8a7c 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -301,7 +301,6 @@ static int setup_mcount_compiler_tramp(unsigned long tramp)
int i;
ppc_inst_t op;
unsigned long ptr;
- ppc_inst_t instr;
static unsigned long ftrace_plt_tramps[NUM_FTRACE_TRAMPS];
/* Is this a known long jump tramp? */
@@ -344,12 +343,6 @@ static int setup_mcount_compiler_tramp(unsigned long tramp)
#else
ptr = ppc_global_function_entry((void *)ftrace_caller);
#endif
- if (create_branch(&instr, (void *)tramp, ptr, 0)) {
- pr_debug("%ps is not reachable from existing mcount tramp\n",
- (void *)ptr);
- return -1;
- }
-
if (patch_branch((u32 *)tramp, ptr, 0)) {
pr_debug("REL24 out of range!\n");
return -1;
@@ -490,7 +483,6 @@ static int
__ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
{
ppc_inst_t op[2];
- ppc_inst_t instr;
void *ip = (void *)rec->ip;
unsigned long entry, ptr, tramp;
struct module *mod = rec->arch.mod;
@@ -539,12 +531,6 @@ __ftrace_make_call(struct dyn_ftrace *rec, unsigned long
addr)
return -EINVAL;
}
- /* Ensure branch is within 24 bits */
- if (create_branch(&instr, ip, tramp, BRANCH_SET_LINK)) {
- pr_err("Branch out of range\n");
- return -EINVAL;
- }
-
if (patch_branch(ip, tramp, BRANCH_SET_LINK)) {
pr_err("REL24 out of range!\n");
return -EINVAL;
@@ -770,12 +756,6 @@ __ftrace_modify_call(struct dyn_ftrace *rec, unsigned long
old_addr,
return -EINVAL;
}
- /* Ensure branch is within 24 bits */
- if (create_branch(&op, (u32 *)ip, tramp, BRANCH_SET_LINK)) {
- pr_err("Branch out of range\n");
- return -EINVAL;
- }
-
if (patch_branch((u32 *)ip, tramp, BRANCH_SET_LINK)) {
pr_err("REL24 out of range!\n");
return -EINVAL;
--
2.35.1
[PATCH v1 07/22] powerpc/ftrace: Use patch_instruction() return directly
Instead of returning -EPERM when patch_instruction() fails, just return what patch_instruction returns. That simplifies ftrace_modify_code(): 0: 94 21 ff c0 stwur1,-64(r1) 4: 93 e1 00 3c stw r31,60(r1) 8: 7c 7f 1b 79 mr. r31,r3 c: 40 80 00 30 bge 3c 10: 93 c1 00 38 stw r30,56(r1) 14: 7c 9e 23 78 mr r30,r4 18: 7c a4 2b 78 mr r4,r5 1c: 80 bf 00 00 lwz r5,0(r31) 20: 7c 1e 28 40 cmplw r30,r5 24: 40 82 00 34 bne 58 28: 83 c1 00 38 lwz r30,56(r1) 2c: 7f e3 fb 78 mr r3,r31 30: 83 e1 00 3c lwz r31,60(r1) 34: 38 21 00 40 addir1,r1,64 38: 48 00 00 00 b 38 38: R_PPC_REL24 patch_instruction Before: 0: 94 21 ff c0 stwur1,-64(r1) 4: 93 e1 00 3c stw r31,60(r1) 8: 7c 7f 1b 79 mr. r31,r3 c: 40 80 00 4c bge 58 10: 93 c1 00 38 stw r30,56(r1) 14: 7c 9e 23 78 mr r30,r4 18: 7c a4 2b 78 mr r4,r5 1c: 80 bf 00 00 lwz r5,0(r31) 20: 7c 08 02 a6 mflrr0 24: 90 01 00 44 stw r0,68(r1) 28: 7c 1e 28 40 cmplw r30,r5 2c: 40 82 00 48 bne 74 30: 7f e3 fb 78 mr r3,r31 34: 48 00 00 01 bl 34 34: R_PPC_REL24 patch_instruction 38: 80 01 00 44 lwz r0,68(r1) 3c: 20 63 00 00 subfic r3,r3,0 40: 83 c1 00 38 lwz r30,56(r1) 44: 7c 63 19 10 subfe r3,r3,r3 48: 7c 08 03 a6 mtlrr0 4c: 83 e1 00 3c lwz r31,60(r1) 50: 38 21 00 40 addir1,r1,64 54: 4e 80 00 20 blr It improves ftrace activation/deactivation duration by about 3%. Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/trace/ftrace.c | 5 + 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c index 98e82fa4980f..1b05d33f96c6 100644 --- a/arch/powerpc/kernel/trace/ftrace.c +++ b/arch/powerpc/kernel/trace/ftrace.c @@ -78,10 +78,7 @@ ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new) } /* replace the text with the new text */ - if (patch_instruction((u32 *)ip, new)) - return -EPERM; - - return 0; + return patch_instruction((u32 *)ip, new); } /* -- 2.35.1
[PATCH v1 08/22] powerpc/ftrace: Make __ftrace_make_{nop/call}() common to PPC32 and PPC64
Since c93d4f6ecf4b ("powerpc/ftrace: Add module_trampoline_target()
for PPC32"), __ftrace_make_nop() for PPC32 is very similar to the
one for PPC64.
Same for __ftrace_make_call().
Make them common.
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/trace/ftrace.c | 108 +++--
1 file changed, 8 insertions(+), 100 deletions(-)
diff --git a/arch/powerpc/kernel/trace/ftrace.c
b/arch/powerpc/kernel/trace/ftrace.c
index 1b05d33f96c6..2c7e42e439bb 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -114,7 +114,6 @@ static unsigned long find_bl_target(unsigned long ip,
ppc_inst_t op)
}
#ifdef CONFIG_MODULES
-#ifdef CONFIG_PPC64
static int
__ftrace_make_nop(struct module *mod,
struct dyn_ftrace *rec, unsigned long addr)
@@ -154,10 +153,11 @@ __ftrace_make_nop(struct module *mod,
return -EINVAL;
}
-#ifdef CONFIG_MPROFILE_KERNEL
- /* When using -mkernel_profile there is no load to jump over */
+ /* When using -mkernel_profile or PPC32 there is no load to jump over */
pop = ppc_inst(PPC_RAW_NOP());
+#ifdef CONFIG_PPC64
+#ifdef CONFIG_MPROFILE_KERNEL
if (copy_inst_from_kernel_nofault(&op, (void *)(ip - 4))) {
pr_err("Fetching instruction at %lx failed.\n", ip - 4);
return -EFAULT;
@@ -201,6 +201,7 @@ __ftrace_make_nop(struct module *mod,
return -EINVAL;
}
#endif /* CONFIG_MPROFILE_KERNEL */
+#endif /* PPC64 */
if (patch_instruction((u32 *)ip, pop)) {
pr_err("Patching NOP failed.\n");
@@ -209,48 +210,6 @@ __ftrace_make_nop(struct module *mod,
return 0;
}
-
-#else /* !PPC64 */
-static int
-__ftrace_make_nop(struct module *mod,
- struct dyn_ftrace *rec, unsigned long addr)
-{
- ppc_inst_t op;
- unsigned long ip = rec->ip;
- unsigned long tramp, ptr;
-
- if (copy_from_kernel_nofault(&op, (void *)ip, MCOUNT_INSN_SIZE))
- return -EFAULT;
-
- /* Make sure that that this is still a 24bit jump */
- if (!is_bl_op(op)) {
- pr_err("Not expected bl: opcode is %s\n", ppc_inst_as_str(op));
- return -EINVAL;
- }
-
- /* lets find where the pointer goes */
- tramp = find_bl_target(ip, op);
-
- /* Find where the trampoline jumps to */
- if (module_trampoline_target(mod, tramp, &ptr)) {
- pr_err("Failed to get trampoline target\n");
- return -EFAULT;
- }
-
- if (ptr != addr) {
- pr_err("Trampoline location %08lx does not match addr\n",
- tramp);
- return -EINVAL;
- }
-
- op = ppc_inst(PPC_RAW_NOP());
-
- if (patch_instruction((u32 *)ip, op))
- return -EPERM;
-
- return 0;
-}
-#endif /* PPC64 */
#endif /* CONFIG_MODULES */
static unsigned long find_ftrace_tramp(unsigned long ip)
@@ -437,13 +396,12 @@ int ftrace_make_nop(struct module *mod,
}
#ifdef CONFIG_MODULES
-#ifdef CONFIG_PPC64
/*
* Examine the existing instructions for __ftrace_make_call.
* They should effectively be a NOP, and follow formal constraints,
* depending on the ABI. Return false if they don't.
*/
-#ifndef CONFIG_MPROFILE_KERNEL
+#ifndef CONFIG_DYNAMIC_FTRACE_WITH_REGS
static int
expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
{
@@ -465,7 +423,7 @@ expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t
op1)
static int
expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
{
- /* look for patched "NOP" on ppc64 with -mprofile-kernel */
+ /* look for patched "NOP" on ppc64 with -mprofile-kernel or ppc32 */
if (!ppc_inst_equal(op0, ppc_inst(PPC_RAW_NOP(
return 0;
return 1;
@@ -484,8 +442,10 @@ __ftrace_make_call(struct dyn_ftrace *rec, unsigned long
addr)
if (copy_inst_from_kernel_nofault(op, ip))
return -EFAULT;
+#ifndef CONFIG_DYNAMIC_FTRACE_WITH_REGS
if (copy_inst_from_kernel_nofault(op + 1, ip + 4))
return -EFAULT;
+#endif
if (!expected_nop_sequence(ip, op[0], op[1])) {
pr_err("Unexpected call sequence at %p: %s %s\n",
@@ -531,58 +491,6 @@ __ftrace_make_call(struct dyn_ftrace *rec, unsigned long
addr)
return 0;
}
-
-#else /* !CONFIG_PPC64: */
-static int
-__ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
-{
- int err;
- ppc_inst_t op;
- u32 *ip = (u32 *)rec->ip;
- struct module *mod = rec->arch.mod;
- unsigned long tramp;
-
- /* read where this goes */
- if (copy_inst_from_kernel_nofault(&op, ip))
- return -EFAULT;
-
- /* It should be pointing to a nop */
- if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_NOP( {
- pr_err("Expected NOP but have %s\n", ppc_inst_as_str(op));
- return -EINVA
[PATCH v1 19/22] powerpc/inst: Remove PPC_INST_BRANCH
Convert last users of PPC_INST_BRANCH to PPC_RAW_BRANCH()
And remove PPC_INST_BRANCH.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/ppc-opcode.h | 3 +--
arch/powerpc/lib/feature-fixups.c | 2 +-
2 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/include/asm/ppc-opcode.h
b/arch/powerpc/include/asm/ppc-opcode.h
index 281754aca0a3..ada8fe17b199 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -290,7 +290,6 @@
#define PPC_INST_ADDIS 0x3c00
#define PPC_INST_ADD 0x7c000214
#define PPC_INST_DIVD 0x7c0003d2
-#define PPC_INST_BRANCH0x4800
#define PPC_INST_BL0x4801
#define PPC_INST_BRANCH_COND 0x4080
@@ -573,7 +572,7 @@
#define PPC_RAW_MTSPR(spr, d) (0x7c0003a6 | ___PPC_RS(d) |
__PPC_SPR(spr))
#define PPC_RAW_EIEIO()(0x7c0006ac)
-#define PPC_RAW_BRANCH(addr) (PPC_INST_BRANCH | ((addr) &
0x03fc))
+#define PPC_RAW_BRANCH(offset) (0x4800 | ((offset) &
PPC_INST_OFFSET24_MASK))
#define PPC_RAW_BL(offset) (0x4801 | ((offset) &
PPC_INST_OFFSET24_MASK))
/* Deal with instructions that older assemblers aren't aware of */
diff --git a/arch/powerpc/lib/feature-fixups.c
b/arch/powerpc/lib/feature-fixups.c
index 343a78826035..993d3f31832a 100644
--- a/arch/powerpc/lib/feature-fixups.c
+++ b/arch/powerpc/lib/feature-fixups.c
@@ -451,7 +451,7 @@ static int __do_rfi_flush_fixups(void *data)
if (types & L1D_FLUSH_FALLBACK)
/* b .+16 to fallback flush */
- instrs[0] = PPC_INST_BRANCH | 16;
+ instrs[0] = PPC_RAW_BRANCH(16);
i = 0;
if (types & L1D_FLUSH_ORI) {
--
2.35.1
[PATCH v1 00/22] powerpc: ftrace optimisation and cleanup and more [v1]
This series provides optimisation and cleanup of ftrace on powerpc.
With this series ftrace activation is about 20% faster on an 8xx.
At the end of the series come additional cleanups around ppc-opcode,
that would likely conflict with this series if posted separately.
Christophe Leroy (22):
powerpc/ftrace: Refactor prepare_ftrace_return()
powerpc/ftrace: Remove redundant create_branch() calls
powerpc/code-patching: Inline is_offset_in_{cond}_branch_range()
powerpc/ftrace: Use is_offset_in_branch_range()
powerpc/code-patching: Inline create_branch()
powerpc/ftrace: Inline ftrace_modify_code()
powerpc/ftrace: Use patch_instruction() return directly
powerpc/ftrace: Make __ftrace_make_{nop/call}() common to PPC32 and
PPC64
powerpc/ftrace: Don't include ftrace.o for CONFIG_FTRACE_SYSCALLS
powerpc/ftrace: Use CONFIG_FUNCTION_TRACER instead of
CONFIG_DYNAMIC_FTRACE
powerpc/ftrace: Remove ftrace_plt_tramps[]
powerpc/ftrace: Use BRANCH_SET_LINK instead of value 1
powerpc/ftrace: Use PPC_RAW_xxx() macros instead of opencoding.
powerpc/ftrace: Use size macro instead of opencoding
powerpc/ftrace: Simplify expected_nop_sequence()
powerpc/ftrace: Minimise number of #ifdefs
powerpc/inst: Add __copy_inst_from_kernel_nofault()
powerpc/ftrace: Don't use copy_from_kernel_nofault() in
module_trampoline_target()
powerpc/inst: Remove PPC_INST_BRANCH
powerpc/modules: Use PPC_INST_BRANCH_MASK instead of opencoding
powerpc/inst: Remove PPC_INST_BL
powerpc/opcodes: Remove unused PPC_INST_XXX macros
arch/powerpc/include/asm/book3s/32/pgtable.h | 2 +-
arch/powerpc/include/asm/book3s/64/pgtable.h | 2 +-
arch/powerpc/include/asm/code-patching.h | 53 ++-
arch/powerpc/include/asm/inst.h | 13 +-
arch/powerpc/include/asm/module.h| 6 +-
arch/powerpc/include/asm/nohash/pgtable.h| 2 +-
arch/powerpc/include/asm/ppc-opcode.h| 20 +-
arch/powerpc/include/asm/sections.h | 24 +-
arch/powerpc/kernel/module_32.c | 29 +-
arch/powerpc/kernel/module_64.c | 10 +-
arch/powerpc/kernel/trace/Makefile | 5 +-
arch/powerpc/kernel/trace/ftrace.c | 390 ++-
arch/powerpc/lib/code-patching.c | 47 ---
arch/powerpc/lib/feature-fixups.c| 2 +-
arch/powerpc/net/bpf_jit.h | 2 +-
15 files changed, 229 insertions(+), 378 deletions(-)
--
2.35.1
[PATCH v1 20/22] powerpc/modules: Use PPC_INST_BRANCH_MASK instead of opencoding
Use PPC_INST_BRANCH_MASK instead of opencoding.
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/module_32.c | 13 ++---
arch/powerpc/kernel/module_64.c | 4 ++--
2 files changed, 8 insertions(+), 9 deletions(-)
diff --git a/arch/powerpc/kernel/module_32.c b/arch/powerpc/kernel/module_32.c
index 1282cfd672f2..344941855e9e 100644
--- a/arch/powerpc/kernel/module_32.c
+++ b/arch/powerpc/kernel/module_32.c
@@ -256,9 +256,8 @@ int apply_relocate_add(Elf32_Shdr *sechdrs,
value, (uint32_t)location);
pr_debug("Location before: %08X.\n",
*(uint32_t *)location);
- value = (*(uint32_t *)location & ~0x03fc)
- | ((value - (uint32_t)location)
- & 0x03fc);
+ value = (*(uint32_t *)location &
~PPC_INST_OFFSET24_MASK) |
+ ((value - (uint32_t)location) &
PPC_INST_OFFSET24_MASK);
if (patch_instruction(location, ppc_inst(value)))
return -EFAULT;
@@ -266,10 +265,10 @@ int apply_relocate_add(Elf32_Shdr *sechdrs,
pr_debug("Location after: %08X.\n",
*(uint32_t *)location);
pr_debug("ie. jump to %08X+%08X = %08X\n",
- *(uint32_t *)location & 0x03fc,
- (uint32_t)location,
- (*(uint32_t *)location & 0x03fc)
- + (uint32_t)location);
+*(uint32_t *)location & PPC_INST_OFFSET24_MASK,
+(uint32_t)location,
+(*(uint32_t *)location &
PPC_INST_OFFSET24_MASK) +
+(uint32_t)location);
break;
case R_PPC_REL32:
diff --git a/arch/powerpc/kernel/module_64.c b/arch/powerpc/kernel/module_64.c
index b13a72665eee..8f70a04aac6c 100644
--- a/arch/powerpc/kernel/module_64.c
+++ b/arch/powerpc/kernel/module_64.c
@@ -653,8 +653,8 @@ int apply_relocate_add(Elf64_Shdr *sechdrs,
}
/* Only replace bits 2 through 26 */
- value = (*(uint32_t *)location & ~0x03fc)
- | (value & 0x03fc);
+ value = (*(uint32_t *)location &
~PPC_INST_OFFSET24_MASK) |
+ (value & PPC_INST_OFFSET24_MASK);
if (patch_instruction((u32 *)location, ppc_inst(value)))
return -EFAULT;
--
2.35.1
[PATCH v1 15/22] powerpc/ftrace: Simplify expected_nop_sequence()
Avoid ifdefs around expected_nop_sequence().
While at it make it a bool.
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/trace/ftrace.c | 22 ++
1 file changed, 6 insertions(+), 16 deletions(-)
diff --git a/arch/powerpc/kernel/trace/ftrace.c
b/arch/powerpc/kernel/trace/ftrace.c
index 15e0746806f8..ffedf8c82ea8 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -390,24 +390,14 @@ int ftrace_make_nop(struct module *mod,
* They should effectively be a NOP, and follow formal constraints,
* depending on the ABI. Return false if they don't.
*/
-#ifndef CONFIG_DYNAMIC_FTRACE_WITH_REGS
-static int
-expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
-{
- if (!ppc_inst_equal(op0, ppc_inst(PPC_RAW_BRANCH(8))) ||
- !ppc_inst_equal(op1, ppc_inst(PPC_INST_LD_TOC)))
- return 0;
- return 1;
-}
-#else
-static int
-expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
+static bool expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
{
- if (!ppc_inst_equal(op0, ppc_inst(PPC_RAW_NOP(
- return 0;
- return 1;
+ if (IS_ENABLED(CONFIG_DYNAMIC_FTRACE_WITH_REGS))
+ return ppc_inst_equal(op0, ppc_inst(PPC_RAW_NOP()));
+ else
+ return ppc_inst_equal(op0, ppc_inst(PPC_RAW_BRANCH(8))) &&
+ ppc_inst_equal(op1, ppc_inst(PPC_INST_LD_TOC));
}
-#endif
static int
__ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr)
--
2.35.1
[PATCH v1 05/22] powerpc/code-patching: Inline create_branch()
create_branch() is a good candidate for inlining because:
- Flags can be folded in.
- Range tests are likely to be already done.
Hence reducing the create_branch() to only a set of instructions.
So inline it.
It improves ftrace activation by 10%.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/code-patching.h | 22 --
arch/powerpc/lib/code-patching.c | 20
2 files changed, 20 insertions(+), 22 deletions(-)
diff --git a/arch/powerpc/include/asm/code-patching.h
b/arch/powerpc/include/asm/code-patching.h
index e7c5df50cb4e..4260e89f62b1 100644
--- a/arch/powerpc/include/asm/code-patching.h
+++ b/arch/powerpc/include/asm/code-patching.h
@@ -49,8 +49,26 @@ static inline bool is_offset_in_cond_branch_range(long
offset)
return offset >= -0x8000 && offset <= 0x7fff && !(offset & 0x3);
}
-int create_branch(ppc_inst_t *instr, const u32 *addr,
- unsigned long target, int flags);
+static inline int create_branch(ppc_inst_t *instr, const u32 *addr,
+ unsigned long target, int flags)
+{
+ long offset;
+
+ *instr = ppc_inst(0);
+ offset = target;
+ if (! (flags & BRANCH_ABSOLUTE))
+ offset = offset - (unsigned long)addr;
+
+ /* Check we can represent the target in the instruction format */
+ if (!is_offset_in_branch_range(offset))
+ return 1;
+
+ /* Mask out the flags and target, so they don't step on each other. */
+ *instr = ppc_inst(0x4800 | (flags & 0x3) | (offset & 0x03FC));
+
+ return 0;
+}
+
int create_cond_branch(ppc_inst_t *instr, const u32 *addr,
unsigned long target, int flags);
int patch_branch(u32 *addr, unsigned long target, int flags);
diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c
index 58262c7e447c..7adbdb05fee7 100644
--- a/arch/powerpc/lib/code-patching.c
+++ b/arch/powerpc/lib/code-patching.c
@@ -230,26 +230,6 @@ bool is_conditional_branch(ppc_inst_t instr)
}
NOKPROBE_SYMBOL(is_conditional_branch);
-int create_branch(ppc_inst_t *instr, const u32 *addr,
- unsigned long target, int flags)
-{
- long offset;
-
- *instr = ppc_inst(0);
- offset = target;
- if (! (flags & BRANCH_ABSOLUTE))
- offset = offset - (unsigned long)addr;
-
- /* Check we can represent the target in the instruction format */
- if (!is_offset_in_branch_range(offset))
- return 1;
-
- /* Mask out the flags and target, so they don't step on each other. */
- *instr = ppc_inst(0x4800 | (flags & 0x3) | (offset & 0x03FC));
-
- return 0;
-}
-
int create_cond_branch(ppc_inst_t *instr, const u32 *addr,
unsigned long target, int flags)
{
--
2.35.1
[PATCH v1 04/22] powerpc/ftrace: Use is_offset_in_branch_range()
Use is_offset_in_branch_range() instead of create_branch()
to check if a target is within branch range.
This patch together with the previous one improves
ftrace activation time by 7%
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/trace/ftrace.c | 8 ++--
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/kernel/trace/ftrace.c
b/arch/powerpc/kernel/trace/ftrace.c
index 3ce3697e8a7c..41c45b9c7f39 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -89,11 +89,9 @@ ftrace_modify_code(unsigned long ip, ppc_inst_t old,
ppc_inst_t new)
*/
static int test_24bit_addr(unsigned long ip, unsigned long addr)
{
- ppc_inst_t op;
addr = ppc_function_entry((void *)addr);
- /* use the create_branch to verify that this offset can be branched */
- return create_branch(&op, (u32 *)ip, addr, 0) == 0;
+ return is_offset_in_branch_range(addr - ip);
}
static int is_bl_op(ppc_inst_t op)
@@ -261,7 +259,6 @@ __ftrace_make_nop(struct module *mod,
static unsigned long find_ftrace_tramp(unsigned long ip)
{
int i;
- ppc_inst_t instr;
/*
* We have the compiler generated long_branch tramps at the end
@@ -270,8 +267,7 @@ static unsigned long find_ftrace_tramp(unsigned long ip)
for (i = NUM_FTRACE_TRAMPS - 1; i >= 0; i--)
if (!ftrace_tramps[i])
continue;
- else if (create_branch(&instr, (void *)ip,
- ftrace_tramps[i], 0) == 0)
+ else if (is_offset_in_branch_range(ftrace_tramps[i] - ip))
return ftrace_tramps[i];
return 0;
--
2.35.1
[PATCH v1 22/22] powerpc/opcodes: Remove unused PPC_INST_XXX macros
The following PPC_INST_XXX macros are not used anymore outside ppc-opcode.h: - PPC_INST_LD - PPC_INST_STD - PPC_INST_ADDIS - PPC_INST_ADD - PPC_INST_DIVD Remove them. Signed-off-by: Christophe Leroy --- arch/powerpc/include/asm/ppc-opcode.h | 13 - 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h index 810a28af9dce..4cdda507e816 100644 --- a/arch/powerpc/include/asm/ppc-opcode.h +++ b/arch/powerpc/include/asm/ppc-opcode.h @@ -285,11 +285,6 @@ #define PPC_INST_TRECHKPT 0x7c0007dd #define PPC_INST_TRECLAIM 0x7c00075d #define PPC_INST_TSR 0x7c0005dd -#define PPC_INST_LD0xe800 -#define PPC_INST_STD 0xf800 -#define PPC_INST_ADDIS 0x3c00 -#define PPC_INST_ADD 0x7c000214 -#define PPC_INST_DIVD 0x7c0003d2 #define PPC_INST_BRANCH_COND 0x4080 #define PPC_INST_OFFSET24_MASK 0x03fc @@ -460,10 +455,10 @@ (0x10c7 | ___PPC_RT(vrt) | ___PPC_RA(vra) | ___PPC_RB(vrb) | __PPC_RC21) #define PPC_RAW_VCMPEQUB_RC(vrt, vra, vrb) \ (0x1006 | ___PPC_RT(vrt) | ___PPC_RA(vra) | ___PPC_RB(vrb) | __PPC_RC21) -#define PPC_RAW_LD(r, base, i) (PPC_INST_LD | ___PPC_RT(r) | ___PPC_RA(base) | IMM_DS(i)) +#define PPC_RAW_LD(r, base, i) (0xe800 | ___PPC_RT(r) | ___PPC_RA(base) | IMM_DS(i)) #define PPC_RAW_LWZ(r, base, i)(0x8000 | ___PPC_RT(r) | ___PPC_RA(base) | IMM_L(i)) #define PPC_RAW_LWZX(t, a, b) (0x7c2e | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b)) -#define PPC_RAW_STD(r, base, i)(PPC_INST_STD | ___PPC_RS(r) | ___PPC_RA(base) | IMM_DS(i)) +#define PPC_RAW_STD(r, base, i)(0xf800 | ___PPC_RS(r) | ___PPC_RA(base) | IMM_DS(i)) #define PPC_RAW_STDCX(s, a, b) (0x7c0001ad | ___PPC_RS(s) | ___PPC_RA(a) | ___PPC_RB(b)) #define PPC_RAW_LFSX(t, a, b) (0x7c00042e | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b)) #define PPC_RAW_STFSX(s, a, b) (0x7c00052e | ___PPC_RS(s) | ___PPC_RA(a) | ___PPC_RB(b)) @@ -474,8 +469,8 @@ #define PPC_RAW_ADDE(t, a, b) (0x7c000114 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b)) #define PPC_RAW_ADDZE(t, a)(0x7c000194 | ___PPC_RT(t) | ___PPC_RA(a)) #define PPC_RAW_ADDME(t, a)(0x7c0001d4 | ___PPC_RT(t) | ___PPC_RA(a)) -#define PPC_RAW_ADD(t, a, b) (PPC_INST_ADD | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b)) -#define PPC_RAW_ADD_DOT(t, a, b) (PPC_INST_ADD | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | 0x1) +#define PPC_RAW_ADD(t, a, b) (0x7c000214 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b)) +#define PPC_RAW_ADD_DOT(t, a, b) (0x7c000214 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | 0x1) #define PPC_RAW_ADDC(t, a, b) (0x7c14 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b)) #define PPC_RAW_ADDC_DOT(t, a, b) (0x7c14 | ___PPC_RT(t) | ___PPC_RA(a) | ___PPC_RB(b) | 0x1) #define PPC_RAW_NOP() PPC_RAW_ORI(0, 0, 0) -- 2.35.1
[PATCH v1 17/22] powerpc/inst: Add __copy_inst_from_kernel_nofault()
On the same model as get_user() versus __get_user(),
introduce __copy_inst_from_kernel_nofault() which doesn't
check address.
To be used by callers that have already checked that the adress
is a kernel address.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/inst.h | 13 +
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/inst.h b/arch/powerpc/include/asm/inst.h
index 80b6d74146c6..b49aae9f6f27 100644
--- a/arch/powerpc/include/asm/inst.h
+++ b/arch/powerpc/include/asm/inst.h
@@ -158,13 +158,10 @@ static inline char *__ppc_inst_as_str(char
str[PPC_INST_STR_LEN], ppc_inst_t x)
__str; \
})
-static inline int copy_inst_from_kernel_nofault(ppc_inst_t *inst, u32 *src)
+static inline int __copy_inst_from_kernel_nofault(ppc_inst_t *inst, u32 *src)
{
unsigned int val, suffix;
- if (unlikely(!is_kernel_addr((unsigned long)src)))
- return -ERANGE;
-
/* See https://github.com/ClangBuiltLinux/linux/issues/1521 */
#if defined(CONFIG_CC_IS_CLANG) && CONFIG_CLANG_VERSION < 14
val = suffix = 0;
@@ -181,4 +178,12 @@ static inline int copy_inst_from_kernel_nofault(ppc_inst_t
*inst, u32 *src)
return -EFAULT;
}
+static inline int copy_inst_from_kernel_nofault(ppc_inst_t *inst, u32 *src)
+{
+ if (unlikely(!is_kernel_addr((unsigned long)src)))
+ return -ERANGE;
+
+ return __copy_inst_from_kernel_nofault(inst, src);
+}
+
#endif /* _ASM_POWERPC_INST_H */
--
2.35.1
[PATCH v1 12/22] powerpc/ftrace: Use BRANCH_SET_LINK instead of value 1
To make it explicit, use BRANCH_SET_LINK instead of value 1 when calling create_branch(). Signed-off-by: Christophe Leroy --- arch/powerpc/kernel/trace/ftrace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/kernel/trace/ftrace.c b/arch/powerpc/kernel/trace/ftrace.c index b6c5223e26b2..fdc0412c1d8a 100644 --- a/arch/powerpc/kernel/trace/ftrace.c +++ b/arch/powerpc/kernel/trace/ftrace.c @@ -45,7 +45,7 @@ ftrace_call_replace(unsigned long ip, unsigned long addr, int link) addr = ppc_function_entry((void *)addr); /* if (link) set op to 'bl' else 'b' */ - create_branch(&op, (u32 *)ip, addr, link ? 1 : 0); + create_branch(&op, (u32 *)ip, addr, link ? BRANCH_SET_LINK : 0); return op; } -- 2.35.1
[PATCH v1 06/22] powerpc/ftrace: Inline ftrace_modify_code()
Inlining ftrace_modify_code(), it increases a bit the
size of ftrace code but brings 5% improvment on ftrace
activation.
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/trace/ftrace.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/trace/ftrace.c
b/arch/powerpc/kernel/trace/ftrace.c
index 41c45b9c7f39..98e82fa4980f 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -53,7 +53,7 @@ ftrace_call_replace(unsigned long ip, unsigned long addr, int
link)
return op;
}
-static int
+static inline int
ftrace_modify_code(unsigned long ip, ppc_inst_t old, ppc_inst_t new)
{
ppc_inst_t replaced;
--
2.35.1
[PATCH v1 01/22] powerpc/ftrace: Refactor prepare_ftrace_return()
When we have CONFIG_DYNAMIC_FTRACE_WITH_ARGS,
prepare_ftrace_return() is called by ftrace_graph_func()
otherwise prepare_ftrace_return() is called from assembly.
Refactor prepare_ftrace_return() into a static
__prepare_ftrace_return() that will be called by both
prepare_ftrace_return() and ftrace_graph_func().
It will allow GCC to fold __prepare_ftrace_return() inside
ftrace_graph_func().
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/trace/ftrace.c | 12 +---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/kernel/trace/ftrace.c
b/arch/powerpc/kernel/trace/ftrace.c
index 4ee04aacf9f1..7a266fd469b7 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -939,8 +939,8 @@ int ftrace_disable_ftrace_graph_caller(void)
* Hook the return address and push it in the stack of return addrs
* in current thread info. Return the address we want to divert to.
*/
-unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
- unsigned long sp)
+static unsigned long
+__prepare_ftrace_return(unsigned long parent, unsigned long ip, unsigned long
sp)
{
unsigned long return_hooker;
int bit;
@@ -969,7 +969,13 @@ unsigned long prepare_ftrace_return(unsigned long parent,
unsigned long ip,
void ftrace_graph_func(unsigned long ip, unsigned long parent_ip,
struct ftrace_ops *op, struct ftrace_regs *fregs)
{
- fregs->regs.link = prepare_ftrace_return(parent_ip, ip,
fregs->regs.gpr[1]);
+ fregs->regs.link = __prepare_ftrace_return(parent_ip, ip,
fregs->regs.gpr[1]);
+}
+#else
+unsigned long prepare_ftrace_return(unsigned long parent, unsigned long ip,
+ unsigned long sp)
+{
+ return __prepare_ftrace_return(parent, ip, sp);
}
#endif
#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
--
2.35.1
[PATCH v1 21/22] powerpc/inst: Remove PPC_INST_BL
Convert last users of PPC_INST_BL to PPC_RAW_BL() And remove PPC_INST_BL. Signed-off-by: Christophe Leroy --- arch/powerpc/include/asm/ppc-opcode.h | 1 - arch/powerpc/net/bpf_jit.h| 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h index ada8fe17b199..810a28af9dce 100644 --- a/arch/powerpc/include/asm/ppc-opcode.h +++ b/arch/powerpc/include/asm/ppc-opcode.h @@ -290,7 +290,6 @@ #define PPC_INST_ADDIS 0x3c00 #define PPC_INST_ADD 0x7c000214 #define PPC_INST_DIVD 0x7c0003d2 -#define PPC_INST_BL0x4801 #define PPC_INST_BRANCH_COND 0x4080 #define PPC_INST_OFFSET24_MASK 0x03fc diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h index 979701d360da..c63324ff9f04 100644 --- a/arch/powerpc/net/bpf_jit.h +++ b/arch/powerpc/net/bpf_jit.h @@ -35,7 +35,7 @@ } while (0) /* bl (unconditional 'branch' with link) */ -#define PPC_BL(dest) EMIT(PPC_INST_BL | (((dest) - (unsigned long)(image + ctx->idx)) & 0x03fc)) +#define PPC_BL(dest) EMIT(PPC_RAW_BL((dest) - (unsigned long)(image + ctx->idx))) /* "cond" here covers BO:BI fields. */ #define PPC_BCC_SHORT(cond, dest)\ -- 2.35.1
[PATCH v1 13/22] powerpc/ftrace: Use PPC_RAW_xxx() macros instead of opencoding.
PPC_RAW_xxx() macros are self explanatory and less error prone
than open coding.
Use them in ftrace.c
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/ppc-opcode.h | 3 +++
arch/powerpc/kernel/trace/ftrace.c| 32 +--
2 files changed, 14 insertions(+), 21 deletions(-)
diff --git a/arch/powerpc/include/asm/ppc-opcode.h
b/arch/powerpc/include/asm/ppc-opcode.h
index 82f1f0041c6f..281754aca0a3 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -294,6 +294,8 @@
#define PPC_INST_BL0x4801
#define PPC_INST_BRANCH_COND 0x4080
+#define PPC_INST_OFFSET24_MASK 0x03fc
+
/* Prefixes */
#define PPC_INST_LFS 0xc000
#define PPC_INST_STFS 0xd000
@@ -572,6 +574,7 @@
#define PPC_RAW_EIEIO()(0x7c0006ac)
#define PPC_RAW_BRANCH(addr) (PPC_INST_BRANCH | ((addr) &
0x03fc))
+#define PPC_RAW_BL(offset) (0x4801 | ((offset) &
PPC_INST_OFFSET24_MASK))
/* Deal with instructions that older assemblers aren't aware of */
#definePPC_BCCTR_FLUSH stringify_in_c(.long
PPC_INST_BCCTR_FLUSH)
diff --git a/arch/powerpc/kernel/trace/ftrace.c
b/arch/powerpc/kernel/trace/ftrace.c
index fdc0412c1d8a..afb1d12838c9 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -90,19 +90,19 @@ static int test_24bit_addr(unsigned long ip, unsigned long
addr)
static int is_bl_op(ppc_inst_t op)
{
- return (ppc_inst_val(op) & 0xfc03) == 0x4801;
+ return (ppc_inst_val(op) & ~PPC_INST_OFFSET24_MASK) == PPC_RAW_BL(0);
}
static int is_b_op(ppc_inst_t op)
{
- return (ppc_inst_val(op) & 0xfc03) == 0x4800;
+ return (ppc_inst_val(op) & ~PPC_INST_OFFSET24_MASK) ==
PPC_RAW_BRANCH(0);
}
static unsigned long find_bl_target(unsigned long ip, ppc_inst_t op)
{
int offset;
- offset = (ppc_inst_val(op) & 0x03fc);
+ offset = (ppc_inst_val(op) & PPC_INST_OFFSET24_MASK);
/* make it signed */
if (offset & 0x0200)
offset |= 0xfe00;
@@ -182,7 +182,7 @@ __ftrace_make_nop(struct module *mod,
* Use a b +8 to jump over the load.
*/
- pop = ppc_inst(PPC_INST_BRANCH | 8);/* b +8 */
+ pop = ppc_inst(PPC_RAW_BRANCH(8)); /* b +8 */
/*
* Check what is in the next instruction. We can see ld r2,40(r1), but
@@ -394,17 +394,8 @@ int ftrace_make_nop(struct module *mod,
static int
expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
{
- /*
-* We expect to see:
-*
-* b +8
-* ld r2,XX(r1)
-*
-* The load offset is different depending on the ABI. For simplicity
-* just mask it out when doing the compare.
-*/
- if (!ppc_inst_equal(op0, ppc_inst(0x4808)) ||
- (ppc_inst_val(op1) & 0x) != 0xe841)
+ if (!ppc_inst_equal(op0, ppc_inst(PPC_RAW_BRANCH(8))) ||
+ !ppc_inst_equal(op1, ppc_inst(PPC_INST_LD_TOC)))
return 0;
return 1;
}
@@ -412,7 +403,6 @@ expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t
op1)
static int
expected_nop_sequence(void *ip, ppc_inst_t op0, ppc_inst_t op1)
{
- /* look for patched "NOP" on ppc64 with -mprofile-kernel or ppc32 */
if (!ppc_inst_equal(op0, ppc_inst(PPC_RAW_NOP(
return 0;
return 1;
@@ -738,11 +728,11 @@ int __init ftrace_dyn_arch_init(void)
int i;
unsigned int *tramp[] = { ftrace_tramp_text, ftrace_tramp_init };
u32 stub_insns[] = {
- 0xe98d | PACATOC, /* ld r12,PACATOC(r13) */
- 0x3d8c, /* addis r12,r12, */
- 0x398c, /* addir12,r12,*/
- 0x7d8903a6, /* mtctr r12 */
- 0x4e800420, /* bctr */
+ PPC_RAW_LD(_R12, _R13, PACATOC),
+ PPC_RAW_ADDIS(_R12, _R12, 0),
+ PPC_RAW_ADDIS(_R12, _R12, 0),
+ PPC_RAW_MTCTR(_R12),
+ PPC_RAW_BCTR()
};
#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
unsigned long addr = ppc_global_function_entry((void
*)ftrace_regs_caller);
--
2.35.1
[PATCH v1 16/22] powerpc/ftrace: Minimise number of #ifdefs
A lot of #ifdefs can be replaced by IS_ENABLED()
Do so.
This requires to have kernel_toc_addr() defined at all time
and PPC_INST_LD_TOC as well.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/code-patching.h | 2 -
arch/powerpc/include/asm/module.h| 2 -
arch/powerpc/include/asm/sections.h | 24 +--
arch/powerpc/kernel/trace/ftrace.c | 201 ---
4 files changed, 113 insertions(+), 116 deletions(-)
diff --git a/arch/powerpc/include/asm/code-patching.h
b/arch/powerpc/include/asm/code-patching.h
index 4260e89f62b1..071fcbec31c5 100644
--- a/arch/powerpc/include/asm/code-patching.h
+++ b/arch/powerpc/include/asm/code-patching.h
@@ -217,7 +217,6 @@ static inline unsigned long ppc_kallsyms_lookup_name(const
char *name)
return addr;
}
-#ifdef CONFIG_PPC64
/*
* Some instruction encodings commonly used in dynamic ftracing
* and function live patching.
@@ -234,6 +233,5 @@ static inline unsigned long ppc_kallsyms_lookup_name(const
char *name)
/* usually preceded by a mflr r0 */
#define PPC_INST_STD_LRPPC_RAW_STD(_R0, _R1, PPC_LR_STKOFF)
-#endif /* CONFIG_PPC64 */
#endif /* _ASM_POWERPC_CODE_PATCHING_H */
diff --git a/arch/powerpc/include/asm/module.h
b/arch/powerpc/include/asm/module.h
index e6f5963fd96e..700d7ecd9012 100644
--- a/arch/powerpc/include/asm/module.h
+++ b/arch/powerpc/include/asm/module.h
@@ -41,9 +41,7 @@ struct mod_arch_specific {
#ifdef CONFIG_FUNCTION_TRACER
unsigned long tramp;
-#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
unsigned long tramp_regs;
-#endif
#endif
/* List of BUG addresses, source line numbers and filenames */
diff --git a/arch/powerpc/include/asm/sections.h
b/arch/powerpc/include/asm/sections.h
index 8be2c491c733..6980eaeb16fe 100644
--- a/arch/powerpc/include/asm/sections.h
+++ b/arch/powerpc/include/asm/sections.h
@@ -29,18 +29,6 @@ extern char start_virt_trampolines[];
extern char end_virt_trampolines[];
#endif
-/*
- * This assumes the kernel is never compiled -mcmodel=small or
- * the total .toc is always less than 64k.
- */
-static inline unsigned long kernel_toc_addr(void)
-{
- unsigned long toc_ptr;
-
- asm volatile("mr %0, 2" : "=r" (toc_ptr));
- return toc_ptr;
-}
-
static inline int overlaps_interrupt_vector_text(unsigned long start,
unsigned long end)
{
@@ -60,5 +48,17 @@ static inline int overlaps_kernel_text(unsigned long start,
unsigned long end)
#endif
+/*
+ * This assumes the kernel is never compiled -mcmodel=small or
+ * the total .toc is always less than 64k.
+ */
+static inline unsigned long kernel_toc_addr(void)
+{
+ unsigned long toc_ptr;
+
+ asm volatile("mr %0, 2" : "=r" (toc_ptr));
+ return toc_ptr;
+}
+
#endif /* __KERNEL__ */
#endif /* _ASM_POWERPC_SECTIONS_H */
diff --git a/arch/powerpc/kernel/trace/ftrace.c
b/arch/powerpc/kernel/trace/ftrace.c
index ffedf8c82ea8..4dd30e396026 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -150,55 +150,55 @@ __ftrace_make_nop(struct module *mod,
return -EINVAL;
}
- /* When using -mkernel_profile or PPC32 there is no load to jump over */
- pop = ppc_inst(PPC_RAW_NOP());
+ if (IS_ENABLED(CONFIG_MPROFILE_KERNEL)) {
+ /* When using -mkernel_profile or PPC32 there is no load to
jump over */
+ pop = ppc_inst(PPC_RAW_NOP());
-#ifdef CONFIG_PPC64
-#ifdef CONFIG_MPROFILE_KERNEL
- if (copy_inst_from_kernel_nofault(&op, (void *)(ip - 4))) {
- pr_err("Fetching instruction at %lx failed.\n", ip - 4);
- return -EFAULT;
- }
+ if (copy_inst_from_kernel_nofault(&op, (void *)(ip - 4))) {
+ pr_err("Fetching instruction at %lx failed.\n", ip - 4);
+ return -EFAULT;
+ }
- /* We expect either a mflr r0, or a std r0, LRSAVE(r1) */
- if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_MFLR(_R0))) &&
- !ppc_inst_equal(op, ppc_inst(PPC_INST_STD_LR))) {
- pr_err("Unexpected instruction %s around bl _mcount\n",
- ppc_inst_as_str(op));
- return -EINVAL;
- }
-#else
- /*
-* Our original call site looks like:
-*
-* bl
-* ld r2,XX(r1)
-*
-* Milton Miller pointed out that we can not simply nop the branch.
-* If a task was preempted when calling a trace function, the nops
-* will remove the way to restore the TOC in r2 and the r2 TOC will
-* get corrupted.
-*
-* Use a b +8 to jump over the load.
-*/
+ /* We expect either a mflr r0, or a std r0, LRSAVE(r1) */
+ if (!ppc_inst_equal(op, ppc_inst(PPC_RAW_MFLR(_R0))) &&
+ !ppc_inst_equal(op, ppc_inst(PPC_INST_STD_LR))) {
+
[PATCH v1 09/22] powerpc/ftrace: Don't include ftrace.o for CONFIG_FTRACE_SYSCALLS
Since commit 7bea7ac0ca01 ("powerpc/syscalls: Fix syscall tracing")
ftrace.o is not needed anymore for CONFIG_FTRACE_SYSCALLS.
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/trace/Makefile | 1 -
1 file changed, 1 deletion(-)
diff --git a/arch/powerpc/kernel/trace/Makefile
b/arch/powerpc/kernel/trace/Makefile
index 542aa7a8b2b4..fc32ec30b297 100644
--- a/arch/powerpc/kernel/trace/Makefile
+++ b/arch/powerpc/kernel/trace/Makefile
@@ -17,7 +17,6 @@ endif
obj-$(CONFIG_FUNCTION_TRACER) += ftrace_low.o
obj-$(CONFIG_DYNAMIC_FTRACE) += ftrace.o
obj-$(CONFIG_FUNCTION_GRAPH_TRACER)+= ftrace.o
-obj-$(CONFIG_FTRACE_SYSCALLS) += ftrace.o
obj-$(CONFIG_TRACING) += trace_clock.o
obj-$(CONFIG_PPC64)+= $(obj64-y)
--
2.35.1
[PATCH v1 10/22] powerpc/ftrace: Use CONFIG_FUNCTION_TRACER instead of CONFIG_DYNAMIC_FTRACE
Since commit 0c0c52306f47 ("powerpc: Only support DYNAMIC_FTRACE not
static"), CONFIG_DYNAMIC_FTRACE is always selected when
CONFIG_FUNCTION_TRACER is selected.
To avoid confusion and have the reader wonder what's happen when
CONFIG_FUNCTION_TRACER is selected and CONFIG_DYNAMIC_FTRACE is not,
use CONFIG_FUNCTION_TRACER in ifdefs instead of CONFIG_DYNAMIC_FTRACE.
As CONFIG_FUNCTION_GRAPH_TRACER depends on CONFIG_FUNCTION_TRACER,
ftrace.o doesn't need to appear for both symbols in Makefile.
Then as ftrace.o is built only when CONFIG_FUNCTION_TRACER is selected
ifdef CONFIG_FUNCTION_TRACER is not needed in ftrace.c
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/book3s/32/pgtable.h | 2 +-
arch/powerpc/include/asm/book3s/64/pgtable.h | 2 +-
arch/powerpc/include/asm/module.h| 4 ++--
arch/powerpc/include/asm/nohash/pgtable.h| 2 +-
arch/powerpc/kernel/module_32.c | 4 ++--
arch/powerpc/kernel/module_64.c | 6 +++---
arch/powerpc/kernel/trace/Makefile | 4 +---
arch/powerpc/kernel/trace/ftrace.c | 4
8 files changed, 11 insertions(+), 17 deletions(-)
diff --git a/arch/powerpc/include/asm/book3s/32/pgtable.h
b/arch/powerpc/include/asm/book3s/32/pgtable.h
index 772e00dc4ef1..992aed626eb4 100644
--- a/arch/powerpc/include/asm/book3s/32/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/32/pgtable.h
@@ -124,7 +124,7 @@ static inline bool pte_user(pte_t pte)
* on platforms where such control is possible.
*/
#if defined(CONFIG_KGDB) || defined(CONFIG_XMON) || defined(CONFIG_BDI_SWITCH)
||\
- defined(CONFIG_KPROBES) || defined(CONFIG_DYNAMIC_FTRACE)
+ defined(CONFIG_KPROBES) || defined(CONFIG_FUNCTION_TRACER)
#define PAGE_KERNEL_TEXT PAGE_KERNEL_X
#else
#define PAGE_KERNEL_TEXT PAGE_KERNEL_ROX
diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h
b/arch/powerpc/include/asm/book3s/64/pgtable.h
index 875730d5af40..cf01b609572f 100644
--- a/arch/powerpc/include/asm/book3s/64/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
@@ -169,7 +169,7 @@
* on platforms where such control is possible.
*/
#if defined(CONFIG_KGDB) || defined(CONFIG_XMON) || defined(CONFIG_BDI_SWITCH)
|| \
- defined(CONFIG_KPROBES) || defined(CONFIG_DYNAMIC_FTRACE)
+ defined(CONFIG_KPROBES) || defined(CONFIG_FUNCTION_TRACER)
#define PAGE_KERNEL_TEXT PAGE_KERNEL_X
#else
#define PAGE_KERNEL_TEXT PAGE_KERNEL_ROX
diff --git a/arch/powerpc/include/asm/module.h
b/arch/powerpc/include/asm/module.h
index 857d9ff24295..e6f5963fd96e 100644
--- a/arch/powerpc/include/asm/module.h
+++ b/arch/powerpc/include/asm/module.h
@@ -39,7 +39,7 @@ struct mod_arch_specific {
unsigned int init_plt_section;
#endif /* powerpc64 */
-#ifdef CONFIG_DYNAMIC_FTRACE
+#ifdef CONFIG_FUNCTION_TRACER
unsigned long tramp;
#ifdef CONFIG_DYNAMIC_FTRACE_WITH_REGS
unsigned long tramp_regs;
@@ -68,7 +68,7 @@ struct mod_arch_specific {
#endif /* MODULE */
#endif
-#ifdef CONFIG_DYNAMIC_FTRACE
+#ifdef CONFIG_FUNCTION_TRACER
#ifdef MODULE
asm(".section .ftrace.tramp,\"ax\",@nobits; .align 3; .previous");
#endif /* MODULE */
diff --git a/arch/powerpc/include/asm/nohash/pgtable.h
b/arch/powerpc/include/asm/nohash/pgtable.h
index ac75f4ab0dba..2e8cf217a191 100644
--- a/arch/powerpc/include/asm/nohash/pgtable.h
+++ b/arch/powerpc/include/asm/nohash/pgtable.h
@@ -23,7 +23,7 @@
* on platforms where such control is possible.
*/
#if defined(CONFIG_KGDB) || defined(CONFIG_XMON) || defined(CONFIG_BDI_SWITCH)
||\
- defined(CONFIG_KPROBES) || defined(CONFIG_DYNAMIC_FTRACE)
+ defined(CONFIG_KPROBES) || defined(CONFIG_FUNCTION_TRACER)
#define PAGE_KERNEL_TEXT PAGE_KERNEL_X
#else
#define PAGE_KERNEL_TEXT PAGE_KERNEL_ROX
diff --git a/arch/powerpc/kernel/module_32.c b/arch/powerpc/kernel/module_32.c
index a0432ef46967..2aa368ce21c9 100644
--- a/arch/powerpc/kernel/module_32.c
+++ b/arch/powerpc/kernel/module_32.c
@@ -39,7 +39,7 @@ static unsigned int count_relocs(const Elf32_Rela *rela,
unsigned int num)
r_addend = rela[i].r_addend;
}
-#ifdef CONFIG_DYNAMIC_FTRACE
+#ifdef CONFIG_FUNCTION_TRACER
_count_relocs++;/* add one for ftrace_caller */
#endif
return _count_relocs;
@@ -288,7 +288,7 @@ int apply_relocate_add(Elf32_Shdr *sechdrs,
return 0;
}
-#ifdef CONFIG_DYNAMIC_FTRACE
+#ifdef CONFIG_FUNCTION_TRACER
int module_trampoline_target(struct module *mod, unsigned long addr,
unsigned long *target)
{
diff --git a/arch/powerpc/kernel/module_64.c b/arch/powerpc/kernel/module_64.c
index 794720530442..b13a72665eee 100644
--- a/arch/powerpc/kernel/module_64.c
+++ b/arch/powerpc/kernel/module_64.c
@@ -207,7 +207,7 @@ static unsigned long get_stubs_size(const Elf64_Ehdr *hdr,
}
}
-#ifdef CONFIG_DYNAMIC_FTRACE
+#ifdef CONFIG_FUNCTI
[PATCH v1 03/22] powerpc/code-patching: Inline is_offset_in_{cond}_branch_range()
Test in is_offset_in_branch_range() and is_offset_in_cond_branch_range()
are simple tests that are worth inlining.
Signed-off-by: Christophe Leroy
---
arch/powerpc/include/asm/code-patching.h | 29 ++--
arch/powerpc/lib/code-patching.c | 27 --
2 files changed, 27 insertions(+), 29 deletions(-)
diff --git a/arch/powerpc/include/asm/code-patching.h
b/arch/powerpc/include/asm/code-patching.h
index 409483b2d0ce..e7c5df50cb4e 100644
--- a/arch/powerpc/include/asm/code-patching.h
+++ b/arch/powerpc/include/asm/code-patching.h
@@ -22,8 +22,33 @@
#define BRANCH_SET_LINK0x1
#define BRANCH_ABSOLUTE0x2
-bool is_offset_in_branch_range(long offset);
-bool is_offset_in_cond_branch_range(long offset);
+/*
+ * Powerpc branch instruction is :
+ *
+ * 0 6 30 31
+ * +-++---+---+
+ * | opcode | LI |AA |LK |
+ * +-++---+---+
+ * Where AA = 0 and LK = 0
+ *
+ * LI is a signed 24 bits integer. The real branch offset is computed
+ * by: imm32 = SignExtend(LI:'0b00', 32);
+ *
+ * So the maximum forward branch should be:
+ * (0x007f << 2) = 0x01fc = 0x1fc
+ * The maximum backward branch should be:
+ * (0xff80 << 2) = 0xfe00 = -0x200
+ */
+static inline bool is_offset_in_branch_range(long offset)
+{
+ return (offset >= -0x200 && offset <= 0x1fc && !(offset & 0x3));
+}
+
+static inline bool is_offset_in_cond_branch_range(long offset)
+{
+ return offset >= -0x8000 && offset <= 0x7fff && !(offset & 0x3);
+}
+
int create_branch(ppc_inst_t *instr, const u32 *addr,
unsigned long target, int flags);
int create_cond_branch(ppc_inst_t *instr, const u32 *addr,
diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c
index 00c68e7fb11e..58262c7e447c 100644
--- a/arch/powerpc/lib/code-patching.c
+++ b/arch/powerpc/lib/code-patching.c
@@ -208,33 +208,6 @@ int patch_branch(u32 *addr, unsigned long target, int
flags)
return patch_instruction(addr, instr);
}
-bool is_offset_in_branch_range(long offset)
-{
- /*
-* Powerpc branch instruction is :
-*
-* 0 6 30 31
-* +-++---+---+
-* | opcode | LI |AA |LK |
-* +-++---+---+
-* Where AA = 0 and LK = 0
-*
-* LI is a signed 24 bits integer. The real branch offset is computed
-* by: imm32 = SignExtend(LI:'0b00', 32);
-*
-* So the maximum forward branch should be:
-* (0x007f << 2) = 0x01fc = 0x1fc
-* The maximum backward branch should be:
-* (0xff80 << 2) = 0xfe00 = -0x200
-*/
- return (offset >= -0x200 && offset <= 0x1fc && !(offset & 0x3));
-}
-
-bool is_offset_in_cond_branch_range(long offset)
-{
- return offset >= -0x8000 && offset <= 0x7fff && !(offset & 0x3);
-}
-
/*
* Helper to check if a given instruction is a conditional branch
* Derived from the conditional checks in analyse_instr()
--
2.35.1
[PATCH v1 11/22] powerpc/ftrace: Remove ftrace_plt_tramps[]
ftrace_plt_tramps table is never filled so it is useless.
Remove it.
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/trace/ftrace.c | 8
1 file changed, 8 deletions(-)
diff --git a/arch/powerpc/kernel/trace/ftrace.c
b/arch/powerpc/kernel/trace/ftrace.c
index 188f59f4ee4a..b6c5223e26b2 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -250,7 +250,6 @@ static int setup_mcount_compiler_tramp(unsigned long tramp)
int i;
ppc_inst_t op;
unsigned long ptr;
- static unsigned long ftrace_plt_tramps[NUM_FTRACE_TRAMPS];
/* Is this a known long jump tramp? */
for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
@@ -259,13 +258,6 @@ static int setup_mcount_compiler_tramp(unsigned long tramp)
else if (ftrace_tramps[i] == tramp)
return 0;
- /* Is this a known plt tramp? */
- for (i = 0; i < NUM_FTRACE_TRAMPS; i++)
- if (!ftrace_plt_tramps[i])
- break;
- else if (ftrace_plt_tramps[i] == tramp)
- return -1;
-
/* New trampoline -- read where this goes */
if (copy_inst_from_kernel_nofault(&op, (void *)tramp)) {
pr_debug("Fetching opcode failed.\n");
--
2.35.1
[PATCH v1 18/22] powerpc/ftrace: Don't use copy_from_kernel_nofault() in module_trampoline_target()
module_trampoline_target() is quite a hot path used when
activating/deactivating function tracer.
Avoid the heavy copy_from_kernel_nofault() by doing four calls
to copy_inst_from_kernel_nofault().
Use __copy_inst_from_kernel_nofault() for the 3 last calls. First call
is done to copy_from_kernel_nofault() to check address is within
kernel space. No risk to wrap out the top of kernel space because the
last page is never mapped so if address is in last page the first copy
will fails and the other ones will never be performed.
And also make it notrace just like all functions that call it.
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/module_32.c | 12 +---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/kernel/module_32.c b/arch/powerpc/kernel/module_32.c
index 2aa368ce21c9..1282cfd672f2 100644
--- a/arch/powerpc/kernel/module_32.c
+++ b/arch/powerpc/kernel/module_32.c
@@ -289,13 +289,19 @@ int apply_relocate_add(Elf32_Shdr *sechdrs,
}
#ifdef CONFIG_FUNCTION_TRACER
-int module_trampoline_target(struct module *mod, unsigned long addr,
-unsigned long *target)
+notrace int module_trampoline_target(struct module *mod, unsigned long addr,
+unsigned long *target)
{
unsigned int jmp[4];
/* Find where the trampoline jumps to */
- if (copy_from_kernel_nofault(jmp, (void *)addr, sizeof(jmp)))
+ if (copy_inst_from_kernel_nofault(jmp, (void *)addr))
+ return -EFAULT;
+ if (__copy_inst_from_kernel_nofault(jmp + 1, (void *)addr + 4))
+ return -EFAULT;
+ if (__copy_inst_from_kernel_nofault(jmp + 2, (void *)addr + 8))
+ return -EFAULT;
+ if (__copy_inst_from_kernel_nofault(jmp + 3, (void *)addr + 12))
return -EFAULT;
/* verify that this is what we expect it to be */
--
2.35.1
[PATCH v1 14/22] powerpc/ftrace: Use size macro instead of opencoding
0x8000 is SZ_2G. Use it.
Signed-off-by: Christophe Leroy
---
arch/powerpc/kernel/trace/ftrace.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/trace/ftrace.c
b/arch/powerpc/kernel/trace/ftrace.c
index afb1d12838c9..15e0746806f8 100644
--- a/arch/powerpc/kernel/trace/ftrace.c
+++ b/arch/powerpc/kernel/trace/ftrace.c
@@ -741,7 +741,7 @@ int __init ftrace_dyn_arch_init(void)
#endif
long reladdr = addr - kernel_toc_addr();
- if (reladdr > 0x7FFF || reladdr < -(0x8000L)) {
+ if (reladdr >= SZ_2G || reladdr < -SZ_2G) {
pr_err("Address of %ps out of range of kernel_toc.\n",
(void *)addr);
return -1;
--
2.35.1
Re: [PATCH 1/2] powerpc: Reject probes on instructions that can't be single stepped
Hi, Naveen.
Some comments below.
On 3/23/22 08:51, Naveen N. Rao wrote:
Per the ISA, a Trace interrupt is not generated for:
- [h|u]rfi[d]
- rfscv
- sc, scv, and Trap instructions that trap
- Power-Saving Mode instructions
- other instructions that cause interrupts (other than Trace interrupts)
- the first instructions of any interrupt handler (applies to Branch and Single
Step tracing;
CIABR matches may still occur)
- instructions that are emulated by software
Add a helper to check for instructions belonging to the first four
categories above and to reject kprobes, uprobes and xmon breakpoints on
such instructions. We reject probing on instructions belonging to these
categories across all ISA versions and across both BookS and BookE.
For trap instructions, we can't know in advance if they can cause a
trap, and there is no good reason to allow probing on those. Also,
uprobes already refuses to probe trap instructions and kprobes does not
allow probes on trap instructions used for kernel warnings and bugs. As
such, stop allowing any type of probes/breakpoints on trap instruction
across uprobes, kprobes and xmon.
For some of the fp/altivec instructions that can generate an interrupt
and which we emulate in the kernel (altivec assist, for example), we
check and turn off single stepping in emulate_single_step().
Instructions generating a DSI are restarted and single stepping normally
completes once the instruction is completed.
In uprobes, if a single stepped instruction results in a non-fatal
signal to be delivered to the task, such signals are "delayed" until
after the instruction completes. For fatal signals, single stepping is
cancelled and the instruction restarted in-place so that core dump
captures proper addresses.
In kprobes, we do not allow probes on instructions having an extable
entry and we also do not allow probing interrupt vectors.
Signed-off-by: Naveen N. Rao
---
arch/powerpc/include/asm/probes.h | 55 +++
arch/powerpc/kernel/kprobes.c | 4 +--
arch/powerpc/kernel/uprobes.c | 5 +++
arch/powerpc/xmon/xmon.c | 11 +++
4 files changed, 67 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/include/asm/probes.h
b/arch/powerpc/include/asm/probes.h
index c5d984700d241a..21ab5b6256b590 100644
--- a/arch/powerpc/include/asm/probes.h
+++ b/arch/powerpc/include/asm/probes.h
@@ -31,6 +31,61 @@ typedef u32 ppc_opcode_t;
#define MSR_SINGLESTEP(MSR_SE)
#endif
+static inline bool can_single_step(u32 inst)
+{
+ switch (inst >> 26) {
Can't ppc_inst_primary_opcode() be used instead?
+ case 2: /* tdi */
+ return false;
+ case 3: /* twi */
+ return false;
+ case 17:/* sc and scv */
+ return false;
+ case 19:
+ switch ((inst >> 1) & 0x3ff) {
+ case 18:/* rfid */
+ return false;
+ case 38:/* rfmci */
+ return false;
+ case 39:/* rfdi */
+ return false;
+ case 50:/* rfi */
+ return false;
+ case 51:/* rfci */
+ return false;
+ case 82:/* rfscv */
+ return false;
+ case 274: /* hrfid */
+ return false;
+ case 306: /* urfid */
+ return false;
+ case 370: /* stop */
+ return false;
+ case 402: /* doze */
+ return false;
+ case 434: /* nap */
+ return false;
+ case 466: /* sleep */
+ return false;
+ case 498: /* rvwinkle */
+ return false;
+ }
+ break;
+ case 31:
+ switch ((inst >> 1) & 0x3ff) {
+ case 4: /* tw */
+ return false;
+ case 68:/* td */
+ return false;
+ case 146: /* mtmsr */
+ return false;
+ case 178: /* mtmsrd */
+ return false;
+ }
+ break;
+ }
+ return true;
+}
+
Can't OP_* definitions from ppc-opcode.h be used for all of these switch-case
statements?
/* Enable single stepping for the current task */
static inline void enable_single_step(struct pt_regs *regs)
{
diff --git a/arch/powerpc/kernel/kprobes.c b/arch/powerpc/kernel/kprobes.c
index 9a492fdec1dfbe..0936a6c8c256b9 100644
--- a/arch/powerpc/kernel/kprobes.c
+++ b/arch/powerpc/kernel/kprobes.c
@@ -129,8 +129,8 @@ int arch_prepare_kprobe(struct kprobe *p)
if ((unsigned long)p->addr & 0x03) {
printk("Attempt to
[PATCH v1] ftrace: Make ftrace_graph_is_dead() static inline
ftrace_graph_is_dead() is used on hot paths, it just reads a variable
in memory and is not worth suffering function call constraints.
For instance, at entry of prepare_ftrace_return(), inlining it avoids
saving prepare_ftrace_return() parameters to stack and restoring them
after calling ftrace_graph_is_dead().
The performance improvement is noticeable.
Signed-off-by: Christophe Leroy
---
include/linux/ftrace.h | 15 ++-
kernel/trace/fgraph.c | 14 +-
2 files changed, 15 insertions(+), 14 deletions(-)
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index e29187de..cecdda515059 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -1006,7 +1006,20 @@ unsigned long ftrace_graph_ret_addr(struct task_struct
*task, int *idx,
extern int register_ftrace_graph(struct fgraph_ops *ops);
extern void unregister_ftrace_graph(struct fgraph_ops *ops);
-extern bool ftrace_graph_is_dead(void);
+/**
+ * ftrace_graph_is_dead - returns true if ftrace_graph_stop() was called
+ *
+ * ftrace_graph_stop() is called when a severe error is detected in
+ * the function graph tracing. This function is called by the critical
+ * paths of function graph to keep those paths from doing any more harm.
+ */
+extern bool kill_ftrace_graph;
+
+static inline bool ftrace_graph_is_dead(void)
+{
+ return kill_ftrace_graph;
+}
+
extern void ftrace_graph_stop(void);
/* The current handlers in use */
diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c
index 22061d38fc00..3961f5884063 100644
--- a/kernel/trace/fgraph.c
+++ b/kernel/trace/fgraph.c
@@ -23,24 +23,12 @@
#define ASSIGN_OPS_HASH(opsname, val)
#endif
-static bool kill_ftrace_graph;
+bool kill_ftrace_graph;
int ftrace_graph_active;
/* Both enabled by default (can be cleared by function_graph tracer flags */
static bool fgraph_sleep_time = true;
-/**
- * ftrace_graph_is_dead - returns true if ftrace_graph_stop() was called
- *
- * ftrace_graph_stop() is called when a severe error is detected in
- * the function graph tracing. This function is called by the critical
- * paths of function graph to keep those paths from doing any more harm.
- */
-bool ftrace_graph_is_dead(void)
-{
- return kill_ftrace_graph;
-}
-
/**
* ftrace_graph_stop - set to permanently disable function graph tracing
*
--
2.35.1
[powerpc/merge] Possible stack corruption while running selftests
I am seeing random crashes(at least to me) with powerpc/selftests on P10 LPAR running powerpc/merge branch code. mitigation-patching.sh test was running in both the instances. In the latest instance it seems like a possible stack corruption ?? [ 711.005150] count-cache-flush: hardware flush enabled. [ 711.005153] link-stack-flush: software flush enabled. [ 711.015306] barrier-nospec: using ORI speculation barrier [ 711.030889] kernel tried to execute exec-protected page (ca70fc80) - exploit attempt? (uid: 0) [ 711.030902] BUG: Unable to handle kernel instruction fetch [ 711.030905] Faulting instruction address: 0xca70fc80 [ 711.030909] Thread overran stack, or stack corrupted [ 711.030913] Oops: Kernel access of bad area, sig: 11 [#1] [ 711.030917] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries [ 711.030924] Modules linked in: dm_mod nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set rfkill nf_tables bonding libcrc32c nfnetlink sunrpc pseries_rng xts vmx_crypto sch_fq_codel ext4 mbcache jbd2 sd_mod t10_pi sg ibmvscsi ibmveth scsi_transport_srp fuse [ 711.030960] CPU: 31 PID: 165 Comm: migration/31 Not tainted 5.17.0-ge8833c5edc59 #1 [ 711.030965] Stopper: multi_cpu_stop+0x0/0x230 <- stop_machine_cpuslocked+0x188/0x1e0 [ 711.030977] NIP: ca70fc80 LR: ca70fc80 CTR: c0293f90 [ 711.030981] REGS: ca70f9a0 TRAP: 0400 Not tainted (5.17.0-ge8833c5edc59) [ 711.030986] MSR: 80001280b033 CR: 48002822 XER: [ 711.031001] CFAR: c0216628 IRQMASK: 0 [ 711.031001] GPR00: ca70fc80 ca70fc40 c2a1fe00 00c57415 [ 711.031001] GPR04: c00efa36ab80 c00efa36ab70 c001e688 [ 711.031001] GPR08: c00efa3ef480 c00efa3ee600 [ 711.031001] GPR12: c00effbe5a80 c018fc98 c72a5f80 [ 711.031001] GPR16: [ 711.031001] GPR20: [ 711.031001] GPR24: 0001 0002 0003 c2a62138 [ 711.031001] GPR28: c0024224fb08 0001 c0024224fb2c 0001 [ 711.031054] NIP [ca70fc80] 0xca70fc80 [ 711.031058] LR [ca70fc80] 0xca70fc80 [ 711.031062] Call Trace: [ 711.031065] [ca70fc40] [ca70fc80] 0xca70fc80 (unreliable) [ 711.031071] [ca70fcb0] [c0293ce4] cpu_stopper_thread+0xe4/0x240 [ 711.031077] [ca70fd60] [000119a59724] 0x119a59724 [ 711.031083] BUG: Unable to handle kernel data access on read at 0xc014c000 [ 711.031088] Faulting instruction address: 0xc001ccfc [ 711.031091] Thread overran stack, or stack corrupted [ 711.031093] Oops: Kernel access of bad area, sig: 11 [#2] [ 711.031097] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries [ 711.031101] Modules linked in: dm_mod nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set rfkill nf_tables bonding libcrc32c nfnetlink sunrpc pseries_rng xts vmx_crypto sch_fq_codel ext4 mbcache jbd2 sd_mod t10_pi sg ibmvscsi ibmveth scsi_transport_srp fuse [ 711.031128] CPU: 31 PID: 165 Comm: Not tainted 5.17.0-ge8833c5edc59 #1 [ 711.031134] BUG: Unable to handle kernel data access at 0xc1000214ab60 [ 711.031138] Faulting instruction address: 0xc0293e70 [ 711.031141] Thread overran stack, or stack corrupted [ 711.031144] Oops: Kernel access of bad area, sig: 11 [#3] ……….. ……….. In another instance I saw following crash in ibmveth [ 714.823524] count-cache-flush: hardware flush enabled. [ 714.823528] link-stack-flush: software flush enabled. [ 714.828529] barrier-nospec: using ORI speculation barrier [ 715.181552] [ cut here ] [ 715.181558] kernel BUG at drivers/net/ethernet/ibm/ibmveth.c:402! [ 715.181563] Oops: Exception in kernel mode, sig: 5 [#1] [ 715.181568] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries [ 715.181572] Modules linked in: dm_mod nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set rfkill nf_tables bonding libcrc32c nfnetlink sunrpc pseries_rng xts vmx_crypto sch_fq_codel ext4 mbcache jbd2 sd_mod t10_pi sg ibmvscsi ibmveth scsi_transport_srp fuse [ 715.181604] CPU: 0 PID: 12 Comm: migration/0 Not tainted 5.17.0-ge8833c5edc59 #1 [ 715.181609] Stopper: multi_cpu_stop+0x0/0x230 <- stop_machine_cpuslocked+0x188/0x1e0 [ 715.181620] NIP: c00800a91fdc LR: c0aca5d4 CTR: c00800a91e48 [ 715.181624] REGS: c772f300 TRAP: 0700 Not tainted (5.17.0-ge8833c5edc59) [ 715.181628] MSR: 80029033 CR: 42004422 XER: [ 715.181640] CFAR: c00800a91f14 IRQMASK: 0 [ 715.181640] GPR00: c0aca5d4 c772f5a0 c00800ac8000 c0003a4c0a10 [ 715.181640] GPR04: 0010 2d89 000
