Re: [PATCH] perf report: Calling available function for stats printing

[Namhyung Kim]

On Fri, 28 Jun 2024 14:32:24 -0400, Abhishek Dubey wrote:

> For printing dump_trace, just use existing stats_print()
> function.
> 
> 

Applied to perf-tools-next, thanks!

Best regards,
Namhyung

Re: [PATCH V5 16/17] tools/perf: Add support for global_die to capture name of variable in case of register defined variable

[Namhyung Kim]

On Mon, Jul 01, 2024 at 10:04:29AM +0530, Athira Rajeev wrote:
> In case of register defined variable (found using
> find_data_type_global_reg), if the type of variable happens to be base
> type (example, long unsigned int), perf report captures it as:
> 
> 12.85%  long unsigned int  long unsigned int +0 (no field)
> 
> The above data type is actually referring to samples captured while
> accessing "r1" which represents current stack pointer in powerpc.
> register void *__stack_pointer asm("r1");
> 
> The dwarf debug contains this as:
> 
> <<>>
>  <1><18dd772>: Abbrev Number: 129 (DW_TAG_variable)
> <18dd774>   DW_AT_name: (indirect string, offset: 0x11ba): 
> current_stack_pointer
> <18dd778>   DW_AT_decl_file   : 51
> <18dd779>   DW_AT_decl_line   : 1468
> <18dd77b>   DW_AT_decl_column : 24
> <18dd77c>   DW_AT_type: <0x18da5cd>
> <18dd780>   DW_AT_external: 1
> <18dd780>   DW_AT_location: 1 byte block: 51(DW_OP_reg1 (r1))
> 
>  where 18da5cd is:
> 
>  <1><18da5cd>: Abbrev Number: 47 (DW_TAG_base_type)
> <18da5ce>   DW_AT_byte_size   : 8
> <18da5cf>   DW_AT_encoding: 7   (unsigned)
> <18da5d0>   DW_AT_name: (indirect string, offset: 0x55c7): long 
> unsigned int
> <<>>
> 
> To make it more clear to the user, capture the DW_AT_name of the
> variable and save it as part of Dwarf_Global. Dwarf_Global is used so
> that it can be used and retrieved while presenting the result.
> 
> Update "dso__findnew_data_type" function to set "var_name" if
> variable name is set as part of Dwarf_Global. Updated
> "hist_entry__typeoff_snprintf" to print var_name if it is set.
> With the changes, along with "long unsigned int" report also says the
> variable name as current_stack_pointer
> 
> Snippet of result:
> 
> 12.85%  long unsigned int  long unsigned int +0 (current_stack_pointer)

But I'm afraid it'd contain other (global) unsigned int type accesses
also.  Currently data_type_cmp() only compares size and type name so it
cannot distinguish two different int variables.

I thought it's ok since we care about types, but we might want to see
global variables (in the same type) - not only register variables -
separately.  Then I think we should compare variable names after type
name (if exists).


>  4.68%  struct paca_struct  struct paca_struct +2312 (__current)
>  4.57%  struct paca_struct  struct paca_struct +2354 (irq_soft_mask)
> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/util/annotate-data.c | 30 --
>  tools/perf/util/dwarf-aux.c |  1 +
>  tools/perf/util/dwarf-aux.h |  1 +
>  tools/perf/util/sort.c  |  7 +--
>  4 files changed, 31 insertions(+), 8 deletions(-)
> 
> diff --git a/tools/perf/util/annotate-data.c b/tools/perf/util/annotate-data.c
> index c919b2259f54..a6d6cf69dbab 100644
> --- a/tools/perf/util/annotate-data.c
> +++ b/tools/perf/util/annotate-data.c
> @@ -273,23 +273,32 @@ static void delete_members(struct annotated_member 
> *member)
>  }
>  
>  static struct annotated_data_type *dso__findnew_data_type(struct dso *dso,
> -   Dwarf_Die *type_die)
> +   Dwarf_Die *type_die, 
> Dwarf_Global *global_die)
>  {
>   struct annotated_data_type *result = NULL;
>   struct annotated_data_type key;
>   struct rb_node *node;
>   struct strbuf sb;
> + struct strbuf sb_var_name;
>   char *type_name;
> + char *var_name = NULL;
>   Dwarf_Word size;
>  
>   strbuf_init(, 32);
> + strbuf_init(_var_name, 32);
>   if (die_get_typename_from_type(type_die, ) < 0)
>   strbuf_add(, "(unknown type)", 14);
> + if (global_die->name) {
> + strbuf_addstr(_var_name, global_die->name);
> + var_name = strbuf_detach(_var_name, NULL);
> + }
>   type_name = strbuf_detach(, NULL);
>   dwarf_aggregate_size(type_die, );
>  
>   /* Check existing nodes in dso->data_types tree */
>   key.self.type_name = type_name;
> + if (global_die->name)
> + key.self.var_name = var_name;

Maybe you can set it unconditionally as it's NULL by default.
But you need to free the var_name if rb_find() returns an existing
node.  I think it should be done after you allocate the result.


>   key.self.size = size;
>   node = rb_find(, dso__data_types(dso), data_type_cmp);
>   if (node) {
> @@ -306,6 +315,8 @@ static struct annotated_data_type 
> *dso__findnew_data_type(struct dso *dso,
>   }
>  
>   result->self.type_name = type_name;
> + if (global_die->name)
> + result->self.var_name = var_name;

Here.

Thanks,
Namhyung


>   result->self.size = size;
>   INIT_LIST_HEAD(>self.children);
>  
> @@ -1183,7 +1194,7 @@ static int find_data_type_block(struct data_loc_info 
> *dloc,
>   * cu_die and match with reg to identify data type die.
>   */
>  static 

Re: [PATCH V5 15/17] tools/perf: Add support to find global register variables using find_data_type_global_reg

[Namhyung Kim]

On Mon, Jul 01, 2024 at 10:04:28AM +0530, Athira Rajeev wrote:
> There are cases where define a global register variable and associate it
> with a specified register. Example, in powerpc, two registers are
> defined to represent variable:
> 1. r13: represents local_paca
> register struct paca_struct *local_paca asm("r13");
> 
> 2. r1: represents stack_pointer
> register void *__stack_pointer asm("r1");
> 
> These regs are present in dwarf debug as DW_OP_reg as part of variables
> in the cu_die (compile unit). These are not present in die search done
> in the list of nested scopes since these are global register variables.
> 
> Example for local_paca represented by r13:
> 
> <<>>
>  <1><18dc6b4>: Abbrev Number: 128 (DW_TAG_variable)
> <18dc6b6>   DW_AT_name: (indirect string, offset: 0x3861): 
> local_paca
> <18dc6ba>   DW_AT_decl_file   : 48
> <18dc6bb>   DW_AT_decl_line   : 36
> <18dc6bc>   DW_AT_decl_column : 30
> <18dc6bd>   DW_AT_type: <0x18dc6c3>
> <18dc6c1>   DW_AT_external: 1
> <18dc6c1>   DW_AT_location: 1 byte block: 5d(DW_OP_reg13 (r13))
> 
>  <1><18dc6c3>: Abbrev Number: 3 (DW_TAG_pointer_type)
> <18dc6c4>   DW_AT_byte_size   : 8
> <18dc6c4>   DW_AT_type: <0x18dc353>
> 
> Where  DW_AT_type : <0x18dc6c3> further points to :
> 
>  <1><18dc6c3>: Abbrev Number: 3 (DW_TAG_pointer_type)
> <18dc6c4>   DW_AT_byte_size   : 8
> <18dc6c4>   DW_AT_type: <0x18dc353>
> 
> which belongs to:
> 
>  <1><18dc353>: Abbrev Number: 67 (DW_TAG_structure_type)
> <18dc354>   DW_AT_name: (indirect string, offset: 0x56cd): 
> paca_struct
> <18dc358>   DW_AT_byte_size   : 2944
> <18dc35a>   DW_AT_alignment   : 128
> <18dc35b>   DW_AT_decl_file   : 48
> <18dc35c>   DW_AT_decl_line   : 61
> <18dc35d>   DW_AT_decl_column : 8
> <18dc35d>   DW_AT_sibling : <0x18dc6b4>
> <<>>
> 
> Similar is case with "r1".
> 
> <<>>
>  <1><18dd772>: Abbrev Number: 129 (DW_TAG_variable)
> <18dd774>   DW_AT_name: (indirect string, offset: 0x11ba): 
> current_stack_pointer
> <18dd778>   DW_AT_decl_file   : 51
> <18dd779>   DW_AT_decl_line   : 1468
> <18dd77b>   DW_AT_decl_column : 24
> <18dd77c>   DW_AT_type: <0x18da5cd>
> <18dd780>   DW_AT_external: 1
> <18dd780>   DW_AT_location: 1 byte block: 51(DW_OP_reg1 (r1))
> 
>  where 18da5cd is:
> 
>  <1><18da5cd>: Abbrev Number: 47 (DW_TAG_base_type)
> <18da5ce>   DW_AT_byte_size   : 8
> <18da5cf>   DW_AT_encoding: 7   (unsigned)
> <18da5d0>   DW_AT_name: (indirect string, offset: 0x55c7): long 
> unsigned int
> <<>>
> 
> To identify data type for these two special cases, iterate over
> variables in the CU die (Compile Unit) and match it with the register.
> If the variable is a base type, ie die_get_real_type will return NULL
> here, set offset to zero. With the changes, data type for "paca_struct"
> and "long unsigned int" for r1 is identified.
> 
> Snippet from ./perf report -s type,type_off
> 
> 12.85%  long unsigned int  long unsigned int +0 (no field)
>  4.68%  struct paca_struct  struct paca_struct +2312 (__current)
>  4.57%  struct paca_struct  struct paca_struct +2354 (irq_soft_mask)
> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/util/annotate-data.c  | 42 
>  tools/perf/util/annotate.c   |  8 ++
>  tools/perf/util/annotate.h   |  1 +
>  tools/perf/util/include/dwarf-regs.h |  1 +
>  4 files changed, 52 insertions(+)
> 
> diff --git a/tools/perf/util/annotate-data.c b/tools/perf/util/annotate-data.c
> index 721235e1e6cf..c919b2259f54 100644
> --- a/tools/perf/util/annotate-data.c
> +++ b/tools/perf/util/annotate-data.c
> @@ -1175,6 +1175,42 @@ static int find_data_type_block(struct data_loc_info 
> *dloc,
>   return ret;
>  }
>  
> +/*
> + * Handle cases where define a global register variable and
> + * associate it with a specified register. These regs are
> + * present in dwarf debug as DW_OP_reg as part of variables
> + * in the cu_die (compile unit). Iterate over variables in the
> + * cu_die and match with reg to identify data type die.
> + */
> +static int find_data_type_global_reg(struct data_loc_info *dloc, int reg, 
> Dwarf_Die *cu_die,
> + Dwarf_Die *type_die)
> +{
> + Dwarf_Die vr_die;
> + int ret = -1;
> + struct die_var_type *var_types = NULL;
> +
> + die_collect_vars(cu_die, _types);
> + while (var_types) {
> + if (var_types->reg == reg) {

Reduce the indent level by using for and continue.  Like

for (vt = var_types ; vt != NULL; vt = vt->next) {
if (vt->reg != reg)
continue;
...
}

> + if (dwarf_offdie(dloc->di->dbg, var_types->die_off, 
> _die)) {
> + if (die_get_real_type(_die, type_die) == 
> NULL) {
> + dloc->type_offset = 0;
> +  

Re: [PATCH V5 05/17] tools/perf: Add support to capture and parse raw instruction in powerpc using dso__data_read_offset utility

[Namhyung Kim]

On Mon, Jul 01, 2024 at 10:04:18AM +0530, Athira Rajeev wrote:
> Add support to capture and parse raw instruction in powerpc.
> Currently, the perf tool infrastructure uses two ways to disassemble
> and understand the instruction. One is objdump and other option is
> via libcapstone.
> 
> Currently, the perf tool infrastructure uses "--no-show-raw-insn" option
> with "objdump" while disassemble. Example from powerpc with this option
> for an instruction address is:
> 
> Snippet from:
> objdump  --start-address= --stop-address=  -d 
> --no-show-raw-insn -C 
> 
> c10224b4: lwz r10,0(r9)
> 
> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> registers names and offset. Also to find whether there is a memory
> reference in the operands, "memory_ref_char" field of objdump is used.
> For x86, "(" is used as memory_ref_char to tackle instructions of the
> form "mov  (%rax), %rcx".
> 
> In case of powerpc, not all instructions using "(" are the only memory
> instructions. Example, above instruction can also be of extended form (X
> form) "lwzx r10,0,r19". Inorder to easy identify the instruction category
> and extract the source/target registers, patch adds support to use raw
> instruction for powerpc. Approach used is to read the raw instruction
> directly from the DSO file using "dso__data_read_offset" utility which
> is already implemented in perf infrastructure in "util/dso.c".
> 
> Example:
> 
> 38 01 81 e8 ld  r4,312(r1)
> 
> Here "38 01 81 e8" is the raw instruction representation. In powerpc,
> this translates to instruction form: "ld RT,DS(RA)" and binary code
> as:
> 
>| 58 |  RT  |  RA |  DS   | |
>-
>06 1116  30 31
> 
> Function "symbol__disassemble_dso" is updated to read raw instruction
> directly from DSO using dso__data_read_offset utility. In case of
> above example, this captures:
> line:38 01 81 e8
> 
> The above works well when perf report is invoked with only sort keys for
> data type ie type and typeoff. Because there is no instruction level
> annotation needed if only data type information is requested for. For
> annotating sample, along with type and typeoff sort key, "sym" sort key
> is also needed. And by default invoking just "perf report" uses sort key
> "sym" that displays the symbol information.
> 
> With approach changes in powerpc which first reads DSO for raw
> instruction, "perf annotate" and "perf report" + a key breaks since
> it doesn't do the instruction level disassembly.
> 
> Snippet of result from perf report:
> 
> Samples: 1K of event 'mem-loads', 4000 Hz, Event count (approx.): 937238
> do_work  /usr/bin/pmlogger [Percent: local period]
> Percent│ea230010
>│3a550010
>│3a60
> 
>│38f60001
>│39490008
>│42400438
>  51.44 │81290008
>│7d485378
> 
> Here, raw instruction is displayed in the output instead of human
> readable annotated form.
> 
> One way to get the appropriate data is to specify "--objdump path", by
> which code annotation will be done. But the default behaviour will be
> changed. To fix this breakage, check if "sym" sort key is set. If so
> fallback and use the libcapstone/objdump way of disassmbling the sample.
> 
> With the changes and "perf report"
> 
> Samples: 1K of event 'mem-loads', 4000 Hz, Event count (approx.): 937238
> do_work  /usr/bin/pmlogger [Percent: local period]
> Percent│ldr17,16(r3)
>│addi  r18,r21,16
>│lir19,0
> 
>│ 8b0:   rldiclr10,r10,63,33
>│addi  r10,r10,1
>│mtctr r10
>│  ↓ b 8e4
>│ 8c0:   addi  r7,r22,1
>│addi  r10,r9,8
>│  ↓ bdz   d00
>  51.44 │lwz   r9,8(r9)
>│mrr8,r10
>│cmpw  r20,r9
> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/util/disasm.c | 101 +++
>  1 file changed, 101 insertions(+)
> 
> diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
> index ddb861a0b043..e400dcab4029 100644
> --- a/tools/perf/util/disasm.c
> +++ b/tools/perf/util/disasm.c
> @@ -25,6 +25,7 @@
>  #include "srcline.h"
>  #include "symbol.h"
>  #include "util.h"
> +#include "sort.h"
>  
>  static regex_tfile_lineno;
>  
> @@ -1645,6 +1646,91 @@ static int symbol__disassemble_capstone(char 
> *filename, struct symbol *sym,
>  }
>  #endif
>  
> +static int symbol__disassemble_dso(char *filename, struct symbol *sym,

Again, I still think this should be named to symbol__disassemble_raw()
because it only uses the raw binary codes.  Using dso__data_read_offset
is not important here and it's just a way of implementing it.

Thanks,
Namhyung


> + struct annotate_args *args)
> +{
> + struct 

Re: [PATCH V5 04/17] tools/perf: Add disasm_line__parse to parse raw instruction for powerpc

[Namhyung Kim]

On Mon, Jul 01, 2024 at 10:04:17AM +0530, Athira Rajeev wrote:
> Currently, the perf tool infrastructure disasm_line__parse function to
> parse disassembled line.
> 
> Example snippet from objdump:
> objdump  --start-address= --stop-address=  -d 
> --no-show-raw-insn -C 
> 
> c10224b4: lwz r10,0(r9)
> 
> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> registers names and offset. In powerpc, the approach for data type
> profiling uses raw instruction instead of result from objdump to identify
> the instruction category and extract the source/target registers.
> 
> Example: 38 01 81 e8 ld  r4,312(r1)
> 
> Here "38 01 81 e8" is the raw instruction representation. Add function
> "disasm_line__parse_powerpc" to handle parsing of raw instruction.
> Also update "struct disasm_line" to save the binary code/
> With the change, function captures:
> 
> line -> "38 01 81 e8 ld  r4,312(r1)"
> raw instruction "38 01 81 e8"
> 
> Raw instruction is used later to extract the reg/offset fields. Macros
> are added to extract opcode and register fields. "struct disasm_line"
> is updated to carry union of "bytes" and "raw_insn" of 32 bit to carry raw
> code (raw). Function "disasm_line__parse_powerpc fills the raw
> instruction hex value and can use macros to get opcode. There is no
> changes in existing code paths, which parses the disassembled code.
> The size of raw instruction depends on architecture. In case of powerpc,
> the parsing the disasm line needs to handle cases for reading binary code
> directly from DSO as well as parsing the objdump result. Hence adding
> the logic into separate function instead of updating "disasm_line__parse".
> The architecture using the instruction name and present approach is
> not altered. Since this approach targets powerpc, the macro
> implementation is added for powerpc as of now.
> 
> Since the disasm_line__parse is used in other cases (perf annotate) and
> not only data tye profiling, the powerpc callback includes changes to
> work with binary code as well as mneumonic representation. Also in case
> if the DSO read fails and libcapstone is not supported, the approach
> fallback to use objdump as option. Hence as option, patch has changes to
> ensure objdump option also works well.
> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/include/linux/string.h  |  2 +
>  tools/lib/string.c| 13 
>  .../perf/arch/powerpc/annotate/instructions.c |  1 +
>  tools/perf/arch/powerpc/util/dwarf-regs.c |  9 +++
>  tools/perf/util/annotate.h|  5 +-
>  tools/perf/util/disasm.c  | 59 ++-
>  6 files changed, 87 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/include/linux/string.h b/tools/include/linux/string.h
> index db5c99318c79..0acb1fc14e19 100644
> --- a/tools/include/linux/string.h
> +++ b/tools/include/linux/string.h
> @@ -46,5 +46,7 @@ extern char * __must_check skip_spaces(const char *);
>  
>  extern char *strim(char *);
>  
> +extern void remove_spaces(char *s);
> +
>  extern void *memchr_inv(const void *start, int c, size_t bytes);
>  #endif /* _TOOLS_LINUX_STRING_H_ */
> diff --git a/tools/lib/string.c b/tools/lib/string.c
> index 8b6892f959ab..3126d2cff716 100644
> --- a/tools/lib/string.c
> +++ b/tools/lib/string.c
> @@ -153,6 +153,19 @@ char *strim(char *s)
>   return skip_spaces(s);
>  }
>  
> +/*
> + * remove_spaces - Removes whitespaces from @s
> + */
> +void remove_spaces(char *s)
> +{
> + char *d = s;
> +
> + do {
> + while (*d == ' ')
> + ++d;
> + } while ((*s++ = *d++));
> +}
> +
>  /**
>   * strreplace - Replace all occurrences of character in string.
>   * @s: The string to operate on.
> diff --git a/tools/perf/arch/powerpc/annotate/instructions.c 
> b/tools/perf/arch/powerpc/annotate/instructions.c
> index a3f423c27cae..d57fd023ef9c 100644
> --- a/tools/perf/arch/powerpc/annotate/instructions.c
> +++ b/tools/perf/arch/powerpc/annotate/instructions.c
> @@ -55,6 +55,7 @@ static int powerpc__annotate_init(struct arch *arch, char 
> *cpuid __maybe_unused)
>   arch->initialized = true;
>   arch->associate_instruction_ops = 
> powerpc__associate_instruction_ops;
>   arch->objdump.comment_char  = '#';
> + annotate_opts.show_asm_raw = true;
>   }
>  
>   return 0;
> diff --git a/tools/perf/arch/powerpc/util/dwarf-regs.c 
> b/tools/perf/arch/powerpc/util/dwarf-regs.c
> index 0c4f4caf53ac..430623ca5612 100644
> --- a/tools/perf/arch/powerpc/util/dwarf-regs.c
> +++ b/tools/perf/arch/powerpc/util/dwarf-regs.c
> @@ -98,3 +98,12 @@ int regs_query_register_offset(const char *name)
>   return roff->ptregs_offset;
>   return -EINVAL;
>  }
> +
> +#define PPC_OP(op)   (((op) >> 26) & 0x3F)
> +#define PPC_RA(a)(((a) >> 16) & 0x1f)
> +#define PPC_RT(t)(((t) >> 21) & 0x1f)
> +#define PPC_RB(b)

Re: [PATCH V5 03/17] tools/perf: Update TYPE_STATE_MAX_REGS to include max of regs in powerpc

[Namhyung Kim]

On Mon, Jul 01, 2024 at 10:04:16AM +0530, Athira Rajeev wrote:
> Add TYPE_STATE_MAX_REGS_X86 and TYPE_STATE_MAX_REGS_PPC. Define
> TYPE_STATE_MAX_REGS to be 32 which is max size of the array. While
> checking if reg is valid using has_reg_type, use the max value
> depending on the architecture. For x86, use TYPE_STATE_MAX_REGS_X86
> since max number of regs is 16. Update has_reg_type to
> pass "struct arch" also as one of the parameters.
> 
> Signed-off-by: Athira Rajeev
> ---
>  tools/perf/arch/x86/annotate/instructions.c | 20 ++--
>  tools/perf/util/annotate-data.c | 13 +
>  tools/perf/util/annotate-data.h |  6 --
>  3 files changed, 23 insertions(+), 16 deletions(-)
> 
> diff --git a/tools/perf/arch/x86/annotate/instructions.c 
> b/tools/perf/arch/x86/annotate/instructions.c
> index 7b7d462c6c6b..ea1dc686e7b4 100644
> --- a/tools/perf/arch/x86/annotate/instructions.c
> +++ b/tools/perf/arch/x86/annotate/instructions.c
> @@ -263,14 +263,14 @@ static void update_insn_state_x86(struct type_state 
> *state,
>   struct map_symbol *ms = dloc->ms;
>   u64 ip = ms->sym->start + dl->al.offset;
>  
> - if (!has_reg_type(state, dst->reg1))
> + if (!has_reg_type(state, dst->reg1, dloc->arch))
>   return;
>  
>   tsr = >regs[dst->reg1];
>  
>   if (src->imm)
>   imm_value = src->offset;
> - else if (has_reg_type(state, src->reg1) &&
> + else if (has_reg_type(state, src->reg1, dloc->arch) &&
>state->regs[src->reg1].kind == TSR_KIND_CONST)
>   imm_value = state->regs[src->reg1].imm_value;
>   else if (src->reg1 == DWARF_REG_PC) {
> @@ -321,7 +321,7 @@ static void update_insn_state_x86(struct type_state 
> *state,
>  
>   /* Case 1. register to register or segment:offset to register transfers 
> */
>   if (!src->mem_ref && !dst->mem_ref) {
> - if (!has_reg_type(state, dst->reg1))
> + if (!has_reg_type(state, dst->reg1, dloc->arch))
>   return;
>  
>   tsr = >regs[dst->reg1];
> @@ -374,7 +374,7 @@ static void update_insn_state_x86(struct type_state 
> *state,
>   return;
>   }
>  
> - if (!has_reg_type(state, src->reg1) ||
> + if (!has_reg_type(state, src->reg1, dloc->arch) ||
>   !state->regs[src->reg1].ok) {
>   tsr->ok = false;
>   return;
> @@ -392,7 +392,7 @@ static void update_insn_state_x86(struct type_state 
> *state,
>   if (src->mem_ref && !dst->mem_ref) {
>   int sreg = src->reg1;
>  
> - if (!has_reg_type(state, dst->reg1))
> + if (!has_reg_type(state, dst->reg1, dloc->arch))
>   return;
>  
>   tsr = >regs[dst->reg1];
> @@ -427,7 +427,7 @@ static void update_insn_state_x86(struct type_state 
> *state,
>   pr_debug_type_name(>type, tsr->kind);
>   }
>   /* And then dereference the pointer if it has one */
> - else if (has_reg_type(state, sreg) && state->regs[sreg].ok &&
> + else if (has_reg_type(state, sreg, dloc->arch) && 
> state->regs[sreg].ok &&
>state->regs[sreg].kind == TSR_KIND_TYPE &&
>die_deref_ptr_type(>regs[sreg].type,
>   src->offset, _die)) {
> @@ -464,7 +464,7 @@ static void update_insn_state_x86(struct type_state 
> *state,
>   pr_debug_type_name(_die, tsr->kind);
>   }
>   /* And check percpu access with base register */
> - else if (has_reg_type(state, sreg) &&
> + else if (has_reg_type(state, sreg, dloc->arch) &&
>state->regs[sreg].kind == TSR_KIND_PERCPU_BASE) {
>   u64 ip = dloc->ms->sym->start + dl->al.offset;
>   u64 var_addr = src->offset;
> @@ -473,7 +473,7 @@ static void update_insn_state_x86(struct type_state 
> *state,
>   if (src->multi_regs) {
>   int reg2 = (sreg == src->reg1) ? src->reg2 : 
> src->reg1;
>  
> - if (has_reg_type(state, reg2) && 
> state->regs[reg2].ok &&
> + if (has_reg_type(state, reg2, dloc->arch) && 
> state->regs[reg2].ok &&
>   state->regs[reg2].kind == TSR_KIND_CONST)
>   var_addr += state->regs[reg2].imm_value;
>   }
> @@ -504,7 +504,7 @@ static void update_insn_state_x86(struct type_state 
> *state,
>   }
>   }
>   /* And then dereference the calculated pointer if it has one */
> - else if (has_reg_type(state, sreg) && state->regs[sreg].ok 

Re: [V4 05/16] tools/perf: Add disasm_line__parse to parse raw instruction for powerpc

[Namhyung Kim]

Hello,

On Wed, Jun 26, 2024 at 09:38:28AM +0530, Athira Rajeev wrote:
> 
> 
> > On 26 Jun 2024, at 12:15 AM, Namhyung Kim  wrote:
> > 
> > On Tue, Jun 25, 2024 at 06:12:51PM +0530, Athira Rajeev wrote:
> >> 
> >> 
> >>> On 25 Jun 2024, at 11:09 AM, Namhyung Kim  wrote:
> >>> 
> >>> On Fri, Jun 14, 2024 at 10:56:20PM +0530, Athira Rajeev wrote:
> >>>> Currently, the perf tool infrastructure disasm_line__parse function to
> >>>> parse disassembled line.
> >>>> 
> >>>> Example snippet from objdump:
> >>>> objdump  --start-address= --stop-address=  -d 
> >>>> --no-show-raw-insn -C 
> >>>> 
> >>>> c10224b4: lwz r10,0(r9)
> >>>> 
> >>>> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> >>>> registers names and offset. In powerpc, the approach for data type
> >>>> profiling uses raw instruction instead of result from objdump to identify
> >>>> the instruction category and extract the source/target registers.
> >>>> 
> >>>> Example: 38 01 81 e8 ld  r4,312(r1)
> >>>> 
> >>>> Here "38 01 81 e8" is the raw instruction representation. Add function
> >>>> "disasm_line__parse_powerpc" to handle parsing of raw instruction.
> >>>> Also update "struct disasm_line" to save the binary code/
> >>>> With the change, function captures:
> >>>> 
> >>>> line -> "38 01 81 e8 ld  r4,312(r1)"
> >>>> raw instruction "38 01 81 e8"
> >>>> 
> >>>> Raw instruction is used later to extract the reg/offset fields. Macros
> >>>> are added to extract opcode and register fields. "struct disasm_line"
> >>>> is updated to carry union of "bytes" and "raw_insn" of 32 bit to carry 
> >>>> raw
> >>>> code (raw). Function "disasm_line__parse_powerpc fills the raw
> >>>> instruction hex value and can use macros to get opcode. There is no
> >>>> changes in existing code paths, which parses the disassembled code.
> >>>> The architecture using the instruction name and present approach is
> >>>> not altered. Since this approach targets powerpc, the macro
> >>>> implementation is added for powerpc as of now.
> >>>> 
> >>>> Since the disasm_line__parse is used in other cases (perf annotate) and
> >>>> not only data tye profiling, the powerpc callback includes changes to
> >>>> work with binary code as well as mneumonic representation. Also in case
> >>>> if the DSO read fails and libcapstone is not supported, the approach
> >>>> fallback to use objdump as option. Hence as option, patch has changes to
> >>>> ensure objdump option also works well.
> >>>> 
> >>>> Signed-off-by: Athira Rajeev 
> >>>> ---
[SNIP]
> >>>> +/*
> >>>> + * Parses the result captured from symbol__disassemble_*
> >>>> + * Example, line read from DSO file in powerpc:
> >>>> + * line:38 01 81 e8
> >>>> + * opcode: fetched from arch specific get_opcode_insn
> >>>> + * rawp_insn: e8810138
> >>>> + *
> >>>> + * rawp_insn is used later to extract the reg/offset fields
> >>>> + */
> >>>> +#define PPC_OP(op) (((op) >> 26) & 0x3F)
> >>>> +
> >>>> +static int disasm_line__parse_powerpc(struct disasm_line *dl)
> >>>> +{
> >>>> + char *line = dl->al.line;
> >>>> + const char **namep = >ins.name;
> >>>> + char **rawp = >ops.raw;
> >>>> + char tmp, *tmp_raw_insn, *name_raw_insn = skip_spaces(line);
> >>>> + char *name = skip_spaces(name_raw_insn + 11);
> >>>> + int objdump = 0;
> >>>> +
> >>>> + if (strlen(line) > 11)
> >>>> + objdump = 1;
> >>>> +
> >>>> + if (name_raw_insn[0] == '\0')
> >>>> + return -1;
> >>>> +
> >>>> + if (objdump) {
> >>>> + *rawp = name + 1;
> >>>> + while ((*rawp)[0] != '\0' && !isspace((*rawp)[0]))
> >>>> + ++*rawp;
> >>>> + tmp = (*rawp)[0];
> >>>> + (*rawp)[0] = '\0';
> >>>> +
> >>

Re: [PATCH V4 1/3] tools/perf: Fix the string match for "/tmp/perf-$PID.map" files in dso__load

[Namhyung Kim]

On Sun, 23 Jun 2024 12:18:48 +0530, Athira Rajeev wrote:

> Perf test for perf probe of function from different CU fails
> as below:
> 
>   ./perf test -vv "test perf probe of function from different CU"
>   116: test perf probe of function from different CU:
>   --- start ---
>   test child forked, pid 2679
>   Failed to find symbol foo in 
> /tmp/perf-uprobe-different-cu-sh.Msa7iy89bx/testfile
> Error: Failed to add events.
>   --- Cleaning up ---
>   "foo" does not hit any event.
> Error: Failed to delete events.
>    end(-1) 
>   116: test perf probe of function from different CU   : 
> FAILED!
> 
> [...]

Applied the series to perf-tools-next, thanks!

Best regards,
Namhyung

Re: [V4 05/16] tools/perf: Add disasm_line__parse to parse raw instruction for powerpc

[Namhyung Kim]

On Tue, Jun 25, 2024 at 06:12:51PM +0530, Athira Rajeev wrote:
> 
> 
> > On 25 Jun 2024, at 11:09 AM, Namhyung Kim  wrote:
> > 
> > On Fri, Jun 14, 2024 at 10:56:20PM +0530, Athira Rajeev wrote:
> >> Currently, the perf tool infrastructure disasm_line__parse function to
> >> parse disassembled line.
> >> 
> >> Example snippet from objdump:
> >> objdump  --start-address= --stop-address=  -d 
> >> --no-show-raw-insn -C 
> >> 
> >> c10224b4: lwz r10,0(r9)
> >> 
> >> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> >> registers names and offset. In powerpc, the approach for data type
> >> profiling uses raw instruction instead of result from objdump to identify
> >> the instruction category and extract the source/target registers.
> >> 
> >> Example: 38 01 81 e8 ld  r4,312(r1)
> >> 
> >> Here "38 01 81 e8" is the raw instruction representation. Add function
> >> "disasm_line__parse_powerpc" to handle parsing of raw instruction.
> >> Also update "struct disasm_line" to save the binary code/
> >> With the change, function captures:
> >> 
> >> line -> "38 01 81 e8 ld  r4,312(r1)"
> >> raw instruction "38 01 81 e8"
> >> 
> >> Raw instruction is used later to extract the reg/offset fields. Macros
> >> are added to extract opcode and register fields. "struct disasm_line"
> >> is updated to carry union of "bytes" and "raw_insn" of 32 bit to carry raw
> >> code (raw). Function "disasm_line__parse_powerpc fills the raw
> >> instruction hex value and can use macros to get opcode. There is no
> >> changes in existing code paths, which parses the disassembled code.
> >> The architecture using the instruction name and present approach is
> >> not altered. Since this approach targets powerpc, the macro
> >> implementation is added for powerpc as of now.
> >> 
> >> Since the disasm_line__parse is used in other cases (perf annotate) and
> >> not only data tye profiling, the powerpc callback includes changes to
> >> work with binary code as well as mneumonic representation. Also in case
> >> if the DSO read fails and libcapstone is not supported, the approach
> >> fallback to use objdump as option. Hence as option, patch has changes to
> >> ensure objdump option also works well.
> >> 
> >> Signed-off-by: Athira Rajeev 
> >> ---
> >> tools/include/linux/string.h  |  2 +
> >> tools/lib/string.c| 13 
> >> .../perf/arch/powerpc/annotate/instructions.c |  1 +
> >> tools/perf/arch/powerpc/util/dwarf-regs.c |  9 +++
> >> tools/perf/util/annotate.h|  5 +-
> >> tools/perf/util/disasm.c  | 59 ++-
> >> 6 files changed, 87 insertions(+), 2 deletions(-)
> >> 
> >> diff --git a/tools/include/linux/string.h b/tools/include/linux/string.h
> >> index db5c99318c79..0acb1fc14e19 100644
> >> --- a/tools/include/linux/string.h
> >> +++ b/tools/include/linux/string.h
> >> @@ -46,5 +46,7 @@ extern char * __must_check skip_spaces(const char *);
> >> 
> >> extern char *strim(char *);
> >> 
> >> +extern void remove_spaces(char *s);
> >> +
> >> extern void *memchr_inv(const void *start, int c, size_t bytes);
> >> #endif /* _TOOLS_LINUX_STRING_H_ */
> >> diff --git a/tools/lib/string.c b/tools/lib/string.c
> >> index 8b6892f959ab..3126d2cff716 100644
> >> --- a/tools/lib/string.c
> >> +++ b/tools/lib/string.c
> >> @@ -153,6 +153,19 @@ char *strim(char *s)
> >> return skip_spaces(s);
> >> }
> >> 
> >> +/*
> >> + * remove_spaces - Removes whitespaces from @s
> >> + */
> >> +void remove_spaces(char *s)
> >> +{
> >> + char *d = s;
> >> +
> >> + do {
> >> + while (*d == ' ')
> >> + ++d;
> >> + } while ((*s++ = *d++));
> >> +}
> >> +
> >> /**
> >>  * strreplace - Replace all occurrences of character in string.
> >>  * @s: The string to operate on.
> >> diff --git a/tools/perf/arch/powerpc/annotate/instructions.c 
> >> b/tools/perf/arch/powerpc/annotate/instructions.c
> >> index a3f423c27cae..d57fd023ef9c 100644
> >> --- a/tools/perf/arch/powerpc/annotate/instructions.c
> >> ++

Re: [V4 03/16] tools/perf: Add support to capture and parse raw instruction in powerpc using dso__data_read_offset utility

[Namhyung Kim]

On Tue, Jun 25, 2024 at 06:08:49PM +0530, Athira Rajeev wrote:
> 
> 
> > On 25 Jun 2024, at 10:59 AM, Namhyung Kim  wrote:
> > 
> > On Fri, Jun 14, 2024 at 10:56:18PM +0530, Athira Rajeev wrote:
> >> Add support to capture and parse raw instruction in powerpc.
> >> Currently, the perf tool infrastructure uses two ways to disassemble
> >> and understand the instruction. One is objdump and other option is
> >> via libcapstone.
> >> 
> >> Currently, the perf tool infrastructure uses "--no-show-raw-insn" option
> >> with "objdump" while disassemble. Example from powerpc with this option
> >> for an instruction address is:
> >> 
> >> Snippet from:
> >> objdump  --start-address= --stop-address=  -d 
> >> --no-show-raw-insn -C 
> >> 
> >> c10224b4: lwz r10,0(r9)
> > 
> > What about removing --no-show-raw-insn and parse the raw byte code in
> > the output for powerpc?  I think it's better to support normal
> > annotation together.
> Hi Namhyung,
> 
> Yes, In the other patch in same series, I have added support for normal 
> annotation together.
> Patch 5 includes changes to work with binary code as well as mneumonic 
> representation.
> 
> Example representation using --show-raw-insn in objdump gives result:
> 
> 38 01 81 e8 ld r4,312(r1)
> 
> Patch5 has changes to use “objdump” with --show-raw-insn to read the raw 
> instruction and also support normal annotation.

Ok, that's good!


> In case of data type profiling, with only sort keys, (type, typeoff) there is 
> no need to disassemble and then get raw byte code.
> Binary code can be read directly from the DSO. Compared to using objdump, 
> directly reading from DSO will be faster in this case.

Sounds like an optimization.  Then I think you'd better handle the
general case first and optimize later.  Probably you want to merge
patch 3 and 4 together.

Thanks,
Namhyung


> In summary, current patchset uses below approach:
> 
> 1. Read directly from DSO using dso__data_read_offset if only “type, typeoff” 
> is needed.
> 2. If in any case reading directly from DSO fails, fallback to using 
> libcapstone. Using libcapstone to read is faster than objdump
> 3. If libcapstone is not supported, approach will use objdump. Patchset has 
> changes to handle objdump result created with show-raw-ins in powerpc. 
> 4. Also for normal perf report or perf annotate, approach will use objdump
> 
> NOTE:
> libcapstone is used currently only for reading raw binary code. Disassemble 
> is currently not enabled. While attempting to do cs_disasm, observation is 
> that some of the instructions were not identified (ex: extswsli, maddld) and 
> it had to fallback to use objdump. Hence enabling "cs_disasm" is added in 
> comment section as a TODO for powerpc. Patch number 13. 
> 
> Thanks
> Athira
> 
> > 
> >> 
> >> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> >> registers names and offset. Also to find whether there is a memory
> >> reference in the operands, "memory_ref_char" field of objdump is used.
> >> For x86, "(" is used as memory_ref_char to tackle instructions of the
> >> form "mov  (%rax), %rcx".
> >> 
> >> In case of powerpc, not all instructions using "(" are the only memory
> >> instructions. Example, above instruction can also be of extended form (X
> >> form) "lwzx r10,0,r19". Inorder to easy identify the instruction category
> >> and extract the source/target registers, patch adds support to use raw
> >> instruction for powerpc. Approach used is to read the raw instruction
> >> directly from the DSO file using "dso__data_read_offset" utility which
> >> is already implemented in perf infrastructure in "util/dso.c".
> >> 
> >> Example:
> >> 
> >> 38 01 81 e8 ld  r4,312(r1)
> >> 
> >> Here "38 01 81 e8" is the raw instruction representation. In powerpc,
> >> this translates to instruction form: "ld RT,DS(RA)" and binary code
> >> as:
> >> 
> >>   | 58 |  RT  |  RA |  DS   | |
> >>   -
> >>   06 1116  30 31
> >> 
> >> Function "symbol__disassemble_dso" is updated to read raw instruction
> >> directly from DSO using dso__data_read_offset utility. In case of
> >> above example, this captures:
> >> line:38 01 81 e8
> >> 
> >> Signed-off-by: A

Re: [PATCH V4 2/3] tools/perf: Use is_perf_pid_map_name helper function to check dso's of pattern /tmp/perf-%d.map

[Namhyung Kim]

On Tue, Jun 25, 2024 at 5:03 AM Adrian Hunter  wrote:
>
> On 23/06/24 09:48, Athira Rajeev wrote:
> > commit 80d496be89ed ("perf report: Add support for profiling JIT
> > generated code") added support for profiling JIT generated code.
> > This patch handles dso's of form "/tmp/perf-$PID.map".
> >
> > Some of the references doesn't check exactly for same pattern.
> > some uses "if (!strncmp(dso_name, "/tmp/perf-", 10))". Fix
> > this by using helper function perf_pid_map_tid and
> > is_perf_pid_map_name which looks for proper pattern of
> > form: "/tmp/perf-$PID.map" for these checks.
> >
> > Signed-off-by: Athira Rajeev 
>
> Add a Fixes tag, then
>
> Reviewed-by: Adrian Hunter 

Thanks, but I'm not sure which commit I can add the Fixes tag because
the original commit 80d496be89ed is too old and I'm sure we added a
lot of changes after that.

Namhyung


>
> > ---
> >  tools/perf/util/dsos.c| 2 +-
> >  tools/perf/util/srcline.c | 2 +-
> >  2 files changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/tools/perf/util/dsos.c b/tools/perf/util/dsos.c
> > index ab3d0c01dd63..846828ea1f00 100644
> > --- a/tools/perf/util/dsos.c
> > +++ b/tools/perf/util/dsos.c
> > @@ -275,7 +275,7 @@ static void dso__set_basename(struct dso *dso)
> >   char *base, *lname;
> >   int tid;
> >
> > - if (sscanf(dso__long_name(dso), "/tmp/perf-%d.map", ) == 1) {
> > + if (perf_pid_map_tid(dso__long_name(dso), )) {
> >   if (asprintf(, "[JIT] tid %d", tid) < 0)
> >   return;
> >   } else {
> > diff --git a/tools/perf/util/srcline.c b/tools/perf/util/srcline.c
> > index 9d670d8c1c08..51eb78993fe2 100644
> > --- a/tools/perf/util/srcline.c
> > +++ b/tools/perf/util/srcline.c
> > @@ -39,7 +39,7 @@ static const char *srcline_dso_name(struct dso *dso)
> >   if (dso_name[0] == '[')
> >   return NULL;
> >
> > - if (!strncmp(dso_name, "/tmp/perf-", 10))
> > + if (is_perf_pid_map_name(dso_name))
> >   return NULL;
> >
> >   return dso_name;
>

Re: [PATCH V4 1/3] tools/perf: Fix the string match for "/tmp/perf-$PID.map" files in dso__load

[Namhyung Kim]

Hello,

On Tue, Jun 25, 2024 at 5:02 AM Adrian Hunter  wrote:
>
> On 25/06/24 14:57, Adrian Hunter wrote:
> > On 23/06/24 09:48, Athira Rajeev wrote:
> >> Perf test for perf probe of function from different CU fails
> >> as below:
> >>
> >>  ./perf test -vv "test perf probe of function from different CU"
> >>  116: test perf probe of function from different CU:
> >>  --- start ---
> >>  test child forked, pid 2679
> >>  Failed to find symbol foo in 
> >> /tmp/perf-uprobe-different-cu-sh.Msa7iy89bx/testfile
> >>Error: Failed to add events.
> >>  --- Cleaning up ---
> >>  "foo" does not hit any event.
> >>Error: Failed to delete events.
> >>   end(-1) 
> >>  116: test perf probe of function from different CU   
> >> : FAILED!
> >>
> >> The test does below to probe function "foo" :
> >>
> >>  # gcc -g -Og -flto -c 
> >> /tmp/perf-uprobe-different-cu-sh.XniNxNEVT7/testfile-foo.c
> >>  -o /tmp/perf-uprobe-different-cu-sh.XniNxNEVT7/testfile-foo.o
> >>  # gcc -g -Og -c 
> >> /tmp/perf-uprobe-different-cu-sh.XniNxNEVT7/testfile-main.c
> >>  -o /tmp/perf-uprobe-different-cu-sh.XniNxNEVT7/testfile-main.o
> >>  # gcc -g -Og -o /tmp/perf-uprobe-different-cu-sh.XniNxNEVT7/testfile
> >>  /tmp/perf-uprobe-different-cu-sh.XniNxNEVT7/testfile-foo.o
> >>  /tmp/perf-uprobe-different-cu-sh.XniNxNEVT7/testfile-main.o
> >>
> >>  # ./perf probe -x 
> >> /tmp/perf-uprobe-different-cu-sh.XniNxNEVT7/testfile foo
> >>  Failed to find symbol foo in 
> >> /tmp/perf-uprobe-different-cu-sh.XniNxNEVT7/testfile
> >> Error: Failed to add events.
> >>
> >> Perf probe fails to find symbol foo in the executable placed in
> >> /tmp/perf-uprobe-different-cu-sh.XniNxNEVT7
> >>
> >> Simple reproduce:
> >>
> >>  # mktemp -d /tmp/perf-checkXX
> >>/tmp/perf-checkcWpuLRQI8j
> >>
> >>  # gcc -g -o test test.c
> >>  # cp test /tmp/perf-checkcWpuLRQI8j/
> >>  # nm /tmp/perf-checkcWpuLRQI8j/test | grep foo
> >>16bc T foo
> >>
> >>  # ./perf probe -x /tmp/perf-checkcWpuLRQI8j/test foo
> >>Failed to find symbol foo in /tmp/perf-checkcWpuLRQI8j/test
> >>   Error: Failed to add events.
> >>
> >> But it works with any files like /tmp/perf/test. Only for
> >> patterns with "/tmp/perf-", this fails.
> >>
> >> Further debugging, commit 80d496be89ed ("perf report: Add support
> >> for profiling JIT generated code") added support for profiling JIT
> >> generated code. This patch handles dso's of form
> >> "/tmp/perf-$PID.map" .
> >>
> >> The check used "if (strncmp(self->name, "/tmp/perf-", 10) == 0)"
> >> to match "/tmp/perf-$PID.map". With this commit, any dso in
> >> /tmp/perf- folder will be considered separately for processing
> >> (not only JIT created map files ). Fix this by changing the
> >> string pattern to check for "/tmp/perf-%d.map". Add a helper
> >> function is_perf_pid_map_name to do this check. In "struct dso",
> >> dso->long_name holds the long name of the dso file. Since the
> >> /tmp/perf-$PID.map check uses the complete name, use dso___long_name for
> >> the string name.
> >>
> >> With the fix,
> >>  # ./perf test "test perf probe of function from different CU"
> >>  117: test perf probe of function from different CU   
> >> : Ok
> >>
> >> Signed-off-by: Athira Rajeev
> >
> > Reviewed-by: Adrian Hunter 
> >
>
> Although it could use a Fixes tag
>

Thanks, I will add

Fixes: 56cbeacf1435 ("perf probe: Add test for regression introduced
by switch to die_get_decl_file()")

Namhyung

Re: [V4 14/16] tools/perf: Add support to find global register variables using find_data_type_global_reg

[Namhyung Kim]

On Fri, Jun 14, 2024 at 10:56:29PM +0530, Athira Rajeev wrote:
> There are cases where define a global register variable and associate it
> with a specified register. Example, in powerpc, two registers are
> defined to represent variable:
> 1. r13: represents local_paca
> register struct paca_struct *local_paca asm("r13");
> 
> 2. r1: represents stack_pointer
> register void *__stack_pointer asm("r1");
> 
> These regs are present in dwarf debug as DW_OP_reg as part of variables
> in the cu_die (compile unit). These are not present in die search done
> in the list of nested scopes since these are global register variables.
> 
> Example for local_paca represented by r13:
> 
> <<>>
>  <1><18dc6b4>: Abbrev Number: 128 (DW_TAG_variable)
> <18dc6b6>   DW_AT_name: (indirect string, offset: 0x3861): 
> local_paca
> <18dc6ba>   DW_AT_decl_file   : 48
> <18dc6bb>   DW_AT_decl_line   : 36
> <18dc6bc>   DW_AT_decl_column : 30
> <18dc6bd>   DW_AT_type: <0x18dc6c3>
> <18dc6c1>   DW_AT_external: 1
> <18dc6c1>   DW_AT_location: 1 byte block: 5d(DW_OP_reg13 (r13))
> 
>  <1><18dc6c3>: Abbrev Number: 3 (DW_TAG_pointer_type)
> <18dc6c4>   DW_AT_byte_size   : 8
> <18dc6c4>   DW_AT_type: <0x18dc353>
> 
> Where  DW_AT_type : <0x18dc6c3> further points to :
> 
>  <1><18dc6c3>: Abbrev Number: 3 (DW_TAG_pointer_type)
> <18dc6c4>   DW_AT_byte_size   : 8
> <18dc6c4>   DW_AT_type: <0x18dc353>
> 
> which belongs to:
> 
>  <1><18dc353>: Abbrev Number: 67 (DW_TAG_structure_type)
> <18dc354>   DW_AT_name: (indirect string, offset: 0x56cd): 
> paca_struct
> <18dc358>   DW_AT_byte_size   : 2944
> <18dc35a>   DW_AT_alignment   : 128
> <18dc35b>   DW_AT_decl_file   : 48
> <18dc35c>   DW_AT_decl_line   : 61
> <18dc35d>   DW_AT_decl_column : 8
> <18dc35d>   DW_AT_sibling : <0x18dc6b4>
> <<>>
> 
> Similar is case with "r1".
> 
> <<>>
>  <1><18dd772>: Abbrev Number: 129 (DW_TAG_variable)
> <18dd774>   DW_AT_name: (indirect string, offset: 0x11ba): 
> current_stack_pointer
> <18dd778>   DW_AT_decl_file   : 51
> <18dd779>   DW_AT_decl_line   : 1468
> <18dd77b>   DW_AT_decl_column : 24
> <18dd77c>   DW_AT_type: <0x18da5cd>
> <18dd780>   DW_AT_external: 1
> <18dd780>   DW_AT_location: 1 byte block: 51(DW_OP_reg1 (r1))
> 
>  where 18da5cd is:
> 
>  <1><18da5cd>: Abbrev Number: 47 (DW_TAG_base_type)
> <18da5ce>   DW_AT_byte_size   : 8
> <18da5cf>   DW_AT_encoding: 7   (unsigned)
> <18da5d0>   DW_AT_name: (indirect string, offset: 0x55c7): long 
> unsigned int
> <<>>
> 
> To identify data type for these two special cases, iterate over
> variables in the CU die (Compile Unit) and match it with the register.
> If the variable is a base type, ie die_get_real_type will return NULL
> here, set offset to zero. With the changes, data type for "paca_struct"
> and "long unsigned int" for r1 is identified.
> 
> Snippet from ./perf report -s type,type_off
> 
> 12.85%  long unsigned int  long unsigned int +0 (no field)
>  4.68%  struct paca_struct  struct paca_struct +2312 (__current)
>  4.57%  struct paca_struct  struct paca_struct +2354 (irq_soft_mask)
> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/util/annotate-data.c  | 40 
>  tools/perf/util/annotate.c   |  8 ++
>  tools/perf/util/annotate.h   |  1 +
>  tools/perf/util/include/dwarf-regs.h |  1 +
>  4 files changed, 50 insertions(+)
> 
> diff --git a/tools/perf/util/annotate-data.c b/tools/perf/util/annotate-data.c
> index 734acdd8c4b7..82232f2d8e16 100644
> --- a/tools/perf/util/annotate-data.c
> +++ b/tools/perf/util/annotate-data.c
> @@ -1170,6 +1170,40 @@ static int find_data_type_block(struct data_loc_info 
> *dloc,
>   return ret;
>  }
>  
> +/*
> + * Handle cases where define a global register variable and
> + * associate it with a specified register. These regs are
> + * present in dwarf debug as DW_OP_reg as part of variables
> + * in the cu_die (compile unit). Iterate over variables in the
> + * cu_die and match with reg to identify data type die.

Ok, if they always point to the same type, you may cache the result and
avoid the repeated search everytime.

> + */
> +static int find_data_type_global_reg(struct data_loc_info *dloc, int reg, 
> Dwarf_Die *cu_die,
> + Dwarf_Die *type_die)
> +{
> + Dwarf_Die vr_die;
> + int ret = -1;
> + struct die_var_type *var_types = NULL;
> +
> + die_collect_vars(cu_die, _types);
> + while (var_types) {
> + if (var_types->reg == reg) {
> + if (dwarf_offdie(dloc->di->dbg, var_types->die_off, 
> _die)) {
> + if (die_get_real_type(_die, type_die) == 
> NULL) {
> + dloc->type_offset = 0;
> + dwarf_offdie(dloc->di->dbg, 
> 

Re: [V4 13/16] tools/perf: Add support to use libcapstone in powerpc

[Namhyung Kim]

On Fri, Jun 14, 2024 at 10:56:28PM +0530, Athira Rajeev wrote:
> Now perf uses the capstone library to disassemble the instructions in
> x86. capstone is used (if available) for perf annotate to speed up.
> Currently it only supports x86 architecture. Patch includes changes to
> enable this in powerpc. For now, only for data type sort keys, this
> method is used and only binary code (raw instruction) is read. This is
> because powerpc approach to understand instructions and reg fields uses
> raw instruction. The "cs_disasm" is currently not enabled. While
> attempting to do cs_disasm, observation is that some of the instructions
> were not identified (ex: extswsli, maddld) and it had to fallback to use
> objdump. Hence enabling "cs_disasm" is added in comment section as a
> TODO for powerpc.

Well.. I'm not sure if I understand it correctly but it seems this
function effectively does nothing more than the raw disassemble.
Can we simply drop this patch for now?  Or did I miss something?

Thanks,
Namhyung

> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/util/disasm.c | 143 +++
>  1 file changed, 143 insertions(+)
> 
> diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
> index 43743ca4bdc9..987bff9f71c3 100644
> --- a/tools/perf/util/disasm.c
> +++ b/tools/perf/util/disasm.c
> @@ -1592,6 +1592,144 @@ static void print_capstone_detail(cs_insn *insn, char 
> *buf, size_t len,
>   }
>  }
>  
> +static int symbol__disassemble_capstone_powerpc(char *filename, struct 
> symbol *sym,
> + struct annotate_args *args)
> +{
> + struct annotation *notes = symbol__annotation(sym);
> + struct map *map = args->ms.map;
> + struct dso *dso = map__dso(map);
> + struct nscookie nsc;
> + u64 start = map__rip_2objdump(map, sym->start);
> + u64 end = map__rip_2objdump(map, sym->end);
> + u64 len = end - start;
> + u64 offset;
> + int i, fd, count;
> + bool is_64bit = false;
> + bool needs_cs_close = false;
> + u8 *buf = NULL;
> + struct find_file_offset_data data = {
> + .ip = start,
> + };
> + csh handle;
> + char disasm_buf[512];
> + struct disasm_line *dl;
> + u32 *line;
> + bool disassembler_style = false;
> +
> + if (args->options->objdump_path)
> + return -1;
> +
> + nsinfo__mountns_enter(dso->nsinfo, );
> + fd = open(filename, O_RDONLY);
> + nsinfo__mountns_exit();
> + if (fd < 0)
> + return -1;
> +
> + if (file__read_maps(fd, /*exe=*/true, find_file_offset, ,
> + _64bit) == 0)
> + goto err;
> +
> + if (!args->options->disassembler_style ||
> + !strcmp(args->options->disassembler_style, "att"))
> + disassembler_style = true;
> +
> + if (capstone_init(maps__machine(args->ms.maps), , is_64bit, 
> disassembler_style) < 0)
> + goto err;
> +
> + needs_cs_close = true;
> +
> + buf = malloc(len);
> + if (buf == NULL)
> + goto err;
> +
> + count = pread(fd, buf, len, data.offset);
> + close(fd);
> + fd = -1;
> +
> + if ((u64)count != len)
> + goto err;
> +
> + line = (u32 *)buf;
> +
> + /* add the function address and name */
> + scnprintf(disasm_buf, sizeof(disasm_buf), "%#"PRIx64" <%s>:",
> +   start, sym->name);
> +
> + args->offset = -1;
> + args->line = disasm_buf;
> + args->line_nr = 0;
> + args->fileloc = NULL;
> + args->ms.sym = sym;
> +
> + dl = disasm_line__new(args);
> + if (dl == NULL)
> + goto err;
> +
> + annotation_line__add(>al, >src->source);
> +
> + /*
> +  * TODO: enable disassm for powerpc
> +  * count = cs_disasm(handle, buf, len, start, len, );
> +  *
> +  * For now, only binary code is saved in disassembled line
> +  * to be used in "type" and "typeoff" sort keys. Each raw code
> +  * is 32 bit instruction. So use "len/4" to get the number of
> +  * entries.
> +  */
> + count = len/4;
> +
> + for (i = 0, offset = 0; i < count; i++) {
> + args->offset = offset;
> + sprintf(args->line, "%x", line[i]);
> +
> + dl = disasm_line__new(args);
> + if (dl == NULL)
> + goto err;
> +
> + annotation_line__add(>al, >src->source);
> +
> + offset += 4;
> + }
> +
> + /* It failed in the middle */
> + if (offset != len) {
> + struct list_head *list = >src->source;
> +
> + /* Discard all lines and fallback to objdump */
> + while (!list_empty(list)) {
> + dl = list_first_entry(list, struct disasm_line, 
> al.node);
> +
> + list_del_init(>al.node);
> + disasm_line__free(dl);
> + }
> + count = -1;
> + }
> +
> +out:
> + if 

Re: [V4 06/16] tools/perf: Update parameters for reg extract functions to use raw instruction on powerpc

[Namhyung Kim]

On Fri, Jun 14, 2024 at 10:56:21PM +0530, Athira Rajeev wrote:
> Use the raw instruction code and macros to identify memory instructions,
> extract register fields and also offset. The implementation addresses
> the D-form, X-form, DS-form instructions. Two main functions are added.
> New parse function "load_store__parse" as instruction ops parser for
> memory instructions. Unlink other parser (like mov__parse), this parser
> fills in the "multi_regs" field for source/target and new added "mem_ref"
> field. No other fields are set because, here there is no need to parse the
> disassembled code and arch specific macros will take care of extracting
> offset and regs which is easier and will be precise.
> 
> In powerpc, all instructions with a primary opcode from 32 to 63
> are memory instructions. Update "ins__find" function to have "raw_insn"
> also as a parameter. Don't use the "extract_reg_offset", instead use
> newly added function "get_arch_regs" which will set these fields: reg1,
> reg2, offset depending of where it is source or target ops.
> 
> Update "parse" callback for "struct ins_ops" to also pass "struct
> disasm_line" as argument. This is needed in parse functions where opcode
> is used to determine whether to set multi_regs.

Can you please split "ins__find" change and "parse" change into separate
commits?

> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/arch/arm64/annotate/instructions.c |  3 +-
>  .../arch/loongarch/annotate/instructions.c|  6 +-
>  .../perf/arch/powerpc/annotate/instructions.c | 16 
>  tools/perf/arch/powerpc/util/dwarf-regs.c | 44 +++
>  tools/perf/arch/s390/annotate/instructions.c  |  5 +-
>  tools/perf/util/annotate.c| 25 ++-
>  tools/perf/util/disasm.c  | 73 ---
>  tools/perf/util/disasm.h  |  6 +-
>  tools/perf/util/include/dwarf-regs.h  |  3 +
>  9 files changed, 159 insertions(+), 22 deletions(-)
> 
> diff --git a/tools/perf/arch/arm64/annotate/instructions.c 
> b/tools/perf/arch/arm64/annotate/instructions.c
> index 4af0c3a0f86e..f86d9f4798bd 100644
> --- a/tools/perf/arch/arm64/annotate/instructions.c
> +++ b/tools/perf/arch/arm64/annotate/instructions.c
> @@ -11,7 +11,8 @@ struct arm64_annotate {
>  
>  static int arm64_mov__parse(struct arch *arch __maybe_unused,
>   struct ins_operands *ops,
> - struct map_symbol *ms __maybe_unused)
> + struct map_symbol *ms __maybe_unused,
> + struct disasm_line *dl __maybe_unused)
>  {
>   char *s = strchr(ops->raw, ','), *target, *endptr;
>  
> diff --git a/tools/perf/arch/loongarch/annotate/instructions.c 
> b/tools/perf/arch/loongarch/annotate/instructions.c
> index 21cc7e4149f7..ab43b1ab51e3 100644
> --- a/tools/perf/arch/loongarch/annotate/instructions.c
> +++ b/tools/perf/arch/loongarch/annotate/instructions.c
> @@ -5,7 +5,8 @@
>   * Copyright (C) 2020-2023 Loongson Technology Corporation Limited
>   */
>  
> -static int loongarch_call__parse(struct arch *arch, struct ins_operands 
> *ops, struct map_symbol *ms)
> +static int loongarch_call__parse(struct arch *arch, struct ins_operands 
> *ops, struct map_symbol *ms,
> + struct disasm_line *dl __maybe_unused)
>  {
>   char *c, *endptr, *tok, *name;
>   struct map *map = ms->map;
> @@ -51,7 +52,8 @@ static struct ins_ops loongarch_call_ops = {
>   .scnprintf = call__scnprintf,
>  };
>  
> -static int loongarch_jump__parse(struct arch *arch, struct ins_operands 
> *ops, struct map_symbol *ms)
> +static int loongarch_jump__parse(struct arch *arch, struct ins_operands 
> *ops, struct map_symbol *ms,
> + struct disasm_line *dl __maybe_unused)
>  {
>   struct map *map = ms->map;
>   struct symbol *sym = ms->sym;
> diff --git a/tools/perf/arch/powerpc/annotate/instructions.c 
> b/tools/perf/arch/powerpc/annotate/instructions.c
> index d57fd023ef9c..10fea5e5cf4c 100644
> --- a/tools/perf/arch/powerpc/annotate/instructions.c
> +++ b/tools/perf/arch/powerpc/annotate/instructions.c
> @@ -49,6 +49,22 @@ static struct ins_ops 
> *powerpc__associate_instruction_ops(struct arch *arch, con
>   return ops;
>  }
>  
> +#define PPC_OP(op)  (((op) >> 26) & 0x3F)
> +
> +static struct ins_ops *check_ppc_insn(int raw_insn)

It'd be nice to use 'u32' instead of 'int' for raw_insn if you want to
do some bit operations.

> +{
> + int opcode = PPC_OP(raw_insn);
> +
> + /*
> +  * Instructions with opcode 32 to 63 are memory
> +  * instructions in powerpc
> +  */
> + if ((opcode & 0x20))
> + return _store_ops;
> +
> + return NULL;
> +}
> +
>  static int powerpc__annotate_init(struct arch *arch, char *cpuid 
> __maybe_unused)
>  {
>   if (!arch->initialized) {
> diff --git a/tools/perf/arch/powerpc/util/dwarf-regs.c 
> b/tools/perf/arch/powerpc/util/dwarf-regs.c
> index 

Re: [V4 05/16] tools/perf: Add disasm_line__parse to parse raw instruction for powerpc

[Namhyung Kim]

On Fri, Jun 14, 2024 at 10:56:20PM +0530, Athira Rajeev wrote:
> Currently, the perf tool infrastructure disasm_line__parse function to
> parse disassembled line.
> 
> Example snippet from objdump:
> objdump  --start-address= --stop-address=  -d 
> --no-show-raw-insn -C 
> 
> c10224b4: lwz r10,0(r9)
> 
> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> registers names and offset. In powerpc, the approach for data type
> profiling uses raw instruction instead of result from objdump to identify
> the instruction category and extract the source/target registers.
> 
> Example: 38 01 81 e8 ld  r4,312(r1)
> 
> Here "38 01 81 e8" is the raw instruction representation. Add function
> "disasm_line__parse_powerpc" to handle parsing of raw instruction.
> Also update "struct disasm_line" to save the binary code/
> With the change, function captures:
> 
> line -> "38 01 81 e8 ld  r4,312(r1)"
> raw instruction "38 01 81 e8"
> 
> Raw instruction is used later to extract the reg/offset fields. Macros
> are added to extract opcode and register fields. "struct disasm_line"
> is updated to carry union of "bytes" and "raw_insn" of 32 bit to carry raw
> code (raw). Function "disasm_line__parse_powerpc fills the raw
> instruction hex value and can use macros to get opcode. There is no
> changes in existing code paths, which parses the disassembled code.
> The architecture using the instruction name and present approach is
> not altered. Since this approach targets powerpc, the macro
> implementation is added for powerpc as of now.
> 
> Since the disasm_line__parse is used in other cases (perf annotate) and
> not only data tye profiling, the powerpc callback includes changes to
> work with binary code as well as mneumonic representation. Also in case
> if the DSO read fails and libcapstone is not supported, the approach
> fallback to use objdump as option. Hence as option, patch has changes to
> ensure objdump option also works well.
> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/include/linux/string.h  |  2 +
>  tools/lib/string.c| 13 
>  .../perf/arch/powerpc/annotate/instructions.c |  1 +
>  tools/perf/arch/powerpc/util/dwarf-regs.c |  9 +++
>  tools/perf/util/annotate.h|  5 +-
>  tools/perf/util/disasm.c  | 59 ++-
>  6 files changed, 87 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/include/linux/string.h b/tools/include/linux/string.h
> index db5c99318c79..0acb1fc14e19 100644
> --- a/tools/include/linux/string.h
> +++ b/tools/include/linux/string.h
> @@ -46,5 +46,7 @@ extern char * __must_check skip_spaces(const char *);
>  
>  extern char *strim(char *);
>  
> +extern void remove_spaces(char *s);
> +
>  extern void *memchr_inv(const void *start, int c, size_t bytes);
>  #endif /* _TOOLS_LINUX_STRING_H_ */
> diff --git a/tools/lib/string.c b/tools/lib/string.c
> index 8b6892f959ab..3126d2cff716 100644
> --- a/tools/lib/string.c
> +++ b/tools/lib/string.c
> @@ -153,6 +153,19 @@ char *strim(char *s)
>   return skip_spaces(s);
>  }
>  
> +/*
> + * remove_spaces - Removes whitespaces from @s
> + */
> +void remove_spaces(char *s)
> +{
> + char *d = s;
> +
> + do {
> + while (*d == ' ')
> + ++d;
> + } while ((*s++ = *d++));
> +}
> +
>  /**
>   * strreplace - Replace all occurrences of character in string.
>   * @s: The string to operate on.
> diff --git a/tools/perf/arch/powerpc/annotate/instructions.c 
> b/tools/perf/arch/powerpc/annotate/instructions.c
> index a3f423c27cae..d57fd023ef9c 100644
> --- a/tools/perf/arch/powerpc/annotate/instructions.c
> +++ b/tools/perf/arch/powerpc/annotate/instructions.c
> @@ -55,6 +55,7 @@ static int powerpc__annotate_init(struct arch *arch, char 
> *cpuid __maybe_unused)
>   arch->initialized = true;
>   arch->associate_instruction_ops = 
> powerpc__associate_instruction_ops;
>   arch->objdump.comment_char  = '#';
> + annotate_opts.show_asm_raw = true;

Right, I think this will add the raw insn in the output of objdump, no?
Why not using the information?

>   }
>  
>   return 0;
> diff --git a/tools/perf/arch/powerpc/util/dwarf-regs.c 
> b/tools/perf/arch/powerpc/util/dwarf-regs.c
> index 0c4f4caf53ac..430623ca5612 100644
> --- a/tools/perf/arch/powerpc/util/dwarf-regs.c
> +++ b/tools/perf/arch/powerpc/util/dwarf-regs.c
> @@ -98,3 +98,12 @@ int regs_query_register_offset(const char *name)
>   return roff->ptregs_offset;
>   return -EINVAL;
>  }
> +
> +#define PPC_OP(op)   (((op) >> 26) & 0x3F)
> +#define PPC_RA(a)(((a) >> 16) & 0x1f)
> +#define PPC_RT(t)(((t) >> 21) & 0x1f)
> +#define PPC_RB(b)(((b) >> 11) & 0x1f)
> +#define PPC_D(D) ((D) & 0xfffe)
> +#define PPC_DS(DS)   ((DS) & 0xfffc)
> +#define OP_LD58
> +#define OP_STD   62
> diff --git a/tools/perf/util/annotate.h 

Re: [V4 04/16] tools/perf: Use sort keys to determine whether to pick objdump to disassemble

[Namhyung Kim]

On Fri, Jun 14, 2024 at 10:56:19PM +0530, Athira Rajeev wrote:
> perf annotate can be done in different ways. One way is to directly use
> "perf annotate" command, other way to annotate specific symbol is to do
> "perf report" and press "a" on the sample in UI mode. The approach
> preferred in powerpc to parse sample for data type profiling is:
> - Read directly from DSO using dso__data_read_offset
> - If that fails for any case, fallback to using libcapstone
> - If libcapstone is not supported, approach will use objdump
> 
> The above works well when perf report is invoked with only sort keys for
> data type ie type and typeoff. Because there is no instruction level
> annotation needed if only data type information is requested for. For
> annotating sample, along with type and typeoff sort key, "sym" sort key
> is also needed. And by default invoking just "perf report" uses sort key
> "sym" that displays the symbol information.
> 
> With approach changes in powerpc which first reads DSO for raw
> instruction, "perf annotate" and "perf report" + a key breaks since
> it doesn't do the instruction level disassembly.

So as I said, it'd be nice you can read the raw insn from the objdump
output directly.

Thanks,
Namhyung

> 
> Snippet of result from perf report:
> 
> Samples: 1K of event 'mem-loads', 4000 Hz, Event count (approx.): 937238
> do_work  /usr/bin/pmlogger [Percent: local period]
> Percent│ea230010
>│3a550010
>│3a60
> 
>│38f60001
>│39490008
>│42400438
>  51.44 │81290008
>│7d485378
> 
> Here, raw instruction is displayed in the output instead of human
> readable annotated form.
> 
> One way to get the appropriate data is to specify "--objdump path", by
> which code annotation will be done. But the default behaviour will be
> changed. To fix this breakage, check if "sym" sort key is set. If so
> fallback and use the libcapstone/objdump way of disassmbling the sample.
> 
> With the changes and "perf report"
> 
> Samples: 1K of event 'mem-loads', 4000 Hz, Event count (approx.): 937238
> do_work  /usr/bin/pmlogger [Percent: local period]
> Percent│ldr17,16(r3)
>│addi  r18,r21,16
>│lir19,0
> 
>│ 8b0:   rldiclr10,r10,63,33
>│addi  r10,r10,1
>│mtctr r10
>│  ↓ b 8e4
>│ 8c0:   addi  r7,r22,1
>│addi  r10,r9,8
>│  ↓ bdz   d00
>  51.44 │lwz   r9,8(r9)
>│mrr8,r10
>│cmpw  r20,r9
> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/util/disasm.c | 9 ++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
> index f19496133bf0..b81cdcf4d6b4 100644
> --- a/tools/perf/util/disasm.c
> +++ b/tools/perf/util/disasm.c
> @@ -25,6 +25,7 @@
>  #include "srcline.h"
>  #include "symbol.h"
>  #include "util.h"
> +#include "sort.h"
>  
>  static regex_tfile_lineno;
>  
> @@ -1803,9 +1804,11 @@ int symbol__disassemble(struct symbol *sym, struct 
> annotate_args *args)
>* not required in case of powerpc.
>*/
>   if (arch__is(args->arch, "powerpc")) {
> - err = symbol__disassemble_dso(symfs_filename, sym, args);
> - if (err == 0)
> - goto out_remove_tmp;
> + if (sort_order && !strstr(sort_order, "sym")) {
> + err = symbol__disassemble_dso(symfs_filename, sym, 
> args);
> + if (err == 0)
> + goto out_remove_tmp;
> + }
>   }
>  
>  #ifdef HAVE_LIBCAPSTONE_SUPPORT
> -- 
> 2.43.0
> 

Re: [V4 03/16] tools/perf: Add support to capture and parse raw instruction in powerpc using dso__data_read_offset utility

[Namhyung Kim]

On Fri, Jun 14, 2024 at 10:56:18PM +0530, Athira Rajeev wrote:
> Add support to capture and parse raw instruction in powerpc.
> Currently, the perf tool infrastructure uses two ways to disassemble
> and understand the instruction. One is objdump and other option is
> via libcapstone.
> 
> Currently, the perf tool infrastructure uses "--no-show-raw-insn" option
> with "objdump" while disassemble. Example from powerpc with this option
> for an instruction address is:
> 
> Snippet from:
> objdump  --start-address= --stop-address=  -d 
> --no-show-raw-insn -C 
> 
> c10224b4: lwz r10,0(r9)

What about removing --no-show-raw-insn and parse the raw byte code in
the output for powerpc?  I think it's better to support normal
annotation together.

> 
> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> registers names and offset. Also to find whether there is a memory
> reference in the operands, "memory_ref_char" field of objdump is used.
> For x86, "(" is used as memory_ref_char to tackle instructions of the
> form "mov  (%rax), %rcx".
> 
> In case of powerpc, not all instructions using "(" are the only memory
> instructions. Example, above instruction can also be of extended form (X
> form) "lwzx r10,0,r19". Inorder to easy identify the instruction category
> and extract the source/target registers, patch adds support to use raw
> instruction for powerpc. Approach used is to read the raw instruction
> directly from the DSO file using "dso__data_read_offset" utility which
> is already implemented in perf infrastructure in "util/dso.c".
> 
> Example:
> 
> 38 01 81 e8 ld  r4,312(r1)
> 
> Here "38 01 81 e8" is the raw instruction representation. In powerpc,
> this translates to instruction form: "ld RT,DS(RA)" and binary code
> as:
> 
>| 58 |  RT  |  RA |  DS   | |
>-
>06 1116  30 31
> 
> Function "symbol__disassemble_dso" is updated to read raw instruction
> directly from DSO using dso__data_read_offset utility. In case of
> above example, this captures:
> line:38 01 81 e8
> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/util/disasm.c | 98 
>  1 file changed, 98 insertions(+)
> 
> diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
> index b5fe3a7508bb..f19496133bf0 100644
> --- a/tools/perf/util/disasm.c
> +++ b/tools/perf/util/disasm.c
> @@ -1586,6 +1586,91 @@ static int symbol__disassemble_capstone(char 
> *filename, struct symbol *sym,
>  }
>  #endif
>  
> +static int symbol__disassemble_dso(char *filename, struct symbol *sym,

Maybe rename to symbol__disassemble_raw() ?

> + struct annotate_args *args)
> +{
> + struct annotation *notes = symbol__annotation(sym);
> + struct map *map = args->ms.map;
> + struct dso *dso = map__dso(map);
> + u64 start = map__rip_2objdump(map, sym->start);
> + u64 end = map__rip_2objdump(map, sym->end);
> + u64 len = end - start;
> + u64 offset;
> + int i, count;
> + u8 *buf = NULL;
> + char disasm_buf[512];
> + struct disasm_line *dl;
> + u32 *line;
> +
> + /* Return if objdump is specified explicitly */
> + if (args->options->objdump_path)
> + return -1;
> +
> + pr_debug("Reading raw instruction from : %s using 
> dso__data_read_offset\n", filename);

You may want to print the actual offset and remove the "using
dso__data_read_offset" part.

Thanks,
Namhyung

> +
> + buf = malloc(len);
> + if (buf == NULL)
> + goto err;
> +
> + count = dso__data_read_offset(dso, NULL, sym->start, buf, len);
> +
> + line = (u32 *)buf;
> +
> + if ((u64)count != len)
> + goto err;
> +
> + /* add the function address and name */
> + scnprintf(disasm_buf, sizeof(disasm_buf), "%#"PRIx64" <%s>:",
> +   start, sym->name);
> +
> + args->offset = -1;
> + args->line = disasm_buf;
> + args->line_nr = 0;
> + args->fileloc = NULL;
> + args->ms.sym = sym;
> +
> + dl = disasm_line__new(args);
> + if (dl == NULL)
> + goto err;
> +
> + annotation_line__add(>al, >src->source);
> +
> + /* Each raw instruction is 4 byte */
> + count = len/4;
> +
> + for (i = 0, offset = 0; i < count; i++) {
> + args->offset = offset;
> + sprintf(args->line, "%x", line[i]);
> + dl = disasm_line__new(args);
> + if (dl == NULL)
> + goto err;
> +
> + annotation_line__add(>al, >src->source);
> + offset += 4;
> + }
> +
> + /* It failed in the middle */
> + if (offset != len) {
> + struct list_head *list = >src->source;
> +
> + /* Discard all lines and fallback to objdump */
> + while (!list_empty(list)) {
> + dl = list_first_entry(list, struct disasm_line, 
> al.node);
> +
> +

Re: [V4 01/16] tools/perf: Move the data structures related to register type to header file

[Namhyung Kim]

Hello,

On Fri, Jun 14, 2024 at 10:56:16PM +0530, Athira Rajeev wrote:
> Data type profiling uses instruction tracking by checking each
> instruction and updating the register type state in some data
> structures. This is useful to find the data type in cases when the
> register state gets transferred from one reg to another. Example, in
> x86, "mov" instruction and in powerpc, "mr" instruction. Currently these
> structures are defined in annotate-data.c and instruction tracking is
> implemented only for x86. Move these data structures to
> "annotate-data.h" header file so that other arch implementations can use
> it in arch specific files as well.
> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/util/annotate-data.c | 53 +--
>  tools/perf/util/annotate-data.h | 55 +
>  2 files changed, 56 insertions(+), 52 deletions(-)
> 
> diff --git a/tools/perf/util/annotate-data.c b/tools/perf/util/annotate-data.c
> index 965da6c0b542..a4c7f98a75e3 100644
> --- a/tools/perf/util/annotate-data.c
> +++ b/tools/perf/util/annotate-data.c
> @@ -31,15 +31,6 @@
>  
>  static void delete_var_types(struct die_var_type *var_types);
>  
> -enum type_state_kind {
> - TSR_KIND_INVALID = 0,
> - TSR_KIND_TYPE,
> - TSR_KIND_PERCPU_BASE,
> - TSR_KIND_CONST,
> - TSR_KIND_POINTER,
> - TSR_KIND_CANARY,
> -};
> -
>  #define pr_debug_dtp(fmt, ...)   \
>  do { \
>   if (debug_type_profile) \
> @@ -140,49 +131,7 @@ static void pr_debug_location(Dwarf_Die *die, u64 pc, 
> int reg)
>   }
>  }
>  
> -/*
> - * Type information in a register, valid when @ok is true.
> - * The @caller_saved registers are invalidated after a function call.
> - */
> -struct type_state_reg {
> - Dwarf_Die type;
> - u32 imm_value;
> - bool ok;
> - bool caller_saved;
> - u8 kind;
> -};
> -
> -/* Type information in a stack location, dynamically allocated */
> -struct type_state_stack {
> - struct list_head list;
> - Dwarf_Die type;
> - int offset;
> - int size;
> - bool compound;
> - u8 kind;
> -};
> -
> -/* FIXME: This should be arch-dependent */
> -#define TYPE_STATE_MAX_REGS  16
> -
> -/*
> - * State table to maintain type info in each register and stack location.
> - * It'll be updated when new variable is allocated or type info is moved
> - * to a new location (register or stack).  As it'd be used with the
> - * shortest path of basic blocks, it only maintains a single table.
> - */
> -struct type_state {
> - /* state of general purpose registers */
> - struct type_state_reg regs[TYPE_STATE_MAX_REGS];
> - /* state of stack location */
> - struct list_head stack_vars;
> - /* return value register */
> - int ret_reg;
> - /* stack pointer register */
> - int stack_reg;
> -};
> -
> -static bool has_reg_type(struct type_state *state, int reg)
> +bool has_reg_type(struct type_state *state, int reg)
>  {
>   return (unsigned)reg < ARRAY_SIZE(state->regs);
>  }
> diff --git a/tools/perf/util/annotate-data.h b/tools/perf/util/annotate-data.h
> index 0a57d9f5ee78..ef235b1b15e1 100644
> --- a/tools/perf/util/annotate-data.h
> +++ b/tools/perf/util/annotate-data.h
> @@ -6,6 +6,9 @@
>  #include 
>  #include 
>  #include 
> +#include "dwarf-aux.h"
> +#include "annotate.h"
> +#include "debuginfo.h"
>  
>  struct annotated_op_loc;
>  struct debuginfo;
> @@ -15,6 +18,15 @@ struct hist_entry;
>  struct map_symbol;
>  struct thread;
>  
> +enum type_state_kind {
> + TSR_KIND_INVALID = 0,
> + TSR_KIND_TYPE,
> + TSR_KIND_PERCPU_BASE,
> + TSR_KIND_CONST,
> + TSR_KIND_POINTER,
> + TSR_KIND_CANARY,
> +};
> +
>  /**
>   * struct annotated_member - Type of member field
>   * @node: List entry in the parent list
> @@ -142,6 +154,48 @@ struct annotated_data_stat {
>  };
>  extern struct annotated_data_stat ann_data_stat;
>  
> +/*
> + * Type information in a register, valid when @ok is true.
> + * The @caller_saved registers are invalidated after a function call.
> + */
> +struct type_state_reg {
> + Dwarf_Die type;
> + u32 imm_value;
> + bool ok;
> + bool caller_saved;
> + u8 kind;
> +};
> +
> +/* Type information in a stack location, dynamically allocated */
> +struct type_state_stack {
> + struct list_head list;
> + Dwarf_Die type;
> + int offset;
> + int size;
> + bool compound;
> + u8 kind;
> +};
> +
> +/* FIXME: This should be arch-dependent */
> +#define TYPE_STATE_MAX_REGS  32

Can you please define this for powerpc separately?  I think x86 should
remain in 16.

Thanks,
Namhyung

> +
> +/*
> + * State table to maintain type info in each register and stack location.
> + * It'll be updated when new variable is allocated or type info is moved
> + * to a new location (register or stack).  As it'd be used with the
> + * 

Re: [V4 00/16] Add data type profiling support for powerpc

[Namhyung Kim]

Hello,

On Thu, Jun 20, 2024 at 09:01:01PM +0530, Athira Rajeev wrote:
> 
> 
> > On 14 Jun 2024, at 10:56 PM, Athira Rajeev  
> > wrote:
> > 
> > The patchset from Namhyung added support for data type profiling
> > in perf tool. This enabled support to associate PMU samples to data
> > types they refer using DWARF debug information. With the upstream
> > perf, currently it possible to run perf report or perf annotate to
> > view the data type information on x86.
> > 
> > Initial patchset posted here had changes need to enable data type
> > profiling support for powerpc.
> > 
> > https://lore.kernel.org/all/6e09dc28-4a2e-49d8-a2b5-ffb3396a9...@csgroup.eu/T/
> > 
> > Main change were:
> > 1. powerpc instruction nmemonic table to associate load/store
> > instructions with move_ops which is use to identify if instruction
> > is a memory access one.
> > 2. To get register number and access offset from the given
> > instruction, code uses fields from "struct arch" -> objump.
> > Added entry for powerpc here.
> > 3. A get_arch_regnum to return register number from the
> > register name string.
> > 
> > But the apporach used in the initial patchset used parsing of
> > disassembled code which the current perf tool implementation does.
> > 
> > Example: lwz r10,0(r9)
> > 
> > This line "lwz r10,0(r9)" is parsed to extract instruction name,
> > registers names and offset. Also to find whether there is a memory
> > reference in the operands, "memory_ref_char" field of objdump is used.
> > For x86, "(" is used as memory_ref_char to tackle instructions of the
> > form "mov  (%rax), %rcx".
> > 
> > In case of powerpc, not all instructions using "(" are the only memory
> > instructions. Example, above instruction can also be of extended form (X
> > form) "lwzx r10,0,r19". Inorder to easy identify the instruction category
> > and extract the source/target registers, second patchset added support to 
> > use
> > raw instruction. With raw instruction, macros are added to extract opcode
> > and register fields.
> > Link to second patchset:
> > https://lore.kernel.org/all/20240506121906.76639-1-atraj...@linux.vnet.ibm.com/
> > 
> > Example representation using --show-raw-insn in objdump gives result:
> > 
> > 38 01 81 e8 ld  r4,312(r1)
> > 
> > Here "38 01 81 e8" is the raw instruction representation. In powerpc,
> > this translates to instruction form: "ld RT,DS(RA)" and binary code
> > as:
> >  _
> >  | 58 |  RT  |  RA |  DS   | |
> >  -
> > 06 1116  30 31
> > 
> > Second patchset used "objdump" again to read the raw instruction.
> > But since there is no need to disassemble and binary code can be read
> > directly from the DSO, third patchset (ie this patchset) uses below
> > apporach. The apporach preferred in powerpc to parse sample for data
> > type profiling in V3 patchset is:
> > - Read directly from DSO using dso__data_read_offset
> > - If that fails for any case, fallback to using libcapstone
> > - If libcapstone is not supported, approach will use objdump
> > 
> > Patchset adds support to pick the opcode and reg fields from this
> > raw/binary instruction code. This approach came in from review comment
> > by Segher Boessenkool and Christophe for the initial patchset.
> > 
> > Apart from that, instruction tracking is enabled for powerpc and
> > support function is added to find variables defined as registers
> > Example, in powerpc, below two registers are
> > defined to represent variable:
> > 1. r13: represents local_paca
> > register struct paca_struct *local_paca asm("r13");
> > 
> > 2. r1: represents stack_pointer
> > register void *__stack_pointer asm("r1");
> > 
> > These are handled in this patchset.
> > 
> > - Patch 1 is to rearrange register state type structures to header file
> > so that it can referred from other arch specific files
> > - Patch 2 is to make instruction tracking as a callback to"struct arch"
> > so that it can be implemented by other archs easily and defined in arch
> > specific files
> > - Patch 3 adds support to capture and parse raw instruction in powerpc
> > using dso__data_read_offset utility
> > - Patch 4 adds logic to support using objdump when doing default "perf
> > report" or "perf annotate" since it that needs disassembled instruction.
> > - Patch 5 adds disasm_line__parse to parse raw instruction for powerpc
> > - Patch 6 update parameters for reg extract functions to use raw
> > instruction on powerpc
> > - Patch 7 add support to identify memory instructions of opcode 31 in
> > powerpc
> > - Patch 8 adds more instructions to support instruction tracking in powerpc
> > - Patch 9 and 10 handles instruction tracking for powerpc.
> > - Patch 11, 12 and 13 add support to use libcapstone in powerpc
> > - Patch 14 and patch 15 handles support to find global register variables
> > - Patch 16 handles insn-stat option for perf annotate
> > 
> > Note:
> > - There are 

Re: [PATCH] tools/perf: Handle perftool-testsuite_probe testcases fail when kernel debuginfo is not present

[Namhyung Kim]

On Mon, 17 Jun 2024 17:51:21 +0530, Athira Rajeev wrote:

> Running "perftool-testsuite_probe" fails as below:
> 
>   ./perf test -v "perftool-testsuite_probe"
>   83: perftool-testsuite_probe  : FAILED
> 
> There are three fails:
> 
> [...]

Applied to perf-tools-next, thanks!

Best regards,
Namhyung

Re: [PATCH] tools/perf: Handle perftool-testsuite_probe testcases fail when kernel debuginfo is not present

[Namhyung Kim]

Hello,

Adding Veronika and Michael to CC.

Thanks,
Namhyung


On Tue, Jun 18, 2024 at 6:44 AM James Clark  wrote:
>
>
>
> On 17/06/2024 17:47, Athira Rajeev wrote:
> >
> >
> >> On 17 Jun 2024, at 8:30 PM, James Clark  wrote:
> >>
> >>
> >>
> >> On 17/06/2024 13:21, Athira Rajeev wrote:
> >>> Running "perftool-testsuite_probe" fails as below:
> >>>
> >>> ./perf test -v "perftool-testsuite_probe"
> >>> 83: perftool-testsuite_probe  : FAILED
> >>>
> >>> There are three fails:
> >>>
> >>> 1. Regexp not found: "\s*probe:inode_permission(?:_\d+)?\s+\(on 
> >>> inode_permission(?:[:\+][0-9A-Fa-f]+)?@.+\)"
> >>>   -- [ FAIL ] -- perf_probe :: test_adding_kernel :: listing added probe 
> >>> :: perf probe -l (output regexp parsing)
> >>>
> >>
> >> On a machine where NO_DEBUGINFO gets set, this one skips for me. But on
> >> a machine where there _is_ debug info this test still fails.
> >>
> >> But in both cases the probe looks like it was added successfully. So I'm
> >> wondering if this one does need to be skipped, or it's just always
> >> failing? Do you have this test passing anywhere where there is debug info?
> >>
> >> The list command looks like it successfully lists the probe for me in
> >> both cases, it just doesn't have an address on the end:
> >>
> >> perf list 'probe:*'
> >>
> >>   probe:inode_permission (on inode_permission)
> >>
> >> Does the missing address mean anything or is it just not handled
> >> properly by the test?
> >>
> >> Ironically the machine that _does_ pass the debug info test also prints
> >> this, but it looks like it still adds and lists the probe correctly:
> >>
> >>  perf probe -l probe:*
> >>
> >>  Failed to find debug information for address 0x80008047ac30
> >>probe:inode_permission (on inode_permission)
> >
> > Hi James,
> >
> > Thanks for checking this patch.
> >
> > In environment where kernel is compiled with debuginfo:
> >
> > 1) Add probe point
> >
> > # ./perf probe --add inode_permission
> > Added new event:
> >   probe:inode_permission (on inode_permission)
> >
> > You can now use it in all perf tools, such as:
> >
> > perf record -e probe:inode_permission -aR sleep 1
> >
> >
> > 2) Check using perf probe -l
> >
> > # ./perf probe -l
> > probe:inode_permission (on inode_permission:2@fs/namei.c)
> >
> > With debuginfo, the result has additional info.
> >  The test looks for matching pattern 
> > "\s*probe:inode_permission(?:_\d+)?\s+\(on 
> > inode_permission(?:[:\+][0-9A-Fa-f]+)?@.+\)” in result
> > where it is expecting "inode_permission:2@fs/namei.c” . The “@fs/namei.c” 
> > info needs debuginfo here.
> >
>
> Hi Athira,
>
> Maybe there is a real bug and this patch is ok to go in and we should leave
> it as failing. Probe -L shows there is debug info available for 
> inode_permission:
>
>$ ./perf probe -L inode_permission
>
>   
>   0  int inode_permission(struct mnt_idmap *idmap,
>  struct inode *inode, int mask)
>   ... more source code ...
>
> But probe -l has an error which could be related to the following
> line not showing the filename details:
>
>   $ ./perf probe -l
>
>   Failed to find debug information for address 0x80008047ac30
> probe:inode_permission (on inode_permission)
>
> I'm running a clang kernel and sometimes I see issues with debug
> info or toolchain stuff, that could be the reason.
>
> > The function I am using in patch to check for debuginfo 
> > (skip_if_no_debuginfo) is from "tests/shell/lib/probe_vfs_getname.sh"
> >
> > skip_if_no_debuginfo() {
> > add_probe_vfs_getname -v 2>&1 | grep -E -q "^(Failed to find the 
> > path for the kernel|Debuginfo-analysis is not supported)|(file has no debug 
> > information)" && return 2
> > return 1
> > }
> >
> > So the debuginfo test passes in your case since the log has "Failed to find 
> > debug information” which is not present in above grep string.
> >
>
> It passes because there is debug info for getname_flags() which is what the
> debug info check looks for. After some greps and seds it ultimately does this
> which succeeds:
>
>  $ perf probe "vfs_getname=getname_flags:72 pathname=result->name:string"
>
>  Added new event:
> probe:vfs_getname(on getname_flags:72 with 
> pathname=result->name:string)
>
> "Failed to find debug information for address" is only ever printed
> with "perf probe -l" when there are probes added. The stderr
> of that command is never piped into any grep anyway, which is why I
> see it on the test output.
>
> So "probe -L" is working but "probe -l" isn't. Ultimately it looks like a real
> issue and we should leave the failure in.
>
> > James,
> >
> > Only “perf probe -l” subtest fails with debuginfo enabled or other two 
> > subtests as well? Can you also share result on how other two subtests 
> > behaves ?
> >
> > 1. Fail 2 :
> >perf probe -nf --max-probes=512 -a 'vfs_* $params’
> >
> >
> > 2. Fail 3 :
> >   perf probe 'vfs_read 
> > 

Re: [PATCH 1/3] tools/perf: Fix the nrcpus in perf bench futex to enable the run when all CPU's are not online

[Namhyung Kim]

On Fri, 07 Jun 2024 10:13:52 +0530, Athira Rajeev wrote:

> Perf bench futex fails as below when attempted to run on
> on a powerpc system:
> 
>  ./perf bench futex all
>  Running futex/hash benchmark...
> Run summary [PID 626307]: 80 threads, each operating on 1024 [private] 
> futexes for 10 secs.
> 
> [...]

Applied to perf-tools-next after updating the commit log a bit, thanks!

Best regards,
Namhyung

Re: [PATCH V3 10/14] tools/perf: Update instruction tracking for powerpc

[Namhyung Kim]

On Sat, Jun 01, 2024 at 11:39:37AM +0530, Athira Rajeev wrote:
> Add instruction tracking function "update_insn_state_powerpc" for
> powerpc. Example sequence in powerpc:
> 
> ld  r10,264(r3)
> mr  r31,r3
> <
> ld  r9,312(r31)
> 
> Consider ithe sample is pointing to: "ld r9,312(r31)".
> Here the memory reference is hit at "312(r31)" where 312 is the offset
> and r31 is the source register. Previous instruction sequence shows that
> register state of r3 is moved to r31. So to identify the data type for r31
> access, the previous instruction ("mr") needs to be tracked and the
> state type entry has to be updated. Current instruction tracking support
> in perf tools infrastructure is specific to x86. Patch adds this support
> for powerpc as well.
> 
> Signed-off-by: Athira Rajeev 
> ---
>  .../perf/arch/powerpc/annotate/instructions.c | 65 +++
>  tools/perf/util/annotate-data.c   |  9 ++-
>  tools/perf/util/disasm.c  |  1 +
>  3 files changed, 74 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/perf/arch/powerpc/annotate/instructions.c 
> b/tools/perf/arch/powerpc/annotate/instructions.c
> index db72148eb857..3ecf5a986037 100644
> --- a/tools/perf/arch/powerpc/annotate/instructions.c
> +++ b/tools/perf/arch/powerpc/annotate/instructions.c
> @@ -231,6 +231,71 @@ static struct ins_ops *check_ppc_insn(int raw_insn)
>   return NULL;
>  }
>  
> +/*
> + * Instruction tracking function to track register state moves.
> + * Example sequence:
> + *ld  r10,264(r3)
> + *mr  r31,r3
> + *<
> + *ld  r9,312(r31)
> + *
> + * Previous instruction sequence shows that register state of r3
> + * is moved to r31. update_insn_state_powerpc tracks these state
> + * changes
> + */
> +#ifdef HAVE_DWARF_SUPPORT
> +static void update_insn_state_powerpc(struct type_state *state,
> + struct data_loc_info *dloc, Dwarf_Die * cu_die __maybe_unused,
> + struct disasm_line *dl)
> +{
> + struct annotated_insn_loc loc;
> + struct annotated_op_loc *src = [INSN_OP_SOURCE];
> + struct annotated_op_loc *dst = [INSN_OP_TARGET];
> + struct type_state_reg *tsr;
> + u32 insn_offset = dl->al.offset;
> +
> + if (annotate_get_insn_location(dloc->arch, dl, ) < 0)
> + return;
> +
> + /*
> +  * Value 444 for bits 21:30 is for "mr"
> +  * instruction. "mr" is extended OR. So set the
> +  * source and destination reg correctly
> +  */
> + if (PPC_21_30(dl->ops.raw_insn) == 444) {
> + int src_reg = src->reg1;
> +
> + src->reg1 = dst->reg1;
> + dst->reg1 = src_reg;
> + }
> +
> + if (!has_reg_type(state, dst->reg1))
> + return;
> +
> + tsr = >regs[dst->reg1];
> +
> + if (!has_reg_type(state, src->reg1) ||
> + !state->regs[src->reg1].ok) {
> + tsr->ok = false;
> + return;
> + }
> +
> + tsr->type = state->regs[src->reg1].type;
> + tsr->kind = state->regs[src->reg1].kind;
> + tsr->ok = true;
> +
> + pr_debug("mov [%x] reg%d -> reg%d",

pr_debug_dtp() ?

Thanks,
Namhyung


> + insn_offset, src->reg1, dst->reg1);
> + pr_debug_type_name(>type, tsr->kind);
> +}
> +#else /* HAVE_DWARF_SUPPORT */
> +static void update_insn_state_powerpc(struct type_state *state 
> __maybe_unused, struct data_loc_info *dloc __maybe_unused,
> + Dwarf_Die * cu_die __maybe_unused, struct disasm_line *dl 
> __maybe_unused)
> +{
> + return;
> +}
> +#endif /* HAVE_DWARF_SUPPORT */
> +
>  static int powerpc__annotate_init(struct arch *arch, char *cpuid 
> __maybe_unused)
>  {
>   if (!arch->initialized) {
> diff --git a/tools/perf/util/annotate-data.c b/tools/perf/util/annotate-data.c
> index 7a48c3d72b89..734acdd8c4b7 100644
> --- a/tools/perf/util/annotate-data.c
> +++ b/tools/perf/util/annotate-data.c
> @@ -1080,6 +1080,13 @@ static int find_data_type_insn(struct data_loc_info 
> *dloc,
>   return ret;
>  }
>  
> +static int arch_supports_insn_tracking(struct data_loc_info *dloc)
> +{
> + if ((arch__is(dloc->arch, "x86")) || (arch__is(dloc->arch, "powerpc")))
> + return 1;
> + return 0;
> +}
> +
>  /*
>   * Construct a list of basic blocks for each scope with variables and try to 
> find
>   * the data type by updating a type state table through instructions.
> @@ -1094,7 +1101,7 @@ static int find_data_type_block(struct data_loc_info 
> *dloc,
>   int ret = -1;
>  
>   /* TODO: other architecture support */
> - if (!arch__is(dloc->arch, "x86"))
> + if (!arch_supports_insn_tracking(dloc))
>   return -1;
>  
>   prev_dst_ip = dst_ip = dloc->ip;
> diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
> index 57af4dc42a58..d8b357055302 100644
> --- a/tools/perf/util/disasm.c
> +++ b/tools/perf/util/disasm.c
> @@ -155,6 +155,7 @@ static struct arch architectures[] = {
>   {
>   

Re: [PATCH V3 06/14] tools/perf: Update parameters for reg extract functions to use raw instruction on powerpc

[Namhyung Kim]

On Sat, Jun 01, 2024 at 11:39:33AM +0530, Athira Rajeev wrote:
> Use the raw instruction code and macros to identify memory instructions,
> extract register fields and also offset. The implementation addresses
> the D-form, X-form, DS-form instructions. Two main functions are added.
> New parse function "load_store__parse" as instruction ops parser for
> memory instructions. Unlink other parser (like mov__parse), this parser
> fills in the "raw_insn" field for source/target and new added "mem_ref"
> field. Also set if it is multi_regs and opcode as well. No other fields
> are set because, here there is no need to parse the disassembled
> code and arch specific macros will take care of extracting offset and
> regs which is easier and will be precise.
> 
> In powerpc, all instructions with a primary opcode from 32 to 63
> are memory instructions. Update "ins__find" function to have "raw_insn"
> also as a parameter. Don't use the "extract_reg_offset", instead use
> newly added function "get_arch_regs" which will set these fields: reg1,
> reg2, offset depending of where it is source or target ops.
> 
> Signed-off-by: Athira Rajeev 
> ---
>  .../perf/arch/powerpc/annotate/instructions.c | 16 +
>  tools/perf/arch/powerpc/util/dwarf-regs.c | 44 +
>  tools/perf/util/annotate.c| 25 +++-
>  tools/perf/util/disasm.c  | 64 +--
>  tools/perf/util/disasm.h  |  4 +-
>  tools/perf/util/include/dwarf-regs.h  |  3 +
>  6 files changed, 147 insertions(+), 9 deletions(-)
> 
> diff --git a/tools/perf/arch/powerpc/annotate/instructions.c 
> b/tools/perf/arch/powerpc/annotate/instructions.c
> index d57fd023ef9c..10fea5e5cf4c 100644
> --- a/tools/perf/arch/powerpc/annotate/instructions.c
> +++ b/tools/perf/arch/powerpc/annotate/instructions.c
> @@ -49,6 +49,22 @@ static struct ins_ops 
> *powerpc__associate_instruction_ops(struct arch *arch, con
>   return ops;
>  }
>  
> +#define PPC_OP(op)  (((op) >> 26) & 0x3F)
> +
> +static struct ins_ops *check_ppc_insn(int raw_insn)
> +{
> + int opcode = PPC_OP(raw_insn);
> +
> + /*
> +  * Instructions with opcode 32 to 63 are memory
> +  * instructions in powerpc
> +  */
> + if ((opcode & 0x20))
> + return _store_ops;
> +
> + return NULL;
> +}
> +
>  static int powerpc__annotate_init(struct arch *arch, char *cpuid 
> __maybe_unused)
>  {
>   if (!arch->initialized) {
> diff --git a/tools/perf/arch/powerpc/util/dwarf-regs.c 
> b/tools/perf/arch/powerpc/util/dwarf-regs.c
> index 430623ca5612..38b74fa01d8b 100644
> --- a/tools/perf/arch/powerpc/util/dwarf-regs.c
> +++ b/tools/perf/arch/powerpc/util/dwarf-regs.c
> @@ -107,3 +107,47 @@ int regs_query_register_offset(const char *name)
>  #define PPC_DS(DS)   ((DS) & 0xfffc)
>  #define OP_LD58
>  #define OP_STD   62
> +
> +static int get_source_reg(unsigned int raw_insn)
> +{
> + return PPC_RA(raw_insn);
> +}
> +
> +static int get_target_reg(unsigned int raw_insn)
> +{
> + return PPC_RT(raw_insn);
> +}
> +
> +static int get_offset_opcode(int raw_insn __maybe_unused)

The argument is used below, no need for __maybe_unused.

> +{
> + int opcode = PPC_OP(raw_insn);
> +
> + /* DS- form */
> + if ((opcode == OP_LD) || (opcode == OP_STD))
> + return PPC_DS(raw_insn);
> + else
> + return PPC_D(raw_insn);
> +}
> +
> +/*
> + * Fills the required fields for op_loc depending on if it
> + * is a source or target.
> + * D form: ins RT,D(RA) -> src_reg1 = RA, offset = D, dst_reg1 = RT
> + * DS form: ins RT,DS(RA) -> src_reg1 = RA, offset = DS, dst_reg1 = RT
> + * X form: ins RT,RA,RB -> src_reg1 = RA, src_reg2 = RB, dst_reg1 = RT
> + */
> +void get_arch_regs(int raw_insn __maybe_unused, int is_source __maybe_unused,
> + struct annotated_op_loc *op_loc __maybe_unused)

Ditto.

Thanks,
Namhyung


> +{
> + if (is_source)
> + op_loc->reg1 = get_source_reg(raw_insn);
> + else
> + op_loc->reg1 = get_target_reg(raw_insn);
> +
> + if (op_loc->multi_regs)
> + op_loc->reg2 = PPC_RB(raw_insn);
> +
> + /* TODO: Implement offset handling for X Form */
> + if ((op_loc->mem_ref) && (PPC_OP(raw_insn) != 31))
> + op_loc->offset = get_offset_opcode(raw_insn);
> +}
> diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c
> index 1451caf25e77..2b8cc759ae35 100644
> --- a/tools/perf/util/annotate.c
> +++ b/tools/perf/util/annotate.c
> @@ -2079,6 +2079,12 @@ static int extract_reg_offset(struct arch *arch, const 
> char *str,
>   return 0;
>  }
>  
> +__weak void get_arch_regs(int raw_insn __maybe_unused, int is_source 
> __maybe_unused,
> + struct annotated_op_loc *op_loc __maybe_unused)
> +{
> + return;
> +}
> +
>  /**
>   * annotate_get_insn_location - Get location of instruction
>   * @arch: the architecture info
> @@ -2123,20 +2129,33 @@ int 

Re: [PATCH V3 05/14] tools/perf: Add disasm_line__parse to parse raw instruction for powerpc

[Namhyung Kim]

Hello,

On Sat, Jun 01, 2024 at 11:39:32AM +0530, Athira Rajeev wrote:
> Currently, the perf tool infrastructure disasm_line__parse function to
> parse disassembled line.
> 
> Example snippet from objdump:
> objdump  --start-address= --stop-address=  -d 
> --no-show-raw-insn -C 
> 
> c10224b4: lwz r10,0(r9)
> 
> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> registers names and offset. In powerpc, the approach for data type
> profiling uses raw instruction instead of result from objdump to identify
> the instruction category and extract the source/target registers.
> 
> Example: 38 01 81 e8 ld  r4,312(r1)
> 
> Here "38 01 81 e8" is the raw instruction representation. Add function
> "disasm_line__parse_powerpc" to handle parsing of raw instruction. Also
> update "struct ins" and "struct ins_operands" to save "opcode" and
> binary code. With the change, function captures:
> 
> line -> "38 01 81 e8 ld  r4,312(r1)"
> opcode and raw instruction "38 01 81 e8"
> 
> Raw instruction is used later to extract the reg/offset fields. Macros
> are added to extract opcode and register fields. "struct ins_operands"
> and "struct ins" is updated to carry opcode and raw instruction binary
> code (raw_insn). Function "disasm_line__parse_powerpc fills the raw
> instruction hex value and opcode in newly added fields. There is no
> changes in existing code paths, which parses the disassembled code.
> The architecture using the instruction name and present approach is
> not altered. Since this approach targets powerpc, the macro
> implementation is added for powerpc as of now.
> 
> Since the disasm_line__parse is used in other cases (perf annotate) and
> not only data tye profiling, the powerpc callback includes changes to
> work with binary code as well as mneumonic representation. Also in case
> if the DSO read fails and libcapstone is not supported, the approach
> fallback to use objdump as option. Hence as option, patch has changes to
> ensure objdump option also works well.
> 
> Signed-off-by: Athira Rajeev 
> ---
>  tools/include/linux/string.h  |  2 +
>  tools/lib/string.c| 13 
>  .../perf/arch/powerpc/annotate/instructions.c |  1 +
>  tools/perf/arch/powerpc/util/dwarf-regs.c |  9 +++
>  tools/perf/util/disasm.c  | 63 ++-
>  tools/perf/util/disasm.h  |  7 +++
>  6 files changed, 94 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/include/linux/string.h b/tools/include/linux/string.h
> index db5c99318c79..0acb1fc14e19 100644
> --- a/tools/include/linux/string.h
> +++ b/tools/include/linux/string.h
> @@ -46,5 +46,7 @@ extern char * __must_check skip_spaces(const char *);
>  
>  extern char *strim(char *);
>  
> +extern void remove_spaces(char *s);
> +
>  extern void *memchr_inv(const void *start, int c, size_t bytes);
>  #endif /* _TOOLS_LINUX_STRING_H_ */
> diff --git a/tools/lib/string.c b/tools/lib/string.c
> index 8b6892f959ab..3126d2cff716 100644
> --- a/tools/lib/string.c
> +++ b/tools/lib/string.c
> @@ -153,6 +153,19 @@ char *strim(char *s)
>   return skip_spaces(s);
>  }
>  
> +/*
> + * remove_spaces - Removes whitespaces from @s
> + */
> +void remove_spaces(char *s)
> +{
> + char *d = s;
> +
> + do {
> + while (*d == ' ')
> + ++d;
> + } while ((*s++ = *d++));
> +}
> +
>  /**
>   * strreplace - Replace all occurrences of character in string.
>   * @s: The string to operate on.
> diff --git a/tools/perf/arch/powerpc/annotate/instructions.c 
> b/tools/perf/arch/powerpc/annotate/instructions.c
> index a3f423c27cae..d57fd023ef9c 100644
> --- a/tools/perf/arch/powerpc/annotate/instructions.c
> +++ b/tools/perf/arch/powerpc/annotate/instructions.c
> @@ -55,6 +55,7 @@ static int powerpc__annotate_init(struct arch *arch, char 
> *cpuid __maybe_unused)
>   arch->initialized = true;
>   arch->associate_instruction_ops = 
> powerpc__associate_instruction_ops;
>   arch->objdump.comment_char  = '#';
> + annotate_opts.show_asm_raw = true;
>   }
>  
>   return 0;
> diff --git a/tools/perf/arch/powerpc/util/dwarf-regs.c 
> b/tools/perf/arch/powerpc/util/dwarf-regs.c
> index 0c4f4caf53ac..430623ca5612 100644
> --- a/tools/perf/arch/powerpc/util/dwarf-regs.c
> +++ b/tools/perf/arch/powerpc/util/dwarf-regs.c
> @@ -98,3 +98,12 @@ int regs_query_register_offset(const char *name)
>   return roff->ptregs_offset;
>   return -EINVAL;
>  }
> +
> +#define PPC_OP(op)   (((op) >> 26) & 0x3F)
> +#define PPC_RA(a)(((a) >> 16) & 0x1f)
> +#define PPC_RT(t)(((t) >> 21) & 0x1f)
> +#define PPC_RB(b)(((b) >> 11) & 0x1f)
> +#define PPC_D(D) ((D) & 0xfffe)
> +#define PPC_DS(DS)   ((DS) & 0xfffc)
> +#define OP_LD58
> +#define OP_STD   62
> diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
> index 

Re: [PATCH V2 4/9] tools/perf: Add support to capture and parse raw instruction in objdump

[Namhyung Kim]

On Thu, May 9, 2024 at 10:27 AM Athira Rajeev
 wrote:
>
>
>
> > On 7 May 2024, at 3:05 PM, Christophe Leroy  
> > wrote:
> >
> >
> >
> > Le 06/05/2024 à 14:19, Athira Rajeev a écrit :
> >> Add support to capture and parse raw instruction in objdump.
> >
> > What's the purpose of using 'objdump' for reading raw instructions ?
> > Can't they be read directly without invoking 'objdump' ? It looks odd to
> > me to use objdump to provide readable text and then parse it back.
>
> Hi Christophe,
>
> Thanks for your review comments.
>
> Current implementation for data type profiling on X86 uses "objdump" tool to 
> get the disassembled code.
> And then the objdump result lines are parsed to get the instruction name and 
> register fields. The initial patchset I posted to enable the data type 
> profiling feature in powerpc was using the same way by getting disassembled 
> code from objdump and parsing the disassembled lines. But in V2, we are 
> introducing change for powerpc to use "raw instruction" and fetch opcode, reg 
> fields from the raw instruction.
>
> I tried to explain below that current objdump uses option 
> "--no-show-raw-insn" which doesn't capture raw instruction.  So to capture 
> raw instruction, V2 patchset has changes to use default option 
> "--show-raw-insn" and get the raw instruction [ for powerpc ] along with 
> human readable annotation [ which is used by other archs ]. Since perf tool 
> already has objdump implementation in place, I went in the direction to 
> enhance it to use "--show-raw-insn" for powerpc purpose.
>
> But as you mentioned, we can directly read raw instruction without using 
> "objdump" tool.
> perf has support to read object code. The dso open/read utilities and helper 
> functions are already present in "util/dso.c" And "dso__data_read_offset" 
> function reads data from dso file offset. We can use these functions and I 
> can make changes to directly read binary instruction without using objdump.
>
> Namhyung, Arnaldo, Christophe
> Looking for your valuable feedback on this approach. Please suggest if this 
> approach looks fine

Looks like you want to implement instruction decoding
like in arch/x86/lib/{insn,inat}.c.  I think it's ok to do that
but you need to decide which way is more convenient.

Also it works on the struct disasm_line so you need to
fill in the necessary info when not using objdump.  As
long as it produces the same output I don't care much
if you use objdump or not.  Actually it uses libcapstone
to disassemble x86 instructions if possible.  Maybe you
can use that on powerpc too.

Thanks,
Namhyung

>
>
> Thanks
> Athira
> >
> >> Currently, the perf tool infrastructure uses "--no-show-raw-insn" option
> >> with "objdump" while disassemble. Example from powerpc with this option
> >> for an instruction address is:
> >
> > Yes and that makes sense because the purpose of objdump is to provide
> > human readable annotations, not to perform automated analysis. Am I
> > missing something ?
> >
> >>
> >> Snippet from:
> >> objdump  --start-address= --stop-address=  -d 
> >> --no-show-raw-insn -C 
> >>
> >> c10224b4: lwz r10,0(r9)
> >>
> >> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> >> registers names and offset. Also to find whether there is a memory
> >> reference in the operands, "memory_ref_char" field of objdump is used.
> >> For x86, "(" is used as memory_ref_char to tackle instructions of the
> >> form "mov  (%rax), %rcx".
> >>
> >> In case of powerpc, not all instructions using "(" are the only memory
> >> instructions. Example, above instruction can also be of extended form (X
> >> form) "lwzx r10,0,r19". Inorder to easy identify the instruction category
> >> and extract the source/target registers, patch adds support to use raw
> >> instruction. With raw instruction, macros are added to extract opcode
> >> and register fields.
> >>
> >> "struct ins_operands" and "struct ins" is updated to carry opcode and
> >> raw instruction binary code (raw_insn). Function "disasm_line__parse"
> >> is updated to fill the raw instruction hex value and opcode in newly
> >> added fields. There is no changes in existing code paths, which parses
> >> the disassembled code. The architecture using the instruction name and
> >> present approach is not altered. Since this approach targets powerpc,
> >> the macro implementation is added for powerpc as of now.
> >>
> >> Example:
> >> representation using --show-raw-insn in objdump gives result:
> >>
> >> 38 01 81 e8 ld  r4,312(r1)
> >>
> >> Here "38 01 81 e8" is the raw instruction representation. In powerpc,
> >> this translates to instruction form: "ld RT,DS(RA)" and binary code
> >> as:
> >> _
> >> | 58 |  RT  |  RA |  DS   | |
> >> -
> >> 06 1116  30 31
> >>
> >> Function "disasm_line__parse" is updated to capture:
> >>
> >> line:38 01 81 e8 ld  r4,312(r1)
> >> 

Re: [PATCH V2 4/9] tools/perf: Add support to capture and parse raw instruction in objdump

[Namhyung Kim]

On Mon, May 6, 2024 at 5:21 AM Athira Rajeev
 wrote:
>
> Add support to capture and parse raw instruction in objdump.
> Currently, the perf tool infrastructure uses "--no-show-raw-insn" option
> with "objdump" while disassemble. Example from powerpc with this option
> for an instruction address is:
>
> Snippet from:
> objdump  --start-address= --stop-address=  -d 
> --no-show-raw-insn -C 
>
> c10224b4:   lwz r10,0(r9)
>
> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> registers names and offset. Also to find whether there is a memory
> reference in the operands, "memory_ref_char" field of objdump is used.
> For x86, "(" is used as memory_ref_char to tackle instructions of the
> form "mov  (%rax), %rcx".
>
> In case of powerpc, not all instructions using "(" are the only memory
> instructions. Example, above instruction can also be of extended form (X
> form) "lwzx r10,0,r19". Inorder to easy identify the instruction category
> and extract the source/target registers, patch adds support to use raw
> instruction. With raw instruction, macros are added to extract opcode
> and register fields.
>
> "struct ins_operands" and "struct ins" is updated to carry opcode and
> raw instruction binary code (raw_insn). Function "disasm_line__parse"
> is updated to fill the raw instruction hex value and opcode in newly
> added fields. There is no changes in existing code paths, which parses
> the disassembled code. The architecture using the instruction name and
> present approach is not altered. Since this approach targets powerpc,
> the macro implementation is added for powerpc as of now.
>
> Example:
> representation using --show-raw-insn in objdump gives result:
>
> 38 01 81 e8 ld  r4,312(r1)
>
> Here "38 01 81 e8" is the raw instruction representation. In powerpc,
> this translates to instruction form: "ld RT,DS(RA)" and binary code
> as:
> _
> | 58 |  RT  |  RA |  DS   | |
> -
> 06 1116  30 31
>
> Function "disasm_line__parse" is updated to capture:
>
> line:38 01 81 e8 ld  r4,312(r1)
> opcode and raw instruction "38 01 81 e8"
> Raw instruction is used later to extract the reg/offset fields.
>
> Signed-off-by: Athira Rajeev 
> ---
>  tools/include/linux/string.h  |  2 +
>  tools/lib/string.c| 13 +++
>  tools/perf/arch/powerpc/util/dwarf-regs.c | 19 ++
>  tools/perf/util/disasm.c  | 46 +++
>  tools/perf/util/disasm.h  |  6 +++
>  tools/perf/util/include/dwarf-regs.h  |  9 +
>  6 files changed, 88 insertions(+), 7 deletions(-)
>
> diff --git a/tools/include/linux/string.h b/tools/include/linux/string.h
> index db5c99318c79..0acb1fc14e19 100644
> --- a/tools/include/linux/string.h
> +++ b/tools/include/linux/string.h
> @@ -46,5 +46,7 @@ extern char * __must_check skip_spaces(const char *);
>
>  extern char *strim(char *);
>
> +extern void remove_spaces(char *s);
> +
>  extern void *memchr_inv(const void *start, int c, size_t bytes);
>  #endif /* _TOOLS_LINUX_STRING_H_ */
> diff --git a/tools/lib/string.c b/tools/lib/string.c
> index 8b6892f959ab..21d273e69951 100644
> --- a/tools/lib/string.c
> +++ b/tools/lib/string.c
> @@ -153,6 +153,19 @@ char *strim(char *s)
> return skip_spaces(s);
>  }
>
> +/*
> + * remove_spaces - Removes whitespaces from @s
> + */
> +void remove_spaces(char *s)
> +{
> +   char *d = s;
> +   do {
> +   while (*d == ' ') {
> +   ++d;
> +   }
> +   } while ((*s++ = *d++));
> +}
> +
>  /**
>   * strreplace - Replace all occurrences of character in string.
>   * @s: The string to operate on.
> diff --git a/tools/perf/arch/powerpc/util/dwarf-regs.c 
> b/tools/perf/arch/powerpc/util/dwarf-regs.c
> index 0c4f4caf53ac..e60a71fd846e 100644
> --- a/tools/perf/arch/powerpc/util/dwarf-regs.c
> +++ b/tools/perf/arch/powerpc/util/dwarf-regs.c
> @@ -98,3 +98,22 @@ int regs_query_register_offset(const char *name)
> return roff->ptregs_offset;
> return -EINVAL;
>  }
> +
> +#definePPC_OP(op)  (((op) >> 26) & 0x3F)
> +#define PPC_RA(a)  (((a) >> 16) & 0x1f)
> +#define PPC_RT(t)  (((t) >> 21) & 0x1f)
> +
> +int get_opcode_insn(unsigned int raw_insn)
> +{
> +   return PPC_OP(raw_insn);
> +}
> +
> +int get_source_reg(unsigned int raw_insn)
> +{
> +   return PPC_RA(raw_insn);
> +}
> +
> +int get_target_reg(unsigned int raw_insn)
> +{
> +   return PPC_RT(raw_insn);
> +}
> diff --git a/tools/perf/util/disasm.c b/tools/perf/util/disasm.c
> index 2de66a092cab..85692f73e78f 100644
> --- a/tools/perf/util/disasm.c
> +++ b/tools/perf/util/disasm.c
> @@ -43,7 +43,7 @@ static int call__scnprintf(struct ins *ins, char *bf, 
> size_t size,
>struct ins_operands *ops, int max_ins_name);
>
>  static void 

Re: [PATCH V2 3/9] tools/perf: Fix a comment about multi_regs in extract_reg_offset function

[Namhyung Kim]

On Mon, May 6, 2024 at 5:19 AM Athira Rajeev
 wrote:
>
> Fix a comment in function which explains how multi_regs field gets set
> for an instruction. In the example, "mov  %rsi, 8(%rbx,%rcx,4)", the
> comment mistakenly referred to "dst_multi_regs = 0". Correct it to use
> "src_multi_regs = 0"
>
> Signed-off-by: Athira Rajeev 

Acked-by: Namhyung Kim 

Thanks,
Namhyung

> ---
>  tools/perf/util/annotate.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c
> index f5b6b5e5e757..0f5e10654d09 100644
> --- a/tools/perf/util/annotate.c
> +++ b/tools/perf/util/annotate.c
> @@ -2093,7 +2093,7 @@ static int extract_reg_offset(struct arch *arch, const 
> char *str,
>   *   mov  0x18, %r8  # src_reg1 = -1, src_mem = 0
>   *   # dst_reg1 = r8, dst_mem = 0
>   *
> - *   mov  %rsi, 8(%rbx,%rcx,4)  # src_reg1 = rsi, src_mem = 0, 
> dst_multi_regs = 0
> + *   mov  %rsi, 8(%rbx,%rcx,4)  # src_reg1 = rsi, src_mem = 0, 
> src_multi_regs = 0
>   *  # dst_reg1 = rbx, dst_reg2 = rcx, dst_mem = 1
>   *  # dst_multi_regs = 1, dst_offset = 8
>   */
> --
> 2.43.0
>

Re: [PATCH V2 0/9] Add data type profiling support for powerpc

[Namhyung Kim]

Hello,

On Mon, May 6, 2024 at 5:19 AM Athira Rajeev
 wrote:
>
> The patchset from Namhyung added support for data type profiling
> in perf tool. This enabled support to associate PMU samples to data
> types they refer using DWARF debug information. With the upstream
> perf, currently it possible to run perf report or perf annotate to
> view the data type information on x86.
>
> Initial patchset posted here had changes need to enable data type
> profiling support for powerpc.
>
> https://lore.kernel.org/all/6e09dc28-4a2e-49d8-a2b5-ffb3396a9...@csgroup.eu/T/
>
> Main change were:
> 1. powerpc instruction nmemonic table to associate load/store
> instructions with move_ops which is use to identify if instruction
> is a memory access one.
> 2. To get register number and access offset from the given
> instruction, code uses fields from "struct arch" -> objump.
> Added entry for powerpc here.
> 3. A get_arch_regnum to return register number from the
> register name string.
>
> But the apporach used in the initial patchset used parsing of
> disassembled code which the current perf tool implementation does.
>
> Example: lwz r10,0(r9)
>
> This line "lwz r10,0(r9)" is parsed to extract instruction name,
> registers names and offset. Also to find whether there is a memory
> reference in the operands, "memory_ref_char" field of objdump is used.
> For x86, "(" is used as memory_ref_char to tackle instructions of the
> form "mov  (%rax), %rcx".
>
> In case of powerpc, not all instructions using "(" are the only memory
> instructions. Example, above instruction can also be of extended form (X
> form) "lwzx r10,0,r19". Inorder to easy identify the instruction category
> and extract the source/target registers, this patchset adds support to use
> raw instruction. With raw instruction, macros are added to extract opcode
> and register fields.
>
> Example representation using --show-raw-insn in objdump gives result:
>
> 38 01 81 e8 ld  r4,312(r1)
>
> Here "38 01 81 e8" is the raw instruction representation. In powerpc,
> this translates to instruction form: "ld RT,DS(RA)" and binary code
> as:
> _
> | 58 |  RT  |  RA |  DS   | |
> -
> 06 1116  30 31
>
> Patchset adds support to pick the opcode and reg fields from this
> raw/binary instruction code. This approach came in from review comment
> by Segher Boessenkool for the initial patchset.
>
> Apart from that, instruction tracking is enabled for powerpc and
> support function is added to find variables defined as registers
> Example, in powerpc, two registers are
> defined to represent variable:
> 1. r13: represents local_paca
> register struct paca_struct *local_paca asm("r13");
>
> 2. r1: represents stack_pointer
> register void *__stack_pointer asm("r1");
>
> These are handled in this patchset.
>
> - Patch 1 is to rearrange register state type structures to header file
> so that it can referred from other arch specific files
> - Patch 2 is to make instruction tracking as a callback to"struct arch"
> so that it can be implemented by other archs easily and defined in arch
> specific files
> - Patch 3 is to fix a small comment
> - Patch 4 adds support to capture and parse raw instruction in objdump
> by keeping existing approach intact.
> - Patch 5 update parameters for reg extract functions to use raw
> instruction on powerpc
> - Patch 6 and patch 7 handles instruction tracking for powerpc.
> - Patch 8 and Patch 8 handles support to find global register variables
>
> With the current patchset:
>
>  ./perf record -a -e mem-loads sleep 1
>  ./perf report -s type,typeoff --hierarchy --group --stdio
>  ./perf annotate --data-type --insn-stat
>
> perf annotate logs:
>
> Annotate Instruction stats
> total 562, ok 441 (78.5%), bad 121 (21.5%)
>
>   Name  :  Good   Bad
> ---
>   ld:   31354
>   lwz   :5132
>   lbz   :31 5
>   ldx   : 621
>   lhz   :23 0
>   lwa   : 4 3
>   lwarx : 5 0
>   lwzx  : 2 2
>   ldarx : 3 0
>   lwzu  : 2 0
>   stdcx.: 0 1
>   nop   : 0 1
>   ldu   : 1 0
>   lbzx  : 0 1
>   lwax  : 0 1
>
> perf report logs:
>
> # Samples: 1K of event 'mem-loads'
> # Event count (approx.): 937238
> #
> # Overhead  Data Type  Data Type Offset
> #   .  
> #
> 48.81%  (unknown)  (unknown) +0 (no field)
> 12.85%  long unsigned int  long unsigned int +0 (current_stack_pointer)
>  4.68%  struct paca_struct  struct paca_struct +2312 (__current)
>  4.57%  struct paca_struct  struct paca_struct +2354 (irq_soft_mask)
>  2.68%  struct paca_struct  struct paca_struct +8 (paca_index)
>  2.64%  struct paca_struct  struct paca_struct +2808 (canary)
>  2.24%  struct paca_struct  

Re: [PATCH 0/3] Add data type profiling support for powerpc

[Namhyung Kim]

Hello,

Sorry for the super late reply.


On Fri, Mar 8, 2024 at 11:25 PM Athira Rajeev
 wrote:
>
> The patchset from Namhyung added support for data type profiling
> in perf tool. This enabled support to associate PMU samples to data
> types they refer using DWARF debug information. With the upstream
> perf, currently it possible to run perf report or perf annotate to
> view the data type information on x86.
>
> This patchset includes changes need to enable data type profiling
> support for powerpc. Main change are:
> 1. powerpc instruction nmemonic table to associate load/store
> instructions with move_ops which is use to identify if instruction
> is a memory access one.
> 2. To get register number and access offset from the given
> instruction, code uses fields from "struct arch" -> objump.
> Add entry for powerpc here.
> 3. A get_arch_regnum to return register number from the
> register name string.
>
> These three patches are the basic foundational patches.
> With these changes, data types is getting identified for kernel
> and user-space samples. There are still samples, which comes as
> "unknown" and needs to be checked. We plan to get those addressed
> in follow up. With the current patchset:
>
>  ./perf record -a -e mem-loads sleep 1
>  ./perf report -s type,typeoff --hierarchy --group --stdio
> Snippet of logs:
>
>  Total Lost Samples: 0
>
>  Samples: 277  of events 'mem-loads, dummy:u'
>  Event count (approx.): 149813
>
> Overhead  Data Type / Data Type Offset
>  ...  
>
> 65.93%   0.00% (unknown)
>65.93%   0.00% (unknown) +0 (no field)
>  8.19%   0.00% struct vm_area_struct
> 8.19%   0.00% struct vm_area_struct +136 (vm_file)
>  4.53%   0.00% struct rq
> 3.14%   0.00% struct rq +0 (__lock.raw_lock.val)
> 0.83%   0.00% struct rq +3216 (avg_irq.runnable_sum)
> 0.24%   0.00% struct rq +4 (nr_running)
> 0.14%   0.00% struct rq +12 (nr_preferred_running)
> 0.12%   0.00% struct rq +2760 (sd)
> 0.06%   0.00% struct rq +3368 (prev_steal_time_rq)
> 0.01%   0.00% struct rq +2592 (curr)
>  3.53%   0.00% struct rb_node
> 3.53%   0.00% struct rb_node +0 (__rb_parent_color)
>  3.43%   0.00% struct slab
> 3.43%   0.00% struct slab +32 (freelist)
>  3.30%   0.00% unsigned int
> 3.30%   0.00% unsigned int +0 (no field)
>  3.22%   0.00% struct vm_fault
> 3.22%   0.00% struct vm_fault +48 (pmd)
>  2.55%   0.00% unsigned char
> 2.55%   0.00% unsigned char +0 (no field)
>  1.06%   0.00% struct task_struct
> 1.06%   0.00% struct task_struct +4 (thread_info.cpu)
>  0.92%   0.00% void*
> 0.92%   0.00% void* +0 (no field)
>  0.74%   0.00% __int128 unsigned
> 0.74%   0.00% __int128 unsigned +8 (no field)
>  0.59%   0.00% struct perf_event
> 0.54%   0.00% struct perf_event +552 (ctx)
> 0.04%   0.00% struct perf_event +152 (pmu)
>  0.20%   0.00% struct sched_entity
> 0.20%   0.00% struct sched_entity +0 (load.weight)
>  0.18%   0.00% struct cfs_rq
> 0.18%   0.00% struct cfs_rq +96 (curr)

Cool!  I'm happy to see it works well on powerpc too.

Lots of codes are still assuming x86 so you may need to
add separate logics like parsing the instruction location.

And instruction tracking is also a great concern.
But we may have something in the clang/LLVM to produce
more debug information and hopefully we can remove the
instruction tracking at all.

https://github.com/llvm/llvm-project/pull/81545

Anyway I'm really looking forward to seeing v2 soon.

Thanks,
Namhyung

>
> Athira Rajeev (3):
>   tools/perf/arch/powerpc: Add load/store in powerpc annotate
> instructions for data type profling
>   tools/erf/util/annotate: Set register_char and memory_ref_char for
> powerpc
>   tools/perf/arch/powerc: Add get_arch_regnum for powerpc
>
>  .../perf/arch/powerpc/annotate/instructions.c | 66 +++
>  tools/perf/arch/powerpc/util/dwarf-regs.c | 29 
>  tools/perf/util/annotate.c|  5 ++
>  3 files changed, 100 insertions(+)
>
> --
> 2.43.0
>

Re: [PATCH 3/3] tools/perf/arch/powerc: Add get_arch_regnum for powerpc

[Namhyung Kim]

Hello,

On Tue, Mar 26, 2024 at 2:35 AM Tiezhu Yang  wrote:
>
> Hi Athira and Namhyung,
>
> On 03/09/2024 03:25 PM, Athira Rajeev wrote:
> > The function get_dwarf_regnum() returns a DWARF register number
> > from a register name string. This calls arch specific function
> > get_arch_regnum to return register number for corresponding arch.
> > Add mappings for register name to register number in powerpc code:
> > arch/powerpc/util/dwarf-regs.c
> >
> > Signed-off-by: Athira Rajeev 
> > ---
> >  tools/perf/arch/powerpc/util/dwarf-regs.c | 29 +++
> >  1 file changed, 29 insertions(+)
>
> I found commit 3eee606757ad ("perf dwarf-regs: Add get_dwarf_regnum()")
> for x86, would you be able to share how to test these changes? What is
> the difference with and without the patches?

Hmm.. I guess it won't work well without the patch.  This is
needed to map register numbers (from objdump) to what
DWARF can understand (IOW they use different numbers).

Thanks,
Namhyung

Re: [PATCH] perf/pmu-events/powerpc: Update json mapfile with Power11 PVR

[Namhyung Kim]

On Mon, 29 Jan 2024 17:38:55 +0530, Madhavan Srinivasan wrote:
> Update the Power11 PVR to json mapfile to enable
> json events. Power11 is PowerISA v3.1 compliant
> and support Power10 events.
> 
> 

Applied to perf-tools-next, thanks!

Best regards,
-- 
Namhyung Kim 

[PATCH 12/14] tools/perf: Update tools's copy of powerpc syscall table

[Namhyung Kim]

tldr; Just FYI, I'm carrying this on the perf tools tree.

Full explanation:

There used to be no copies, with tools/ code using kernel headers
directly. From time to time tools/perf/ broke due to legitimate kernel
hacking. At some point Linus complained about such direct usage. Then we
adopted the current model.

The way these headers are used in perf are not restricted to just
including them to compile something.

There are sometimes used in scripts that convert defines into string
tables, etc, so some change may break one of these scripts, or new MSRs
may use some different #define pattern, etc.

E.g.:

  $ ls -1 tools/perf/trace/beauty/*.sh | head -5
  tools/perf/trace/beauty/arch_errno_names.sh
  tools/perf/trace/beauty/drm_ioctl.sh
  tools/perf/trace/beauty/fadvise.sh
  tools/perf/trace/beauty/fsconfig.sh
  tools/perf/trace/beauty/fsmount.sh
  $
  $ tools/perf/trace/beauty/fadvise.sh
  static const char *fadvise_advices[] = {
[0] = "NORMAL",
[1] = "RANDOM",
[2] = "SEQUENTIAL",
[3] = "WILLNEED",
[4] = "DONTNEED",
[5] = "NOREUSE",
  };
  $

The tools/perf/check-headers.sh script, part of the tools/ build
process, points out changes in the original files.

So its important not to touch the copies in tools/ when doing changes in
the original kernel headers, that will be done later, when
check-headers.sh inform about the change to the perf tools hackers.

Cc: Michael Ellerman 
Cc: Nicholas Piggin 
Cc: Christophe Leroy 
Cc: linuxppc-dev@lists.ozlabs.org
Signed-off-by: Namhyung Kim 
---
 tools/perf/arch/powerpc/entry/syscalls/syscall.tbl | 4 
 1 file changed, 4 insertions(+)

diff --git a/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl 
b/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl
index e1412519b4ad..7fab411378f2 100644
--- a/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl
+++ b/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl
@@ -539,3 +539,7 @@
 450nospu   set_mempolicy_home_node sys_set_mempolicy_home_node
 451common  cachestat   sys_cachestat
 452common  fchmodat2   sys_fchmodat2
+453common  map_shadow_stacksys_ni_syscall
+454common  futex_wake  sys_futex_wake
+455common  futex_wait  sys_futex_wait
+456common  futex_requeue   sys_futex_requeue
-- 
2.43.0.rc1.413.gea7ed67945-goog

Re: [PATCH] tools/perf/arch/powerpc: Fix the CPU ID const char* value by adding 0x prefix

[Namhyung Kim]

On Mon, 9 Oct 2023 10:30:52 +0530, Athira Rajeev wrote:
> Simple expression parser test fails in powerpc as below:
> 
> 4: Simple expression parser
> test child forked, pid 170385
> Using CPUID 004e2102
> division by zero
> syntax error
> syntax error
> FAILED tests/expr.c:65 parse test failed
> test child finished with -1
> Simple expression parser: FAILED!
> 
> [...]

Applied to perf-tools-next, thanks!

Re: [PATCH V2 0/3] Fix for shellcheck issues with latest scripts in tests/shell

[Namhyung Kim]

On Fri, 13 Oct 2023 13:00:18 +0530, Athira Rajeev wrote:
> shellcheck was run on perf tool shell scripts as a pre-requisite
> to include a build option for shellcheck discussed here:
> https://www.spinics.net/lists/linux-perf-users/msg25553.html
> 
> And fixes were added for the coding/formatting issues in
> two patchsets:
> https://lore.kernel.org/linux-perf-users/20230613164145.50488-1-atraj...@linux.vnet.ibm.com/
> https://lore.kernel.org/linux-perf-users/20230709182800.53002-1-atraj...@linux.vnet.ibm.com/
> 
> [...]

Applied to perf-tools-next, thanks!

Re: [PATCH V2 0/3] Fix for shellcheck issues with latest scripts in tests/shell

[Namhyung Kim]

Hello,

On Fri, Oct 13, 2023 at 12:31 AM Athira Rajeev
 wrote:
>
> shellcheck was run on perf tool shell scripts as a pre-requisite
> to include a build option for shellcheck discussed here:
> https://www.spinics.net/lists/linux-perf-users/msg25553.html
>
> And fixes were added for the coding/formatting issues in
> two patchsets:
> https://lore.kernel.org/linux-perf-users/20230613164145.50488-1-atraj...@linux.vnet.ibm.com/
> https://lore.kernel.org/linux-perf-users/20230709182800.53002-1-atraj...@linux.vnet.ibm.com/
>
> Three additional issues were observed and fixes are part of:
> git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git
>
> With recent commits in perf, other three issues are observed.
> shellcheck version: 0.6.0
> The changes are with recent commits ( which is mentioned in each patch)
> for lock_contention, record_sideband and stat_all_metricgroups test.
> Patch series fixes these testcases and patches are on top of:
> git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git
>
> The version 1 patchset had fix patch for test_arm_coresight.sh.
> Dropping that in V2 based on discussion here:
> https://lore.kernel.org/linux-perf-users/f265857d-0d37-4878-908c-d20732f21...@linux.vnet.ibm.com/T/#u
>
> Athira Rajeev (3):
>   tools/perf/tests Ignore the shellcheck SC2046 warning in
> lock_contention
>   tools/perf/tests: Fix shellcheck warning in record_sideband.sh test
>   tools/perf/tests/shell: Fix shellcheck warning SC2112 with
> stat_all_metricgroups

For the series,
Acked-by: Namhyung Kim 

Thanks,
Namhyung

Re: [PATCH V5 1/3] tools/perf: Add text_end to "struct dso" to save .text section size

[Namhyung Kim]

On Mon, Oct 2, 2023 at 11:47 PM Athira Rajeev
 wrote:
>
>
>
> > On 03-Oct-2023, at 9:58 AM, Namhyung Kim  wrote:
> >
> > Hello,
> >
> > On Thu, Sep 28, 2023 at 12:52 AM Athira Rajeev
> >  wrote:
> >>
> >> Update "struct dso" to include new member "text_end".
> >> This new field will represent the offset for end of text
> >> section for a dso. For elf, this value is derived as:
> >> sh_size (Size of section in byes) + sh_offset (Section file
> >> offst) of the elf header for text.
> >>
> >> For bfd, this value is derived as:
> >> 1. For PE file,
> >> section->size + ( section->vma - dso->text_offset)
> >> 2. Other cases:
> >> section->filepos (file position) + section->size (size of
> >> section)
> >>
> >> To resolve the address from a sample, perf looks at the
> >> DSO maps. In case of address from a kernel module, there
> >> were some address found to be not resolved. This was
> >> observed while running perf test for "Object code reading".
> >> Though the ip falls beteen the start address of the loaded
> >> module (perf map->start ) and end address ( perf map->end),
> >> it was unresolved.
> >>
> >> Example:
> >>
> >>Reading object code for memory address: 0xc00807f0142c
> >>File is: /lib/modules/6.5.0-rc3+/kernel/fs/xfs/xfs.ko
> >>On file address is: 0x1114cc
> >>Objdump command is: objdump -z -d --start-address=0x11142c 
> >> --stop-address=0x1114ac /lib/modules/6.5.0-rc3+/kernel/fs/xfs/xfs.ko
> >>objdump read too few bytes: 128
> >>test child finished with -1
> >>
> >> Here, module is loaded at:
> >># cat /proc/modules | grep xfs
> >>xfs 2228224 3 - Live 0xc00807d0
> >>
> >> From objdump for xfs module, text section is:
> >>text 0010f7bc    00a0 2**4
> >>
> >> Here the offset for 0xc00807f0142c ie  0x112074 falls out
> >> .text section which is up to 0x10f7bc.
> >>
> >> In this case for module, the address 0xc00807e11fd4 is pointing
> >> to stub instructions. This address range represents the module stubs
> >> which is allocated on module load and hence is not part of DSO offset.
> >>
> >> To identify such  address, which falls out of text
> >> section and within module end, added the new field "text_end" to
> >> "struct dso".
> >>
> >> Reported-by: Disha Goel 
> >> Signed-off-by: Athira Rajeev 
> >> Reviewed-by: Adrian Hunter 
> >> Reviewed-by: Kajol Jain 
> >
> > For the series,
> > Acked-by: Namhyung Kim 
> >
> > Thanks,
> > Namhyung
>
> Thanks for checking Namhyung,

Applied to perf-tools-next, thanks!

Re: [PATCH V2] tools/perf: Add perf binary dependent rule for shellcheck log in Makefile.perf

[Namhyung Kim]

Hello,

Sorry for the late reply.

On Thu, Oct 5, 2023 at 8:27 AM Athira Rajeev
 wrote:
>
>
>
> > On 29-Sep-2023, at 12:19 PM, Athira Rajeev  
> > wrote:
> >
> > Add rule in new Makefile "tests/Makefile.tests" for running
> > shellcheck on shell test scripts. This automates below shellcheck
> > into the build.
> >
> > $ for F in $(find tests/shell/ -perm -o=x -name '*.sh'); do shellcheck -S 
> > warning $F; done
> >
> > Condition for shellcheck is added in Makefile.perf to avoid build
> > breakage in the absence of shellcheck binary. Update Makefile.perf
> > to contain new rule for "SHELLCHECK_TEST" which is for making
> > shellcheck test as a dependency on perf binary. Added
> > "tests/Makefile.tests" to run shellcheck on shellscripts in
> > tests/shell. The make rule "SHLLCHECK_RUN" ensures that, every
> > time during make, shellcheck will be run only on modified files
> > during subsequent invocations. By this, if any newly added shell
> > scripts or fixes in existing scripts breaks coding/formatting
> > style, it will get captured during the perf build.
> >
> > Example build failure with present scripts in tests/shell:
> >
> > INSTALL libsubcmd_headers
> > INSTALL libperf_headers
> > INSTALL libapi_headers
> > INSTALL libsymbol_headers
> > INSTALL libbpf_headers
> > make[3]: *** 
> > [/root/athira/namhyung/perf-tools-next/tools/perf/tests/Makefile.tests:17: 
> > output/tests/shell/record_sideband.dep] Error 1
> > make[3]: *** Waiting for unfinished jobs
> > make[3]: *** 
> > [/root/athira/namhyung/perf-tools-next/tools/perf/tests/Makefile.tests:17: 
> > output/tests/shell/test_arm_coresight.dep] Error 1
> > make[3]: *** 
> > [/root/athira/namhyung/perf-tools-next/tools/perf/tests/Makefile.tests:17: 
> > output/tests/shell/lock_contention.dep] Error 1
> > make[2]: *** [Makefile.perf:675: SHELLCHECK_TEST] Error 2
> > make[1]: *** [Makefile.perf:238: sub-make] Error 2
> > make: *** [Makefile:70: all] Error 2
> >
> > After this, for testing, changed "tests/shell/record.sh" to
> > break shellcheck format. In the next make run, it is
> > also captured:

Where can I see the actual failure messages?

> >
> > make[3]: *** 
> > [/root/athira/namhyung/perf-tools-next/tools/perf/tests/Makefile.tests:17: 
> > output/tests/shell/record_sideband.dep] Error 1
> > make[3]: *** Waiting for unfinished jobs
> > make[3]: *** 
> > [/root/athira/namhyung/perf-tools-next/tools/perf/tests/Makefile.tests:17: 
> > output/tests/shell/record.dep] Error 1
> > make[3]: *** 
> > [/root/athira/namhyung/perf-tools-next/tools/perf/tests/Makefile.tests:17: 
> > output/tests/shell/test_arm_coresight.dep] Error 1
> > make[3]: *** 
> > [/root/athira/namhyung/perf-tools-next/tools/perf/tests/Makefile.tests:17: 
> > output/tests/shell/lock_contention.dep] Error 1
> > make[2]: *** [Makefile.perf:675: SHELLCHECK_TEST] Error 2
> > make[1]: *** [Makefile.perf:238: sub-make] Error 2
> > make: *** [Makefile:70: all] Error 2

So is this reported at build time before I run the test command?
I'm ok with that but maybe I need to build it even though I know
some test is broken.  Can we have an option to do that like with
`make NO_SHELLCHECK=1` ?

> >
> > Signed-off-by: Athira Rajeev 
> > ---
> > Changelog:
> > v1 -> v2:
> > Version1 had shellcheck in feature check which is not
> > required since shellcheck is already a binary. Presence
> > of binary can be checked using:
> > $(shell command -v shellcheck)
> > Addressed these changes as suggested by Namhyung in V2
> > Feature test logic is removed in V2. Also added example
> > for build breakage when shellcheck fails in commit message
>
> Hi All,
>
> Looking for feedback on this patch
>
> Thanks
> Athira
> >
> > tools/perf/Makefile.perf| 14 +-
> > tools/perf/tests/Makefile.tests | 24 
> > 2 files changed, 37 insertions(+), 1 deletion(-)
> > create mode 100644 tools/perf/tests/Makefile.tests
> >
> > diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> > index 98604e396ac3..56a66ca253ab 100644
> > --- a/tools/perf/Makefile.perf
> > +++ b/tools/perf/Makefile.perf
> > @@ -667,7 +667,18 @@ $(PERF_IN): prepare FORCE
> > $(PMU_EVENTS_IN): FORCE prepare
> > $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events 
> > obj=pmu-events
> >
> > -$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN)
> > +# Runs shellcheck on perf test shell scripts
> > +
> > +SHELLCHECK := $(shell command -v shellcheck)
> > +ifneq ($(SHELLCHECK),)
> > +SHELLCHECK_TEST: FORCE prepare
> > + $(Q)$(MAKE) -f $(srctree)/tools/perf/tests/Makefile.tests
> > +else
> > +SHELLCHECK_TEST:
> > + @:
> > +endif
> > +
> > +$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN) SHELLCHECK_TEST
> > $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) \
> > $(PERF_IN) $(PMU_EVENTS_IN) $(LIBS) -o $@
> >
> > @@ -1130,6 +1141,7 @@ bpf-skel-clean:
> > $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS)
> >
> > clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean 
> > 

Re: [PATCH 3/3] tools/perf/tests: Fix shellcheck warning in record_sideband.sh test

[Namhyung Kim]

On Thu, Sep 28, 2023 at 9:11 PM Athira Rajeev
 wrote:
>
> Running shellcheck on record_sideband.sh throws below
> warning:
>
> In tests/shell/record_sideband.sh line 25:
>   if ! perf record -o ${perfdata} -BN --no-bpf-event -C $1 true 2>&1 
> >/dev/null
> ^--^ SC2069: To redirect stdout+stderr, 2>&1 must be last (or use 
> '{ cmd > file; } 2>&1' to clarify).
>
> This shows shellcheck warning SC2069 where the redirection
> order needs to be fixed. Use { cmd > file; } 2>&1 to fix the
> redirection of perf record output
>
> Fixes: 23b97c7ee963 ("perf test: Add test case for record sideband events")
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/tests/shell/record_sideband.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/perf/tests/shell/record_sideband.sh 
> b/tools/perf/tests/shell/record_sideband.sh
> index 5024a7ce0c51..7e036763a43c 100755
> --- a/tools/perf/tests/shell/record_sideband.sh
> +++ b/tools/perf/tests/shell/record_sideband.sh
> @@ -22,7 +22,7 @@ trap trap_cleanup EXIT TERM INT
>
>  can_cpu_wide()
>  {
> -if ! perf record -o ${perfdata} -BN --no-bpf-event -C $1 true 2>&1 
> >/dev/null
> +if ! { perf record -o ${perfdata} -BN --no-bpf-event -C $1 true > 
> /dev/null; } 2>&1

I think we usually go without braces.

Thanks,
Namhyung


>  then
>  echo "record sideband test [Skipped cannot record cpu$1]"
>  err=2
> --
> 2.31.1
>

Re: [PATCH 1/3] perf tests test_arm_coresight: Fix the shellcheck warning in latest test_arm_coresight.sh

[Namhyung Kim]

On Thu, Sep 28, 2023 at 9:11 PM Athira Rajeev
 wrote:
>
> Running shellcheck on tests/shell/test_arm_coresight.sh
> throws below warnings:
>
> In tests/shell/test_arm_coresight.sh line 15:
> cs_etm_path=$(find  /sys/bus/event_source/devices/cs_etm/ -name cpu* 
> -print -quit)
>   ^--^ SC2061: Quote the parameter to -name so the shell 
> won't interpret it.
>
> In tests/shell/test_arm_coresight.sh line 20:
> if [ $archhver -eq 5 -a "$(printf "0x%X\n" $archpart)" = 
> "0xA13" ] ; then
>  ^-- SC2166: Prefer [ p ] && [ q ] as [ p 
> -a q ] is not well defined
>
> This warning is observed after commit:
> "commit bb350847965d ("perf test: Update cs_etm testcase for Arm ETE")"
>
> Fixed this issue by using quoting 'cpu*' for SC2061 and
> using "&&" in line number 20 for SC2166 warning
>
> Fixes: bb350847965d ("perf test: Update cs_etm testcase for Arm ETE")
> Signed-off-by: Athira Rajeev 

You'd better add Coresight folks on this.
Maybe this file was missing in the MAINTAINERS file.

Thanks,
Namhyung


> ---
>  tools/perf/tests/shell/test_arm_coresight.sh | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/tools/perf/tests/shell/test_arm_coresight.sh 
> b/tools/perf/tests/shell/test_arm_coresight.sh
> index fe78c4626e45..f2115dfa24a5 100755
> --- a/tools/perf/tests/shell/test_arm_coresight.sh
> +++ b/tools/perf/tests/shell/test_arm_coresight.sh
> @@ -12,12 +12,12 @@
>  glb_err=0
>
>  cs_etm_dev_name() {
> -   cs_etm_path=$(find  /sys/bus/event_source/devices/cs_etm/ -name cpu* 
> -print -quit)
> +   cs_etm_path=$(find  /sys/bus/event_source/devices/cs_etm/ -name 
> 'cpu*' -print -quit)
> trcdevarch=$(cat ${cs_etm_path}/mgmt/trcdevarch)
> archhver=$((($trcdevarch >> 12) & 0xf))
> archpart=$(($trcdevarch & 0xfff))
>
> -   if [ $archhver -eq 5 -a "$(printf "0x%X\n" $archpart)" = "0xA13" ] ; 
> then
> +   if [ $archhver -eq 5 ] && [ "$(printf "0x%X\n" $archpart)" = "0xA13" 
> ] ; then
> echo "ete"
> else
> echo "etm"
> --
> 2.31.1
>

Re: [PATCH V5 1/3] tools/perf: Add text_end to "struct dso" to save .text section size

[Namhyung Kim]

Hello,

On Thu, Sep 28, 2023 at 12:52 AM Athira Rajeev
 wrote:
>
> Update "struct dso" to include new member "text_end".
> This new field will represent the offset for end of text
> section for a dso. For elf, this value is derived as:
> sh_size (Size of section in byes) + sh_offset (Section file
> offst) of the elf header for text.
>
> For bfd, this value is derived as:
> 1. For PE file,
> section->size + ( section->vma - dso->text_offset)
> 2. Other cases:
> section->filepos (file position) + section->size (size of
> section)
>
> To resolve the address from a sample, perf looks at the
> DSO maps. In case of address from a kernel module, there
> were some address found to be not resolved. This was
> observed while running perf test for "Object code reading".
> Though the ip falls beteen the start address of the loaded
> module (perf map->start ) and end address ( perf map->end),
> it was unresolved.
>
> Example:
>
> Reading object code for memory address: 0xc00807f0142c
> File is: /lib/modules/6.5.0-rc3+/kernel/fs/xfs/xfs.ko
> On file address is: 0x1114cc
> Objdump command is: objdump -z -d --start-address=0x11142c 
> --stop-address=0x1114ac /lib/modules/6.5.0-rc3+/kernel/fs/xfs/xfs.ko
> objdump read too few bytes: 128
> test child finished with -1
>
> Here, module is loaded at:
> # cat /proc/modules | grep xfs
> xfs 2228224 3 - Live 0xc00807d0
>
> From objdump for xfs module, text section is:
> text 0010f7bc    00a0 2**4
>
> Here the offset for 0xc00807f0142c ie  0x112074 falls out
> .text section which is up to 0x10f7bc.
>
> In this case for module, the address 0xc00807e11fd4 is pointing
> to stub instructions. This address range represents the module stubs
> which is allocated on module load and hence is not part of DSO offset.
>
> To identify such  address, which falls out of text
> section and within module end, added the new field "text_end" to
> "struct dso".
>
> Reported-by: Disha Goel 
> Signed-off-by: Athira Rajeev 
> Reviewed-by: Adrian Hunter 
> Reviewed-by: Kajol Jain 

For the series,
Acked-by: Namhyung Kim 

Thanks,
Namhyung


> ---
> Changelog:
> v2 -> v3:
>  Added Reviewed-by from Adrian
>
>  v1 -> v2:
>  Added text_end for bfd also by updating dso__load_bfd_symbols
>  as suggested by Adrian.
>
>  tools/perf/util/dso.h| 1 +
>  tools/perf/util/symbol-elf.c | 4 +++-
>  tools/perf/util/symbol.c | 2 ++
>  3 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
> index b41c9782c754..70fe0fe69bef 100644
> --- a/tools/perf/util/dso.h
> +++ b/tools/perf/util/dso.h
> @@ -181,6 +181,7 @@ struct dso {
> u8   rel;
> struct build_id  bid;
> u64  text_offset;
> +   u64  text_end;
> const char   *short_name;
> const char   *long_name;
> u16  long_name_len;
> diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
> index 95e99c332d7e..9e7eeaf616b8 100644
> --- a/tools/perf/util/symbol-elf.c
> +++ b/tools/perf/util/symbol-elf.c
> @@ -1514,8 +1514,10 @@ dso__load_sym_internal(struct dso *dso, struct map 
> *map, struct symsrc *syms_ss,
> }
>
> if (elf_section_by_name(runtime_ss->elf, _ss->ehdr, ,
> -   ".text", NULL))
> +   ".text", NULL)) {
> dso->text_offset = tshdr.sh_addr - tshdr.sh_offset;
> +   dso->text_end = tshdr.sh_offset + tshdr.sh_size;
> +   }
>
> if (runtime_ss->opdsec)
> opddata = elf_rawdata(runtime_ss->opdsec, NULL);
> diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
> index 3f36675b7c8f..f25e4e62cf25 100644
> --- a/tools/perf/util/symbol.c
> +++ b/tools/perf/util/symbol.c
> @@ -1733,8 +1733,10 @@ int dso__load_bfd_symbols(struct dso *dso, const char 
> *debugfile)
> /* PE symbols can only have 4 bytes, so use .text 
> high bits */
> dso->text_offset = section->vma - (u32)section->vma;
> dso->text_offset += 
> (u32)bfd_asymbol_value(symbols[i]);
> +   dso->text_end = (section->vma - dso->text_offset) + 
> section->size;
> } else {
> dso->text_offset = section->vma - section->filepos;
> +   dso->text_end = section->filepos + section->size;
> }
> }
>
> --
> 2.31.1
>

Re: [PATCH V3] perf test: Fix parse-events tests to skip parametrized events

[Namhyung Kim]

On Wed, Sep 27, 2023 at 11:17 AM Athira Rajeev
 wrote:
>
> Testcase "Parsing of all PMU events from sysfs" parse events for
> all PMUs, and not just cpu. In case of powerpc, the PowerVM
> environment supports events from hv_24x7 and hv_gpci PMU which
> is of example format like below:
>
> - hv_24x7/CPM_ADJUNCT_INST,domain=?,core=?/
> - hv_gpci/event,partition_id=?/
>
> The value for "?" needs to be filled in depending on system
> configuration. It is better to skip these parametrized events
> in this test as it is done in:
> 'commit b50d691e50e6 ("perf test: Fix "all PMU test" to skip
> parametrized events")' which handled a simialr instance with
> "all PMU test".
>
> Fix parse-events test to skip parametrized events since
> it needs proper setup of the parameters.
>
> Signed-off-by: Athira Rajeev 
> Tested-by: Ian Rogers 
> Tested-by: Sachin Sant 
> Reviewed-by: Kajol Jain 

Applied to perf-tools-next, thanks!

Re: [PATCH 0/3] Fix for shellcheck issues with version "0.6"

[Namhyung Kim]

On Tue, Sep 26, 2023 at 9:29 PM Athira Rajeev
 wrote:
>
>
>
> > On 25-Sep-2023, at 1:34 PM, kajoljain  wrote:
> >
> >
> >
> > On 9/7/23 22:45, Athira Rajeev wrote:
> >> From: root 
> >>
> >> shellcheck was run on perf tool shell scripts s a pre-requisite
> >> to include a build option for shellcheck discussed here:
> >> https://www.spinics.net/lists/linux-perf-users/msg25553.html
> >>
> >> And fixes were added for the coding/formatting issues in
> >> two patchsets:
> >> https://lore.kernel.org/linux-perf-users/20230613164145.50488-1-atraj...@linux.vnet.ibm.com/
> >> https://lore.kernel.org/linux-perf-users/20230709182800.53002-1-atraj...@linux.vnet.ibm.com/
> >>
> >> Three additional issues are observed with shellcheck "0.6" and
> >> this patchset covers those. With this patchset,
> >>
> >> # for F in $(find tests/shell/ -perm -o=x -name '*.sh'); do shellcheck -S 
> >> warning $F; done
> >> # echo $?
> >> 0
> >>
> >
> > Patchset looks good to me.
> >
> > Reviewed-by: Kajol Jain 
> >
> > Thanks,
> > Kajol Jain
> >
>
> Hi Namhyunbg,
>
> Can you please check for this patchset also

Sure, it's applied to perf-tools-next, thanks!

Re: [PATCH V4 2/2] tools/perf/tests: Fix object code reading to skip address that falls out of text section

[Namhyung Kim]

On Thu, Sep 14, 2023 at 10:40 PM Athira Rajeev
 wrote:
>
> The testcase "Object code reading" fails in somecases
> for "fs_something" sub test as below:
>
> Reading object code for memory address: 0xc00807f0142c
> File is: /lib/modules/6.5.0-rc3+/kernel/fs/xfs/xfs.ko
> On file address is: 0x1114cc
> Objdump command is: objdump -z -d --start-address=0x11142c 
> --stop-address=0x1114ac /lib/modules/6.5.0-rc3+/kernel/fs/xfs/xfs.ko
> objdump read too few bytes: 128
> test child finished with -1
>
> This can alo be reproduced when running perf record with
> workload that exercises fs_something() code. In the test
> setup, this is exercising xfs code since root is xfs.
>
> # perf record ./a.out
> # perf report -v |grep "xfs.ko"
>   0.76% a.out /lib/modules/6.5.0-rc3+/kernel/fs/xfs/xfs.ko  
> 0xc00807de5efc B [k] xlog_cil_commit
>   0.74% a.out  /lib/modules/6.5.0-rc3+/kernel/fs/xfs/xfs.ko  
> 0xc00807d5ae18 B [k] xfs_btree_key_offset
>   0.74% a.out  /lib/modules/6.5.0-rc3+/kernel/fs/xfs/xfs.ko  
> 0xc00807e11fd4 B [k] 0x00112074
>
> Here addr "0xc00807e11fd4" is not resolved. since this is a
> kernel module, its offset is from the DSO. Xfs module is loaded
> at 0xc00807d0
>
># cat /proc/modules | grep xfs
> xfs 2228224 3 - Live 0xc00807d0
>
> And size is 0x22. So its loaded between  0xc00807d0
> and 0xc00807f2. From objdump, text section is:
> text 0010f7bc    00a0 2**4
>
> Hence perf captured ip maps to 0x112074 which is:
> ( ip - start of module ) + a0
>
> This offset 0x112074 falls out .text section which is up to 0x10f7bc
> In this case for module, the address 0xc00807e11fd4 is pointing
> to stub instructions. This address range represents the module stubs
> which is allocated on module load and hence is not part of DSO offset.
>
> To address this issue in "object code reading", skip the sample if
> address falls out of text section and is within the module end.
> Use the "text_end" member of "struct dso" to do this check.
>
> To address this issue in "perf report", exploring an option of
> having stubs range as part of the /proc/kallsyms, so that perf
> report can resolve addresses in stubs range
>
> However this patch uses text_end to skip the stub range for
> Object code reading testcase.
>
> Reported-by: Disha Goel 
> Signed-off-by: Athira Rajeev 
> Tested-by: Disha Goel
> Reviewed-by: Adrian Hunter 
> ---
> Changelog:
>  v3 -> v4:
>  Fixed indent in V3
>
>  v2 -> v3:
>  Used strtailcmp in comparison for module check and added Reviewed-by
>  from Adrian, Tested-by from Disha.
>
>  v1 -> v2:
>  Updated comment to add description on which arch has stub and
>  reason for skipping as suggested by Adrian
>
>  tools/perf/tests/code-reading.c | 10 ++
>  1 file changed, 10 insertions(+)
>
> diff --git a/tools/perf/tests/code-reading.c b/tools/perf/tests/code-reading.c
> index ed3815163d1b..9e6e6c985840 100644
> --- a/tools/perf/tests/code-reading.c
> +++ b/tools/perf/tests/code-reading.c
> @@ -269,6 +269,16 @@ static int read_object_code(u64 addr, size_t len, u8 
> cpumode,
> if (addr + len > map__end(al.map))
> len = map__end(al.map) - addr;
>
> +   /*
> +* Some architectures (ex: powerpc) have stubs (trampolines) in kernel
> +* modules to manage long jumps. Check if the ip offset falls in stubs
> +* sections for kernel modules. And skip module address after text end
> +*/
> +   if (!strtailcmp(dso->long_name, ".ko") && al.addr > dso->text_end) {

There's a is_kernel_module() that can check compressed modules
too but I think we need a simpler way to check it like dso->kernel.

Thanks,
Namhyung


> +   pr_debug("skipping the module address %#"PRIx64" after text 
> end\n", al.addr);
> +   goto out;
> +   }
> +
> /* Read the object code using perf */
> ret_len = dso__data_read_offset(dso, 
> maps__machine(thread__maps(thread)),
> al.addr, buf1, len);
> --
> 2.31.1
>

Re: [PATCH 2/2] tools/perf: Add perf binary dependent rule for shellcheck log in Makefile.perf

[Namhyung Kim]

On Thu, Sep 14, 2023 at 10:18 AM Athira Rajeev
 wrote:
>
> Add rule in new Makefile "tests/Makefile.tests" for running
> shellcheck on shell test scripts. This automates below shellcheck
> into the build.
>
> $ for F in $(find tests/shell/ -perm -o=x -name '*.sh'); do 
> shellcheck -S warning $F; done

I think you can do it if $(shell command -v shellcheck) returns
non-empty string (the path to the shellcheck).  Then the feature
test logic can be gone.

>
> CONFIG_SHELLCHECK check is added to avoid build breakage in
> the absence of shellcheck binary. Update Makefile.perf to contain
> new rule for "SHELLCHECK_TEST" which is for making shellcheck
> test as a dependency on perf binary. Added "tests/Makefile.tests"
> to run shellcheck on shellscripts in tests/shell. The make rule
> "SHLLCHECK_RUN" ensures that, every time during make, shellcheck
> will be run only on modified files during subsequent invocations.
> By this, if any newly added shell scripts or fixes in existing
> scripts breaks coding/formatting style, it will get captured
> during the perf build.

Can you show me the example output?

>
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/Makefile.perf| 12 +++-
>  tools/perf/tests/Makefile.tests | 24 
>  2 files changed, 35 insertions(+), 1 deletion(-)
>  create mode 100644 tools/perf/tests/Makefile.tests
>
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index f6fdc2d5a92f..c27f54771e90 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -667,7 +667,16 @@ $(PERF_IN): prepare FORCE
>  $(PMU_EVENTS_IN): FORCE prepare
> $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events 
> obj=pmu-events
>
> -$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN)
> +# Runs shellcheck on perf test shell scripts
> +ifeq ($(CONFIG_SHELLCHECK),y)
> +SHELLCHECK_TEST: FORCE prepare
> +   $(Q)$(MAKE) -f $(srctree)/tools/perf/tests/Makefile.tests
> +else
> +SHELLCHECK_TEST:
> +   @:
> +endif
> +
> +$(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN) SHELLCHECK_TEST
> $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) \
> $(PERF_IN) $(PMU_EVENTS_IN) $(LIBS) -o $@
>
> @@ -1129,6 +1138,7 @@ bpf-skel-clean:
> $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS)
>
>  clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean 
> $(LIBSYMBOL)-clean $(LIBPERF)-clean fixdep-clean python-clean bpf-skel-clean 
> tests-coresight-targets-clean
> +   $(Q)$(MAKE) -f $(srctree)/tools/perf/tests/Makefile.tests clean
> $(call QUIET_CLEAN, core-objs)  $(RM) $(LIBPERF_A) 
> $(OUTPUT)perf-archive $(OUTPUT)perf-iostat $(LANG_BINDINGS)
> $(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' 
> -delete -o -name '\.*.d' -delete
> $(Q)$(RM) $(OUTPUT).config-detected
> diff --git a/tools/perf/tests/Makefile.tests b/tools/perf/tests/Makefile.tests
> new file mode 100644
> index ..e74575559e83
> --- /dev/null
> +++ b/tools/perf/tests/Makefile.tests
> @@ -0,0 +1,24 @@
> +# SPDX-License-Identifier: GPL-2.0
> +# Athira Rajeev , 2023
> +-include $(OUTPUT).config-detected
> +
> +log_file = $(OUTPUT)shellcheck_test.log
> +PROGS = $(subst ./,,$(shell find tests/shell -perm -o=x -type f -name 
> '*.sh'))
> +DEPS = $(addprefix output/,$(addsuffix .dep,$(basename $(PROGS
> +DIRS = $(shell echo $(dir $(DEPS)) | xargs -n1 | sort -u | xargs)
> +
> +.PHONY: all
> +all: SHELLCHECK_RUN
> +   @:
> +
> +SHELLCHECK_RUN: $(DEPS) $(DIRS)
> +
> +output/%.dep: %.sh | $(DIRS)
> +   $(call rule_mkdir)
> +   $(Q)$(call frecho-cmd,test)@touch $@
> +   $(Q)$(call frecho-cmd,test)@shellcheck -S warning $(subst 
> output/,./,$(patsubst %.dep, %.sh, $@)) 1> ${log_file} && ([[ ! -s 
> ${log_file} ]])

This line is too long, please wrap it with some backslashes.

Thanks,
Namhyung


> +$(DIRS):
> +   @mkdir -p $@
> +
> +clean:
> +   @rm -rf $(log_file) output
> --
> 2.31.1
>

Re: [PATCH 1/2] tools/perf: Add new CONFIG_SHELLCHECK for detecting shellcheck binary

[Namhyung Kim]

Hello,

On Thu, Sep 14, 2023 at 10:18 AM Athira Rajeev
 wrote:
>
> shellcheck tool can detect coding/formatting issues on
> shell scripts. In perf directory "tests/shell", there are lot
> of shell test scripts and this tool can detect coding/formatting
> issues on these scripts.
>
> Example to use shellcheck for severity level for
> errors and warnings, below command is used:
>
># for F in $(find tests/shell/ -perm -o=x -name '*.sh'); do shellcheck -S 
> warning $F; done
># echo $?
>  0
>
> This testing needs to be automated into the build so that it
> can avoid regressions and also run the check for newly added
> during build test itself. Add a new feature check to detect
> presence of shellcheck. Add CONFIG_SHELLCHECK feature check in
> the build to avoid not having shellcheck breaking the build.
>
> Signed-off-by: Athira Rajeev 
> ---
>  tools/build/Makefile.feature |  6 --
>  tools/build/feature/Makefile |  8 +++-
>  tools/perf/Makefile.config   | 10 ++
>  3 files changed, 21 insertions(+), 3 deletions(-)
>
> diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
> index 934e2777a2db..23f56b95babf 100644
> --- a/tools/build/Makefile.feature
> +++ b/tools/build/Makefile.feature
> @@ -72,7 +72,8 @@ FEATURE_TESTS_BASIC :=  \
>  libzstd\
>  disassembler-four-args \
>  disassembler-init-styled   \
> -file-handle
> +file-handle\
> +shellcheck
>
>  # FEATURE_TESTS_BASIC + FEATURE_TESTS_EXTRA is the complete list
>  # of all feature tests
> @@ -138,7 +139,8 @@ FEATURE_DISPLAY ?=  \
>   get_cpuid  \
>   bpf   \
>   libaio\
> - libzstd
> + libzstd   \
> + shellcheck
>
>  #
>  # Declare group members of a feature to display the logical OR of the 
> detection
> diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
> index 3184f387990a..44ba6d0c98d0 100644
> --- a/tools/build/feature/Makefile
> +++ b/tools/build/feature/Makefile
> @@ -76,7 +76,8 @@ FILES=  \
>   test-libzstd.bin  \
>   test-clang-bpf-co-re.bin  \
>   test-file-handle.bin  \
> - test-libpfm4.bin
> + test-libpfm4.bin  \
> + test-shellcheck.bin
>
>  FILES := $(addprefix $(OUTPUT),$(FILES))
>
> @@ -92,6 +93,8 @@ __BUILD = $(CC) $(CFLAGS) -MD -Wall -Werror -o $@ 
> $(patsubst %.bin,%.c,$(@F)) $(
>  __BUILDXX = $(CXX) $(CXXFLAGS) -MD -Wall -Werror -o $@ $(patsubst 
> %.bin,%.cpp,$(@F)) $(LDFLAGS)
>BUILDXX = $(__BUILDXX) > $(@:.bin=.make.output) 2>&1
>
> +  BUILD_BINARY = sh -c $1 > $(@:.bin=.make.output) 2>&1
> +
>  ###
>
>  $(OUTPUT)test-all.bin:
> @@ -207,6 +210,9 @@ $(OUTPUT)test-libslang-include-subdir.bin:
>  $(OUTPUT)test-libtraceevent.bin:
> $(BUILD) -ltraceevent
>
> +$(OUTPUT)test-shellcheck.bin:
> +   $(BUILD_BINARY) "shellcheck --version"

I don't think it'd generate the .bin file.

Anyway, it's a binary file already.  Can we check it with
`command -v` and get rid of the feature test?

Thanks,
Namhyung


> +
>  $(OUTPUT)test-libtracefs.bin:
>  $(BUILD) $(shell $(PKG_CONFIG) --cflags libtraceevent 2>/dev/null) 
> -ltracefs
>
> diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
> index d66b52407e19..e71fe95ad865 100644
> --- a/tools/perf/Makefile.config
> +++ b/tools/perf/Makefile.config
> @@ -779,6 +779,16 @@ ifndef NO_SLANG
>endif
>  endif
>
> +ifneq ($(NO_SHELLCHECK),1)
> +  $(call feature_check,shellcheck)
> +  ifneq ($(feature-shellcheck), 1)
> +msg := $(warning No shellcheck found. please install ShellCheck);
> +  else
> +$(call detected,CONFIG_SHELLCHECK)
> +NO_SHELLCHECK := 0
> +  endif
> +endif
> +
>  ifdef GTK2
>FLAGS_GTK2=$(CFLAGS) $(LDFLAGS) $(EXTLIBS) $(shell $(PKG_CONFIG) --libs 
> --cflags gtk+-2.0 2>/dev/null)
>$(call feature_check,gtk2)
> --
> 2.31.1
>

Re: [PATCH V2] perf test: Fix parse-events tests to skip parametrized events

[Namhyung Kim]

Hello,

On Mon, Sep 25, 2023 at 10:37 AM Arnaldo Carvalho de Melo
 wrote:
>
>
>
> On Wed, Sep 13, 2023, 7:40 AM Athira Rajeev  
> wrote:
>>
>>
>>
>> > On 08-Sep-2023, at 7:48 PM, Athira Rajeev  
>> > wrote:
>> >
>> >
>> >
>> >> On 08-Sep-2023, at 11:04 AM, Sachin Sant  wrote:
>> >>
>> >>
>> >>
>> >>> On 07-Sep-2023, at 10:29 PM, Athira Rajeev  
>> >>> wrote:
>> >>>
>> >>> Testcase "Parsing of all PMU events from sysfs" parse events for
>> >>> all PMUs, and not just cpu. In case of powerpc, the PowerVM
>> >>> environment supports events from hv_24x7 and hv_gpci PMU which
>> >>> is of example format like below:
>> >>>
>> >>> - hv_24x7/CPM_ADJUNCT_INST,domain=?,core=?/
>> >>> - hv_gpci/event,partition_id=?/
>> >>>
>> >>> The value for "?" needs to be filled in depending on system
>> >>> configuration. It is better to skip these parametrized events
>> >>> in this test as it is done in:
>> >>> 'commit b50d691e50e6 ("perf test: Fix "all PMU test" to skip
>> >>> parametrized events")' which handled a simialr instance with
>> >>> "all PMU test".
>> >>>
>> >>> Fix parse-events test to skip parametrized events since
>> >>> it needs proper setup of the parameters.
>> >>>
>> >>> Signed-off-by: Athira Rajeev 
>> >>> ---
>> >>> Changelog:
>> >>> v1 -> v2:
>> >>> Addressed review comments from Ian. Updated size of
>> >>> pmu event name variable and changed bool name which is
>> >>> used to skip the test.
>> >>>
>> >>
>> >> The patch fixes the reported issue.
>> >>
>> >> 6.2: Parsing of all PMU events from sysfs  : Ok
>> >> 6.3: Parsing of given PMU events from sysfs: Ok
>> >>
>> >> Tested-by: Sachin Sant 
>> >>
>> >> - Sachin
>> >
>> > Hi Sachin, Ian
>> >
>> > Thanks for testing the patch
>>
>> Hi Arnaldo
>>
>> Can you please check and pull this if it looks good to go .
>
>
> Namhyung, can you please take a look?

Yep sure.  I think it needs to close the file when getline() fails.

Athira, can you please send v3 with that?

Thanks,
Namhyung

Re: [PATCH 0/4] Introduce perf build subcommand

[Namhyung Kim]

Hello,

On Thu, Aug 24, 2023 at 11:11 PM Aditya Gupta  wrote:
>
> The Problem
> ===
>
> Currently the presence of a feature is checked with a combination of
> perf version --build-options and greps, such as:
>
> perf version --build-options | grep " on .* HAVE_FEATURE"
>
> Proposed solution
> =
>
> As suggested by contributors in:
> https://lore.kernel.org/linux-perf-users/zmpwk5k63tadm...@kernel.org/
>
> Introduce a subcommand "perf build --has", with which
> scripts can test for presence of a feature, such as:
>
> perf build --has HAVE_FEATURE
>
> The usage of "perf version --build-options | grep" has been replaced in two
> tests, with "perf build --has" command

I'm not sure 'perf build' is a good name, it sounds like it needs to build
something.  Maybe 'perf check --feature XXX' ?

Then we can extend the perf check command to *check* system
settings like perf_event_paranoid, kptr_restrict, nmi_watchdog
and so on, and possibly provides some advice or even change
the values easily.

What do you think?

Thanks,
Namhyung


>
> Also, to not duplicate the same feature list at multiple places, a new global
> 'supported_features' array has been introduced in builtin.h, so both commands
> 'perf build --has' and 'perf version --build-options' use the same array
>
> 'supported_features' feature is an array of 'struct feature_support', which
> also has the name of the feature, macro used to test it's presence, and a
> is_builtin member, which will be 0 if feature not built-in, and 1 if built-in
>
> Architectures Tested
> 
> * x86_64
> * ppc64le
>
> Git tree
> 
>
> Git tree with this patch series applied for testing:
> https://github.com/adi-g15-ibm/linux/tree/perf-build-has
>
> Aditya Gupta (3):
>   perf build: introduce build subcommand
>   perf version: update --build-options to use 'supported_features' array
>   perf tests task_analyzer: check perf build for libtraceevent support
>
> Athira Rajeev (1):
>   tools/perf/tests: Update probe_vfs_getname.sh script to use perf build
> --has
>
>  tools/perf/Build  |  1 +
>  tools/perf/builtin-build.c| 94 +++
>  tools/perf/builtin-version.c  | 39 ++--
>  tools/perf/builtin.h  | 47 ++
>  tools/perf/perf.c |  1 +
>  .../perf/tests/shell/lib/probe_vfs_getname.sh |  4 +-
>  .../shell/record+probe_libc_inet_pton.sh  |  5 +-
>  .../shell/record+script_probe_vfs_getname.sh  |  5 +-
>  tools/perf/tests/shell/test_task_analyzer.sh  |  4 +-
>  9 files changed, 163 insertions(+), 37 deletions(-)
>  create mode 100644 tools/perf/builtin-build.c
>
> --
> 2.41.0
>

Re: [PATCH v4 4/5] tools headers UAPI: Sync files changed by new fchmodat2 syscall

[Namhyung Kim]

Hello,

On Tue, Jul 11, 2023 at 9:18 AM Alexey Gladkov  wrote:
>
> From: Palmer Dabbelt 
>
> That add support for this new syscall in tools such as 'perf trace'.
>
> Signed-off-by: Palmer Dabbelt 
> Signed-off-by: Alexey Gladkov 
> ---
>  tools/include/uapi/asm-generic/unistd.h | 5 -
>  tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl | 2 ++
>  tools/perf/arch/powerpc/entry/syscalls/syscall.tbl  | 2 ++
>  tools/perf/arch/s390/entry/syscalls/syscall.tbl | 2 ++
>  tools/perf/arch/x86/entry/syscalls/syscall_64.tbl   | 2 ++
>  5 files changed, 12 insertions(+), 1 deletion(-)

It'd be nice if you route this patch separately through the
perf tools tree.  We can add this after the kernel change
is accepted.

Thanks,
Namhyung


>
> diff --git a/tools/include/uapi/asm-generic/unistd.h 
> b/tools/include/uapi/asm-generic/unistd.h
> index dd7d8e10f16d..76b5922b0d39 100644
> --- a/tools/include/uapi/asm-generic/unistd.h
> +++ b/tools/include/uapi/asm-generic/unistd.h
> @@ -817,8 +817,11 @@ __SYSCALL(__NR_futex_waitv, sys_futex_waitv)
>  #define __NR_set_mempolicy_home_node 450
>  __SYSCALL(__NR_set_mempolicy_home_node, sys_set_mempolicy_home_node)
>
> +#define __NR_fchmodat2 452
> +__SYSCALL(__NR_fchmodat2, sys_fchmodat2)
> +
>  #undef __NR_syscalls
> -#define __NR_syscalls 451
> +#define __NR_syscalls 453
>
>  /*
>   * 32 bit systems traditionally used different
> diff --git a/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl 
> b/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl
> index 3f1886ad9d80..434728af4eaa 100644
> --- a/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl
> +++ b/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl
> @@ -365,3 +365,5 @@
>  448n64 process_mreleasesys_process_mrelease
>  449n64 futex_waitv sys_futex_waitv
>  450common  set_mempolicy_home_node sys_set_mempolicy_home_node
> +# 451 reserved for cachestat
> +452n64 fchmodat2   sys_fchmodat2
> diff --git a/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl 
> b/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl
> index a0be127475b1..6b70b6705bd7 100644
> --- a/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl
> +++ b/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl
> @@ -537,3 +537,5 @@
>  448common  process_mreleasesys_process_mrelease
>  449common  futex_waitv sys_futex_waitv
>  450nospu   set_mempolicy_home_node sys_set_mempolicy_home_node
> +# 451 reserved for cachestat
> +452common  fchmodat2   sys_fchmodat2
> diff --git a/tools/perf/arch/s390/entry/syscalls/syscall.tbl 
> b/tools/perf/arch/s390/entry/syscalls/syscall.tbl
> index b68f47541169..0ed90c9535b0 100644
> --- a/tools/perf/arch/s390/entry/syscalls/syscall.tbl
> +++ b/tools/perf/arch/s390/entry/syscalls/syscall.tbl
> @@ -453,3 +453,5 @@
>  448  commonprocess_mreleasesys_process_mrelease
> sys_process_mrelease
>  449  commonfutex_waitv sys_futex_waitv 
> sys_futex_waitv
>  450  commonset_mempolicy_home_node sys_set_mempolicy_home_node 
> sys_set_mempolicy_home_node
> +# 451 reserved for cachestat
> +452  commonfchmodat2   sys_fchmodat2   
> sys_fchmodat2
> diff --git a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl 
> b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
> index c84d12608cd2..a008724a1f48 100644
> --- a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
> +++ b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
> @@ -372,6 +372,8 @@
>  448common  process_mreleasesys_process_mrelease
>  449common  futex_waitv sys_futex_waitv
>  450common  set_mempolicy_home_node sys_set_mempolicy_home_node
> +# 451 reserved for cachestat
> +452common  fchmodat2   sys_fchmodat2
>
>  #
>  # Due to a historical design error, certain syscalls are numbered differently
> --
> 2.33.8
>

Re: [PATCH 1/2] tools/perf/tests: perf all metrics test fails when perf_event access is restricted

[Namhyung Kim]

On Mon, Jul 3, 2023 at 10:04 PM Athira Rajeev
 wrote:
>
>
>
> > On 15-Jun-2023, at 1:08 PM, Athira Rajeev  
> > wrote:
> >
> > Perf all metrics test fails as below when perf_event access
> > is restricted.
> >
> >./perf test -v "perf all metrics test"
> >Metric 'Memory_RD_BW_Chip' not printed in:
> >Error:
> >Access to performance monitoring and observability operations is limited.
> >Enforced MAC policy settings (SELinux) can limit access to performance
> >—
> >access to performance monitoring and observability operations for 
> > processes
> >without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
> >—
> >test child finished with -1
> > end 
> >perf all metrics test: FAILED!

In my system, it fails like below:

  $ ./perf test -v 101
  101: perf all metrics test   :
  --- start ---
  test child forked, pid 398458
  Testing branch_misprediction_ratio
  Testing all_remote_links_outbound
  Metric 'all_remote_links_outbound' not printed in:
  Error:
  Invalid event (remote_outbound_data_controller_3:u) in per-thread
mode, enable system wide with '-a'.
  Testing nps1_die_to_dram
  ...

Thanks,
Namhyung

>
>
> Hi,
>
> Looking for review comments on this patch.
>
> Thanks
> >
> > The perf all metrics test picks the input events from
> > "perf list --raw-dump metrics" and runs "perf stat -M "$m""
> > for each of the metrics in the list. It fails here for some
> > of the metrics which needs access, since it collects system
> > wide resource details/statistics. Fix the testcase to skip
> > those metric events.
> >
> > Signed-off-by: Athira Rajeev 
> > ---
> > tools/perf/tests/shell/stat_all_metrics.sh | 4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/perf/tests/shell/stat_all_metrics.sh 
> > b/tools/perf/tests/shell/stat_all_metrics.sh
> > index 54774525e18a..14b96484a359 100755
> > --- a/tools/perf/tests/shell/stat_all_metrics.sh
> > +++ b/tools/perf/tests/shell/stat_all_metrics.sh
> > @@ -6,7 +6,9 @@ err=0
> > for m in $(perf list --raw-dump metrics); do
> >   echo "Testing $m"
> >   result=$(perf stat -M "$m" true 2>&1)
> > -  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "" ]]
> > +  # Skip if there is no access to perf_events monitoring
> > +  # and observability operations
> > +  if [[ "$result" =~ ${m:0:50} ]] || [[ "$result" =~ "" ]] 
> > || [[ "$result" =~ "Access to performance monitoring and observability 
> > operations is limited" ]]
> >   then
> > continue
> >   fi
> > --
> > 2.31.1
> >
>

Re: [PATCH] tools/perf/tests: Fix objdump in Object code reading test to look for all sections

[Namhyung Kim]

On Mon, Jul 3, 2023 at 10:04 PM Athira Rajeev
 wrote:
>
> Object code reading test fails intermittently with below logs:
>
>Reading object code for memory address: 0xc00801dd34fc
>File is: /lib/modules/6.3.0-rc7+/kernel/fs/xfs/xfs.ko
>On file address is: 0x11359c
>Objdump command is: objdump -z -d --start-address=0x1134fc 
> --stop-address=0x11357c /lib/modules/6.3.0-rc7+/kernel/fs/xfs/xfs.ko
>objdump read too few bytes: 128
>test child finished with -1
> end 
>Object code reading: FAILED!
>
> This issue happens ramdomly depending on the sample ip
> captured during the test run. In some cases, the same ip falls
> in the xfs module. The test does an objdump on the xfs.ko file and
> compares it with the dump from the dso that perf captures. But
> since the range of ip address falls in debug info section, it fails

This is strange.  Why did it fall into the debug section?

Thanks,
Namhyung


> to find the address range with objdump. Fix the objdump option so
> as to disasseble all sections to check the address range.
>
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/tests/code-reading.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/perf/tests/code-reading.c b/tools/perf/tests/code-reading.c
> index ed3815163d1b..02132478856a 100644
> --- a/tools/perf/tests/code-reading.c
> +++ b/tools/perf/tests/code-reading.c
> @@ -184,7 +184,7 @@ static int read_via_objdump(const char *filename, u64 
> addr, void *buf,
> FILE *f;
> int ret;
>
> -   fmt = "%s -z -d --start-address=0x%"PRIx64" 
> --stop-address=0x%"PRIx64" %s";
> +   fmt = "%s -z -D --start-address=0x%"PRIx64" 
> --stop-address=0x%"PRIx64" %s";
> ret = snprintf(cmd, sizeof(cmd), fmt, "objdump", addr, addr + len,
>filename);
> if (ret <= 0 || (size_t)ret >= sizeof(cmd))
> --
> 2.35.3
>

Re: [PATCH] tools/perf/tests: Fix Basic BPF llvm compile to check for libbpf support

[Namhyung Kim]

Hello,

On Mon, Jul 3, 2023 at 10:03 PM Athira Rajeev
 wrote:
>
> Basic BPF llvm compile fails in systems with libbpf
> that doesn't support BTF. Log shows below information.
>
> libbpf: BTF is required, but is missing or corrupted.
> Failed to parse test case 'Basic BPF llvm compile'
> test child finished with -2
>  end 
>
> Here BPF llvm compile fails due to missing BTF support.
> Fix the llvm test to skip the test incase BTF support is
> missing.
>
> Signed-off-by: Athira Rajeev 
> ---
>  tools/perf/tests/llvm.c | 7 ++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/tests/llvm.c b/tools/perf/tests/llvm.c
> index 0bc25a56cfef..4c73c9eab0bb 100644
> --- a/tools/perf/tests/llvm.c
> +++ b/tools/perf/tests/llvm.c
> @@ -4,6 +4,7 @@
>  #include 
>  #include "tests.h"
>  #include "debug.h"
> +#include 
>
>  #ifdef HAVE_LIBBPF_SUPPORT
>  #include 
> @@ -14,8 +15,12 @@ static int test__bpf_parsing(void *obj_buf, size_t 
> obj_buf_sz)
> struct bpf_object *obj;
>
> obj = bpf_object__open_mem(obj_buf, obj_buf_sz, NULL);
> -   if (libbpf_get_error(obj))
> +   if (libbpf_get_error(obj)) {
> +   /* Skip if there is no BTF support */
> +   if (errno == ENOENT)

Can we use the return value of libbpf_get_error() instead of
using the errno?

Also I'm curious why it requires BTF even if the BPF program
doesn't use it.  Is there an option or something for that?

Thanks,
Namhyung


> +   return TEST_SKIP;
> return TEST_FAIL;
> +   }
> bpf_object__close(obj);
> return TEST_OK;
>  }
> --
> 2.27.0
>

Re: [PATCH] syscalls: Cleanup references to sys_lookup_dcookie()

[Namhyung Kim]

Hello,

On Wed, Jun 28, 2023 at 4:44 PM Randy Dunlap  wrote:
>
>
>
> On 6/28/23 16:09, Sohil Mehta wrote:
> > commit 'be65de6b03aa ("fs: Remove dcookies support")' removed the
> > syscall definition for lookup_dcookie.  However, syscall tables still
> > point to the old sys_lookup_dcookie() definition. Update syscall tables
> > of all architectures to directly point to sys_ni_syscall() instead.
> >
> > Signed-off-by: Sohil Mehta 
>
> Reviewed-by: Randy Dunlap 

I was about to say that it'd be nice if you split the tools/perf part
since it can support old kernels.  But if the syscall is only used for
oprofile then probably perf doesn't need to care about it. :)

For the perf part,
Acked-by: Namhyung Kim 

Thanks,
Namhyung

Re: [PATCH 00/22] tools/perf: Fix shellcheck coding/formatting issues of perf tool shell scripts

[Namhyung Kim]

Hello,

On Wed, Jun 21, 2023 at 1:30 AM Athira Rajeev
 wrote:
>
> Patchset covers a set of fixes for coding/formatting issues observed while
> running shellcheck tool on the perf shell scripts.
>
> This cleanup is a pre-requisite to include a build option for shellcheck
> discussed here: https://www.spinics.net/lists/linux-perf-users/msg25553.html
> First set of patches were posted here:
> https://lore.kernel.org/linux-perf-users/53b7d823-1570-4289-a632-2205ee2b5...@linux.vnet.ibm.com/T/#t
>
> This patchset covers remaining set of shell scripts which needs
> fix. Patch 1 is resubmission of patch 6 from the initial series.
> Patch 15, 16 and 22 touches code from tools/perf/trace/beauty.
> Other patches are fixes for scripts from tools/perf/tests.

Thanks for this work.  But it seems there are some issues
even after applying this series.

  $ for F in $(find tests/shell/ -perm -o=x -name '*.sh'); do
  > shellcheck $F; done | grep -c -e '(info)' -e '(warning)'
  29

  $ for F in $(find tests/shell/ -perm -o=x -name '*.sh'); do
  > shellcheck $F; done | grep ^In
  In tests/shell/pipe_test.sh line 10:
  In tests/shell/pipe_test.sh line 15:
  In tests/shell/pipe_test.sh line 20:
  In tests/shell/pipe_test.sh line 21:
  In tests/shell/pipe_test.sh line 26:
  In tests/shell/pipe_test.sh line 27:
  In tests/shell/pipe_test.sh line 33:
  In tests/shell/record+zstd_comp_decomp.sh line 16:
  In tests/shell/record+zstd_comp_decomp.sh line 22:
  In tests/shell/record+zstd_comp_decomp.sh line 27:
  In tests/shell/record+zstd_comp_decomp.sh line 28:
  In tests/shell/record+zstd_comp_decomp.sh line 29:
  In tests/shell/record+zstd_comp_decomp.sh line 30:
  In tests/shell/record+zstd_comp_decomp.sh line 36:
  In tests/shell/coresight/thread_loop_check_tid_2.sh line 8:
  In tests/shell/coresight/thread_loop_check_tid_2.sh line 11:
  In tests/shell/coresight/thread_loop_check_tid_2.sh line 14:

Also unfortunately there's some update in the shell tests.
Please rebase onto the perf-tools-next.

Thanks,
Namhyung


>
> Akanksha J N (1):
>   tools/perf/tests: Fix shellcheck warnings for
> trace+probe_vfs_getname.sh
>
> Athira Rajeev (11):
>   tools/perf/tests: fix test_arm_spe_fork.sh signal case issues
>   tools/perf/tests: Fix unused variable references in
> stat+csv_summary.sh testcase
>   tools/perf/tests: fix shellcheck warning for
> test_perf_data_converter_json.sh testcase
>   tools/perf/tests: Fix shellcheck issue for stat_bpf_counters.sh
> testcase
>   tools/perf/tests: Fix shellcheck issues in
> tests/shell/stat+shadow_stat.sh tetscase
>   tools/perf/tests: Fix shellcheck warnings for
> thread_loop_check_tid_10.sh
>   tools/perf/tests: Fix shellcheck warnings for unroll_loop_thread_10.sh
>   tools/perf/tests: Fix shellcheck warnings for lib/probe_vfs_getname.sh
>   tools/perf/tests: Fix the shellcheck wanrings in lib/waiting.sh
>   tools/perf/trace: Fix x86_arch_prctl.sh to address shellcheck warnings
>   tools/perf/arch/x86: Fix syscalltbl.sh to address shellcheck warnings
>
> Kajol Jain (10):
>   tools/perf/tests: Fix shellcheck warning for probe_vfs_getname.sh
> testcase
>   tools/perf/tests: Fix shellcheck warning for record_offcpu.sh testcase
>   tools/perf/tests: Fix shellcheck issue for lock_contention.sh testcase
>   tools/perf/tests: Fix shellcheck issue for stat_bpf_counters_cgrp.sh
> testcase
>   tools/perf/tests: Fix shellcheck warning for asm_pure_loop.sh shell
> script
>   tools/perf/tests: Fix shellcheck warning for memcpy_thread_16k_10.sh
> shell script
>   tools/perf/tests: Fix shellcheck warning for coresight.sh shell script
>   tools/perf/tests: Fix shellcheck warning for probe.sh shell script
>   tools/perf/trace: Fix shellcheck issue for arch_errno_names.sh script
>   tools/perf: Fix shellcheck issue for check-headers.sh script
>
>  .../arch/x86/entry/syscalls/syscalltbl.sh |  2 +-
>  tools/perf/check-headers.sh   |  6 ++--
>  .../tests/shell/coresight/asm_pure_loop.sh|  2 +-
>  .../shell/coresight/memcpy_thread_16k_10.sh   |  2 +-
>  .../coresight/thread_loop_check_tid_10.sh |  2 +-
>  .../shell/coresight/unroll_loop_thread_10.sh  |  2 +-
>  tools/perf/tests/shell/lib/coresight.sh   |  3 +-
>  tools/perf/tests/shell/lib/probe.sh   |  1 +
>  .../perf/tests/shell/lib/probe_vfs_getname.sh |  5 ++--
>  tools/perf/tests/shell/lib/waiting.sh |  1 +
>  tools/perf/tests/shell/lock_contention.sh | 12 
>  tools/perf/tests/shell/probe_vfs_getname.sh   |  4 +--
>  tools/perf/tests/shell/record_offcpu.sh   |  6 ++--
>  tools/perf/tests/shell/stat+csv_summary.sh|  4 +--
>  tools/perf/tests/shell/stat+shadow_stat.sh|  4 +--
>  tools/perf/tests/shell/stat_bpf_counters.sh   |  4 +--
>  .../tests/shell/stat_bpf_counters_cgrp.sh | 28 ---
>  tools/perf/tests/shell/test_arm_spe_fork.sh   |  2 +-
>  .../shell/test_perf_data_converter_json.sh|  2 +-
>  

Re: [PATCH 15/17] perf tests task_analyzer: fix bad substitution ${$1}

[Namhyung Kim]

Hello Aditya,

On Fri, Jun 23, 2023 at 11:19 AM Aditya Gupta  wrote:
>
> Hello, Namhyung,
>
> On 23/06/23 04:55, Namhyung Kim wrote:
> > Ok, I found two problems.
> >
> > ...
> >
> > The first one is related to this message.  It couldn't find the script
> > (task-analyzer.py) because PERF_EXEC_PATH is not set.
> > Running with --exec-path=$PWD was ok, but I got a segfault.
> >
> > The other problem is in set_regs_in_dict().  It tries to capture
> > register values in the sample and save them to a dictionary.
> > The sample doesn't have registers so it should have no problem.
> > But the 'bf' was not initialized properly when size is 0, and it led
> > PyUnicode_FromString() returning NULL.
> >
> > After the changes, it ran ok:
> >
> > $ sudo ./perf test -v task
> > 116: perf script task-analyzer tests :
> > --- start ---
> > test child forked, pid 204088
> > PASS: "test_basic"
> > PASS: "test_ns_rename"
> > PASS: "test_ms_filtertasks_highlight"
> > PASS: "test_extended_times_timelimit_limittasks"
> > PASS: "test_summary"
> > PASS: "test_summaryextended"
> > PASS: "test_summaryonly"
> > PASS: "test_extended_times_summary_ns"
> > PASS: "test_extended_times_summary_ns"
> > PASS: "test_csv"
> > PASS: "test_csvsummary"
> > PASS: "test_csv_extended_times"
> > PASS: "test_csvsummary_extended"
> > test child finished with 0
> >  end 
> > perf script task-analyzer tests: Ok
> This is interesting. I did not encounter these earlier.
>
> > I'll send the fixes soon.
> >
> Thanks for fixing it.

No problem.

> >>> You can try the perf-tools-next branch in the perf/perft-tools-next.git 
> >>> repo.
> >>>
> >>>git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git
> >>>
> >>>$ make clean all
> >>>$ sudo ./perf test -v task
> >>>
>
> I tested with the steps provided, i.e.. clone, /*apply patch 15 (added
> this step myself)*/, clean, make, ./perf test -v test. But still- test
> passed:
>
> '''
> ~/temp_clones/perf-tools-next/tools/perf git:(master) sudo ./perf test
> -v test
>
> ...
>
> --- start ---
> test child forked, pid 75261
> PASS: "test_basic"
> PASS: "test_ns_rename"
> PASS: "test_ms_filtertasks_highlight"
> PASS: "test_extended_times_timelimit_limittasks"
> PASS: "test_summary"
> PASS: "test_summaryextended"
> PASS: "test_summaryonly"
> PASS: "test_extended_times_summary_ns"
> PASS: "test_extended_times_summary_ns"
> PASS: "test_csv"
> PASS: "test_csvsummary"
> PASS: "test_csv_extended_times"
> PASS: "test_csvsummary_extended"
> test child finished with 0
>  end 
> '''
>
> Maybe my environment had that other things set in a way that, I did not
> face any issue (I don't recall doing any special thing with my
> environment though, and these patches have been tested multiple times by
> more people also).
> But thanks for your efforts to debug and fix the issue.

I don't know but you might either set PERF_EXEC_PATH env or
install the scripts somewhere that can be found by default.

Thanks,
Namhyung

Re: [PATCH 15/17] perf tests task_analyzer: fix bad substitution ${$1}

[Namhyung Kim]

Ok, I found two problems.

On Thu, Jun 22, 2023 at 2:33 PM Namhyung Kim  wrote:
>
> Hello Aditya,
>
> On Wed, Jun 21, 2023 at 11:43 AM Aditya Gupta  wrote:
> >
> > Hello Namhyung,
> >
> > On 21/06/23 20:53, Namhyung Kim wrote:
> > > Hello Aditya,
> > >
> > > On Wed, Jun 21, 2023 at 3:05 AM Aditya Gupta  
> > > wrote:
> > >> Hello Namhyung,
> > >>
> > >> On 21/06/23 06:18, Namhyung Kim wrote:
> > >>> ...
> > >>>
> > >>> $ sudo ./perf test -v task
> > >>> 114: perf script task-analyzer tests :
> > >>> --- start ---
> > >>> test child forked, pid 1771042
> > >>> Please specify a valid report script(see 'perf script -l' for listing)

The first one is related to this message.  It couldn't find the script
(task-analyzer.py) because PERF_EXEC_PATH is not set.
Running with --exec-path=$PWD was ok, but I got a segfault.

The other problem is in set_regs_in_dict().  It tries to capture
register values in the sample and save them to a dictionary.
The sample doesn't have registers so it should have no problem.
But the 'bf' was not initialized properly when size is 0, and it led
PyUnicode_FromString() returning NULL.

After the changes, it ran ok:

$ sudo ./perf test -v task
116: perf script task-analyzer tests :
--- start ---
test child forked, pid 204088
PASS: "test_basic"
PASS: "test_ns_rename"
PASS: "test_ms_filtertasks_highlight"
PASS: "test_extended_times_timelimit_limittasks"
PASS: "test_summary"
PASS: "test_summaryextended"
PASS: "test_summaryonly"
PASS: "test_extended_times_summary_ns"
PASS: "test_extended_times_summary_ns"
PASS: "test_csv"
PASS: "test_csvsummary"
PASS: "test_csv_extended_times"
PASS: "test_csvsummary_extended"
test child finished with 0
 end 
perf script task-analyzer tests: Ok

I'll send the fixes soon.

Thanks,
Namhyung


> > >>> FAIL: "invocation of perf command failed" Error message: ""
> > >>> FAIL: "test_basic" Error message: "Failed to find required 
> > >>> string:'Comm'."
> > >>> Please specify a valid report script(see 'perf script -l' for listing)
> > >>> FAIL: "invocation of perf command failed" Error message: ""
> > >>> FAIL: "test_ns_rename" Error message: "Failed to find required 
> > >>> string:'Comm'."
> > >>> ...
> > >>> test child finished with -1
> > >>>  end 
> > >>> perf script task-analyzer tests: FAILED!
> > >> Can you please check if your environment has libtraceevent devel
> > >> libraries (or did you compile with `make NO_LIBTRACEEVENT=1`) ? When
> > >> libtraceevent support is not there, perf record fails and so perf.data
> > >> doesn't contain the strings it's searching for and hence those errors
> > >>
> > >> The error you mentioned has been mentioned and fixed in patch 17/17 of
> > >> this series.
> > > Thanks for your reply but It has libtraceevent.  Also, shouldn't it
> > > skip if it's not?
> > >
> > > Thanks,
> > > Namhyung
> >
> > The skipping is handled in the 17th patch in this series, and
> > considering that patch has also been applied, it will skip the tests if
> > perf wasn't built with proper libtraceevent support.
>
> Right, I tested it with the whole series applied.
> It skipped to run when I disable libtraceevent build.
>
> >
> > Back to the error, Sorry but I tested again on my system and am unable to 
> > reproduce the issue you are seeing when built with libtraceevent support.
> >
> > This is what I did:
> >
> > 0. git clone --depth=1 https://github.com/torvalds/linux
> > 0. cd linux/tools/perf
> > 0. git am patch_15/17.patch
> >
> > > I applied ONLY this patch (15/17) of this series, to a fresh linux tree
> >
> > 1. dnf install libtraceevent-devel
> > 2. make clean && make
> > 3. sudo ./perf test -v "perf script task-analyzer tests"   # Working 
> > fine, tests passed
> >
> > 4. dnf remove libtraceevent-devel
> > 5. make clean && make  # There will 
> > also be a warning during build: "libtraceevent is missing limiting 
> > functionality"
> > 6. sudo ./perf test -v "perf script task-analyzer tests" 

Re: [PATCH 15/17] perf tests task_analyzer: fix bad substitution ${$1}

[Namhyung Kim]

Hello Aditya,

On Wed, Jun 21, 2023 at 11:43 AM Aditya Gupta  wrote:
>
> Hello Namhyung,
>
> On 21/06/23 20:53, Namhyung Kim wrote:
> > Hello Aditya,
> >
> > On Wed, Jun 21, 2023 at 3:05 AM Aditya Gupta  wrote:
> >> Hello Namhyung,
> >>
> >> On 21/06/23 06:18, Namhyung Kim wrote:
> >>> ...
> >>>
> >>> $ sudo ./perf test -v task
> >>> 114: perf script task-analyzer tests :
> >>> --- start ---
> >>> test child forked, pid 1771042
> >>> Please specify a valid report script(see 'perf script -l' for listing)
> >>> FAIL: "invocation of perf command failed" Error message: ""
> >>> FAIL: "test_basic" Error message: "Failed to find required string:'Comm'."
> >>> Please specify a valid report script(see 'perf script -l' for listing)
> >>> FAIL: "invocation of perf command failed" Error message: ""
> >>> FAIL: "test_ns_rename" Error message: "Failed to find required 
> >>> string:'Comm'."
> >>> ...
> >>> test child finished with -1
> >>>  end 
> >>> perf script task-analyzer tests: FAILED!
> >> Can you please check if your environment has libtraceevent devel
> >> libraries (or did you compile with `make NO_LIBTRACEEVENT=1`) ? When
> >> libtraceevent support is not there, perf record fails and so perf.data
> >> doesn't contain the strings it's searching for and hence those errors
> >>
> >> The error you mentioned has been mentioned and fixed in patch 17/17 of
> >> this series.
> > Thanks for your reply but It has libtraceevent.  Also, shouldn't it
> > skip if it's not?
> >
> > Thanks,
> > Namhyung
>
> The skipping is handled in the 17th patch in this series, and
> considering that patch has also been applied, it will skip the tests if
> perf wasn't built with proper libtraceevent support.

Right, I tested it with the whole series applied.
It skipped to run when I disable libtraceevent build.

>
> Back to the error, Sorry but I tested again on my system and am unable to 
> reproduce the issue you are seeing when built with libtraceevent support.
>
> This is what I did:
>
> 0. git clone --depth=1 https://github.com/torvalds/linux
> 0. cd linux/tools/perf
> 0. git am patch_15/17.patch
>
> > I applied ONLY this patch (15/17) of this series, to a fresh linux tree
>
> 1. dnf install libtraceevent-devel
> 2. make clean && make
> 3. sudo ./perf test -v "perf script task-analyzer tests"   # Working 
> fine, tests passed
>
> 4. dnf remove libtraceevent-devel
> 5. make clean && make  # There will 
> also be a warning during build: "libtraceevent is missing limiting 
> functionality"
> 6. sudo ./perf test -v "perf script task-analyzer tests"   # Fails with 
> the error you posted, which was the case till now, it's skipped when the 17th 
> patch is also applied and perf built without libtraceevent support
>
> The error in the second case (without proper libtraceevent support) is 
> expected, as it was the case till now, that is fixed by the 17th patch, try 
> applying that also and build perf with `make NO_LIBTRACEEVENT=1`, it will 
> skip then.
>
> Can you guide me with the steps to reproduce the error ?

You can try the perf-tools-next branch in the perf/perft-tools-next.git repo.

  git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git

  $ make clean all
  $ sudo ./perf test -v task

>
> Sidenote: Just in case, please ensure you are running the perf as root here 
> as `perf record -e sched:sched_switch -a -- sleep 1` requires root, which has 
> been used in `prepare_perf_data`

Sure, I ran the perf test as root.

Thanks,
Namhyung

Re: [PATCH 15/17] perf tests task_analyzer: fix bad substitution ${$1}

[Namhyung Kim]

Hello Aditya,

On Wed, Jun 21, 2023 at 3:05 AM Aditya Gupta  wrote:
>
> Hello Namhyung,
>
> On 21/06/23 06:18, Namhyung Kim wrote:
> > Hello,
> >
> > On Tue, Jun 13, 2023 at 1:06 PM Arnaldo Carvalho de Melo
> >  wrote:
> >> Em Tue, Jun 13, 2023 at 10:11:43PM +0530, Athira Rajeev escreveu:
> >>> This issue due to ${$1} caused all function calls to give error in
> >>> `find_str_or_fail` line, and so no test runs completely. But
> >>> 'perf test "perf script task-analyzer tests"' wrongly reports
> >>> that tests passed with the status OK, which is wrong considering
> >>> the tests didn't even run completely
> >>>
> >>> Fixes: e8478b84d6ba ("perf test: add new task-analyzer tests")
> >>> Signed-off-by: Athira Rajeev 
> >>> Signed-off-by: Kajol Jain 
> >>> Signed-off-by: Aditya Gupta 
> >>> ---
> > I'm seeing a different error even after this fix.
> > Can you please take a look?
> >
> > Thanks,
> > Namhyung
> >
> >
> > $ sudo ./perf test -v task
> > 114: perf script task-analyzer tests :
> > --- start ---
> > test child forked, pid 1771042
> > Please specify a valid report script(see 'perf script -l' for listing)
> > FAIL: "invocation of perf command failed" Error message: ""
> > FAIL: "test_basic" Error message: "Failed to find required string:'Comm'."
> > Please specify a valid report script(see 'perf script -l' for listing)
> > FAIL: "invocation of perf command failed" Error message: ""
> > FAIL: "test_ns_rename" Error message: "Failed to find required 
> > string:'Comm'."
> > ...
> > test child finished with -1
> >  end 
> > perf script task-analyzer tests: FAILED!
> Can you please check if your environment has libtraceevent devel
> libraries (or did you compile with `make NO_LIBTRACEEVENT=1`) ? When
> libtraceevent support is not there, perf record fails and so perf.data
> doesn't contain the strings it's searching for and hence those errors
>
> The error you mentioned has been mentioned and fixed in patch 17/17 of
> this series.

Thanks for your reply but It has libtraceevent.  Also, shouldn't it
skip if it's not?

Thanks,
Namhyung

Re: [PATCH 15/17] perf tests task_analyzer: fix bad substitution ${$1}

[Namhyung Kim]

Hello,

On Tue, Jun 13, 2023 at 1:06 PM Arnaldo Carvalho de Melo
 wrote:
>
> Em Tue, Jun 13, 2023 at 10:11:43PM +0530, Athira Rajeev escreveu:
> > From: Aditya Gupta 
> >
> > ${$1} gives bad substitution error on sh, bash, and zsh. This seems like
> > a typo, and this patch modifies it to $1, since that is what it's usage
> > looks like from wherever `check_exec_0` is called.
>
> Nicely spotted!
>
> Please add the people that last touched the problem to the cc list,
> specially when it fixes a bug.
>
> Thanks for adding a Fixes tag, that helps the sta...@kernel.org guys to
> get this propagated to supported kernel releases.
>
> I've added the test author to the CC list in this message.
>
> thanks!
>
> - Arnaldo
>
> > This issue due to ${$1} caused all function calls to give error in
> > `find_str_or_fail` line, and so no test runs completely. But
> > 'perf test "perf script task-analyzer tests"' wrongly reports
> > that tests passed with the status OK, which is wrong considering
> > the tests didn't even run completely
> >
> > Fixes: e8478b84d6ba ("perf test: add new task-analyzer tests")
> > Signed-off-by: Athira Rajeev 
> > Signed-off-by: Kajol Jain 
> > Signed-off-by: Aditya Gupta 
> > ---

I'm seeing a different error even after this fix.
Can you please take a look?

Thanks,
Namhyung


$ sudo ./perf test -v task
114: perf script task-analyzer tests :
--- start ---
test child forked, pid 1771042
Please specify a valid report script(see 'perf script -l' for listing)
FAIL: "invocation of perf command failed" Error message: ""
FAIL: "test_basic" Error message: "Failed to find required string:'Comm'."
Please specify a valid report script(see 'perf script -l' for listing)
FAIL: "invocation of perf command failed" Error message: ""
FAIL: "test_ns_rename" Error message: "Failed to find required string:'Comm'."
...
test child finished with -1
 end 
perf script task-analyzer tests: FAILED!

[PATCH 4/8] perf/core: Add perf_sample_save_brstack() helper

[Namhyung Kim]

When it saves the branch stack to the perf sample data, it needs to
update the sample flags and the dynamic size.  To make sure this,
add the perf_sample_save_brstack() helper and convert all call sites.

Cc: linuxppc-dev@lists.ozlabs.org
Cc: x...@kernel.org
Suggested-by: Peter Zijlstra 
Acked-by: Jiri Olsa 
Tested-by: Jiri Olsa 
Signed-off-by: Namhyung Kim 
---
 arch/powerpc/perf/core-book3s.c |  3 +-
 arch/x86/events/amd/core.c  |  6 +--
 arch/x86/events/intel/core.c|  6 +--
 arch/x86/events/intel/ds.c  |  9 ++---
 include/linux/perf_event.h  | 66 -
 kernel/events/core.c| 16 +++-
 6 files changed, 53 insertions(+), 53 deletions(-)

diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c
index bf318dd9b709..8c1f7def596e 100644
--- a/arch/powerpc/perf/core-book3s.c
+++ b/arch/powerpc/perf/core-book3s.c
@@ -2313,8 +2313,7 @@ static void record_and_restart(struct perf_event *event, 
unsigned long val,
struct cpu_hw_events *cpuhw;
cpuhw = this_cpu_ptr(_hw_events);
power_pmu_bhrb_read(event, cpuhw);
-   data.br_stack = >bhrb_stack;
-   data.sample_flags |= PERF_SAMPLE_BRANCH_STACK;
+   perf_sample_save_brstack(, event, 
>bhrb_stack);
}
 
if (event->attr.sample_type & PERF_SAMPLE_DATA_SRC &&
diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
index d6f3703e4119..463f3eb8bbd7 100644
--- a/arch/x86/events/amd/core.c
+++ b/arch/x86/events/amd/core.c
@@ -928,10 +928,8 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs)
if (!x86_perf_event_set_period(event))
continue;
 
-   if (has_branch_stack(event)) {
-   data.br_stack = >lbr_stack;
-   data.sample_flags |= PERF_SAMPLE_BRANCH_STACK;
-   }
+   if (has_branch_stack(event))
+   perf_sample_save_brstack(, event, 
>lbr_stack);
 
if (perf_event_overflow(event, , regs))
x86_pmu_stop(event, 0);
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 29d2d0411caf..14f0a746257d 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -3036,10 +3036,8 @@ static int handle_pmi_common(struct pt_regs *regs, u64 
status)
 
perf_sample_data_init(, 0, event->hw.last_period);
 
-   if (has_branch_stack(event)) {
-   data.br_stack = >lbr_stack;
-   data.sample_flags |= PERF_SAMPLE_BRANCH_STACK;
-   }
+   if (has_branch_stack(event))
+   perf_sample_save_brstack(, event, 
>lbr_stack);
 
if (perf_event_overflow(event, , regs))
x86_pmu_stop(event, 0);
diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c
index 158cf845fc80..07c8a2cdc3ee 100644
--- a/arch/x86/events/intel/ds.c
+++ b/arch/x86/events/intel/ds.c
@@ -1720,10 +1720,8 @@ static void setup_pebs_fixed_sample_data(struct 
perf_event *event,
data->sample_flags |= PERF_SAMPLE_TIME;
}
 
-   if (has_branch_stack(event)) {
-   data->br_stack = >lbr_stack;
-   data->sample_flags |= PERF_SAMPLE_BRANCH_STACK;
-   }
+   if (has_branch_stack(event))
+   perf_sample_save_brstack(data, event, >lbr_stack);
 }
 
 static void adaptive_pebs_save_regs(struct pt_regs *regs,
@@ -1883,8 +1881,7 @@ static void setup_pebs_adaptive_sample_data(struct 
perf_event *event,
 
if (has_branch_stack(event)) {
intel_pmu_store_pebs_lbrs(lbr);
-   data->br_stack = >lbr_stack;
-   data->sample_flags |= PERF_SAMPLE_BRANCH_STACK;
+   perf_sample_save_brstack(data, event, >lbr_stack);
}
}
 
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 569dfac5887f..7db0e9cc2682 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1102,6 +1102,31 @@ extern u64 perf_event_read_value(struct perf_event 
*event,
 
 extern struct perf_callchain_entry *perf_callchain(struct perf_event *event, 
struct pt_regs *regs);
 
+static inline bool branch_sample_no_flags(const struct perf_event *event)
+{
+   return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_FLAGS;
+}
+
+static inline bool branch_sample_no_cycles(const struct perf_event *event)
+{
+   return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_CYCLES;
+}
+
+static inline bool branch_sample_type(const struct perf_event *event)
+{
+   return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_TYPE_SAVE;
+}
+
+static inline

[PATCH 4/8] perf/core: Add perf_sample_save_brstack() helper

[Namhyung Kim]

When it saves the branch stack to the perf sample data, it needs to
update the sample flags and the dynamic size.  To make sure this,
add the perf_sample_save_brstack() helper and convert all call sites.

Cc: linuxppc-dev@lists.ozlabs.org
Cc: x...@kernel.org
Suggested-by: Peter Zijlstra 
Signed-off-by: Namhyung Kim 
---
 arch/powerpc/perf/core-book3s.c |  3 +-
 arch/x86/events/amd/core.c  |  6 +--
 arch/x86/events/intel/core.c|  6 +--
 arch/x86/events/intel/ds.c  |  9 ++---
 include/linux/perf_event.h  | 66 -
 kernel/events/core.c| 16 +++-
 6 files changed, 53 insertions(+), 53 deletions(-)

diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c
index bf318dd9b709..8c1f7def596e 100644
--- a/arch/powerpc/perf/core-book3s.c
+++ b/arch/powerpc/perf/core-book3s.c
@@ -2313,8 +2313,7 @@ static void record_and_restart(struct perf_event *event, 
unsigned long val,
struct cpu_hw_events *cpuhw;
cpuhw = this_cpu_ptr(_hw_events);
power_pmu_bhrb_read(event, cpuhw);
-   data.br_stack = >bhrb_stack;
-   data.sample_flags |= PERF_SAMPLE_BRANCH_STACK;
+   perf_sample_save_brstack(, event, 
>bhrb_stack);
}
 
if (event->attr.sample_type & PERF_SAMPLE_DATA_SRC &&
diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
index d6f3703e4119..463f3eb8bbd7 100644
--- a/arch/x86/events/amd/core.c
+++ b/arch/x86/events/amd/core.c
@@ -928,10 +928,8 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs)
if (!x86_perf_event_set_period(event))
continue;
 
-   if (has_branch_stack(event)) {
-   data.br_stack = >lbr_stack;
-   data.sample_flags |= PERF_SAMPLE_BRANCH_STACK;
-   }
+   if (has_branch_stack(event))
+   perf_sample_save_brstack(, event, 
>lbr_stack);
 
if (perf_event_overflow(event, , regs))
x86_pmu_stop(event, 0);
diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 29d2d0411caf..14f0a746257d 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -3036,10 +3036,8 @@ static int handle_pmi_common(struct pt_regs *regs, u64 
status)
 
perf_sample_data_init(, 0, event->hw.last_period);
 
-   if (has_branch_stack(event)) {
-   data.br_stack = >lbr_stack;
-   data.sample_flags |= PERF_SAMPLE_BRANCH_STACK;
-   }
+   if (has_branch_stack(event))
+   perf_sample_save_brstack(, event, 
>lbr_stack);
 
if (perf_event_overflow(event, , regs))
x86_pmu_stop(event, 0);
diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c
index 158cf845fc80..07c8a2cdc3ee 100644
--- a/arch/x86/events/intel/ds.c
+++ b/arch/x86/events/intel/ds.c
@@ -1720,10 +1720,8 @@ static void setup_pebs_fixed_sample_data(struct 
perf_event *event,
data->sample_flags |= PERF_SAMPLE_TIME;
}
 
-   if (has_branch_stack(event)) {
-   data->br_stack = >lbr_stack;
-   data->sample_flags |= PERF_SAMPLE_BRANCH_STACK;
-   }
+   if (has_branch_stack(event))
+   perf_sample_save_brstack(data, event, >lbr_stack);
 }
 
 static void adaptive_pebs_save_regs(struct pt_regs *regs,
@@ -1883,8 +1881,7 @@ static void setup_pebs_adaptive_sample_data(struct 
perf_event *event,
 
if (has_branch_stack(event)) {
intel_pmu_store_pebs_lbrs(lbr);
-   data->br_stack = >lbr_stack;
-   data->sample_flags |= PERF_SAMPLE_BRANCH_STACK;
+   perf_sample_save_brstack(data, event, >lbr_stack);
}
}
 
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 569dfac5887f..7db0e9cc2682 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1102,6 +1102,31 @@ extern u64 perf_event_read_value(struct perf_event 
*event,
 
 extern struct perf_callchain_entry *perf_callchain(struct perf_event *event, 
struct pt_regs *regs);
 
+static inline bool branch_sample_no_flags(const struct perf_event *event)
+{
+   return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_FLAGS;
+}
+
+static inline bool branch_sample_no_cycles(const struct perf_event *event)
+{
+   return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_NO_CYCLES;
+}
+
+static inline bool branch_sample_type(const struct perf_event *event)
+{
+   return event->attr.branch_sample_type & PERF_SAMPLE_BRANCH_TYPE_SAVE;
+}
+
+static inline bool branch_sample_hw_in

[PATCH for v5.10] perf symbol: Remove arch__symbols__fixup_end()

[Namhyung Kim]

Now the generic code can handle kallsyms fixup properly so no need to
keep the arch-functions anymore.

Signed-off-by: Namhyung Kim 
Acked-by: Ian Rogers 
Cc: Heiko Carstens 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: Leo Yan 
Cc: Mark Rutland 
Cc: Masami Hiramatsu 
Cc: Mathieu Poirier 
Cc: Michael Ellerman 
Cc: Michael Petlan 
Cc: Peter Zijlstra 
Cc: Song Liu 
Cc: Will Deacon 
Cc: linux-s...@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Link: https://lore.kernel.org/r/20220416004048.1514900-4-namhy...@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo 
---
The original commit id is a5d20d42a2f2dc2b2f9e9361912062732414090d
 tools/perf/arch/arm64/util/Build |  1 -
 tools/perf/arch/arm64/util/machine.c | 27 ---
 tools/perf/arch/s390/util/machine.c  | 16 
 tools/perf/util/symbol.c |  5 -
 tools/perf/util/symbol.h |  1 -
 5 files changed, 50 deletions(-)
 delete mode 100644 tools/perf/arch/arm64/util/machine.c

diff --git a/tools/perf/arch/arm64/util/Build b/tools/perf/arch/arm64/util/Build
index b53294d74b01..eddaf9bf5729 100644
--- a/tools/perf/arch/arm64/util/Build
+++ b/tools/perf/arch/arm64/util/Build
@@ -1,5 +1,4 @@
 perf-y += header.o
-perf-y += machine.o
 perf-y += perf_regs.o
 perf-y += tsc.o
 perf-$(CONFIG_DWARF) += dwarf-regs.o
diff --git a/tools/perf/arch/arm64/util/machine.c 
b/tools/perf/arch/arm64/util/machine.c
deleted file mode 100644
index d41b27e781d3..
--- a/tools/perf/arch/arm64/util/machine.c
+++ /dev/null
@@ -1,27 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include 
-#include 
-#include "debug.h"
-#include "symbol.h"
-
-/* On arm64, kernel text segment start at high memory address,
- * for example 0x  8xxx . Modules start at a low memory
- * address, like 0x  00ax . When only samll amount of
- * memory is used by modules, gap between end of module's text segment
- * and start of kernel text segment may be reach 2G.
- * Therefore do not fill this gap and do not assign it to the kernel dso map.
- */
-
-#define SYMBOL_LIMIT (1 << 12) /* 4K */
-
-void arch__symbols__fixup_end(struct symbol *p, struct symbol *c)
-{
-   if ((strchr(p->name, '[') && strchr(c->name, '[') == NULL) ||
-   (strchr(p->name, '[') == NULL && strchr(c->name, '[')))
-   /* Limit range of last symbol in module and kernel */
-   p->end += SYMBOL_LIMIT;
-   else
-   p->end = c->start;
-   pr_debug4("%s sym:%s end:%#lx\n", __func__, p->name, p->end);
-}
diff --git a/tools/perf/arch/s390/util/machine.c 
b/tools/perf/arch/s390/util/machine.c
index 724efb2d842d..7219ecdb8423 100644
--- a/tools/perf/arch/s390/util/machine.c
+++ b/tools/perf/arch/s390/util/machine.c
@@ -34,19 +34,3 @@ int arch__fix_module_text_start(u64 *start, u64 *size, const 
char *name)
 
return 0;
 }
-
-/* On s390 kernel text segment start is located at very low memory addresses,
- * for example 0x1. Modules are located at very high memory addresses,
- * for example 0x3ff  . The gap between end of kernel text segment
- * and beginning of first module's text segment is very big.
- * Therefore do not fill this gap and do not assign it to the kernel dso map.
- */
-void arch__symbols__fixup_end(struct symbol *p, struct symbol *c)
-{
-   if (strchr(p->name, '[') == NULL && strchr(c->name, '['))
-   /* Last kernel symbol mapped to end of page */
-   p->end = roundup(p->end, page_size);
-   else
-   p->end = c->start;
-   pr_debug4("%s sym:%s end:%#lx\n", __func__, p->name, p->end);
-}
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 8f63cf8d0669..33954835c823 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -101,11 +101,6 @@ static int prefix_underscores_count(const char *str)
return tail - str;
 }
 
-void __weak arch__symbols__fixup_end(struct symbol *p, struct symbol *c)
-{
-   p->end = c->start;
-}
-
 const char * __weak arch__normalize_symbol_name(const char *name)
 {
return name;
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index 66d5b732bb7a..28721d761d91 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -230,7 +230,6 @@ const char *arch__normalize_symbol_name(const char *name);
 #define SYMBOL_A 0
 #define SYMBOL_B 1
 
-void arch__symbols__fixup_end(struct symbol *p, struct symbol *c);
 int arch__compare_symbol_names(const char *namea, const char *nameb);
 int arch__compare_symbol_names_n(const char *namea, const char *nameb,
 unsigned int n);
-- 
2.36.0.464.gb9c8b46e94-goog

[PATCH for v5.15] perf symbol: Remove arch__symbols__fixup_end()

[Namhyung Kim]

Now the generic code can handle kallsyms fixup properly so no need to
keep the arch-functions anymore.

Signed-off-by: Namhyung Kim 
Acked-by: Ian Rogers 
Cc: Heiko Carstens 
Cc: Ingo Molnar 
Cc: Jiri Olsa 
Cc: John Garry 
Cc: Leo Yan 
Cc: Mark Rutland 
Cc: Masami Hiramatsu 
Cc: Mathieu Poirier 
Cc: Michael Ellerman 
Cc: Michael Petlan 
Cc: Peter Zijlstra 
Cc: Song Liu 
Cc: Will Deacon 
Cc: linux-s...@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Link: https://lore.kernel.org/r/20220416004048.1514900-4-namhy...@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo 
---
The original commit id is a5d20d42a2f2dc2b2f9e9361912062732414090d

 tools/perf/arch/arm64/util/Build   |  1 -
 tools/perf/arch/arm64/util/machine.c   | 28 --
 tools/perf/arch/powerpc/util/Build |  1 -
 tools/perf/arch/powerpc/util/machine.c | 25 ---
 tools/perf/arch/s390/util/machine.c| 16 ---
 tools/perf/util/symbol.c   |  5 -
 tools/perf/util/symbol.h   |  1 -
 7 files changed, 77 deletions(-)
 delete mode 100644 tools/perf/arch/arm64/util/machine.c
 delete mode 100644 tools/perf/arch/powerpc/util/machine.c

diff --git a/tools/perf/arch/arm64/util/Build b/tools/perf/arch/arm64/util/Build
index 9fcb4e68add9..78dfc282e5e2 100644
--- a/tools/perf/arch/arm64/util/Build
+++ b/tools/perf/arch/arm64/util/Build
@@ -1,5 +1,4 @@
 perf-y += header.o
-perf-y += machine.o
 perf-y += perf_regs.o
 perf-y += tsc.o
 perf-y += pmu.o
diff --git a/tools/perf/arch/arm64/util/machine.c 
b/tools/perf/arch/arm64/util/machine.c
deleted file mode 100644
index 7e7714290a87..
--- a/tools/perf/arch/arm64/util/machine.c
+++ /dev/null
@@ -1,28 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include 
-#include 
-#include 
-#include "debug.h"
-#include "symbol.h"
-
-/* On arm64, kernel text segment starts at high memory address,
- * for example 0x  8xxx . Modules start at a low memory
- * address, like 0x  00ax . When only small amount of
- * memory is used by modules, gap between end of module's text segment
- * and start of kernel text segment may reach 2G.
- * Therefore do not fill this gap and do not assign it to the kernel dso map.
- */
-
-#define SYMBOL_LIMIT (1 << 12) /* 4K */
-
-void arch__symbols__fixup_end(struct symbol *p, struct symbol *c)
-{
-   if ((strchr(p->name, '[') && strchr(c->name, '[') == NULL) ||
-   (strchr(p->name, '[') == NULL && strchr(c->name, '[')))
-   /* Limit range of last symbol in module and kernel */
-   p->end += SYMBOL_LIMIT;
-   else
-   p->end = c->start;
-   pr_debug4("%s sym:%s end:%#" PRIx64 "\n", __func__, p->name, p->end);
-}
diff --git a/tools/perf/arch/powerpc/util/Build 
b/tools/perf/arch/powerpc/util/Build
index 8a79c4126e5b..0115f3166568 100644
--- a/tools/perf/arch/powerpc/util/Build
+++ b/tools/perf/arch/powerpc/util/Build
@@ -1,5 +1,4 @@
 perf-y += header.o
-perf-y += machine.o
 perf-y += kvm-stat.o
 perf-y += perf_regs.o
 perf-y += mem-events.o
diff --git a/tools/perf/arch/powerpc/util/machine.c 
b/tools/perf/arch/powerpc/util/machine.c
deleted file mode 100644
index e652a1aa8132..
--- a/tools/perf/arch/powerpc/util/machine.c
+++ /dev/null
@@ -1,25 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include 
-#include 
-#include 
-#include  // page_size
-#include "debug.h"
-#include "symbol.h"
-
-/* On powerpc kernel text segment start at memory addresses, 0xc000
- * whereas the modules are located at very high memory addresses,
- * for example 0xc0080xxx. The gap between end of kernel text segment
- * and beginning of first module's text segment is very high.
- * Therefore do not fill this gap and do not assign it to the kernel dso map.
- */
-
-void arch__symbols__fixup_end(struct symbol *p, struct symbol *c)
-{
-   if (strchr(p->name, '[') == NULL && strchr(c->name, '['))
-   /* Limit the range of last kernel symbol */
-   p->end += page_size;
-   else
-   p->end = c->start;
-   pr_debug4("%s sym:%s end:%#" PRIx64 "\n", __func__, p->name, p->end);
-}
diff --git a/tools/perf/arch/s390/util/machine.c 
b/tools/perf/arch/s390/util/machine.c
index 7644a4f6d4a4..98bc3f39d5f3 100644
--- a/tools/perf/arch/s390/util/machine.c
+++ b/tools/perf/arch/s390/util/machine.c
@@ -35,19 +35,3 @@ int arch__fix_module_text_start(u64 *start, u64 *size, const 
char *name)
 
return 0;
 }
-
-/* On s390 kernel text segment start is located at very low memory addresses,
- * for example 0x1. Modules are located at very high memory addresses,
- * for example 0x3ff  . The gap between end of kernel text segment
- * and beginning of first module's text segment is very big.
- * Therefore do not f

Re: [PATCH 1/3] perf symbol: Pass is_kallsyms to symbols__fixup_end()

[Namhyung Kim]

Hi Ian,

On Sat, Apr 16, 2022 at 7:59 AM Ian Rogers  wrote:
>
> On Fri, Apr 15, 2022 at 8:40 PM Namhyung Kim  wrote:
> >
> > The symbol fixup is necessary for symbols in kallsyms since they don't
> > have size info.  So we use the next symbol's address to calculate the
> > size.  Now it's also used for user binaries because sometimes they
> > miss size for hand-written asm functions.
> >
> > There's a arch-specific function to handle kallsyms differently but
> > currently it cannot distinguish kallsyms from others.  Pass this
> > information explicitly to handle it properly.  Note that those arch
> > functions will be moved to the generic function so I didn't added it
> > to the arch-functions.
>
> Thanks Namhyung, in:
> https://lore.kernel.org/linux-perf-users/20220412154817.2728324-3-irog...@google.com/
> I used "dso->kernel != DSO_SPACE__USER" in symbol-elf to make this
> more than just kallsyms as presumably kernel code is the issue. Do we
> know elf kernel code has correctly sized symbols?

Yeah, IIUC the whole point of the symbol end fixup is because the
kallsyms doesn't have the symbol size info.  Every ELF binaries
should have the size except for some hand-written asm functions
which missed adding it manually.  I guess that's the reason it was
added to other DSO loading paths.

Also considering "[" (and "]") in the symbol name is specific to
kallsyms which has both kernel and module symbols together.

Thanks,
Namhyung

[PATCH 3/3] perf symbol: Remove arch__symbols__fixup_end()

[Namhyung Kim]

Now the generic code can handle kallsyms fixup properly so no need to
keep the arch-functions anymore.

Signed-off-by: Namhyung Kim 
---
 tools/perf/arch/arm64/util/machine.c   | 21 -
 tools/perf/arch/powerpc/util/Build |  1 -
 tools/perf/arch/powerpc/util/machine.c | 25 -
 tools/perf/arch/s390/util/machine.c| 16 
 tools/perf/util/symbol.c   |  5 -
 tools/perf/util/symbol.h   |  1 -
 6 files changed, 69 deletions(-)
 delete mode 100644 tools/perf/arch/powerpc/util/machine.c

diff --git a/tools/perf/arch/arm64/util/machine.c 
b/tools/perf/arch/arm64/util/machine.c
index d2ce31e28cd7..41c1596e5207 100644
--- a/tools/perf/arch/arm64/util/machine.c
+++ b/tools/perf/arch/arm64/util/machine.c
@@ -8,27 +8,6 @@
 #include "callchain.h"
 #include "record.h"
 
-/* On arm64, kernel text segment starts at high memory address,
- * for example 0x  8xxx . Modules start at a low memory
- * address, like 0x  00ax . When only small amount of
- * memory is used by modules, gap between end of module's text segment
- * and start of kernel text segment may reach 2G.
- * Therefore do not fill this gap and do not assign it to the kernel dso map.
- */
-
-#define SYMBOL_LIMIT (1 << 12) /* 4K */
-
-void arch__symbols__fixup_end(struct symbol *p, struct symbol *c)
-{
-   if ((strchr(p->name, '[') && strchr(c->name, '[') == NULL) ||
-   (strchr(p->name, '[') == NULL && strchr(c->name, '[')))
-   /* Limit range of last symbol in module and kernel */
-   p->end += SYMBOL_LIMIT;
-   else
-   p->end = c->start;
-   pr_debug4("%s sym:%s end:%#" PRIx64 "\n", __func__, p->name, p->end);
-}
-
 void arch__add_leaf_frame_record_opts(struct record_opts *opts)
 {
opts->sample_user_regs |= sample_reg_masks[PERF_REG_ARM64_LR].mask;
diff --git a/tools/perf/arch/powerpc/util/Build 
b/tools/perf/arch/powerpc/util/Build
index 8a79c4126e5b..0115f3166568 100644
--- a/tools/perf/arch/powerpc/util/Build
+++ b/tools/perf/arch/powerpc/util/Build
@@ -1,5 +1,4 @@
 perf-y += header.o
-perf-y += machine.o
 perf-y += kvm-stat.o
 perf-y += perf_regs.o
 perf-y += mem-events.o
diff --git a/tools/perf/arch/powerpc/util/machine.c 
b/tools/perf/arch/powerpc/util/machine.c
deleted file mode 100644
index e652a1aa8132..
--- a/tools/perf/arch/powerpc/util/machine.c
+++ /dev/null
@@ -1,25 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#include 
-#include 
-#include 
-#include  // page_size
-#include "debug.h"
-#include "symbol.h"
-
-/* On powerpc kernel text segment start at memory addresses, 0xc000
- * whereas the modules are located at very high memory addresses,
- * for example 0xc0080xxx. The gap between end of kernel text segment
- * and beginning of first module's text segment is very high.
- * Therefore do not fill this gap and do not assign it to the kernel dso map.
- */
-
-void arch__symbols__fixup_end(struct symbol *p, struct symbol *c)
-{
-   if (strchr(p->name, '[') == NULL && strchr(c->name, '['))
-   /* Limit the range of last kernel symbol */
-   p->end += page_size;
-   else
-   p->end = c->start;
-   pr_debug4("%s sym:%s end:%#" PRIx64 "\n", __func__, p->name, p->end);
-}
diff --git a/tools/perf/arch/s390/util/machine.c 
b/tools/perf/arch/s390/util/machine.c
index 7644a4f6d4a4..98bc3f39d5f3 100644
--- a/tools/perf/arch/s390/util/machine.c
+++ b/tools/perf/arch/s390/util/machine.c
@@ -35,19 +35,3 @@ int arch__fix_module_text_start(u64 *start, u64 *size, const 
char *name)
 
return 0;
 }
-
-/* On s390 kernel text segment start is located at very low memory addresses,
- * for example 0x1. Modules are located at very high memory addresses,
- * for example 0x3ff  . The gap between end of kernel text segment
- * and beginning of first module's text segment is very big.
- * Therefore do not fill this gap and do not assign it to the kernel dso map.
- */
-void arch__symbols__fixup_end(struct symbol *p, struct symbol *c)
-{
-   if (strchr(p->name, '[') == NULL && strchr(c->name, '['))
-   /* Last kernel symbol mapped to end of page */
-   p->end = roundup(p->end, page_size);
-   else
-   p->end = c->start;
-   pr_debug4("%s sym:%s end:%#" PRIx64 "\n", __func__, p->name, p->end);
-}
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 623094e866fd..f72baf636724 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -101,11 +101,6 @@ static int prefix_underscores_count(const char *str)
return tail - str;
 }
 
-void __weak arch__symbols__fixup_end(struct symbol *p, struct symbol 

[PATCH 2/3] perf symbol: Update symbols__fixup_end()

[Namhyung Kim]

Now arch-specific functions all do the same thing.  When it fixes the
symbol address it needs to check the boundary between the kernel image
and modules.  For the last symbol in the previous region, it cannot
know the exact size as it's discarded already.  Thus it just uses a
small page size (4096) and rounds it up like the last symbol.

Signed-off-by: Namhyung Kim 
---
 tools/perf/util/symbol.c | 29 +
 1 file changed, 25 insertions(+), 4 deletions(-)

diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 1b85cc1422a9..623094e866fd 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -217,8 +217,8 @@ void symbols__fixup_duplicate(struct rb_root_cached 
*symbols)
}
 }
 
-void symbols__fixup_end(struct rb_root_cached *symbols,
-   bool is_kallsyms __maybe_unused)
+/* Update zero-sized symbols using the address of the next symbol */
+void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms)
 {
struct rb_node *nd, *prevnd = rb_first_cached(symbols);
struct symbol *curr, *prev;
@@ -232,8 +232,29 @@ void symbols__fixup_end(struct rb_root_cached *symbols,
prev = curr;
curr = rb_entry(nd, struct symbol, rb_node);
 
-   if (prev->end == prev->start || prev->end != curr->start)
-   arch__symbols__fixup_end(prev, curr);
+   /*
+* On some architecture kernel text segment start is located at
+* some low memory address, while modules are located at high
+* memory addresses (or vice versa).  The gap between end of
+* kernel text segment and beginning of first module's text
+* segment is very big.  Therefore do not fill this gap and do
+* not assign it to the kernel dso map (kallsyms).
+*
+* In kallsyms, it determines module symbols using '[' character
+* like in:
+*   c1937000 T hdmi_driver_init  [snd_hda_codec_hdmi]
+*/
+   if (prev->end == prev->start) {
+   /* Last kernel/module symbol mapped to end of page */
+   if (is_kallsyms && (!strchr(prev->name, '[') !=
+   !strchr(curr->name, '[')))
+   prev->end = roundup(prev->end + 4096, 4096);
+   else
+   prev->end = curr->start;
+
+   pr_debug4("%s sym:%s end:%#" PRIx64 "\n",
+ __func__, prev->name, prev->end);
+   }
}
 
/* Last entry */
-- 
2.36.0.rc0.470.gd361397f0d-goog

[PATCH 1/3] perf symbol: Pass is_kallsyms to symbols__fixup_end()

[Namhyung Kim]

The symbol fixup is necessary for symbols in kallsyms since they don't
have size info.  So we use the next symbol's address to calculate the
size.  Now it's also used for user binaries because sometimes they
miss size for hand-written asm functions.

There's a arch-specific function to handle kallsyms differently but
currently it cannot distinguish kallsyms from others.  Pass this
information explicitly to handle it properly.  Note that those arch
functions will be moved to the generic function so I didn't added it
to the arch-functions.

Signed-off-by: Namhyung Kim 
---
 tools/perf/util/symbol-elf.c | 2 +-
 tools/perf/util/symbol.c | 7 ---
 tools/perf/util/symbol.h | 2 +-
 3 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index 31cd59a2b66e..ecd377938eea 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -1290,7 +1290,7 @@ dso__load_sym_internal(struct dso *dso, struct map *map, 
struct symsrc *syms_ss,
 * For misannotated, zeroed, ASM function sizes.
 */
if (nr > 0) {
-   symbols__fixup_end(>symbols);
+   symbols__fixup_end(>symbols, false);
symbols__fixup_duplicate(>symbols);
if (kmap) {
/*
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index dea0fc495185..1b85cc1422a9 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -217,7 +217,8 @@ void symbols__fixup_duplicate(struct rb_root_cached 
*symbols)
}
 }
 
-void symbols__fixup_end(struct rb_root_cached *symbols)
+void symbols__fixup_end(struct rb_root_cached *symbols,
+   bool is_kallsyms __maybe_unused)
 {
struct rb_node *nd, *prevnd = rb_first_cached(symbols);
struct symbol *curr, *prev;
@@ -1467,7 +1468,7 @@ int __dso__load_kallsyms(struct dso *dso, const char 
*filename,
if (kallsyms__delta(kmap, filename, ))
return -1;
 
-   symbols__fixup_end(>symbols);
+   symbols__fixup_end(>symbols, true);
symbols__fixup_duplicate(>symbols);
 
if (dso->kernel == DSO_SPACE__KERNEL_GUEST)
@@ -1659,7 +1660,7 @@ int dso__load_bfd_symbols(struct dso *dso, const char 
*debugfile)
 #undef bfd_asymbol_section
 #endif
 
-   symbols__fixup_end(>symbols);
+   symbols__fixup_end(>symbols, false);
symbols__fixup_duplicate(>symbols);
dso->adjust_symbols = 1;
 
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index fbf866d82dcc..5fcdd1f94c56 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -203,7 +203,7 @@ void __symbols__insert(struct rb_root_cached *symbols, 
struct symbol *sym,
   bool kernel);
 void symbols__insert(struct rb_root_cached *symbols, struct symbol *sym);
 void symbols__fixup_duplicate(struct rb_root_cached *symbols);
-void symbols__fixup_end(struct rb_root_cached *symbols);
+void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms);
 void maps__fixup_end(struct maps *maps);
 
 typedef int (*mapfn_t)(u64 start, u64 len, u64 pgoff, void *data);
-- 
2.36.0.rc0.470.gd361397f0d-goog

[PATCH 0/3] perf tools: Tidy up symbol end fixup (v3)

[Namhyung Kim]

Hello,

This work is a follow-up of Ian's previous one at
  https://lore.kernel.org/all/20220412154817.2728324-1-irog...@google.com/

Fixing up more symbol ends as introduced in:
  https://lore.kernel.org/lkml/20220317135536.805-1-mpet...@redhat.com/

it caused perf annotate to run into memory limits - every symbol holds
all the disassembled code in the annotation, and so making symbols
ends further away dramatically increased memory usage (40MB to >1GB).

Modify the symbol end fixup logic so that special kernel cases aren't
applied in the common case.

v3 changes)
 * rename is_kernel to is_kallsyms
 * move the logic to generic function
 * remove arch-specific functions

Thanks,
Namhyung


Namhyung Kim (3):
  perf symbol: Pass is_kallsyms to symbols__fixup_end()
  perf symbol: Update symbols__fixup_end()
  perf symbol: Remove arch__symbols__fixup_end()

 tools/perf/arch/arm64/util/machine.c   | 21 ---
 tools/perf/arch/powerpc/util/Build |  1 -
 tools/perf/arch/powerpc/util/machine.c | 25 -
 tools/perf/arch/s390/util/machine.c| 16 ---
 tools/perf/util/symbol-elf.c   |  2 +-
 tools/perf/util/symbol.c   | 37 +++---
 tools/perf/util/symbol.h   |  3 +--
 7 files changed, 29 insertions(+), 76 deletions(-)
 delete mode 100644 tools/perf/arch/powerpc/util/machine.c


base-commit: 41204da4c16071be9090940b18f566832d46becc
-- 
2.36.0.rc0.470.gd361397f0d-goog

Re: [PATCH] perf tools: Resolve symbols against debug file first

[Namhyung Kim]

Hello,

On Thu, Jan 14, 2021 at 8:17 PM Michael Ellerman  wrote:
>
> Namhyung Kim  writes:
> > On Wed, Jan 13, 2021 at 8:43 PM Jiri Slaby  wrote:
> >>
> >> On 13. 01. 21, 11:46, Jiri Olsa wrote:
> >> > On Wed, Jan 13, 2021 at 09:01:28AM +0100, Jiri Slaby wrote:
> >> >> With LTO, there are symbols like these:
> >> >> /usr/lib/debug/usr/lib64/libantlr4-runtime.so.4.8-4.8-1.4.x86_64.debug
> >> >>   10305: 00955fa4 0 NOTYPE  LOCAL  DEFAULT   29 
> >> >> Predicate.cpp.2bc410e7
> >> >>
> >> >> This comes from a runtime/debug split done by the standard way:
> >> >> objcopy --only-keep-debug $runtime $debug
> >> >> objcopy --add-gnu-debuglink=$debugfn -R .comment -R .GCC.command.line 
> >> >> --strip-all $runtime
> >> >>
> ...
> >> >> diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
> >> >> index f3577f7d72fe..a31b716fa61c 100644
> >> >> --- a/tools/perf/util/symbol-elf.c
> >> >> +++ b/tools/perf/util/symbol-elf.c
> >> >> @@ -1226,12 +1226,20 @@ int dso__load_sym(struct dso *dso, struct map 
> >> >> *map, struct symsrc *syms_ss,
> >> >>  if (sym.st_shndx == SHN_ABS)
> >> >>  continue;
> >> >>
> >> >> -sec = elf_getscn(runtime_ss->elf, sym.st_shndx);
> >> >> +sec = elf_getscn(syms_ss->elf, sym.st_shndx);
> >> >>  if (!sec)
> >> >>  goto out_elf_end;
> >> >
> >> > we iterate symbols from syms_ss, so the fix seems to be correct
> >> > to call elf_getscn on syms_ss, not on runtime_ss as we do now
> >> >
> >> > I'd think this worked only when runtime_ss == syms_ss
> >>
> >> No, because the headers are copied 1:1 from runtime_ss to syms_ss. And
> >> runtime_ss is then stripped, so only .debug* sections are removed there.
> >> (And syms_ss's are set as NOBITS.)
> >>
> >> We iterated .debug* sections in syms_ss and used runtime_ss section
> >> _headers_ only to adjust symbols (sometimes). That worked.
> >
> > It seems PPC has an opd section only in the runtime_ss and that's why
> > we use it for section headers.
>
> At least on my system (Ubuntu 20.04.1) I see .opd in the debug file with
> NOBITS set:
>
> $ readelf -e vmlinux.debug | grep opd
>   [37] .opd  NOBITS   c1c1f548  01202e14
>
>
> But possibly that's not the case with older toolchains?

I was referring to this commit:

commit 261360b6e90a782f0a63d8f61a67683c376c88cf
Author: Cody P Schafer 
Date:   Fri Aug 10 15:23:01 2012 -0700

perf symbols: Convert dso__load_syms to take 2 symsrc's

To properly handle platforms with an opd section, both a runtime image
(which contains the opd section but possibly lacks symbols) and a symbol
image (which probably lacks an opd section but has symbols).

The next patch ("perf symbol: use both runtime and debug images")
adjusts the callsite in dso__load() to take advantage of being able to
pass both runtime & debug images.

Assumptions made here:

 - The opd section, if it exists in the runtime image, has headers in
   both the runtime image and the debug/syms image.

 - The index of the opd section (again, only if it exists in the runtime
   image) is the same in both the runtime and debug/symbols image.

Both of these are true on RHEL, but it is unclear how accurate they are
in general (on platforms with function descriptors in opd sections).

Signed-off-by: Cody P Schafer 


Thanks,
Namhyung

Re: [PATCH] perf tools: Resolve symbols against debug file first

[Namhyung Kim]

Hi both of Jiri,

On Wed, Jan 13, 2021 at 8:43 PM Jiri Slaby  wrote:
>
> On 13. 01. 21, 11:46, Jiri Olsa wrote:
> > On Wed, Jan 13, 2021 at 09:01:28AM +0100, Jiri Slaby wrote:
> >> With LTO, there are symbols like these:
> >> /usr/lib/debug/usr/lib64/libantlr4-runtime.so.4.8-4.8-1.4.x86_64.debug
> >>   10305: 00955fa4 0 NOTYPE  LOCAL  DEFAULT   29 
> >> Predicate.cpp.2bc410e7
> >>
> >> This comes from a runtime/debug split done by the standard way:
> >> objcopy --only-keep-debug $runtime $debug
> >> objcopy --add-gnu-debuglink=$debugfn -R .comment -R .GCC.command.line 
> >> --strip-all $runtime
> >>
> >> perf currently cannot resolve such symbols (relicts of LTO), as section
> >> 29 exists only in the debug file (29 is .debug_info). And perf resolves
> >> symbols only against runtime file. This results in all symbols from such
> >> a library being unresolved:
> >>   0.38%  main2libantlr4-runtime.so.4.8  [.] 0x000671e0
> >>
> >> So try resolving against the debug file first. And only if it fails (the
> >> section has NOBITS set), try runtime file. We can do this, as "objcopy
> >> --only-keep-debug" per documentation preserves all sections, but clears
> >> data of some of them (the runtime ones) and marks them as NOBITS.
> >>
> >> The correct result is now:
> >>   0.38%  main2libantlr4-runtime.so.4.8  [.] 
> >> antlr4::IntStream::~IntStream
> >>
> >> Note that these LTO symbols are properly skipped anyway as they belong
> >> neither to *text* nor to *data* (is_label && !elf_sec__filter(,
> >> secstrs) is true).
> >>
> >> Signed-off-by: Jiri Slaby 
> >> Cc: Peter Zijlstra 
> >> Cc: Ingo Molnar 
> >> Cc: Arnaldo Carvalho de Melo 
> >> Cc: Mark Rutland 
> >> Cc: Alexander Shishkin 
> >> Cc: Jiri Olsa 
> >> Cc: Namhyung Kim 
> >> ---
> >>   tools/perf/util/symbol-elf.c | 10 +-
> >>   1 file changed, 9 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
> >> index f3577f7d72fe..a31b716fa61c 100644
> >> --- a/tools/perf/util/symbol-elf.c
> >> +++ b/tools/perf/util/symbol-elf.c
> >> @@ -1226,12 +1226,20 @@ int dso__load_sym(struct dso *dso, struct map 
> >> *map, struct symsrc *syms_ss,
> >>  if (sym.st_shndx == SHN_ABS)
> >>  continue;
> >>
> >> -sec = elf_getscn(runtime_ss->elf, sym.st_shndx);
> >> +sec = elf_getscn(syms_ss->elf, sym.st_shndx);
> >>  if (!sec)
> >>  goto out_elf_end;
> >
> > we iterate symbols from syms_ss, so the fix seems to be correct
> > to call elf_getscn on syms_ss, not on runtime_ss as we do now
> >
> > I'd think this worked only when runtime_ss == syms_ss
>
> No, because the headers are copied 1:1 from runtime_ss to syms_ss. And
> runtime_ss is then stripped, so only .debug* sections are removed there.
> (And syms_ss's are set as NOBITS.)
>
> We iterated .debug* sections in syms_ss and used runtime_ss section
> _headers_ only to adjust symbols (sometimes). That worked.

It seems PPC has an opd section only in the runtime_ss and that's why
we use it for section headers.

>
> >>  gelf_getshdr(sec, );
> >>
> >> +if (shdr.sh_type == SHT_NOBITS) {
> >> +sec = elf_getscn(runtime_ss->elf, sym.st_shndx);
> >> +if (!sec)
> >> +goto out_elf_end;
> >> +
> >> +gelf_getshdr(sec, );
> >> +}
> >
> > is that fallback necessary? the symbol is from syms_ss
>
> Provided the above, we don't need the section data here, only headers,
> so the NOBITS test is superfluous and the fallback shouldn't be needed.
> Let me test it.

We need to talk to PPC folks like I said.  Or maybe we can change the
default ss depending on the arch.

Thanks,
Namhyung

Re: [PATCH 1/3] perf/core: Flush PMU internal buffers for per-CPU events

[Namhyung Kim]

Hello,

On Mon, Nov 23, 2020 at 8:00 PM Michael Ellerman  wrote:
>
> Namhyung Kim  writes:
> > Hi Peter and Kan,
> >
> > (Adding PPC folks)
> >
> > On Tue, Nov 17, 2020 at 2:01 PM Namhyung Kim  wrote:
> >>
> >> Hello,
> >>
> >> On Thu, Nov 12, 2020 at 4:54 AM Liang, Kan  
> >> wrote:
> >> >
> >> >
> >> >
> >> > On 11/11/2020 11:25 AM, Peter Zijlstra wrote:
> >> > > On Mon, Nov 09, 2020 at 09:49:31AM -0500, Liang, Kan wrote:
> >> > >
> >> > >> - When the large PEBS was introduced (9c964efa4330), the sched_task() 
> >> > >> should
> >> > >> be invoked to flush the PEBS buffer in each context switch. However, 
> >> > >> The
> >> > >> perf_sched_events in account_event() is not updated accordingly. The
> >> > >> perf_event_task_sched_* never be invoked for a pure per-CPU context. 
> >> > >> Only
> >> > >> per-task event works.
> >> > >> At that time, the perf_pmu_sched_task() is outside of
> >> > >> perf_event_context_sched_in/out. It means that perf has to double
> >> > >> perf_pmu_disable() for per-task event.
> >> > >
> >> > >> - The patch 1 tries to fix broken per-CPU events. The CPU context 
> >> > >> cannot be
> >> > >> retrieved from the task->perf_event_ctxp. So it has to be tracked in 
> >> > >> the
> >> > >> sched_cb_list. Yes, the code is very similar to the original codes, 
> >> > >> but it
> >> > >> is actually the new code for per-CPU events. The optimization for 
> >> > >> per-task
> >> > >> events is still kept.
> >> > >>For the case, which has both a CPU context and a task context, 
> >> > >> yes, the
> >> > >> __perf_pmu_sched_task() in this patch is not invoked. Because the
> >> > >> sched_task() only need to be invoked once in a context switch. The
> >> > >> sched_task() will be eventually invoked in the task context.
> >> > >
> >> > > The thing is; your first two patches rely on PERF_ATTACH_SCHED_CB and
> >> > > only set that for large pebs. Are you sure the other users (Intel LBR
> >> > > and PowerPC BHRB) don't need it?
> >> >
> >> > I didn't set it for LBR, because the perf_sched_events is always enabled
> >> > for LBR. But, yes, we should explicitly set the PERF_ATTACH_SCHED_CB
> >> > for LBR.
> >> >
> >> > if (has_branch_stack(event))
> >> > inc = true;
> >> >
> >> > >
> >> > > If they indeed do not require the pmu::sched_task() callback for CPU
> >> > > events, then I still think the whole perf_sched_cb_{inc,dec}() 
> >> > > interface
> >> >
> >> > No, LBR requires the pmu::sched_task() callback for CPU events.
> >> >
> >> > Now, The LBR registers have to be reset in sched in even for CPU events.
> >> >
> >> > To fix the shorter LBR callstack issue for CPU events, we also need to
> >> > save/restore LBRs in pmu::sched_task().
> >> > https://lore.kernel.org/lkml/1578495789-95006-4-git-send-email-kan.li...@linux.intel.com/
> >> >
> >> > > is confusing at best.
> >> > >
> >> > > Can't we do something like this instead?
> >> > >
> >> > I think the below patch may have two issues.
> >> > - PERF_ATTACH_SCHED_CB is required for LBR (maybe PowerPC BHRB as well) 
> >> > now.
> >> > - We may disable the large PEBS later if not all PEBS events support
> >> > large PEBS. The PMU need a way to notify the generic code to decrease
> >> > the nr_sched_task.
> >>
> >> Any updates on this?  I've reviewed and tested Kan's patches
> >> and they all look good.
> >>
> >> Maybe we can talk to PPC folks to confirm the BHRB case?
> >
> > Can we move this forward?  I saw patch 3/3 also adds PERF_ATTACH_SCHED_CB
> > for PowerPC too.  But it'd be nice if ppc folks can confirm the change.
>
> Sorry I've read the whole thread, but I'm still not entirely sure I
> understand the question.

Thanks for your time and sorry about not being clear enough.

We found per-cpu events are not calling pmu::sched_task()
on context switches.  So PERF_ATTACH_SCHED_CB was
added to indicate the core logic that it needs to invoke the
callback.

The patch 3/3 added the flag to PPC (for BHRB) with other
changes (I think it should be split like in the patch 2/3) and
want to get ACKs from the PPC folks.

Thanks,
Namhyung

Re: [PATCH 1/3] perf/core: Flush PMU internal buffers for per-CPU events

[Namhyung Kim]

Hi Peter and Kan,

(Adding PPC folks)

On Tue, Nov 17, 2020 at 2:01 PM Namhyung Kim  wrote:
>
> Hello,
>
> On Thu, Nov 12, 2020 at 4:54 AM Liang, Kan  wrote:
> >
> >
> >
> > On 11/11/2020 11:25 AM, Peter Zijlstra wrote:
> > > On Mon, Nov 09, 2020 at 09:49:31AM -0500, Liang, Kan wrote:
> > >
> > >> - When the large PEBS was introduced (9c964efa4330), the sched_task() 
> > >> should
> > >> be invoked to flush the PEBS buffer in each context switch. However, The
> > >> perf_sched_events in account_event() is not updated accordingly. The
> > >> perf_event_task_sched_* never be invoked for a pure per-CPU context. Only
> > >> per-task event works.
> > >> At that time, the perf_pmu_sched_task() is outside of
> > >> perf_event_context_sched_in/out. It means that perf has to double
> > >> perf_pmu_disable() for per-task event.
> > >
> > >> - The patch 1 tries to fix broken per-CPU events. The CPU context cannot 
> > >> be
> > >> retrieved from the task->perf_event_ctxp. So it has to be tracked in the
> > >> sched_cb_list. Yes, the code is very similar to the original codes, but 
> > >> it
> > >> is actually the new code for per-CPU events. The optimization for 
> > >> per-task
> > >> events is still kept.
> > >>For the case, which has both a CPU context and a task context, yes, 
> > >> the
> > >> __perf_pmu_sched_task() in this patch is not invoked. Because the
> > >> sched_task() only need to be invoked once in a context switch. The
> > >> sched_task() will be eventually invoked in the task context.
> > >
> > > The thing is; your first two patches rely on PERF_ATTACH_SCHED_CB and
> > > only set that for large pebs. Are you sure the other users (Intel LBR
> > > and PowerPC BHRB) don't need it?
> >
> > I didn't set it for LBR, because the perf_sched_events is always enabled
> > for LBR. But, yes, we should explicitly set the PERF_ATTACH_SCHED_CB
> > for LBR.
> >
> > if (has_branch_stack(event))
> > inc = true;
> >
> > >
> > > If they indeed do not require the pmu::sched_task() callback for CPU
> > > events, then I still think the whole perf_sched_cb_{inc,dec}() interface
> >
> > No, LBR requires the pmu::sched_task() callback for CPU events.
> >
> > Now, The LBR registers have to be reset in sched in even for CPU events.
> >
> > To fix the shorter LBR callstack issue for CPU events, we also need to
> > save/restore LBRs in pmu::sched_task().
> > https://lore.kernel.org/lkml/1578495789-95006-4-git-send-email-kan.li...@linux.intel.com/
> >
> > > is confusing at best.
> > >
> > > Can't we do something like this instead?
> > >
> > I think the below patch may have two issues.
> > - PERF_ATTACH_SCHED_CB is required for LBR (maybe PowerPC BHRB as well) now.
> > - We may disable the large PEBS later if not all PEBS events support
> > large PEBS. The PMU need a way to notify the generic code to decrease
> > the nr_sched_task.
>
> Any updates on this?  I've reviewed and tested Kan's patches
> and they all look good.
>
> Maybe we can talk to PPC folks to confirm the BHRB case?

Can we move this forward?  I saw patch 3/3 also adds PERF_ATTACH_SCHED_CB
for PowerPC too.  But it'd be nice if ppc folks can confirm the change.

Thanks,
Namhyung

Re: [PATCH 06/18] pstore: Extract common arguments into structure

[Namhyung Kim]

On Tue, Mar 7, 2017 at 6:55 AM, Kees Cook  wrote:
> The read/mkfile pair pass the same arguments and should be cleared
> between calls. Move to a structure and wipe it after every loop.
>
> Signed-off-by: Kees Cook 
> ---
>  fs/pstore/platform.c   | 55 
> +++---
>  include/linux/pstore.h | 28 -
>  2 files changed, 57 insertions(+), 26 deletions(-)
>
> diff --git a/fs/pstore/platform.c b/fs/pstore/platform.c
> index 320a673ecb5b..3fa1575a6e36 100644
> --- a/fs/pstore/platform.c
> +++ b/fs/pstore/platform.c
> @@ -766,16 +766,9 @@ EXPORT_SYMBOL_GPL(pstore_unregister);
>  void pstore_get_records(int quiet)
>  {
> struct pstore_info *psi = psinfo;
> -   char*buf = NULL;
> -   ssize_t size;
> -   u64 id;
> -   int count;
> -   enum pstore_type_id type;
> -   struct timespec time;
> +   struct pstore_recordrecord = { .psi = psi, };
> int failed = 0, rc;
> -   boolcompressed;
> int unzipped_len = -1;
> -   ssize_t ecc_notice_size = 0;
>
> if (!psi)
> return;
> @@ -784,39 +777,51 @@ void pstore_get_records(int quiet)
> if (psi->open && psi->open(psi))
> goto out;
>
> -   while ((size = psi->read(, , , , , ,
> -_notice_size, psi)) > 0) {
> -   if (compressed && (type == PSTORE_TYPE_DMESG)) {
> +   while ((record.size = psi->read(, ,
> +, ,
> +, ,
> +_notice_size,
> +record.psi)) > 0) {
> +   if (record.compressed &&
> +   record.type == PSTORE_TYPE_DMESG) {
> if (big_oops_buf)
> -   unzipped_len = pstore_decompress(buf,
> -   big_oops_buf, size,
> +   unzipped_len = pstore_decompress(
> +   record.buf,
> +   big_oops_buf,
> +   record.size,
> big_oops_buf_sz);
>
> if (unzipped_len > 0) {
> -   if (ecc_notice_size)
> +   if (record.ecc_notice_size)
> memcpy(big_oops_buf + unzipped_len,
> -  buf + size, ecc_notice_size);
> -   kfree(buf);
> -   buf = big_oops_buf;
> -   size = unzipped_len;
> -   compressed = false;
> +  record.buf + recorrecord.size,

A typo on record.size.

Thanks,
Namhyung

Re: [PATCH 03/18] pstore: Avoid race in module unloading

[Namhyung Kim]

Hi Kees,

On Tue, Mar 7, 2017 at 6:55 AM, Kees Cook  wrote:
> Technically, it might be possible for struct pstore_info to go out of
> scope after the module_put(), so report the backend name first.

But in that case, using pstore will crash the kernel anyway, right?
If so, why pstore doesn't keep a reference until unregister?
Do I miss something?

Thanks,
Namhyung


>
> Signed-off-by: Kees Cook 
> ---
>  fs/pstore/platform.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/fs/pstore/platform.c b/fs/pstore/platform.c
> index 074fe85a2078..d69ef8a840b9 100644
> --- a/fs/pstore/platform.c
> +++ b/fs/pstore/platform.c
> @@ -722,10 +722,10 @@ int pstore_register(struct pstore_info *psi)
>  */
> backend = psi->name;
>
> -   module_put(owner);
> -
> pr_info("Registered %s as persistent store backend\n", psi->name);
>
> +   module_put(owner);
> +
> return 0;
>  }
>  EXPORT_SYMBOL_GPL(pstore_register);
> --
> 2.7.4
>

Re: [PATCH v6 2/7] perf annotate: Add cross arch annotate support

[Namhyung Kim]

Hi,

On Tue, Aug 23, 2016 at 03:36:17PM -0500, Kim Phillips wrote:
> On Tue, 23 Aug 2016 11:17:16 +0900
> Namhyung Kim <namhy...@kernel.org> wrote:
> 
> > On Tue, Aug 23, 2016 at 8:01 AM, Kim Phillips <kim.phill...@arm.com> wrote:
> > > On Fri, 19 Aug 2016 18:29:33 +0530
> > > Ravi Bangoria <ravi.bango...@linux.vnet.ibm.com> wrote:
> > >
> > >> Changes in v6:
> > >>   - Instead of adding only those instructions defined in #ifdef __arm__,
> > >> add all instructions from default table to arm table.
> > > Thanks, I've gone through the list and removed all not-ARM
> > > instructions, and added some missing ARM branch instructions:
> > 
> > Can we use regex patterns instead?
> 
> Yes, that helps prevent mistakes updating instruction lists - how does
> this look?:

Much better!

Thanks,
Namhyung


> 
> diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c
> index b2c6cf3..52316f3 100644
> --- a/tools/perf/util/annotate.c
> +++ b/tools/perf/util/annotate.c
> @@ -26,6 +26,7 @@
>  const char   *disassembler_style;
>  const char   *objdump_path;
>  static regex_tfile_lineno;
> +static regex_tarm_call_insn, arm_jump_insn;
>  
>  static struct ins *ins__find(const char *name, const char *norm_arch);
>  static int disasm_line__parse(char *line, char **namep, char **rawp);
> @@ -449,98 +450,7 @@ static struct ins instructions_x86[] = {
>   { .name = "retq",  .ops  = _ops, },
>  };
>  
> -static struct ins instructions_arm[] = {
> - { .name = "add",   .ops  = _ops, },
> - { .name = "addl",  .ops  = _ops, },
> - { .name = "addq",  .ops  = _ops, },
> - { .name = "addw",  .ops  = _ops, },
> - { .name = "and",   .ops  = _ops, },
> - { .name = "b", .ops  = _ops, }, /* might also be a call */
> - { .name = "bcc",   .ops  = _ops, },
> - { .name = "bcs",   .ops  = _ops, },
> - { .name = "beq",   .ops  = _ops, },
> - { .name = "bge",   .ops  = _ops, },
> - { .name = "bgt",   .ops  = _ops, },
> - { .name = "bhi",   .ops  = _ops, },
> - { .name = "bl",.ops  = _ops, },
> - { .name = "bls",   .ops  = _ops, },
> - { .name = "blt",   .ops  = _ops, },
> - { .name = "blx",   .ops  = _ops, },
> - { .name = "bne",   .ops  = _ops, },
> - { .name = "bts",   .ops  = _ops, },
> - { .name = "call",  .ops  = _ops, },
> - { .name = "callq", .ops  = _ops, },
> - { .name = "cmp",   .ops  = _ops, },
> - { .name = "cmpb",  .ops  = _ops, },
> - { .name = "cmpl",  .ops  = _ops, },
> - { .name = "cmpq",  .ops  = _ops, },
> - { .name = "cmpw",  .ops  = _ops, },
> - { .name = "cmpxch", .ops  = _ops, },
> - { .name = "dec",   .ops  = _ops, },
> - { .name = "decl",  .ops  = _ops, },
> - { .name = "imul",  .ops  = _ops, },
> - { .name = "inc",   .ops  = _ops, },
> - { .name = "incl",  .ops  = _ops, },
> - { .name = "ja",.ops  = _ops, },
> - { .name = "jae",   .ops  = _ops, },
> - { .name = "jb",.ops  = _ops, },
> - { .name = "jbe",   .ops  = _ops, },
> - { .name = "jc",.ops  = _ops, },
> - { .name = "jcxz",  .ops  = _ops, },
> - { .name = "je",.ops  = _ops, },
> - { .name = "jecxz", .ops  = _ops, },
> - { .name = "jg",.ops  = _ops, },
> - { .name = "jge",   .ops  = _ops, },
> - { .name = "jl",.ops  = _ops, },
> - { .name = "jle",   .ops  = _ops, },
> - { .name = "jmp",   .ops  = _ops, },
> - { .name = "jmpq",  .ops  = _ops, },
> - { .name = "jna",   .ops  = _ops, },
> - { .name = "jnae",  .ops  = _ops, },
> - { .name = "jnb",   .ops  = _ops, },
> - { .name = "jnbe",  .ops  = _ops, },
> - { .name = "jnc",   .ops  = _ops, },
> - { .name = "jne",   .ops  = _ops, },
> - { .name = "jng",   .ops  = _ops, },
> - { .name = "jnge",  .ops  = _ops, },
> - { .name = "jnl",   .ops  = _ops, },
> - { .name = "jnle",  .ops  = _ops, },
> - { .name = "jno",   .ops  = _ops, },
> - { .name = "jnp",   .ops  = _ops, },
> - { .name = "

Re: [PATCH v6 2/7] perf annotate: Add cross arch annotate support

[Namhyung Kim]

Hello,

On Tue, Aug 23, 2016 at 8:01 AM, Kim Phillips  wrote:
> On Fri, 19 Aug 2016 18:29:33 +0530
> Ravi Bangoria  wrote:
>
>> Changes in v6:
>>   - Instead of adding only those instructions defined in #ifdef __arm__,
>> add all instructions from default table to arm table.
> ..
>> +static struct ins instructions_arm[] = {
>>   { .name = "add",   .ops  = _ops, },
>>   { .name = "addl",  .ops  = _ops, },
>>   { .name = "addq",  .ops  = _ops, },
>>   { .name = "addw",  .ops  = _ops, },
>>   { .name = "and",   .ops  = _ops, },
>> -#ifdef __arm__
>> - { .name = "b", .ops  = _ops, }, // might also be a call
>> + { .name = "b", .ops  = _ops, }, /* might also be a call */
>>   { .name = "bcc",   .ops  = _ops, },
>>   { .name = "bcs",   .ops  = _ops, },
>>   { .name = "beq",   .ops  = _ops, },
>> @@ -382,7 +462,6 @@ static struct ins instructions[] = {
>>   { .name = "blt",   .ops  = _ops, },
>>   { .name = "blx",   .ops  = _ops, },
>>   { .name = "bne",   .ops  = _ops, },
>> -#endif
>>   { .name = "bts",   .ops  = _ops, },
>>   { .name = "call",  .ops  = _ops, },
>>   { .name = "callq", .ops  = _ops, },
>> @@ -471,24 +550,48 @@ static int ins__cmp(const void *a, const void *b)
>>   return strcmp(ia->name, ib->name);
>>  }
>
> Thanks, I've gone through the list and removed all not-ARM
> instructions, and added some missing ARM branch instructions:

Can we use regex patterns instead?

Thanks
Namhyung


>
> diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c
> index b2c6cf3..9d686504 100644
> --- a/tools/perf/util/annotate.c
> +++ b/tools/perf/util/annotate.c
> @@ -451,9 +451,6 @@ static struct ins instructions_x86[] = {
>
>  static struct ins instructions_arm[] = {
> { .name = "add",   .ops  = _ops, },
> -   { .name = "addl",  .ops  = _ops, },
> -   { .name = "addq",  .ops  = _ops, },
> -   { .name = "addw",  .ops  = _ops, },
> { .name = "and",   .ops  = _ops, },
> { .name = "b", .ops  = _ops, }, /* might also be a call */
> { .name = "bcc",   .ops  = _ops, },
> @@ -463,81 +460,20 @@ static struct ins instructions_arm[] = {
> { .name = "bgt",   .ops  = _ops, },
> { .name = "bhi",   .ops  = _ops, },
> { .name = "bl",.ops  = _ops, },
> +   { .name = "ble",   .ops  = _ops, },
> +   { .name = "bleq",  .ops  = _ops, },
> +   { .name = "blne",  .ops  = _ops, },
> { .name = "bls",   .ops  = _ops, },
> { .name = "blt",   .ops  = _ops, },
> { .name = "blx",   .ops  = _ops, },
> +   { .name = "blxne", .ops  = _ops, },
> +   { .name = "bmi",   .ops  = _ops, },
> { .name = "bne",   .ops  = _ops, },
> -   { .name = "bts",   .ops  = _ops, },
> -   { .name = "call",  .ops  = _ops, },
> -   { .name = "callq", .ops  = _ops, },
> +   { .name = "bpl",   .ops  = _ops, },
> { .name = "cmp",   .ops  = _ops, },
> -   { .name = "cmpb",  .ops  = _ops, },
> -   { .name = "cmpl",  .ops  = _ops, },
> -   { .name = "cmpq",  .ops  = _ops, },
> -   { .name = "cmpw",  .ops  = _ops, },
> -   { .name = "cmpxch", .ops  = _ops, },
> -   { .name = "dec",   .ops  = _ops, },
> -   { .name = "decl",  .ops  = _ops, },
> -   { .name = "imul",  .ops  = _ops, },
> -   { .name = "inc",   .ops  = _ops, },
> -   { .name = "incl",  .ops  = _ops, },
> -   { .name = "ja",.ops  = _ops, },
> -   { .name = "jae",   .ops  = _ops, },
> -   { .name = "jb",.ops  = _ops, },
> -   { .name = "jbe",   .ops  = _ops, },
> -   { .name = "jc",.ops  = _ops, },
> -   { .name = "jcxz",  .ops  = _ops, },
> -   { .name = "je",.ops  = _ops, },
> -   { .name = "jecxz", .ops  = _ops, },
> -   { .name = "jg",.ops  = _ops, },
> -   { .name = "jge",   .ops  = _ops, },
> -   { .name = "jl",.ops  = _ops, },
> -   { .name = "jle",   .ops  = _ops, },
> -   { .name = "jmp",   .ops  = _ops, },
> -   { .name = "jmpq",  .ops  = _ops, },
> -   { .name = "jna",   .ops  = _ops, },
> -   { .name = "jnae",  .ops  = _ops, },
> -   { .name = "jnb",   .ops  = _ops, },
> -   { .name = "jnbe",  .ops  = _ops, },
> -   { .name = "jnc",   .ops  = _ops, },
> -   { .name = "jne",   .ops  = _ops, },
> -   { .name = "jng",   .ops  = _ops, },
> -   { .name = "jnge",  .ops  = _ops, },
> -   { .name = "jnl",   .ops  = _ops, },
> -   { .name = "jnle",  .ops  = _ops, },
> -   { .name = "jno",   .ops  = _ops, },
> -   { .name = "jnp",   .ops  = _ops, },
> -   { .name = "jns",   .ops  = _ops, },
> -   { .name = "jnz",   .ops  = _ops, },
> -   { .name = "jo",.ops  = _ops, },
> -   { .name = "jp",.ops  = _ops, },
> -   { .name = "jpe",   .ops  = _ops, },
> -   { .name = "jpo",   .ops  = _ops, },
> -   { .name = "jrcxz", 

Re: [PATCH 2/4] perf: jevents: Program to convert JSON file to C style file

[Namhyung Kim]

Hi Sukadev,

On Tue, May 19, 2015 at 05:02:08PM -0700, Sukadev Bhattiprolu wrote:
 From: Andi Kleen a...@linux.intel.com
 
 This is a modified version of an earlier patch by Andi Kleen.
 
 We expect architectures to describe the performance monitoring events
 for each CPU in a corresponding JSON file, which look like:
 
   [
   {
   EventCode: 0x00,
   UMask: 0x01,
   EventName: INST_RETIRED.ANY,
   BriefDescription: Instructions retired from execution.,
   PublicDescription: Instructions retired from execution.,
   Counter: Fixed counter 1,
   CounterHTOff: Fixed counter 1,
   SampleAfterValue: 203,
   SampleAfterValue: 203,
   MSRIndex: 0,
   MSRValue: 0,
   TakenAlone: 0,
   CounterMask: 0,
   Invert: 0,
   AnyThread: 0,
   EdgeDetect: 0,
   PEBS: 0,
   PRECISE_STORE: 0,
   Errata: null,
   Offcore: 0
   }
   ]
 
 We also expect the architectures to provide a mapping between individual
 CPUs to their JSON files. Eg:
 
   GenuineIntel-6-1E,V1,/NHM-EP/NehalemEP_core_V1.json,core
 
 which maps each CPU, identified by [vendor, family, model, version, type]
 to a JSON file.
 
 Given these files, the program, jevents::
   - locates all JSON files for the architecture,
   - parses each JSON file and generates a C-style PMU-events table
 (pmu-events.c)
   - locates a mapfile for the architecture
   - builds a global table, mapping each model of CPU to the
 corresponding PMU-events table.

So we build tables of all models in the architecture, and choose
matching one when compiling perf, right?  Can't we do that when
building the tables?  IOW, why don't we check the VFM and discard
non-matching tables?  Those non-matching tables are also needed?

Sorry if I missed something..


 
 The 'pmu-events.c' is generated when building perf and added to libperf.a.
 The global table pmu_events_map[] table in this pmu-events.c will be used
 in perf in a follow-on patch.
 
 If the architecture does not have any JSON files or there is an error in
 processing them, an empty mapping file is created. This would allow the
 build of perf to proceed even if we are not able to provide aliases for
 events.
 
 The parser for JSON files allows parsing Intel style JSON event files. This
 allows to use an Intel event list directly with perf. The Intel event lists
 can be quite large and are too big to store in unswappable kernel memory.
 
 The conversion from JSON to C-style is straight forward.  The parser knows
 (very little) Intel specific information, and can be easily extended to
 handle fields for other CPUs.
 
 The parser code is partially shared with an independent parsing library,
 which is 2-clause BSD licenced. To avoid any conflicts I marked those
 files as BSD licenced too. As part of perf they become GPLv2.
 
 Signed-off-by: Andi Kleen a...@linux.intel.com
 Signed-off-by: Sukadev Bhattiprolu suka...@linux.vnet.ibm.com
 
 v2: Address review feedback. Rename option to --event-files
 v3: Add JSON example
 v4: Update manpages.
 v5: Don't remove dot in fixname. Fix compile error. Add include
   protection. Comment realloc.
 v6: Include debug/util.h
 v7: (Sukadev Bhattiprolu)
   Rebase to 4.0 and fix some conflicts.
 v8: (Sukadev Bhattiprolu)
   Move jevents.[hc] to tools/perf/pmu-events/
   Rewrite to locate and process arch specific JSON and map files;
   and generate a C file.
   (Removed acked-by Namhyung Kim due to modest changes to patch)
   Compile the generated pmu-events.c and add the pmu-events.o to
   libperf.a
 ---

[SNIP]
 +/* Call func with each event in the json file */
 +int json_events(const char *fn,
 +   int (*func)(void *data, char *name, char *event, char *desc),
 +   void *data)
 +{
 + int err = -EIO;
 + size_t size;
 + jsmntok_t *tokens, *tok;
 + int i, j, len;
 + char *map;
 +
 + if (!fn)
 + return -ENOENT;
 +
 + tokens = parse_json(fn, map, size, len);
 + if (!tokens)
 + return -EIO;
 + EXPECT(tokens-type == JSMN_ARRAY, tokens, expected top level array);
 + tok = tokens + 1;
 + for (i = 0; i  tokens-size; i++) {
 + char *event = NULL, *desc = NULL, *name = NULL;
 + struct msrmap *msr = NULL;
 + jsmntok_t *msrval = NULL;
 + jsmntok_t *precise = NULL;
 + jsmntok_t *obj = tok++;
 +
 + EXPECT(obj-type == JSMN_OBJECT, obj, expected object);
 + for (j = 0; j  obj-size; j += 2) {
 + jsmntok_t *field, *val;
 + int nz;
 +
 + field = tok + j;
 + EXPECT(field-type == JSMN_STRING, tok + j,
 +Expected field name);
 + val = tok + j + 1;
 + EXPECT(val-type == JSMN_STRING, tok + j + 1,
 +Expected string value

Re: [PATCH 4/4] perf: Add power8 PMU events in JSON format

[Namhyung Kim]

On Tue, May 19, 2015 at 05:02:10PM -0700, Sukadev Bhattiprolu wrote:
 The power8.json and 004d0100.json files describe the PMU events in the
 Power8 processor.
 
 The jevents program from the prior patches will use these JSON files
 to create tables which will then be used in perf to build aliases for
 PMU events. This in turn would allow users to specify these PMU events
 by name:
 
   $ perf stat -e pm_1plus_ppc_cmpl sleep 1
 
 Signed-off-by: Sukadev Bhattiprolu suka...@linux.vnet.ibm.com
 ---

[SNIP]
 +  {
 +EventCode: 0x2505e,
 +EventName: PM_BACK_BR_CMPL,
 +BriefDescription: Branch instruction completed with a target address 
 less than current instruction address,,
 +PublicDescription: Branch instruction completed with a target address 
 less than current instruction address.,

Can't we remove PublicDescription field if it's identical to
BriefDescription?  It seems just wasting spaces..

Thanks,
Namhyung


 +  },
 +  {
 +EventCode: 0x4082,
 +EventName: PM_BANK_CONFLICT,
 +BriefDescription: Read blocked due to interleave conflict. The ifar 
 logic will detect an interleave conflict and kill the data that was read that 
 cycle.,,
 +PublicDescription: Read blocked due to interleave conflict. The ifar 
 logic will detect an interleave conflict and kill the data that was read that 
 cycle.,
 +  },
 +  {
 +EventCode: 0x10068,
 +EventName: PM_BRU_FIN,
 +BriefDescription: Branch Instruction Finished,,
 +PublicDescription: Branch Instruction Finished .,
 +  },
 +  {
 +EventCode: 0x20036,
 +EventName: PM_BR_2PATH,
 +BriefDescription: two path branch,,
 +PublicDescription: two path branch.,
 +  },
 +  {
 +EventCode: 0x5086,
 +EventName: PM_BR_BC_8,
 +BriefDescription: Pairable BC+8 branch that has not been converted to 
 a Resolve Finished in the BRU pipeline,,
 +PublicDescription: Pairable BC+8 branch that has not been converted 
 to a Resolve Finished in the BRU pipeline,
 +  },
 +  {
 +EventCode: 0x5084,
 +EventName: PM_BR_BC_8_CONV,
 +BriefDescription: Pairable BC+8 branch that was converted to a 
 Resolve Finished in the BRU pipeline.,,
 +PublicDescription: Pairable BC+8 branch that was converted to a 
 Resolve Finished in the BRU pipeline.,
 +  },
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH 4/4] perf: Add power8 PMU events in JSON format

[Namhyung Kim]

On Wed, May 27, 2015 at 11:41 PM, Andi Kleen a...@linux.intel.com wrote:
  +  {
  +EventCode: 0x2505e,
  +EventName: PM_BACK_BR_CMPL,
  +BriefDescription: Branch instruction completed with a target 
  address less than current instruction address,,
  +PublicDescription: Branch instruction completed with a target 
  address less than current instruction address.,

 Can't we remove PublicDescription field if it's identical to
 BriefDescription?  It seems just wasting spaces..

 It's not always identical. There are events where PublicDescription is much 
 longer (several paragraphs)

I know.  What I said is make it optional so that we can drop if it's identical.

Thanks,
Namhyung
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH 2/4] perf: jevents: Program to convert JSON file to C style file

[Namhyung Kim]

Hi Andi,

On Wed, May 27, 2015 at 11:40 PM, Andi Kleen a...@linux.intel.com wrote:
 So we build tables of all models in the architecture, and choose
 matching one when compiling perf, right?  Can't we do that when
 building the tables?  IOW, why don't we check the VFM and discard
 non-matching tables?  Those non-matching tables are also needed?

 We build it for all cpus in an architecture, not all architectures.
 So e.g. for an x86 binary power is not included, and vice versa.

OK.

 It always includes all CPUs for a given architecture, so it's possible
 to use the perf binary on other systems than just the one it was
 build on.

So it selects one at run-time not build-time, good.  But I worry about
the size of the intel tables.  How large are they?  Maybe we can make
it dynamic-loadable if needed..

Thanks,
Namhyung
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH v5 07/45] CPU hotplug: Provide APIs to prevent CPU offline from atomic context

[Namhyung Kim]

Hi Srivatsa,

On Tue, 22 Jan 2013 13:04:54 +0530, Srivatsa S. Bhat wrote:
 @@ -246,15 +291,21 @@ struct take_cpu_down_param {
  static int __ref take_cpu_down(void *_param)
  {
   struct take_cpu_down_param *param = _param;
 - int err;
 + unsigned long flags;
 + int err = 0;

It seems no need to set 'err' to 0.

Thanks,
Namhyung

 +
 + percpu_write_lock_irqsave(hotplug_pcpu_rwlock, flags);
  
   /* Ensure this CPU doesn't handle any more interrupts. */
   err = __cpu_disable();
   if (err  0)
 - return err;
 + goto out;
  
   cpu_notify(CPU_DYING | param-mod, param-hcpu);
 - return 0;
 +
 +out:
 + percpu_write_unlock_irqrestore(hotplug_pcpu_rwlock, flags);
 + return err;
  }
  
  /* Requires cpu_add_remove_lock to be held */

 --
 To unsubscribe from this list: send the line unsubscribe linux-pm in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: [PATCH v5 04/45] percpu_rwlock: Implement the core design of Per-CPU Reader-Writer Locks

[Namhyung Kim]

On Thu, 24 Jan 2013 10:00:04 +0530, Srivatsa S. Bhat wrote:
 On 01/24/2013 01:27 AM, Tejun Heo wrote:
 On Thu, Jan 24, 2013 at 01:03:52AM +0530, Srivatsa S. Bhat wrote:
 CPU 0  CPU 1

 read_lock(rwlock)

   write_lock(rwlock) //spins, because CPU 0
   //has acquired the lock for read

 read_lock(rwlock)
^
 What happens here? Does CPU 0 start spinning (and hence deadlock) or will
 it continue realizing that it already holds the rwlock for read?
 
 I don't think rwlock allows nesting write lock inside read lock.
 read_lock(); write_lock() will always deadlock.
 

 Sure, I understand that :-) My question was, what happens when *two* CPUs
 are involved, as in, the read_lock() is invoked only on CPU 0 whereas the
 write_lock() is invoked on CPU 1.

 For example, the same scenario shown above, but with slightly different
 timing, will NOT result in a deadlock:

 Scenario 2:
   CPU 0CPU 1

 read_lock(rwlock)


 read_lock(rwlock) //doesn't spin

 write_lock(rwlock) //spins, because CPU 0
 //has acquired the lock for read


 So I was wondering whether the fairness logic of rwlocks would cause
 the second read_lock() to spin (in the first scenario shown above) because
 a writer is already waiting (and hence new readers should spin) and thus
 cause a deadlock.

In my understanding, current x86 rwlock does basically this (of course,
in an atomic fashion):


#define RW_LOCK_BIAS 0x1

rwlock_init(rwlock)
{
rwlock-lock = RW_LOCK_BIAS;
}

arch_read_lock(rwlock)
{
retry:
if (--rwlock-lock = 0)
return;

rwlock-lock++;
while (rwlock-lock  1)
continue;

goto retry;
}

arch_write_lock(rwlock)
{
retry:
if ((rwlock-lock -= RW_LOCK_BIAS) == 0)
return;

rwlock-lock += RW_LOCK_BIAS;
while (rwlock-lock != RW_LOCK_BIAS)
continue;

goto retry;
}


So I can't find where the 'fairness' logic comes from..

Thanks,
Namhyung
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev