Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[Jiri Olsa]

On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> I meet strange behavior with below commands on my gentoo box:
> 1:perf kmem record
> 2:CTRL-C to stop 1
> 3:perf report
> 4:"Enter", "Enter", "Run scripts for all samples",
>   "event_analyzing_sample".
> 
> Then perf report:
> "
> No kallsyms or vmlinux with build-id  was found
> /lib/modules/4.10.0+/build/vmlinux with build id  not found,
> continuing without symbols
> ".
> 
> It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> is right for perf.data.
> 
> After digging, I find out the reason is "perf report" generates many
> open fds, then "script_browse" uses popen to run "perf script" which
> run out of open files. The gentoo box has a small default value for
> "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> fix it, but I think that using O_CLOEXEC in do_open is a better way.
> 
> Signed-off-by: Wang YanQing 

Acked-by: Jiri Olsa 

thanks,
jirka

> ---
>  tools/perf/util/dso.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> index d5b6f7f..36ef45b 100644
> --- a/tools/perf/util/dso.c
> +++ b/tools/perf/util/dso.c
> @@ -446,7 +446,7 @@ static int do_open(char *name)
>   char sbuf[STRERR_BUFSIZE];
>  
>   do {
> - fd = open(name, O_RDONLY);
> + fd = open(name, O_RDONLY|O_CLOEXEC);
>   if (fd >= 0)
>   return fd;
>  
> -- 
> 1.8.5.6.2.g3d8a54e.dirty

Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[Jiri Olsa]

On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> I meet strange behavior with below commands on my gentoo box:
> 1:perf kmem record
> 2:CTRL-C to stop 1
> 3:perf report
> 4:"Enter", "Enter", "Run scripts for all samples",
>   "event_analyzing_sample".
> 
> Then perf report:
> "
> No kallsyms or vmlinux with build-id  was found
> /lib/modules/4.10.0+/build/vmlinux with build id  not found,
> continuing without symbols
> ".
> 
> It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> is right for perf.data.
> 
> After digging, I find out the reason is "perf report" generates many
> open fds, then "script_browse" uses popen to run "perf script" which
> run out of open files. The gentoo box has a small default value for
> "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> fix it, but I think that using O_CLOEXEC in do_open is a better way.
> 
> Signed-off-by: Wang YanQing 

Acked-by: Jiri Olsa 

thanks,
jirka

> ---
>  tools/perf/util/dso.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> index d5b6f7f..36ef45b 100644
> --- a/tools/perf/util/dso.c
> +++ b/tools/perf/util/dso.c
> @@ -446,7 +446,7 @@ static int do_open(char *name)
>   char sbuf[STRERR_BUFSIZE];
>  
>   do {
> - fd = open(name, O_RDONLY);
> + fd = open(name, O_RDONLY|O_CLOEXEC);
>   if (fd >= 0)
>   return fd;
>  
> -- 
> 1.8.5.6.2.g3d8a54e.dirty

Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[Wang YanQing]

On Mon, Jan 15, 2018 at 10:45:06AM +0100, Jiri Olsa wrote:
> On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> > I meet strange behavior with below commands on my gentoo box:
> > 1:perf kmem record
> > 2:CTRL-C to stop 1
> > 3:perf report
> > 4:"Enter", "Enter", "Run scripts for all samples",
> >   "event_analyzing_sample".
> > 
> > Then perf report:
> > "
> > No kallsyms or vmlinux with build-id  was found
> > /lib/modules/4.10.0+/build/vmlinux with build id  not found,
> > continuing without symbols
> > ".
> > 
> > It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> > is right for perf.data.
> > 
> > After digging, I find out the reason is "perf report" generates many
> > open fds, then "script_browse" uses popen to run "perf script" which
> > run out of open files. The gentoo box has a small default value for
> > "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> > fix it, but I think that using O_CLOEXEC in do_open is a better way.
> 
> that seems right.. so why does kmem record open those
> files before calling cmd_record? should that be fixed?
The problem is "perf report", isn't "perf kmem record",
see the above reproduction steps again.

The problem is "perf report" generates may open fds with
dso:do_open for symbol analysis, and "perf report" uses
popen to run another command "perf script", then the cmd
"perf script" will run out of "max open files".

Thanks.
> 
> thanks,
> jirka
> 
> > 
> > Signed-off-by: Wang YanQing 
> > ---
> >  tools/perf/util/dso.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> > index d5b6f7f..36ef45b 100644
> > --- a/tools/perf/util/dso.c
> > +++ b/tools/perf/util/dso.c
> > @@ -446,7 +446,7 @@ static int do_open(char *name)
> > char sbuf[STRERR_BUFSIZE];
> >  
> > do {
> > -   fd = open(name, O_RDONLY);
> > +   fd = open(name, O_RDONLY|O_CLOEXEC);
> > if (fd >= 0)
> > return fd;
> >  
> > -- 
> > 1.8.5.6.2.g3d8a54e.dirty

Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[Wang YanQing]

On Mon, Jan 15, 2018 at 10:45:06AM +0100, Jiri Olsa wrote:
> On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> > I meet strange behavior with below commands on my gentoo box:
> > 1:perf kmem record
> > 2:CTRL-C to stop 1
> > 3:perf report
> > 4:"Enter", "Enter", "Run scripts for all samples",
> >   "event_analyzing_sample".
> > 
> > Then perf report:
> > "
> > No kallsyms or vmlinux with build-id  was found
> > /lib/modules/4.10.0+/build/vmlinux with build id  not found,
> > continuing without symbols
> > ".
> > 
> > It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> > is right for perf.data.
> > 
> > After digging, I find out the reason is "perf report" generates many
> > open fds, then "script_browse" uses popen to run "perf script" which
> > run out of open files. The gentoo box has a small default value for
> > "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> > fix it, but I think that using O_CLOEXEC in do_open is a better way.
> 
> that seems right.. so why does kmem record open those
> files before calling cmd_record? should that be fixed?
The problem is "perf report", isn't "perf kmem record",
see the above reproduction steps again.

The problem is "perf report" generates may open fds with
dso:do_open for symbol analysis, and "perf report" uses
popen to run another command "perf script", then the cmd
"perf script" will run out of "max open files".

Thanks.
> 
> thanks,
> jirka
> 
> > 
> > Signed-off-by: Wang YanQing 
> > ---
> >  tools/perf/util/dso.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> > index d5b6f7f..36ef45b 100644
> > --- a/tools/perf/util/dso.c
> > +++ b/tools/perf/util/dso.c
> > @@ -446,7 +446,7 @@ static int do_open(char *name)
> > char sbuf[STRERR_BUFSIZE];
> >  
> > do {
> > -   fd = open(name, O_RDONLY);
> > +   fd = open(name, O_RDONLY|O_CLOEXEC);
> > if (fd >= 0)
> > return fd;
> >  
> > -- 
> > 1.8.5.6.2.g3d8a54e.dirty

Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[Jiri Olsa]

On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> I meet strange behavior with below commands on my gentoo box:
> 1:perf kmem record
> 2:CTRL-C to stop 1
> 3:perf report
> 4:"Enter", "Enter", "Run scripts for all samples",
>   "event_analyzing_sample".
> 
> Then perf report:
> "
> No kallsyms or vmlinux with build-id  was found
> /lib/modules/4.10.0+/build/vmlinux with build id  not found,
> continuing without symbols
> ".
> 
> It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> is right for perf.data.
> 
> After digging, I find out the reason is "perf report" generates many
> open fds, then "script_browse" uses popen to run "perf script" which
> run out of open files. The gentoo box has a small default value for
> "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> fix it, but I think that using O_CLOEXEC in do_open is a better way.

that seems right.. so why does kmem record open those
files before calling cmd_record? should that be fixed?

thanks,
jirka

> 
> Signed-off-by: Wang YanQing 
> ---
>  tools/perf/util/dso.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> index d5b6f7f..36ef45b 100644
> --- a/tools/perf/util/dso.c
> +++ b/tools/perf/util/dso.c
> @@ -446,7 +446,7 @@ static int do_open(char *name)
>   char sbuf[STRERR_BUFSIZE];
>  
>   do {
> - fd = open(name, O_RDONLY);
> + fd = open(name, O_RDONLY|O_CLOEXEC);
>   if (fd >= 0)
>   return fd;
>  
> -- 
> 1.8.5.6.2.g3d8a54e.dirty

Re: [PATCH] perf:util:dso: Using O_CLOEXEC in do_open

[Jiri Olsa]

On Mon, Jan 15, 2018 at 01:04:48PM +0800, Wang YanQing wrote:
> I meet strange behavior with below commands on my gentoo box:
> 1:perf kmem record
> 2:CTRL-C to stop 1
> 3:perf report
> 4:"Enter", "Enter", "Run scripts for all samples",
>   "event_analyzing_sample".
> 
> Then perf report:
> "
> No kallsyms or vmlinux with build-id  was found
> /lib/modules/4.10.0+/build/vmlinux with build id  not found,
> continuing without symbols
> ".
> 
> It is strange because I am sure /lib/modules/4.10.0+/build/vmlinux
> is right for perf.data.
> 
> After digging, I find out the reason is "perf report" generates many
> open fds, then "script_browse" uses popen to run "perf script" which
> run out of open files. The gentoo box has a small default value for
> "max open files", 1024. Yes, "ulimit -n " with a bigger number could
> fix it, but I think that using O_CLOEXEC in do_open is a better way.

that seems right.. so why does kmem record open those
files before calling cmd_record? should that be fixed?

thanks,
jirka

> 
> Signed-off-by: Wang YanQing 
> ---
>  tools/perf/util/dso.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
> index d5b6f7f..36ef45b 100644
> --- a/tools/perf/util/dso.c
> +++ b/tools/perf/util/dso.c
> @@ -446,7 +446,7 @@ static int do_open(char *name)
>   char sbuf[STRERR_BUFSIZE];
>  
>   do {
> - fd = open(name, O_RDONLY);
> + fd = open(name, O_RDONLY|O_CLOEXEC);
>   if (fd >= 0)
>   return fd;
>  
> -- 
> 1.8.5.6.2.g3d8a54e.dirty