Re: [PATCH v4 00/12] Introduce threaded trace streaming for basic perf record operation

2021-04-09 Thread Bayduraev, Alexey V
On 09.04.2021 0:52, Jiri Olsa wrote: > On Tue, Apr 06, 2021 at 11:37:26AM +0300, Bayduraev, Alexey V wrote: >> >> Changes in v4: >> - renamed 'comm' structure to 'pipes' >> - moved thread fd/maps messages to verbose=2 >> - fixed leaks during

Re: [PATCH v4 02/12] perf record: introduce thread specific data array

2021-04-09 Thread Bayduraev, Alexey V
Hi, On 08.04.2021 17:16, Andi Kleen wrote: >> +} else { >> +thread_data[t].tid = syscall(SYS_gettid); > > That always fills in the tid of the setup thread instead of the target > threads? > Here we set the tid for 0-thread (for cpu 0), other threads we setup late

[PATCH v4 12/12] perf session: use reader functions to load perf data file

2021-04-06 Thread Bayduraev, Alexey V
Use the reader functions to load data file similar to loading of data directory files. Signed-off-by: Alexey Bayduraev --- tools/perf/util/session.c | 215 -- 1 file changed, 66 insertions(+), 149 deletions(-) diff --git a/tools/perf/util/session.c b/tools/

[PATCH v4 11/12] perf session: load data directory files for analysis

2021-04-06 Thread Bayduraev, Alexey V
Introduce decompressor into trace reader object so that decompression could be executed on per data file basis separately for every data file located in data directory. Load data directory files and provide basic raw dump and aggregated analysis support of data directories in report mode, still

[PATCH v4 10/12] perf report: output data file name in raw trace dump

2021-04-06 Thread Bayduraev, Alexey V
Print path and name of a data file into raw dump (-D) @. Print offset of PERF_RECORD_COMPRESSED record instead of zero for decompressed records: 0x22...@perf.data [0x30]: event: 9 or 0x15c...@perf.data/data.7 [0x30]: event: 9 Acked-by: Namhyung Kim Signed-off-by: Alexey Bayduraev --- tool

[PATCH v4 09/12] perf record: document parallel data streaming mode

2021-04-06 Thread Bayduraev, Alexey V
Document --threads option syntax and parallel data streaming modes in Documentation/perf-record.txt. Implement compatibility checks for other modes and related command line options: asynchronous(--aio) trace streaming and affinity (--affinity) modes, pipe mode, AUX area tracing --snapshot and --a

[PATCH v4 08/12] perf record: introduce --threads= command line option

2021-04-06 Thread Bayduraev, Alexey V
Provide --threads option in perf record command line interface. The option can have a value in the form of masks that specify cpus to be monitored with data streaming threads and its layout in system topology. The masks can be filtered using cpu mask provided via -C option. The specification val

[PATCH v4 07/12] perf record: init data file at mmap buffer object

2021-04-06 Thread Bayduraev, Alexey V
Initialize data files located at mmap buffer objects so trace data can be written into several data file located at data directory. Signed-off-by: Alexey Bayduraev --- tools/perf/builtin-record.c | 41 ++--- tools/perf/util/record.h| 1 + 2 files changed, 3

[PATCH v4 06/12] perf record: introduce data file at mmap buffer object

2021-04-06 Thread Bayduraev, Alexey V
Introduce data file and compressor objects into mmap object so they could be used to process and store data stream from the corresponding kernel data buffer. Introduce bytes_transferred and bytes_compressed stats so they would capture statistics for the related data buffer transfers. Make use of

[PATCH v4 05/12] perf record: start threads in the beginning of trace streaming

2021-04-06 Thread Bayduraev, Alexey V
Start thread in detached state because its management is implemented via messaging to avoid any scaling issues. Block signals prior thread start so only main tool thread would be notified on external async signals during data collection. Thread affinity mask is used to assign eligible cpus for th

[PATCH v4 04/12] perf record: stop threads in the end of trace streaming

2021-04-06 Thread Bayduraev, Alexey V
Signal thread to terminate by closing write fd of msg pipe. Receive THREAD_MSG__READY message as the confirmation of the thread's termination. Stop threads created for parallel trace streaming prior their stats processing. Signed-off-by: Alexey Bayduraev --- tools/perf/builtin-record.c | 30 ++

[PATCH v4 03/12] perf record: introduce thread local variable

2021-04-06 Thread Bayduraev, Alexey V
Introduce thread local variable and use it for threaded trace streaming. Use thread affinity mask instead or record affinity mask in affinity modes. Introduce and use evlist__ctlfd_update() function to propagate external control commands to global evlist object. Signed-off-by: Alexey Bayduraev

[PATCH v4 02/12] perf record: introduce thread specific data array

2021-04-06 Thread Bayduraev, Alexey V
Introduce thread specific data object and array of such objects to store and manage thread local data. Implement functions to allocate, initialize, finalize and release thread specific data. Thread local maps and overwrite_maps arrays keep pointers to mmap buffer objects to serve according to ma

[PATCH v4 01/12] perf record: introduce thread affinity and mmap masks

2021-04-06 Thread Bayduraev, Alexey V
Introduce affinity and mmap thread masks. Thread affinity mask defines cpus that a thread is allowed to run on. Thread maps mask defines mmap data buffers the thread serves to stream profiling data from. Signed-off-by: Alexey Bayduraev --- tools/perf/builtin-record.c | 117

[PATCH v4 00/12] Introduce threaded trace streaming for basic perf record operation

2021-04-06 Thread Bayduraev, Alexey V
Changes in v4: - renamed 'comm' structure to 'pipes' - moved thread fd/maps messages to verbose=2 - fixed leaks during allocation of thread_data structures - fixed leaks during allocation of thread masks - fixed possible fails when releasing thread masks v3: https://lore.kernel.org/lkml/7d197a2

Re: [PATCH v3 07/12] perf record: init data file at mmap buffer object

2021-03-01 Thread Bayduraev, Alexey V
On 01.03.2021 14:44, Namhyung Kim wrote: > Hello, > > On Mon, Mar 1, 2021 at 8:16 PM Bayduraev, Alexey V > wrote: >> >> Hi, >> >> On 20.11.2020 13:49, Namhyung Kim wrote: >>> On Mon, Nov 16, 2020 at 03:19:41PM +0300, Alexey Budankov wrote: >>

Re: [PATCH v3 07/12] perf record: init data file at mmap buffer object

2021-03-01 Thread Bayduraev, Alexey V
Hi, On 20.11.2020 13:49, Namhyung Kim wrote: > On Mon, Nov 16, 2020 at 03:19:41PM +0300, Alexey Budankov wrote: >> >> @@ -1400,8 +1417,12 @@ static int record__mmap_read_evlist(struct record >> *rec, struct evlist *evlist, >> /* >> * Mark the round finished in case we wrote >>

Re: [PATCH v2 1/3] Revert "perf session: Fix decompression of PERF_RECORD_COMPRESSED records"

2020-12-02 Thread Bayduraev, Alexey V
Hi, I was able to reproduce "Couldn't allocate memory for decompression" on 32-bit perf with long perf.data. On my side mmap() in perf_session__process_compressed_event() fails with ENOMEM, due to exceeded memory limit for 32-bit applications. This happens with or without Petr's patch. As I can