----- On Mar 5, 2019, at 4:43 AM, Thomas Schickentanz <thomas.schickent...@harman.com> wrote:
> Hi Mathieu, > Thanks for clarifying the restrictions of ‘shm-path’ option! > Our purpose is to use exclusively reserved memory for tracing. Our customers > request a certain amount of free memory on the system for future development. > In development stage chunks of this “free” memory are grabbed by the > applications and tracing lacks memory. We want to allocate this “free” memory > in an early boot stage to ensure that we get the amount of memory we need for > tracing. E.g. if we have reserved 512 Mbyte of memory, we know that we can > start lttng at a later point of time with 16 buffers of 4 Mbyte on a 8 core > system. But this works only, if we can provide the previously reserved memory > to lttng for buffer allocation. > Is there some general concern about using POSIX shared memory for buffers in > kernel tracing mode (apart from performance reasons)? The kernel tracer does not use POSIX shared memory at all at the moment. Only lttng-ust does. lttng-modules allocates memory for its buffers directly from the kernel page allocator all by itself. Switching to a design where an application process (kernel consumerd) would map memory associated to files on a filesystem, and then hand over that memory to the kernel so it can trace there would be a rather large design change. For the needs you have, I think a simpler option would be to implement a new setup argument to lttng-modules so it can reserve some memory from the page allocator when the lttng modules are loaded (e.g. at system start), and then use that pre-reserved pool to allocate the buffers when a tracing session is created. All this would need to be NUMA-aware, with e.g. one pool per NUMA node. This would be a new feature request against lttng-modules. In order to reserve memory for lttng-ust, you could ensure your /dev/shm mountpoint is a tmpfs filesystem which contains enough memory to hold the lttng-ust buffers. An alternative there would be to use pmem + dax to hold the buffers with the --shm-path option. Thanks, Mathieu > I used lttng-crash only for testing, because I wanted to verify, if tracing > data > is available from provided shared memory. If this is ensured, I agree with > you, > that ‘lttng snapshot’ can do the work. > Regards, > Thomas > From: Mathieu Desnoyers <mathieu.desnoy...@efficios.com> > Sent: Montag, 4. März 2019 17:01 > To: Schickentanz, Thomas <thomas.schickent...@harman.com> > Cc: lttng-dev <lttng-dev@lists.lttng.org> > Subject: [EXTERNAL] Re: [lttng-dev] Tracing directly into shared memory > ----- On Mar 4, 2019, at 4:28 AM, Thomas Schickentanz < [ > mailto:thomas.schickent...@harman.com | thomas.schickent...@harman.com ] > > wrote: >> Hi all, >> My idea is to pre-allocate memory on system startup which can later be used >> by >> lttng for tracing. It should work like flight recorder mode without >> extracting >> data from ringbuffers to trace file while recording data. Data extraction of >> shm ringbuffers should be done when recording has been finished. >> As per documention with option `--shm-path‘ ringbuffer for tracing can be >> created in shared memory. With lttng-crash data can be extracted from >> ringbuffer. >> Can this option be used with shared memory in volatile memory (default ram), >> too? And is it possible to record kernel space traces with this option? >> I tried >> lttng create testshm --shm-path=/dev/shm --snapshot >> lttng enable-channel c0 -s testshm -k >> lttng enable-event -s testshm -c c0 -k -a >> lttng start >> lttng stop >> lttng-crash -extract=/home/test /dev/shm >> With lttng and lttng-crash everything seems to work fine (no error messages). >> However the extract folder keeps empty. >> Has someone researched into this option and can help to clarify the points? > Hi Thomas, > About "--shm-path" and lttng-crash, those are discussed here: [ > https://clicktime.symantec.com/3YZ15pbJVSD47yeXxhXiwqE7Vc?u=https%3A%2F%2Flttng.org%2Fdocs%2Fv2.10%2F%23doc-persistent-memory-file-systems > | > https://lttng.org/docs/v2.10/#doc-persistent-memory-file-systems ] > I'm not sure I completely understand your use-case and whether your intent is > to > extract trace data after a > system crash or if the "snapshot" mode is enough for your needs. > Note that the "shm-path" option only applies to UST (user-space) traces at the > moment. It has not been implemented > for kernel tracing yet. > The way we typically use it for extraction of trace data from user-space when > a > system crashes is to use the pmem driver to > create a block device out of a range of the system memory, and then use ext2 > or > ext4 with the DAX mount option to ensure > the filesystem bypasses the page cache. > Then we ensure that kexec is used to boot a new kernel on crash, or make sure > the BIOS does not erase the > memory content on soft reboot, which gives access to the shm-path buffers > after > crash/reboot. > We don't usually use /dev/shm as an argument to --shm-path. This path is > normally used to hold POSIX shared memory > files. I would expect a different mount point path as argument to --shm-path > which points to a DAX-enabled filesystem > backed by a PMEM block device. > Thanks, > Mathieu > -- > Mathieu Desnoyers > EfficiOS Inc. > [ > https://clicktime.symantec.com/32vnSPUsVCfHCkkwoyPymVx7Vc?u=http%3A%2F%2Fwww.efficios.com > | > https://clicktime.symantec.com/32vnSPUsVCfHCkkwoyPymVx7Vc?u=http%3A%2F%2Fwww.efficios.com > ] -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
_______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
