Hi On Thu, Oct 29, 2026 at 9:33 PM Daniel P. Berrangé <[email protected]> wrote: > > On Fri, Oct 16, 2026 at 11:20:11AM +0400, Marc-André Lureau wrote: > > Hi > > > > On Thu, Oct 15, 2026 at 4:32 PM Daniel P. Berrangé <[email protected]> > > wrote: > > > > > > On Thu, Oct 15, 2026 at 04:15:52PM +0400, [email protected] > > > wrote: > > > > > This series introduces a new facility called 'tldr' which is a > > > > > live data recorder. This is intended to aid support of production > > > > > QEMU deployments, by providing a data collection facility that > > > > > can be left "always on", such that in the event of a QEMU crash > > > > > or guest OS crash, certain important information will be recorded. > > > > > > > > > > The data recorder has to be enabled by adding a "tldr-config" > > > > > object instance with -object / object_add. > > > > > > > > > > With this series, the data recorder collects > > > > > > > > > > * error_report/warn_report/error_printf messages > > > > > * QMP events > > > > > > > > It should probably be recorded at emission, not per monitor > > > > > > Opps. > > > > > > > > * QMP errors > > > > > * QMP commands, which are not query-XXX/qom-list, > > > > > or any command associated with a QMP error. > > > > > * qemu_log output > > > > > > > > > > The last item is intended primarily to be used in combination > > > > > with tracing and its 'log' backend, to have continuous > > > > > recording of certain interesting probes. > > > > > > > > > > Each type of data is collected in a distinct circular > > > > > buffer, such that high volumes of one type of data, does > > > > > not result in overwriting other types of low volume data. > > > > > > > > > > The data recorder is set to trigger on SIGBUS, SIGSEGV, > > > > > SIGILL, SIGFPE and SIGABRT, or manually via QMP 'tldr-dump'. > > > > > > > > Instead of allocating and dumping by a signal handler, why not use a > > > > file-backed > > > > mmap or a memfd? We would need to define a small fixed header for the > > > > file > > > > layout/buckets. This would avoid all the issues with signal handlers. > > > > > > That would impose a continuous I/O workload on data recording, which > > > is something I wanted to explicitly avoid, until the time the data is > > > > Yes, a file-backed mapping shouldn't be the default, but memfd/tmpfs > > should have the same performance. > > > > > needed to be preserved. This only persists the final data, not any > > > intermediate data that is overwritten as the circular buffer wraps. > > > > Same design, the reader would handle the ring data > > > > > Coupling the in-memory layout to the on-disk layout also means that > > > dumps would always be the max size, since we would not have grow-on > > > demand buffers. > > > > The dump output would not be the raw data, a reader would emit the > > same text format, skipping the unused parts. > > IMHO that reader has to be QEMU owned code, as otherwise we're > exposing a private data structure to a 3rd party and that makes > it public ABI/API which is highly undesirable for this.
The resulting data will be ABI/API anyway, no? > > > mapping reserves address space only, pages are allocated on first write. > > > > The benefits: > > - no signal handler, no conflict with MCE, ASAN etc > > A signal handler is integral to some extent, as captured info about > the signal is itself is an important part of the data record. IOW, > even if moving writing of the dump out of the signal handler in some > manner, I still want to be able to capture several siginfo_t fields. > > You can get that info from a core dump, but the working premise > is that a core dump is not guaranteed available, whereas the data > record dumps are something we can more easily acquire. right.. this is something you don't touch in the cover letter. I assume most systems have a core dump handler already (perhaps differently on Windows), thus we could also rely on the dumped data instead of a signal handler. We should be explicit about why we don't follow that path systemd/coredumpctl does store siginfo_t of the process too > > > - data survives SIGKILL, OOM, or a crash in the dump path itself > > Simplifying the signal handler to just stash the siginfo_t would > help. > > > - data can be read from a hung QEMU > > Data can always be read from a hung QEMU by sending it a signal. > We could wire up a non-fatal signal for this purpose if we want > to read hung QEMUs without killing them. > > > Someone needs to hold the memfd to outlive QEMU, otherwise QEMU could > > create a file under a runtime dir on tmpfs > > I wonder if QEMU could simply fork a child process itself that just > hangs around waiting for death, and has a shared memory region with > the main QEMU. > > We already have a '-run-with async-teardown' comamnd line, which > forks a child process and uses prctl(PR_SET_PDEATHSIG) to get > notified with the parent goes away. We could expand that child > process to also handle crash dump writing. Yes, that should work, although I would like to have a rationale why/when core dump aren't good enough. thanks
