On Thu, Jun 26, 2025 at 6:27 PM Gunnar Guðvarðarson <gun...@meh.is> wrote:
> I have attempted to use systemd-journald as a centralized log collector, > it works fine and the interface provided by journalctl is very convenient > to quickly find things, following logs and other things. > > But I quickly discovered the performance implications of doing so, as when > the journal gets over a certain size, it practically ceases to function. > > See https://github.com/systemd/systemd/issues/2460 for details. > "Takes 50 seconds to display logs" is slow, yes, but that's nowhere near the same as "practically ceases to function"? > This makes rsyslog's omjournal, and systemd-journal-remote basically > useless, right? > > So, serious question, can the journal be migrated to a better (indexed?) > database format, for example SQLite? > Maybe it could, but journal entries do not have fixed columns (except for timestamps and seqnum), so you wouldn't have one neat SQL table of entries – you'd probably end up with a large <id, fieldname, value> "pile of attributes" table and I'm not sure if that would perform any better. (The current format *is* indexed – each field has its own index, not just specific ones.) I think the journal is closer to a "wide-column store" or "NoSQL database" than a traditional SQL database? I'm not even close to being knowledgeable in this topic, though. But I'm not sure if a format change would help on HDDs, anyway... I have a small project that involves a ~10 GB SQLite database on an HDD, and it too sometimes takes minutes to perform the first query from cold cache (long enough that I had to extend the web server's request timeouts).
