On Sat, Mar 14, 2015 at 10:59:42AM -0400, Chris Morgan wrote: > On Fri, Mar 13, 2015 at 11:03 PM, Zbigniew Jędrzejewski-Szmek > <zbys...@in.waw.pl> wrote: > > On Fri, Mar 13, 2015 at 06:35:38PM -0400, Chris Morgan wrote: > >> Hello. > >> > >> I posted this, > >> http://lists.freedesktop.org/archives/systemd-devel/2013-July/011926.html, > >> some time ago about tiered logging for embedded systems. > >> > >> The goal is to guarantee that the flash memory will last the duration > >> of the product by carefully controlling who writes to it. > >> > >> I'm back looking at the issue and wanted to re-open the discussion. > >> > >> One approach that came up would be to set "Storage=volatile", a limit > >> of say "10MB" for the journal size, and then write a separate program > >> that would filter out the journal entries and write them to a file on > >> a physical disk. > > You should be able to do something like this right now with journal-remote: > > journalctl --directory /run/log/<bootid> -o export | \ > > systemd-journal-remote -o /var/log/<bootid>/ - > > > > Thats pretty cool. I didn't realize systemd-journal-remote existed. > Using journalctl would mean we have the ability to add filters to the > output stream. Yes.
> > This is a bit too hacky to put into production, and it would be better > > to have a single binary which does this. But all the parts are there: > > > > - following the journal and filtering, > > - opening of specific directory as input, > > - saving of state (i.e. the last cursor written), > > - writing to a directory and rotating files the same as journald does. > > > > So adding a new binary (or extending one of the existing ones) > > supporting your use case would be a matter of hooking stuff together. > > > >> The filtering portion is required as we are using the > >> journal to persist some important information that we'd like to log. > >> We'd also like to preserve high priority messages but don't mind if we > >> lose low priority ones across reboots. > >> > >> An upside of the external program is that we can filter on both high > >> priority messages as well as those with specific "FIELD=value" > >> entries. Downside is a custom format file and can't use journalctl to > >> search through it, no log rotation, no size limits etc. > >> > >> At the time there was some thought of putting this into journald > >> itself. I'm wondering how that would fit given the desire to use more > >> complicated matching to decide which entries were put into the > >> persisted journal. > > > > Adding filtering and splitting functionality to journald is another > > story of course. It probably would work better: more efficent, and > > journal entries would not be duplicated at all. > > > > The piping example you gave does seem to be a bit more heavyweight > than a process that was using the sd_journal_xxx() calls to interact > with the journal. > > It doesn't look like systemd-journal-remote supports size limits or > rotation, the man page doesn't have any options for that. It does. They are not configurable, so it just uses the defaults. I guess that this should be added. > Would it be much less efficient to make yet another program that used > sd_journal_xxx() calls, and whatever functionality > systemd-journal-remote used to write to a new journal vs. trying to > extend journald to do this internally? I'd rather not try to force > things into journald proper that don't belong there if the separate > application approach is nearly as performant and much cleaner. One thing which implementing this in journald would give over implementing is in something separate: journald could be configured to split messages between /run, or /var. journalctl has the ability to transparently merge events from multiple directories, so this split would be invisible to users. I don't see how this could be done so nicely with a separate tool. > Or possibly enhance systemd-journal-remote to also support following > the journal in addition to its current behavior of taking input from > stdin? Yeah, that seems like a good idea. > >> If it would fit inside of journald I'd be willing to implement it but > >> we would need to come up with a way to configure the filtering, where > >> the files are persisted etc. The filtering is a new requirement since > >> the last time this was discussed. > > I think that supporting a set of similar filters to journalctl would > > be a good start. Options which limit the number of messages or > > filter based on time would not make sense for journald, but most > > other probably would. > > > > Not following what you are referring to when you say "supporting a set > of similar filters to journalctl would be a good start". _FIELD=value, --priority, --unit, --user-unit, --dmesg, --identifier, --user, --system. Zbyszek _______________________________________________ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
