Re: [gentoo-user] syslog-ng: how to read the log files
On Tue, Feb 17, 2015 at 1:26 PM, lee wrote: > Hi, > > how do you read the log files when using syslog-ng? > > The log file seem to be some sort of binary that doesn't display too > well in less, and there doesn't seem to be any way to read them. That's news to me. Are you sure you're not looking at wtmp or something like that (which isn't maintained by syslog)? -- Rich
Re: [gentoo-user] syslog-ng: how to read the log files
On Tue, Feb 17, 2015 at 8:26 PM, lee wrote: > Hi, > > how do you read the log files when using syslog-ng? > > The log file seem to be some sort of binary that doesn't display too > well in less, and there doesn't seem to be any way to read them. > > > -- > Again we must be afraid of speaking of daemons for fear that daemons > might swallow us. Finally, this fear has become reasonable. > > If you're talking about /var/log/messages, which is: messages: data I use cat(1).
Re: [gentoo-user] syslog-ng: how to read the log files
> On Feb 17, 2015, at 20:26, lee wrote: > > Hi, > > how do you read the log files when using syslog-ng? > > The log file seem to be some sort of binary that doesn't display too > well in less, and there doesn't seem to be any way to read them. This was discussed earlier on this list... Actually what syslog-ng produces is plain text. There seemed to be a bug that creates some binary (i.e. unreadable characters) and that causes less to consider files to be binary and show them incorrectly. To work around you can use -r flag with less, or replace/remove unreadable chars from log, or delete the log file. -- -Matti
Re: [gentoo-user] syslog-ng: how to read the log files
Hello, Lee. On Tue, Feb 17, 2015 at 07:26:05PM +0100, lee wrote: > Hi, > how do you read the log files when using syslog-ng? > The log file seem to be some sort of binary that doesn't display too > well in less, and there doesn't seem to be any way to read them. When I try "less /var/log/messages", less gives me what is basically a hex dump of the file. I'm assuming you see the same. less searches part of the buffer (presumably the first few KB) and if it finds non-printable characters, uses an input filter first to convert to the hex dump. This same filter is what enables less to expand compressed files and man pages. What I do is to disable this input filter with # LESSOPEN="" less /var/log/messages . It is evident that every now and then, syslog-ng writes a stream of several hundred null bytes to /var/log/messages. It seems to do this when logging the system startup messages. This is probably a bug. By the way, the LESSOPEN="" trick can sometimes leave your display corrupted, displaying wierd glyphs on the screen when you type. To restore your screen, output ^o. To do this, type (blindly) # echo . > -- > Again we must be afraid of speaking of daemons for fear that daemons > might swallow us. Finally, this fear has become reasonable. -- Alan Mackenzie (Nuremberg, Germany).
Re: [gentoo-user] syslog-ng: how to read the log files
On Tue, Feb 17, 2015 at 8:38 PM, Alexander Kapshuk < alexander.kaps...@gmail.com> wrote: > On Tue, Feb 17, 2015 at 8:26 PM, lee wrote: > >> Hi, >> >> how do you read the log files when using syslog-ng? >> >> The log file seem to be some sort of binary that doesn't display too >> well in less, and there doesn't seem to be any way to read them. >> >> >> -- >> Again we must be afraid of speaking of daemons for fear that daemons >> might swallow us. Finally, this fear has become reasonable. >> >> > If you're talking about /var/log/messages, which is: > messages: data > > I use cat(1). > > Just tried 'sed p /var/log/messages', which seems to work as well.
Re: [gentoo-user] syslog-ng: how to read the log files
Alan Mackenzie writes: > Hello, Lee. > > On Tue, Feb 17, 2015 at 07:26:05PM +0100, lee wrote: >> Hi, > >> how do you read the log files when using syslog-ng? > >> The log file seem to be some sort of binary that doesn't display too >> well in less, and there doesn't seem to be any way to read them. > > When I try "less /var/log/messages", less gives me what is basically a > hex dump of the file. I'm assuming you see the same. Yes, that's what I was looking at. > less searches part of the buffer (presumably the first few KB) and if it > finds non-printable characters, uses an input filter first to convert to > the hex dump. Is that a new feature of less? I've never had this problem with any other file. IIRC, unprintable characters, like null, used to be displayed like ^@, and less always did a great job in preventing the display from needing a reset without switching to an equivalent of hexl-mode. BTW, what happens when something writes to /var/log/messages? I noticed today that the default shorewall.conf that ships with gentoo has that set as logfile for shorewall. Shouldn't all messages going into /var/log/messages go to syslog-ng instead when syslog-ng is used, with nothing else writing to this file? -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
Alexander Kapshuk wrote: > On Tue, Feb 17, 2015 at 8:26 PM, lee wrote: > > > Hi, > > > > how do you read the log files when using syslog-ng? > > > > The log file seem to be some sort of binary that doesn't display too > > well in less, and there doesn't seem to be any way to read them. > > > > > > -- > > Again we must be afraid of speaking of daemons for fear that daemons > > might swallow us. Finally, this fear has become reasonable. > > > > > If you're talking about /var/log/messages, which is: > messages: data > > I use cat(1). I wonder if the OP is using systemd and trying to read the journal files? -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] syslog-ng: how to read the log files
On Tue, Feb 17, 2015 at 2:29 PM, wrote: > > Alexander Kapshuk wrote: > > > On Tue, Feb 17, 2015 at 8:26 PM, lee wrote: > > > > > Hi, > > > > > > how do you read the log files when using syslog-ng? > > > > > > The log file seem to be some sort of binary that doesn't display too > > > well in less, and there doesn't seem to be any way to read them. > > > > > > > > > -- > > > Again we must be afraid of speaking of daemons for fear that daemons > > > might swallow us. Finally, this fear has become reasonable. > > > > > > > > If you're talking about /var/log/messages, which is: > > messages: data > > > > I use cat(1). > > I wonder if the OP is using systemd and trying to read the journal > files? Those live under /var/lib/journal (which you need to create; Gentoo doesn't do it by default last time I saw), in the directory named after the machine ID (cat /etc/machine-id). And there are several journal files, of the kind: system@1df50cd49c7f4a089c9414561f65aac7-0006f091-000507235df68768.journal I think it would be really difficult to mix up that with /var/log/messages. I think it's just that some part of /var/log/messages got corrupted (happens a lot of times), and therefore /usr/bin/less identifies it as a binary files since it contains non-printable characters. Regards. -- Canek Peláez Valdés Profesor de asignatura, Facultad de Ciencias Universidad Nacional Autónoma de México
Re: [gentoo-user] syslog-ng: how to read the log files
On Tuesday 17 Feb 2015 19:17:20 lee wrote: > Alan Mackenzie writes: > > Hello, Lee. > > > > On Tue, Feb 17, 2015 at 07:26:05PM +0100, lee wrote: > >> Hi, > >> > >> how do you read the log files when using syslog-ng? > >> > >> The log file seem to be some sort of binary that doesn't display too > >> well in less, and there doesn't seem to be any way to read them. > > > > When I try "less /var/log/messages", less gives me what is basically a > > hex dump of the file. I'm assuming you see the same. > > Yes, that's what I was looking at. > > > less searches part of the buffer (presumably the first few KB) and if it > > finds non-printable characters, uses an input filter first to convert to > > the hex dump. > > Is that a new feature of less? I've never had this problem with any > other file. IIRC, unprintable characters, like null, used to be > displayed like ^@, and less always did a great job in preventing the > display from needing a reset without switching to an equivalent of > hexl-mode. > > > BTW, what happens when something writes to /var/log/messages? I noticed > today that the default shorewall.conf that ships with gentoo has that > set as logfile for shorewall. Shouldn't all messages going into > /var/log/messages go to syslog-ng instead when syslog-ng is used, with > nothing else writing to this file? It depends on what filters have been set in the configuration file of the application in question or syslog-ng. I use less -L /var/log/messages to see the content of the log files in plain text. At boot up I get a load of: Feb 16 07:54:04 ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ kernel: Initializing cgroup subsys cpuset Feb 16 07:54:04 ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ being printed up. Perhaps I will disable cgroups in the kernel and see what gives. I don't use containers anyway. -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] syslog-ng: how to read the log files
Am Tue, 17 Feb 2015 13:45:38 -0600 schrieb Canek Peláez Valdés : > On Tue, Feb 17, 2015 at 2:29 PM, wrote: > > > > Alexander Kapshuk wrote: > > > > > On Tue, Feb 17, 2015 at 8:26 PM, lee wrote: > > > > > > > Hi, > > > > > > > > how do you read the log files when using syslog-ng? > > > > > > > > The log file seem to be some sort of binary that doesn't display too > > > > well in less, and there doesn't seem to be any way to read them. > > > > > > > > > > > > -- > > > > Again we must be afraid of speaking of daemons for fear that daemons > > > > might swallow us. Finally, this fear has become reasonable. > > > > > > > > > > > If you're talking about /var/log/messages, which is: > > > messages: data > > > > > > I use cat(1). > > > > I wonder if the OP is using systemd and trying to read the journal > > files? > > Those live under /var/lib/journal (which you need to create; Gentoo doesn't > do it by default last time I saw) [...] It did on my laptop after I migrated it to systemd over the weekend (on a whim, no less -- apparently I'm adventurous?). Or, to be more precise, I didn't have to create the directory myself. And wouldn't it be created at run-time, anyway? That's what I would expect, at least. [...] -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup pgpATVahurywy.pgp Description: Digitale Signatur von OpenPGP
Re: [gentoo-user] syslog-ng: how to read the log files
On Tuesday 17 February 2015 20:41:06 Matti Nykyri wrote: > > On Feb 17, 2015, at 20:26, lee wrote: > > how do you read the log files when using syslog-ng? > > The log file seem to be some sort of binary that doesn't display too > > well in less, and there doesn't seem to be any way to read them. > > This was discussed earlier on this list... Actually what syslog-ng > produces is plain text. There seemed to be a bug that creates some > binary (i.e. unreadable characters) and that causes less to consider > files to be binary and show them incorrectly. Yes, that was me. I found that something had marked /var/log/messages as a binary file. There's nothing in it that can't be read, no mysterious characters or anything; it's just marked as binary. All you have to do is to move it, then cat it back into place. I did that with no ill effects. -- Rgds Peter.
Re: [gentoo-user] syslog-ng: how to read the log files
On Tuesday 17 February 2015 22:51:55 Peter Humphrey wrote: > On Tuesday 17 February 2015 20:41:06 Matti Nykyri wrote: > > > On Feb 17, 2015, at 20:26, lee wrote: > > > how do you read the log files when using syslog-ng? > > > The log file seem to be some sort of binary that doesn't display > > > too > > > well in less, and there doesn't seem to be any way to read them. > > > > This was discussed earlier on this list... Actually what syslog-ng > > produces is plain text. There seemed to be a bug that creates some > > binary (i.e. unreadable characters) and that causes less to consider > > files to be binary and show them incorrectly. > > Yes, that was me. I found that something had marked /var/log/messages > as a binary file. There's nothing in it that can't be read, no > mysterious characters or anything; it's just marked as binary. All > you have to do is to move it, then cat it back into place. I did that > with no ill effects. Actually, this is what I did, as I reported here on 26/12: > 1.Boot rescue system and mount main system > 2.# cd /mnt/main/var/log > 3.# mv messages messages.bin > 4.# strings messages.bin > messages > 5.# rm messages.bin > 6.Reboot. -- Rgds Peter.
Re: [gentoo-user] syslog-ng: how to read the log files
On 02/18/2015 12:13 AM, Peter Humphrey wrote: > On Tuesday 17 February 2015 22:51:55 Peter Humphrey wrote: >> On Tuesday 17 February 2015 20:41:06 Matti Nykyri wrote: On Feb 17, 2015, at 20:26, lee wrote: how do you read the log files when using syslog-ng? The log file seem to be some sort of binary that doesn't display too well in less, and there doesn't seem to be any way to read them. >>> >>> This was discussed earlier on this list... Actually what syslog-ng >>> produces is plain text. There seemed to be a bug that creates some >>> binary (i.e. unreadable characters) and that causes less to consider >>> files to be binary and show them incorrectly. >> >> Yes, that was me. I found that something had marked /var/log/messages >> as a binary file. There's nothing in it that can't be read, no >> mysterious characters or anything; it's just marked as binary. All >> you have to do is to move it, then cat it back into place. I did that >> with no ill effects. > > Actually, this is what I did, as I reported here on 26/12: > >> 1.Boot rescue system and mount main system >> 2.# cd /mnt/main/var/log >> 3.# mv messages messages.bin >> 4.# strings messages.bin > messages >> 5.# rm messages.bin >> 6.Reboot. > When I had similar problem, I changed threaded(yes) to threaded(no) in syslog-ng.conf and the problem disappeared. Maybe it helps you too. -- Jan Sever
Re: [gentoo-user] syslog-ng: how to read the log files
On Tue, Feb 17, 2015 at 5:51 PM, Peter Humphrey wrote: > > Yes, that was me. I found that something had marked /var/log/messages as > a binary file. There's nothing in it that can't be read, no mysterious > characters or anything; it's just marked as binary. All you have to do > is to move it, then cat it back into place. I did that with no ill > effects. > Uh, of what binary "mark" are you speaking? Seems likely that however you processed the file stripped it of whatever was causing less to consider it as binary. I don't think cat alone would do anything to the file, but I'm not certain of that. -- Rich
Re: [gentoo-user] syslog-ng: how to read the log files
On Wednesday 18 February 2015 00:23:19 Jan Sever wrote: > On 02/18/2015 12:13 AM, Peter Humphrey wrote: > > On Tuesday 17 February 2015 22:51:55 Peter Humphrey wrote: > >> On Tuesday 17 February 2015 20:41:06 Matti Nykyri wrote: > On Feb 17, 2015, at 20:26, lee wrote: > how do you read the log files when using syslog-ng? > The log file seem to be some sort of binary that doesn't display > too > well in less, and there doesn't seem to be any way to read them. > >>> > >>> This was discussed earlier on this list... Actually what syslog-ng > >>> produces is plain text. There seemed to be a bug that creates some > >>> binary (i.e. unreadable characters) and that causes less to > >>> consider > >>> files to be binary and show them incorrectly. > >> > >> Yes, that was me. I found that something had marked > >> /var/log/messages > >> as a binary file. There's nothing in it that can't be read, no > >> mysterious characters or anything; it's just marked as binary. All > >> you have to do is to move it, then cat it back into place. I did > >> that > >> with no ill effects. > > > > Actually, this is what I did, as I reported here on 26/12: > >> 1.Boot rescue system and mount main system > >> 2.# cd /mnt/main/var/log > >> 3.# mv messages messages.bin > >> 4.# strings messages.bin > messages > >> 5.# rm messages.bin > >> 6.Reboot. > > When I had similar problem, I changed threaded(yes) to threaded(no) > in syslog-ng.conf and the problem disappeared. Maybe it helps you too. Good idea. Sounds like a bug report is needed, unless it's already been superseded. -- Rgds Peter.
Re: [gentoo-user] syslog-ng: how to read the log files
On Tuesday 17 February 2015 18:52:07 Rich Freeman wrote: > On Tue, Feb 17, 2015 at 5:51 PM, Peter Humphrey wrote: > > Yes, that was me. I found that something had marked > > /var/log/messages as a binary file. There's nothing in it that > > can't be read, no mysterious characters or anything; it's just > > marked as binary. All you have to do is to move it, then cat it > > back into place. I did that with no ill effects. > > Uh, of what binary "mark" are you speaking? Seems likely that however > you processed the file stripped it of whatever was causing less to > consider it as binary. I don't think cat alone would do anything to > the file, but I'm not certain of that. I don't know. Are we talking magic here? -- Rgds Peter.
Re: [gentoo-user] syslog-ng: how to read the log files
On Tuesday 17 Feb 2015 23:13:08 Peter Humphrey wrote: > On Tuesday 17 February 2015 22:51:55 Peter Humphrey wrote: > > On Tuesday 17 February 2015 20:41:06 Matti Nykyri wrote: > > > > On Feb 17, 2015, at 20:26, lee wrote: > > > > how do you read the log files when using syslog-ng? > > > > The log file seem to be some sort of binary that doesn't display > > > > too > > > > well in less, and there doesn't seem to be any way to read them. > > > > > > This was discussed earlier on this list... Actually what syslog-ng > > > produces is plain text. There seemed to be a bug that creates some > > > binary (i.e. unreadable characters) and that causes less to consider > > > files to be binary and show them incorrectly. > > > > Yes, that was me. I found that something had marked /var/log/messages > > as a binary file. There's nothing in it that can't be read, no > > mysterious characters or anything; it's just marked as binary. All > > you have to do is to move it, then cat it back into place. I did that > > with no ill effects. > > Actually, this is what I did, as I reported here on 26/12: > > 1.Boot rescue system and mount main system > > 2.# cd /mnt/main/var/log > > 3.# mv messages messages.bin > > 4.# strings messages.bin > messages > > 5.# rm messages.bin > > 6.Reboot. How often do you have to do this? -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] syslog-ng: how to read the log files
On Wednesday 18 February 2015 07:38:46 Mick wrote: > On Tuesday 17 Feb 2015 23:13:08 Peter Humphrey wrote: > > Actually, this is what I did, as I reported here on 26/12: > > > 1.Boot rescue system and mount main system > > > 2.# cd /mnt/main/var/log > > > 3.# mv messages messages.bin > > > 4.# strings messages.bin > messages > > > 5.# rm messages.bin > > > 6.Reboot. > > How often do you have to do this? Just the once. Whatever bug caused it seems to have been fixed. -- Rgds Peter.
Re: [gentoo-user] syslog-ng: how to read the log files
On Tue, 17 February 2015, at 6:26 pm, lee wrote: > > The log file seem to be some sort of binary that doesn't display too > well in less, and there doesn't seem to be any way to read them. I believe this may be bug 406623. https://bugs.gentoo.org/show_bug.cgi?id=406623 Note comment #2 - the "binary zero characters" are not visible with every editor, but if I recollect I was able to see them and delete them when I opened the log files in vim. I think they displayed as "@^" in vim. Once you know where to look, you can also identify the binary characters using `hexdump -C /var/log/messages`. I am now running app-admin/syslog-ng-3.4.8 and have threading enabled and the problem is now no longer occurring. You can establish whether you're affected by 406623 simply by deleting the character(s) (renaming the log file would probably work, too) and rebooting the system. My experience was that the text logfile is "turned to binary" on reboot - the binary characters were logged as part of the kernel's startup messages. This was repeatable and predictable. Stroller.
Re: [gentoo-user] syslog-ng: how to read the log files
On Tue, Feb 17 2015, Canek Peláez Valdés wrote: > On Tue, Feb 17, 2015 at 2:29 PM, wrote: >> >> I wonder if the OP is using systemd and trying to read the journal >> files? > > Those live under /var/lib/journal (which you need to create; Gentoo doesn't > do it by default last time I saw), Wow! I just checked and indeed I do not have /var/lib/journal. I run systemd (thanks to canek) and use journalctl, which I *thought* was displaying the journal). Need I make some changes? thanks allan
Re: [gentoo-user] syslog-ng: how to read the log files
On Wed, Feb 18, 2015 at 12:22 PM, wrote: > > On Tue, Feb 17 2015, Canek Peláez Valdés wrote: > > > On Tue, Feb 17, 2015 at 2:29 PM, wrote: > >> > >> I wonder if the OP is using systemd and trying to read the journal > >> files? > > > > Those live under /var/lib/journal (which you need to create; Gentoo doesn't > > do it by default last time I saw), > > Wow! I just checked and indeed I do not have /var/lib/journal. > I run systemd (thanks to canek) and use journalctl, which I *thought* > was displaying the journal). The journal works without permanent storage (one more of its many advantages); in that case, it keeps a small amount of logs in memory (you can set how much memory to reserve for it). > Need I make some changes? Only if you want to have logs in permanent storage. In that case, you only need to create the /var/log/journal dir with systemd-journal GID, and 2755 permissions (with setgid). systemd-journald will automatically rotate the logs when they use 10% of the free disk available (you can also change that). Since the logs are compressed and indexed, each entry on them is accesible in O(1), and they don't use that much space (with 280 megabytes reserved in my laptop for journal logs, I have logs since Sep 20, 2014; that's 5 months worth of logs, although my laptop doesn't run that many daemons). Anyway, the journal works perfectly without permanent storage (as you can see); if you are happy that way, you don't need to enable it. Regards. -- Canek Peláez Valdés Profesor de asignatura, Facultad de Ciencias Universidad Nacional Autónoma de México
Re: [gentoo-user] syslog-ng: how to read the log files
cov...@ccs.covici.com writes: > Alexander Kapshuk wrote: > >> On Tue, Feb 17, 2015 at 8:26 PM, lee wrote: >> >> > Hi, >> > >> > how do you read the log files when using syslog-ng? >> > >> > The log file seem to be some sort of binary that doesn't display too >> > well in less, and there doesn't seem to be any way to read them. >> > >> > >> > -- >> > Again we must be afraid of speaking of daemons for fear that daemons >> > might swallow us. Finally, this fear has become reasonable. >> > >> > >> If you're talking about /var/log/messages, which is: >> messages: data >> >> I use cat(1). > > I wonder if the OP is using systemd and trying to read the journal > files? Nooo, I hate systemd ... What good are log files you can't read? If syslog-ng would actually use some binary format, I'd have switched to something else. -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
Stroller writes: > On Tue, 17 February 2015, at 6:26 pm, lee wrote: >> >> The log file seem to be some sort of binary that doesn't display too >> well in less, and there doesn't seem to be any way to read them. > > I believe this may be bug 406623. > > https://bugs.gentoo.org/show_bug.cgi?id=406623 That's almost three years old and should apparently be fixed? > You can establish whether you're affected by 406623 simply by deleting > the character(s) (renaming the log file would probably work, too) and > rebooting the system. My experience was that the text logfile is > "turned to binary" on reboot - the binary characters were logged as > part of the kernel's startup messages. This was repeatable and > predictable. Maybe I'll try it tomorrow --- it's on a server at work which I plan to reboot anyway. -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
On Wed, 18 February 2015, at 8:40 pm, lee wrote: >>> >>> The log file seem to be some sort of binary that doesn't display too >>> well in less, and there doesn't seem to be any way to read them. >> >> I believe this may be bug 406623. >> >> https://bugs.gentoo.org/show_bug.cgi?id=406623 > > That's almost three years old and should apparently be fixed? It's only been closed in the last few weeks. See for example, comment 36, November last year (i.e. 3 or 4 months old), "This isn't resolved unless commit f4ae768 is backported or >=3.5.6 is stabilised." https://bugs.gentoo.org/show_bug.cgi?id=406623#c36 Since you haven't told us what version of syslog-ng you're running, I think it's reasonable to suspect you've not updated it recently. Of course the characters could be left in your logfile from months ago, if you've not been rotating logs. If it's not that bug, though, you should prolly file a new one. Stroller.
Re: [gentoo-user] syslog-ng: how to read the log files
On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > I wonder if the OP is using systemd and trying to read the journal > > files? > > Nooo, I hate systemd ... > > What good are log files you can't read? You can't read syslog-ng log files without some reading software, usually a combination of cat, grep and less. systemd does it all with journalctl. There are good reasons to not use systemd, this isn't one of them. -- Neil Bothwick Weird enough for government work. pgpfUGXDctrCx.pgp Description: OpenPGP digital signature
Re: [gentoo-user] syslog-ng: how to read the log files
On Wed, Feb 18 2015, Canek Peláez Valdés wrote: > On Wed, Feb 18, 2015 at 12:22 PM, wrote: >> >> On Tue, Feb 17 2015, Canek Peláez Valdés wrote: >> >> > On Tue, Feb 17, 2015 at 2:29 PM, wrote: >> >> >> >> I wonder if the OP is using systemd and trying to read the journal >> >> files? >> > >> > Those live under /var/lib/journal (which you need to create; Gentoo >> > doesn't do it by default last time I saw), >> >> Wow! I just checked and indeed I do not have /var/lib/journal. >> I run systemd (thanks to canek) and use journalctl, which I *thought* >> was displaying the journal). > > The journal works without permanent storage (one more of its many > advantages); in that case, it keeps a small amount of logs in memory (you > can set how much memory to reserve for it). > >> Need I make some changes? > > Only if you want to have logs in permanent storage. In that case, you only > need to create the /var/log/journal dir with systemd-journal GID, and 2755 > permissions (with setgid). systemd-journald will automatically rotate the > logs when they use 10% of the free disk available (you can also change > that). > > Since the logs are compressed and indexed, each entry on them is accesible > in O(1), and they don't use that much space (with 280 megabytes reserved in > my laptop for journal logs, I have logs since Sep 20, 2014; that's 5 months > worth of logs, although my laptop doesn't run that many daemons). > > Anyway, the journal works perfectly without permanent storage (as you can > see); if you are happy that way, you don't need to enable it. > > Regards. > -- > Canek Peláez Valdés Thank you for another lucid explanation. allan
Re: [gentoo-user] syslog-ng: how to read the log files
On Tuesday, February 17, 2015 7:26:05 PM lee wrote: > Hi, > > how do you read the log files when using syslog-ng? > > The log file seem to be some sort of binary that doesn't display too > well in less, and there doesn't seem to be any way to read them. > > > You can just pipe the output of strings /var/log/messages to less. You can use strings(1) for systemd journal files also. -- Fernando Rodriguez signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] syslog-ng: how to read the log files
On 02/19/2015 08:02 PM, Fernando Rodriguez wrote: > On Tuesday, February 17, 2015 7:26:05 PM lee wrote: >> Hi, >> >> how do you read the log files when using syslog-ng? >> >> The log file seem to be some sort of binary that doesn't display too >> well in less, and there doesn't seem to be any way to read them. >> >> >> > > You can just pipe the output of strings /var/log/messages to less. You can use > strings(1) for systemd journal files also. Yeah and you can check whether it contains any binary data by diff <(strings /var/log/messages) /var/log/messages -- Jan Sever
Re: [gentoo-user] syslog-ng: how to read the log files
Stroller writes: > On Wed, 18 February 2015, at 8:40 pm, lee wrote: The log file seem to be some sort of binary that doesn't display too well in less, and there doesn't seem to be any way to read them. >>> >>> I believe this may be bug 406623. >>> >>> https://bugs.gentoo.org/show_bug.cgi?id=406623 >> >> That's almost three years old and should apparently be fixed? > > It's only been closed in the last few weeks. Still I wonder why it took so long to fix it. > See for example, comment 36, November last year (i.e. 3 or 4 months old), > "This isn't resolved unless commit f4ae768 is backported or >=3.5.6 is > stabilised." > > https://bugs.gentoo.org/show_bug.cgi?id=406623#c36 > > Since you haven't told us what version of syslog-ng you're running, I think > it's reasonable to suspect you've not updated it recently. The server was installed the week before the last, starting with the latest live DVD. It has been updated. I can't tell what version it is because it's at work. > Of course the characters could be left in your logfile from months ago, if > you've not been rotating logs. > > If it's not that bug, though, you should prolly file a new one. Dunno, I've edited the file and removed the null characters. Time will tell whether new ones will be logged or not. IIUC, syslog-ng handles rotating the logs. Do I need to do something to make it rotate them? -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
"Jan Sever" writes: > On 02/19/2015 08:02 PM, Fernando Rodriguez wrote: >> On Tuesday, February 17, 2015 7:26:05 PM lee wrote: >>> Hi, >>> >>> how do you read the log files when using syslog-ng? >>> >>> The log file seem to be some sort of binary that doesn't display too >>> well in less, and there doesn't seem to be any way to read them. >>> >>> >>> >> >> You can just pipe the output of strings /var/log/messages to less. You can > use >> strings(1) for systemd journal files also. > > Yeah and you can check whether it contains any binary data by > diff <(strings /var/log/messages) /var/log/messages Well, yes; see it this way: I never used syslog-ng before. When I had to fix some problems, I wanted to read the log files. Less showed them in some hexl-mode, and it has never done anything like that before. I didn't even know that it has this hexl-mode. So I googled for how to read those log files because I figured that syslog-ng perhaps uses some stupid binary format and that there might be some program you're supposed to read them with. That didn't turn up anything and it really sucked. How am I supposed to know that there's a combination of a three-year-old bug and totally unexpected behaviour of less preventing me from reading these logs? Try something like 'less /boot/vmlinuz-3.17.8-gentoo-r1' and you don't get hexl-mode. Why such a surprise when trying to read a log file? -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
Neil Bothwick writes: > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > >> > I wonder if the OP is using systemd and trying to read the journal >> > files? >> >> Nooo, I hate systemd ... >> >> What good are log files you can't read? > > You can't read syslog-ng log files without some reading software, usually > a combination of cat, grep and less. systemd does it all with journalctl. > > There are good reasons to not use systemd, this isn't one of them. To me it is one of the good reasons, and an important one. Plain text can usually always be read without further ado, be it from rescue systems you booted or with software available on different operating systems. It can be also be processed with scripts and sent as email. You can probably even read it on your cell phone. You can still read log files that were created 20 years ago when they are plain text. Can you do all that with the binary files created by systemd? I can't even read them on a working system. -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
On Sun, Feb 22, 2015 at 6:41 PM, lee wrote: > > Neil Bothwick writes: > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > >> > I wonder if the OP is using systemd and trying to read the journal > >> > files? > >> > >> Nooo, I hate systemd ... > >> > >> What good are log files you can't read? > > > > You can't read syslog-ng log files without some reading software, usually > > a combination of cat, grep and less. systemd does it all with journalctl. > > > > There are good reasons to not use systemd, this isn't one of them. > > To me it is one of the good reasons, and an important one. Plain text > can usually always be read without further ado, be it from rescue > systems you booted or with software available on different operating > systems. It can be also be processed with scripts and sent as email. > You can probably even read it on your cell phone. You can still read > log files that were created 20 years ago when they are plain text. > > Can you do all that with the binary files created by systemd? Yes, you can. > I can't even read them on a working system. If that's true (which I highly doubt, more probably you don't know how to read them), then it's a bug and should be reported and fixed. Regards. -- Canek Peláez Valdés Profesor de asignatura, Facultad de Ciencias Universidad Nacional Autónoma de México
Re: [gentoo-user] syslog-ng: how to read the log files
On Sun, Feb 22, 2015 at 6:48 PM, lee wrote: > > Stroller writes: > > > On Wed, 18 February 2015, at 8:40 pm, lee wrote: > > The log file seem to be some sort of binary that doesn't display too > well in less, and there doesn't seem to be any way to read them. > >>> > >>> I believe this may be bug 406623. > >>> > >>> https://bugs.gentoo.org/show_bug.cgi?id=406623 > >> > >> That's almost three years old and should apparently be fixed? > > > > It's only been closed in the last few weeks. > > Still I wonder why it took so long to fix it. > > > See for example, comment 36, November last year (i.e. 3 or 4 months old), "This isn't resolved unless commit f4ae768 is backported or >=3.5.6 is stabilised." > > > > https://bugs.gentoo.org/show_bug.cgi?id=406623#c36 > > > > Since you haven't told us what version of syslog-ng you're running, I think it's reasonable to suspect you've not updated it recently. > > The server was installed the week before the last, starting with the > latest live DVD. It has been updated. I can't tell what version it is > because it's at work. > > > Of course the characters could be left in your logfile from months ago, if you've not been rotating logs. > > > > If it's not that bug, though, you should prolly file a new one. > > Dunno, I've edited the file and removed the null characters. Time will > tell whether new ones will be logged or not. > > > IIUC, syslog-ng handles rotating the logs. Do I need to do something to > make it rotate them? syslog-ng, as long as I remember, has never rotated its logs. You need logrotate (or something similar) to do it for you. Regards. -- Canek Peláez Valdés Profesor de asignatura, Facultad de Ciencias Universidad Nacional Autónoma de México
Re: [gentoo-user] syslog-ng: how to read the log files
On Sun, Feb 22, 2015 at 6:41 PM, lee wrote: > > To me it is one of the good reasons, and an important one. Plain text > can usually always be read without further ado, be it from rescue > systems you booted or with software available on different operating > systems. It can be also be processed with scripts and sent as email. > You can probably even read it on your cell phone. You can still read > log files that were created 20 years ago when they are plain text. Doing any of that stuff requires the use of software capable of reading text files. It isn't like you can just interpret the magnetic fields on your disk with your eyes. Sure, there are a lot more utilities that can read text files than journal files, but you just need to arrange to have them handy. They'll be ubiquitous before long since every distro around will end up needing them. > > Can you do all that with the binary files created by systemd? I can't > even read them on a working system. > You just type journalctl to read the live system logs. For offline use you just type journalctl --file=filename. Or you can just run strings on the file I imagine if you're desperate. If it doesn't work on a "working system" then your system isn't working. -- Rich
Re: [gentoo-user] syslog-ng: how to read the log files
lee wrote: > IIUC, syslog-ng handles rotating the logs. Do I need to do something > to make it rotate them? I think you need this: app-admin/logrotate Then I think a cron package is needed to run that, set to daily here I think. Hope that helps. Dale :-) :-)
Re: [gentoo-user] syslog-ng: how to read the log files
On Sunday 22 February 2015 20:57:43 Dale wrote: > I think you need this: > > app-admin/logrotate > > Then I think a cron package is needed to run that, set to daily here I > think. It comes with logrotate: /etc/cron.daily/logrotate -- Rgds Peter.
Re: [gentoo-user] syslog-ng: how to read the log files
Peter Humphrey wrote: > On Sunday 22 February 2015 20:57:43 Dale wrote: >> I think you need this: >> >> app-admin/logrotate >> >> Then I think a cron package is needed to run that, set to daily here I >> think. > It comes with logrotate: > > /etc/cron.daily/logrotate > The script does but if you don't have a cron package installed, nothing will run to rotate the logs. Maybe my message wasn't worded correctly? It's been a long week. ;-) Dale :-) :-)
Re: [gentoo-user] syslog-ng: how to read the log files
Am Mon, 23 Feb 2015 00:41:50 +0100 schrieb lee : > Neil Bothwick writes: > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > >> > I wonder if the OP is using systemd and trying to read the journal > >> > files? > >> > >> Nooo, I hate systemd ... > >> > >> What good are log files you can't read? > > > > You can't read syslog-ng log files without some reading software, usually > > a combination of cat, grep and less. systemd does it all with journalctl. > > > > There are good reasons to not use systemd, this isn't one of them. > > To me it is one of the good reasons, and an important one. Plain text > can usually always be read without further ado, be it from rescue > systems you booted or with software available on different operating > systems. It can be also be processed with scripts and sent as email. > You can probably even read it on your cell phone. You can still read > log files that were created 20 years ago when they are plain text. > > Can you do all that with the binary files created by systemd? I can't > even read them on a working system. What Canek and Rich already said is good, but I'll just add this: it's not like you can't run a classic syslog implementation alongside the systemd journal. On my systems, by *default*, syslog-ng kept working as usual, getting the logs from the systemd journal. If you want to go further, you can even configure the journal to not store logs permanently, so that you *only* end up with plain-text logs on your system (Duncan on gentoo-amd64 went this way). So no, the format that the systemd journal uses is most decidedly *not* a reason against using systemd. Personally, I'm probably going to uninstall syslog-ng, because journalctl is *such* a nice way to read logs, so why run something whose output I'll never read again? I recommend reading http://0pointer.net/blog/projects/journalctl.html for examples of the kind of stuff you can do that would be cumbersome, if not *impossible* with regular syslog. HTH -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup pgp64Eza5OoST.pgp Description: Digitale Signatur von OpenPGP
Re: [gentoo-user] syslog-ng: how to read the log files
Marc Joliet wrote: > Am Mon, 23 Feb 2015 00:41:50 +0100 > schrieb lee : > > > Neil Bothwick writes: > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > > > >> > I wonder if the OP is using systemd and trying to read the journal > > >> > files? > > >> > > >> Nooo, I hate systemd ... > > >> > > >> What good are log files you can't read? > > > > > > You can't read syslog-ng log files without some reading software, usually > > > a combination of cat, grep and less. systemd does it all with journalctl. > > > > > > There are good reasons to not use systemd, this isn't one of them. > > > > To me it is one of the good reasons, and an important one. Plain text > > can usually always be read without further ado, be it from rescue > > systems you booted or with software available on different operating > > systems. It can be also be processed with scripts and sent as email. > > You can probably even read it on your cell phone. You can still read > > log files that were created 20 years ago when they are plain text. > > > > Can you do all that with the binary files created by systemd? I can't > > even read them on a working system. > > What Canek and Rich already said is good, but I'll just add this: it's not > like > you can't run a classic syslog implementation alongside the systemd journal. > On my systems, by *default*, syslog-ng kept working as usual, getting the logs > from the systemd journal. If you want to go further, you can even configure > the journal to not store logs permanently, so that you *only* end up with > plain-text logs on your system (Duncan on gentoo-amd64 went this way). > > So no, the format that the systemd journal uses is most decidedly *not* a > reason > against using systemd. > > Personally, I'm probably going to uninstall syslog-ng, because journalctl is > *such* a nice way to read logs, so why run something whose output I'll never > read again? I recommend reading > http://0pointer.net/blog/projects/journalctl.html for examples of the kind of > stuff you can do that would be cumbersome, if not *impossible* with regular > syslog. Except that I get lots of messages about the system journal missing messages when forwarding to syslog, so how can I make sure this does not happening? -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] syslog-ng: how to read the log files
On Sunday 22 February 2015 22:28:07 Dale wrote: > Peter Humphrey wrote: > > On Sunday 22 February 2015 20:57:43 Dale wrote: > >> I think you need this: > >> > >> app-admin/logrotate > >> > >> Then I think a cron package is needed to run that, set to daily > >> here I think. > > > > It comes with logrotate: > > > > /etc/cron.daily/logrotate > > The script does but if you don't have a cron package installed, > nothing will run to rotate the logs. Maybe my message wasn't worded > correctly? It's been a long week. ;-) Ah, I see what you mean. My misread. -- Rgds Peter.
Re: [gentoo-user] syslog-ng: how to read the log files
On Mon, Feb 23, 2015 at 3:41 AM, wrote: > > Marc Joliet wrote: > > > Am Mon, 23 Feb 2015 00:41:50 +0100 > > schrieb lee : > > > > > Neil Bothwick writes: > > > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > > > > > >> > I wonder if the OP is using systemd and trying to read the journal > > > >> > files? > > > >> > > > >> Nooo, I hate systemd ... > > > >> > > > >> What good are log files you can't read? > > > > > > > > You can't read syslog-ng log files without some reading software, usually > > > > a combination of cat, grep and less. systemd does it all with journalctl. > > > > > > > > There are good reasons to not use systemd, this isn't one of them. > > > > > > To me it is one of the good reasons, and an important one. Plain text > > > can usually always be read without further ado, be it from rescue > > > systems you booted or with software available on different operating > > > systems. It can be also be processed with scripts and sent as email. > > > You can probably even read it on your cell phone. You can still read > > > log files that were created 20 years ago when they are plain text. > > > > > > Can you do all that with the binary files created by systemd? I can't > > > even read them on a working system. > > > > What Canek and Rich already said is good, but I'll just add this: it's not like > > you can't run a classic syslog implementation alongside the systemd journal. > > On my systems, by *default*, syslog-ng kept working as usual, getting the logs > > from the systemd journal. If you want to go further, you can even configure > > the journal to not store logs permanently, so that you *only* end up with > > plain-text logs on your system (Duncan on gentoo-amd64 went this way). > > > > So no, the format that the systemd journal uses is most decidedly *not* a reason > > against using systemd. > > > > Personally, I'm probably going to uninstall syslog-ng, because journalctl is > > *such* a nice way to read logs, so why run something whose output I'll never > > read again? I recommend reading > > http://0pointer.net/blog/projects/journalctl.html for examples of the kind of > > stuff you can do that would be cumbersome, if not *impossible* with regular > > syslog. > > Except that I get lots of messages about the system journal missing > messages when forwarding to syslog, so how can I make sure this does not > happening? Could you please show those messages? systemd sends *everything* to the journal, and then the journal (optionally) can send it too to a regular syslog. In that sense, it's impossible for the journal to miss any message. The only way in which the journal could miss messages is at very early boot stages; but with a proper initramfs (like the ones generated with dracut), even those get caught. You get to put an instance of systemd and the journal inside the initramfs, and so it's available almost from the beginning. And if you use gummiboot, then you can even log from the moment the UEFI firmware comes to life. Regards. -- Canek Peláez Valdés Profesor de asignatura, Facultad de Ciencias Universidad Nacional Autónoma de México
Re: [gentoo-user] syslog-ng: how to read the log files
Canek Peláez Valdés wrote: > On Mon, Feb 23, 2015 at 3:41 AM, wrote: > > > > Marc Joliet wrote: > > > > > Am Mon, 23 Feb 2015 00:41:50 +0100 > > > schrieb lee : > > > > > > > Neil Bothwick writes: > > > > > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > > > > > > > >> > I wonder if the OP is using systemd and trying to read the > journal > > > > >> > files? > > > > >> > > > > >> Nooo, I hate systemd ... > > > > >> > > > > >> What good are log files you can't read? > > > > > > > > > > You can't read syslog-ng log files without some reading software, > usually > > > > > a combination of cat, grep and less. systemd does it all with > journalctl. > > > > > > > > > > There are good reasons to not use systemd, this isn't one of them. > > > > > > > > To me it is one of the good reasons, and an important one. Plain text > > > > can usually always be read without further ado, be it from rescue > > > > systems you booted or with software available on different operating > > > > systems. It can be also be processed with scripts and sent as email. > > > > You can probably even read it on your cell phone. You can still read > > > > log files that were created 20 years ago when they are plain text. > > > > > > > > Can you do all that with the binary files created by systemd? I can't > > > > even read them on a working system. > > > > > > What Canek and Rich already said is good, but I'll just add this: it's > not like > > > you can't run a classic syslog implementation alongside the systemd > journal. > > > On my systems, by *default*, syslog-ng kept working as usual, getting > the logs > > > from the systemd journal. If you want to go further, you can even > configure > > > the journal to not store logs permanently, so that you *only* end up > with > > > plain-text logs on your system (Duncan on gentoo-amd64 went this way). > > > > > > So no, the format that the systemd journal uses is most decidedly *not* > a reason > > > against using systemd. > > > > > > Personally, I'm probably going to uninstall syslog-ng, because > journalctl is > > > *such* a nice way to read logs, so why run something whose output I'll > never > > > read again? I recommend reading > > > http://0pointer.net/blog/projects/journalctl.html for examples of the > kind of > > > stuff you can do that would be cumbersome, if not *impossible* with > regular > > > syslog. > > > > Except that I get lots of messages about the system journal missing > > messages when forwarding to syslog, so how can I make sure this does not > > happening? > > Could you please show those messages? systemd sends *everything* to the > journal, and then the journal (optionally) can send it too to a regular > syslog. In that sense, it's impossible for the journal to miss any message. > > The only way in which the journal could miss messages is at very early boot > stages; but with a proper initramfs (like the ones generated with dracut), > even those get caught. You get to put an instance of systemd and the > journal inside the initramfs, and so it's available almost from the > beginning. > > And if you use gummiboot, then you can even log from the moment the UEFI > firmware comes to life. So, I get lots of messages in my regular syslog-ng /var/log/messages like the following: Feb 23 12:47:52 ccs.covici.com systemd-journal[715]: Forwarding to syslog missed 15 messages. So, I saw a post on Google to up the queue length, and I uped it to 200, but no joy, still get the messages like the one above. -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] syslog-ng: how to read the log files
On Mon, Feb 23, 2015 at 11:49 AM, wrote: > > Canek Peláez Valdés wrote: > > > On Mon, Feb 23, 2015 at 3:41 AM, wrote: > > > > > > Marc Joliet wrote: > > > > > > > Am Mon, 23 Feb 2015 00:41:50 +0100 > > > > schrieb lee : > > > > > > > > > Neil Bothwick writes: > > > > > > > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > > > > > > > > > >> > I wonder if the OP is using systemd and trying to read the > > journal > > > > > >> > files? > > > > > >> > > > > > >> Nooo, I hate systemd ... > > > > > >> > > > > > >> What good are log files you can't read? > > > > > > > > > > > > You can't read syslog-ng log files without some reading software, > > usually > > > > > > a combination of cat, grep and less. systemd does it all with > > journalctl. > > > > > > > > > > > > There are good reasons to not use systemd, this isn't one of them. > > > > > > > > > > To me it is one of the good reasons, and an important one. Plain text > > > > > can usually always be read without further ado, be it from rescue > > > > > systems you booted or with software available on different operating > > > > > systems. It can be also be processed with scripts and sent as email. > > > > > You can probably even read it on your cell phone. You can still read > > > > > log files that were created 20 years ago when they are plain text. > > > > > > > > > > Can you do all that with the binary files created by systemd? I can't > > > > > even read them on a working system. > > > > > > > > What Canek and Rich already said is good, but I'll just add this: it's > > not like > > > > you can't run a classic syslog implementation alongside the systemd > > journal. > > > > On my systems, by *default*, syslog-ng kept working as usual, getting > > the logs > > > > from the systemd journal. If you want to go further, you can even > > configure > > > > the journal to not store logs permanently, so that you *only* end up > > with > > > > plain-text logs on your system (Duncan on gentoo-amd64 went this way). > > > > > > > > So no, the format that the systemd journal uses is most decidedly *not* > > a reason > > > > against using systemd. > > > > > > > > Personally, I'm probably going to uninstall syslog-ng, because > > journalctl is > > > > *such* a nice way to read logs, so why run something whose output I'll > > never > > > > read again? I recommend reading > > > > http://0pointer.net/blog/projects/journalctl.html for examples of the > > kind of > > > > stuff you can do that would be cumbersome, if not *impossible* with > > regular > > > > syslog. > > > > > > Except that I get lots of messages about the system journal missing > > > messages when forwarding to syslog, so how can I make sure this does not > > > happening? > > > > Could you please show those messages? systemd sends *everything* to the > > journal, and then the journal (optionally) can send it too to a regular > > syslog. In that sense, it's impossible for the journal to miss any message. > > > > The only way in which the journal could miss messages is at very early boot > > stages; but with a proper initramfs (like the ones generated with dracut), > > even those get caught. You get to put an instance of systemd and the > > journal inside the initramfs, and so it's available almost from the > > beginning. > > > > And if you use gummiboot, then you can even log from the moment the UEFI > > firmware comes to life. > > So, I get lots of messages in my regular syslog-ng /var/log/messages > like the following: > Feb 23 12:47:52 ccs.covici.com systemd-journal[715]: Forwarding to > syslog missed 15 messages. > > So, I saw a post on Google to up the queue length, and I uped it to 200, > but no joy, still get the messages like the one above. Are you using the unit file provided by syslog-ng (systemd-delta doesn't mention syslog)? Also, is /etc/systemd/system/syslog.service is a link to /usr/lib/systemd/system/syslog-ng.service? I do, and I don't get any of those messages. I use the default journal configuration. According to [1], this should be fixed. Regards. https://github.com/balabit/syslog-ng/issues/314 -- Canek Peláez Valdés Profesor de asignatura, Facultad de Ciencias Universidad Nacional Autónoma de México
Re: [gentoo-user] syslog-ng: how to read the log files
Am Mon, 23 Feb 2015 12:10:18 -0600
schrieb Canek Peláez Valdés :
> On Mon, Feb 23, 2015 at 11:49 AM, wrote:
> >
> > Canek Peláez Valdés wrote:
> >
> > > On Mon, Feb 23, 2015 at 3:41 AM, wrote:
> > > >
> > > > Marc Joliet wrote:
> > > >
> > > > > Am Mon, 23 Feb 2015 00:41:50 +0100
> > > > > schrieb lee :
> > > > >
> > > > > > Neil Bothwick writes:
> > > > > >
> > > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote:
> > > > > > >
> > > > > > >> > I wonder if the OP is using systemd and trying to read the
> > > journal
> > > > > > >> > files?
> > > > > > >>
> > > > > > >> Nooo, I hate systemd ...
> > > > > > >>
> > > > > > >> What good are log files you can't read?
> > > > > > >
> > > > > > > You can't read syslog-ng log files without some reading
> software,
> > > usually
> > > > > > > a combination of cat, grep and less. systemd does it all with
> > > journalctl.
> > > > > > >
> > > > > > > There are good reasons to not use systemd, this isn't one of
> them.
> > > > > >
> > > > > > To me it is one of the good reasons, and an important one. Plain
> text
> > > > > > can usually always be read without further ado, be it from rescue
> > > > > > systems you booted or with software available on different
> operating
> > > > > > systems. It can be also be processed with scripts and sent as
> email.
> > > > > > You can probably even read it on your cell phone. You can still
> read
> > > > > > log files that were created 20 years ago when they are plain text.
> > > > > >
> > > > > > Can you do all that with the binary files created by systemd? I
> can't
> > > > > > even read them on a working system.
> > > > >
> > > > > What Canek and Rich already said is good, but I'll just add this:
> it's
> > > not like
> > > > > you can't run a classic syslog implementation alongside the systemd
> > > journal.
> > > > > On my systems, by *default*, syslog-ng kept working as usual,
> getting
> > > the logs
> > > > > from the systemd journal. If you want to go further, you can even
> > > configure
> > > > > the journal to not store logs permanently, so that you *only* end up
> > > with
> > > > > plain-text logs on your system (Duncan on gentoo-amd64 went this
> way).
> > > > >
> > > > > So no, the format that the systemd journal uses is most decidedly
> *not*
> > > a reason
> > > > > against using systemd.
> > > > >
> > > > > Personally, I'm probably going to uninstall syslog-ng, because
> > > journalctl is
> > > > > *such* a nice way to read logs, so why run something whose output
> I'll
> > > never
> > > > > read again? I recommend reading
> > > > > http://0pointer.net/blog/projects/journalctl.html for examples of
> the
> > > kind of
> > > > > stuff you can do that would be cumbersome, if not *impossible* with
> > > regular
> > > > > syslog.
> > > >
> > > > Except that I get lots of messages about the system journal missing
> > > > messages when forwarding to syslog, so how can I make sure this does
> not
> > > > happening?
> > >
> > > Could you please show those messages? systemd sends *everything* to the
> > > journal, and then the journal (optionally) can send it too to a regular
> > > syslog. In that sense, it's impossible for the journal to miss any
> message.
> > >
> > > The only way in which the journal could miss messages is at very early
> boot
> > > stages; but with a proper initramfs (like the ones generated with
> dracut),
> > > even those get caught. You get to put an instance of systemd and the
> > > journal inside the initramfs, and so it's available almost from the
> > > beginning.
> > >
> > > And if you use gummiboot, then you can even log from the moment the UEFI
> > > firmware comes to life.
> >
> > So, I get lots of messages in my regular syslog-ng /var/log/messages
> > like the following:
> > Feb 23 12:47:52 ccs.covici.com systemd-journal[715]: Forwarding to
> > syslog missed 15 messages.
> >
> > So, I saw a post on Google to up the queue length, and I uped it to 200,
> > but no joy, still get the messages like the one above.
>
> Are you using the unit file provided by syslog-ng (systemd-delta doesn't
> mention syslog)? Also, is /etc/systemd/system/syslog.service is a link
> to /usr/lib/systemd/system/syslog-ng.service?
>
> I do, and I don't get any of those messages. I use the default journal
> configuration. According to [1], this should be fixed.
I remember getting a small number of messages like that, too, on my laptop.
However, it's at the university, so I can't check now to see what types of
messages were missed (if any; if I understand [1] correctly, those messages are
most likely bogus?).
But yeah, that's any idea, Covici: see what's in /var/log/messages, compare that
to the journalctl output, and check if any messages were actually missed ("diff
-U" might be of help here). And if/once you did that, what kinds of messages
were missed, if any? If those messages really are bogus, you shouldn't see any
differences between the two.
> Regards.
>
> https://github.com/balabit/syslog-
Re: [gentoo-user] syslog-ng: how to read the log files
Canek Peláez Valdés wrote: > On Mon, Feb 23, 2015 at 11:49 AM, wrote: > > > > Canek Peláez Valdés wrote: > > > > > On Mon, Feb 23, 2015 at 3:41 AM, wrote: > > > > > > > > Marc Joliet wrote: > > > > > > > > > Am Mon, 23 Feb 2015 00:41:50 +0100 > > > > > schrieb lee : > > > > > > > > > > > Neil Bothwick writes: > > > > > > > > > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > > > > > > > > > > > >> > I wonder if the OP is using systemd and trying to read the > > > journal > > > > > > >> > files? > > > > > > >> > > > > > > >> Nooo, I hate systemd ... > > > > > > >> > > > > > > >> What good are log files you can't read? > > > > > > > > > > > > > > You can't read syslog-ng log files without some reading > software, > > > usually > > > > > > > a combination of cat, grep and less. systemd does it all with > > > journalctl. > > > > > > > > > > > > > > There are good reasons to not use systemd, this isn't one of > them. > > > > > > > > > > > > To me it is one of the good reasons, and an important one. Plain > text > > > > > > can usually always be read without further ado, be it from rescue > > > > > > systems you booted or with software available on different > operating > > > > > > systems. It can be also be processed with scripts and sent as > email. > > > > > > You can probably even read it on your cell phone. You can still > read > > > > > > log files that were created 20 years ago when they are plain text. > > > > > > > > > > > > Can you do all that with the binary files created by systemd? I > can't > > > > > > even read them on a working system. > > > > > > > > > > What Canek and Rich already said is good, but I'll just add this: > it's > > > not like > > > > > you can't run a classic syslog implementation alongside the systemd > > > journal. > > > > > On my systems, by *default*, syslog-ng kept working as usual, > getting > > > the logs > > > > > from the systemd journal. If you want to go further, you can even > > > configure > > > > > the journal to not store logs permanently, so that you *only* end up > > > with > > > > > plain-text logs on your system (Duncan on gentoo-amd64 went this > way). > > > > > > > > > > So no, the format that the systemd journal uses is most decidedly > *not* > > > a reason > > > > > against using systemd. > > > > > > > > > > Personally, I'm probably going to uninstall syslog-ng, because > > > journalctl is > > > > > *such* a nice way to read logs, so why run something whose output > I'll > > > never > > > > > read again? I recommend reading > > > > > http://0pointer.net/blog/projects/journalctl.html for examples of > the > > > kind of > > > > > stuff you can do that would be cumbersome, if not *impossible* with > > > regular > > > > > syslog. > > > > > > > > Except that I get lots of messages about the system journal missing > > > > messages when forwarding to syslog, so how can I make sure this does > not > > > > happening? > > > > > > Could you please show those messages? systemd sends *everything* to the > > > journal, and then the journal (optionally) can send it too to a regular > > > syslog. In that sense, it's impossible for the journal to miss any > message. > > > > > > The only way in which the journal could miss messages is at very early > boot > > > stages; but with a proper initramfs (like the ones generated with > dracut), > > > even those get caught. You get to put an instance of systemd and the > > > journal inside the initramfs, and so it's available almost from the > > > beginning. > > > > > > And if you use gummiboot, then you can even log from the moment the UEFI > > > firmware comes to life. > > > > So, I get lots of messages in my regular syslog-ng /var/log/messages > > like the following: > > Feb 23 12:47:52 ccs.covici.com systemd-journal[715]: Forwarding to > > syslog missed 15 messages. > > > > So, I saw a post on Google to up the queue length, and I uped it to 200, > > but no joy, still get the messages like the one above. > > Are you using the unit file provided by syslog-ng (systemd-delta doesn't > mention syslog)? Also, is /etc/systemd/system/syslog.service is a link > to /usr/lib/systemd/system/syslog-ng.service? > > I do, and I don't get any of those messages. I use the default journal > configuration. According to [1], this should be fixed. > > Regards. > > https://github.com/balabit/syslog-ng/issues/314 At the time when I did this there was no syslog-ng.service in /usr/lib/systemd/system, now there is, but my unit file is like this: [Unit] Description=System Logger Daemon Documentation=man:syslog-ng(8) [Service] Sockets=syslog.socket ExecStart=/usr/sbin/syslog-ng -F ExecReload=/bin/kill -HUP $MAINPID #Restart=on-failure [Install] WantedBy=multi-user.target Alias=syslog.service Is there a reason why this should not work? -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] syslog-ng: how to read the log files
Marc Joliet wrote: > Am Mon, 23 Feb 2015 12:10:18 -0600 > schrieb Canek Peláez Valdés : > > > On Mon, Feb 23, 2015 at 11:49 AM, wrote: > > > > > > Canek Peláez Valdés wrote: > > > > > > > On Mon, Feb 23, 2015 at 3:41 AM, wrote: > > > > > > > > > > Marc Joliet wrote: > > > > > > > > > > > Am Mon, 23 Feb 2015 00:41:50 +0100 > > > > > > schrieb lee : > > > > > > > > > > > > > Neil Bothwick writes: > > > > > > > > > > > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > > > > > > > > > > > > > >> > I wonder if the OP is using systemd and trying to read the > > > > journal > > > > > > > >> > files? > > > > > > > >> > > > > > > > >> Nooo, I hate systemd ... > > > > > > > >> > > > > > > > >> What good are log files you can't read? > > > > > > > > > > > > > > > > You can't read syslog-ng log files without some reading > > software, > > > > usually > > > > > > > > a combination of cat, grep and less. systemd does it all with > > > > journalctl. > > > > > > > > > > > > > > > > There are good reasons to not use systemd, this isn't one of > > them. > > > > > > > > > > > > > > To me it is one of the good reasons, and an important one. Plain > > text > > > > > > > can usually always be read without further ado, be it from rescue > > > > > > > systems you booted or with software available on different > > operating > > > > > > > systems. It can be also be processed with scripts and sent as > > email. > > > > > > > You can probably even read it on your cell phone. You can still > > read > > > > > > > log files that were created 20 years ago when they are plain text. > > > > > > > > > > > > > > Can you do all that with the binary files created by systemd? I > > can't > > > > > > > even read them on a working system. > > > > > > > > > > > > What Canek and Rich already said is good, but I'll just add this: > > it's > > > > not like > > > > > > you can't run a classic syslog implementation alongside the systemd > > > > journal. > > > > > > On my systems, by *default*, syslog-ng kept working as usual, > > getting > > > > the logs > > > > > > from the systemd journal. If you want to go further, you can even > > > > configure > > > > > > the journal to not store logs permanently, so that you *only* end up > > > > with > > > > > > plain-text logs on your system (Duncan on gentoo-amd64 went this > > way). > > > > > > > > > > > > So no, the format that the systemd journal uses is most decidedly > > *not* > > > > a reason > > > > > > against using systemd. > > > > > > > > > > > > Personally, I'm probably going to uninstall syslog-ng, because > > > > journalctl is > > > > > > *such* a nice way to read logs, so why run something whose output > > I'll > > > > never > > > > > > read again? I recommend reading > > > > > > http://0pointer.net/blog/projects/journalctl.html for examples of > > the > > > > kind of > > > > > > stuff you can do that would be cumbersome, if not *impossible* with > > > > regular > > > > > > syslog. > > > > > > > > > > Except that I get lots of messages about the system journal missing > > > > > messages when forwarding to syslog, so how can I make sure this does > > not > > > > > happening? > > > > > > > > Could you please show those messages? systemd sends *everything* to the > > > > journal, and then the journal (optionally) can send it too to a regular > > > > syslog. In that sense, it's impossible for the journal to miss any > > message. > > > > > > > > The only way in which the journal could miss messages is at very early > > boot > > > > stages; but with a proper initramfs (like the ones generated with > > dracut), > > > > even those get caught. You get to put an instance of systemd and the > > > > journal inside the initramfs, and so it's available almost from the > > > > beginning. > > > > > > > > And if you use gummiboot, then you can even log from the moment the UEFI > > > > firmware comes to life. > > > > > > So, I get lots of messages in my regular syslog-ng /var/log/messages > > > like the following: > > > Feb 23 12:47:52 ccs.covici.com systemd-journal[715]: Forwarding to > > > syslog missed 15 messages. > > > > > > So, I saw a post on Google to up the queue length, and I uped it to 200, > > > but no joy, still get the messages like the one above. > > > > Are you using the unit file provided by syslog-ng (systemd-delta doesn't > > mention syslog)? Also, is /etc/systemd/system/syslog.service is a link > > to /usr/lib/systemd/system/syslog-ng.service? > > > > I do, and I don't get any of those messages. I use the default journal > > configuration. According to [1], this should be fixed. > > I remember getting a small number of messages like that, too, on my laptop. > However, it's at the university, so I can't check now to see what types of > messages were missed (if any; if I understand [1] correctly, those messages > are > most likely bogus?). > > But yeah, that's any idea, Covici: see what's in /var/log/messages, compare > that > to the journalctl output, and check if
Re: [gentoo-user] syslog-ng: how to read the log files
On Mon, Feb 23, 2015 at 1:31 PM, wrote: > > Marc Joliet wrote: > > > Am Mon, 23 Feb 2015 12:10:18 -0600 > > schrieb Canek Peláez Valdés : > > > > > On Mon, Feb 23, 2015 at 11:49 AM, wrote: > > > > > > > > Canek Peláez Valdés wrote: > > > > > > > > > On Mon, Feb 23, 2015 at 3:41 AM, wrote: > > > > > > > > > > > > Marc Joliet wrote: > > > > > > > > > > > > > Am Mon, 23 Feb 2015 00:41:50 +0100 > > > > > > > schrieb lee : > > > > > > > > > > > > > > > Neil Bothwick writes: > > > > > > > > > > > > > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > > > > > > > > > > > > > > > >> > I wonder if the OP is using systemd and trying to read the > > > > > journal > > > > > > > > >> > files? > > > > > > > > >> > > > > > > > > >> Nooo, I hate systemd ... > > > > > > > > >> > > > > > > > > >> What good are log files you can't read? > > > > > > > > > > > > > > > > > > You can't read syslog-ng log files without some reading > > > software, > > > > > usually > > > > > > > > > a combination of cat, grep and less. systemd does it all with > > > > > journalctl. > > > > > > > > > > > > > > > > > > There are good reasons to not use systemd, this isn't one of > > > them. > > > > > > > > > > > > > > > > To me it is one of the good reasons, and an important one. Plain > > > text > > > > > > > > can usually always be read without further ado, be it from rescue > > > > > > > > systems you booted or with software available on different > > > operating > > > > > > > > systems. It can be also be processed with scripts and sent as > > > email. > > > > > > > > You can probably even read it on your cell phone. You can still > > > read > > > > > > > > log files that were created 20 years ago when they are plain text. > > > > > > > > > > > > > > > > Can you do all that with the binary files created by systemd? I > > > can't > > > > > > > > even read them on a working system. > > > > > > > > > > > > > > What Canek and Rich already said is good, but I'll just add this: > > > it's > > > > > not like > > > > > > > you can't run a classic syslog implementation alongside the systemd > > > > > journal. > > > > > > > On my systems, by *default*, syslog-ng kept working as usual, > > > getting > > > > > the logs > > > > > > > from the systemd journal. If you want to go further, you can even > > > > > configure > > > > > > > the journal to not store logs permanently, so that you *only* end up > > > > > with > > > > > > > plain-text logs on your system (Duncan on gentoo-amd64 went this > > > way). > > > > > > > > > > > > > > So no, the format that the systemd journal uses is most decidedly > > > *not* > > > > > a reason > > > > > > > against using systemd. > > > > > > > > > > > > > > Personally, I'm probably going to uninstall syslog-ng, because > > > > > journalctl is > > > > > > > *such* a nice way to read logs, so why run something whose output > > > I'll > > > > > never > > > > > > > read again? I recommend reading > > > > > > > http://0pointer.net/blog/projects/journalctl.html for examples of > > > the > > > > > kind of > > > > > > > stuff you can do that would be cumbersome, if not *impossible* with > > > > > regular > > > > > > > syslog. > > > > > > > > > > > > Except that I get lots of messages about the system journal missing > > > > > > messages when forwarding to syslog, so how can I make sure this does > > > not > > > > > > happening? > > > > > > > > > > Could you please show those messages? systemd sends *everything* to the > > > > > journal, and then the journal (optionally) can send it too to a regular > > > > > syslog. In that sense, it's impossible for the journal to miss any > > > message. > > > > > > > > > > The only way in which the journal could miss messages is at very early > > > boot > > > > > stages; but with a proper initramfs (like the ones generated with > > > dracut), > > > > > even those get caught. You get to put an instance of systemd and the > > > > > journal inside the initramfs, and so it's available almost from the > > > > > beginning. > > > > > > > > > > And if you use gummiboot, then you can even log from the moment the UEFI > > > > > firmware comes to life. > > > > > > > > So, I get lots of messages in my regular syslog-ng /var/log/messages > > > > like the following: > > > > Feb 23 12:47:52 ccs.covici.com systemd-journal[715]: Forwarding to > > > > syslog missed 15 messages. > > > > > > > > So, I saw a post on Google to up the queue length, and I uped it to 200, > > > > but no joy, still get the messages like the one above. > > > > > > Are you using the unit file provided by syslog-ng (systemd-delta doesn't > > > mention syslog)? Also, is /etc/systemd/system/syslog.service is a link > > > to /usr/lib/systemd/system/syslog-ng.service? > > > > > > I do, and I don't get any of those messages. I use the default journal > > > configuration. According to [1], this should be fixed. > > > > I remember getting a small number of messages like that, too, on my laptop. > > However, it's at the universit
Re: [gentoo-user] syslog-ng: how to read the log files
Canek Peláez Valdés wrote: > On Mon, Feb 23, 2015 at 1:31 PM, wrote: > > > > Marc Joliet wrote: > > > > > Am Mon, 23 Feb 2015 12:10:18 -0600 > > > schrieb Canek Peláez Valdés : > > > > > > > On Mon, Feb 23, 2015 at 11:49 AM, wrote: > > > > > > > > > > Canek Peláez Valdés wrote: > > > > > > > > > > > On Mon, Feb 23, 2015 at 3:41 AM, wrote: > > > > > > > > > > > > > > Marc Joliet wrote: > > > > > > > > > > > > > > > Am Mon, 23 Feb 2015 00:41:50 +0100 > > > > > > > > schrieb lee : > > > > > > > > > > > > > > > > > Neil Bothwick writes: > > > > > > > > > > > > > > > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > > > > > > > > > > > > > > > > > >> > I wonder if the OP is using systemd and trying to read > the > > > > > > journal > > > > > > > > > >> > files? > > > > > > > > > >> > > > > > > > > > >> Nooo, I hate systemd ... > > > > > > > > > >> > > > > > > > > > >> What good are log files you can't read? > > > > > > > > > > > > > > > > > > > > You can't read syslog-ng log files without some reading > > > > software, > > > > > > usually > > > > > > > > > > a combination of cat, grep and less. systemd does it all > with > > > > > > journalctl. > > > > > > > > > > > > > > > > > > > > There are good reasons to not use systemd, this isn't one > of > > > > them. > > > > > > > > > > > > > > > > > > To me it is one of the good reasons, and an important one. > Plain > > > > text > > > > > > > > > can usually always be read without further ado, be it from > rescue > > > > > > > > > systems you booted or with software available on different > > > > operating > > > > > > > > > systems. It can be also be processed with scripts and sent > as > > > > email. > > > > > > > > > You can probably even read it on your cell phone. You can > still > > > > read > > > > > > > > > log files that were created 20 years ago when they are > plain text. > > > > > > > > > > > > > > > > > > Can you do all that with the binary files created by > systemd? I > > > > can't > > > > > > > > > even read them on a working system. > > > > > > > > > > > > > > > > What Canek and Rich already said is good, but I'll just add > this: > > > > it's > > > > > > not like > > > > > > > > you can't run a classic syslog implementation alongside the > systemd > > > > > > journal. > > > > > > > > On my systems, by *default*, syslog-ng kept working as usual, > > > > getting > > > > > > the logs > > > > > > > > from the systemd journal. If you want to go further, you can > even > > > > > > configure > > > > > > > > the journal to not store logs permanently, so that you *only* > end up > > > > > > with > > > > > > > > plain-text logs on your system (Duncan on gentoo-amd64 went > this > > > > way). > > > > > > > > > > > > > > > > So no, the format that the systemd journal uses is most > decidedly > > > > *not* > > > > > > a reason > > > > > > > > against using systemd. > > > > > > > > > > > > > > > > Personally, I'm probably going to uninstall syslog-ng, because > > > > > > journalctl is > > > > > > > > *such* a nice way to read logs, so why run something whose > output > > > > I'll > > > > > > never > > > > > > > > read again? I recommend reading > > > > > > > > http://0pointer.net/blog/projects/journalctl.html for > examples of > > > > the > > > > > > kind of > > > > > > > > stuff you can do that would be cumbersome, if not > *impossible* with > > > > > > regular > > > > > > > > syslog. > > > > > > > > > > > > > > Except that I get lots of messages about the system journal > missing > > > > > > > messages when forwarding to syslog, so how can I make sure this > does > > > > not > > > > > > > happening? > > > > > > > > > > > > Could you please show those messages? systemd sends *everything* > to the > > > > > > journal, and then the journal (optionally) can send it too to a > regular > > > > > > syslog. In that sense, it's impossible for the journal to miss any > > > > message. > > > > > > > > > > > > The only way in which the journal could miss messages is at very > early > > > > boot > > > > > > stages; but with a proper initramfs (like the ones generated with > > > > dracut), > > > > > > even those get caught. You get to put an instance of systemd and > the > > > > > > journal inside the initramfs, and so it's available almost from > the > > > > > > beginning. > > > > > > > > > > > > And if you use gummiboot, then you can even log from the moment > the UEFI > > > > > > firmware comes to life. > > > > > > > > > > So, I get lots of messages in my regular syslog-ng /var/log/messages > > > > > like the following: > > > > > Feb 23 12:47:52 ccs.covici.com systemd-journal[715]: Forwarding to > > > > > syslog missed 15 messages. > > > > > > > > > > So, I saw a post on Google to up the queue length, and I uped it to > 200, > > > > > but no joy, still get the messages like the one above. > > > > > > > > Are you using the unit file provided by syslog-ng (systemd-delta > doesn't > > > > mention syslog)? Also, is /etc/systemd/system/syslog.service is
Re: [gentoo-user] syslog-ng: how to read the log files
Canek Peláez Valdés wrote: > On Mon, Feb 23, 2015 at 1:31 PM, wrote: > > > > Marc Joliet wrote: > > > > > Am Mon, 23 Feb 2015 12:10:18 -0600 > > > schrieb Canek Peláez Valdés : > > > > > > > On Mon, Feb 23, 2015 at 11:49 AM, wrote: > > > > > > > > > > Canek Peláez Valdés wrote: > > > > > > > > > > > On Mon, Feb 23, 2015 at 3:41 AM, wrote: > > > > > > > > > > > > > > Marc Joliet wrote: > > > > > > > > > > > > > > > Am Mon, 23 Feb 2015 00:41:50 +0100 > > > > > > > > schrieb lee : > > > > > > > > > > > > > > > > > Neil Bothwick writes: > > > > > > > > > > > > > > > > > > > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: > > > > > > > > > > > > > > > > > > > >> > I wonder if the OP is using systemd and trying to read > the > > > > > > journal > > > > > > > > > >> > files? > > > > > > > > > >> > > > > > > > > > >> Nooo, I hate systemd ... > > > > > > > > > >> > > > > > > > > > >> What good are log files you can't read? > > > > > > > > > > > > > > > > > > > > You can't read syslog-ng log files without some reading > > > > software, > > > > > > usually > > > > > > > > > > a combination of cat, grep and less. systemd does it all > with > > > > > > journalctl. > > > > > > > > > > > > > > > > > > > > There are good reasons to not use systemd, this isn't one > of > > > > them. > > > > > > > > > > > > > > > > > > To me it is one of the good reasons, and an important one. > Plain > > > > text > > > > > > > > > can usually always be read without further ado, be it from > rescue > > > > > > > > > systems you booted or with software available on different > > > > operating > > > > > > > > > systems. It can be also be processed with scripts and sent > as > > > > email. > > > > > > > > > You can probably even read it on your cell phone. You can > still > > > > read > > > > > > > > > log files that were created 20 years ago when they are > plain text. > > > > > > > > > > > > > > > > > > Can you do all that with the binary files created by > systemd? I > > > > can't > > > > > > > > > even read them on a working system. > > > > > > > > > > > > > > > > What Canek and Rich already said is good, but I'll just add > this: > > > > it's > > > > > > not like > > > > > > > > you can't run a classic syslog implementation alongside the > systemd > > > > > > journal. > > > > > > > > On my systems, by *default*, syslog-ng kept working as usual, > > > > getting > > > > > > the logs > > > > > > > > from the systemd journal. If you want to go further, you can > even > > > > > > configure > > > > > > > > the journal to not store logs permanently, so that you *only* > end up > > > > > > with > > > > > > > > plain-text logs on your system (Duncan on gentoo-amd64 went > this > > > > way). > > > > > > > > > > > > > > > > So no, the format that the systemd journal uses is most > decidedly > > > > *not* > > > > > > a reason > > > > > > > > against using systemd. > > > > > > > > > > > > > > > > Personally, I'm probably going to uninstall syslog-ng, because > > > > > > journalctl is > > > > > > > > *such* a nice way to read logs, so why run something whose > output > > > > I'll > > > > > > never > > > > > > > > read again? I recommend reading > > > > > > > > http://0pointer.net/blog/projects/journalctl.html for > examples of > > > > the > > > > > > kind of > > > > > > > > stuff you can do that would be cumbersome, if not > *impossible* with > > > > > > regular > > > > > > > > syslog. > > > > > > > > > > > > > > Except that I get lots of messages about the system journal > missing > > > > > > > messages when forwarding to syslog, so how can I make sure this > does > > > > not > > > > > > > happening? > > > > > > > > > > > > Could you please show those messages? systemd sends *everything* > to the > > > > > > journal, and then the journal (optionally) can send it too to a > regular > > > > > > syslog. In that sense, it's impossible for the journal to miss any > > > > message. > > > > > > > > > > > > The only way in which the journal could miss messages is at very > early > > > > boot > > > > > > stages; but with a proper initramfs (like the ones generated with > > > > dracut), > > > > > > even those get caught. You get to put an instance of systemd and > the > > > > > > journal inside the initramfs, and so it's available almost from > the > > > > > > beginning. > > > > > > > > > > > > And if you use gummiboot, then you can even log from the moment > the UEFI > > > > > > firmware comes to life. > > > > > > > > > > So, I get lots of messages in my regular syslog-ng /var/log/messages > > > > > like the following: > > > > > Feb 23 12:47:52 ccs.covici.com systemd-journal[715]: Forwarding to > > > > > syslog missed 15 messages. > > > > > > > > > > So, I saw a post on Google to up the queue length, and I uped it to > 200, > > > > > but no joy, still get the messages like the one above. > > > > > > > > Are you using the unit file provided by syslog-ng (systemd-delta > doesn't > > > > mention syslog)? Also, is /etc/systemd/system/syslog.service is
Re: [gentoo-user] syslog-ng: how to read the log files
On Mon, 23 Feb 2015 18:18:36 -0500, cov...@ccs.covici.com wrote: > I did change the unit file, but no joy, I still get messages like this: > Feb 23 18:16:05 ccs.covici.com systemd-journal[715]: Forwarding to > syslog missed 13 messages. I used to get messages like that. Sometimes substantial numbers of messages, 100+ was far from uncommon. But the last such message in my journal was on November 6th. That's on my laptop, my desktop doesn't have a single such message. -- Neil Bothwick Disinformation is not as good as datinformation. pgpkAJRo3OjyK.pgp Description: OpenPGP digital signature
Re: [gentoo-user] syslog-ng: how to read the log files
On Monday 23 February 2015 23:29:49 Neil Bothwick wrote: > On Mon, 23 Feb 2015 18:18:36 -0500, cov...@ccs.covici.com wrote: > > I did change the unit file, but no joy, I still get messages like > > this: Feb 23 18:16:05 ccs.covici.com systemd-journal[715]: > > Forwarding to syslog missed 13 messages. > > I used to get messages like that. Sometimes substantial numbers of > messages, 100+ was far from uncommon. But the last such message in my > journal was on November 6th. > > That's on my laptop, my desktop doesn't have a single such message. Thank Goodness! Someone who knows enough to trim out the bits of the message he's not replying to. Why do you others make me page-down eight times to find what you've written in reply to the last three lines of the preceding message? -- Rgds Peter.
Re: [gentoo-user] syslog-ng: how to read the log files
Am Tue, 24 Feb 2015 00:50:30 + schrieb Peter Humphrey : > On Monday 23 February 2015 23:29:49 Neil Bothwick wrote: > > On Mon, 23 Feb 2015 18:18:36 -0500, cov...@ccs.covici.com wrote: > > > I did change the unit file, but no joy, I still get messages like > > > this: Feb 23 18:16:05 ccs.covici.com systemd-journal[715]: > > > Forwarding to syslog missed 13 messages. > > > > I used to get messages like that. Sometimes substantial numbers of > > messages, 100+ was far from uncommon. But the last such message in my > > journal was on November 6th. > > > > That's on my laptop, my desktop doesn't have a single such message. > > Thank Goodness! Someone who knows enough to trim out the bits of the > message he's not replying to. > > Why do you others make me page-down eight times to find what you've > written in reply to the last three lines of the preceding message? *Checks to see if he's guilty.* *Hangs head in shame.* -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup pgp7l3HzDsiJp.pgp Description: Digitale Signatur von OpenPGP
Re: [gentoo-user] syslog-ng: how to read the log files
> On Feb 24, 2015, at 2:50, Peter Humphrey wrote: > > Thank Goodness! Someone who knows enough to trim out the bits of the > message he's not replying to. > > Why do you others make me page-down eight times to find what you've > written in reply to the last three lines of the preceding message? +1 -- -Matti
Re: [gentoo-user] syslog-ng: how to read the log files
On Sun, 22 February 2015, at 11:48 pm, lee wrote: > I believe this may be bug 406623. https://bugs.gentoo.org/show_bug.cgi?id=406623 >>> >>> That's almost three years old and should apparently be fixed? >> >> It's only been closed in the last few weeks. > > Still I wonder why it took so long to fix it. That's hardly unusual - Gentoo is massively understaffed. Even version bumps may sometimes take weeks to be actioned. Stroller.
Re: [gentoo-user] syslog-ng: how to read the log files
Canek Peláez Valdés writes: > On Sun, Feb 22, 2015 at 6:41 PM, lee wrote: >> >> Neil Bothwick writes: >> >> > On Wed, 18 Feb 2015 21:49:54 +0100, lee wrote: >> > >> >> > I wonder if the OP is using systemd and trying to read the journal >> >> > files? >> >> >> >> Nooo, I hate systemd ... >> >> >> >> What good are log files you can't read? >> > >> > You can't read syslog-ng log files without some reading software, > usually >> > a combination of cat, grep and less. systemd does it all with > journalctl. >> > >> > There are good reasons to not use systemd, this isn't one of them. >> >> To me it is one of the good reasons, and an important one. Plain text >> can usually always be read without further ado, be it from rescue >> systems you booted or with software available on different operating >> systems. It can be also be processed with scripts and sent as email. >> You can probably even read it on your cell phone. You can still read >> log files that were created 20 years ago when they are plain text. >> >> Can you do all that with the binary files created by systemd? > > Yes, you can. You can predict the next 20 years? >> I can't even read them on a working system. > > If that's true (which I highly doubt, more probably you don't know how to > read them), then it's a bug and should be reported and fixed. I read log files with less. The bug is that systemd uses some sort of binary files, and they aren't going to fix it. They even won't fix their misunderstanding of what "disabled" means. So why make bug reports? -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
Rich Freeman writes: > On Sun, Feb 22, 2015 at 6:41 PM, lee wrote: >> >> To me it is one of the good reasons, and an important one. Plain text >> can usually always be read without further ado, be it from rescue >> systems you booted or with software available on different operating >> systems. It can be also be processed with scripts and sent as email. >> You can probably even read it on your cell phone. You can still read >> log files that were created 20 years ago when they are plain text. > > Doing any of that stuff requires the use of software capable of > reading text files. It isn't like you can just interpret the magnetic > fields on your disk with your eyes. Yes, and it doesn't seem very likely that it'll become impossible to read text files in the next 20 years. > Sure, there are a lot more utilities that can read text files than > journal files, but you just need to arrange to have them handy. > They'll be ubiquitous before long since every distro around will end > up needing them. Hopefully not, systemd is a bad thing for many reasons. >> Can you do all that with the binary files created by systemd? I can't >> even read them on a working system. >> > > You just type journalctl to read the live system logs. For offline > use you just type journalctl --file=filename. Or you can just run > strings on the file I imagine if you're desperate. If it doesn't work > on a "working system" then your system isn't working. See, ppl already claim that when something that comes from systemd isn't working, then the system isn't working. Unfortunately, they overlook that when things systemd don't work by design, it's bad design or a problem of systemd rather than the system not working. -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
Marc Joliet writes: >> Can you do all that with the binary files created by systemd? I can't >> even read them on a working system. > > What Canek and Rich already said is good, but I'll just add this: it's not > like > you can't run a classic syslog implementation alongside the systemd journal. > On my systems, by *default*, syslog-ng kept working as usual, getting the logs > from the systemd journal. If you want to go further, you can even configure > the journal to not store logs permanently, so that you *only* end up with > plain-text logs on your system (Duncan on gentoo-amd64 went this way). > > So no, the format that the systemd journal uses is most decidedly *not* a > reason > against using systemd. It is only one of the many reasons. I don't find it advantageous to have to waste additional resources to be able to read the log files. > Personally, I'm probably going to uninstall syslog-ng, because journalctl is > *such* a nice way to read logs, so why run something whose output I'll never > read again? If you like it, nobody prevents you from using it. It's good to have many options. Just don't force others to use it as well. -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
On Mon, May 4, 2015 at 2:14 AM, lee wrote: > Marc Joliet writes: > >> Personally, I'm probably going to uninstall syslog-ng, because journalctl is >> *such* a nice way to read logs, so why run something whose output I'll never >> read again? > > If you like it, nobody prevents you from using it. It's good to have > many options. Just don't force others to use it as well. > Who is forcing anybody to use anything? Did Lennart break into your house with an RHEL 7 disk and force you to install it at gunpoint or something? You did a great job holding out under the torture - that would explain your 2.5 month absence from this long-dead thread. Fortunately, while you were gone nobody treecleaned sysvinit, not that treecleaning a package prevents anybody from using it. -- Rich
Re: [gentoo-user] syslog-ng: how to read the log files
On Mon, May 4, 2015 at 1:57 AM, lee wrote: > Canek Peláez Valdés writes: >> On Sun, Feb 22, 2015 at 6:41 PM, lee wrote: >>> >>> I can't even read them on a working system. >> >> If that's true (which I highly doubt, more probably you don't know how to >> read them), then it's a bug and should be reported and fixed. > > I read log files with less. The bug is that systemd uses some sort of > binary files, and they aren't going to fix it. They even won't fix > their misunderstanding of what "disabled" means. So why make bug > reports? The systemd developers' use of disable/mask isn't wrong simply because you disagree with them. "systemctl disable unit" is the same as "blacklist module": the unit/module can be loaded manually or as a dependency. "systemctl mask unit" is the same as "install module /bin/true": the unit/module can't be loaded.
Re: [gentoo-user] syslog-ng: how to read the log files
Tom H writes: > On Mon, May 4, 2015 at 1:57 AM, lee wrote: >> Canek Peláez Valdés writes: >>> On Sun, Feb 22, 2015 at 6:41 PM, lee wrote: I can't even read them on a working system. >>> >>> If that's true (which I highly doubt, more probably you don't know how to >>> read them), then it's a bug and should be reported and fixed. >> >> I read log files with less. The bug is that systemd uses some sort of >> binary files, and they aren't going to fix it. They even won't fix >> their misunderstanding of what "disabled" means. So why make bug >> reports? > > The systemd developers' use of disable/mask isn't wrong simply because > you disagree with them. No, it's wrong because they don't know what "disabled" means. Feel free to look into dictionaries and to examine the use of the word "disabled" in it's language to find out what it means. -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
Rich Freeman writes: > On Mon, May 4, 2015 at 2:14 AM, lee wrote: >> Marc Joliet writes: >> >>> Personally, I'm probably going to uninstall syslog-ng, because journalctl is >>> *such* a nice way to read logs, so why run something whose output I'll never >>> read again? >> >> If you like it, nobody prevents you from using it. It's good to have >> many options. Just don't force others to use it as well. >> > > Who is forcing anybody to use anything? Look around and you will find that systemd has taken over Linux, with a few exceptions as in distributions like Gentoo. The taking over will probably continue until you cannot use Linux anymore without using systemd. If that's not forcing, then what is it? -- Again we must be afraid of speaking of daemons for fear that daemons might swallow us. Finally, this fear has become reasonable.
Re: [gentoo-user] syslog-ng: how to read the log files
On Saturday 09 May 2015 17:01:00 lee wrote: > Tom H writes: > > The systemd developers' use of disable/mask isn't wrong simply because > > you disagree with them. > > No, it's wrong because they don't know what "disabled" means. Feel free > to look into dictionaries and to examine the use of the word "disabled" > in it's language to find out what it means. I've been through this before, and a certain contributor to e-mail lists is still in my kill filter because of it. Some computing people, mostly American in my experience, insist that "disabled" means the same as "switched off". No amount of pointing out the error of this makes any difference. They merely shrug and cite custom and practice. It was never custom or practice in my patch of the forest. Incidentally, there's another stupidity in an ancient CPU instruction set, I think 8080. If I move something from A to B it's no longer at A, but in the mov instruction it finishes up in both places. Sometimes I wish the language were still extended only by scholars. -- Rgds Peter
Re: [gentoo-user] syslog-ng: how to read the log files
On Sat, May 9, 2015 at 11:08 AM, lee wrote: > Rich Freeman writes: >> >> Who is forcing anybody to use anything? > > Look around and you will find that systemd has taken over Linux, with a > few exceptions as in distributions like Gentoo. The taking over will > probably continue until you cannot use Linux anymore without using > systemd. If that's not forcing, then what is it? They're not forcing you to use Linux at all, let alone Linux with systemd. Fedora v13 never contained systemd as far as I'm aware. Today it still works EXACTLY as it did then. Nobody is preventing you from using that. The same is true for any other distro that has adopted systemd. Your complaint isn't that people are forcing you to use systemd. Your complaint is that somebody isn't building a linux distro to your personal specifications free of charge. With FOSS we all contribute code that anybody can use for as long as they wish. The fact that you don't want to use the code that somebody wrote a few years ago and would rather they write updated software (perhaps with security vulnerabilities removed, features added, and so on) doesn't obligate others to create it for you. And that is my issue with this line of argument. It assumes that you have a right to demand that others create free software for you, and that they do it to your specifications. I get your frustration. There have been FOSS projects that were discontinued that I've thought were really valuable. However, while I mourn their indefinite slumber, I'm not going to complain that the devs chose to move on. I never paid them for what they gave me in the first place, and I have no right to demand more. The same is true of whoever maintains your init scripts. If you have a support contract that requires somebody to backport fixes to whatever you're using for 10 years, and they're breaking that contract, then I can only agree that you're in the right. Short of that, you've gotten what you've paid for. I understand your frustration, but I don't think the use of terms like "force" is justified. -- Rich
systemd journal location (was: Re: [gentoo-user] syslog-ng: how to read the log files)
Am Tue, 17 Feb 2015 23:31:26 +0100 schrieb Marc Joliet : > Am Tue, 17 Feb 2015 13:45:38 -0600 > schrieb Canek Peláez Valdés : > > > On Tue, Feb 17, 2015 at 2:29 PM, wrote: > > > > > > Alexander Kapshuk wrote: > > > > > > > On Tue, Feb 17, 2015 at 8:26 PM, lee wrote: > > > > > > > > > Hi, > > > > > > > > > > how do you read the log files when using syslog-ng? > > > > > > > > > > The log file seem to be some sort of binary that doesn't display too > > > > > well in less, and there doesn't seem to be any way to read them. > > > > > > > > > > > > > > > -- > > > > > Again we must be afraid of speaking of daemons for fear that daemons > > > > > might swallow us. Finally, this fear has become reasonable. > > > > > > > > > > > > > > If you're talking about /var/log/messages, which is: > > > > messages: data > > > > > > > > I use cat(1). > > > > > > I wonder if the OP is using systemd and trying to read the journal > > > files? > > > > Those live under /var/lib/journal (which you need to create; Gentoo doesn't > > do it by default last time I saw) > [...] > > It did on my laptop after I migrated it to systemd over the weekend (on a > whim, > no less -- apparently I'm adventurous?). Or, to be more precise, I didn't have > to create the directory myself. And wouldn't it be created at run-time, > anyway? > That's what I would expect, at least. Dammit, I *wanted* to mention that I didn't have my laptop there to look, and now I regret not doing it, because I was *actually* thinking of /var/log/journal/ (which I still didn't create by hand, BTW). I mean, it still contains journal files, and systemd-journald(8) says its the default *persistent* journal location. However, it is structured differently than what you showed, namely: % tree /var/log/journal/ /var/log/journal/ ├── b3a495d35e890b80816684a4521fc1cc │ ├── system.journal │ └── user-1000.journal └── remote So it creates a directory named after the machine ID, which contains a system journal and one journal per user. And if it receives logs from remote machines, those go into the remote folder. Just, uh, just so you know... -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup pgpa6dFL7j83g.pgp Description: Digitale Signatur von OpenPGP
