Re: What in /var/logs shows system reboot?

[francis picabia]

On Tue, Jan 24, 2017 at 9:47 AM, Greg Wooledge  wrote:

> On Mon, Jan 23, 2017 at 08:28:08PM -0400, francis picabia wrote:
> > Here is the exercise anyone reading can try:
> >
> > Prove to yourself exactly when you rebooted your Debian system(s)
>
> arc3:~$ uptime
>  08:44:40 up 8 days, 31 min,  1 user,  load average: 0.02, 0.02, 0.00
>
> Everything's easy on a live, running system.
>

> > to ensure you were safe against dirty cow.
>
> Oh, *that*?  In that case, you don't give a flying leap how long ago you
> rebooted.  What you care about is the *exact running kernel version*.
>
> arc3:~$ uname -v
> #1 SMP Debian 3.16.39-1 (2016-12-30)
>
> Then you compare 3.16.39-1 against the changelog.Debian.gz to see if it's
> got the bug fixes you want.
>
> http://mywiki.wooledge.org/XyProblem
>
> I had unattended upgrades on, but didn't have reboot set to automatically
trigger.  I found evidence a user tried the Dirty COW
exploit a couple of days after the kernel was upgraded,
but I needed to know exactly when the system had
been rebooted, in October 2016, to see whether
the exploit had possibly worked.  With the old dmesg files, I would
likely have that on hand as they don't rotate away too quickly.

The solution was to restore /var/log from backup tapes and I see
when the reboot happened in kern.log.  I'll consider increasing the
number of kern.log to keep in logrotate so I might not need to wait
for backup tapes in the future.

Re: What in /var/logs shows system reboot?

[Joe]

On Tue, 24 Jan 2017 23:33:37 +
Jonathan de Boyne Pollard 
wrote:

> Joe:
> 
> > Using strings /var/log/journal/*/* | grep debian-kernel [...]
> >  
> 
> Or one could just use journalctl .
> 

Hit send too early.

On a systemd-based installation, yes. For looking at logs from a systemd
system using an installation which doesn't have systemd, no.

But the *question* was 'what files in /var/log have reboot
information?', to which the answer is not 'journalctl'.

-- 
Joe

Re: What in /var/logs shows system reboot?

[Joe]

On Tue, 24 Jan 2017 23:33:37 +
Jonathan de Boyne Pollard 
wrote:

> Joe:
> 
> > Using strings /var/log/journal/*/* | grep debian-kernel [...]
> >  
> 
> Or one could just use journalctl .
> 

On a systemd-based installation, yes. For looking at logs from a systemd
system using an installation which doesn't have systemd, no.

-- 
Joe

Re: What in /var/logs shows system reboot?

[Greg Wooledge]

On Mon, Jan 23, 2017 at 08:28:08PM -0400, francis picabia wrote:
> Here is the exercise anyone reading can try:
> 
> Prove to yourself exactly when you rebooted your Debian system(s)

arc3:~$ uptime
 08:44:40 up 8 days, 31 min,  1 user,  load average: 0.02, 0.02, 0.00

Everything's easy on a live, running system.

> to ensure you were safe against dirty cow.

Oh, *that*?  In that case, you don't give a flying leap how long ago you
rebooted.  What you care about is the *exact running kernel version*.

arc3:~$ uname -v
#1 SMP Debian 3.16.39-1 (2016-12-30)

Then you compare 3.16.39-1 against the changelog.Debian.gz to see if it's
got the bug fixes you want.

http://mywiki.wooledge.org/XyProblem

Re: What in /var/logs shows system reboot?

[francis picabia]

On Mon, Jan 23, 2017 at 4:04 PM, Joe  wrote:

> On Mon, 23 Jan 2017 14:28:33 -0400
> francis picabia  wrote:
>
> > On Mon, Jan 23, 2017 at 2:18 PM, Greg Wooledge 
> > wrote:
> >
> > > On Mon, Jan 23, 2017 at 02:12:04PM -0400, francis picabia wrote:
> > > > I'm running Debian 8.6, and looking at old logs.  I'd like to
> > > > confirm
> > > when
> > > > the system was rebooted to invoke the newer kernel which fixed
> > > > the Dirty COW bug.
> > >
> > > last | grep boot
> > >
> > > or, apparently (according to the man page):
> > >
> > > last reboot
> > >
> > > > If I have a complete
> > > > copy of my /var/log from last October,
> > >
> > > Urghhh.  So it's not on a live system?  It's on a chrooted disk
> > > image? Then I think you need to use:
> > >
> > > last -f /some/path/wtmp reboot
> > >
> > > where /some/path/wtmp is the location of your chrooted disk image's
> > > wtmp file.
> > >
> > >
> > I think I said I already tried that.  There are no results from last
> > pointed to any recovered wtmp file.  It is a recovery of /var/log
> > from backup tape.
> >
> > I even have psacct on that system.  A command like
> > lastcomm -f pacct | grep boot
> > is returning nothing.
>
> How about grep -R debian-kernel /var/log/*
>
> On my system (on a workstation) it returns the last dozen boots
> from /var/log/kernel.log.1 and /var/log/messages.1.
>
> You'll need recent logs after the reboot, as logrotate is in action
> here. In the old days, dmesg.N would have stored several reboots, not on
> a time basis.
>

Yes, that is what I remember.  On a system up for 60 days since
the last reboot, there is nothing to go by these days, at least
with the default logrotate settings.

Thanks for the explanation on the journal files.  I might try that.

Here is the exercise anyone reading can try:

Prove to yourself exactly when you rebooted your Debian system(s)
to ensure you were safe against dirty cow.



> Going back further than logrotate, it is (still) possible to pull small
> strings out of systemd journal files, but without timestamps, because
> that's the bit journald handles.
>
> Using strings /var/log/journal/*/* | grep debian-kernel will confirm
> the records of what kernels systemd remembers, grepping for rtc_cmos
> will return times of clock settings during boot. Once you see the line
> that the debian-kernel grep returns, you can try a direct grep with the
> exact kernel name to find the appropriate journal file, then using
> strings | grep to match kernel version to boot time. Hopefully you only
> need to search a couple of files if you don't boot often, but the
> filenames are painfully long.
>
> --
> Joe
>
>

Re: What in /var/logs shows system reboot?

[Joe]

On Mon, 23 Jan 2017 14:28:33 -0400
francis picabia  wrote:

> On Mon, Jan 23, 2017 at 2:18 PM, Greg Wooledge 
> wrote:
> 
> > On Mon, Jan 23, 2017 at 02:12:04PM -0400, francis picabia wrote:  
> > > I'm running Debian 8.6, and looking at old logs.  I'd like to
> > > confirm  
> > when  
> > > the system was rebooted to invoke the newer kernel which fixed
> > > the Dirty COW bug.  
> >
> > last | grep boot
> >
> > or, apparently (according to the man page):
> >
> > last reboot
> >  
> > > If I have a complete
> > > copy of my /var/log from last October,  
> >
> > Urghhh.  So it's not on a live system?  It's on a chrooted disk
> > image? Then I think you need to use:
> >
> > last -f /some/path/wtmp reboot
> >
> > where /some/path/wtmp is the location of your chrooted disk image's
> > wtmp file.
> >
> >  
> I think I said I already tried that.  There are no results from last
> pointed to any recovered wtmp file.  It is a recovery of /var/log
> from backup tape.
> 
> I even have psacct on that system.  A command like
> lastcomm -f pacct | grep boot
> is returning nothing.

How about grep -R debian-kernel /var/log/*

On my system (on a workstation) it returns the last dozen boots 
from /var/log/kernel.log.1 and /var/log/messages.1.

You'll need recent logs after the reboot, as logrotate is in action
here. In the old days, dmesg.N would have stored several reboots, not on
a time basis.

Going back further than logrotate, it is (still) possible to pull small
strings out of systemd journal files, but without timestamps, because
that's the bit journald handles.

Using strings /var/log/journal/*/* | grep debian-kernel will confirm
the records of what kernels systemd remembers, grepping for rtc_cmos
will return times of clock settings during boot. Once you see the line
that the debian-kernel grep returns, you can try a direct grep with the
exact kernel name to find the appropriate journal file, then using
strings | grep to match kernel version to boot time. Hopefully you only
need to search a couple of files if you don't boot often, but the
filenames are painfully long.

-- 
Joe

Re: What in /var/logs shows system reboot?

[Sven Joachim]

On 2017-01-23 14:12 -0400, francis picabia wrote:

> I'm running Debian 8.6, and looking at old logs.  I'd like to confirm when
> the system was rebooted to invoke the newer kernel which fixed
> the Dirty COW bug.  With systemd, I'm not seeing the old signs
> like the announcement of the kernel version.

Something like this should do the trick:

$ zgrep "\[0.00] Linux version" /var/log/kern.log*

Provided that the files actually go back that far, in the default
logrotate configuration only the last four weeks of kernel logs are
kept.

> This should be a simple question but I'm not seeing the usual
> stuff like dmesg file activity in /var/log

There is no /var/log/dmesg file with systemd because it is redundant,
all that info is in the journal (and in the other log files like
/var/log/kern.log if you have a syslog daemon installed).

Cheers,
   Sven

Re: What in /var/logs shows system reboot?

[francis picabia]

On Mon, Jan 23, 2017 at 2:18 PM, Greg Wooledge  wrote:

> On Mon, Jan 23, 2017 at 02:12:04PM -0400, francis picabia wrote:
> > I'm running Debian 8.6, and looking at old logs.  I'd like to confirm
> when
> > the system was rebooted to invoke the newer kernel which fixed
> > the Dirty COW bug.
>
> last | grep boot
>
> or, apparently (according to the man page):
>
> last reboot
>
> > If I have a complete
> > copy of my /var/log from last October,
>
> Urghhh.  So it's not on a live system?  It's on a chrooted disk image?
> Then I think you need to use:
>
> last -f /some/path/wtmp reboot
>
> where /some/path/wtmp is the location of your chrooted disk image's
> wtmp file.
>
>
I think I said I already tried that.  There are no results from last
pointed to any recovered wtmp file.  It is a recovery of /var/log
from backup tape.

I even have psacct on that system.  A command like
lastcomm -f pacct | grep boot
is returning nothing.

Re: What in /var/logs shows system reboot?

[Greg Wooledge]

On Mon, Jan 23, 2017 at 02:12:04PM -0400, francis picabia wrote:
> I'm running Debian 8.6, and looking at old logs.  I'd like to confirm when
> the system was rebooted to invoke the newer kernel which fixed
> the Dirty COW bug.

last | grep boot

or, apparently (according to the man page):

last reboot

> If I have a complete
> copy of my /var/log from last October,

Urghhh.  So it's not on a live system?  It's on a chrooted disk image?
Then I think you need to use:

last -f /some/path/wtmp reboot

where /some/path/wtmp is the location of your chrooted disk image's
wtmp file.