Bug#987839: apt-listbugs: daily cleanup runs hourly

[Francesco Poli]

Control: tags -1 - moreinfo


On Thu, 27 May 2021 10:21:17 -0700 Ross Boylan wrote:

> This bug is still tagged "need more info".  Is there anything more you need?
> 
> Also, I submitted a bug against logcheck to add additional patterns:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=988389.
> I think these would cover apt-listbugs as well as other packages.
> I've made similar changes locally.

Hello Ross,
thanks for reminding me about the moreinfo tag!

I wanted to try and write some logcheck rules (at that point I would
have asked you to test them...), but got distracted by other
activities.   :-(

However, I see that you are already proposing a more general solution
to the logcheck package, which is definitely better, I think.

Good, thanks for doing so, and let's see how it goes!

I am dropping the moreinfo tag, as I am no longer waiting for any
further feedback from you.

As previously said, I will think about changing the Description field
into something less misleading. But not now.


Have a nice day!


-- 
 http://www.inventati.org/frx/
 There's not a second to spare! To the laboratory!
. Francesco Poli .
 GnuPG key fpr == CA01 1147 9CD2 EFDF FB82  3925 3E1C 27E1 1F69 BFFE


pgpfODjiMZAIA.pgp
Description: PGP signature

Bug#987839: apt-listbugs: daily cleanup runs hourly

[Ross Boylan]

This bug is still tagged "need more info".  Is there anything more you need?

Also, I submitted a bug against logcheck to add additional patterns:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=988389.
I think these would cover apt-listbugs as well as other packages.
I've made similar changes locally.

Ross

Bug#987839: apt-listbugs: daily cleanup runs hourly

[Francesco Poli]

On Tue, 4 May 2021 21:41:03 -0700 Ross Boylan wrote:

[...]
> On Mon, May 3, 2021 at 2:24 PM Francesco Poli  
> wrote:
[...]
> > I would be grateful, if you could find out why.
> > I am asking you, since you seem to be more knowledgeable about logcheck
> > than me.
> 
> This strikes me as secondary to avoiding having hourly runs for a
> daily job, but I've tracked down what's going on.

It's not so secondary: anacron runs hourly for very similar reasons,
and you do not seem to complain about it!
Maybe just because its hourly check does not trigger a logcheck mail
message?!?;-)

> 
> There are specific patterns to ignore the relevant messages.  For
> example, for anacron
[...]
> The file ships as part of the logcheck-database package, although
> individual packages can also install such files directly,
> i.e., it could have shipped as part of anacron.

I see: there are a number of such files which are indeed shipped in
other packages.
I do not have logcheck installed, but I have some logcheck
configuration snippets:

  $ dpkg -S /etc/logcheck/ignore.d.server/mdadm 
  mdadm: /etc/logcheck/ignore.d.server/mdadm
  $ dpkg -S /etc/logcheck/ignore.d.server/rsyslog 
  rsyslog: /etc/logcheck/ignore.d.server/rsyslog

and so forth...

> 
> But
> == ...ignore.d.server/systemd 
> =
> ^[[:alpha:]]{3} [ :[:digit:]]{11} [._[:alnum:]\-]+ systemd\[[0-9]+\]:
> (Starting|Stopping) [ +[:alnum:]/]+\.(\.\.)?$
> 
> looks as if it should match the Starting..., but the expression [
> +[:alnum:]/]+ does not match "-" in "apt-listbugs".  Changing it to
> [- +[:alnum:]/]+ does get it to match.  I think "-" needs to be first
> to avoid having it interpreted as indicating a range.  There is also a
> rule for Finished, but it enumerates specific text to look for and
> apt-listbugs isn't on it

I will probably add a /etc/logcheck/ignore.d.server/apt-listbugs file
sooner or later (after bullseye is released as stable).

[...]
> Granted that detecting if the network is "really" up has no perfect
> solution, it seems to me there's no strong reason for apt-listbugs to
> be more paranoid about it than other packages, where by paranoid I
> mean checking every hour because it's possible there was a network
> problem.

Please note that anacron only has daily, weekly, and monthly tasks to
execute.  Nonetheless, it checks hourly whether there's anything still
to be done.  And its tasks do not necessarily involve network access...

I am convinced that apt-listbugs should not risk wasting opportunities
to successfully run its cleanup task.  Otherwise users will report bugs
complaining that package pins are never removed or removed too late.

[...]
> > So, unless you reboot at least once a day (and be online shortly
> > after!), you may fail several attempts in a row, just because you are
> > not online at the "right" time...
> 
> There is a 5 minute delay,

Exactly: by "shortly after" I meant "within 5 minutes".

> and network targets to try to assure connectivity.

These are not too reliable, as already said.

[...]
> > But your suggestion is not too bad.
> > Maybe something like:
> >
> >   Description=Hourly check for daily apt-listbugs preferences cleanup
> >
> 
> That sounds pretty good to me.
[...]

Thanks: I will think about it.


-- 
 http://www.inventati.org/frx/
 There's not a second to spare! To the laboratory!
. Francesco Poli .
 GnuPG key fpr == CA01 1147 9CD2 EFDF FB82  3925 3E1C 27E1 1F69 BFFE


pgpzPMAlGTapV.pgp
Description: PGP signature

Bug#987839: apt-listbugs: daily cleanup runs hourly

[Ross Boylan]

A couple comments below.

On Mon, May 3, 2021 at 2:24 PM Francesco Poli  wrote:
>
> On Sun, 2 May 2021 14:11:59 -0700 Ross Boylan wrote:
>
> [...]
> > On Sun, May 2, 2021 at 11:08 AM Francesco Poli
> >  wrote:
> [...]
> > > I wonder why logcheck does not send hourly mail messages about
> > > anacron...
> > >
> >
> > For logcheck to send a message there must be something in the logs it
> > checks and it must either match a pattern logcheck thinks is
> > noteworthy or (from memory) fail to match a pattern for things that
> > are OK.  I haven't reviewed exactly why the messages are being noted,
> > albeit at logcheck's lowest severity level.  I configured logcheck  to
> > use workstation mode.
>
> In my humble opinion, this is the point that needs to be better
> scrutinized.
> Anacron systemd timer fires hourly, but logcheck does not consider it
> as something worth a mail message.
> On the other hand, it annoys users with messages about apt-listbugs
> systemd timer triggers, which are equally "normal".
>
> I would be grateful, if you could find out why.
> I am asking you, since you seem to be more knowledgeable about logcheck
> than me.

This strikes me as secondary to avoiding having hourly runs for a
daily job, but I've tracked down what's going on.

There are specific patterns to ignore the relevant messages.  For
example, for anacron
==/etc/logcheck/logcheck.ignore.d/server/anacron

^\w{3} [ :0-9]{11} [._[:alnum:]-]+ anacron\[[0-9]+\]: Anacron
[.[:alnum:]]+ started on [0-9-]+$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ anacron\[[0-9]+\]: Jobs will be
executed sequentially$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ anacron\[[0-9]+\]: Normal exit
\([0-9]+ jobs* run\)$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ anacron\[[0-9]+\]: Will run job
`[._[:alnum:]-]+' in [0-9]+ min\.$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ anacron\[[0-9]+\]: Job
`[._[:alnum:]-]+' started$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ anacron\[[0-9]+\]: Job
`[._[:alnum:]-]+' terminated$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ anacron\[[0-9]+\]: Job
`[._[:alnum:]-]+' terminated (\(exit status: [[:digit:]]+\)
)?\(mailing output\)$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ anacron\[[0-9]+\]: Updated
timestamp for job `[._[:alnum:]-]+' to [0-9-]+$
=

The file ships as part of the logcheck-database package, although
individual packages can also install such files directly,
i.e., it could have shipped as part of anacron.

But
== ...ignore.d.server/systemd =
^[[:alpha:]]{3} [ :[:digit:]]{11} [._[:alnum:]\-]+ systemd\[[0-9]+\]:
(Starting|Stopping) [ +[:alnum:]/]+\.(\.\.)?$

looks as if it should match the Starting..., but the expression [
+[:alnum:]/]+ does not match "-" in "apt-listbugs".  Changing it to
[- +[:alnum:]/]+ does get it to match.  I think "-" needs to be first
to avoid having it interpreted as indicating a range.  There is also a
rule for Finished, but it enumerates specific text to look for and
apt-listbugs isn't on it (I'm having trouble pasting from the testing
VM--the rule has changed since buster; the snips above are from
buster).

>
> [...]
> > Let's see how apt-daily-upgrade works:
> [...]
> > ExecStartPre=-/usr/lib/apt/apt-helper wait-online
> [...]
> >
> > 3. network targets in [Unit] to (try to?) assure connectivity.  BTW,
> > for some insight into why the network target doesn't work as expected,
> > see https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/.
> > I still find it annoying, but I appreciate the point that it is not
> > straightforward to say exactly what the network being "up" means.
>
> This is one of the trickiest parts.
> There seems to be no reliable method to check whether a box is "online".
>
> Please note that apt-daily-upgrade uses "/usr/lib/apt/apt-helper
> wait-online", which is shipped by package 'apt', see its [source].
>
> [source]: 
> 
>
> I could not find any man page for that helper command.
>
> It seems to me that that helper fails to detect whether an Ethernet
> interface (managed with ifup/ifdown) is configured.
>
>   # ifdown enp0s25
>
> Internet is not reachable... but, nonetheless the command immediately
> exits reporting a zero exit status:
>
>   $ /usr/lib/apt/apt-helper wait-online
>   $ echo $?
>   0
>
> After bringing the interface back up:
>
>   # ifup enp0s25
>
> Internet is again reachable... but... no difference:
>
>   $ /usr/lib/apt/apt-helper wait-online
>   $ echo $?
>   0
>
> If it cannot see that a network interface of the box it is running on
> is down... what about issues farther away from the local host?
>
> I don't think this approach could work for apt-listbugs...

Granted that detecting if the network is "really" up has no perfect
solution, it seems to me there's no strong reason for apt-listbugs to
be more 

Bug#987839: apt-listbugs: daily cleanup runs hourly

[Francesco Poli]

On Sun, 2 May 2021 14:11:59 -0700 Ross Boylan wrote:

[...]
> On Sun, May 2, 2021 at 11:08 AM Francesco Poli
>  wrote:
[...]
> > I wonder why logcheck does not send hourly mail messages about
> > anacron...
> >
> 
> For logcheck to send a message there must be something in the logs it
> checks and it must either match a pattern logcheck thinks is
> noteworthy or (from memory) fail to match a pattern for things that
> are OK.  I haven't reviewed exactly why the messages are being noted,
> albeit at logcheck's lowest severity level.  I configured logcheck  to
> use workstation mode.

In my humble opinion, this is the point that needs to be better
scrutinized.
Anacron systemd timer fires hourly, but logcheck does not consider it
as something worth a mail message.
On the other hand, it annoys users with messages about apt-listbugs
systemd timer triggers, which are equally "normal".

I would be grateful, if you could find out why.
I am asking you, since you seem to be more knowledgeable about logcheck
than me.

[...]
> Let's see how apt-daily-upgrade works:
[...]
> ExecStartPre=-/usr/lib/apt/apt-helper wait-online
[...]
>
> 3. network targets in [Unit] to (try to?) assure connectivity.  BTW,
> for some insight into why the network target doesn't work as expected,
> see https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/.
> I still find it annoying, but I appreciate the point that it is not
> straightforward to say exactly what the network being "up" means.

This is one of the trickiest parts.
There seems to be no reliable method to check whether a box is "online".

Please note that apt-daily-upgrade uses "/usr/lib/apt/apt-helper
wait-online", which is shipped by package 'apt', see its [source].

[source]: 


I could not find any man page for that helper command.

It seems to me that that helper fails to detect whether an Ethernet
interface (managed with ifup/ifdown) is configured.

  # ifdown enp0s25 

Internet is not reachable... but, nonetheless the command immediately
exits reporting a zero exit status:

  $ /usr/lib/apt/apt-helper wait-online
  $ echo $?
  0

After bringing the interface back up:

  # ifup enp0s25 

Internet is again reachable... but... no difference:

  $ /usr/lib/apt/apt-helper wait-online
  $ echo $?
  0

If it cannot see that a network interface of the box it is running on
is down... what about issues farther away from the local host?

I don't think this approach could work for apt-listbugs...

[...]
> > The timer would only trigger once every (slightly less than one) day:
> > if your system is not online during that only attempt, you are out of
> > luck for another day or so...
> 
> I think OnActiveSec triggers an initial run 5minutes after system startup.
> Have I misunderstood the semantics?

That's correct, I think.
But then, after the boot, you only have one attempt a day, more or less.

So, unless you reboot at least once a day (and be online shortly
after!), you may fail several attempts in a row, just because you are
not online at the "right" time...

[...]
> > I am open to suggestions on how to change the Description field for the
> > timer.
> > I see that the Description for the anacron timer is "Trigger anacron
> > every hour": maybe I should think about a Description that uses the
> > word "hourly", rather than "daily".
> 
> If sticking with the current setup but changing the message, it currently is
>  Starting Daily apt-listbugs preferences cleanup...
> (or Finishing ).
> Maybe
>  Starting check for daily apt-listbugs preferences cleanup...
> and/or different messages depending on what happened:
>  Skipping daily apt-listbugs preferences cleanup. Too soon.
> or
>  Finished run of daily apt-listbugs preferences cleanup.

I don't think I can customize the message generated by systemd: it's
apparently "Starting ..." and "Finished .".
I can only change the Description field, as far as I can tell.

But your suggestion is not too bad.
Maybe something like:

  Description=Hourly check for daily apt-listbugs preferences cleanup

A bit clumsy, perhaps.
I have to think about it.

> 
> BTW, I don't know what the job is doing,

Please take a look at the apt-listbugs(1) man page:

[...]
|  Each package pin is automatically removed by a daily cron job (or by an
|  equivalent systemd timer), as soon as the corresponding bug is fixed in
|  (or no longer affects) the package version available  for  installation
|  or upgrade. When the pin is removed, the installation or upgrade of the
|  package becomes possible again.
[...]

> but should it also do it after
> someone dpkg-reconfigure's apt-listbugs?

Does "dpkg-reconfigure apt-listbugs" do anything?
Maybe I am just tired right now, but I think it does nothing...


-- 
 http://www.inventati.org/frx/
 There's not a second to spare! To the laboratory!
. Francesco 

Bug#987839: apt-listbugs: daily cleanup runs hourly

[Ross Boylan]

See below.

On Sun, May 2, 2021 at 11:08 AM Francesco Poli
 wrote:
>
> On Sat, 1 May 2021 11:31:19 -0700 Ross Boylan wrote:
>
> [...]
> > On Fri, Apr 30, 2021 at 2:47 PM Francesco Poli
> >  wrote:
> [...]
> > > Does logcheck send e-mail messages for all the other systemd timers?
> > >
> > > As you may already know, you can get a list of active systemd timers on
> > > your box with the following command:
> > >
> > >   $ systemctl list-timers
> > >
> > Thanks for the tip; I wasn't familiar with list-timers.  There are 11
> > on my list, but the only things I see in my hourly reports from
> > logcheck were  apt-listbugs and complaints about time synchronization.
>
> Do you have anacron installed?

Yes.

> It's another package that has an hourly systemd timer.
>
> I wonder why logcheck does not send hourly mail messages about
> anacron...
>

For logcheck to send a message there must be something in the logs it
checks and it must either match a pattern logcheck thinks is
noteworthy or (from memory) fail to match a pattern for things that
are OK.  I haven't reviewed exactly why the messages are being noted,
albeit at logcheck's lowest severity level.  I configured logcheck  to
use workstation mode.

> > Of course, I don't know how many of the timers are hourly.
>
> The command 'systemctl list-timers' should tell you: just look at the
> LEFT and PASSED columns.
Sun 02 May 2021 01:19:46 PM PDT
NEXTLEFTLAST
PASSED   UNIT ACTIVATES
Sun 2021-05-02 13:30:14 PDT 10min left  Sun 2021-05-02 12:31:06 PDT
48min agoanacron.timeranacron.service
Sun 2021-05-02 22:56:18 PDT 9h left Sun 2021-05-02 12:50:06 PDT
29min agoapt-daily.timer  apt-daily.service
Mon 2021-05-03 00:00:00 PDT 10h leftSun 2021-05-02 00:00:01 PDT
13h ago  exim4-base.timer exim4-base.service
Mon 2021-05-03 00:00:00 PDT 10h leftSun 2021-05-02 00:00:01 PDT
13h ago  logrotate.timer  logrotate.service
Mon 2021-05-03 00:00:00 PDT 10h leftSun 2021-05-02 00:00:01 PDT
13h ago  man-db.timer man-db.service
Mon 2021-05-03 01:25:15 PDT 12h leftMon 2021-04-26 16:32:27 PDT 5
days ago   fstrim.timer fstrim.service
Mon 2021-05-03 06:45:23 PDT 17h leftSun 2021-05-02 06:28:28 PDT 6h
ago   apt-daily-upgrade.timer  apt-daily-upgrade.service
Mon 2021-05-03 10:05:28 PDT 20h leftSun 2021-05-02 10:05:28 PDT 3h
14min ago etckeeper.timer  etckeeper.service
Mon 2021-05-03 10:05:28 PDT 20h leftSun 2021-05-02 10:05:28 PDT 3h
14min ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.serv…
Mon 2021-05-03 11:42:24 PDT 22h leftSun 2021-05-02 11:50:28 PDT 1h
29min ago apt-listbugs.timer   apt-listbugs.service
Sun 2021-05-09 03:10:37 PDT 6 days left Sun 2021-05-02 03:11:06 PDT
10h ago  e2scrub_all.timere2scrub_all.service

So it looks as if anacron is the only one that is running hourly.
apt-listbugs would too if I hadn't messed with it.

Let's see how apt-daily-upgrade works:
=apt-daily-upgrade.timer==
[Unit]
Description=Daily apt upgrade and clean activities
After=apt-daily.timer

[Timer]
OnCalendar=*-*-* 6:00
RandomizedDelaySec=60m
Persistent=true

[Install]
WantedBy=timers.target


=apt-daily-upgrade.service=
[Unit]
Description=Daily apt upgrade and clean activities
Documentation=man:apt(8)
ConditionACPower=true
After=apt-daily.service network.target network-online.target
systemd-networkd.service NetworkManager.service connman.service

[Service]
Type=oneshot
ExecStartPre=-/usr/lib/apt/apt-helper wait-online
ExecStart=/usr/lib/apt/apt.systemd.daily install
KillMode=process
TimeoutStopSec=900
=

1. So the timer is 6am every day.
2. Persistent=true means if you restart the system and a timer would
have run while it was off, it is run.  Got that from
https://wiki.archlinux.org/title/Systemd/Timers, though it actually is
documented on the man page for systemd.timer.
3. network targets in [Unit] to (try to?) assure connectivity.  BTW,
for some insight into why the network target doesn't work as expected,
see https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/.
I still find it annoying, but I appreciate the point that it is not
straightforward to say exactly what the network being "up" means.

That seems like a model that could work for apt-listbugs, assuming the
script were changed to update unconditionally when invoked.

>
> Or you could read the timer definitions, if you know (or learn) the
> syntax and semantics...
>
> [...]
> > I tried this modification to apt-listbugs.timer:
> > [Timer]
> > OnActiveSec=5min
> > #OnCalendar=*-*-* *:20
> > OnUnitActiveSec=23h 50m
> > RandomizedDelaySec=20min
> >
> > which did  fix the "running every hour" problem (even if the run is
> > only to check if a real  

Bug#987839: apt-listbugs: daily cleanup runs hourly

[Francesco Poli]

On Sat, 1 May 2021 11:31:19 -0700 Ross Boylan wrote:

[...]
> On Fri, Apr 30, 2021 at 2:47 PM Francesco Poli
>  wrote:
[...]
> > Does logcheck send e-mail messages for all the other systemd timers?
> >
> > As you may already know, you can get a list of active systemd timers on
> > your box with the following command:
> >
> >   $ systemctl list-timers
> >
> Thanks for the tip; I wasn't familiar with list-timers.  There are 11
> on my list, but the only things I see in my hourly reports from
> logcheck were  apt-listbugs and complaints about time synchronization.

Do you have anacron installed?
It's another package that has an hourly systemd timer.

I wonder why logcheck does not send hourly mail messages about
anacron...

> Of course, I don't know how many of the timers are hourly.

The command 'systemctl list-timers' should tell you: just look at the
LEFT and PASSED columns.

Or you could read the timer definitions, if you know (or learn) the
syntax and semantics...

[...]
> I tried this modification to apt-listbugs.timer:
> [Timer]
> OnActiveSec=5min
> #OnCalendar=*-*-* *:20
> OnUnitActiveSec=23h 50m
> RandomizedDelaySec=20min
> 
> which did  fix the "running every hour" problem (even if the run is
> only to check if a real  update is necessary).
> But this does not entirely meet the desired behavior expressed in 932995:
[...]

Not only that, but there's also another issue with this modification.

The timer would only trigger once every (slightly less than one) day:
if your system is not online during that only attempt, you are out of
luck for another day or so...

> > >
> > > Another work-around for the visible annoyance would be for me to tell
> > > logcheck to ignore the relevant messages.
> >
> > I think this should be the way to avoid the annoyance.
> 
> It avoids the annoyance of seeing the message, but it leaves the job
> firing every hour.

I think this is needed to have some reasonable chance to get one
successful cleanup operation a day.

[...]
> > If you do not object, I will close this bug report.
> >
> I suppose, though, as observed in the other bug you mentioned, the
> message about "daily update" is confusing.  The fact that other
> packages aren't exhibiting such behavior suggests there is some other
> way to handle this, but I certainly don't know what it is.

I am open to suggestions on how to change the Description field for the
timer.
I see that the Description for the anacron timer is "Trigger anacron
every hour": maybe I should think about a Description that uses the
word "hourly", rather than "daily".

But then I am afraid I would get bug reports from people saying that an
hourly cleanup is too frequent and asking to reduce the frequency to
daily!

I think that applying the principle of least surprise is especially
tricky in this case...


-- 
 http://www.inventati.org/frx/
 There's not a second to spare! To the laboratory!
. Francesco Poli .
 GnuPG key fpr == CA01 1147 9CD2 EFDF FB82  3925 3E1C 27E1 1F69 BFFE


pgpCBPJIoZoVX.pgp
Description: PGP signature

Bug#987839: apt-listbugs: daily cleanup runs hourly

[Ross Boylan]

Thanks for the quick response.  Answers interspersed below.

On Fri, Apr 30, 2021 at 2:47 PM Francesco Poli
 wrote:
>
> Control: tags -1 + moreinfo
>
>
> On Fri, 30 Apr 2021 10:46:04 -0700 Ross Boylan wrote:
>
> [...]
> > Dear Maintainer,
>
> Hello Ross,
> thanks for your bug report.
>
> >
> > It would be nice to clean up this minor annoyance before buster's release.
>
> I am afraid that we are too late for *buster* release.
> Maybe you meant *bullseye*?!?;-)

Yes.

> Anyway, it's late for bullseye, as well: Debian testing is already in
> hard freeze...
>
> But I don't think there's anything really in need for a fix.
> Please read on.
>
> >
> >* What led up to the situation?
> >
> >Installed apt-listbugs and logcheck on a debian testing system.
> >Every *hour* I get email from logcheck that includes lines like this:
> >Apr 30 08:37:06 debtest systemd[1]: Starting Daily apt-listbugs
> > preferences cleanup...
> >Apr 30 08:37:06 debtest systemd[1]: Finished Daily apt-listbugs
> > preferences cleanup.
> > (Obviously logcheck and the email are incidental; the point is the
> >"Daily" cleanup is actually run hourly.  However, the emails are
> >part of why I find it annoying.)
>
> Does logcheck send e-mail messages for all the other systemd timers?
>
> As you may already know, you can get a list of active systemd timers on
> your box with the following command:
>
>   $ systemctl list-timers
>
Thanks for the tip; I wasn't familiar with list-timers.  There are 11
on my list, but the only things I see in my hourly reports from
logcheck were  apt-listbugs and complaints about time synchronization.
Of course, I don't know how many of the timers are hourly.  Note the
time sync messages reflect a real problem, and so seem appropriate.

I do get occasional messages reflecting successful runs, e.g.,
Apr 30 23:15:11 debtest systemd-timesyncd[38257]: Timed out waiting
for reply from 192.168.1.10:123 (192.168.1.10).
Apr 30 23:49:30 debtest systemd-timesyncd[38257]: Timed out waiting
for reply from 192.168.1.10:123 (192.168.1.10).
May  1 00:00:01 debtest systemd[1]: Starting exim4-base housekeeping...
May  1 00:00:01 debtest systemd[1]: Starting Daily man-db regeneration...
May  1 00:00:01 debtest systemd[1]: Finished exim4-base housekeeping.

> >
> >* What exactly did you do (or not do) that was effective (or
> >  ineffective)?
> >
> >  Investigated why this is happening.  The package has an entry in
> >/etc/cron.daily, which seems correct.  But it also has
> >/lib/systemd/system/apt-listbugs.timer which includes
> >OnCalendar=*-*-* *:20
> >I believe this is telling it to trigger something at 20 minutes
> >after the hour for every hour.  There is also
> >RandomizedDelaySec=20min
> >in the same file, which may explain why it seems to run between 20
> >and 40 minutes after the hour for me.
>
> Yes, that is correct.
> Please read bug [#932995] for a more detailed discussion about the same
> surprise you experienced.

Ah, I see: it's running, but it's mostly not *really* running (i.e.,
check but no update).

>
> [#932995]: 
>
> >
> >This seems more like a task for anacron for systems that may not be
> >up all the time, but I don't know enough about systemd to be sure
> >how to turn it off, or if it can handle these situations.
>
> If you use systemd as your init system (PID 1), please do not turn it
> off. It's the only apt-listbugs cleanup routine running, since the
> cron.daily job does nothing, if systemd is PID 1.

Good to know.  It's a standard bullseye install, and so systemd is in control.

>
> >
> >* What was the outcome of this action?
> >Only observations so far, and so no change.  I might change to
> >OnCalendar=*-*-* 20:20
> >which I think means run at 8:20pm every day.  But the VM is not up
> >all the time and so that might miss some times it should run,
> >unless the /etc/cron.daily/ entry acts as a safety net.
>
> That is exactly the reason why the cleanup is attempted hourly, but
> performed at most once a day.
> Again, see [#932995] for an explanation.
>
> >
> >* What outcome did you expect instead?
> >That a daily cleanup job would only run once a day.
>
> That's already happening: the cleanup runs at most once a day.
>

I tried this modification to apt-listbugs.timer:
[Timer]
OnActiveSec=5min
#OnCalendar=*-*-* *:20
OnUnitActiveSec=23h 50m
RandomizedDelaySec=20min

which did  fix the "running every hour" problem (even if the run is
only to check if a real  update is necessary).
But this does not entirely meet the desired behavior expressed in 932995:
 1) The "anniversary" time will be whenever the system started,
not early AM if available.
 2) At least with the current script, which just checks if it
should run, the result could be as
 much as a 48 hour delay between real updates.  This would
happen if the system was
 turned on just 

Bug#987839: apt-listbugs: daily cleanup runs hourly

[Francesco Poli]

Control: tags -1 + moreinfo


On Fri, 30 Apr 2021 10:46:04 -0700 Ross Boylan wrote:

[...]
> Dear Maintainer,

Hello Ross,
thanks for your bug report.

> 
> It would be nice to clean up this minor annoyance before buster's release.

I am afraid that we are too late for *buster* release.
Maybe you meant *bullseye*?!?;-)
Anyway, it's late for bullseye, as well: Debian testing is already in
hard freeze...

But I don't think there's anything really in need for a fix.
Please read on.

> 
>* What led up to the situation?
> 
>Installed apt-listbugs and logcheck on a debian testing system.
>Every *hour* I get email from logcheck that includes lines like this:
>Apr 30 08:37:06 debtest systemd[1]: Starting Daily apt-listbugs
> preferences cleanup...
>Apr 30 08:37:06 debtest systemd[1]: Finished Daily apt-listbugs
> preferences cleanup.
> (Obviously logcheck and the email are incidental; the point is the
>"Daily" cleanup is actually run hourly.  However, the emails are
>part of why I find it annoying.)

Does logcheck send e-mail messages for all the other systemd timers?

As you may already know, you can get a list of active systemd timers on
your box with the following command:

  $ systemctl list-timers

> 
>* What exactly did you do (or not do) that was effective (or
>  ineffective)?
> 
>  Investigated why this is happening.  The package has an entry in
>/etc/cron.daily, which seems correct.  But it also has
>/lib/systemd/system/apt-listbugs.timer which includes
>OnCalendar=*-*-* *:20
>I believe this is telling it to trigger something at 20 minutes
>after the hour for every hour.  There is also
>RandomizedDelaySec=20min
>in the same file, which may explain why it seems to run between 20
>and 40 minutes after the hour for me.

Yes, that is correct.
Please read bug [#932995] for a more detailed discussion about the same
surprise you experienced.

[#932995]: 

> 
>This seems more like a task for anacron for systems that may not be
>up all the time, but I don't know enough about systemd to be sure
>how to turn it off, or if it can handle these situations.

If you use systemd as your init system (PID 1), please do not turn it
off. It's the only apt-listbugs cleanup routine running, since the
cron.daily job does nothing, if systemd is PID 1.  

> 
>* What was the outcome of this action?
>Only observations so far, and so no change.  I might change to
>OnCalendar=*-*-* 20:20
>which I think means run at 8:20pm every day.  But the VM is not up
>all the time and so that might miss some times it should run,
>unless the /etc/cron.daily/ entry acts as a safety net.

That is exactly the reason why the cleanup is attempted hourly, but
performed at most once a day.
Again, see [#932995] for an explanation.

> 
>* What outcome did you expect instead?
>That a daily cleanup job would only run once a day.

That's already happening: the cleanup runs at most once a day.

> 
> Another work-around for the visible annoyance would be for me to tell
> logcheck to ignore the relevant messages.

I think this should be the way to avoid the annoyance.
I am not knowledgeable enough about logcheck to help you in this.
Sorry about that.

> 
> Thanks for your work on the package:)

You're welcome, I try to do my best...

If you do not object, I will close this bug report.


-- 
 http://www.inventati.org/frx/
 There's not a second to spare! To the laboratory!
. Francesco Poli .
 GnuPG key fpr == CA01 1147 9CD2 EFDF FB82  3925 3E1C 27E1 1F69 BFFE


pgpbMqLTlSKbo.pgp
Description: PGP signature

Bug#987839: apt-listbugs: daily cleanup runs hourly

[Ross Boylan]

Package: apt-listbugs
Version: 0.1.35
Severity: minor
X-Debbugs-Cc: rossboy...@stanfordalumni.org

Dear Maintainer,

It would be nice to clean up this minor annoyance before buster's release.

   * What led up to the situation?

   Installed apt-listbugs and logcheck on a debian testing system.
   Every *hour* I get email from logcheck that includes lines like this:
   Apr 30 08:37:06 debtest systemd[1]: Starting Daily apt-listbugs
preferences cleanup...
   Apr 30 08:37:06 debtest systemd[1]: Finished Daily apt-listbugs
preferences cleanup.
(Obviously logcheck and the email are incidental; the point is the
   "Daily" cleanup is actually run hourly.  However, the emails are
   part of why I find it annoying.)

   * What exactly did you do (or not do) that was effective (or
 ineffective)?

 Investigated why this is happening.  The package has an entry in
   /etc/cron.daily, which seems correct.  But it also has
   /lib/systemd/system/apt-listbugs.timer which includes
   OnCalendar=*-*-* *:20
   I believe this is telling it to trigger something at 20 minutes
   after the hour for every hour.  There is also
   RandomizedDelaySec=20min
   in the same file, which may explain why it seems to run between 20
   and 40 minutes after the hour for me.

   This seems more like a task for anacron for systems that may not be
   up all the time, but I don't know enough about systemd to be sure
   how to turn it off, or if it can handle these situations.

   * What was the outcome of this action?
   Only observations so far, and so no change.  I might change to
   OnCalendar=*-*-* 20:20
   which I think means run at 8:20pm every day.  But the VM is not up
   all the time and so that might miss some times it should run,
   unless the /etc/cron.daily/ entry acts as a safety net.

   * What outcome did you expect instead?
   That a daily cleanup job would only run once a day.

Another work-around for the visible annoyance would be for me to tell
logcheck to ignore the relevant messages.

Thanks for your work on the package:)

-- System Information:
Debian Release: bullseye/sid
  APT prefers testing-security
  APT policy: (500, 'testing-security'), (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 5.10.0-6-amd64 (SMP w/3 CPU threads)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages apt-listbugs depends on:
ii  apt 2.2.3
ii  ruby1:2.7+2
ii  ruby-debian 0.3.10+b4
ii  ruby-gettext3.3.3-2
ii  ruby-soap4r 2.0.5-5
ii  ruby-unicode0.4.4.4-1+b1
ii  ruby-xmlparser  0.7.3-4

Versions of packages apt-listbugs recommends:
ii  ruby-httpclient  2.8.3-2

Versions of packages apt-listbugs suggests:
ii  konqueror [www-browser]  4:20.12.0-4
ii  reportbug7.10.3
ii  sensible-utils   0.0.14
ii  xdg-utils1.1.3-4

-- no debconf information