Bug#955765: Error: Received invalid X.509 certificate from ACME server!

[Guilhem Moulin]

On Sun, 05 Apr 2020 at 01:22:46 +0200, Jonas Smedegaard wrote:
>> Ah, yes: The cause is that the system lack the hashes in /etc/ssl/certs.
>> 
>> Sorry for bothering you with a bug outside of lacme - sharing these 
>> details only in case it might inspire you to provide clues when lacme 
>> runs into weird scenarios like that.
> 
> In case you are curious, the cause is (most likely) bugs#923479 - my 
> server is an armhf box bootstrapped with QEMU.  I _did_ put a workaround 
> in place for that exact bug, but evidently that failed.

I see, thanks for the follow-up!  lacme *does* hostname verification by
default (ouf) but this is done internally by OpenSSL with
IO::Socket::SSL::default_ca() as default CA stores.


https://metacpan.org/pod/IO::Socket::SSL#IO::Socket::SSL::default_ca([-path|dir|-SSL_ca_file-=-...,-SSL_ca_path-=%3E-...-])%3E

On your system IO::Socket::SSL::default_ca() appears to have found a
suitable default.  But with bogus /etc/ssl/certs hashes verify(1ssl)
seems equivalent to -no-CApath, and I'm indeed able to reproduce the
“error 2 at 1 depth lookup” error if I disable /etc/ssl/certs hashes:

$ curl -s 
https://acme-v02.api.letsencrypt.org/acme/cert/036c9c4c3720c2241c7f32cb5920470555db
 \
| openssl verify -CAfile /usr/share/lacme/lets-encrypt-x3-cross-signed.pem 
-no-CApath -purpose sslserver -x509_strict
C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
error 2 at 1 depth lookup: unable to get issuer certificate
error stdin: verification failed

Let's close this, no? :-)

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#955765: Error: Received invalid X.509 certificate from ACME server!

[Guilhem Moulin]

On Sun, 05 Apr 2020 at 01:00:04 +0200, Jonas Smedegaard wrote:
> Weirdly the cause seems to be that curl doesn't get the cert at all:
> 
> debian@everton:~$ curl -s 
> https://acme-v02.api.letsencrypt.org/acme/cert/036c9c4c3720c2241c7f32cb5920470555db
> debian@everton:~$ echo $?
> 60
> 
> On another host I have no problem fetching the cert.
> 
> So seems like an issue unrelated to lacme :-/

Not sure if it's unrelated, curl(1)'s exit status 60 is “Peer
certificate cannot be authenticated with known CA certificates.” Is
ca-certificates installed on that machine?  It occurs to me that a
dependency might be missing here.

Oddly enough lacme is able to talk to the server, even though its
‘SSL_verify’ option defaults to ‘Yes’, which AFAICT causes
IO::Socket::SSL to use the peer verification logic from openssl.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#916564: python-pyte: new upstream version: 0.8.0

[Guilhem Moulin]

Hi Andrej,

On Tue, 1 Oct 2019 at 16:21:54 +0200, Andrej Shadura wrote
> Control: tag -1 pending

Anything still blocking for 0.8.0-1? :-)

> Why? 0.4.8 should be good enough for anyone for a few years more :)

FWIW termtosvg, which I just ITP'ed, needs pyte.graphics.FG_BG_256 which
was added in 0.6.0 :-P

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#955765: Error: Received invalid X.509 certificate from ACME server!

[Guilhem Moulin]

Control: tag -1 moreinfo

Hi Jonas,

On Sat, 04 Apr 2020 at 20:18:28 +0200, Jonas Smedegaard wrote:
> C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
> error 2 at 1 depth lookup: unable to get issuer certificate
> [live] Error: Received invalid X.509 certificate from ACME server!

This indicates that the received X.509 certificate isn't signed by the
CA specified as ‘CAfile’.  More precisely, that

openssl verify -CAfile $CAfile -purpose sslserver -x509_strict 
https://acme-v02.api.letsencrypt.org/acme/cert/036c9c4c3720c2241c7f32cb5920470555db
 \
  | openssl verify -CAfile 
/usr/share/lacme/lets-encrypt-x3-cross-signed.pem -purpose sslserver 
-x509_strict
stdin: OK

Does this command work on your system?  I've not been able to reproduce
the “error 2 at 1 depth lookup” error, but for a completely different CA
verify(1ssl) fails with:

$ curl -s 
https://acme-v02.api.letsencrypt.org/acme/cert/036c9c4c3720c2241c7f32cb5920470555db
 \
  | openssl verify -CAfile 
/usr/share/lacme/lets-encrypt-x1-cross-signed.pem -purpose sslserver 
-x509_strict
CN = live.homebase.dk
error 20 at 0 depth lookup: unable to get local issuer certificate
error stdin: verification failed

(Adding --debug will indicate the exact `openssl verify -CAfile …` that
fails.)

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#955738: ITP: termtosvg -- record terminal sessions as standalone SVG animations

[Guilhem Moulin]

Package: wnpp
Severity: wishlist
Owner: Guilhem Moulin 

* Package name: termtosvg
  Version : 1.1.0
  Upstream Author : Nicolas Bedos 
* URL : https://nbedos.github.io/termtosvg/
* License : BSD-3-clause
  Programming Lang: Python
  Description : record terminal sessions as standalone SVG animations

termtosvg is a Unix terminal recorder written in Python that renders
command line sessions as standalone SVG animations.  Features:

 * Produce lightweight and clean looking animations embeddable on a
   project page
 * Custom color themes, terminal UI and animation controls via SVG
   templates
 * Compatible with asciinema recording format

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#955384: dropbear-bin: dropbearconvert cannot convert SSH keys from OpenSSH to dropbear format

[Guilhem Moulin]

Control: severity -1 minor
Control: retitle -1 dropbear-bin: dropbearconvert doesn't understand new 
OpenSSH (>=7.8) private key format

On Mon, 30 Mar 2020 at 22:04:31 +0100, André Rodier wrote:
> The dropbear-bin package, comes with a tool to convert keys fron
> OpenSSL. This tool worked on Stretch, but is now broken on Buster.

dropbearconvert(1) only understands the legacy PEM private key format.
(It should be documented, but I guess at the time the tool was written
the new format didn't exist yet.)  Since 1:7.8p1-1 ssh-keygen(1) writes
private keys in a new format, see the OpenSSH changes and/or release
notes.  Keys can be generated and converted to PEM using ‘-m PEM’:

$ ssh-keygen -N "" -f /tmp/openssh.key
[…]
$ grep ^--- /tmp/openssh.key
-BEGIN OPENSSH PRIVATE KEY-
-END OPENSSH PRIVATE KEY-
$ dropbearconvert openssh dropbear /tmp/openssh.key /tmp/dropbear.key
Error: Unrecognised key type
Error reading key from '/tmp/openssh.key'

Convert to PEM:

$ ssh-keygen -p -N "" -m PEM -f /tmp/openssh.key
[…]
Your identification has been saved with the new passphrase.
$ grep ^--- /tmp/openssh.key
-BEGIN RSA PRIVATE KEY-
-END RSA PRIVATE KEY-
$ dropbearconvert openssh dropbear /tmp/openssh.key /tmp/dropbear.key
Key is a ssh-rsa key
Wrote key to '/tmp/dropbear.key'

Or generate to PEM directly:

$ ssh-keygen -N "" -m PEM -f /tmp/openssh.key
[…]
$ grep ^--- /tmp/openssh.key
-BEGIN RSA PRIVATE KEY-
-END RSA PRIVATE KEY-
$ dropbearconvert openssh dropbear /tmp/openssh.key /tmp/dropbear.key
Key is a ssh-rsa key
Wrote key to '/tmp/dropbear.key'

I'm keeping that bug open but it's really not that important, at most a
documentation issue or a wishlist upstream bug to add support for the
new OpenSSH format in dropbearconvert(1).

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#817050: netcat-openbsd differs from netcat-traditional when passed "-u -q0"

[Guilhem Moulin]

On Sun, 29 Mar 2020 at 18:02:00 +0200, Guilhem Moulin wrote:
>> Using TCP, ^D to server nc has no visible efect. Maybe an update to your 
>> patch
>> can fix this also. I will investicate over the next few days.
> 
> Sorry, my bad for claiming earlier that ‘-q0’ was an alias ‘-N’.  What's
> correct is what's written in the manual, namely that it *implies* ‘-N’.
> All ‘-N’ does is to call shutdown(2) after EOF on stdin; which it does,
> so I'm removing the ‘upstream’ tag on the bug.

Attached is an updated version that also causes nc to exit on EOF when
‘-l’ is set.  Will test it shortly :-)

-- 
Guilhem.
--- a/netcat.c
+++ b/netcat.c
@@ -1607,6 +1607,12 @@
 			if (pfd[POLL_NETOUT].fd != -1 && Nflag)
 shutdown(pfd[POLL_NETOUT].fd, SHUT_WR);
 			pfd[POLL_NETOUT].fd = -1;
+			if ((lflag || uflag) && pfd[POLL_NETIN].fd != -1 &&
+			qflag >= 0 && netinbufpos == 0) {
+shutdown(pfd[POLL_NETIN].fd, SHUT_RD);
+pfd[POLL_NETIN].fd = -1;
+kflag = 0;
+			}
 		}
 		/* net in gone and queue empty? */
 		if (pfd[POLL_NETIN].fd == -1 && netinbufpos == 0) {


signature.asc
Description: PGP signature

Bug#817050: netcat-openbsd differs from netcat-traditional when passed "-u -q0"

[Guilhem Moulin]

Control: tag -1 - upstream

Hi Duncan,

On Sun, 08 Mar 2020 at 14:25:50 +1100, Duncan Roe wrote:
>> The enclosed patch seems to work for me.  (Untested though, so not
>> pushing just yet.)
> 
> Good idea not to apply that patch. After applying, I can send server nc into a
> hard loop by entering ^D after it has responded to the its 2nd client, as per
> attached scenario.
> 
> Should be a simple fix.

Thanks for testing and fixing the patch :-)  I'm reattaching what you
sent me privately so it shows in the BTS.

> Using TCP, ^D to server nc has no visible efect. Maybe an update to your patch
> can fix this also. I will investicate over the next few days.

Sorry, my bad for claiming earlier that ‘-q0’ was an alias ‘-N’.  What's
correct is what's written in the manual, namely that it *implies* ‘-N’.
All ‘-N’ does is to call shutdown(2) after EOF on stdin; which it does,
so I'm removing the ‘upstream’ tag on the bug.

> Input in parentheses
> 
> Screen 1 Screen 2
>13:40:42$ (nc -k -u -N -6 -l ::1 1042)
> 13:42:11$ (nc -u -N -6 ::1 1042)
> (aaa)
>aaa
> (^D)
> 13:42:50$
> (nc -u -N -6 ::1 1042)
> (bbb)
>bbb
>(^D)
> 
> Screen 2 nc is now in a hard loop:
> 
>> poll([{fd=0, events=POLLIN}, {fd=3, events=0}, {fd=3, events=POLLIN}, {fd=1, 
>> events=0}], 4, -1) = 2 ([{fd=3, revents=POLLHUP}, {fd=3, 
>> revents=POLLIN|POLLHUP}])
>> shutdown(3, SHUT_WR)= -1 ENOTCONN (Transport endpoint is 
>> not connected)
>> read(3, "", 16384)  = 0
>> shutdown(3, SHUT_RD)= -1 ENOTCONN (Transport endpoint is 
>> not connected)
>> poll([{fd=0, events=POLLIN}, {fd=3, events=0}, {fd=3, events=POLLIN}, {fd=1, 
>> events=0}], 4, -1) = 2 ([{fd=3, revents=POLLHUP}, {fd=3, 
>> revents=POLLIN|POLLHUP}])
>> shutdown(3, SHUT_WR)= -1 ENOTCONN (Transport endpoint is 
>> not connected)

-- 
Guilhem.
--- a/netcat.c
+++ b/netcat.c
@@ -1431,8 +1431,11 @@ readwrite(int net_fd)
 		/* both inputs are gone, buffers are empty, we are done */
 		if (pfd[POLL_STDIN].fd == -1 && pfd[POLL_NETIN].fd == -1 &&
 		stdinbufpos == 0 && netinbufpos == 0) {
-			if (qflag <= 0)
+			if (qflag <= 0) {
+if (kflag)
+	kflag = 0;
 return;
+			}
 			goto delay_exit;
 		}
 		/* both outputs are gone, we can't continue */
@@ -1607,6 +1610,10 @@ delay_exit:
 			if (pfd[POLL_NETOUT].fd != -1 && Nflag)
 shutdown(pfd[POLL_NETOUT].fd, SHUT_WR);
 			pfd[POLL_NETOUT].fd = -1;
+			if (uflag && pfd[POLL_NETIN].fd != -1 && Nflag && netinbufpos == 0) {
+shutdown(pfd[POLL_NETIN].fd, SHUT_RD);
+pfd[POLL_NETIN].fd = -1;
+			}
 		}
 		/* net in gone and queue empty? */
 		if (pfd[POLL_NETIN].fd == -1 && netinbufpos == 0) {


signature.asc
Description: PGP signature

Bug#911331: ploticus: randomly fails drawing lines

[Guilhem Moulin]

Control: usertag -1 bsp-2020-03-se-gothenburg

Hi,

On Thu, 18 Oct 2018 at 21:19:51 +0300, Antti Kuparinen wrote:
> Lines are sometimes drawn with one end extending to lower left.
> Rendering the same input might usually look fine, but fail randomly.
> Example input and output of 100 iterations attached.
> 
> The cause seems to be reading outside of allocated memory on line
> execline.c:489. This leads to a wrong argument count causing the
> following warning:
>
> pl proc line:  2959: warning: points must have either 4 or 2 values per line

Just uploaded 2.42-4.1 but I didn't include your patch as it doesn't
apply after applying ‘debian/patches/upstream-bug-fixes.patch’ (which
seems to be a fix for 
https://bugs.launchpad.net/ubuntu/+source/ploticus/+bug/692567).
CC'ing upstream for feedback.

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#880778: ploticus build depends on removed libgd2*-dev provides

[Guilhem Moulin]

Control: tag -1 pending
Control: usertags -1 bsp-2020-03-se-gothenburg

Hi there, about to NMU with the attached debdiff.

-- 
Guilhem.
diffstat for ploticus-2.42 ploticus-2.42

 changelog |8 
 control   |2 +-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff -Nru ploticus-2.42/debian/changelog ploticus-2.42/debian/changelog
--- ploticus-2.42/debian/changelog  2017-07-25 10:58:08.0 +0200
+++ ploticus-2.42/debian/changelog  2020-03-29 11:06:52.0 +0200
@@ -1,3 +1,11 @@
+ploticus (2.42-4.1) UNRELEASED; urgency=medium
+
+  * Non-maintainer upload.
+  * d/control: Replace removed libgd2-noxpm-dev build-deb with libgd-dev to
+fix FTBS (closes: #880778)
+
+ -- Guilhem Moulin   Sun, 29 Mar 2020 11:06:52 +0200
+
 ploticus (2.42-4) unstable; urgency=medium
 
   * Fix debian/watch - thanks to Iain Learmonth
diff -Nru ploticus-2.42/debian/control ploticus-2.42/debian/control
--- ploticus-2.42/debian/control2017-07-25 10:57:12.0 +0200
+++ ploticus-2.42/debian/control2020-03-29 11:04:16.0 +0200
@@ -3,7 +3,7 @@
 Priority: optional
 Maintainer: Colin Tuckley 
 Homepage: http://ploticus.sourceforge.net
-Build-Depends: debhelper (>= 9), zlib1g-dev, libgd2-noxpm-dev, libjpeg-dev, 
libfreetype6-dev, libx11-dev, x11proto-core-dev, libpng-dev, dpkg-dev (>= 
1.16.1~)
+Build-Depends: debhelper (>= 9), zlib1g-dev, libgd-dev, libjpeg-dev, 
libfreetype6-dev, libx11-dev, x11proto-core-dev, libpng-dev, dpkg-dev (>= 
1.16.1~)
 Standards-Version: 4.0.0
 
 Package: ploticus


signature.asc
Description: PGP signature

Bug#954934: MD5SIG connection support

[Guilhem Moulin]

Control: tag -1 pending

On Wed, 25 Mar 2020 at 09:32:38 -0400, tho...@habets.se wrote:
> I have now made patches for TCP_MD5SIG that fixes both of these
> issues.

Thanks, applied: 
https://salsa.debian.org/debian/netcat-openbsd/-/commit/04482d7bdaa49dfef2126683b017c1e8d5047525
 .

> I have emailed the Debian package maintainer for netcat-openbsd with
> these patches for a few months and have not gotten any reply.

Please use the BTS (or salsa) to interact with package maintainers :-P

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#817050: netcat-openbsd differs from netcat-traditional when passed "-u -q0"

[Guilhem Moulin]

Control: tag -1 patch

On Fri, 06 Mar 2020 at 17:50:38 +0100, Guilhem Moulin wrote:
> Also, calling exit() in fillbuf() is the wrong approach as the buffer
> might not have been drained yet.

The enclosed patch seems to work for me.  (Untested though, so not
pushing just yet.)  Since for UDP sockets we won't get revents=POLLIN
after calling ‘shutdown(pfd[POLL_NETOUT].fd, SHUT_WR)’, a workaround is
to set pfd[POLL_NETIN].fd to -1 as well at the end of readwrite().

-- 
Guilhem.
--- a/netcat.c
+++ b/netcat.c
@@ -1607,6 +1607,10 @@
 			if (pfd[POLL_NETOUT].fd != -1 && Nflag)
 shutdown(pfd[POLL_NETOUT].fd, SHUT_WR);
 			pfd[POLL_NETOUT].fd = -1;
+			if (uflag && pfd[POLL_NETIN].fd != -1 && Nflag && netinbufpos == 0) {
+shutdown(pfd[POLL_NETIN].fd, SHUT_RD);
+pfd[POLL_NETIN].fd = -1;
+			}
 		}
 		/* net in gone and queue empty? */
 		if (pfd[POLL_NETIN].fd == -1 && netinbufpos == 0) {


signature.asc
Description: PGP signature

Bug#952703: [cryptsetup] needs breaks on old cryptsetup-bin

[Guilhem Moulin]

On Fri, 28 Feb 2020 at 05:40:10 +, jnq...@gmail.com wrote:
> edit: ah, so I see there were two reorganisations, one in June 2018 and
> one more in June 2019...

More precisely, one two-steps reorganisation with a Debian release in
between ;-)
 
> Indeed if I had performed a stable->stable->stable upgrade then I would
> not have encountered the error. I only question whether limiting
> upgrade compatibility so strictly is the best idea. Especially when a
> couple of breaks entries is a such a tiny things for a package to carry
> and enhances robustness.

As I wrote previously, adding the break isn't the only thing that would
have made your upgrade you smooth, the postinst script would like need
to be adapted too. 

While the side effect isn't intentional, not having the Breaks: on older
cryptsetups causes dpkg to fail, which I'd argue is better than ending
up with a broken (unbootable) systems.  Sure, we could test upgrade
paths from very old src:cryptsetups, but I'm not personally interested
in working on something that we as a project don't support.  Not keen
about the extra clutter either.  So right now I'm leaning towards
closing this as ‘wontfix’. :-P

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#952703: [cryptsetup] needs breaks on old cryptsetup-bin

[Guilhem Moulin]

On Thu, 27 Feb 2020 at 23:55:49 +, jnq...@gmail.com wrote:
> It is thus not a question of 'FrankenDebian' but of how old a sid
> installation upgrading is supported for.

The starting baseline doesn't matter, if you start from sid and don't
upgrade for longer than a full recycle, then most of your packages are
likely to be as old as the ones in oldstable.  Mixing stable and sid
packages is asking for trouble, but at least the upgrade path from
stable to sid is tested (the package in sid will be transitioning to
testing, which is stable +1); upgrading from *old*stable to testing/sid
is not, regardless whether if it's a selective or full upgrade.

> I don't think too much focus should be paid to the fact of just how old
> the version is in this case. Surely this affects all sid/testing
> installations that have not been upgraded since the reorganisation of
> cryptsetup began (~May 2019), and became an issue at some point towards
> the end of the reorganisation, which nobody encountered or bothered to
> report until now.

We started the package split in *June 2018* with 2:2.0.3-1, and ended July
2019 with 2:2.1.0-6.  Between the two buster was released, and it ships
2:2.1.0-5 (+deb10u2 right now).  So your upgrade leaps over a debian
stable release, which is not something supported.  If you had upgraded
from stretch's cryptsetup to buster's *and then* to bullseye/sid's, and
the upgrade path had broken, then I would have raised the severity to RC.

> Also, big upgrades, such as stretch-> buster do not always go cleanly,
> for which users may resort to performing some degree of selective
> upgrades in their upgrade path. Would they not potentially run into
> this error doing `aptitude upgrade cryptsetup` in that situation for
> such a stretch to buster upgrade in this case?

No, that's stable → stable+1 so fully supported.

FYI you don't even need bullseye/sid's cryptsetup for LUKSv2.  It's even
mentioned in buster's release notes:
https://www.debian.org/releases/buster/amd64/release-notes/ch-whats-new.en.html#cryptsetup-luks2

> I think it should be simple policy that when things get moved about
> between packages, that a breaks is specified for the older versions to
> prevent such dpkg file-clash errors that could occur in situations such
> as selective upgrades, and remains permanently in place for a
> reasonable number of years to avoid any issues for a reasonably wide
> range of scenarios.

And that's what we did, except that we count in release cycles not in
years.  Passed a full release cycle we decided to drop the clutter,
because going from oldstable to testing (or stable to stable+2), i.e.
skipping an upgrade, has never been something supported.  This is
something stated in each release note, see for instance

“Direct upgrades from Debian releases older than 9 (stretch) are not
 supported. Please follow the instructions in the Release Notes for
 Debian 9 to upgrade to Debian 9 first.”
— 
https://www.debian.org/releases/buster/amd64/release-notes/ch-upgrading.en.html

Or 

“Please always upgrade to Debian Testing from current stable.
Upgrading from oldstable is not supported and might encounter
unexpected errors.”
— https://wiki.debian.org/DebianTesting

Also, the package relationship isn't the only thing to care about when
upgrading.  There are also warning for deprecated settings which are
later removed, compatibility symlinks, etc.  Skipping (Debian) releases
upon upgrades has never been something supported, and we decided to
simply remove the clutter.  For the same reason we might just close this
as ‘wontfix’.

> I expected that a simple oversight had been made at the time the
> cryptsetup reorganisation was done, though your focus on upgrade paths
> suggests otherwise, that you perhaps do not give as much consideration
> to supporting sid & testing installations as stable ones?

Uh?  testing will be the next stable, so we're very careful about what
goes inside.  And same thing for sid.  Sorry, a sid system deployed
before stretch was released, with a direct (selective or full) upgrade
to sid isn't a sid system, that's very much a FrankenDebian.

> edit: I just tried looking up Debian policy regarding sid/unstable and
> breaks info. I came across 
> https://www.debian.org/doc/debian-policy/ch-relationships.html, but
> nothing specific about 'sid'/'unstable' was mentioned, nor in fact
> 'stable'... nor in https://www.debian.org/doc/debian-policy/

We have proper package relationship in place to support upgrading from
stretch to buster (or more precisely, from ≥2:1.7.3-4, <2:2.1.0-5 to 2:2.1.0-5)
and others fro buster to bullseye/sid (or more precisely, from
≥2:2.1.0-5, <2:2.2.2-3 to 2:2.2.2-3, or whichever version we'll ship to
bullseye). 

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#952703: [cryptsetup] needs breaks on old cryptsetup-bin

[Guilhem Moulin]

Control: retitle -1 unsupported upgrade from  ```
> dpkg: error processing archive
> /var/cache/apt/archives/cryptsetup_2%3a2.2.2-3_deb (--unpack):
> trying to overwrite '/usr/sbin/luksformat', which is also in package
> cryptsetup-bin 2:1.7.0-2
> ```

That seems to be an upgrade from a package version even older that
stretch's (stretch has 2:1.7.3-4) all the way to bullseye's.  That's not
a supported upgrade path.  The only supported one is stable → stable+1
(for which we have suitable Breaks relations in place),
https://wiki.debian.org/DontBreakDebian#Don.27t_make_a_FrankenDebian .

> I expect you forgot to place a suitable breaks on the older packages
> after the reorganisation.
> […]
> Clearly what should be expected from a targetted upgrade specifying
> just `cryptsetup` and not `cryptsetup cryptsetup-bin
> cryptsetup-initramfs` should be that it either pulls in all three as
> necessary upgrades or at least complains about version conflicts. It
> should never just fail like this.

I suppose adding Breaks on pre-Buster packages won't hurt so I'm keeping
that open (with a lower severity) but you're very much on your own once
you start mixing releases like this :-P

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#951194: [Pkg-roundcube-maintainers] Bug#951194: roundcube-core: update should not change config file permissions

[Guilhem Moulin]

Control: found -1 1.2.3+dfsg.1-4+deb9u3
Control: severity -1 serious
Control: tag -1 + confirmed pending

Hi,

On Wed, 12 Feb 2020 at 10:40:22 +0100, Daniel wrote:
> After an `apt upgrade` which upgrade roundcube-core, the group of 
> /etc/roundcube/config.inc.php 
> has changed, set to www-data,

Ack, and oldstable is affected too (and probably all versions before
that as the chown can already be found in 0.1-1).  Making this RC since
it's AFAICT a violation of Policy §10.7.3 (assuming ownership and/or
mode changes count as local changes, which sounds sensible).

> which in my case broke roundcube (the php user handling roundcube
> is not in this group)

Note that changing ownership and/or mode of config files (config.inc.php +
debian-db.php) might not be enough, one might need to apply similar
changes to /var/log/roundcube and /var/lib/roundcube/temp.  As these are
not configurations files, ownership and/or mode changes are usually made
sticky using dpkg-statoverride(1).  Unfortunately we didn't honor these
overrides either. :-/

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#950254: initramfs-tools: Please make libpthread pull in libgcc_s

[Guilhem Moulin]

On Mon, 03 Feb 2020 at 13:22:44 +0100, Vincent Bernat wrote:
> The hack in cryptroot hook doesn't find it anymore.

FWIW a mitigation was meanwhile uploaded as cryptsetup-initramfs 2:2.2.2-3,
cf. #950628.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#939766: [pkg-cryptsetup-devel] Bug#939766: Bug#939766: cryptsetup-initramfs: Trying to boot linux-image-5.2.0-2-amd64 fails, linux-image-4.19.0-5-amd64 works.

[Guilhem Moulin]

On Tue, 14 Jan 2020 at 03:22:20 +0100, Guilhem Moulin wrote:
> If there is a need for a complex logic I guess this could be done once
> and for all in /usr/share/initramfs-tools/hook-functions.

Quick follow up about this: asked Ben for feedback and he agreed that it
would make sense to do it in initramfs-tools.  That's #950254; I'm not
merging the bugs on closing -1 until this is actually implemented, but
we'll hopefully have a fine solution for all packaging using
pthread_cancel() & friends soon :-)

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#916649: [pkg-cryptsetup-devel] Bug#916649: `/etc/init.d/cryptdisks stop` should ignore devices holding / and /usr

[Guilhem Moulin]

Control: tag -1 pending

Hi there,

I believe


https://salsa.debian.org/cryptsetup-team/cryptsetup/commit/863e91f0e763b92a5f70d84278116a28357e74eb

should fix this, but I had to refactor a bit so would appreciate some
extra tests before releasing the fix.  You can apply the above manually
(after its parent 1d97d98a), but I can also upload to experimental if
you prefer :-)

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#950254: initramfs-tools: Please make libpthread pull in libgcc_s

[Guilhem Moulin]

Package: initramfs-tools
Version: 0.136
Severity: wishlist

Dear Maintainer,

libthread appears to load libgcc_s via dlopen, so copy_exec() won't copy
it to the initramfs image.  This causes pthread_cancel to fail with

LIBGCC_S_SO must be installed for pthread_cancel to work

https://sources.debian.org/src/glibc/2.29-9/fbtl/sysdeps/pthread/unwind-forcedunwind.c/#L64

Some packages (cryptsetup-initramfs, open-iscsi, btrfs-progs, and
probably others) have added brittle workarounds to their initramfs hooks
to copy libgcc_s.  As discussed here at HSBXL, please consider making
libthread pull libgcc_s.  (Or alternatively, providing a helper function
so these packages don't need to reinvent the wheel.)

Thanks,
cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#949888: [pkg-cryptsetup-devel] Bug#949888: cryptsetup-initramfs: cryptroot hook doesn't recognize devices with authenticated encryption

[Guilhem Moulin]

On Wed, 29 Jan 2020 at 21:25:02 +0100, hede wrote:
> On Wed, 29 Jan 2020 20:17:28 +0100 Guilhem Moulin  wrote:
> 
>> Given upstream's loud warning in cryptsetup(8) “WARNING: All support for
>> authenticated modes is experimental”, fixing this is not personally
>> really high on my list for the moment :-P
> 
> That's understandable. But don't say you haven't been warned the time
> when authenticated encryption is stable with cryptsetup but using it
> with debian is still prone to errors like this one. ;-)

FWIW I wrote the code making the hook print a warning on “capi:”
prefixes, so I was already self-warned :-P

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#949888: [pkg-cryptsetup-devel] Bug#949888: cryptsetup-initramfs: cryptroot hook doesn't recognize devices with authenticated encryption

[Guilhem Moulin]

Hi,

On Sun, 26 Jan 2020 at 17:34:50 +, hede wrote:
> I've switched to Authenticated Encryption, i.e:
> […]
> Nevertheless, the initramfs doesn't get created as expected. 

Given upstream's loud warning in cryptsetup(8) “WARNING: All support for
authenticated modes is experimental”, fixing this is not personally
really high on my list for the moment :-P

On Sun, 26 Jan 2020 at 18:44:28 +0100, hede wrote:
> cryptsetup: WARNING: Couldn't determine cipher modules to load for sdXX_crypt 
>   (kernel crypto API format isn't supported yet)
> 
> -- it seems also here the sdXX_crypt_dif should be used!?

No, the source device is fine AFAICT, but `dmsetup table` lists its
cipher in crypto API format, for which we have no parser yet.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/admin-guide/device-mapper/dm-crypt.rst#n34

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#891410:

[Guilhem Moulin]

Hi Christoph,

On Mon, 27 Jan 2020 at 07:23:02 +0100, Christoph Biedl wrote:
> As far as I understand however, while such a situation _can_ happen, it
> is not very likely.

FWIW our last heavy refactoring was in spring/summer 2018.  We obviously
won't change the internal interface just “for fun”, but I'm not making
the promise it won't change again without prior notice :-P  (In the past
I've tried to use sources.debian.org and chase uses of our internal
interfaces, cf. #904899 for instance, but that's quite tedious and I'm
unsure I'll do the same next time.)

> So my suggestions, in decreasing order of preference:
>
> * Creating/using an API for the initramfs code, something you suggested in
> another message. Honestly, I haven't checked the code yet whether this
> is feasablle.

In case cryptsetup-initramfs' current interface is not expressive enough
I'm open to extend it.  What I object to though, is to be “forced” to
freeze & publish our internal interface because some projects blindly
started relying on it.

> For the moment I'll mostly likely do an upload to experimental.

Ack, and just to be clear I won't stand in the way here, only trying to
warn you about possible RC bugs further down the road :-P  (And bugs
that might be difficult to fix gracefully in case the fix requires
updating /etc/crypttab.)

> I've been waiting for this feature for a long time, therefore I'd like
> to collect first experiences with it soon. We should have a sound
> solution before the freeze which is still several months away.¹
> […]
> ¹ But keep in mind dates in calendars are closer than they appear.

Hehe :-)

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#948501: [Pkg-roundcube-maintainers] Bug#948501: Wrong Dependency

[Guilhem Moulin]

Hi,

On Sat, 25 Jan 2020 at 02:16:01 +0100, mar...@mitzlaff.eu wrote:
> I just faced a similar problem. I tried to install the roundcube 1.4.2 from
> Sid into my Buster system.

Doesn't seem to match Marc's environment.

> I think roundcube 1.4.2 needs a dependency to libjs-bootstrap4 (>=4.4.1).

See https://bugs.debian.org/948011#15 .

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#891410:

[Guilhem Moulin]

On Wed, 22 Jan 2020 at 14:37:54 -0600, Mario Limonciello wrote:
> Would you mind suggesting something upstream with the relevant changes
> that make sense?

As written earlier in this bug our public interface for this kind of
things is to use keyscripts.  See crypttab(5) for the few environment
variables are exported to the script, and we're also discussing
documenting a some helper functions in #901795.

Unfortunately, that means users who have a working crypttab now will
have to change it to use our interface.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#949623: [pkg-cryptsetup-devel] Bug#949623: cryptsetup: cryptdisks_stop/start bash completion broken

[Guilhem Moulin]

Control: found -1 2:1.7.0-1

On Wed, 22 Jan 2020 at 23:28:37 +0100, Guilhem Moulin wrote:
> If that's a regression it's older than 2:2.0.3-1, AFAICT stretch has
> the same problem.

Before 2:1.7.0-1 the completion file was copied into
/etc/bash_completion.d, which AFAICT bash traverses and sources at
startup.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#949623: [pkg-cryptsetup-devel] Bug#949623: cryptsetup: cryptdisks_stop/start bash completion broken

[Guilhem Moulin]

Control: tag -1 moreinfo
Control: severity -1 minor

On Wed, 22 Jan 2020 at 23:08:32 +0100, Christoph Anton Mitterer wrote:
> in .bashrc, and that's enough for all other completions... e.g. the one
> for cryptsetup work (more or less).

~$ foo looks for /usr/share/bash-completion/completions/foo (and
also with a ‘.bash’ extension and a ‘_’ prefix.  We therefore need to
ship /usr/share/bash-completion/completions/cryptdisks_start (plus a
symlink to cryptdisks_stop, or just split it) if we want autocompletion
to work out of the box.  If that's a regression it's older than
2:2.0.3-1, AFAICT stretch has the same problem.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#949623: [pkg-cryptsetup-devel] Bug#949623: cryptsetup: cryptdisks_stop/start bash completion broken

[Guilhem Moulin]

Control: tag -1 moreinfo

On Wed, 22 Jan 2020 at 22:31:49 +0100, Christoph Anton Mitterer wrote:
> Instead of possible device target names it just completes to
> the files of the local directories.

Works fine here, also as non-root.  In a sid chroot:

~$ dd if=/dev/zero of=/tmp/disk.img bs=1M count=64 status=none
~$ /sbin/cryptsetup luksFormat /tmp/disk.img --pbkdf-force-iterations 4 
--pbkdf-memory 32 <<>"$TABFILE"
~$ . /usr/share/bash-completion/completions/cryptdisks
~$ /usr/sbin/cryptdisks_start 
--  --readonly  -r  test_crypt
~$ /usr/sbin/cryptdisks_stop 

Did you source /usr/share/bash-completion/completions/cryptdisks?  Is
the TABFILE environment variable set to something else than your normal
crypttab(5)?

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#949336: Mapped integrity devices of size ≥2TiB are unusable on 32-bits platforms

[Guilhem Moulin]

Control: tag -1 + moreinfo

On Mon, 20 Jan 2020 at 21:25:56 +0100, Milan Broz wrote:
> but I definitely need a clear reproducer (with the latest stable - 2.2.2 or 
> 2.3.0-rc0) - ideally with attached debug and system log.
> (Debug log will provide all versions I need - kernel and dm targets versions 
> are important here)
> 
> It could be also kernel problem, so if you can attach .config for the kernel, 
> it helps.

Ack, tagging this ‘moreinfo’ in the meantime.  n.b.f., assuming you're using
the stock kernel you'll find the .config under /boot/config-$VERSION-$ARCH.
As of 2:2.2.2-2 we don't have custom Debian patches against the libcryptsetup
and cryptsetup binary we're shipping.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#891410:

[Guilhem Moulin]

On Mon, 20 Jan 2020 at 07:12:37 -0600, Mario Limonciello wrote:
> FYI, the newly released version 12 has initramfs support.

Hmm.  I guess I'm part of the problem since I haven't found time to help
with this unfortunately, but on a quick look it appears that my comments
from msg#27 and msg#32 still hold.

cryptsetup's initramfs integration isn't part of cryptsetup upstream,
but is development and maintained by the Debian packaging team, which
I'm part of.  AFAICT clevis upstream seem to have taken shell scripts
from src:cryptsetup and adapted them to their needs.  Might work right
now, but these file use *internal* / *undocumented* interfaces which are
subject to change without notice.  I fear clevis initramfs users have a
real risk of ending up with an unbootable system when we do update these
interfaces.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#949336: Mapped integrity devices of size ≥2TiB are unusable on 32-bits platforms

[Guilhem Moulin]

Control: tag -1 upstream

Hi,

On Sun, 19 Jan 2020 at 23:41:15 +, n...@waifu.club wrote:
> clarification: I am testing it with a volume I created and used with
> cryptsetup 2:2.0. With 2:2.1 and 2:2.2 integritysetup-open seems to succeed,
> but the embedded ext4 filesystem cannot be used. Attempt to read the
> superblock raise I/O errors due to integrity mismatches.

I won't be able to help debugging this as I don't have a >2TiB disk
around.  (Could fake the size and format with --no-wipe, but the
resulting device contains invalid checksums for obvious reasons,
regardless of the architecture.)  Milan, should I forward that upstream
(verbatim)?

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#939766: [pkg-cryptsetup-devel] Bug#939766: cryptsetup-initramfs: Trying to boot linux-image-5.2.0-2-amd64 fails, linux-image-4.19.0-5-amd64 works.

[Guilhem Moulin]

Hi Guillem!

On Tue, 14 Jan 2020 at 02:01:09 +0100, Guillem Jover wrote:
> The problem I've got is that I upgraded to gcc-10 from experimental,
> which pulled in libgcc1-s1 and libgcc1 (that I later removed), neither
> of which install libgcc_s.so.1 under /lib/$MULTIARCH anymore. The first
> installs the shared library under /usr/lib/$MULTIARCH, the latter
> under /lib/ (although that one might be in error, as there's an empty
> multiarch dir under /lib in that package), so that logic does not find
> it.

I see, thanks for the follow-up!

> IMO the assumption that libgcc will be located in the same directory
> as libc is incorrect, first because it comes from a different source
> package, and second because nothing says they need to be on the same
> directory. :)

Fair enough, and also ldd(1)'s output isn't meant to be parsable.
Finding libgcc that way a dirty hack which has served us well so far.

Not sure which one was first but FWIW there are a couple of other
packages using the same libc-based logic to determine which runtime
library to copy into the initramfs:

https://codesearch.debian.net/search?q=%2Flibc%5C%5C.so=0

but rather than reinventing the wheel each on our end I'd welcome a tip
to find where libgcc_s (and other runtime libraries such as libnss) is
installed in a robust fashion.  At least the one needed for a given
binary, but if there are multiple versions I guess shipping them all
wouldn't cause too much overhead.  Parsing the output of `ldconfig -p`
sounds a bit overkill, but perhaps it's better (or at least not worse,
the output isn't meant to be parsable either AFAICT).  If there is a
need for a complex logic I guess this could be done once and for all in
/usr/share/initramfs-tools/hook-functions.

Otherwise, I guess we could amend the sed script to allow an optional
/usr prefix (until the library moves elsewhere again).

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#948593: Unable to open LUKS device (error allocating crypto tfm) for aes / cbc-essiv:sha256 sha1 LUKS header

[Guilhem Moulin]

Control: retitle -1 cryptsetup-initramfs: Can't open aes-cbc-essiv:sha256 
dm-crypt targets with a 5.4 kernel and an initramfs built with MODULES=dep
Control: found -1 2:1.6.6-5
Control: tag -1 pending

Hi,

On Mon, 13 Jan 2020 at 08:47:43 +0100, Didier 'OdyX' Raboud wrote:
>> Devices formatted since 2:1.6.1-1 (June 2013) use XTS by default and
>> AFAICT aren't affected.  For other devices and when the initramfs is built
>> with MODULES!="most" I guess we should change populate_CRYPTO_MODULES() so
>> the ivmode is appended too, not only cipher+chainmode+ivopts.
> 
> https://sources.debian.org/src/cryptsetup/2:2.2.2-1/debian/initramfs/hooks/cryptroot/?hl=318#L318
> 
> That'd be useful yes!

This should fix it: 
https://salsa.debian.org/cryptsetup-team/cryptsetup/commit/6b75e4bda81ec63f42c46368e7b078c827ef0aad
 .
AFAICT all versions of the initramfs hook are affected since 2006 (but
only on 5.4 kernels and for initramfs images built with MODULES=dep).

Kernel modules named after the IV generator are now added to the
initramfs image when found under /kernel/crypto/.  If there is no
matching modules (for instance with aes-cbc-essiv:sha256 on older
kernels, or with aes-xts-plain64 on any kernel) the initramfs image
should be identical.

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#948593: [pkg-cryptsetup-devel] Bug#948593: Unable to open LUKS device (error allocating crypto tfm) for aes / cbc-essiv:sha256 sha1 LUKS header

[Guilhem Moulin]

Hi OdyX,

On Sat, 11 Jan 2020 at 11:56:35 +, Didier 'OdyX' Raboud wrote:
> From diffing the initramfs'es, I see that kernel/arch/x86/crypto/aes-x86_64.ko
> was present in 5.3.0-3 kernels, but not present anymore in 5.4.0-1 or 5.4.0-2
> kernels.

kernel/arch/x86/crypto/aes-x86_64.ko isn't in 5.4.0-2's module tree.  Do
you build the initramfs with MODULES="most", MODULES="dep", or something
else?  Looking at the output of

cut -d" " -f1 /proc/modules | xargs -d"\\n" /sbin/modinfo -F filename | 
grep /crypto/

before and after (formatting and) opening a cbc-essiv:sha256 device
with

$ cryptsetup luksFormat --type luks1 --cipher aes-cbc-essiv:sha256 --hash 
sha1 /tmp/disk.img <<

signature.asc
Description: PGP signature

Bug#948501: [Pkg-roundcube-maintainers] Bug#948501: roundcube-core: Elastic theme deps not accessible

[Guilhem Moulin]

Control: tag -1 unreproducible moreinfo

On Thu, 09 Jan 2020 at 23:39:57 +0900, Marc Dequènes wrote:
> The new Elastic theme dependencies cannot be loaded.

Works just fine for me with the supplied apache2 and lighttpd config
files, and also on nginx.  Please use reportbug(1) next time, or at
least tell which HTTPd you're using.  Did you perhaps disable symlinks?

> they stop pointing to the right place when accessed via the vhost
> document_root (/var/lib/roundcube/).

They do?

$ readlink /var/lib/roundcube/skins/elastic
/usr/share/roundcube/skins/elastic
$ readlink /var/lib/roundcube/skins/elastic/deps/*
../../../../nodejs/bootstrap/dist/js/bootstrap.bundle.min.js
../../../../nodejs/bootstrap/dist/css/bootstrap.min.css
../../../../javascript/less/less.min.js
$ readlink -e /var/lib/roundcube/skins/elastic/deps/*
/usr/share/nodejs/bootstrap/dist/js/bootstrap.bundle.min.js
/usr/share/nodejs/bootstrap/dist/css/bootstrap.min.css
/usr/share/javascript/less/less.min.js
$ strace -estat stat -L --printf="" /var/lib/roundcube/skins/elastic/deps/*
stat("/var/lib/roundcube/skins/elastic/deps/bootstrap.bundle.min.js", 
{st_mode=S_IFREG|0644, st_size=133657, ...}) = 0
stat("/var/lib/roundcube/skins/elastic/deps/bootstrap.min.css", 
{st_mode=S_IFREG|0644, st_size=146226, ...}) = 0
stat("/var/lib/roundcube/skins/elastic/deps/less.min.js", 
{st_mode=S_IFREG|0644, st_size=97984, ...}) = 0

FWIW ‘foo/bar/baz/../..’ isn't necessarily the same as directory ‘foo’;
that depends if there are any symlinks on the way.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#948514: [Pkg-roundcube-maintainers] Bug#948514: roundcube-core: missing dependency on libjs-less

[Guilhem Moulin]

On Fri, 10 Jan 2020 at 00:15:25 +0100, Guilhem Moulin wrote:
> But I guess we could ship symlinks though in case someone wants to
> manually install libjs-less.

Didn't recall I did precisely that last summer:


https://salsa.debian.org/roundcube-team/roundcube/commit/9512c4378c9386840a0b369bd311b18a7767c7f6#1779df0686eda665ee449958bfd19da823410d45

The web console spews some warnings but AFAICT devel mode 
(‘$config['devel_mode'] = true;’)
works out of the box once libjs-less is installed.  Still I don't see the need 
to add it as
dependency, not even as Suggests:.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#948514: [Pkg-roundcube-maintainers] Bug#948514: roundcube-core: missing dependency on libjs-less

[Guilhem Moulin]

Control: tag -1 moreinfo

Hi,

On Fri, 10 Jan 2020 at 01:40:56 +0900, Marc Dequènes wrote:
> libjs-less is missing for the new Elastic theme.

Are clients being served .less files?  The intention is to use the .css
files (compiled from the .less sources) instead.  AFAICT .less files are
only used used in devel_mode, which doesn't warrant a new dependency.
But I guess we could ship symlinks though in case someone wants to
manually install libjs-less.

See also the “KNOWN ISSUES” section at the bottom of skins/elastic/README.md .

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#948011: [Pkg-roundcube-maintainers] Bug#948011: roundcube-core: Needs dependency on newer libjs-bootstrap4 libjs-popper.js

[Guilhem Moulin]

Control: retitle -1 roundcube-core: Needs versioned dependency on 
libjs-bootstrap4: >=4.4.1+dfsg1-1.
Control: tag -1 pending

Hi,

On Fri, 03 Jan 2020 at 11:10:06 +0100, Frederik Himpe wrote:
> I installed libjs-bootstrap4=4.4.1+dfsg1-1 which also pulled in an upgrade of
> libjs-popper.js=1.16.0+ds2-1, and now it's behaving as I expected.

AFAICT libjs-popper.js is irrelevant, and Roundcube 1.4.2 should work with
Bootstrap 4.3.1.

The problem is that ‘/usr/share/roundcube/skins/elastic/deps/bootstrap.min.css’
and ‘/usr/share/roundcube/skins/elastic/deps/bootstrap.bundle.min.js’
respectively link to ‘/usr/lib/nodejs/bootstrap/dist/css/bootstrap.min.css’ and
‘/usr/lib/nodejs/bootstrap/dist/js/bootstrap.bundle.min.js’.  Roundcube can't
load Buster's libjs-bootstrap4 as these targets were changed in 4.4.1+dfsg1-1,
cf. https://salsa.debian.org/js-team/twitter-bootstrap4/commit/08ebbd5 .

I changed the target to ‘/usr/share/javascript/bootstrap4/css/bootstrap.min.css’
and ‘/usr/share/javascript/bootstrap4/js/bootstrap.bundle.min.js’ respectively.
This can be manually applied as workaround and doesn't require upgrading
libjs-bootstrap4 to >4.3.1+dfsg2-1.

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#947320: roundcube-core: Retry to connect to IMAP server

[Guilhem Moulin]

Hi Sandro,

On Tue, 24 Dec 2019 at 17:21:40 +0100, Sandro Knauß wrote:
> An IMAP server may have temporally issues, like to much load and roundcube 
> fails with
> "Empty startup gretting".

Did you try to upstream that patch?  Also FWIW the code snippet seems to
be unchanged since <1.0.0, AFAICT no one complained so far so I'm unsure
that Severity: important is really appropriate here, or if it's worth
fixing in stable :-P

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#946727: interimap: typo in html documentation: though → through

[Guilhem Moulin]

Control: tag -1 pending
Control: retitle -1 interimap: typo in documentation: though → through

Hi,

On Sat, 14 Dec 2019 at 21:06:14 +0100, Jonas Smedegaard wrote:
> Reads more sensible to me with word "though" replaced by "through".

Ah yeah, thanks!  Removed “html” in the title: the source, in pandoc
markdown, can be found under /usr/share/doc/interimap.  There is also a
suggested workflow for multi-remote setups under the same directory, by
the way (though again pure IMAP, not sure how that plays along with non
IMAP-based MUAs).

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#934753: dropbear-initramfs: please add an autopkgtest

[Guilhem Moulin]

Hi josch,

On Wed, 20 Nov 2019 at 11:21:31 +0100, Johannes Schauer wrote:
> Indeed it would not. The autopkgtest does not even test upgrades. But when I
> re-installed the system after the upgrade failed I also was unsure in many
> points of how to do it correctly. The autopkgtest does not only serve as a
> constant assurance that the current state does work but also serves as
> documentation of which knobs have to be turned to successfully set it up. It
> can thus serve as an addition to
> /usr/share/doc/dropbear-initramfs/README.Debian

Fair enough :-)

>> I'd also bind to INADDR_LOOPBACK, change the NIC and drive model from the
>> default (resp. e1000 and ide) to virtio, and pass `-no-user-config
>> -nodefaults`.  Maybe also set the CPU model to host.  Might also help to
>> create a virtio-rng device, given that key material is generated on the
>> guest.
> 
> I'm unsure about -no-user-config -nodefaults. This will require a much larger
> command line for what benefit?

For ‘-nodefaults’ it might be reasonable to assume that the qemu
maintainers make sure to have sane default that don't break on upgrade,
however ‘-no-user-config’ shouldn't require a larger command line and will
be really useful for those who run the pkgtest in unclean environments.
(One could have tons of stuff there, including extra disks, different
boot priorities, etc. and shoot oneself in the foot.)
 
> What benefit would changing the nic and drive to virtio have? Sorry, I'm not 
> an
> expert on this. :)

I guess QEMU defaults to “full virtualization” (incl. hardware
emulation) because not all guest OS are VirtIO-aware.  When the guest OS
can speak VirtIO it makes little sense to emulate a particular hardware.
That said saving the abstraction cost won't give a huge perf gain here
because there is not much I/O.  Another advantage is the reliable block
device paths (/dev/vda for the first drive), while otherwise that
depends on the bus (SATA, IDE) and driver in use.
 
> Yes, adding something like this might be useful:
> 
> -object rng-random,filename=/dev/urandom,id=rng0 -device 
> virtio-rng-pci,rng=rng0
> 
> I'll benchmark this to see whether there is a noticeable difference.

That's a hard requirement when the guest calls getrandom(,,GRND_RANDOM)
or reads from the blocking PRNG.  dropbear(8) doesn't *right now*, but
other components in the software stack do and on headless VMs there is a
real risk of entropy starvation without virtual hardware RNG (cf. last
year's “Handling of entropy during boot” thread on debian-devel.)

>>> printf myinsecurepassphrase | cryptsetup luksFormat /dev/sdb3 -
>> To speep up things I suggest to skip the the PBKDF benchmark by passing
>> `--pbkdf-force-iterations 4 --pbkdf-memory 32` (for Argon2), or
>> `--pbkdf-force-iterations 1000` (for PBKDF2).  cryptsetup <2.0 (up to
>> Stretch) are only able to format and open LUKSv1 volumes, which only
>> supports PBKDF2 as PBKDF algorithm; since cryptsetup 2.0 a new LUKS
>> version format is available (and is the default as for Buster) with
>> support for both Argon2i/d (default) and PBKDF2.
> 
> Is this only for a potential speedup? Is it not better to test the default
> setup?

PBKDF are by design expensive functions; when a new keyslot is added, a
benchmark is run to choose parameters that are 1/ expensive enough to
defeat bruteforce attacks, and 2/ reasonable enough to avoid waiting too
long for the device to unlock.  When `luksFormat` and `luksOpen` are
called from machines with different clock speed and/or RAM size, the
benchmark is just moot; and when the former machine is much more beefy
than the latter, which is likely the case here (though maybe not in the
CI pipeline), unlocking might take a lng time, or even trigger the
OOM killer and cause the test to fail.  With my cryptsetup maintainer
hat on, the general recommendation when `luksFormat` and `luksOpen` are
called from different machines, is to collect parameters on the latter
machine, and use them when creating the keyslot.  For a test though it
makes sense to skip the benchmark altogether and PBKDF parameters that
are as cheap as possible, to avoid wasting RAM and CPU cycles.  FWIW
this is also what I do in the tests for cryptsetup-initramfs (which I
mean to turn into autopkgtests as well). :-)

Thanks!
Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#945795: [signing-party] gpg-key2ps prints ed25519 key as eddsa256 key

[Guilhem Moulin]

Control: severity -1 minor
Control: retitle -1 gpg-key2ps should list curve names for ECC keys rather than 
$ALGO$LENGTH

Hi,

On Thu, 28 Nov 2019 at 21:59:18 +0200, Mikaela Suomalainen wrote:
> my key is printed as eddsa256/BAE30723, while I believe it should be
> ed25519 as reported by `gpg --list-keys`.

Lowered the severity as there is AFAICT no promise that the output
format matches the one from `gpg --list-keys`.  Showing $ALGO$LENGTH for
ECC keys, like what's done for RSA, DSA and ElGalmal keys isn't wrong,
even though showing the curve name would be more precise.  I don't
really speak PostScript so I'm unlikely to change that myself; patch
welcome :-)  See also https://bugs.debian.org/869398 .

In the meantime one can use gpg-key2latex(1) from the same package,
which doesn't have this issue, at the expense of heavier dependencies
(the TexLive suite).

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944958: interimap: please optionally sync flags with local notmuch database

[Guilhem Moulin]

Hi Jonas,

On Sun, 17 Nov 2019 at 21:19:21 +0100, Jonas Smedegaard wrote:
> I would love something geared towards standards-compliant IMAP,
> enabled by adding one interimap config line pointing to my notmuch
> database.

Such a MUA-specific feature is most likely a won't fix I'm afraid,
because interimap is intentionally MUA-agnostic and I'm very, very keen
to keep it that way…

Not tagging the bug as ‘wontfix’ as hooks *might* be added at some point
in the distant future (or by someone else).  But I'm not convinced it'd
work, because AFAIK notmuch works with filenames in the Maildir, while
interimap has only access to the message UID and sequence number.  (One
could FETCH X-GUIDs but that becomes Dovecot-specific.)  It works with
OfflineIMAP because it speak Maildir natively.  Furthermore, while a
hook system could in principle replicate remote flag updates to a custom
format locally, for the other way around we'd need active polling which
sound rather clunky, especially with NOTIFY.

Sorry to give up on that one, but I'm also very keen on preserving
layering :-P  IMHO the better fix would be to add an IMAP backend to
notmuch.  Unfortunately, that's also the most difficult :-(  (And not
something I plan to work on.)  Perhaps a stand-alone tool to sync a
notmuch tag database against an IMAP server would be easier?  Or even —
further breaking layering — directly in the Maildir.  Both options would
be IMAPd-specific though.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

On Sun, 17 Nov 2019 at 20:54:47 +0100, Jonas Smedegaard wrote:
> Ohh - problem seems gone indeed now:

Woo! \o/

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

On Sun, 17 Nov 2019 at 19:35:51 +0100, Jonas Smedegaard wrote:
> Seems you edited above quote: I had a dot between INBOX and olpc - is 
> that the "typo" you are talking about then the command I ran included 
> that dot (as did the email I sent).

Oh sorry for that, my finger must have ripped as I was replying, and
doveadm-force-resync(1) appears not to complain when the mailbox doesn't
exist :-P
 
>> If it still doesn't work with the right name, here is what I would do if
>> I were you:
>> 
>>   1. ~$ cp /path/to/maildir/.INBOX.olpc /path/to/maildir/.INBOX.olpc.back
>>   2. ~$ rm -vf /path/to/maildir/.INBOX.olpc/dovecot.*
>>   3. Run `doveadm -f flow fetch "uid modseq guid flags text" mailbox 
>> INBOX.olpc | grep -iE "(^| )uid=97( |$)"`
>>  again.
>>   4a. If 3. doesn't match anymore, then `interimap --repair INBOX.olpc`
>>   should be able to reconcile the mailboxes (it'll complain about
>>   missed updates because of the reset HIGHESTMODSEQ, but that's
>>   harmless), and subsequent `interimap --repair INBOX.olpc`
>>   shouldn't spew anny warning.
>>   4b. If 3. still matches, then also remove 
>> /path/to/maildir/.INBOX.olpc/dovecot-uid*
>>   However that will invalidate the UID mapping, so interimap
>>   won't be able to reconcile, you'll need to remove the mailbox
>>   from the database and the local server.
> 
> Not sure what you mean by "doesn't match anymore" - if I understand 
> correctly it didn't match before either.

I mean grep(1) has exit status 1.  At the bottom of 
https://bugs.debian.org/944812#57 you wrote

| jonas@auryn:~$ ssh jonas-deb...@xayide.jones.dk 'doveadm -f flow fetch "uid 
modseq guid flags text" mailbox INBOX.olpc' | grep -iE "(^| )uid=97( |$)"
| doveadm(jonas-debian): Error: net_connect_unix() failed: Connection refused
| doveadm(jonas-debian): Error: fetch(guid) failed for box=INBOX.olpc uid=97: 
Message was expunged
| doveadm(jonas-debian): Error: fetch(text) failed for box=INBOX.olpc uid=97: 
Message was expunged
| uid=97 modseq=1 guid= flags=11662 text=2

Which seems to be the source of the problem (the mail is gone and
doesn't show up when queried specifically, but *does* show up when its
UID is included in a larger range).
 
> I tried now to remove all dovecot.index* files for that Maildir, and (as 
> earlier) the command greps nothing.

Sounds good to me, and hopefully UID 97 no longer show up in
mailbox-wise UID FETCH commands:

b UID FETCH 1:* (MODSEQ FLAGS INTERNALDATE BODY.PEEK[])

If that's indeed the case then the problem seems gone to me.
 
>> Beside that I'm not sure which Dovecot magic could help.  Perhaps double
>> check that all files in that directory look alright?
>> 
>>   find /path/to/maildir/.INBOX.olpc/{cur,new,tmp} -mindepth 1 \
>>   \! -type f -o \! -name "*.M*,S=*"
> 
> That find command finds almost every all mails in that Maildir:

Ah fair enough, I'm not so familiar with the Maildir format, seems like
file names are more diverse than I thought.  At least this one shouldn't
list anything:

find /path/to/maildir/.INBOX.olpc/{cur,new,tmp} -mindepth 1 \! -type f

(I have no idea how Dovecot deals with mails that aren't regular files,
broken symlinks, etc.)

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

On Sun, 17 Nov 2019 at 17:54:23 +0100, Jonas Smedegaard wrote:
> Seems the "force-repair" command didn't make any change:

Grmbl, but seems you typoed the mailbox name:

> jonas@auryn:~$ ssh jonas-deb...@xayide.jones.dk 'doveadm force-resync 
> INBOXolpc'

If it still doesn't work with the right name, here is what I would do if
I were you:

1. ~$ cp /path/to/maildir/.INBOX.olpc /path/to/maildir/.INBOX.olpc.back
2. ~$ rm -vf /path/to/maildir/.INBOX.olpc/dovecot.*
3. Run `doveadm -f flow fetch "uid modseq guid flags text" mailbox 
INBOX.olpc | grep -iE "(^| )uid=97( |$)"`
   again.
4a. If 3. doesn't match anymore, then `interimap --repair INBOX.olpc`
should be able to reconcile the mailboxes (it'll complain about
missed updates because of the reset HIGHESTMODSEQ, but that's
harmless), and subsequent `interimap --repair INBOX.olpc`
shouldn't spew anny warning.
4b. If 3. still matches, then also remove 
/path/to/maildir/.INBOX.olpc/dovecot-uid*
However that will invalidate the UID mapping, so interimap
won't be able to reconcile, you'll need to remove the mailbox
from the database and the local server.

Beside that I'm not sure which Dovecot magic could help.  Perhaps double
check that all files in that directory look alright?

find /path/to/maildir/.INBOX.olpc/{cur,new,tmp} -mindepth 1 \
\! -type f -o \! -name "*.M*,S=*"

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

On Sun, 17 Nov 2019 at 17:12:54 +0100, Jonas Smedegaard wrote:
> local(INBOX.olpc): WARNING: No match for 1 vanished remote UID(s) 97. 
> Ignoring.

This means the remote server send a VANISHED response for a message in
the “known range” (ie with UID strictly lower than the remote's cache
UIDNEXT value), but that message was not synced locally so interimap
can't replicate the STORE/EXPUNGE command to remove it from the local
server.  Details are in RFC 7162 sec. 3.2.5 and 3.2.10 :-)

You can probably see an untagged VANISHED (EARLIER) response with a set
containing 97 if you do

a ENABLE QRESYNC
b EXAMINE INBOX.olpc (QRESYNC (1225292233 1 1:100))

(1225292233 is the mailbox's UIDVALIDITY, 1 is the lowest mod-sequence,
and 1:100 is an UID range containing 97.  If that doesn't trigger an
untagged VANISHED (EARLIER) response then replacing ‘1:100’ with
‘1:12584’ should.)

> jonas@auryn:~$ interimap --config debian
> jonas@auryn:~$ interimap --config debian
> jonas@auryn:~$ interimap --config debian

That's expected, it doesn't complain about unexpected VANISHED responses
anymore because it went on.  QRESYNC-based synchronization is so
efficient because it's stateful, and UID 97 message disappeared before
the servers could agree on a state.  The mailbox's HIGHESTMODSEQ counter
is increased when a message is EXPUNGed, added, flagged, etc.; once
synchronization successfully terminates, the invariant is that all
changes up to the cached UIDNEXT / HIGHESTMODSEQ values have been
synced, so only further changes are picked up next time.  (To reconcile
earlier changes one needs to do a so called “full-sync” with --repair.)
The server claims that message has MODSEQ 1, so I guess the VANISHED
(EARLIER) response won't show up when QRESYNC's MODSEQ attribute is ≥2.
That said I'm not sure server responses about that UID can be relied
upon.

> jonas@auryn:~$ interimap --config debian --repair
> local(INBOX.olpc): WARNING: No match for modified remote UID 97. Downloading 
> again.

That's also expected, because as we saw earlier the server sends
conflicting untagged FETCH responses for that UID.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

Control: tag -1 pending

On Sun, 17 Nov 2019 at 16:07:38 +0100, Jonas Smedegaard wrote:
> * 3396 FETCH (UID 97 MODSEQ (1) FLAGS () INTERNALDATE "01-Jan-1970 00:00:00 
> +" BODY[] NIL)

Many thanks, that's the first time I see ‘BODY[] NIL’ in an untagged
FETCH response :-)  That explains the warning in your original bug
report.  RFC 3501 allows ‘BODY[] NIL’ in untagged FETCH responses, but
not in APPEND commands so in the context of mailbox synchronization I
see no option other than gracefully ignoring these (just like
zero-length messages).

>> Does ‘doveadm force-resync INBOX.olpc’
>> help in convincing dovecot to forget about that message?
> 
> I avoid above command to be able to test your later bugfix.

The following commit should silence the “uninitialized value” warning
(which is benign/cosmetic anyway):


https://git.guilhem.org/interimap/commit/?id=5d55a14fa52981ae2a8bb6e65a65bf410c773464

However for the reproducible warning in --repair mode I don't see what
can be done because we receive conflicting responses from the server:
first it says there is a message that wasn't copied to the other side
(hence the note about re-downloading), then that there is no such
message.

> Tell me if that's a wrong approach and you have no need for my mailbox
> staying broken.

Feel free to repair your mailbox on the remote now :-)  Thanks for
holding that, though!

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944859: interimap: BAD Error in IMAP command UID FETCH: Too long argument

[Guilhem Moulin]

On Sun, 17 Nov 2019 at 15:20:51 +0100, Jonas Smedegaard wrote:
> Quoting Guilhem Moulin (2019-11-16 17:26:29)
>> On Sat, 16 Nov 2019 at 15:57:16 +0100, Jonas Smedegaard wrote:
>>> Seems interimap can generate commands exceeding what Dovecot can 
>>> handle.
> [...]
>> If you have access to the Dovecot config on both ends, a workaround is
>> to raise `imap_max_line_length` to something sensible (the default is
>> 64k).
> 
> Thanks for the suggested workaround.  I however get same error when 
> (uncommenting and) bumping to 128k/256k/640k in file 
> set_presence/etc/dovecot/conf.d/20-imap.conf on a Debian stable system, 
> and then restarting dovecot.

Ah yeah, forgot to add that Dovecot's `imap` binary doesn't honor that
setting when set in the configuration file.  Dunno if that's intentional
or if it's a Dovecot bug; anyway one needs to pass it to the command
line instead: `doveadm exec imap -oimap_max_line_length=256k`.  (No need
to reload or restart Dovecot.)

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

On Sun, 17 Nov 2019 at 14:38:43 +0100, Jonas Smedegaard wrote:
> doveadm is easier to run as a oneliner (please do tell if interesting 
> for you to have also the output of the raw imap commands):

In this case I'd like to see the IMAP server response so I can push a
fix for the “uninitialized value” warning.
 
> jonas@auryn:~$ ssh jonas-deb...@xayide.jones.dk 'doveadm -f flow fetch "uid 
> modseq flags text" mailbox INBOX.olpc' | grep -iE "(^| )uid=97( |$)"
> doveadm(jonas-debian): Error: fetch(text) failed for box=INBOX.olpc uid=97: 
> Message was expunged
> uid=97 modseq=1 flags= text=11662
> […] 
> jonas@auryn:~$ ssh jonas-deb...@xayide.jones.dk 'doveadm -f flow fetch "uid 
> modseq guid flags text" mailbox INBOX.olpc' | grep -iE "(^| )uid=97( |$)"
> doveadm(jonas-debian): Error: fetch(guid) failed for box=INBOX.olpc uid=97: 
> Message was expunged
> doveadm(jonas-debian): Error: fetch(text) failed for box=INBOX.olpc uid=97: 
> Message was expunged
> uid=97 modseq=1 guid= flags=11662 text=2

Fairly odd, especially the ‘11662’ which moved from text to flags…
Still nothing in the server log?  Does ‘doveadm force-resync INBOX.olpc’
help in convincing dovecot to forget about that message?  Otherwise,
does

grep "^97 " /path/to/maildir/.INBOX.olpc/dovecot-uidlist

matches anything?

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

Control: severity -1 minor

On Sat, 16 Nov 2019 at 23:35:31 +0100, Jonas Smedegaard wrote:
> I guess you mean this:
> […]
> * 3396 FETCH (UID 97 MODSEQ (1) FLAGS ()) 

> jonas@auryn:~$ ssh jonas-deb...@xayide.jones.dk 'doveadm -f flow fetch "uid 
> modseq flags" mailbox INBOX.olpc' | grep -w 97
> uid=97 modseq=1 flags=

Yup, sorry for the lack of clarity and thanks for reading between the
lines.  I'm lowering the severity of this bug to ‘minor’ now, because
AFAICT it's cosmetic at worst.

The above is consistent with the rest of this report, even though I
don't get why you receive a response with UID 97 when the FETCH command
spans over all UIDs, but not when limited to said UID.  Sounds like a
server issue to me, but I'd need to double check RFC 3501 to be sure.

We've already seen that FETCHing the BODY of UID 97 produces no
response.  Does it help to add “INTERNALDATE BODY.PEEK[]” to the
fetch-attr list?  I.e., replace the UID FETCH command with

b UID FETCH 1:* (MODSEQ FLAGS INTERNALDATE BODY.PEEK[])

That was the command used in the initial sync (the one producing the
warning about uninitialized value value).  If my theory is correct, the
response will be something like

* 3396 FETCH (UID 97 MODSEQ (1) FLAGS () … BODY[] NIL)

(That IMAP command will produce a lot of output on large mailboxes; you
might want to grep remotely if your connection to the server is slow or
unreliable.)  Or using doveadm:

$ doveadm -f flow fetch "uid modseq flags text" mailbox INBOX.olpc | grep 
-iE "(^| )uid=97( |$)"

Perhaps it's enough to dump UIDs 90 to 100 than the entire mailbox, I
don't know.  Hopefully your mailbox is not so large that dumping its
content DoS'es the server :-P

If these command do produce an untagged FETCh output, you could try to
add the GUID to the fetch-attr list (with doveadm and/or in the IMAP
session) to see if that message points anywhere at all.

b UID FETCH 1:* (MODSEQ X-GUID FLAGS INTERNALDATE BODY.PEEK[])

and

$ doveadm -f flow fetch "uid modseq guid flags text" mailbox INBOX.olpc | 
grep -iE "(^| )uid=97( |$)"

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

On Sat, 16 Nov 2019 at 20:47:53 +0100, Jonas Smedegaard wrote:
> Doesn't seem succesful (I creatively prepended "a EXAMINE INBOX.olpc", 
> hope that is correct):

Yup, sorry for the incomplete commands ;-)
 
> a UID FETCH 97 MODSEQ
> * OK [HIGHESTMODSEQ 17] Highest
> a OK Fetch completed (0.001 + 0.000 secs).
> b UID FETCH 97 FLAGS
> b OK Fetch completed (0.001 + 0.000 secs).
> c UID FETCH 97 INTERNALDATE
> c OK Fetch completed (0.001 + 0.000 secs).
> d UID FETCH 97 BODY.PEEK[]
> d OK Fetch completed (0.001 + 0.000 secs).
> e UID FETCH 97 ENVELOPE
> e OK Fetch completed (0.001 + 0.000 secs).

OK, at least that is consistent with a lack of response for ‘UID FETCH
97 ($FETCH_ATTRS)’, but doesn't explain why that UID shows up when
trying to repair.  In your debug log we see

| remote(INBOX.olpc): S: * 3396 FETCH (UID 97 MODSEQ (1) FLAGS ())

AFAICT that untagged FETCH response comes from an ‘UID FETCH 1:* (MODSEQ
FLAGS)’ (most likely prefixed with ‘C: 03’ in the debug log).  Could
you try that command manually and double check that the same untagged
line appears?  We can then narrow it down from there.

S: * PREAUTH [CAPABILITY …]
C: a EXAMINE INBOX.olpc
S: […]
S: a OK [READ-ONLY] Examine completed
C: b UID FETCH 1:* (MODSEQ FLAGS)
S: […]
S: b OK Fetch completed

(I don't need the full server response, just the line containig “UID
97”.  Hopefully doveadm-fetch(1) agrees with that ouput as well:

doveadm -f flow fetch "uid modseq flags" mailbox INBOX.olpc

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

On Sat, 16 Nov 2019 at 18:19:37 +0100, Jonas Smedegaard wrote:
> Quoting Guilhem Moulin (2019-11-16 17:50:14)
>> On Sat, 16 Nov 2019 at 16:25:15 +0100, Jonas Smedegaard wrote:
>>> jonas@auryn:~$ interimap --config debian --repair --debug INBOX.olpc 2>&1 | 
>>> grep -Fw 97
>>> […]
>>> local(INBOX.olpc): WARNING: No match for modified remote UID 97. 
>>> Downloading again.
>>> remote(INBOX.olpc): C: 04 UID FETCH 97 (MODSEQ FLAGS INTERNALDATE 
>>> BODY.PEEK[] ENVELOPE)
>> 
>> I'm suprised grep didn't match the server response for that command.
> 
> Perhaps if you talking about matching the originally reported warning, 
> then beware that the warning mesage initially reported happened only 
> while initially syncronizing that mailbox from scratch.  I am unable to 
> repeat that exact warning message - but instead the warning message 
> above is now repeatable for me.

Gotcha

> Nope:
> […]
> remote(INBOX.olpc): S: * 3396 FETCH (UID 97 MODSEQ (1) FLAGS ())
> […]
> remote(INBOX.olpc): C: 04 UID FETCH 97 (MODSEQ FLAGS INTERNALDATE 
> BODY.PEEK[] ENVELOPE)
> remote(INBOX.olpc): S: 04 OK Fetch completed (0.001 + 0.000 secs).

I'm puzzled by that sequence, the response at the top indicates the
presence of a message (at position #3396) with UID 97, mod-sequence 1,
and empty flag list.  But the server doesn't appear to provide any
response when BODY[] in is the list of attributes to FETCH.  (That would
be fine if there was no message with that UID, but since you can
reproduce it doesn't seem to be the case here.)

> a EXAMINE INBOX.olpc
> […]
> b UID FETCH 97 BODY.PEEK[]
> b OK Fetch completed (0.001 + 0.000 secs).
 
> Seems I didn't get the expected response.  So does that indicate broken 
> data at the server?

Not sure yet, but I'm leaning towards that conclusion.  What if you
query the attributes one at a time, does any of these IMAP commands
produce an untagged FETCH response?

a UID FETCH 97 MODSEQ
b UID FETCH 97 FLAGS
c UID FETCH 97 INTERNALDATE
d UID FETCH 97 BODY.PEEK[]
e UID FETCH 97 ENVELOPE

Do you have access to the server log?  If so, is there something
appearing upon ‘b UID FETCH 97 BODY.PEEK[]’?

You could also try to dump the message using doveadm-fetch(1):

doveadm fetch text mailbox INBOX.olpc uid 97

Or even look directly into the maildir (‘x UID FETCH 97 X-GUID’ should
tell you the file name holding that message).

>> I appreciate the assistance in debugging.  Thanks!
> 
> I am having fun :-)

^^

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

On Sat, 16 Nov 2019 at 16:25:15 +0100, Jonas Smedegaard wrote:
> jonas@auryn:~$ interimap --config debian --repair --debug INBOX.olpc 2>&1 | 
> grep -Fw 97
> […]
> local(INBOX.olpc): WARNING: No match for modified remote UID 97. Downloading 
> again.
> remote(INBOX.olpc): C: 04 UID FETCH 97 (MODSEQ FLAGS INTERNALDATE 
> BODY.PEEK[] ENVELOPE)

I'm suprised grep didn't match the server response for that command.  Do
you have anything at all between

remote(INBOX.olpc): C: 04 UID FETCH 97 (MODSEQ FLAGS INTERNALDATE 
BODY.PEEK[] ENVELOPE)

 and

remote(INBOX.olpc): S: 04 OK Fetch completed

> UID FETCH 97 BODY.PEEK[]
> * 97 FETCH (BODY[] {6520}
> […])
> UID OK Fetch completed (0.132 + 0.000 + 0.131 secs).

Oh my bad, seems I typoed the IMAP command I gave you; that message is
sequence number #97 not UID 97.  It's a ‘FETCH’ command with tag ‘UID’,
while I wanted to send an ‘UID FETCH’ command with tag $foo, like what's
being found in the debug output above.

Please try again prefixing the command with ‘b ’ (or any other tag):

b UID FETCH 97 BODY.PEEK[]

The server response should be

* $SEQ FETCH (UID 97 BODY[] $BODY)
b OK Fetch completed

Where $SEQ is the sequence number (not necessarily 97) and $BODY is a
literal ‘{3}\r\nfoo’, quoted string ‘"bar"’, or NIL.  “UID 97” is
required to be present somewhere in the server response.

> Seems like this _isn't_ a zero-length message.

Message with sequence number #97 is likely not be the one we're looking
for.  (In your debug output I can see that UID 97 had sequence number
#3396 at the time.)

I appreciate the assistance in debugging.  Thanks!

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944859: interimap: BAD Error in IMAP command UID FETCH: Too long argument

[Guilhem Moulin]

Control: tag -1 confirmed pending

On Sat, 16 Nov 2019 at 15:57:16 +0100, Jonas Smedegaard wrote:
> Seems interimap can generate commands exceeding what Dovecot can handle.

Ack, that's a long-standing bug which breaks syncronisation in some
scenarios, such as large mailboxes with many non-contiguous UIDs being
changed at once.  Fixed it a few days ago :-)  It wasn't caught in the
test-suite before as test UID sets are typically way less sparse.


https://git.guilhem.org/interimap/commit/?id=0a2558aabfefd6800fe74c24e5aff2b0d47cc5e2

interimap is at fault here as there was no logic in place to avoid UID
sets from growing indefinitely.  But there is also a bug with Dovecot
(reported upstream at 
https://dovecot.org/pipermail/dovecot/2019-November/117522.html )
which makes thing even worse when COMPRESS=DEFLATE is used.

> Currently with my setup I experience this:

If you have access to the Dovecot config on both ends, a workaround is
to raise `imap_max_line_length` to something sensible (the default is
64k).

That said it's a very anoying bug when it bites, and I should rush the
next release.  I don't forsee a lot of code change anymore, but I'd like
to improve the docs before releasing.  Would you be willing to install a
package from experimental in the meantime if I were to upload one? :-)

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944812: interimap: uninitialized value

[Guilhem Moulin]

On Fri, 15 Nov 2019 at 19:10:56 +0100, Jonas Smedegaard wrote:
> Use of uninitialized value $length in numeric eq (==) at /usr/bin/interimap 
> line 955.
> remote(INBOX.olpc): WARNING: Ignoring new 0-length message (UID 97)
> 
> Wanted to let you know in case it is a bug in your code (rather than
> just a broken email in my mailbox).

Thanks, it's either cosmetic (it shouldn't complain about the
uninitialized value), or it's a more severe bug in the parser.  I guess
this is reproducible with `interimap --repair INBOX.olpc`?  If so, could
you please enable debug mode and run `grep -Fw "UID 97"` on the logfile?
Failing that, you could run the following two commands through an IMAP
session with the remote server:

C: a EXAMINE INBOX.olpc
S: […]
S: a OK [READ-ONLY] …
C: UID FETCH 97 BODY.PEEK[]
S: * $SEQ FETCH (UID 97 BODY[] …)

I'd like to see the first line of the server response past “BODY[] ”.
If it's the keyword ‘NIL’ then it's a cosmetic bug; if it's a string,
either in quoted (‘"foo"’) or literal (‘{3}\r\nfoo’) form it's something
more problematic.

About zero-length messages: they are intentionally ignored as RFC 3502
sec. 6.3.11 explicitly forbids them.  IMHO the WARNING makes sense if
the body of that message is NIL or the empty string, but we should
silence the bit mentioning line 955.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944163: interimap: seems --repair doesn't work

[Guilhem Moulin]

On Thu, 14 Nov 2019 at 19:13:59 +0100, Jonas Smedegaard wrote:
> Heh.  What actually happened was that at first I ran
> 
> interimap --config hb --delete --target=database INBOX
> […]
> Seems to me both local and remote INBOX UIDVALIDITY is now 1571588814:

I see, that explains it.  No magic with recovered UIDVALIDITY, that's a
relief at least :-)

> I see now what mislead me: The bash-style syntax in documentation:
> 
>  --target={local,remote,database}
> 
> It is the only option where you use such unusual compact syntax.
> 
> May I suggest that to
> 
>  --target=TARGET
>Where TARGET is one of local,remote, or database.
>Option can be repeated to cover multiple targets.
> 
> ..or maybe that's irrelevant with your soon-to-be-released change?

Right, sorry for that.  I meant to support comma-separated values
anyway (and croak on invalid targets instead of silently ignoring them),
so I'll call that a now-fixed bug :-)

> I guess you mean this:
> 
> jonas@auryn:~$ grep -Po "\s+V\K\S+" ~/Maildir/hb/.Drafts/dovecot-uidlist
> 1571588832
> jonas@auryn:~$ ssh mail.homebase.dk 'grep -Po "\s+V\K\S+" 
> ~/Maildir/.Drafts/dovecot-uidlist'
> 1571588832

Yup, and is the first line of those files identical?  The 4th field, in
particular.  That's the mailbox GUID, a Dovecot specific attribute
that's AFAIK not exposed to IMAP clients (not in a modifiable way at
least).  It's a 128bits integer built from the current time, PID, and
hostname.


https://sources.debian.org/src/dovecot/1:2.3.4.1-5+deb10u1/src/lib/guid.c/#L49

Granted, this doesn't use getrandom(), but you should never see the same
value accross mailboxes, or hosts outside the same Dovecot cluster.
AFAIK only Dovecot's internal tools (over)write that attribute (we
already know the file wasn't copied using `rsync` or similar, because we
see 3 states: old remote, old local, and new which is… a combination of
these two).  Anyway I'm quite sure neither interimap nor any other
non-malicious IMAP client is responsible for the UIDVALIDITY change.
Since your Draft mailbox seems empty you shouldn't have any bad surprise
with `interimap --config hb --target=database --target=local --delete Drafts`.

>> Would be nice to see the output of `stat -c"%y  %z"` on these files,
>> also.
> 
> jonas@auryn:~$ stat -c"%y  %z" ~/Maildir/hb/.Drafts/dovecot-uidlist
> 2019-10-20 18:56:40.128528124 +0200  2019-10-20 18:56:40.152528125 +0200
> jonas@auryn:~$ ssh mail.homebase.dk 'stat -c"%y  %z" 
> ~/Maildir/.Drafts/dovecot-uidlist'
> 2019-10-21 13:47:24.454650388 +0200  2019-10-21 13:47:24.486649045 +0200

Alright, nothing relevant then :-(

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944163: interimap: seems --repair doesn't work

[Guilhem Moulin]

On Thu, 14 Nov 2019 at 14:01:33 +0100, Jonas Smedegaard wrote:
> interimap --config hb --delete --target=database,local INBOX
> interimap --config hb
>
> ..which seemingly succeeded in throwing away local copy of INBOX

That confuses me even more, ‘--target=database,local’ is a no-op for
0.4-1 because there is no target with that name.  That should be be
‘--target=database --target=local’ instead (comma-separated targets are
allowed since ce35ea5 but that code hasn't been released yet) as written
earlier.  However, unlike what I claimed before, deleting INBOX will
only succeed for the database target, as RFC 3501 src 6.3.4 explicitly
forbids this.  Sigh.  (A workaround to delete one's INBOX is to rename
it to something else and delete the new name.)

> and re-fetching it from remote INBOX,

I guess only the new remote messages were copied?  Could you confirm
that INBOX's local (resp. remote) UIDVALIDITY is 1571588814 (resp.
1154884797)?

However it's beyond me why the remote UIDVALIDITY appears to have
changed again back to its previous value.  Are Dovecot's remote index
files stored on another drive, that might not be mounted all the time?

> ..because surprisingly to me, attempting to throw away local copy of
> _Drafts_ failed:
>
> jonas@auryn:~$ interimap --config hb --delete --target=database,local Drafts

Should work with `--target=database --target=local --delete Drafts`; you
should see notifications of successful deletion in the DB and on the
local server.

> Am I perhaps misunderstanding and UIDVALIDITY is tied to whole account,
> not each mailbox?

Nope, this is indeed a per-mailbox attribute (like UIDNEXT annd HIGHESTMODSEQ).
Or more precisely, a “mailbox” is the combination of a *name* and an 
UIDVALIDITY.

> local: S: * STATUS Drafts (UIDNEXT 962 UIDVALIDITY 1571588832 HIGHESTMODSEQ 4)
> remote: S: * STATUS Drafts (UIDNEXT 962 UIDVALIDITY 1571588832 HIGHESTMODSEQ 
> 4)
> local: Update last clean state for Drafts: (UIDVALIDITY 1571588832 
> HIGHESTMODSEQ 1 UIDNEXT 1)

> lUIDVALIDITY|lUIDNEXT|lHIGHESTMODSEQ|rUIDVALIDITY|rUIDNEXT|rHIGHESTMODSEQ
> 1571588832  |1   |1 |1154884797  |962 |4

Thanks, so the local values make sense for a mailbox created on Oct 20,
which had never been populated by the time interimap last synced it.
Right now the UIDVALIDITY/UIDNEXT/HIGHESTMODSEQ values appear to be
identical on both servers, oddly set to their respective maximum.

That's extremely unlikely the doing of an IMAP client (it'd need to
delete Drafts on the remote server, recreate it at 1571588832s — or be
lucky enough that the server picks that UIDVALIDITY, then on both
servers create 961 messages in 3 batches, mark them all as deleted, then
expunge).

> jonas@auryn:~$ ssh mail.homebase.dk doveadm mailbox status "messages uidnext 
> uidvalidity highestmodseq firstsaved" Drafts
> Drafts messages=0

You need to double quote/escape the spaces in the string query,
otherwise it runs `doveadm mailbox status messages uidnext uidvalidity
highestmodseq firstsaved Drafts` (‘uidnext’ and beyond interpreted as
mailbox names).

Could you please also check that the UIDVALIDITY value is found on the
second field (prefixed with a ‘V’) of the first line [0] of
/path/to/Maildir/.Drafts/dovecot-uidlist both locally and remotely?
Would be nice to see the output of `stat -c"%y  %z"` on these files,
also.

> I have triple-checked, and am preeetty sure that I completely stopped
> using dsync after I engaged with interimap.  Cannot verify for certain
> e.g. by examining .bash_history because I use a local wrapper script,
> but all lines in that wrapper script which previously called dsync was
> commented out before I initially ran interimap (and that commenting out
> is what I triple-checked).

OK, thanks for checking.  Can't explain why the new internal mailbox
state seems to be a mix of the old remote and local ones though :-(

> Possibly related sidenote: I noticed suspecious interimap summary notes
> about bandwidth consumption that seemed flipped around (new messages
> transfered from remote to local, but interimap saying that remote
> received far more than sending, and local received far less than
> sending).  Possible local/remote mixup in interimap code somewhere?
> Sidenote only here - if I notice again I will report separately!

I hope the test suite would have caught delivery to the wrong server
(there are unit tests as well as randomized tests) :-P  The stats stuff
is just cosmetic so there are no tests though; I understand your
confusion, but these numbers come from interimap's perspective: recv
(resp. sent) stands for received by (resp. to) interimap from the
server, so when remote a message as copied locally the “remote” IMAP
client *receives* a large FETCH reply and the “local” one *sends* a
large literal in an APPEND command.  That was my thinking at least :-)

-- 
Guilhem.

[0] https://wiki.dovecot.org/MailboxFormat/Maildir


signature.asc
Description: PGP signature

Bug#942725: interimap: please support name to appear in log lines

[Guilhem Moulin]

Control: tag -1 pending

On Mon, 21 Oct 2019 at 00:26:58 +0200, Jonas Smedegaard wrote:
>> I can certainly add an option with a simple printf-like format to 
>> customize the log :-)
> 
> I was thinking something along the lines of printf-like hack, yes.

I took a hammer for this, and added a new option

log-prefix

A printf(3)-like format string to use as prefix for each log
message. Interpreted sequences are %n and %m, expanding respectively
to the component name (local/remote) and to the name of the mailbox
relevant for the log entry. Conditions on a specifier %X can be
obtained with %?X?then? or %?X?then?, which expands to ‘then’
if the %X specifier expands to a non-empty string, and to ‘else’ (or
the empty string if there is no else branch) otherwise. Literal %
need to be escaped as %%, while &, ? and \ characters need to be
\\-escaped.  (Default: %?n?%?m?%n(%m)&%n?: ?.)

It's quite overkill, but was also the occasion to refactor the logging
code :-)

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#944163: interimap: seems --repair doesn't work

[Guilhem Moulin]

On Wed, 06 Nov 2019 at 12:44:08 +0100, Jonas Smedegaard wrote:
> It seems that interimap would do far better in such an extreme
> scenario, as it seems to syncronize small chunks at a time which would
> make it possible to continue-where-left-off in far smaller chunks than
> possible (or rather comprehensible to me) to do with either offineimap
> or dsync.

Correct, the targeted use-case is when one wants to synchronize often,
meaning most of the time there are no changes to send over.  offlineimap
(and `interimap --repair`) selects each mailbox, fetches all messages
UIDs and flags, then locally checks for differences between the local
and remote list.  This is slow, I/O-intensive, and causes a significant
amount of network traffic.  OTOH with QRESYNC-based synchronization,
it's enough to compare the output of the LIST command, which is much
shorter (one line per message vs. two lines per mailbox).  In practice
the LIST/STATUS response for 100-150 mailboxes fits in a single TLS
record (max size 16kiB); for servers supporting the COMPRESS=DEFLATE
extension that number is typically over 100× higher (it's also possible
to compress at a lower layer, but the advantage of doing it at the IMAP
layer is that it gives more control on the deflate dictionary; we for
instance trigger flush points before sending/receiving large messages,
which likely wouldn't compress well anyway).  When there are new
changes, be it flag updates, new messages, or deletions, only what's new
since the last successful sync needs to be replayed on the other side.
Thanks to that logic, interimap should be able to scale to mailboxes
containing many millions messages; AFAICT Dovecot is quite able to cope
with these too, but the MUA might not.
 
> When I switched, I removed the local folder and had interimap syncronize 
> it from scratch.  Or at least that is my belief: Does your forensics 
> analysis above imply that I failed at that, and the local copy cannot 
> have been initialized by interimap?

I see 3 states in your original bug report:

| local: Update last clean state for INBOX: (HIGHESTMODSEQ 165 UIDVALIDITY 
1571588814 UIDNEXT 9598)

These are the numbers found in the database, and correspond to the state
of the local INBOX at the time it was last synchronized by interimap.
The numbers make sense for a mailbox that was recently created and
populated en masse: low HIGHESTMODSEQ (a strictly increasing sequence of
all modifications to the mailbox, incl. flags, message creations and
deletion) and highest UID <9598 suggests there are slightly under 10k
messages in that mailbox.  Based on how Dovecot allocates UIDVALIDITY, I
guess the mailbox was created on Sun, 20 Oct 2019 at 18:26:54 +0200.

| local: S: * STATUS INBOX (UIDNEXT 290008 UIDVALIDITY 1571588814 HIGHESTMODSEQ 
71671)

This the *current* state of the local INBOX.  The new UIDNEXT and
HIGHESTMODSEQ values suggest (IMAP4rev1 mandates the sequences to be
strictly increasing, but in practice Dovecot uses continuous numbers)
that within two weeks that mailbox saw 280k *new* messages (possibly now
deleted) and over 71k *new* changes (such as flag updates, and message
creation/deletion).  It's not impossible, but certainly suspicious.

I'd be interested to know the last good remote state, could you please
run:

sqlite3 /path/to/interimap/database.db -header <<-EOF
SELECT l.UIDVALIDITY AS lUIDVALIDITY, l.UIDNEXT AS lUIDNEXT, 
l.HIGHESTMODSEQ AS lHIGHESTMODSEQ,
   r.UIDVALIDITY AS rUIDVALIDITY, r.UIDNEXT AS rUIDNEXT, 
r.HIGHESTMODSEQ AS rHIGHESTMODSEQ
 FROM mailboxes m JOIN local l ON m.idx = l.idx JOIN remote r 
ON m.idx = r.idx
WHERE mailbox = 'INBOX';

SELECT COUNT(*) AS messages
 FROM mailboxes NATURAL JOIN mapping
WHERE mailbox = 'INBOX';
EOF

and paste the output?  (The database file can normally be found under
‘${XDG_SHARE_HOME:-~/.local/share}/interimap’, and the default filename
is the hostname of your remote server.)

We already know the values of lUIDVALIDITY, lUIDNEXT, lHIGHESTMODSEQ
and rUIDVALIDITY: respectively 1571588814, 9598, 165 and 1571588814.
I'm interested in rUIDNEXT and rHIGHESTMODSEQ though.  I'd also like to
see the output of

doveadm mailbox status "messages uidnext uidvalidity highestmodseq 
firstsaved" INBOX

both locally and remotely.

| remote: S: * STATUS INBOX (UIDNEXT 280739 UIDVALIDITY 1571588814 
HIGHESTMODSEQ 74538)

This the *current* state of the remote INBOX.  The UIDVALIDITY value
suggests the mailbox was also created on Sun, 20 Oct 2019 at 18:26:54 +0200
(just like the local INBOX; not impossible of course, but suspicious),
while the high UIDNEXT and HIGHESTMODSEQ values hint at a much older
mailbox (unless that mailbox is very active, again not impossible but
suspicious).

| remote: ERROR: UIDVALIDITY changed! (1571588814 != 1154884797)  Need to 
invalidate the UID cache.

This indicates that the UIDVALIDITY of 

Bug#944163: interimap: seems --repair doesn't work

[Guilhem Moulin]

Hi Jonas,

On Tue, 05 Nov 2019 at 13:55:59 +0100, Jonas Smedegaard wrote:
> $ interimap --config hb INBOX
> remote: ERROR: UIDVALIDITY changed! (1571588814 != 1154884797)  Need to 
> invalidate the UID cache.
> 
> ..and it seems the --repair option doesn't do its job:

I guess naming that command ‘--repair’ wasn't the best choice; it's not
meant to reconcile UIDVALIDITY mismatch anyway, but to perform a
so-called “full synchronization” à la offlineimap to compare the
UID+flag list between the local and remote mailboxes.  (Unless there is
a bug in interimap or the IMAPd's QRESYNC implementation there shouldn't
be anything to repair, but it's still useful for integrity checking
etc.)

IMAP4rev1 allows servers without mechanism for storing persistent UID and
UIDVALIDITY values.  However synchronizing such server is out of scope
for interimap(1), and I guess other IMAP-based synchronization software,
cf. RFC 3501 sec. 2.3.1.1: “Persistent unique identifiers are required
for a client to resynchronize its state from a previous session with the
server […] The combination of mailbox name, UIDVALIDITY, and UID must
refer to a single immutable message on that server forever.”

There is nothing interimap can do so resolve UIDVALIDITY conflicts.  It
needs to invalidate its cache, because per RFC 3501 the lUID ↔ rUID
mapping can't be relied upon anymore.  (Rebuilding the mapping by
comparing message RFC 5355 data is not reliable either, because IMAP
does allow duplicates.)

> local: S: * STATUS INBOX (UIDNEXT 290008 UIDVALIDITY 1571588814 HIGHESTMODSEQ 
> 71671)
> […]
> remote: S: * STATUS INBOX (UIDNEXT 280739 UIDVALIDITY 1571588814 
> HIGHESTMODSEQ 74538)
> […]
> remote: ERROR: UIDVALIDITY changed! (1571588814 != 1154884797)  Need to 
> invalidate the UID cache.

I find it suspicious that both mailboxes have the same UIDVALIDITY
value.  By RFC 3501 that value is an arbitrary unsigned >0 32-bits
integer.  That value (like UID values) are set by the server and can't
be chosen or altered by the client (otherwise interimap wouldn't need a
database to keep track of the lUID ↔ rUID mapping, a mere state file
would be sufficient).  Dovecot doesn't pick it at random, but AFAICT
uses the timestamp at which the mailbox was created.

It looks kind of weird to me that both your local and remote INBOX
appears to have been created at the very same time (Oct 20 18:26 CEST)
while the timestamp found in the database is many years older (Aug 06
2006).  The UIDVALIDITY conflict can't be caused by interimap or any
other IMAP client, because that's beyond the IMAP protocol.  Did you
maybe run a lower-level synchronization tool like dsync(1) between these
mailboxes?  IIRC dsync preserve states, so it could explain why the
UIDVALIDITY value was reset to the very same timestamp.  (UIDVALIDITY
reset could also happen when the dovecot index files are removed, but it
seems unlikely that they would be set to the very same value.)  It
should never happen with a “normal” layered IMAP stack.  Dovecot does
have persistent UID values, at least when the index files are not stored
on a volatile medium.

Now, how to fix this?  The easiest is to remove the database entry for
that mailbox: `interimap --target=database delete INBOX` (this won't
touch your mails, just the database) and then try to sync again.
However every message in the local INBOX will be copied remotely and
vice-versa, so if you have a lot of messages in both mailboxes this will
create a lot of duplicates :-(  This won't cause any data loss though.

Another thing which seems to suggest that there is a lower-level
synchronization running and causing cache corruption, is the huge spike
between the local INBOX's current state and the one from the database
(ie the last successful sync): UIDNEXT=9598 / HIGHESTMODSEQ=165 vs.
UIDNEXT=290008 / HIGHESTMODSEQ=71671.  While sudden heavy-traffic
mailboxes are certainly possible, it's still a bit suspicious.

The first step is to identify what's causing the cache corruption as
it's likely to happen again otherwise, and that's beyond the scope of
interimap and other plain IMAP clients :-P  If it was indeed dsync(1)
causing the cache corruption, then it's possible to avoid duplicates:
you could run it once more so both sides are in sync (with matching/
overwritten UIDs, UIDVALIDITY, etc. values), then nuke *one* end along
with the database (using `interimap --target=database --target=local
delete INBOX`) and finally sync again.  I reckon it's a bit scary
though, so feel free to poke me over IRC if you'd like.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#943727: chkrootkit: cron.daily exits with status 1 when `chkrootkit $RUN_DAILY_OPTS` produces no output

[Guilhem Moulin]

Package: chkrootkit
Version: 0.52-3
Severity: normal
File: /etc/cron.daily/chkrootkit
Tags: patch

Dear Maintainer,

As of Buster, chkrootkit's cron.daily script contains the following line [0]

eval $CHKROOTKIT $RUN_DAILY_OPTS | egrep -v -f "${IGNORE_FILE}" > 
$LOG_DIR/log.today.raw 2>&1

egrep(1) exits with status 1 when it does not select any line (because
the left hand-side of the pipe produces no output, or because each
output line was matching a pattern from $IGNORE_FILE).

Since the script is run under `set -e`, it causes the entire cronjob to
fail.  Adding a trailing ‘|| true’ fixes this (though one might argue
it's changing the semantics if $IGNORE_FILE is not a readable file.).

Cheers,
-- 
Guilhem.

[0] 
https://salsa.debian.org/pkg-security-team/chkrootkit/blob/debian/0.52-3/debian/cron.daily#L25
--- a/etc/cron.daily/chkrootkit
+++ b/etc/cron.daily/chkrootkit
@@ -22,7 +22,7 @@
 
 if [ "$RUN_DAILY" = "true" ]; then
 if [ "$DIFF_MODE" = "true" ]; then
-eval $CHKROOTKIT $RUN_DAILY_OPTS | egrep -v -f "${IGNORE_FILE}" > $LOG_DIR/log.today.raw 2>&1
+eval $CHKROOTKIT $RUN_DAILY_OPTS | { egrep -v -f "${IGNORE_FILE}" || true; } > $LOG_DIR/log.today.raw 2>&1
 # the sed expression replaces the messages about /sbin/dhclient3 /usr/sbin/dhcpd3
 # with a message that is the same whatever order eth0 and eth1 were scanned
 sed -r -e 's,eth(0|1)(:[0-9])?: PACKET SNIFFER\((/sbin/dhclient|/usr/sbin/dhcpd)\[[0-9]+\]\),eth\[0|1\]: PACKET SNIFFER\([dhclient|dhcpd]{PID}\),' \


signature.asc
Description: PGP signature

Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode

[Guilhem Moulin]

Control: retitle -1 ITP: dropbear-rescue -- A set of initramfs scripts to add 
and run dropbear when the system boots in rescue mode
Control: tag -1 - pending

On Sat, 26 Oct 2019 at 01:50:02 +0200, Guilhem Moulin wrote:
> Control: retitle -1 race condition: init-bottom script doesn't abort/cleanup 
> configure_networking()
> Control: tag -1 pending

Oops sorry, meant to reply to the clone (#943459) not the original :-/

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#942808: Bug#943459: dropbear-initramfs: race condition: init-bottom script doesn't abort/cleanup configure_networking()

[Guilhem Moulin]

Control: retitle -1 race condition: init-bottom script doesn't abort/cleanup 
configure_networking()
Control: tag -1 pending

On Fri, 25 Oct 2019 at 02:26:39 +0200, Guilhem Moulin wrote:
> Ah right, I understand the problem now.  Whether configure_networking()
> is run (at premount stage) in the background or not depends on the boot
> method.  On local (non-NFS) mounts it's done in the background, and
> should be interrupted at bottom stage.  However if no other script is
> waiting for interactive user input the bottom script might run before
> dropbear had a chance to run yielding a race condition at bottom stage.
> This is a bug.

Implemented a fix at https://salsa.debian.org/debian/dropbear/commit/1ab168b9 ,
could you please confirm that it solves the race for you?

It appears that ipconfig doesn't react to SIGTERM, so I've not been able
to properly abort configure_networking().  Instead, the init-bottom
script now waits (for up to 1min) until dropbear is started before
bringing the network down.  Of course it's a bit of a waste as it
needlessly delays the boot process (it's no longer possible to log in at
that stage anyway), but at least when execution is handed over to init(1)
it's with a clean network stack and not with a running ipconfig process.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode

[Guilhem Moulin]

Control: clone -1 -2
Control: reassign -2 dropbear-initramfs 2019.78-2
Control: retitle  -2 race condition: init-bottom doesn't abort/cleanup 
run_networking()
Control: severity -2 normal

On Thu, 24 Oct 2019 at 18:48:12 -0400, Anton Avramov wrote:
> However I've ran into a problem where if there is no panic and
> initramfs ipconfig command uses dhcp it would actually finish after
> the boot process is complete. And if the system itself uses static ip
> it gets overwritten by the dhcp.

Ah right, I understand the problem now.  Whether configure_networking()
is run (at premount stage) in the background or not depends on the boot
method.  On local (non-NFS) mounts it's done in the background, and
should be interrupted at bottom stage.  However if no other script is
waiting for interactive user input the bottom script might run before
dropbear had a chance to run yielding a race condition at bottom stage.
This is a bug.

> So the starting script in premount for dropbear should take care not
> to start if there is no /etc/crypttab and only start in case of panic.

dropbear-initramfs has nothing to do with /etc/crypttab, and it's not
because there are encrypted volumes to unlock that the boot process is
race-free: for instance these volumes might be unlocked in an unattended
fashion with a cheap PBKDF (or just plain dm-crypt).  Furthermore right
now the way the encrypted volumes are unlocked at initramfs stage, incl.
the location of the crypttab(5), is considered as an internal detail of
the cryptsetup-initramfs package :-P  That said I maintain that package
too, so I could help documenting the necessary interface if needs be :-D

> Considering my comment above should there be an option to activate
> dropbear in case of panic but no crypttab or there should be a config
> option to include and start dropbear in initramfs in case crypttab
> doesn't exists?

Once the race is eliminated from the init-bottom script (bug #-2),
run_dropbear() will unconditionally start, but be properly
terminated/cleaned before starting the init(1) binary.  That's your
second alternative; it's the simplest solution (either way there is a
bug that needs fixing) and it doesn't require further documentation from
initramfs-tools(7).  I also think it'd be reasonable to have an option
to run the premount script at panic stage instead.

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode

[Guilhem Moulin]

On Wed, 23 Oct 2019 at 16:04:31 -0400, Anton Avramov wrote:
> On Mon., Oct. 21, 2019, 21:08 Guilhem Moulin,  wrote:
>> Given the scope of this package, I strongly believe it'd make more sense
>> to merge it with src:dropbear rather than shipping a separate source
>> package.
> 
> I agree. However I don't know how I can contribute to that.

It's never too late :-)  `apt showsrc dropbear-initramfs` gives a link
to the repository (hosted at salsa in the ‘Debian’ group).  Bugs,
including feature requests, are to be filed against the Debian BTS like
for any other package: https://bugs.debian.org/dropbear-initramfs .

>>> Following dropbear-initramfs package
>> You seem to suggest that your code takes inspiration from
>> dropbear-initramfs, but your copyright information doesn't reflect
>> that.
> 
> Sorry about that! I've tried to change that, but I'm not absolutely
> certain I've done it right.

AFAICT only src/* and etc/* are taken from src:dropbear's debian/initramfs/*
(and modified by you), the rest is your own work.  Also AFAICT upstream
isn't a copyright holder for these scripts.

Regarding debian/control, your package shouldn't break or replace
dropbear <2015.68-1.  This was the case the case for drobear-initramfs
after the package split, but it doesn't apply here.  And AFAIK there is
no need to add a lower-bound on dropbear-bin's version.
 
>>> that will install dropbear and would have the appropriate initramfs
>>> scripts to start it in case the system enters rescue mode.
>>
>> Why couldn't that be done in dropbear-initramfs instead?  Right now the
>> script is run at premount stage, but I guess we could have an option to
>> run it at another stage instead.
> 
> The only argument I can think of is to give the option to have separate
> access for panic and crypt unlock.
> […]
> And in principle I can give someone the right to unlock the fs, but
> not login in case of panic where in that case and admin with higher
> access should step in.

One can already separate access levels with a single authorized_keys(5)
file containing multiple keys and different restriction options.

dropbear-initramfs spawns the sshd at init-premount stage, ie after udev
has been started and modules loaded.  AFAIK it's unreliable to start it
earlier, because there might not be any network stack yet.  So if
initramfs-tools panics earlier you'll not be able to log in remotely
will you?  (Anyway, again ‘panic’ is not a documented initramfs-tools(7)
subdirectory.)  And if it panics later, for instance due to a missing
block device, then one should be able to log-in using dropbear-initramfs
alone.

> By compatible I mean they can run together without interfering given they
> use different port. Since there are only 2 scripts in initramfs I'm not
> sure how would they suppose to share the code if we assume they are
> different packages

Using Depends: plus maybe some refactoring on the drobpear-initramfs
side would be a way, but IMHO no separate binary package (let alone a
source package) is needed :-P  dropbear-initramfs's purpose is to expose
the dropbear sshd at initramfs stage, starting as early as possible and
for as long as possible.  It can be useful to unlock encrypted volumes,
but it's not its only use-case.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#942725: interimap: please support name to appear in log lines

[Guilhem Moulin]

On Mon, 21 Oct 2019 at 00:26:58 +0200, Jonas Smedegaard wrote:
> I syncronize multiple accounts into subdirs below ~/Maildir and track 
> them as a whole using a single notmuch database.

AFAIK notmuch doesn't speak IMAP so unfortunately that approach breaks
layering.  I'm unsure how well IMAP servers cope with other processes
messing up with the storage backend.

At best the server will notice (Dovecot seems to do that unless the
*_very_dirty_syncs optimization setting is turned on) and automatically
repair indices, invalidate caches, etc.  Not sure what's the overhead
though (the server might have to stat(2) each message, which on large
mailboxes could be rather slow).  

I don't have any data, but I'd be surprised if all IMAP servers were
able to automatically gracefully recover from external writes :-P The
standard says nothing about storage backend, so personally I prefer
treating it as a black box / implementation detail and give the IMAP
server exclusive access (hence point the MUA and other components to the
IMAP backend rather than the storage backend).  Sadly not everything has
an IMAP backend though; most mail software have their own cache, thread,
sort, search, etc. logic, while MUAs could in principle rely on the IMAP
servers for that :-/

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#942808: ITP: dropbear-rescue -- A set of initramfs scripts to add and run dropbear when the system boots in rescue mode

[Guilhem Moulin]

Hi Anton,

[dropbear package maintainer here.]

On Mon, 21 Oct 2019 at 16:07:11 -0400, Anton Avramov wrote:
> For a long time now I've maintained servers remotely. One problem that
> I've faced is what when there is problem booting I lose ability to login
> remotely and help the person on premises. Further more that person can
> be not technically capable or comfortable with writing console 
> commands. There are instances where there is not even a keyboard or
> monitor attached. 

Given the scope of this package, I strongly believe it'd make more sense
to merge it with src:dropbear rather than shipping a separate source
package.

> Following dropbear-initramfs package

You seem to suggest that your code takes inspiration from dropbear-initramfs,
but your copyright information doesn't reflect that.

> that will install dropbear and would have the appropriate initramfs
> scripts to start it in case the system enters rescue mode.

Why couldn't that be done in dropbear-initramfs instead?  Right now the
script is run at premount stage, but I guess we could have an option to
run it at another stage instead.  However the first step would be to
document the ‘panic’ stage in initramfs-tools(7) (the “boot scripts”
section of the manual doesn't mention it AFAICT).

> I've tried to keep the package compatible with dropbear-initramfs
> package. 

Depending what you mean by “compatible”, please note that the only
public interface of that package is the configuration file.  Relying on
something else would be another argument in favor of integrating this
into src:dropbear.  Also I didn't review the code but at first glance I
see similarities, please see §4.13 of the Debian Policy.

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#942726: interimap: specially treated mailboxes Trash and Junk are undocumented

[Guilhem Moulin]

Control: tag -1 moreinfo

On Sun, 20 Oct 2019 at 17:58:55 +0200, Jonas Smedegaard wrote:
> Apparently interimap treats some mailboxes specially:

It shouldn't, one can trim the output of the LIST command with the
‘list-*’ options, and perform further filtering with ‘ignore-mailbox’,
but then all mailboxes listed without ‘\NonExistent’ nor ‘\NoSelect’
attributes should be treated equally.  It doesn't use the Special-Use
IMAP extension [RFC 6154] to associate mailboxes under the hood.

> If mailbox Junk or Trash exist and is non-empty,
> then interimap fails with an error requesting to delete those first.

I think it's just a coincidence that it happened for your Junk/Trash
mailbox :-P  What error message did you get exactly?  Was it

database: ERROR: Mailbox $MAILBOX exists.  Run `interimap --delete 
$MAILBOX` to delete.

?  That one appears when 1/ there is a known association in the database
for $MAILBOX between the local and remote servers, and 2/ $MAILBOX is
present on the local or remote server, but not both.  A typical
situation is when one runs `interimap` (so $MAILBOX is present on both
servers, and an association is present in the database), then later
either deletes $MAILBOX on one (and only one) server, or renames it to
something else.  Could it be what happened?   Otherwise, do you have a
reproducer?

Unfortunately I'm not sure how to best solve the above (which thus
should be added to the “Known Bugs And Limitations” section of the
manual :-P).  If you deleted $MAILBOX on one server, chances are that
you also want to delete it on the other one and in the database; and if
you renamed it to $MAILBOX2, chances are that you similarly want to
mirror the situation.  However IMAP4rev1 doesn't provide a way to
distinguish between these situations (the NOTIFY extension from RFC 5465
would help, but Dovecot support is not working properly and AFAICT no
other server supports it).  So instead it aborts and wait for manual
intervention :-/  The safest way out of that issue is to run

interimap --target=database --delete $MAILBOX

(That's also what's suggested in the development version.)  That will
remove the association in the database, but not touch the IMAP sides.
The worst that can lead to is duplicates (if the mailbox was in fact
renamed to $MAILBOX2 you'll end up with $MAILBOX and $MAILBOX2 on both
servers), not mail loss.

Silently deleting IMAP mailboxes is not an option due to the risk of
data loss.  We can't reliably determine whether the mailbox was deleted
or renamed.  (The UIDVALIDITY value can't be relied upon as there there
is no guaranty it uniquely defines a mailbox — there might even be some
collisions.)  Moreover the mailbox could have been renamed to something
not LISTed for whatever reason, in which case the program would
incorrectly believe it's been deleted. 

That said we could use the UIDVALIDITY value to *suggest* a --rename
command instead of --delete when a mailbox has disappeared on one server
*and* a new mailbox with same UIDVALIDITY value and ≥UIDNEXT/HIGHESTMODSEQ
values has appeared.  Any help to improve the heuristic and/or
documentation would be welcome, of course :-)

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#942725: interimap: please support name to appear in log lines

[Guilhem Moulin]

Hej Jonas!

On Sun, 20 Oct 2019 at 17:54:39 +0200, Jonas Smedegaard wrote:
> I happily use interimap with multiple configs.

\o/  I'd be interested to hear about the topology of your system.  I
confess that running multiple instances in parallel haven't crossed my
mind until a few months ago when another user convinced me of the
validity of the case where multiple remote accounts (say, personal +
work) are synced each with a distinct namespace of the local IMAP
server.

It's already possible with 0.4-1, but unfortunately a bit clunky and not
documented, as you may have noticed :-P.  It'll be easier with the next
version, thanks to a new ‘list-reference’ directive and a systemd
template unit file.

Doing this properly turned out not to be trivial, because it's not
always possible to ensure that the hierarchy delimiter of the different
servers is the same.  (IMAP namespaces can have their own delimiter of
course, but MUA support is not ideal so it's still desirable to include
them in LIST responses, at which point all delimiters need to match for
0.4-1 to work.)  The development version no longer has this limitation,
but it's using null characters (the only character one can't have in a
mailbox name, so the only one usable as a placeholder) internally in
place of hierarchy delimiters, which unfortunately required an
(automatic) DB schema migration.

If you have an even more “exotic” setup not covered by the above, it
might still make sense to improve the doc a bit ^^

> One thing that annoys me is that when I syncronize in parallel,
> I cannot distinguish which log lines relate to which account.
> 
> Please consider adding a "name = " option for the default section,
> which (if defined) is added as prefix to log lines.
> ..or something even more elegant :-)

I can certainly add an option with a simple printf-like format to
customize the log :-)

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#939766: [pkg-cryptsetup-devel] Bug#939766: Bug#939766: cryptsetup-initramfs: Trying to boot linux-image-5.2.0-2-amd64 fails, linux-image-4.19.0-5-amd64 works.

[Guilhem Moulin]

Control: retitle -1 cryptsetup-initramfs: Missing libgcc_s on 
linux-image-5.2.0-2-amd64

On Mon, 09 Sep 2019 at 02:55:06 +0200, Guilhem Moulin wrote:
> This on a sid system upgraded from buster with a ‘usrmerge’ layout:
> 
>  root@kvm-10487:~# ldd /sbin/cryptsetup | grep -F libc.so
>  libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f075b205000)
>  root@kvm-10487:~# readlink -f /lib/*linux-gnu/libc.so*
>  /usr/lib/x86_64-linux-gnu/libc-2.28.so
>  root@kvm-10487:~# lsinitramfs /initrd.img | grep libgcc
>  usr/lib/x86_64-linux-gnu/libgcc_s.so.1

And also:

root@kvm-10487:~# uname -r
5.2.0-2-amd64
root@kvm-10487:~# dpkg -l | grep -Fe linux-image-5.2.0-2-amd64 -e libc6 -e 
libgcc1
ii  libc6:amd642.29-1   amd64   
 GNU C Library: Shared libraries
ii  libgcc1:amd64  1:9.2.1-7amd64   
 GCC support library
ii  linux-image-5.2.0-2-amd64  5.2.9-2  amd64   
 Linux 5.2 for 64-bit PCs (signed)
root@kvm-10487:~# dpkg -L libc6 | grep -F libc.so
/lib/x86_64-linux-gnu/libc.so.6
root@kvm-10487:~# dpkg -L libgcc1 | grep -F libgcc_s.so
/lib/x86_64-linux-gnu/libgcc_s.so.1
root@kvm-10487:~# readlink -e /lib/x86_64-linux-gnu/libc.so.6 
/lib/x86_64-linux-gnu/libgcc_s.so.1
/usr/lib/x86_64-linux-gnu/libc-2.29.so
/usr/lib/x86_64-linux-gnu/libgcc_s.so.1
root@kvm-10487:~# cat /etc/ld.so.conf.d/x86_64-linux-gnu.conf 
# Multiarch support
/usr/local/lib/x86_64-linux-gnu
/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu

So AFAICT libc6 installs its soname under /lib/$MULTIARCH, and with the
default ld.so.conf ldd(1) looks there before its counterpart in /usr, so
that's what we want LIBC_DIR to be.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#939766: [pkg-cryptsetup-devel] Bug#939766: cryptsetup-initramfs: Trying to boot linux-image-5.2.0-2-amd64 fails, linux-image-4.19.0-5-amd64 works.

[Guilhem Moulin]

Control: tag -1 moreinfo

Hi,

On Mon, 09 Sep 2019 at 00:53:44 +0200, Alexander Brock wrote:
> but on my machine there is no /bin/libc.so.[0-9.-]+ instead it is in
> /usr/lib/x86_64-linux-gnu.

(I assume you meant ‘/lib.*/libc\.so\.[0-9.-]+’.)  How did you end up
with a system where /lib/*-linux-gnu/libc.so.6 is neither a symlink to a
shared library under /lib nor /usr/lib?  I can't reproduce that with a
clean bullseye nor with an upgrading system from an upgrading system
from a buster netinst.  AFAICT the existing regexp work for these
systems in both normal and ‘usrmerge’ layouts.  This on a sid system
upgraded from buster with a ‘usrmerge’ layout:

root@kvm-10487:~# ldd /sbin/cryptsetup | grep -F libc.so
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f075b205000)
root@kvm-10487:~# readlink -f /lib/*linux-gnu/libc.so*
/usr/lib/x86_64-linux-gnu/libc-2.28.so
root@kvm-10487:~# lsinitramfs /initrd.img | grep libgcc
usr/lib/x86_64-linux-gnu/libgcc_s.so.1

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#939357: cryptsetup-run: invoking "sudo cryptdisks_start" with "decrypt_keyctl" in crypttab fails

[Guilhem Moulin]

Control: reassign -1 sudo 1.8.27-1
Control: affects -1 cryptsetup
Control: merge -1 906752

On Thu, 05 Sep 2019 at 02:03:34 +0200, Guilhem Moulin wrote:
> Perhaps keyctl(1) could provide a wrapper using thread-keyring(7) as
> temporary keyring, like the attached PoC.

Of course I forgot the attachment :-P  That said I'm not sure that
using a temporary keyring and changing ownership is the way to go, it
adds complexity and not having a reachable user-keyring(7) might cause
other problems.

I was about to reassign that to sudo but noticed there is already a bug
open: https://bugs.debian.org/906752

-- 
Guilhem.
#include 
#include 
#include 
#include 
#include 

int main(int argc, char *argv[]) {
if (argc != 4)
exit(EXIT_FAILURE);
key_serial_t key = add_key(argv[1], argv[2], argv[3], strlen(argv[3]), KEY_SPEC_THREAD_KEYRING);
if (key == -1)
err(EXIT_FAILURE, "add_key");

if (keyctl_set_timeout(key, 60) == -1)
err(EXIT_FAILURE, "keyctl_set_timeout");
if (keyctl_setperm(key, KEY_POS_ALL|KEY_USR_VIEW|KEY_USR_READ|KEY_USR_WRITE|KEY_USR_SEARCH) == -1)
err(EXIT_FAILURE, "keyctl_setperm");
if (keyctl_link(key, KEY_SPEC_USER_KEYRING))
err(EXIT_FAILURE, "keyctl_link");
keyctl_unlink(key, KEY_SPEC_THREAD_KEYRING);

exit(EXIT_SUCCESS);
}


signature.asc
Description: PGP signature

Bug#939357: [pkg-cryptsetup-devel] Bug#939357: cryptsetup-run: invoking "sudo cryptdisks_start" with "decrypt_keyctl" in crypttab fails

[Guilhem Moulin]

Control: retitle -1 `decrypt_keyctl` fails when the user-keyring(7) isn't 
attached to the calling process

Hi Sebastian,

Thanks for the detailed report!  I was able to reproduce this in a fresh
Buster netinstall, taking SSH sessions and sudo(8)'s ‘-i’ flag out of
the picture.  This is what I get right after a reboot:

user@kvm-31992:~$ sudo keyctl show @u
Keyring
  99804791 --alswrv  0 65534  keyring: _uid.0
user@kvm-31992:~$ sudo keyctl show @s
Keyring
1021507217 --alswrv   1000  1000  keyring: _ses
 872133423 ---lswrv   1000 65534   \_ keyring: _uid.1000
user@kvm-31992:~$ sudo keyctl show @us
Keyring
 133933438 --alswrv  0 65534  keyring: _uid_ses.0
  99804791 --alswrv  0 65534   \_ keyring: _uid.0

Keys attached to root's user-keyring(7) can't be read.

user@kvm-31992:~$ sudo keyctl add user "foo" "bar" @u
875128385
user@kvm-31992:~$ sudo keyctl show @us
Keyring
 133933438 --alswrv  0 65534  keyring: _uid_ses.0
  99804791 --alswrv  0 65534   \_ keyring: _uid.0
 875128385 --alswrv  0 0   \_ user: foo
user@kvm-31992:~$ sudo keyctl print 875128385
keyctl_read_alloc: Permission denied

So LUKS2 volumes can't be open using token keyrings:

user@kvm-31992:~$ sudo dd if=/dev/zero of=/tmp/disk.img bs=1M count=64 
status=none
user@kvm-31992:~$ sudo cryptsetup luksFormat -q \
--pbkdf-force-iterations 4 --pbkdf-memory 32 /tmp/disk.img </dev/null
Caching passphrase for foo:  ***
root@kvm-31992:/home/user# exit

I suppose /etc/pam.d/sudo could use pam_keyinit(8) to initialize a new
session instead of doing the above dance.  Anyway cryptsetup calls
request_key("user",,NULL,0) at the moment, so unlocking LUKS2 devices
via keyring tokens will only works in sessions where the user-keyring(7)
is linked to the session-keyring(7).

On Wed, 04 Sep 2019 at 00:18:57 +0200, Sebastian Mohr wrote:
> I don't know whether there are further security implications or
> something like that,

AFAICT blindly using session-keyring(7) as it might open a window during
which any program in the user session is able to read the payload:

user@kvm-31992:~$ sudo keyctl add user "foo" "foo" @s
137160032
user@kvm-31992:~$ keyctl show @s
Keyring
 461059914 --alswrv   1000  1000  keyring: _ses
 641525861 --alswrv   1000 65534   \_ keyring: _uid.1000
 137160032 --alswrv  0 0   \_ user: foo
user@kvm-31992:~$ keyctl print 137160032
foo

(While the key is owned by the privileged user, it's possessed by the
caller.  It might even be possible to poison the session-keyring(7)
by creating %user:foo with non-root ownership.)  I think the best place
to solve that is in /etc/pam.d/sudo itself, but failing that I suppose
it's acceptable to have the payload readable by root (perhaps only as
fallback, when the caller doesn't have enough right on the key it's
adding).

Perhaps keyctl(1) could provide a wrapper using thread-keyring(7) as
temporary keyring, like the attached PoC.  Or we could even rewrite
decrypt_keyctl in C, or merge it with askpass.  In fact one of our goals
for this release cycle is to better integrate crypttab(5) with LUKS2
native keyring support, and that probably goes via adding keyring
support in askpass. 

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#939413: qemu-system: Regression: `-drive file=/dev/fdset/$FD` fails with EPERM

[Guilhem Moulin]

On Wed, 04 Sep 2019 at 19:54:51 +0200, Guilhem Moulin wrote:
>  $ qemu-system-x86_64 -display none \
>  -add-fd "fd=3,set=0" \
>  -drive "file=/dev/fdset/0,format=raw,media=disk" \
>  3<>/tmp/disk.img
>  qemu-system-x86_64: -drive file=/dev/fdset/0,format=raw,media=disk: Could 
> not open '/dev/fdset/0': Permission denied

Unsurprisingly it doesn't work if the file descriptor is read-only

$ qemu-system-x86_64 -display none \
-add-fd "fd=3,set=0" \
-drive "file=/dev/fdset/0,format=raw,media=disk" \
3/tmp/disk.img 4

signature.asc
Description: PGP signature

Bug#939413: qemu-system: Regression: `-drive file=/dev/fdset/$FD` fails with EPERM

[Guilhem Moulin]

Package: qemu-system-x86
Version: 1:4.1-1
Severity: normal
File: qemu-system

Dear Maintainer,

Using a pre-opened file descriptor to a disk image no longer work as
documented (and as it used to with ≤1:3.1+dfsg-8~deb10u1):

$ touch /tmp/disk.img
$ qemu-system-x86_64 -display none \
-add-fd "fd=3,set=0" \
-drive "file=/dev/fdset/0,format=raw,media=disk" \
3<>/tmp/disk.img
qemu-system-x86_64: -drive file=/dev/fdset/0,format=raw,media=disk: Could 
not open '/dev/fdset/0': Permission denied

Interestingly, for read-only media it does work if the file descriptor
is opened in read-only mode:

$ qemu-system-x86_64 -display none \
-add-fd "fd=3,set=0" \
-drive "file=/dev/fdset/0,format=raw,media=cdrom,readonly" \
3
ii  qemu-system-gui  1:4.1-1
pn  qemu-utils   

Versions of packages qemu-system-x86 suggests:
pn  qemu-block-extra  
pn  samba 
pn  sgabios   
pn  vde2  

-- no debconf information


signature.asc
Description: PGP signature

Bug#935827: buster-pu: package cryptsetup/2:2.1.0-5+deb10u2

[Guilhem Moulin]

Thanks, uploaded!  Hope this makes it to 10.1 :-)

And again, many thanks for your work on Buster!
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#935827: buster-pu: package cryptsetup/2:2.1.0-5+deb10u2

[Guilhem Moulin]

Package: release.debian.org
Severity: normal
Tags: buster
User: release.debian@packages.debian.org
Usertags: pu

Dear release team,

Another regression was found in cryptsetup :-/  Its scope is quite narrow as
it only affects mapped device size ≥2TiB (2³² 512-bits sectors) on 32-bits
platforms.  And AFAICT ‘crypt’ targets are not affected, only ‘integrity’ ones
are; both standalone dm-integrity volumes set up with integritysetup(8), as
well as volumes used for *experimental* authenticated disk encryption and set
up with cryptsetup(8).

In these scenarios the size overflows (due to size_t being incorrectly used in
place of uint64_t) and the device is mapped with a truncated size.  There is a
risk of data loss if the user writes inside the container, for instance while
trying to recover it, so that should IMHO be fixed via s-p-u.

This is an upstream regression from 2.1.0, so Stretch is not affected.
2:2.2.0-3 from Sid contains the cherry-picked upstream fix, but Buster's
2:2.1.0-5 (and 2:2.1.0-5+deb10u1) is affected.  Changelog since 2:2.1.0-5 is
as follows, and debdiff against 2:2.1.0-5 and 2:2.1.0-5+deb10u1 attached.

--8<->8--

cryptsetup (2:2.1.0-5+deb10u2) buster; urgency=medium

  * Cherry pick upstream commit 8f8f0b32: Fix mapped segments overflow on
32bit architectures.  Regression since 2:2.1.0-1.  (Closes: #935702)

 -- Guilhem Moulin   Mon, 26 Aug 2019 14:54:10 +0200

cryptsetup (2:2.1.0-5+deb10u1) buster; urgency=high

  * Backport upstream commits c03e3fe8, 725720df and fe4e1de5 to fix support
for LUKS2 headers without any bound keyslot.  Adding a new key slot using
the volume key was failing, both via the crypt_keyslot_add_by_volume_key()
API call and with `luksAddKey --master-key`.  The former in particular
might yield data loss if, in order to change a passphrase, an application
destroys the keyslot before adding a new one (using the volume key), cf.
#928893.  Note that doing so is *unsafe*: applications should instead use
crypt_keyslot_change_by_passphrase() from libcryptsetup >=1.6.0.
Trying to open LUKS2 volume by supplying the volume key on the command
line was also failing if there were no bound keyslot on the header.
(Closes: #934715)

 -- Guilhem Moulin   Fri, 16 Aug 2019 19:18:10 +0200

--8<->8--

A s-p-u was previously filed (#934956) — and accepted — for 2:2.1.0-5+deb10u1.
The new commit cherry-picked from upstream also includes a unit test; like
most of the test suite it'll be ignored by the build daemons as it requires
root access, but I did verify that the entire test suite still passes on amd64
and i386 (and that indeed large devices no longer overflow).

Given that Buster currently has 2:2.1.0-5, should the .changes include all
changes since that version, or only since 2:2.1.0-5+deb10u1?

Thanks for considering its inclusion in Buster!  CC'ing KiBi for the d-i ack.
Cheers,
-- 
Guilhem.
diffstat for cryptsetup-2.1.0 cryptsetup-2.1.0

 changelog  |   23 +
 gbp.conf   |1 
 patches/Fix-getting-default-LUKS2-keyslot-encryption-paramet.patch |   56 +++
 patches/Fix-mapped-segments-overflow-on-32bit-architectures.patch  |  151 
++
 patches/Fix-volume-key-file-if-no-LUKS2-keyslots-are-present.patch |   86 +
 patches/Mention-limitation-of-crypt_get_volume_key_size.patch  |   20 +
 patches/series |4 
 7 files changed, 341 insertions(+)

diff -Nru cryptsetup-2.1.0/debian/changelog cryptsetup-2.1.0/debian/changelog
--- cryptsetup-2.1.0/debian/changelog   2019-06-10 14:51:15.0 +0200
+++ cryptsetup-2.1.0/debian/changelog   2019-08-26 14:54:10.0 +0200
@@ -1,3 +1,26 @@
+cryptsetup (2:2.1.0-5+deb10u2) buster; urgency=medium
+
+  * Cherry pick upstream commit 8f8f0b32: Fix mapped segments overflow on
+32bit architectures.  Regression since 2:2.1.0-1.  (Closes: #935702)
+
+ -- Guilhem Moulin   Mon, 26 Aug 2019 14:54:10 +0200
+
+cryptsetup (2:2.1.0-5+deb10u1) buster; urgency=high
+
+  * Backport upstream commits c03e3fe8, 725720df and fe4e1de5 to fix support
+for LUKS2 headers without any bound keyslot.  Adding a new key slot using
+the volume key was failing, both via the crypt_keyslot_add_by_volume_key()
+API call and with `luksAddKey --master-key`.  The former in particular
+might yield data loss if, in order to change a passphrase, an application
+destroys the keyslot before adding a new one (using the volume key), cf.
+#928893.  Note that doing so is *unsafe*: applications should instead use
+crypt_keyslot_change_by_passphrase() from libcryptsetup >=1.6.0.
+Trying to open LUKS2 volume by supplying the volume key on the command
+line was also fai

Bug#935370: buster-pu: package lacme/0.5-1+deb10u1

[Guilhem Moulin]

Hi KiBi,

On Mon, 26 Aug 2019 at 08:22:47 +0200, Cyril Brulebois wrote:
> I'll let someone else comment on that point, to ensure I'm not making
> you jump through hoops needlessly…

I vaguely recall seeing debdiffs without ‘Closes: #XXX’, but better safe
than sorry ^^  It makes sense to track the deprecation in our BTS anyway
(#935799).  New debdiff attached.

Thanks!
Cheers,
-- 
Guilhem.
diffstat for lacme-0.5 lacme-0.5

 changelog |   11 +
 gbp.conf  |2 
 patches/0002-Issue-GET-and-POST-as-GET-requests.patch |  121 ++
 patches/series|1 
 4 files changed, 134 insertions(+), 1 deletion(-)

diff -Nru lacme-0.5/debian/changelog lacme-0.5/debian/changelog
--- lacme-0.5/debian/changelog  2018-05-09 14:17:19.0 +0200
+++ lacme-0.5/debian/changelog  2019-08-22 00:14:42.0 +0200
@@ -1,3 +1,14 @@
+lacme (0.5-1+deb10u1) buster; urgency=medium
+
+  * Link to RFC 8555 <https://tools.ietf.org/html/rfc8555> instead of the
+ACME I-D URL.
+  * Issue GET and POST-as-GET requests (RFC 8555 sec. 6.3) for the
+authorizations, order and certificate URLs.   Let's Encrypt will remove
+support of unauthenticated GETs from the V2 API on 01 Nov 2019.
+Closes: #935799.
+
+ -- Guilhem Moulin   Thu, 22 Aug 2019 00:14:42 +0200
+
 lacme (0.5-1) unstable; urgency=medium
 
   * New upstream release, adding support for v2 ACME endpoints.
diff -Nru lacme-0.5/debian/gbp.conf lacme-0.5/debian/gbp.conf
--- lacme-0.5/debian/gbp.conf   2018-05-09 14:17:19.0 +0200
+++ lacme-0.5/debian/gbp.conf   2019-08-22 00:14:42.0 +0200
@@ -1,6 +1,6 @@
 [DEFAULT]
 upstream-branch = master
-debian-branch = debian
+debian-branch = debian-buster
 upstream-tag = upstream/%(version)s
 debian-tag = debian/%(version)s
 pristine-tar = False
diff -Nru 
lacme-0.5/debian/patches/0002-Issue-GET-and-POST-as-GET-requests.patch 
lacme-0.5/debian/patches/0002-Issue-GET-and-POST-as-GET-requests.patch
--- lacme-0.5/debian/patches/0002-Issue-GET-and-POST-as-GET-requests.patch  
1970-01-01 01:00:00.0 +0100
+++ lacme-0.5/debian/patches/0002-Issue-GET-and-POST-as-GET-requests.patch  
2019-08-22 00:14:42.0 +0200
@@ -0,0 +1,121 @@
+From f9d5e53cac1c002e5983efc18e42f5a21444b182 Mon Sep 17 00:00:00 2001
+From: Guilhem Moulin 
+Date: Wed, 21 Aug 2019 17:29:19 +0200
+Subject: Issue GET and POST-as-GET requests (RFC 8555 sec. 6.3)
+
+For the  authorizations, order and certificate URLs.
+See RFC 8555 sec. 7.1.
+---
+ client|   22 +++---
+ lacme-accountd.md |2 +-
+ lacme.md  |2 +-
+ 3 files changed, 13 insertions(+), 13 deletions(-)
+
+--- a/client
 b/client
+@@ -165,16 +165,16 @@ sub request_json_decode($;$$) {
+ #
+ # JSON-encode the hash reference $h and send it to the ACME server $uri
+ # encapsulated it in a JSON Web Signature (JWS).
+-# https://tools.ietf.org/html/draft-ietf-acme-acme-12
++# https://tools.ietf.org/html/rfc8555
+ #
+-sub acme($@) {
+-my $uri = shift;
++sub acme($;$) {
++my ($uri, $h) = @_;
+ die "Missing nonce\n" unless defined $NONCE;
+ 
+ # Produce the JSON Web Signature: RFC 7515 section 5
+ my %header = ( alg => 'RS256', nonce => $NONCE, url => $uri );
+ defined $KID ? ($header{kid} = $KID) : ($header{jwk} = $JWK);
+-my $payload = encode_base64url(json()->encode({ @_ }));
++my $payload = defined $h ? encode_base64url(json()->encode($h)) : "";
+ my $protected = encode_base64url(json()->encode(\%header));
+ my $data = $protected .'.'. $payload;
+ $S->printflush($data, "\r\n");
+@@ -204,7 +204,7 @@ sub acme_resource($%) {
+ request(HEAD => $RES{newNonce});
+ }
+ my $uri = $RES{$r} // die "Unknown resource '$r'\n";
+-acme($uri, @_);
++acme($uri, {@_});
+ }
+ 
+ # Set the key ID (registration URI)
+@@ -237,7 +237,7 @@ if ($COMMAND eq 'account') {
+ 
+ if ($r->is_success()) {
+ $KID = $r->header('Location');
+-$r = acme($KID, %h);
++$r = acme($KID, \%h);
+ request_json_decode($r, 1, \*STDOUT)
+ if $r->is_success() and $r->content_type() eq 'application/json';
+ }
+@@ -264,7 +264,7 @@ elsif ($COMMAND eq 'newOrder') {
+ my $order = request_json_decode($r);
+ 
+ foreach (@{$order->{authorizations}}) {
+-my $authz = request_json_decode(request(GET => $_));
++my $authz = request_json_decode(acme($_));
+ next unless $authz->{status} eq 'pending';
+ 
+ my $identifier = $authz->{identifier}->{value};
+@@ -288,7 +288,7 @@ elsif ($COMMAND eq 'newOrder') {
+ die "Can't open $challenge->{token}: $!";
+ }
+ 
+-$r = acme($challenge->{url}

Bug#935799: lacme: `client` uses unauthenticated GETs instead of POST-as-GETs

[Guilhem Moulin]

Source: lacme
Version: 0.5-1
Severity: important

Per RFC 8555 sec 6.3 the Let's Encrypt folks are deprecating unauthenticated
GETs from their v2 API.  Support for these requests will be removed on
*Nov 01 2019* [0].

lacme uses the v2 API since 0.5, and removing support for unauthenticated
GETs means that applying for certificate issuance will stop working on
Nov 01.  (Hence the severity of this bug should be RC after that.)

-- 
Guilhem.

[0] 
https://community.letsencrypt.org/t/acme-v2-scheduled-deprecation-of-unauthenticated-resource-gets


signature.asc
Description: PGP signature

Bug#935702: [pkg-cryptsetup-devel] Bug#935702: Wrong DM device size due to integer truncation

[Guilhem Moulin]

Control: tag -1 fixed-upstream

On Mon, 26 Aug 2019 at 11:08:35 +0200, Milan Broz wrote:
> Fixed here 
> https://gitlab.com/cryptsetup/cryptsetup/commit/8f8f0b3258152a260c6a40be89b485f943f81484

Thanks, Milan!

> I'll do minor release soon, but perhaps it would be better to
> cherrypick the patch directly.

We'll likely have to backport your patch to 2.2.0-3 since we're a bit in
a rush if we want the bugfix in the upcoming point release (10.1 is
scheduled on Sep 07 after a freeze of one week): bug fixes should land
to unstable first per Release Team policy.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#935702: [pkg-cryptsetup-devel] Bug#935702: Wrong DM device size due to integer truncation

[Guilhem Moulin]

Control: retitle -1 DM device size ≥2³² 512-bits sectors is truncated on 
32-bits platforms
Control: tag -1 + upstream

Hi,

On Sun, 25 Aug 2019 at 12:43:26 +, n...@waifu.club wrote:
> Not only the access to protected data is lost, the integritysetup's "open"
> operation actually succeeds. All reads on the incorrectly created DM device
> will of course fail with I/O errors due to bad integrity tags, but all
> writes will happily write wrong tags at wrong places! This makes it very
> easy for the administrator to destroy the data while trying to recover with
> --integrity-recovery-mode.

Would you mind sharing your test vector, either here or the upstream bug
tracker at https://gitlab.com/cryptsetup/cryptsetup/issues ?  While it's
clear there is a bug, I was unable to reproduce your observations in the
arguably most common cases, namely block devices formatted as LUKS1 or
LUKS2 with the default parameters (and a payload size exceeding ≥2³²
512-bits sectors).  The only function of the dm_*_target_set() family
called is dm_crypt_target_set(), and always with 0 as segment offset and
size.

`cryptsetup plainOpen -b $LARGE_NUMBER` does yield a call to 
dm_crypt_target_set()
with a truncated dmd.size, but the proper size is used before opening
the device thanks to device_block_adjust().

Or does the bug only applies to integrity targets?  I'm indeed able to
reproduce for these targets:

$ uname -r
4.19.0-5-686
$ truncate -s3T /tmp/disk.img
$ losetup /dev/loop0 /tmp/disk.img

$ integritysetup format -s 512 --no-wipe -q /dev/loop0
$ integritysetup dump /dev/loop0 | grep provided_data_sectors
provided_data_sectors 6392379512
$ integritysetup open /dev/loop0 integrity_test
$ echo $?
0
$ dmsetup table integrity_test
0 2097412216 integrity …

syslog contains (checksum errors are due to no-wipe):
[…] device-mapper: integrity: Checksum failed at sector 0x7d03f780
[…] device-mapper: integrity: Checksum failed at sector 0x7d03f780
[…] Buffer I/O error on dev dm-2, logical block 262176496, async page read


## experimental --integrity extension
$ cryptsetup luksFormat -q --integrity hmac-sha256 --integrity-no-wipe \
--pbkdf-force-iterations 4 --pbkdf-memory 32 /dev/loop0 <<

signature.asc
Description: PGP signature

Bug#935727: [gpg-key2ps] Print the long keyID instead of the short ID

[Guilhem Moulin]

Control: severity -1 wishlist

Hi,

On Sun, 25 Aug 2019 at 19:11:10 +0200, Jörg Frings-Fürst wrote:
> please print the long keyID instead of the short keyID.

We were matching the output of `gpg --fingerprint --list-key`.  They now
only show the fingerprint and I guess it makes sense to do the same (and
also for gpg-key2latex).

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#935650: netcat-openbsd: valid arguments disallowed

[Guilhem Moulin]

Control: retitle -1 netcat-openbsd: Unable to specify client socket for 
UNIX-domain datagram sockets
Control: found -1 1.187-1
Control: found -1 1.195-2

Hi,

On Sat, 24 Aug 2019 at 20:25:00 +, astian wrote:
> Looking at the patch I don't trust this is the only behaviour change.  I
> don't understand why this divergence from upstream was introduced and I
> wish it was reverted altogether.

The patch was added to support the following calls for consistency with
netcat-traditional:

nc -l -s 127.0.0.1 -p 12345 ## listen on AF_INET socket 
INADDR_LOOPBACK:12345
nc -l -p 12345  ## listen on AF_INET socket ADDR_ANY:12345

It'd be a regression to stop supporting these forms, or any of the
others from https://bugs.debian.org/897020#17 .

> But if that's not possible below is a patch that you can fold into the
> existing one.
> […]
> - } else if (argc == 1 && !pflag && !sflag) {
> + } else if (argc == 1 && !pflag && (!sflag || family == AF_UNIX)) {

In combination with ‘-U’ the ‘-s’ option only makes sense for datagrams,
no?  AFAICT it's a no-op for stream sockets; might make sense to error
out unless ‘-u’ is set.

As for your other patches, I'm personally not keen to further diverge
with upstream (especially when it comes to adding new flags/options) —
but hopefully they'll eventually pick up your fixes or address your
concerns ;-)

Thanks for the reports and patches!
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#935370: buster-pu: package lacme/0.5-1+deb10u1

[Guilhem Moulin]

Package: release.debian.org
Severity: normal
Tags: buster
User: release.debian@packages.debian.org
Usertags: pu

Dear release team,

Per RFC 8555 sec 6.3 the Let's Encrypt folks are deprecating
unauthenticated GETs from their v2 API.  Support for these requests will
be removed on *Nov 01 2019* (so likely between Debian 10.1 and 10.2) [0].

lacme uses the v2 API by default since 0.5, and removing support for
unauthenticated GETs means that applying for certificate issuance will
stop working.  Replacing GETs with POST-as-GETs is trivial (debdiff
attached), and I'd like to fix that in Buster via s-p-u.

(0.6 from Sid is not affected, and neither is 0.2 from Stretch as the
latter supports only the v1 API.)

Cheers,
-- 
Guilhem.

[0] 
https://community.letsencrypt.org/t/acme-v2-scheduled-deprecation-of-unauthenticated-resource-gets
diffstat for lacme-0.5 lacme-0.5

 changelog |   10 +
 gbp.conf  |2 
 patches/0002-Issue-GET-and-POST-as-GET-requests.patch |  121 ++
 patches/series|1 
 4 files changed, 133 insertions(+), 1 deletion(-)

diff -Nru lacme-0.5/debian/changelog lacme-0.5/debian/changelog
--- lacme-0.5/debian/changelog  2018-05-09 14:17:19.0 +0200
+++ lacme-0.5/debian/changelog  2019-08-22 00:14:42.0 +0200
@@ -1,3 +1,13 @@
+lacme (0.5-1+deb10u1) buster; urgency=medium
+
+  * Link to RFC 8555 <https://tools.ietf.org/html/rfc8555> instead of the
+ACME I-D URL.
+  * Issue GET and POST-as-GET requests (RFC 8555 sec. 6.3) for the
+authorizations, order and certificate URLs.   Let's Encrypt will remove
+support of unauthenticated GETs from the V2 API on 01 Nov 2019.
+
+ -- Guilhem Moulin   Thu, 22 Aug 2019 00:14:42 +0200
+
 lacme (0.5-1) unstable; urgency=medium
 
   * New upstream release, adding support for v2 ACME endpoints.
diff -Nru lacme-0.5/debian/gbp.conf lacme-0.5/debian/gbp.conf
--- lacme-0.5/debian/gbp.conf   2018-05-09 14:17:19.0 +0200
+++ lacme-0.5/debian/gbp.conf   2019-08-22 00:14:42.0 +0200
@@ -1,6 +1,6 @@
 [DEFAULT]
 upstream-branch = master
-debian-branch = debian
+debian-branch = debian-buster
 upstream-tag = upstream/%(version)s
 debian-tag = debian/%(version)s
 pristine-tar = False
diff -Nru 
lacme-0.5/debian/patches/0002-Issue-GET-and-POST-as-GET-requests.patch 
lacme-0.5/debian/patches/0002-Issue-GET-and-POST-as-GET-requests.patch
--- lacme-0.5/debian/patches/0002-Issue-GET-and-POST-as-GET-requests.patch  
1970-01-01 01:00:00.0 +0100
+++ lacme-0.5/debian/patches/0002-Issue-GET-and-POST-as-GET-requests.patch  
2019-08-22 00:14:42.0 +0200
@@ -0,0 +1,121 @@
+From f9d5e53cac1c002e5983efc18e42f5a21444b182 Mon Sep 17 00:00:00 2001
+From: Guilhem Moulin 
+Date: Wed, 21 Aug 2019 17:29:19 +0200
+Subject: Issue GET and POST-as-GET requests (RFC 8555 sec. 6.3)
+
+For the  authorizations, order and certificate URLs.
+See RFC 8555 sec. 7.1.
+---
+ client|   22 +++---
+ lacme-accountd.md |2 +-
+ lacme.md  |2 +-
+ 3 files changed, 13 insertions(+), 13 deletions(-)
+
+--- a/client
 b/client
+@@ -165,16 +165,16 @@ sub request_json_decode($;$$) {
+ #
+ # JSON-encode the hash reference $h and send it to the ACME server $uri
+ # encapsulated it in a JSON Web Signature (JWS).
+-# https://tools.ietf.org/html/draft-ietf-acme-acme-12
++# https://tools.ietf.org/html/rfc8555
+ #
+-sub acme($@) {
+-my $uri = shift;
++sub acme($;$) {
++my ($uri, $h) = @_;
+ die "Missing nonce\n" unless defined $NONCE;
+ 
+ # Produce the JSON Web Signature: RFC 7515 section 5
+ my %header = ( alg => 'RS256', nonce => $NONCE, url => $uri );
+ defined $KID ? ($header{kid} = $KID) : ($header{jwk} = $JWK);
+-my $payload = encode_base64url(json()->encode({ @_ }));
++my $payload = defined $h ? encode_base64url(json()->encode($h)) : "";
+ my $protected = encode_base64url(json()->encode(\%header));
+ my $data = $protected .'.'. $payload;
+ $S->printflush($data, "\r\n");
+@@ -204,7 +204,7 @@ sub acme_resource($%) {
+ request(HEAD => $RES{newNonce});
+ }
+ my $uri = $RES{$r} // die "Unknown resource '$r'\n";
+-acme($uri, @_);
++acme($uri, {@_});
+ }
+ 
+ # Set the key ID (registration URI)
+@@ -237,7 +237,7 @@ if ($COMMAND eq 'account') {
+ 
+ if ($r->is_success()) {
+ $KID = $r->header('Location');
+-$r = acme($KID, %h);
++$r = acme($KID, \%h);
+ request_json_decode($r, 1, \*STDOUT)
+ if $r->is_success() and $r->content_type() eq 'application/json';
+ }
+@@ -264,7 +264,7 @@ elsif ($COMMAND eq 'newOrder') {
+ my $order = request_json_decode($r);
+ 
+ foreach (@{$order->{authorizations}})

Bug#934956: buster-pu: package cryptsetup/2:2.1.0-5+deb10u1

[Guilhem Moulin]

Thanks, uploaded.  And sorry the wall of text in the original report ^^

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#934956: buster-pu: package cryptsetup/2:2.1.0-5+deb10u1

[Guilhem Moulin]

Package: release.debian.org
Severity: normal
Tags: buster
User: release.debian@packages.debian.org
Usertags: pu

Dear release team,

Buster's cryptsetup (2:2.1.0-5) doesn't cope well with LUKS2 headers
without any bound keyslot: adding a new key slot to such a header fails,
both via the crypt_keyslot_add_by_volume_key() API call and with
`luksAddKey --master-key`.

This alone doesn't warrant a s-p-u, but some applications seem to use
the following sequences of API calls to change a passphrase:

keyslot = crypt_volume_key_get(cd, …, volume_key, old_passphrase);
crypt_keyslot_destroy(cd, keyslot);
crypt_keyslot_add_by_volume_key(cd, 0, volume_key, new_passphrase);

This is a *very* bad idea in the first place, and they should use
crypt_keyslot_change_by_passphrase() instead: if there was only one
active keyslot and for some reason the crypt_keyslot_add_by_volume_key()
call fails (for instance due to an hardware failure, or because the
PBKDF benchmark triggers the OOM killer) then the entire volume is lost…
However the libcryptsetup bug makes it much, much worse as the call
*always* fails on LUKS2 headers without any bound keyslot.  This is what
happened in #928893 (gnome-disk-utility: disk content permanently lost
when changing LUKS password).

Using codesearch.d.n I found that (as far as sid is concerned) beside
src:cryptsetup, only src:libblockdev and src:cryptmount are calling
crypt_keyslot_destroy().  AFAICT src:cryptmount is making a sane use of
the call [0]; libblockdev is affected in Buster but per #932588 will be
fixed to use crypt_keyslot_change_by_passphrase() in the upcoming point
release.  Still there might be third party programs we don't know about,
and considering the serious risk of data loss I think it makes sense to
fix the libcryptsetup bug in the next point release, too.  Changelog
since 2:2.1.0-5 is as follows, and debdiff attached.

cryptsetup (2:2.1.0-5+deb10u1) buster; urgency=high

  * Backport upstream commits c03e3fe8, 725720df and fe4e1de5 to fix support
for LUKS2 headers without any bound keyslot.  Adding a new key slot using
the volume key was failing, both via the crypt_keyslot_add_by_volume_key()
API call and with `luksAddKey --master-key`.  The former in particular
might yield data loss if, in order to change a passphrase, an application
destroys the keyslot before adding a new one (using the volume key), cf.
#928893.  Note that doing so is *unsafe*: applications should instead use
crypt_keyslot_change_by_passphrase() from libcryptsetup >=1.6.0.
Trying to open LUKS2 volume by supplying the volume key on the command
line was also failing if there were no bound keyslot on the header.
(Closes: #934715)

 -- Guilhem Moulin   Fri, 16 Aug 2019 19:18:10 +0200

The 3 cherry-picked patches are all backported from 2.2.0 [1,2], and the
version in sid is not affected.  (The one in Stretch is not affected
either as it doesn't have LUKS2 support.)  The diff also includes unit
tests, but note that the tests in question need root access hence are
ignored by the build daemons.  I did ensure that the whole test-suite
still passes on amd64, though.

In case you're unhappy with this changeset, then I propose to only
include the first 2 patches.  IMHO what should really be fixed in Buster
is the libcryptsetup part.  For the CLI part (third patch) the risk of
data loss is lower as the volume key is stored in a file.

Thanks for considering its inclusion in Buster!  CC'ing KiBi for the d-i ack.
Cheers,
-- 
Guilhem.

[0] 
https://codesearch.debian.net/show?file=cryptmount_5.3.1-1%2Farmour-luks.c=247
[1] https://gitlab.com/cryptsetup/cryptsetup/issues/466
[2] https://gitlab.com/cryptsetup/cryptsetup/issues/470
diffstat for cryptsetup-2.1.0 cryptsetup-2.1.0

 changelog  |   16 +
 gbp.conf   |1 
 patches/Fix-getting-default-LUKS2-keyslot-encryption-paramet.patch |   56 
++
 patches/Fix-volume-key-file-if-no-LUKS2-keyslots-are-present.patch |   86 
++
 patches/Mention-limitation-of-crypt_get_volume_key_size.patch  |   20 ++
 patches/series |3 
 6 files changed, 182 insertions(+)

diff -Nru cryptsetup-2.1.0/debian/changelog cryptsetup-2.1.0/debian/changelog
--- cryptsetup-2.1.0/debian/changelog   2019-06-10 14:51:15.0 +0200
+++ cryptsetup-2.1.0/debian/changelog   2019-08-16 19:18:10.0 +0200
@@ -1,3 +1,19 @@
+cryptsetup (2:2.1.0-5+deb10u1) buster; urgency=high
+
+  * Backport upstream commits c03e3fe8, 725720df and fe4e1de5 to fix support
+for LUKS2 headers without any bound keyslot.  Adding a new key slot using
+the volume key was failing, both via the crypt_keyslot_add_by_volume_key()
+API call and with `luksAddKey --master-key`.  The former in particular
+might yield data loss if, in order to change a passphrase, an applicat

Bug#934753: dropbear-initramfs: please add an autopkgtest

[Guilhem Moulin]

On Fri, 16 Aug 2019 at 14:45:17 +0200, Guilhem Moulin wrote:
> On Wed, 14 Aug 2019 at 14:54:08 +0200, Johannes 'josch' Schauer wrote:
>> when I upgraded my Squeeze box to Jessie, remote unlocking via dropbear
>> in my initramfs stopped working. This is a remote host in a datacenter,
>> so I cannot directly investigate the issue.
> 
> Interesting, once you manage to boot I'd be interested to know the
> reason.

Oh wait, you upgraded from Squeeze to Jessie via Wheezy, right?
Directly upgrading from Squeeze to Jessie is not a supported upgrade
path.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#934753: dropbear-initramfs: please add an autopkgtest

[Guilhem Moulin]

On Wed, 14 Aug 2019 at 14:54:08 +0200, Johannes 'josch' Schauer wrote:
> when I upgraded my Squeeze box to Jessie, remote unlocking via dropbear
> in my initramfs stopped working. This is a remote host in a datacenter,
> so I cannot directly investigate the issue.

Interesting, once you manage to boot I'd be interested to know the
reason.  Also FWIW I also use remote unlocking via dropbear on
production systems, and my setups have survived all upgrade paths, incl.
Squeeze → Jessie.  And AFAICT you're the first to report a breakage at
dist-upgrade stage, so I'm not entirely convinced this would have been
caught by a simple autopkgtest :-P

> If you like the script, then I could prepare a patch against
> src:dropbear which implements an autopkgtest that runs the script.

Can't hurt indeed, thanks!  A few comments inlined below.

> pkgs="linux-image-amd64,openssh-server,systemd-sysv,libpam-systemd,policykit-1"
> pkgs="$pkgs,iproute2,util-linux,e2fsprogs,ifupdown,net-tools,netbase"
> pkgs="$pkgs,iputils-ping,isc-dhcp-client,lvm2,parted,cryptsetup"
> pkgs="$pkgs,dropbear-initramfs,busybox,fdisk,mmdebstrap,udev"

If you include ‘dropbear-initramfs’ I guess you want ‘cryptsetup-initramfs’
not ‘cryptsetup’.  Also AFAICT ‘iputils-ping’, ‘parted’ and ‘busybox’
are not needed (the latter will be pulled by ‘cryptsetup-initramfs’ and
‘dropbear-initramfs’).

> auto ens3

Is the interface name reliable?  I was under the impression it wasn't
because it depends on how QEMU arranges its devices, unlike the use of
‘eth0’ after adding ‘net.ifnames=0’ to the kernel cmdline.

> qemu-img convert -O qcow2 debian-unstable.img debian-unstable.qcow2

The conversion from raw to qcow2 format is not needed, is it?

> qemu-system-x86_64 -enable-kvm -m 4G -net user,hostfwd=tcp::10022-:22 \

4GiB sounds really overkill here, surely 1GiB is enough?  This is what I
use for testing the various device stacks before src:cryptsetup uploads.

I'd also bind to INADDR_LOOPBACK, change the NIC and drive model from
the default (resp. e1000 and ide) to virtio, and pass `-no-user-config
-nodefaults`.  Maybe also set the CPU model to host.  Might also help to
create a virtio-rng device, given that key material is generated on the
guest.

> printf myinsecurepassphrase | cryptsetup luksFormat /dev/sdb3 -

To speep up things I suggest to skip the the PBKDF benchmark by passing
`--pbkdf-force-iterations 4 --pbkdf-memory 32` (for Argon2), or
`--pbkdf-force-iterations 1000` (for PBKDF2).  cryptsetup <2.0 (up to
Stretch) are only able to format and open LUKSv1 volumes, which only
supports PBKDF2 as PBKDF algorithm; since cryptsetup 2.0 a new LUKS
version format is available (and is the default as for Buster) with
support for both Argon2i/d (default) and PBKDF2.

> cat > "/mnt/etc/initramfs-tools/conf.d/dropbear" << END
> IP=":ens3:dhcp"
> END

AFAICT it's redundant since you have the same thing as boot parameter.
 
> chroot "/mnt" apt-get -y install lvm2 grub2 linux-image-amd64 openssl 
> cryptsetup dropbear-initramfs busybox udev mount systemd-sysv util-linux 
> e2fsprogs initramfs-tools cryptsetup-initramfs cryptsetup-run console-setup 
> openssh-server ifupdown net-tools netbase iproute2 libpam-systemd policykit-1 
> iputils-ping isc-dhcp-client

Some of these are redundant, and might not be marked as manually
installed on a normal installation.  ‘cryptsetup-run’, ‘busybox’,
‘initramfs-tools’ at least.

Thanks!
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#934863: apticron-systemd: Directory '/var/lib/apticron' and manpage missing

[Guilhem Moulin]

Package: apticron-systemd
Version: 1.2.1
Severity: normal

Dear Maintainer,

‘debian/dirs’ and ‘debian/manpages’ are only installed into the first
binary package acted on, namely apticron.  Hence apticron-systemd is
lacking /usr/share/man/man1/apticron.1.gz and /var/lib/apticron/.

The latter in particular causes the service to fail with

mv: cannot move '/tmp/apticron.XX' to '/var/lib/apticron/last_run': 
No such file or directory

But the notification message is sent out anyway, so I'm unsure about the
severity of this bug.

Cheers,
-- 
Guilhem.

-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 4.19.0-5-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), 
LANGUAGE=en_US:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages apticron-systemd depends on:
ii  apt1.8.3
ii  bsd-mailx [mailx]  8.1.2-0.20180807cvs-1+b1
ii  bzip2  1.0.6-9.2
ii  dpkg   1.19.7
ii  systemd241-7

Versions of packages apticron-systemd recommends:
ii  apt-listchanges  3.20
ii  gpg  2.2.17-3
ii  iproute2 5.2.0-1

apticron-systemd suggests no packages.

-- no debconf information


signature.asc
Description: PGP signature

Bug#934715: libcryptsetup12: crypt_keyslot_add_by_volume_key() fails on a LUKS2 header where all bound key slots were deleted

[Guilhem Moulin]

Package: libcryptsetup12
Version: 2:2.1.0-7
Severity: important
Tags: upstream

(Cloning upstream issue #466 so we can track it for Buster, Bullseye and sid.)

Even when all (bound) key slots were removed from a LUKS header, the header is
still salvageable given a copy of the master key.

The crypt_keyslot_add_by_volume_key() API call works for LUKSv1 headers
without keyslot, but fails for LUKSv2:

$ dd if=/dev/zero of=./disk.img bs=1M count=64
$ cryptsetup luksFormat --pbkdf-force-iterations 1000 \
--type luks1 -q ./disk.img <<#include 
#include 
#include 

#include "libcryptsetup.h"

int main(int argc, char *argv[]) {
struct crypt_device *cd = NULL;
if (crypt_init(, argv[1]))
errx(EXIT_FAILURE, "Error: crypt_init");

if (crypt_load(cd, NULL, NULL))
errx(EXIT_FAILURE, "Error: crypt_load");

size_t vk_size = crypt_get_volume_key_size(cd);
char *volume_key = malloc(vk_size);

int keyslot = crypt_volume_key_get(cd, CRYPT_ANY_SLOT, volume_key, _size,
argv[2], strlen(argv[2]));
if (keyslot < 0)
errx(EXIT_FAILURE, "Error: crypt_volume_key_get");

if (crypt_keyslot_destroy(cd, keyslot))
errx(EXIT_FAILURE, "Error: crypt_keyslot_destroy");

if (crypt_keyslot_add_by_volume_key(cd, keyslot, volume_key, vk_size,
argv[3], strlen(argv[3])))
errx(EXIT_FAILURE, "Error: crypt_keyslot_add_by_volume_key");

return EXIT_SUCCESS;
}


signature.asc
Description: PGP signature

Bug#934146: README.initramfs directs reader to nonexistent /usr/share/doc/cryptsetup/README.Debian

[Guilhem Moulin]

Control: severity -1 minor
Control: tag -1 pending

On Wed, 07 Aug 2019 at 09:16:33 -0400, Ian Kelling wrote:
> "You can unlock your rootfs on bootup remotely, using SSH to log in to
> the booting system while it's running with the initramfs mounted.
> Consult cryptsetup's /usr/share/doc/cryptsetup/README.Debian section 8
> for details."

On src:cryptsetup <2:2.0.3-1 the file is shipped by the ‘cryptsetup’
binary package as /usr/share/doc/cryptsetup/README.Debian.gz.

On src:cryptsetup ≥2:2.0.3-1 and <2:2.1.0-6 the file is shipped by the
‘cryptsetup-run’ binary package as 
/usr/share/doc/cryptsetup-run/README.Debian.gz.

On src:cryptsetup ≥2:2.1.0-6 the file is shipped by the
‘cryptsetup’ binary package as /usr/share/doc/cryptsetup/README.Debian.gz.

See also #904916 and #929922.

> And, the instructions in the older
> /usr/share/doc/cryptsetup/README.Debian no longer work, they said:
> # echo -n "my_secret_passphrase" > /lib/cryptsetup/passfifo

It doesn't?  That command has always been racy because the SSHd is
launched before the passfifo is created.  (See …/README.Debian.gz sec. 8
for a proper solution.)  But it should not be *more* racy with Buster
than with Stretch or any earlier release.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#933836: cryptkeyctl: When using keyscript "decrypt_keyctl" in crypttab, update-initramfs fails

[Guilhem Moulin]

Control: retitle -1 cryptsetup-initramfs: hook files should give hints about 
missing packages to install
Control: severity -1 minor

Hi,

On Sun, 04 Aug 2019 at 10:45:33 +0200, Sebastian Mohr wrote:
> After some debugging, I found out, that this script copies the file
> "/bin/keyctl" to the initramfs. But this file, belonging to the package
> "keyutils", is not installed.

FWIW this is documented in /usr/share/doc/cryptsetup/README.keyctl (or
/usr/share/doc/cryptsetup-run/README.keyctl for src:cryptsetup between
2:2.0.3-1 and 2:2.1.0-5).

> I would suggest at least suggesting or recommending "keyutils" (and other
> packages being needed for the other keyscripts)

Correct dependency declarations would introduce a lot of clutter here,
for ‘keyscript=decrypt_keyctl’ alone we would need two more binary
packages:

  Package: cryptsetup-keyscript-keyctl
  Depends: cryptsetup, keyctl
  [Ships /lib/cryptsetup/scripts/decrypt_keyctl.]

  Package: cryptsetup-initramfs-keyscript-keyctl
  Depends: cryptsetup-initramfs, cryptsetup-keyscript-keyctl
  [Ships /usr/share/initramfs-tools/hooks/cryptkeyctl.]

And similarily for other keyscripts.  Last time we talked about it we
decided that it was not worth the clutter.  We don't want the less
fine-grained dependency declaration via Recommends either (which should
be on ‘cryptsetup’ not ‘cryptsetup-initramfs’, by the way: keyscripts
can be used outside the initramfs stage too), because that would mean on
systems without --no-install-recommends (ie the default), installing
‘cryptsetup’ would clutter the system with the OpenSC daemon and other
tools that are likely not needed.

Instead we decided to document keyscript setup under
/usr/share/doc/cryptsetup/README.*.

> or giving out a clearer error message on failure, like 'File
> "/bin/keyctl" not found, please install package "keyutils".' or
> something like that.

I guess we could do that in hook files.  Won't help when the device is
opened outside initramfs stage though (for instance via `cryptdisks_start`
or sysvinit scripts).

Perhaps /usr/share/initramfs-tools/hook-functions:copy_exec() could give
a more helpful message mentioning the name of the file that couldn't be
copied to the initramfs.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#933610: signify-openbsd-keys: Please upload 2018.5

[Guilhem Moulin]

Package: signify-openbsd-keys
Version: 2018.4
Severity: wishlist

Dear Maintainer,

I noticed you prepared a release with the keys for OpenBSD 66


https://salsa.debian.org/debian/signify-openbsd-keys/commit/14edcb216bf56cbeec6cf872042350488a75b1ab

but didn't follow with an upload to sid.  Could you please upload now
that Buster is released? :-)  (I'm relying on these keys to repack new
netcat-openbsd releases.)

Thanks!
Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#930228: partman-crypto: cryptsetup's initramfs integration was moved to a separate package

[Guilhem Moulin]

Control: severity -1 grave

Hi,

On Thu, 25 Jul 2019 at 22:31:36 +0100, Ben Hutchings wrote:
>> Thanks to the Recommends: d-i will automatically pull the initramfs
>> integration, at least on systems where APT::Install-Recommends hasn't
>> been turned off by preseeding.  (The Recommends: cryptsetup-run is there
>> to improve the upgrade path, cf. #932625.)  I'm therefore only raising
>> the severity to ‘normal’.
> 
> APT::Install-Recommends is only enabled after the base-installer phase.
> of installation.  I don't know what stage cryptsetup is installed at,
> but I suggest it's worth checking that this assumption is correct.

Thanks for the hint, ‘cryptsetup-initramfs’ is not pulled in indeed.  I
guess someone would have found out sooner or better, but the sooner the
better :-)

The attached patch fixes this.  I'll leave it to you if you want to
clone this bug and close -1, or alternatively downgrade its severity to
conditionally implement the initramfs integration:

| The real fix would be to have a detection logic triggering `apt-install
| cryptsetup` whenever there are crypt targets in the dm table, and
| `apt-install cryptsetup-initramfs` if any volume needs to be unlocked at
| initramfs stage, i.e., holding /, /usr, and/or the resume device(s).

Cheers,
-- 
Guilhem.
From b72b0934eb4c729d5fef462bb832aec6665513c8 Mon Sep 17 00:00:00 2001
From: Guilhem Moulin 
Date: Fri, 26 Jul 2019 23:24:33 +0200
Subject: [PATCH] finish.d/crypto_aptinstall: Install cryptsetup-initramfs, not
 cryptsetup.

cryptsetup's initramfs integration was moved to a separate package.
Cf. #930228.
---
 finish.d/crypto_aptinstall | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/finish.d/crypto_aptinstall b/finish.d/crypto_aptinstall
index 047d1a3..a26f5f0 100755
--- a/finish.d/crypto_aptinstall
+++ b/finish.d/crypto_aptinstall
@@ -34,6 +34,6 @@ if grep -q " device-mapper$" /proc/misc; then
 # on an LVM LV on top of an encrypted device
 if type dmsetup >/dev/null 2>&1 && \
 	   dmsetup table | cut -d' ' -f4 | grep -q "crypt" 2>/dev/null; then
-		apt-install cryptsetup || true
+		apt-install cryptsetup-initramfs || true
 	fi
 fi
-- 
2.22.0



signature.asc
Description: PGP signature

Bug#930228: partman-crypto: cryptsetup's initramfs integration was moved to a separate package

[Guilhem Moulin]

Control: severity -1 normal

On Sat, 08 Jun 2019 at 22:05:42 +0200, Guilhem Moulin wrote:
> Our (cryptsetup maintaining team) plan is to rename ‘cryptsetup-run’ to
> ‘cryptsetup’ once Buster is released, hence this bug should be RC at
> this point: with `apt-install cryptsetup` the initramfs integration
> won't be installed anymore.

Since 2:2.1.0-6 we reclaimed the name ‘cryptsetup’:

Package: cryptsetup
Recommends: cryptsetup-initramfs, cryptsetup-run
Description: disk encryption support - startup scripts

Package: cryptsetup-initramfs
Depends: cryptsetup
Description: disk encryption support - initramfs integration

Package: cryptsetup-run
Depends: cryptsetup
Description: transitional dummy package for cryptsetup

Thanks to the Recommends: d-i will automatically pull the initramfs
integration, at least on systems where APT::Install-Recommends hasn't
been turned off by preseeding.  (The Recommends: cryptsetup-run is there
to improve the upgrade path, cf. #932625.)  I'm therefore only raising
the severity to ‘normal’.

> The real fix would be to have a detection logic triggering `apt-install
> cryptsetup` whenever there are crypt targets in the dm table, and
> `apt-install cryptsetup-initramfs` if any volume needs to be unlocked at
> initramfs stage, i.e., holding /, /usr, and/or the resume device(s).

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#932891: [pkg-cryptsetup-devel] Bug#932891: cryptsetup: WARNING: Couldn't determine root device on ZFS error

[Guilhem Moulin]

Control: reassign -1 cryptsetup-initramfs
Control: forcemerge -1 820888

Hi,

On Wed, 24 Jul 2019 at 12:42:45 +0200, Mátyás Csere wrote:
> The symptoms are exactly as described on:
> https://bugs.launchpad.net/debian/+source/cryptsetup/+bug/1830110
> The proposed patch works on Buster too.

Please the merged bugs, in particular the discussion at the end of
#902449, for the reason why we don't want to apply that patch.

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#932643: [pkg-cryptsetup-devel] Bug#932643: cryptsetup upgrade causes cryptsetup-initramfs autoremoval and boot failure

[Guilhem Moulin]

On Mon, 22 Jul 2019 at 11:37:24 +1200, Ben Caradoc-Davies wrote:
> cryptsetup 2:2.1.0-6 has no dependency on cryptsetup-initramfs so the
> latter will be autoremoved if only cryptsetup was marked manual by the
> installer.

Ooops.  We don't want ‘cryptsetup’ to hard-depend on ‘cryptsetup-initramfs’,
as that would make the 2:2.0.3-1 package split moot.  I just uploaded
2:2.1.0-7 where ‘cryptsetup’ *Recommends* ‘cryptsetup-initramfs’, which
is enough to convince APT not to auto-remove the latter upon `apt
upgrade --autoremove`.

In addition, I added a NEWS entry (forgot to do it for 2:2.1.0-6…) and
the prerm script now triggers a loud warning upon package removal if
there are mapped crypt devices.

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#932625: [pkg-cryptsetup-devel] Bug#932625: Bug#932625: cryptsetup: removing transitional cryptsetup-run produces scary debconf question

[Guilhem Moulin]

On Sun, 21 Jul 2019 at 21:57:09 -0300, Guilhem Moulin wrote:
> cryptsetup <2:2.0.3-1's (≤Stretch) functionalities have been subsumed
> by the combination of cryptsetup-run and cryptsetup-initramfs between
> 2:2.0.3-1 and 2:2.0.3-5 (Buster); and the combination of
> cryptsetup-run and cryptsetup-initramfs since 2:2.0.3-6 (≥Bullseye).

2:2.0.3-5 and 2:2.0.3-6 should respectively be 2:2.1.0-5 and 2:2.1.0-6.

> I'm about to upload 2:2.0.3-6 where:

And I meant 2:2.1.0-7 there…

-- 
Guilhem.


signature.asc
Description: PGP signature

Bug#932625: [pkg-cryptsetup-devel] Bug#932625: cryptsetup: removing transitional cryptsetup-run produces scary debconf question

[Guilhem Moulin]

Control: tag -1 pending

Hi,

On Sun, 21 Jul 2019 at 12:58:28 +0100, Simon McVittie wrote:
> My understanding is that it's fine for me to remove cryptsetup-run, because
> its functionality has been subsumed by the combination of cryptsetup and
> cryptsetup-initramfs?

Yup it's safe to remove 'cryptsetup-run'.  cryptsetup <2:2.0.3-1's
(≤Stretch) functionalities have been subsumed by the combination of
cryptsetup-run and cryptsetup-initramfs between 2:2.0.3-1 and 2:2.0.3-5
(Buster); and the combination of cryptsetup-run and cryptsetup-initramfs
since 2:2.0.3-6 (≥Bullseye).

> I suspect that swapping the transitional/non-transitional status of two
> pre-existing packages is only going to cause confusion on upgrade from
> buster to bullseye. It might be more robust to leave cryptsetup-run as
> the real package, drop the transitional cryptsetup, and choose one:
> either keep that situation indefinitely, or reinstate cryptsetup
> (with a transition from cryptsetup-run) in bullseye+1 or later.

I'm about to upload 2:2.0.3-6 where:

  * cryptsetup-run is a transitional depending on cryptsetup (like for
2:2.0.3-5); and
  * cryptsetup is a “real” (non-dummy) package recommending
cryptsetup-run.

That seems to be enough to avoid triggering cryptsetup-run.prerm with
action “remove” upon `apt upgrade --autoremove`, and that also on
systems where APT::Install-Recommends is set to ‘no’.

The package can be safely removed later, for instance with `deborphan
--guess-dummy`.  Once Bullseye is released we'll be able to remove the
‘Recommends: cryptsetup-run’, or even cryptsetup-run altogether.

Cheers,
-- 
Guilhem.


signature.asc
Description: PGP signature