Re: signify and ftp.eu.openbsd.org
2018-04-11 22:44 GMT+02:00 Peter J. Philipp : > On Wed, Apr 11, 2018 at 10:20:37PM +0200, Paul de Weerd wrote: > > ftp doesn't do this itself, but the error detection in tcp and ssl > > (ok, so that's linked into the ftp binary) do. > > > > The file is unlikely to have been changed in flight. > > OK, odd. What I find odd is that the SHA256.sig file I got (which was > downloaded after the .iso) differs from the very first time I made contact > with ftp.eu > today with the SHA256.sig that I downloaded to verify. Could this be an > atomicity thing with the FTP mirror? > The mirror tries to download and replace files dir-by-dir, but not edit files in-place, so its not impossible to hit the window when one of the files but not the other is replaced even if its normally a very small chance. Each snapshot is something like 100G, so there just is no way (for me) to atomically replace the lot so everything in between is a tradeoff between disk-usage, no instant writeable snaps and simplicity. -- May the most significant bit of your life be positive.
Re: signify and ftp.eu.openbsd.org
On Wed, Apr 11, 2018 at 10:44:28PM +0200, Peter J. Philipp wrote: | On Wed, Apr 11, 2018 at 10:20:37PM +0200, Paul de Weerd wrote: | > ftp doesn't do this itself, but the error detection in tcp and ssl | > (ok, so that's linked into the ftp binary) do. | > | > The file is unlikely to have been changed in flight. | | OK, odd. What I find odd is that the SHA256.sig file I got (which | was downloaded after the .iso) differs from the very first time I | made contact with ftp.eu today with the SHA256.sig that I downloaded | to verify. Could this be an atomicity thing with the FTP mirror? So you start your download of the ISO while the mirror is in the process of updating from upstream. The download takes some time, maybe five minutes (which would be ~1MB/s). In the mean time the mirror finishes its update to the latest snap (probably using rsync's --delay-updates flag), but it keeps sending you the file that was just deleted (your download is "atomic"). Then you download the (new) SHA256.sig. Then the ISO will be from an older snap and it will fail verification. Snapshots are produced so often (since I started keeping a backlog of old kernels back in 2011, I've seen 18 days with 4 amd64 snaps *from the same day*, 170 days with 3 amd64 snaps... I update 4 times a day) that this is actually not an unlikely scenario. I think this is what happened to you. The only way I can think of to guard against this is by downloading the .sig file twice, both before and after downloading other files. If they're different, you hit this corner case. Sorry, no big conspiracy where 3-letter agencies are trying to sabotage the installs of our favorite OS ;-) Your checking of the signify signature and the SHA256 checksum does help to detect such sabotage, but it's still not very likely. Cheers, Paul 'WEiRD' de Weerd -- >[<++>-]<+++.>+++[<-->-]<.>+++[<+ +++>-]<.>++[<>-]<+.--.[-] http://www.weirdnet.nl/
Re: signify and ftp.eu.openbsd.org
On Wed, Apr 11, 2018 at 10:20:37PM +0200, Paul de Weerd wrote: > ftp doesn't do this itself, but the error detection in tcp and ssl > (ok, so that's linked into the ftp binary) do. > > The file is unlikely to have been changed in flight. OK, odd. What I find odd is that the SHA256.sig file I got (which was downloaded after the .iso) differs from the very first time I made contact with ftp.eu today with the SHA256.sig that I downloaded to verify. Could this be an atomicity thing with the FTP mirror? > | 4. Is it possible that there was an attack on the OpenBSD network > infrastructure? > > Unlikely. Probably, the bytes you got match 1:1 with the file I got, > minus those last 4096 bytes. There's probably nothing all that > interesting in those last 4K of the ISO, so my guess would be that > everything would still be fine. You can verify that by checking the > contents of the mounted file system against the SHA256 file. > > | 5. Or mine? > > Also unlikely. Could the download have been interrupted at the last > moment? Not by me with any signals, I let the download finish. Theo does want me to investigate this box because it has problems waking/sleeping, and I was going to do that tonight but now it's too late, this signature anomaly wasted my time for tonight. It's entirely possible the drive is dying on it. After swapping SATA cables I think I'll put an SSD in it. Thanks Paul (and everyone else in this thread). -peter > Cheers, > > Paul
Re: signify and ftp.eu.openbsd.org
On Wed, Apr 11, 2018 at 09:52:41PM +0200, Peter J. Philipp wrote: | So I re-downloaded this file on another machine. It turned out to have your | checksum (sha256 -b) and upon downloading another SHA256.sig it signify'ed | correctly. But this leads me to ask questions: | | 1. doesn't the https mode ftp use some sort of authentication/integrity | checking while it's downloading? It's encrypted and anyone with a bit of | knowledge knows that if encryption is changed in-flight it will flip bits, | but with a MAC/HMAC/GMAC it should detect any anomalies. Does ftp not check | for this? ftp doesn't do this itself, but the error detection in tcp and ssl (ok, so that's linked into the ftp binary) do. The file is unlikely to have been changed in flight. | 2. I use quite recently a new unencrypted wifi network but I have IPSEC enabled | which uses aes-256 and auth hmac-sha2-256: | | SAD: | esp tunnel from fd43:5602:29bd:16:0:dead:beef:16 to fd43:5602:29bd:16:0:dead:beef:1 spi 0x6ab04db8 auth hmac-sha2-256 enc aes-256 | esp tunnel from fd43:5602:29bd:16:0:dead:beef:1 to fd43:5602:29bd:16:0:dead:beef:16 spi 0x780a14d0 auth hmac-sha2-256 enc aes-256 | | It is entirely possible that my IPSEC was off minutely and the gif tunnel inside it was not protected, but there was still the https... This is probably also not relevant to the result you got. | 3. The argument that this may have been modified by ftpmirror (or similar software) is good but I stat'ed the file and it's larger than what was downloaded by | us on the second try. A cmp -l shows a lot of differences... here is the stat: | theta$ touch test | theta$ date | Wed Apr 11 21:45:16 CEST 2018 | theta$ stat test | 1034 8616981 -rw-r--r-- 1 pjp pjp 0 0 "Apr 11 21:45:15 2018" "Apr 11 21:45:15 2018" "Apr 11 21:45:15 2018" 32768 0 0 test | theta$ stat install63.iso | 1034 8616961 -rw-r--r-- 1 pjp pjp 34424768 355905536 "Apr 11 20:59:23 2018" "Apr 11 19:33:47 2018" "Apr 11 19:33:47 2018" 32768 695360 0 install63.iso That's actually *smaller* than what I have here. The install63.iso I got is 355909632 bytes in size. Your file is missing 4096 bytes. | I did mount this iso with vnconfig is it possible that it was modified then? | (and grew?). No, vnd's don't grow in size during mount. Their atime may change, but I don't think much else will (not in the case of ISOs, which are read-only by nature). | 4. Is it possible that there was an attack on the OpenBSD network infrastructure? Unlikely. Probably, the bytes you got match 1:1 with the file I got, minus those last 4096 bytes. There's probably nothing all that interesting in those last 4K of the ISO, so my guess would be that everything would still be fine. You can verify that by checking the contents of the mounted file system against the SHA256 file. | 5. Or mine? Also unlikely. Could the download have been interrupted at the last moment? Cheers, Paul -- >[<++>-]<+++.>+++[<-->-]<.>+++[<+ +++>-]<.>++[<>-]<+.--.[-] http://www.weirdnet.nl/
Re: signify and ftp.eu.openbsd.org
On Wed, Apr 11, 2018 at 08:45:40PM +0200, Paul de Weerd wrote: > Hi Peter, Hello Paul, > I downloaded those exact two files from the same IP addresses and the > signature verified OK for me: > > [weerd@pom] $ ftp -4 https://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/amd64/i > > > Trying 193.156.26.18... > Requesting > https://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/amd64/install63.iso > 100% |**| 339 MB 01:41 > 355909632 bytes received in 101.67 seconds (3.34 MB/s) > [weerd@pom] $ nBSD/snapshots/amd64/SHA256.sig > < > Trying 193.156.26.18... > Requesting > https://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/amd64/SHA256.sig > 100% |**| 2152 00:00 > 2152 bytes received in 0.00 seconds (1.02 MB/s) > [weerd@pom] $ sd-63-base.pub -x SHA256.sig install63.iso > < > Signature Verified > install63.iso: OK > > (Note that I forced IPv4 as IPv6 was rather slow for me, but both > addresses match what your dig gave you). > > So, some options: > > Maybe you caught the mirror mid-update. Perhaps a bit fell over due > to cosmic radiation hitting your machine at the wrong moment, > affecting a bit of RAM. Maybe your storage medium is dying. > > Could be various reasons why it failed; can you try again and see if > it still fails? Try also on another machine, if you have one around. > I got WxAW3clMg3BLs/NBq58q9lMGlWFQLAOW5ToeltQlSyU= as a sha256 hash. > > Cheers, > > Paul 'WEiRD' de Weerd So I re-downloaded this file on another machine. It turned out to have your checksum (sha256 -b) and upon downloading another SHA256.sig it signify'ed correctly. But this leads me to ask questions: 1. doesn't the https mode ftp use some sort of authentication/integrity checking while it's downloading? It's encrypted and anyone with a bit of knowledge knows that if encryption is changed in-flight it will flip bits, but with a MAC/HMAC/GMAC it should detect any anomalies. Does ftp not check for this? 2. I use quite recently a new unencrypted wifi network but I have IPSEC enabled which uses aes-256 and auth hmac-sha2-256: SAD: esp tunnel from fd43:5602:29bd:16:0:dead:beef:16 to fd43:5602:29bd:16:0:dead:beef:1 spi 0x6ab04db8 auth hmac-sha2-256 enc aes-256 esp tunnel from fd43:5602:29bd:16:0:dead:beef:1 to fd43:5602:29bd:16:0:dead:beef:16 spi 0x780a14d0 auth hmac-sha2-256 enc aes-256 It is entirely possible that my IPSEC was off minutely and the gif tunnel inside it was not protected, but there was still the https... 3. The argument that this may have been modified by ftpmirror (or similar software) is good but I stat'ed the file and it's larger than what was downloaded by us on the second try. A cmp -l shows a lot of differences... here is the stat: theta$ touch test theta$ date Wed Apr 11 21:45:16 CEST 2018 theta$ stat test 1034 8616981 -rw-r--r-- 1 pjp pjp 0 0 "Apr 11 21:45:15 2018" "Apr 11 21:45:15 2018" "Apr 11 21:45:15 2018" 32768 0 0 test theta$ stat install63.iso 1034 8616961 -rw-r--r-- 1 pjp pjp 34424768 355905536 "Apr 11 20:59:23 2018" "Apr 11 19:33:47 2018" "Apr 11 19:33:47 2018" 32768 695360 0 install63.iso I did mount this iso with vnconfig is it possible that it was modified then? (and grew?). 4. Is it possible that there was an attack on the OpenBSD network infrastructure? 5. Or mine? Best regards, -peter
Re: signify and ftp.eu.openbsd.org
On 2018-04-11, Peter J. Philipp wrote: > Sorry to be writing this twice. Not my day. > > The complete path where I downloaded this from was > https://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/amd64/install63.iso > > including the SHA256.sig which I check against. > > Also: > > beta$ dig ftp.eu.openbsd.org +short > 193.156.26.18 > beta$ dig ftp.eu.openbsd.org +short > 2001:700:3:4017::100 > > that is in my DNS cache is this all correct? > > Best Regards, > > -peter > > > On 04/11/18 19:54, Peter J. Philipp wrote: >> Hi, >> >> I just downloaded this install63.iso from https://ftp.eu.openbsd.org: >> >> beta$ signify -C -p /etc/signify/openbsd-63-base.pub \ >>> -x SHA256.sig bsd >> Signature Verified >> bsd: OK >> beta$ signify -C -p /etc/signify/openbsd-63-base.pub \ >>> -x SHA256.sig install63.iso >> Signature Verified >> install63.iso: FAIL >> beta$ sha256 -C SHA256 install63.iso >> >> (SHA256) install63.iso: FAILED >> >> What's going on ? Why has the checksum failed? >> >> Regards, >> -peter > > A new snapshot may have been produced while the files were transferring, so SHA256.sig is from one snapshot and install63.iso from another. If you try another mirror at the same time and get the same error and file contents from both, it's quite likely that is what happened.
Re: signify and ftp.eu.openbsd.org
Hi Peter, On Wed, Apr 11, 2018 at 07:58:56PM +0200, Peter J. Philipp wrote: | Sorry to be writing this twice. Not my day. | | The complete path where I downloaded this from was | https://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/amd64/install63.iso | | including the SHA256.sig which I check against. | | Also: | | beta$ dig ftp.eu.openbsd.org +short | 193.156.26.18 | beta$ dig ftp.eu.openbsd.org +short | 2001:700:3:4017::100 I downloaded those exact two files from the same IP addresses and the signature verified OK for me: [weerd@pom] $ ftp -4 https://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/amd64/i > Trying 193.156.26.18... Requesting https://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/amd64/install63.iso 100% |**| 339 MB 01:41 355909632 bytes received in 101.67 seconds (3.34 MB/s) [weerd@pom] $ nBSD/snapshots/amd64/SHA256.sig < Trying 193.156.26.18... Requesting https://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/amd64/SHA256.sig 100% |**| 2152 00:00 2152 bytes received in 0.00 seconds (1.02 MB/s) [weerd@pom] $ sd-63-base.pub -x SHA256.sig install63.iso < Signature Verified install63.iso: OK (Note that I forced IPv4 as IPv6 was rather slow for me, but both addresses match what your dig gave you). So, some options: Maybe you caught the mirror mid-update. Perhaps a bit fell over due to cosmic radiation hitting your machine at the wrong moment, affecting a bit of RAM. Maybe your storage medium is dying. Could be various reasons why it failed; can you try again and see if it still fails? Try also on another machine, if you have one around. I got WxAW3clMg3BLs/NBq58q9lMGlWFQLAOW5ToeltQlSyU= as a sha256 hash. Cheers, Paul 'WEiRD' de Weerd | that is in my DNS cache is this all correct? | | Best Regards, | | -peter | | | On 04/11/18 19:54, Peter J. Philipp wrote: | > Hi, | > | > I just downloaded this install63.iso from https://ftp.eu.openbsd.org: | > | > beta$ signify -C -p /etc/signify/openbsd-63-base.pub \ | >> -x SHA256.sig bsd | > Signature Verified | > bsd: OK | > beta$ signify -C -p /etc/signify/openbsd-63-base.pub \ | >> -x SHA256.sig install63.iso | > Signature Verified | > install63.iso: FAIL | > beta$ sha256 -C SHA256 install63.iso | > (SHA256) install63.iso: FAILED | > | > What's going on ? Why has the checksum failed? | > | > Regards, | > -peter | -- >[<++>-]<+++.>+++[<-->-]<.>+++[<+ +++>-]<.>++[<>-]<+.--.[-] http://www.weirdnet.nl/
Re: signify and ftp.eu.openbsd.org
Sorry to be writing this twice. Not my day. The complete path where I downloaded this from was https://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/amd64/install63.iso including the SHA256.sig which I check against. Also: beta$ dig ftp.eu.openbsd.org +short 193.156.26.18 beta$ dig ftp.eu.openbsd.org +short 2001:700:3:4017::100 that is in my DNS cache is this all correct? Best Regards, -peter On 04/11/18 19:54, Peter J. Philipp wrote: > Hi, > > I just downloaded this install63.iso from https://ftp.eu.openbsd.org: > > beta$ signify -C -p /etc/signify/openbsd-63-base.pub \ >> -x SHA256.sig bsd > Signature Verified > bsd: OK > beta$ signify -C -p /etc/signify/openbsd-63-base.pub \ >> -x SHA256.sig install63.iso > Signature Verified > install63.iso: FAIL > beta$ sha256 -C SHA256 install63.iso > > (SHA256) install63.iso: FAILED > > What's going on ? Why has the checksum failed? > > Regards, > -peter
signify and ftp.eu.openbsd.org
Hi, I just downloaded this install63.iso from https://ftp.eu.openbsd.org: beta$ signify -C -p /etc/signify/openbsd-63-base.pub \ > -x SHA256.sig bsd Signature Verified bsd: OK beta$ signify -C -p /etc/signify/openbsd-63-base.pub \ > -x SHA256.sig install63.iso Signature Verified install63.iso: FAIL beta$ sha256 -C SHA256 install63.iso (SHA256) install63.iso: FAILED What's going on ? Why has the checksum failed? Regards, -peter