Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Thu, Aug 8, 2013 at 10:21 PM, Wouter Verhelst wou...@debian.org wrote: On 05-08-13 02:16, Ben Hutchings wrote: On Sun, 2013-08-04 at 16:45 +0200, Wouter Verhelst wrote: On 03-08-13 13:45, Ondřej Surý wrote: I think it's useless to upgrade to SHA512 (or SHA-3), It's never useless to upgrade to a stronger hash. The cost might outweight the benefit, yes. But that's a different matter. What makes you think these are stronger? Simple mathematics. To me, a strong hash is a hash for which collisions are unlikely. A SHA512 hash is longer than a SHA1 hash. Therefore it has more bits. Therefore it has more possible values, which decreases the likelihood that two collections of bits will produce the same hash value by accident. This is a very dangerous fallacy. More bits != stronger. It's the algorithm properties that makes the hash stronger, not the number of the bits in the resulting hash. O. -- Ondřej Surý ond...@sury.org Have you tried Knot DNS – https://www.knot-dns.cz/ – a high-performance authoritative-only DNS server
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On 05-08-13 02:16, Ben Hutchings wrote: On Sun, 2013-08-04 at 16:45 +0200, Wouter Verhelst wrote: On 03-08-13 13:45, Ondřej Surý wrote: I think it's useless to upgrade to SHA512 (or SHA-3), It's never useless to upgrade to a stronger hash. The cost might outweight the benefit, yes. But that's a different matter. What makes you think these are stronger? Simple mathematics. To me, a strong hash is a hash for which collisions are unlikely. A SHA512 hash is longer than a SHA1 hash. Therefore it has more bits. Therefore it has more possible values, which decreases the likelihood that two collections of bits will produce the same hash value by accident. In addition, there are some concerns today about the strength of SHA1. It's not yet broken, but it's not right to think of it as fully safe anymore, either. Hashes don't get stronger over time; they get weaker. Of course, the very fact that SHA512 produces a longer hash does mean that there is a cost involved; and as said, that cost might outweigh the benefits. But that doesn't make it useless. -- This end should point toward the ground if you want to go to space. If it starts pointing toward space you are having a bad problem and you will not go to space today. -- http://xkcd.com/1133/ signature.asc Description: OpenPGP digital signature
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Thu, 2013-08-08 at 22:21 +0200, Wouter Verhelst wrote: On 05-08-13 02:16, Ben Hutchings wrote: On Sun, 2013-08-04 at 16:45 +0200, Wouter Verhelst wrote: On 03-08-13 13:45, Ondřej Surý wrote: I think it's useless to upgrade to SHA512 (or SHA-3), It's never useless to upgrade to a stronger hash. The cost might outweight the benefit, yes. But that's a different matter. What makes you think these are stronger? Simple mathematics. To me, a strong hash is a hash for which collisions are unlikely. [...] There is a big difference between *likelihood* of a random collision, and *difficulty* of deliberately constructing a collision. The latter case is not simple mathematics. Still, if I understand correctly, current attacks on SHA-256 and SHA-512 only improve by a few orders of magnitude over a brute force search, which does make SHA-512 much stronger. If I understand correctly, SHA-3 is a very different algorithm, but not necessarily stronger. It's probably worth designing into cryptographic hardware for the next few decades, but there's no need to start using it. I think SHA-2 (with any of the specified hash lengths) is good enough for now - it's just not going to be the weak link in authenticating Debian packages. Ben. -- Ben Hutchings The two most common things in the universe are hydrogen and stupidity. signature.asc Description: This is a digitally signed message part
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
Wouter Verhelst wou...@debian.org writes: Simple mathematics. To me, a strong hash is a hash for which collisions are unlikely. A SHA512 hash is longer than a SHA1 hash. Therefore it has more bits. Therefore it has more possible values, which decreases the likelihood that two collections of bits will produce the same hash value by accident. SHA-1 is already sufficiently unlikely that, barring a break in the underlying mathematics, it's not clear that you're gaining anything. Increasing the number of multiples of the age of the universe that it takes to brute force something doesn't make any actual, practical difference. In both cases, the primary concern is around breaks in the underlying mathematics, rather than in comparative brute force. I find it very hard to get excited about simple counts of the number of bits in the hash when the important factor for whether it's a secure hash is basically independent of length. The length is adequate for even theoretical computation models that use every atom in the solar system. In addition, there are some concerns today about the strength of SHA1. It's not yet broken, but it's not right to think of it as fully safe anymore, either. Hashes don't get stronger over time; they get weaker. This is the part that's more interesting. However, SHA-256 and SHA-512 are the same algorithm, and therefore are probably subject to the same attacks. So adding SHA-512 when we already have SHA-256 seems rather pointless. Adding SHA-3, which is a different algorithm and therefore might resist mathematical attacks that break SHA-2, is much more interesting. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87r4e3zyps@windlord.stanford.edu
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
David Kalnischkies wrote: On Fri, Aug 2, 2013 at 2:52 PM, Paul Wise p...@debian.org wrote: If so, here is the list of software that probably needs updating: dak apt/apt-ftparchive reprepro launchpad dpkg-dev devscripts derivatives census (c)debootstrap Also, apt-get is forcing MD5 in --print-uris by default because not doing it used to break all kinds of scripts. I think jigdo was one of them, no idea if that is really the case and/or if this changed by now. (not saying they shouldn't be fixed, just that the list is probably longer) jigdo and debian-cd both use MD5 for tracking and indexing files - debian-cd uses them to assist in generating jigdo files and also as a verification of archive contents as images are built. There should be no security implications in either case as more/stronger checksums are used for verifying the complete images. Changing jigdo to use a different checksum would not be impossible, but very involved and I'm not really convinced it would be worth it. -- Steve McIntyre, Cambridge, UK.st...@einval.com Support the Campaign for Audiovisual Free Expression: http://www.eff.org/cafe/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/e1v6yxw-0005ru...@mail.einval.com
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
Ondřej Surý writes (Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?): SHA512 doesn't bring any advantage over SHA256. AIUI SHA-512 is faster than SHA-256 on many processors, and not usually slower on the others. If the hashes are too long, they can be truncated. Ian. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20991.39828.481089.77...@chiark.greenend.org.uk
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Mon, Aug 05, 2013 at 01:33:24PM +0100, Ian Jackson wrote: AIUI SHA-512 is faster than SHA-256 on many processors, and not usually slower on the others. If the hashes are too long, they can be truncated. Not that, I think it matters, but this got me interested. It appears that in practice this depends entirely on the word size. So SHA-256 is faster on 32bit architectures and SHA-512 is faster on 64bit architectures. The other aspect is that a block update of SHA-256 uses 64 rounds for a 64 byte block. Whereas SHA-512 uses 80 rounds for a 128 byte block update. So SHA-512 lowers the rounds/byte ratio. Now what can we do with this knowledge? Probably negligible. Helmut -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130805162104.ga32...@alf.mars
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On 03-08-13 13:45, Ondřej Surý wrote: I think it's useless to upgrade to SHA512 (or SHA-3), It's never useless to upgrade to a stronger hash. The cost might outweight the benefit, yes. But that's a different matter. -- This end should point toward the ground if you want to go to space. If it starts pointing toward space you are having a bad problem and you will not go to space today. -- http://xkcd.com/1133/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51fe6907.7020...@debian.org
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Sun, 2013-08-04 at 16:45 +0200, Wouter Verhelst wrote: On 03-08-13 13:45, Ondřej Surý wrote: I think it's useless to upgrade to SHA512 (or SHA-3), It's never useless to upgrade to a stronger hash. The cost might outweight the benefit, yes. But that's a different matter. What makes you think these are stronger? Ben. -- Ben Hutchings This sentence contradicts itself - no actually it doesn't. signature.asc Description: This is a digitally signed message part
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
* Paul Wise p...@debian.org [130802 15:54]: In any case, removing md5 support seems like a bad idea to me right now, as older software might not have been adapted to check the other hashes, or would imply breaking the current .dsc and ,changes formats, as the Files field uses md5. We've had SHA1 since before snapshot.d.o data started (2005), I would guess any relevant software would have been updated in the last 8 years. In 2008 ubuntu had Sha256Sums wrong which showed that back then almost not software even bothered to check those fields: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/243630 non-md5sum hashses in Sources generated by DAK were incomplete until the generation code moved away from apt-ftparchive (early 2011 I think), thus only the Files: part with md5sums was the only reliable way to get the list of all files belonging to it. Support for non-md5sum hashes was added to dpkg-scansources/apt-ftparchive with apt (0.7.25.3) released to unstable 2010-02-01, first released with squeeze. So it is not some 8 years. It is more since squeeze that Debian and some of the common tools even produce complete non-md5sum hashes in Sources indices. reprepro for example only tries to support source indices without Files (i.e. md5sum hashes) since 4.12.0 (i.e. since wheezy). Bernhard R. Link -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130803075234.ga3...@client.brlink.eu
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Fri, 2013-08-02 at 15:29 +0200, Guillem Jover wrote: I was wondering if it is time to drop or deprecate MD5 from the apt metadata and replace it with SHA512 and or SHA-3. Thoughts? Adding stronger hashes support seems in general like a good idea, but I've never quite understood the urge to remove weaker ones in case these get accumulated instead of replaced, as more hashes should also in general imply a harder time coming up with data that will produce all the same hashes. You don't need to match all of them though, just the strongest hash that your target is actually checking. So if we drop md5 we will flush out all those utilities which rely only on md5 which will, eventually, lead to an increase in the strongest hash which targets are checking and prevent attackers from only supplying md5. In any case, removing md5 support seems like a bad idea to me right now, as older software might not have been adapted to check the other hashes, or would imply breaking the current .dsc and ,changes formats, as the Files field uses md5. Right, but perhaps dropping md5 should become a longer term goal? Did debian-devel have not this same conversation not so long ago? I'm getting that deja vu feeling... Ian. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1375525839.15681.63.ca...@dagon.hellion.org.uk
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Sat, Aug 3, 2013 at 12:30 PM, Ian Campbell wrote: Did debian-devel have not this same conversation not so long ago? I'm getting that deja vu feeling... Yes: http://lists.debian.org/1349911198.3341.117.ca...@fermat.scientia.net I probably should have searched the archives before posting, sorry. -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAKTje6EOEEL3KOajiUeUOGCL+bq0gOYcp7uRM8OPt=szjqk...@mail.gmail.com
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Fri, Aug 2, 2013 at 8:57 PM, David Kalnischkies kalnischk...@gmail.comwrote: On Fri, Aug 2, 2013 at 6:33 PM, Ondřej Surý ond...@sury.org wrote: On Fri, Aug 2, 2013 at 2:52 PM, Paul Wise p...@debian.org wrote: So, yeah let's drop MD5, but don't introduce neither SHA512 nor SHA-3 unless there's a cryptographical need (there isn't at the moment). Actually, it might be less controversial to drop SHA1[0] as the MD5 has fieldnames (as Guillem already mentioned) which are probably assumed to be present. I have not check(-ETIME) that for APT now, but somehow I would be surprised if it wouldn't dislike (some) missing MD5 sections even if it isn't using the sections for providing MD5, but because they have a wonderfully stable name like Files. Its not like we are anywhere near to a cryptographical need to drop MD5 (as you have to do (at least) two pre-image attacks in a row with the same file (aka compressed and uncompressed) – and as a bonus, the filesize has to match as well – not to mention that the file has to make sense…) and at the time we do SHA1 is probably not an interesting candidate. [IANACryptoguy] As far as I understand the MD5 attacks the length doesn't matter. You just need to pick the package big enough to hold your evil content and the filling which you use to compute the same MD5 (e.g. collision vulnerability). I think that the lengths of the files do not add enough bits. As for compressed/uncompressed - again I am unsure if this adds enough bits to circumvent the attacks on MD5. In my simplistic view - if you can find a collision in digital certificate then I am quite sure you can find a collision in debian package. I would not rely on MD5 with anything, so the dropping of MD5 for good (together with SHA-1) might be a good release goal. O. -- Ondřej Surý ond...@sury.org
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Sat, Aug 3, 2013 at 1:34 PM, Paul Wise p...@debian.org wrote: On Sat, Aug 3, 2013 at 12:30 PM, Ian Campbell wrote: Did debian-devel have not this same conversation not so long ago? I'm getting that deja vu feeling... Yes: http://lists.debian.org/1349911198.3341.117.ca...@fermat.scientia.net I probably should have searched the archives before posting, sorry. JFTR (from re-reading the dejavu :) I think it's useless to upgrade to SHA512 (or SHA-3), but at the same time I think we should drop MD5/SHA-1 in .changes/.dsc files (and Release.gpg). Using MD5 for debsums is just fine - the algorithm there needs different properties and any good checksum algorithm would do. (Even CRC-32 or Alder-32 would be fine, I guess...) O. -- Ondřej Surý ond...@sury.org
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Sat, 03 Aug 2013, Ondřej Surý wrote: [IANACryptoguy] As far as I understand the MD5 attacks the length doesn't matter. You just need to pick the package big enough to hold your evil content and the filling which you use to compute the same MD5 (e.g. collision vulnerability). I think that the lengths of the files do not add enough bits. For length to make a sucessfull collision attack considerably harder, your signature must include the length, i.e. it should be something like hash, length, not just the hash. I.e. you need to know the length of the original message/data somehow, not just its hash. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130803173235.gc10...@khazad-dum.debian.net
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
Hi! On Fri, 2013-08-02 at 14:52:33 +0200, Paul Wise wrote: I noted[1] that some derivatives have introduced SHA512 into their Release files (and probably Packages/etc). This will increase those files (Packages, Sources, etc) by quite a bit, at least 128 bytes per entry. Is that something we want, and is it really worth it? I was wondering if it is time to drop or deprecate MD5 from the apt metadata and replace it with SHA512 and or SHA-3. Thoughts? Adding stronger hashes support seems in general like a good idea, but I've never quite understood the urge to remove weaker ones in case these get accumulated instead of replaced, as more hashes should also in general imply a harder time coming up with data that will produce all the same hashes. In any case, removing md5 support seems like a bad idea to me right now, as older software might not have been adapted to check the other hashes, or would imply breaking the current .dsc and ,changes formats, as the Files field uses md5. It might be good to create a similar wiki page (to DebSupport) with the repository format support, so that we can get a better idea of the current status of the software around. If so, here is the list of software that probably needs updating: dpkg-dev I've got a local patch to add sha512 support to dpkg-dev, which I could commit for 1.17.x, if there's no opposition to this proposal. Thanks, Guillem -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130802132914.ga7...@gaara.hadrons.org
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Fri, Aug 2, 2013 at 3:29 PM, Guillem Jover wrote: Adding stronger hashes support seems in general like a good idea, but I've never quite understood the urge to remove weaker ones in case these get accumulated instead of replaced, as more hashes should also in general imply a harder time coming up with data that will produce all the same hashes. The only argument to remove them would be that they take up space in the apt metadata. In any case, removing md5 support seems like a bad idea to me right now, as older software might not have been adapted to check the other hashes, or would imply breaking the current .dsc and ,changes formats, as the Files field uses md5. We've had SHA1 since before snapshot.d.o data started (2005), I would guess any relevant software would have been updated in the last 8 years. http://snapshot.debian.org/archive/debian/20050312T00Z/dists/sid/Release It might be good to create a similar wiki page (to DebSupport) with the repository format support, so that we can get a better idea of the current status of the software around. Agreed, created one here, minimal content though: https://wiki.debian.org/RepositorySupport -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caktje6fr1ikdx3ctueiduee1v1jogzsbha-00vdqzzw_zs1...@mail.gmail.com
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Fri, Aug 2, 2013 at 2:52 PM, Paul Wise p...@debian.org wrote: If so, here is the list of software that probably needs updating: dak apt/apt-ftparchive reprepro launchpad dpkg-dev devscripts derivatives census (c)debootstrap Also, apt-get is forcing MD5 in --print-uris by default because not doing it used to break all kinds of scripts. I think jigdo was one of them, no idea if that is really the case and/or if this changed by now. (not saying they shouldn't be fixed, just that the list is probably longer) Side note; is SHA512 accepted/checked by apt in Release files yet? If so it would be great if the spec at [2] could be updated for that. Yes, APT is supporting SHA512 in in/output, but more as a by-product of the SHA2 group as a whole than a specific feature. This, and a bit that APT is just one implementation of this spec is the reason that it isn't mentioned in the wiki. Best regards David Kalnischkies -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caaz6_fbswkxvu5ugcw8qjnerfxfqp8azcfxj5otecf1zsnf...@mail.gmail.com
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Fri, Aug 2, 2013 at 2:52 PM, Paul Wise p...@debian.org wrote: I noted[1] that some derivatives have introduced SHA512 into their Release files (and probably Packages/etc). I was wondering if it is time to drop or deprecate MD5 from the apt metadata and replace it with SHA512 and or SHA-3. Thoughts? SHA512 doesn't bring any advantage over SHA256. SHA-3 hasn't been standardized yet by NIST as Secure Hash Standard and doesn't bring any advantages over SHA-2 (yet). So, yeah let's drop MD5, but don't introduce neither SHA512 nor SHA-3 unless there's a cryptographical need (there isn't at the moment). O. -- Ondřej Surý ond...@sury.org
Re: new hashes (SHA512, SHA3) in apt metadata and .changes files?
On Fri, Aug 2, 2013 at 6:33 PM, Ondřej Surý ond...@sury.org wrote: On Fri, Aug 2, 2013 at 2:52 PM, Paul Wise p...@debian.org wrote: So, yeah let's drop MD5, but don't introduce neither SHA512 nor SHA-3 unless there's a cryptographical need (there isn't at the moment). Actually, it might be less controversial to drop SHA1[0] as the MD5 has fieldnames (as Guillem already mentioned) which are probably assumed to be present. I have not check(-ETIME) that for APT now, but somehow I would be surprised if it wouldn't dislike (some) missing MD5 sections even if it isn't using the sections for providing MD5, but because they have a wonderfully stable name like Files. Its not like we are anywhere near to a cryptographical need to drop MD5 (as you have to do (at least) two pre-image attacks in a row with the same file (aka compressed and uncompressed) – and as a bonus, the filesize has to match as well – not to mention that the file has to make sense…) and at the time we do SHA1 is probably not an interesting candidate. Best regards David Kalnischkies [0] expect in pdiffs as that is the only supported in there so far -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAAZ6_fBiOFv-S�tVvZ=Y+UJbNBCcd64f�vp7ybnkvos...@mail.gmail.com
