Thanks Marco:
...and thanks for every detailed comment. I just noticed that there were a couple of things where I was not clear: Sorry about the lengthy post, but it's worth to seem or be redundant. I give proofs 1.- On your first quote about canonicalization, . I mentioned that not only MIME message headers require canonicalization, for the following reason: It is true that the MIC resulting from doing the "sha1sum" on a file with canonicalized MIME headers and canonicalized content produces a different MIC to that when you don't canonicalize content (of course when the content has end of lines to canonicalize). But my point IS NOT about about differences ammong MICs resulting from changing the original message!!!, but rather ABOUT THE DIFFERENCES between: THE MIC calculated with sha1sum on the original message USING EITHER modality on canonicalization [CANONICALIZING: (1) headers only, (2) headers and content, (3) nothing canonicalized] AND the messageDigest already stored in the signature of the message, this is the field inside the ASN1 structure of the signature labeled "messageDigest"; I found that commercial packages agree that THIS messageDigest is the VALID MIC to return on the MDN If you make your message and you sign it, you will find this. If all your message is canonicalized both in headers AND body, AND THEN SIGN IT, the messageDigest (I mean the one stored in the ASN1 structure of the resulting signature) will match the MIC you calculated on your message before signing it. If you only canonicalize headers but not body (and if the body has end of lines of course), the MIC on your message WILL NOT MATCH the messageDigest stored in the signature. 2.- From the above comes my second point: what I call THE TRICK Noticing that one can make match the MIC (the one you can calculate with a simple sha1sum over the message before signing it) with the messageDigest (the one stored in the signature of the signed message) by just canonicalizing the whole original message (not just headers), there comes THE FIRST footnote about the matching of those 2 values (MIC and messageDigest). Precisely because one can calculate many MICs, I point out which is the one that makes match those two values. Try a message with ONLY canonicalized MIME headers and dont't worry about the content, and the MIC and messageDigest WON'T match. Try canonicalizing both headers and content, this is THE WHOLE message, AND THE MIC and messageDigest WILL MATCH. This is not precisely what the RFC-4130 says, because 4130 says you should canonicalize MIME headers (regardless of what you do to the content), and doing just that will lead to that the MIC (the one you calculate with sha1sum) and messageDigest (inside the signature) will not match. Despite other mismatches explained below. What I mention above is already the first trick, because there is no warning about the matter The second trick comes in commercial packages, where YOU CAN NOT MAKE MATCH MIC on the original content with the messageDigest in the signature by just canonizalizing the original content. MAKING JUST THAT IS NOT ENOUGH, for that reason I say there is A TRICK to make those 2 numbers (MIC and messageDigest) MATCH. Too much wording, here comes the PROOF (the (N) and (RN) are the end of lines found and translated-to-be-displayed by small script "findeol.pl" The canonicaliza.pl only canonicalizes mime headers. #this is the original message with ONLY \n as end of line [EMAIL PROTECTED]:~/Documents/openssl> cat anyoriginalmsg.txt|~/bin/findeol.pl Content-Type: text/plain(N) (N) This is my text(N) written in THREE lines(N) THE-END (N) #this is the original message with ONLY \r\n as end of line AT MIME HEADERS [EMAIL PROTECTED]:~/Documents/openssl> cat anyoriginalmsg.txt|~/bin/canonicaliza.pl|~/bin/findeol.pl Content-Type: text/plain(RN) (RN) This is my text(N) written in THREE lines(N) THE-END (N) #this is the original message with \r\n as end of line everywhere [EMAIL PROTECTED]:~/Documents/openssl> cat anyoriginalmsgDOS.txt|~/bin/findeol.pl Content-Type: text/plain(RN) (RN) This is my text(RN) written in THREE lines(RN) THE-END (RN) #these are the respective MIC's [EMAIL PROTECTED]:~/Documents/openssl> cat anyoriginalmsg.txt|sha1sum db4286d4753be683b3bfb7897aa6959525f93353 - [EMAIL PROTECTED]:~/Documents/openssl> cat anyoriginalmsg.txt|~/bin/canonicaliza.pl|sha1sum 4602e1d628295e64cb66570a9558478bde0ff353 - [EMAIL PROTECTED]:~/Documents/openssl> cat anyoriginalmsgDOS.txt|sha1sum 07fe4193fac34fe84b48278c66c842f0ed0ee1d7 - [EMAIL PROTECTED]:~/Documents/openssl> Now, I signed every one of the above, then took the base64 encoded signature and piped it to openssl asn1parse and then, piped to grep to get the messageDigest portion: [EMAIL PROTECTED]:~/Documents/openssl> openssl asn1parse -in sign1.txt|grep -C2 -i messagedigest 2296:d=8 hl=2 l= 13 prim: UTCTIME :080807013825Z 2311:d=6 hl=2 l= 35 cons: SEQUENCE 2313:d=7 hl=2 l= 9 prim: OBJECT :messageDigest 2324:d=7 hl=2 l= 22 cons: SET 2326:d=8 hl=2 l= 20 prim: OCTET STRING [HEX DUMP]:07FE4193FAC34FE84B48278C66C842F0ED0EE1D7 [EMAIL PROTECTED]:~/Documents/openssl> openssl asn1parse -in sign2.txt|grep -C2 -i messagedigest 2296:d=8 hl=2 l= 13 prim: UTCTIME :080807013920Z 2311:d=6 hl=2 l= 35 cons: SEQUENCE 2313:d=7 hl=2 l= 9 prim: OBJECT :messageDigest 2324:d=7 hl=2 l= 22 cons: SET 2326:d=8 hl=2 l= 20 prim: OCTET STRING [HEX DUMP]:07FE4193FAC34FE84B48278C66C842F0ED0EE1D7 [EMAIL PROTECTED]:~/Documents/openssl> openssl asn1parse -in sign3.txt|grep -C2 -i messagedigest 2296:d=8 hl=2 l= 13 prim: UTCTIME :080807013957Z 2311:d=6 hl=2 l= 35 cons: SEQUENCE 2313:d=7 hl=2 l= 9 prim: OBJECT :messageDigest 2324:d=7 hl=2 l= 22 cons: SET 2326:d=8 hl=2 l= 20 prim: OCTET STRING [HEX DUMP]:07FE4193FAC34FE84B48278C66C842F0ED0EE1D7 [EMAIL PROTECTED]:~/Documents/openssl> Notice that ALL 3 signatures have the same MESSAGE DIGEST Notice that EVERY one of the MICs CALCULATED FIRST are different AMMONG THEMSELVES, this is what you said, and I agree, because data is different But also notice that ONLY ONE OF THE MICs MATCH THE messageDigest inside all signatures Which ONES MATCH? Not the one from the text not canonicalized Not the one from the text WITH ONLY MIME HEADERS canonicalized BUT THE ONE from text WITH ***ALL CANONICALIZED*** >From the above hence my points on (1) WHAT TO CANONICALIZE and ALSO, (2) to MATCH MIC and messageDigest in certain commercial packages, is not even enough to canonicalize ALL the original message. There is another TRICK which so far I haven't found. I get those messages every day, with signed messages, with messageDigest inside the signature, I take that one and return to sender in the MDN as the MIC and my business partner SAYS "OK". If I canonicalize original content; make sha1sum over that, and send that MIC... business partner says "error". If I canonicalize only headers of original message, business partner says "error". I have gone through another test about weird UTF-8 encoding but without success. They send me UTF-8 data (containing a,e,i,o,u "acute" and n with tilde in their UTF-8 encoding) and all that is further encoded again in UTF-8. I've done the double decoding, again the things of the canonicalization but still the MIC and messageDigest don't match. If they signed some original document and that produced the signature and corresponding messageDigest inside, what they are placing in the first multipart section where the original data is (RFC1767 or EDI object) is no longer the same they signed, it has been changed and even with the above shufflings and jugglings, the MIC and messageDigest DON'T match. These are just facts (not opinions) placed in words meaning "this happens" if only MIME headers are canonicalized: THE BUSINESS PARTNER'S "certified software" WILL REJECT your calculated MIC unless you watch the messageDigest inside the signature and make it MATCH with the MIC calculated. (more práctical if you forget about calculating MIC and simply fetch the messageDigest inside the signature and send it back in the MDN's MIC. Thanks for the reading and for the support -- View this message in context: http://www.nabble.com/RFC-4130-checksum-in-SHA1-tp18034577p18863502.html Sent from the OpenSSL - User mailing list archive at Nabble.com.
