Re: [tor-dev] Sanitizing bridge descriptors containing ed25519 fields
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/06/15 20:02, Karsten Loesing wrote: On 01/06/15 17:48, Nick Mathewson wrote: On Mon, Jun 1, 2015 at 3:27 AM, Karsten Loesing kars...@torproject.org wrote: Which one, the extended extra-info line or the additional fingerprint-ed25519 line? :) Not sure. I haven't actually added either yet; does the status quo not work? Well, it's the same use case. People would be able to grep extra-info descriptors for a given identity string, rather than having to use a specialized tool for that. It think it would be useful to have. (And it would allow me to ignore the identity-ed25519 crypto block entirely rather than having to parse the contained data structure and pick the bytes I want.) I think the master-key-ed25519 line is the likeliest way; I don't know if adding an extra arg to the first line is clever. I'm fine with either solution. If your preference is to add another master-key-ed25519 line (and if you agree that it would make sense to have the plain-text master key in extra-info descriptors at all), sounds good to me. Please find the following ticket for a first implementation of the new sanitizing code. It comes with two samples of sanitized descriptors containing new fields. https://trac.torproject.org/projects/tor/ticket/16359 All the best, Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJVe+TxAAoJEJD5dJfVqbCrkDMIAJ96cYxpSai+KHhWec4brGLj NPZ1w6yLyNiWtdVkeSTXWDUJ4QwzuiHz4qUGP600rkuDDwDFxKqq7KuR0a/e66k7 rT/lVzZJeXWOoE2rzolcFl8QXW0VhwBCwyyr/c+xHs8KLJDrjG/xCLL/nM8tZEra n0BnLNt2kCgnLl1aSnHL83y3BwpGe4BEnvwTLag1xj+opbuTOS9nx+cFaC0uWolq fT2Vi5PKq9a1lHy/UTRN2athhVKCLepBWS9fuFRLIY3T7w0GNSa6s4ky+BpeDoL2 kIPCdT2FPAms2aqy8Di9DS7oZqXsJsIWF6P+A/xcL5Mh+DO5gFOidvM7h2gHXEs= =5an+ -END PGP SIGNATURE- ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Sanitizing bridge descriptors containing ed25519 fields
On Mon, Jun 1, 2015 at 3:27 AM, Karsten Loesing kars...@torproject.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Nick, On 31/05/15 16:21, Nick Mathewson wrote: On Sat, May 30, 2015 at 5:16 PM, Karsten Loesing kars...@torproject.org wrote: So, I think a fingerprint-ed25519 line would be useful. It would make the bridge descriptor sanitizing process much easier. It would also facilitate debugging network problems, because people can just grep descriptors rather than using specialized tools that know how to decode the cert. And with microdescriptors in place it should be okay to add this line even if it's redundant information, because clients would never download it. Hm. Okay, that sounds solid enough. I'll try to hack it in tonight or Monday, and add it to prop220. Sounds good. Thanks! Added to code; documenting now. It's called master-key-ed25519. (No point in using a fingerprint, since the public key itself is only 32 bytes long.) How bad would it be to just SHA256 values for sanitizing bridge descriptors for the sake of simplicity? Probably not too bad; but use a differentiated hash wherever possible (like, for new stuff). extra-info Truie SHA1-of-RSA SHA256-of-ed25519 Possible downsides are that this additional value might break existing code and that it might be problematic to get rid of the SHA1-of-RSA part later. But the same issues would come up with the extra-info-digest line in server descriptors, and maybe there are good solutions. Otherwise, a separate fingerprint-ed25519 line might work here, too. Plausible. Which one, the extended extra-info line or the additional fingerprint-ed25519 line? :) Not sure. I haven't actually added either yet; does the status quo not work? I think the master-key-ed25519 line is the likeliest way; I don't know if adding an extra arg to the first line is clever. -- Nick ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Sanitizing bridge descriptors containing ed25519 fields
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/06/15 17:48, Nick Mathewson wrote: On Mon, Jun 1, 2015 at 3:27 AM, Karsten Loesing kars...@torproject.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Nick, On 31/05/15 16:21, Nick Mathewson wrote: On Sat, May 30, 2015 at 5:16 PM, Karsten Loesing kars...@torproject.org wrote: So, I think a fingerprint-ed25519 line would be useful. It would make the bridge descriptor sanitizing process much easier. It would also facilitate debugging network problems, because people can just grep descriptors rather than using specialized tools that know how to decode the cert. And with microdescriptors in place it should be okay to add this line even if it's redundant information, because clients would never download it. Hm. Okay, that sounds solid enough. I'll try to hack it in tonight or Monday, and add it to prop220. Sounds good. Thanks! Added to code; documenting now. It's called master-key-ed25519. (No point in using a fingerprint, since the public key itself is only 32 bytes long.) Great, thanks! How bad would it be to just SHA256 values for sanitizing bridge descriptors for the sake of simplicity? Probably not too bad; but use a differentiated hash wherever possible (like, for new stuff). Okay. extra-info Truie SHA1-of-RSA SHA256-of-ed25519 Possible downsides are that this additional value might break existing code and that it might be problematic to get rid of the SHA1-of-RSA part later. But the same issues would come up with the extra-info-digest line in server descriptors, and maybe there are good solutions. Otherwise, a separate fingerprint-ed25519 line might work here, too. Plausible. Which one, the extended extra-info line or the additional fingerprint-ed25519 line? :) Not sure. I haven't actually added either yet; does the status quo not work? Well, it's the same use case. People would be able to grep extra-info descriptors for a given identity string, rather than having to use a specialized tool for that. It think it would be useful to have. (And it would allow me to ignore the identity-ed25519 crypto block entirely rather than having to parse the contained data structure and pick the bytes I want.) I think the master-key-ed25519 line is the likeliest way; I don't know if adding an extra arg to the first line is clever. I'm fine with either solution. If your preference is to add another master-key-ed25519 line (and if you agree that it would make sense to have the plain-text master key in extra-info descriptors at all), sounds good to me. Thanks! All the best, Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJVbJ5CAAoJEJD5dJfVqbCrowUH/RTUoKDkCVVC6fT9DhF4CpSj 6RivAXo9wt3LZSNngWfXRMhv5y9cNnG0GuCOw7BPTo9/78nq8apJsFlVNE9ll2I7 0P0FQ97HV4IOAczMyGe8d27hazK+Nm/V//aPGuTzI7ZxpniAJNaOWzs/K8V0291k pDGzouX5gmtZkVlsGP+ngy5tA68xZIcPKB1I5IgJYQCYM1TeQ6Ra4ZoJTMTx26vP +UHiyN+diqOkO1+KkHwCwHXbdkSjpIjKNZFhMa9MvkKbD6SAPpTKCvni5b9bcDbx MSc2VsU6TTdRBZTaPhBfnO9CaSBrg9yzDIVUCriCY+xgXi+pwZSV70i6v5ukXjA= =BIV9 -END PGP SIGNATURE- ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Sanitizing bridge descriptors containing ed25519 fields
On Sat, May 30, 2015 at 5:16 PM, Karsten Loesing kars...@torproject.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 30/05/15 16:44, Nick Mathewson wrote: On Fri, May 29, 2015 at 4:23 PM, Karsten Loesing kars...@torproject.org wrote: router euler [scrubbed] 8000 0 0 identity-ed25519 -BEGIN ED25519 CERT- [scrubbed] -END ED25519 CERT- Base64-decode that block, throw it into SHA256(), base64-encode the result, format as block. But wouldn't the result be much shorter? There's no new fingerprint equivalent, like fingerprint-ed25519, is there? Oh dear. That blob is a certificate, not a key. It changes over time, because the key that it certifies varies over time. The format is documented in section 2.1 of proposal 220; the actual identity key is in an extension labeled with type 04 (see section 2.2.1). Maybe we should add a fingerprint-ed25519 line? It would be redundant, but maybe valuable. What do you think? I took a quick look at proposal 220, but not to the point that makes me entirely certain about this ed25519 stuff. If anything below remains unclear, that might be because I'm not clear about it myself. Please question everything I'm saying, even more than usual. Let's also include Damian on this thread. He usually has good ideas about descriptors, and he knows about sanitizing bridge descriptors. And let's add Isis who is working a lot with bridge descriptors, too. (There may be more people we should ask, but hey, it's tor-dev@.) So, I think a fingerprint-ed25519 line would be useful. It would make the bridge descriptor sanitizing process much easier. It would also facilitate debugging network problems, because people can just grep descriptors rather than using specialized tools that know how to decode the cert. And with microdescriptors in place it should be okay to add this line even if it's redundant information, because clients would never download it. Hm. Okay, that sounds solid enough. I'll try to hack it in tonight or Monday, and add it to prop220. If the server descriptor in #16227 were a bridge descriptor, I'd do the following steps for sanitizing it: - Remove identity-ed25519 line and subsequent crypto block. +1 - Keep yet-to-be-added fingerprint-ed25519 line, but SHA256 its value. - Keep extra-info digest line, but SHA1 and SHA256 its values. Suggestion: Don't use raw SHA256(x) but instead use SHA256(sanitize ID for bridge descriptor | x) or SHA256(sanitize extra-info digest for bridge descriptor | x). That way we don't need to worry about colliding with something else that decides to SHA256 these. - Remove onion-key line and subsequent crypto block. - Remove signing-key line and subsequent crypto block. - Remove onion-key-crosscert line and subsequent crypto block. - Remove ntor-onion-key-crosscert line and subsequent crypto block. - Keep ntor-onion-key line, mostly because we didn't remove it so far; unless it should be removed? - Remove router-sig-ed25519 line. - Remove router-signature line and subsequent crypto block. - Add router-digest line with SHA1 of SHA1 of descriptor content and SHA256 of SHA256 of descriptor content; the rationale is that we don't have to write entirely new digests into the network status in order to match r lines with descriptors. That all sounds fine. I also added the extra-info descriptor that corresponds to the server descriptor to #16227: https://trac.torproject.org/projects/tor/ticket/16227#comment:5 I wonder, what's the best way for including the ed25519 identity in the extra-info descriptor? How about extending the first line to: extra-info Truie SHA1-of-RSA SHA256-of-ed25519 Possible downsides are that this additional value might break existing code and that it might be problematic to get rid of the SHA1-of-RSA part later. But the same issues would come up with the extra-info-digest line in server descriptors, and maybe there are good solutions. Otherwise, a separate fingerprint-ed25519 line might work here, too. Plausible. In order to sanitize such an extra-info descriptor, I'd do these steps: - Keep extra-info line, but SHA1 (and possibly SHA256) its value(s). See above. - Or, keep yet-to-be-added fingerprint-ed25519 line, but SHA256 its value. See above. - Remove router-sig-ed25519 line. - Remove router-signature line and subsequent crypto block. - Add router-digest line with SHA1 of SHA1 of descriptor content and SHA256 of SHA256 of descriptor content; same rationale as above, but with the extra-info-digest line in server descriptors. What do you think? Sounds fine. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Sanitizing bridge descriptors containing ed25519 fields
On Fri, May 29, 2015 at 4:23 PM, Karsten Loesing kars...@torproject.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Forwarding from a private thread with Nick. - Forwarded Message Subject: Re: Whoops Date: Fri, 29 May 2015 21:20:57 +0200 From: Karsten Loesing kars...@torproject.org To: Nick Mathewson ni...@torproject.org Ugh, long mail ahead. This turns out to be more difficult than expected... Just like life itself! Never fear, we will solve all problems and build a better world. On 29/05/15 19:29, Nick Mathewson wrote: On Fri, May 29, 2015 at 11:04 AM, Karsten Loesing kars...@torproject.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sure! My main question is which of these new fields we'll have to sanitize in bridge descriptors. The current idea of sanitizing bridge identities is that Tonga would give out server descriptors if you give it a bridge identity. We want to avoid that, which is why we're SHA1()'ing fingerprints and removing cryptographic material. What about the new identity? Would we have to sanitize that in any way? And if so, would we want to SHA1() it, or is there a more ed25519y way to do this? I guess the better question might be: are there plans for Tonga to give out descriptors if you tell it an ed25519 identity? If not, do you see any potential trouble in leaving it unchanged in sanitized bridge descriptors? I would suggest that we sanitize all the crosscert stuff, and the ed25519 identity, and the ed25519 signing cert. Does this need to be done using some language I know? If so I'll be happy to hack it up for you if you point me to the current code that does it. Thanks for the offer, really, but if I can, I'd rather want to write this code myself once I know what it's supposed to do. The reason is that setting up this code and providing you with sample data might be more effort than writing it myself. Hope that's okay, too. Only if you're curious, here's the current code that sanitizes bridge descriptors: https://gitweb.torproject.org/metrics-db.git/tree/src/org/torproject/ernie/db/bridgedescs/SanitizedBridgesWriter.java But feel free to ignore that code, and let's talk conceptually or by example. Okay. (Sadly for me, it's Java. I haven't touched Java in about 13 years, and probably shouldn't be trusted with it.) (The authority might someday give out bridges based on this information. Who knows? Not me. Better to be safe than sorry IMO.) Okay. To sanitize an ed25519 identity, i'd suggest SHA256. Best avoid SHA1. Sure, that would work. By the way, here's how we're currently sanitizing bridge descriptors: https://collector.torproject.org/formats.html#bridge-descriptors Following those steps, I'd do the following things (quoting an actual bridge descriptor as input here; edit: scrubbed potentially sensitive fields, sorry for the linebreaks!): router euler [scrubbed] 8000 0 0 identity-ed25519 -BEGIN ED25519 CERT- [scrubbed] -END ED25519 CERT- Base64-decode that block, throw it into SHA256(), base64-encode the result, format as block. But wouldn't the result be much shorter? There's no new fingerprint equivalent, like fingerprint-ed25519, is there? Oh dear. That blob is a certificate, not a key. It changes over time, because the key that it certifies varies over time. The format is documented in section 2.1 of proposal 220; the actual identity key is in an extension labeled with type 04 (see section 2.2.1). Maybe we should add a fingerprint-ed25519 line? It would be redundant, but maybe valuable. What do you think? -- Nick ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Sanitizing bridge descriptors containing ed25519 fields
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Forwarding from a private thread with Nick. - Forwarded Message Subject: Re: Whoops Date: Fri, 29 May 2015 21:20:57 +0200 From: Karsten Loesing kars...@torproject.org To: Nick Mathewson ni...@torproject.org Ugh, long mail ahead. This turns out to be more difficult than expected... On 29/05/15 19:29, Nick Mathewson wrote: On Fri, May 29, 2015 at 11:04 AM, Karsten Loesing kars...@torproject.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sure! My main question is which of these new fields we'll have to sanitize in bridge descriptors. The current idea of sanitizing bridge identities is that Tonga would give out server descriptors if you give it a bridge identity. We want to avoid that, which is why we're SHA1()'ing fingerprints and removing cryptographic material. What about the new identity? Would we have to sanitize that in any way? And if so, would we want to SHA1() it, or is there a more ed25519y way to do this? I guess the better question might be: are there plans for Tonga to give out descriptors if you tell it an ed25519 identity? If not, do you see any potential trouble in leaving it unchanged in sanitized bridge descriptors? I would suggest that we sanitize all the crosscert stuff, and the ed25519 identity, and the ed25519 signing cert. Does this need to be done using some language I know? If so I'll be happy to hack it up for you if you point me to the current code that does it. Thanks for the offer, really, but if I can, I'd rather want to write this code myself once I know what it's supposed to do. The reason is that setting up this code and providing you with sample data might be more effort than writing it myself. Hope that's okay, too. Only if you're curious, here's the current code that sanitizes bridge descriptors: https://gitweb.torproject.org/metrics-db.git/tree/src/org/torproject/ernie/db/bridgedescs/SanitizedBridgesWriter.java But feel free to ignore that code, and let's talk conceptually or by example. (The authority might someday give out bridges based on this information. Who knows? Not me. Better to be safe than sorry IMO.) Okay. To sanitize an ed25519 identity, i'd suggest SHA256. Best avoid SHA1. Sure, that would work. By the way, here's how we're currently sanitizing bridge descriptors: https://collector.torproject.org/formats.html#bridge-descriptors Following those steps, I'd do the following things (quoting an actual bridge descriptor as input here; edit: scrubbed potentially sensitive fields, sorry for the linebreaks!): router euler [scrubbed] 8000 0 0 identity-ed25519 -BEGIN ED25519 CERT- [scrubbed] -END ED25519 CERT- Base64-decode that block, throw it into SHA256(), base64-encode the result, format as block. But wouldn't the result be much shorter? There's no new fingerprint equivalent, like fingerprint-ed25519, is there? Whatever we do here, we should make it simple enough for bridge operators to do, too, if they want to look up their bridge in Atlas or Globe. hex-encode(SHA1(hex-decode(fingerprint))) seemed okay. We should try to find something similar for ed25519 identities. (We could also have tor log its hashed ed25519 identity, but that only works for people looking at logs.) platform Tor 0.2.7.1-alpha-dev on Linux protocols Link 1 2 Circuit 1 published 2015-05-29 17:53:38 fingerprint [scrubbed] uptime 2 bandwidth 14971520 104857600 63488 extra-info-digest [scrubbed] [scrubbed] IIRC (and it's been a while...), I'd replace E07A... with its SHA1. So, I guess I'd replace YZpo... with its SHA256. Note that the extra-info descriptor would contain a router-digest line with the SHA1 of E07A... and be stored in a file using that identifier. In the distant future, we'd probably store extra-info descriptors under their SHA256. onion-key -BEGIN RSA PUBLIC KEY- [scrubbed] -END RSA PUBLIC KEY- signing-key -BEGIN RSA PUBLIC KEY- [scrubbed] -END RSA PUBLIC KEY- onion-key-crosscert -BEGIN CROSSCERT- [scrubbed] -END CROSSCERT- I'd remove the last six lines starting at onion-key-crosscert. (The onion-key and signing-key lines/blocks would also be removed in the current code.) ntor-onion-key-crosscert 0 -BEGIN ED25519 CERT- [scrubbed] -END ED25519 CERT- That block would go away, too. hidden-service-dir contact [scrubbed] ntor-onion-key [scrubbed] reject *:* router-sig-ed25519 [scrubbed] I guess that can go away, too. We'll probably want to put in a router-dig-ed25519 with the SHA256 of that value. router-signature -BEGIN SIGNATURE- [scrubbed] -END SIGNATURE- And here's the corresponding extra-info descriptor: extra-info euler [scrubbed] identity-ed25519 -BEGIN ED25519 CERT- [scrubbed] -END ED25519 CERT- Same as above, whatever that will be. published 2015-05-29 17:53:38
