Re: [tor-dev] [tor-assistants] Python metrics-lib
[Moving this thread to tor-dev as suggested by Damian in separate mail.] Hi Damian, hi Beck, On 5/4/12 6:59 PM, Damian Johnson wrote: Hi Beck. I noticed that stem has not been updated during the last ten days Yup, I've been busy for a while with a new arm release [1] and a variety of other chores. Ravi is currently working on safe cookie support and I'll be getting back into stem development this weekend. it's a good time to start doing some substantial work Great! So for me, a reasonable next step is to build some subclass on top of the base Descriptor class. What do you think? Sounds good. What descriptor type would you like to tackle first? Karsten, do you have any suggestions? Ravi plans to do network status entries and microdescriptors so I'd suggest something besides those. There are quite a few options. How about extending stem to implement #2768? There's already Java code for verifying server descriptors and consensuses in metrics-tasks.git [0]. On stem's side this requires parsing the crypto parts of server descriptors, consensuses, and certificates. If the network statuses aren't there yet, starting with server descriptors should be fine. And once there's verification code for server descriptors and consensuses, there are more things to verify in Tor's various descriptors. Another option would be to write the parsing code for extra-info descriptors including all the stats. Once we have consensuses, server descriptors, and extra-info descriptors in stem, we can start writing a Python version of Onionoo [1], the service that provides the data for Atlas [2]. Let me know if you need more options or more input on these. Thanks, Karsten [0] https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-2768 [1] https://www.torproject.org/projects/onionoo.html.en [2] https://atlas.torproject.org/ Also, since you're interested in this area I'd suggest giving this a read... https://trac.torproject.org/projects/tor/ticket/5651 There's nothing that needs to be done for it on our end at the moment - just related to the stem/metrics-lib work. Cheers! -Damian [1] https://blog.torproject.org/blog/arm-release-145 On Fri, May 4, 2012 at 9:32 AM, Beck Chen csyb...@gmail.com wrote: Hi Damian, Greetings! I've read through the metrics-lib and stem.descriptor code and skimmed the directory server specs since you sent me the discussion thread between you and Karsten. I noticed that stem has not been updated during the last ten days, and since I've finished my semester and gone back to China, it's a good time to start doing some substantial work. In stem.descriptor, we already have counterparts for Descriptor.java, DescriptorReader.java, and ServerDescriptor.java now, and as you mentioned, they pretty much behave in the same way. So for me, a reasonable next step is to build some subclass on top of the base Descriptor class. What do you think? I believe you have discussed this with Karsten, so it will be helpful if you can forward me some of your discussions. Beck ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Can we stop sanitizing nicknames in bridge descriptors?
On Sat, May 05, 2012 at 01:47:45AM +0200, Ondrej Mikle wrote: One trick I had in mind was create secret hash function (take the following with a grain of salt, algebra is not my primary thing): - you keep generators g_i secret, which turns the problem from discrete-log into a problem of finding n-th root in finite field (n is the value of the digraph, trigraph or few characters, e.g. encoded value of 'ec2bridge', possibly blinded by another multiplication with secret constant c_i) - in general, computing n-th root is quite slow [1], but there are many special cases like square root (quadratic residue) - the above properties would make it slow for attacker to brute-force all possible values - i.e. attacker can't just compute the result values of such homomorphic hash, because he doesn't know the function parameters (e.g. without computing the generators), but everyone can use the homomorphic property for testing parts It sounds like you're talking about the homomorphic hashing paper you linked to in your last email. But there, the exponentiations are in Z_p, and taking n-th roots in Z_p is totally trivial. I seem to have lost the thread of why we're doing this. Is it just to count how many bridges are running our ec2 / rackspace / etc. bridge images? Can't we just report that out of band? Is the EC2 IP space not known? I must be missing something. - Ian ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Win32/MSVC: tinytest.c problems
On May 5, 2012, at 3:30 PM, Gisle Vanem wrote: There are 2 problems with the use of 'stricmp()' (line 295) in tinytest.c. 1. As it is now, I'm getting an undefined reference to 'stricmp' when linking tinytest.exe and using '-nodefault:oldnames' in the link-flags. Can we please remove this lib from the requirements? 2. 'stricmp()' is used nowhere else in Tor. But rather 'strcasecmp()'. Can we use that for consistency instead? Need to include or.h to include src/compat.h. Like the lines from src/compat.h says: /* Windows names string functions differently from most other platforms. */ #define strncasecmp _strnicmp #define strcasecmp _stricmp How about this little patch: Tinytest is supposed to be an independent library, so including a tor-specific header file doesn't work here. There are more MSVC-issues in other test/*.c files. I can get back to that. Please do, thanks. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Can we stop sanitizing nicknames in bridge descriptors?
On 05/05/2012 03:05 PM, Ian Goldberg wrote: On Sat, May 05, 2012 at 01:47:45AM +0200, Ondrej Mikle wrote: One trick I had in mind was create secret hash function (take the following with a grain of salt, algebra is not my primary thing): - you keep generators g_i secret, which turns the problem from discrete-log into a problem of finding n-th root in finite field (n is the value of the digraph, trigraph or few characters, e.g. encoded value of 'ec2bridge', possibly blinded by another multiplication with secret constant c_i) - in general, computing n-th root is quite slow [1], but there are many special cases like square root (quadratic residue) - the above properties would make it slow for attacker to brute-force all possible values - i.e. attacker can't just compute the result values of such homomorphic hash, because he doesn't know the function parameters (e.g. without computing the generators), but everyone can use the homomorphic property for testing parts It sounds like you're talking about the homomorphic hashing paper you linked to in your last email. But there, the exponentiations are in Z_p, and taking n-th roots in Z_p is totally trivial. I seem to have lost the thread of why we're doing this. Is it just to count how many bridges are running our ec2 / rackspace / etc. bridge images? Can't we just report that out of band? Is the EC2 IP space not known? I must be missing something. As I wrote before, the idea is over-engineered (I got a bit carried away; the EC2 IP space is known, it was an attempt to hide nicknames not on EC2 or similar cloud services potentially leaking information). Ondrej ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] tbb-build for Windows
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The last couple of days I've been working on recreating my build environment. I remember from the last time that it was a major PITA. (Especially with outdated docs, that aside) I decided to automate as much as possible and I came up with the following bootstrapper. It's a minimal bundle and contains 7zip and mingw-get needed to completely deploy everything. How it works is like follows: The user downloads and extracts the package to C:\MinGW (for example) Then double-clicks '.\install.cmd' and the process starts. It will first copy the files to the necessary places and then runs mingw-get to retrieve all packages. After that it downloads and unpacks MsysGit (Yes, it integrates git into mingw ;) It downloads and unpacks CMake and lastly it sets some default settings to integrate msysgit features into mingw, like git auto-completion. Note that the entire thing is self-contained. It installs nothing. For now, the user still needs to download the following manually, but they can use the default install locations: Qt, MSVS C++ Express, WinSDK 7.0, DirectX SDK I could only use sha1sum for msysgit because it's the only one that offers it. mingw-get might use hashes internally. This entire process differs none from manually downloading all packages in terms of integrity checking. I'm not sure where I was going with this lil' project, but personally, I'll keep using it. I just thought sharing it would be a good idea. It might be a step in the right direction for deterministic builds. The source can be found at: https://github.com/Shondoit/tbb-build The latest package can be downloaded from: https://github.com/downloads/Shondoit/tbb-build/tbb-build-1.2.1.zip Extract it to C:\MinGW and run install.cmd, after it's finished you can start mingw with start.cmd If you have a clean VM, with a default install of Windows 7, then please try it out and let me know. If you have a dirty Windows XP in the back of your closet, try it out as well. I'd love to hear how it breaks and then improving it. Note: you need to use my 'personal-build' branch from my personal tpo repo since the latest patches have not been merged yet. And you need to install Qt, MSVC, WinSDK and DX SDK, found here: http://releases.qt-project.org/qt4/source/qt-win-opensource-4.8.1-mingw.exe http://download.microsoft.com/download/A/E/7/AE743F1F-632B-4809-87A9-AA1BB3458E31/DXSDK_Jun10.exe http://download.microsoft.com/download/A/5/4/A54BADB6-9C3F-478D-8657-93B3FC9FE62D/vcsetup.exe http://download.microsoft.com/download/7/A/B/7ABD2203-C472-4036-8BA0-E505528CCCB7/winsdk_web.exe Qt needs to be installed /after/ tbb-build finished completely. Don't forget to set-up Git (git config [...]), clone torbrowser.git, add my personal repo, checkout 'personal-build' And then the following should Just Work*?*: $ make -f build-scripts/windows.mk build-all-binaries bundle Good luck, Thanks, And regards, Shondoit -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPpdGcAAoJEPkeD+x3AmlW2h0IAJBDV96W/ttwW/AlGJqNhbWB Qjqz23ToX9oIhm4S3XVJ4UmnrBSMbgG611+aFVl1BRieOTCaaCNFZADF9YUrunex dXPI0v+MhLu31Lc/yjIA8wbf8lhKQT35nLn4SQLG6lKV9tbxZYSIkNsYB4oVzLXr 9n512ILU48t8E4yZzobsqdJEvgW89V91tG6nPoW27oDJnG9vvU6c93yLVAZhCtgK TMRGa48Vvw+vyC+Gr5XMYEMD3D4LB7jpf3yIrd7USKGDrrgRAXn+av6PYLxmizhV GlCD+PYjcaCIx1CO+gK5UnuZfEQUFXuQAvtw+unXVVDq1oHb7lMYpkOpcUt1BPA= =tpTj -END PGP SIGNATURE- ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [tor-assistants] Python metrics-lib
Hi Beck, hi Karsten. First I'd like to make sure that I'm clear on what we're trying to do. The javadocs for VerifyDescriptors [1] says that it... Verify server descriptors using the contained signing key. Verify that 1) a contained fingerprint is actually a hash of the signing key and 2) a router signature was created using the signing key. Verify consensuses using the separate certs. Verify that 1) the fingerprint in a cert is actually a hash of the identity key, 2) a cert was signed using the identity key, 3) a consensus was signed using the signing key from the cert. Honestly I'm not yet sure what most of this means. The first #2 is simply checking that the descriptor content can be verified using the router-signature and signing-key, right? If so then this sounds like a good place to start since it's entirely self-contained within the descriptor and just involves implementation and testing of... https://gitweb.torproject.org/stem.git/blob/HEAD:/stem/descriptor/server_descriptor.py#l624 However, I need some suggestions for the choice of Python cryptography API, since I haven't used any before. Nor have I. At present stem does not have any dependencies outside of python's builtin functions. If we need PyCrypto and it's the best choice then so be it, but be sure to wrap the imports in a try/catch so we only raise an ImportError when executing the function that requires the PyCrypto library. Cheers! -Damian [1] https://gitweb.torproject.org/metrics-tasks.git/blob/HEAD:/task-2768/VerifyDescriptors.java ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [tor-assistants] Python metrics-lib
Hi Karsten Damian, Extending stem to implement #2768 seems a good option to start with; I looked at the Java code and felt doing it in stem should not be complicated. However, I need some suggestions for the choice of Python cryptography API, since I haven't used any before. A simple Google search says PyCrypto. I believe you have an answer, Damian. Cheers, Beck Quoting Karsten Loesing kars...@torproject.org: [Moving this thread to tor-dev as suggested by Damian in separate mail.] Hi Damian, hi Beck, There are quite a few options. How about extending stem to implement #2768? There's already Java code for verifying server descriptors and consensuses in metrics-tasks.git [0]. On stem's side this requires parsing the crypto parts of server descriptors, consensuses, and certificates. If the network statuses aren't there yet, starting with server descriptors should be fine. And once there's verification code for server descriptors and consensuses, there are more things to verify in Tor's various descriptors. Another option would be to write the parsing code for extra-info descriptors including all the stats. Once we have consensuses, server descriptors, and extra-info descriptors in stem, we can start writing a Python version of Onionoo [1], the service that provides the data for Atlas [2]. Let me know if you need more options or more input on these. Thanks, Karsten [0] https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-2768 [1] https://www.torproject.org/projects/onionoo.html.en [2] https://atlas.torproject.org/ ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
