[tor-dev] Performance testing using chutney (was Re: Env variables for chutney)
> On 5 Jul 2015, at 05:45 , Cory Pruce wrote: > > One more thing for right now: how should I do benchmarks with chutney. > Should I measure the averages of how long it takes to complete the make > test-network command? make test-network is dominated by the 25 second delay waiting for the Tor test network to bootstrap. So it's not going to help much. I'm working on a chutney branch to measure bandwidth on "chutney verify", but it doesn't have any command-line arguments yet (it's all constants in the code). I'll see if I can pull it into shape today. https://trac.torproject.org/projects/tor/ticket/14175 Even with these bandwidth measurement changes, there's something else to think about: chutney will measure the combined throughput of 4-5 tor instances, and 4n - 5n cpuworker threads, where n is the number of cores on your machine. But this isn't the performance you're interested in for multithreaded crypto changes - you want to know how a single instance + n cpuworker threads performs. (A chutney test network is far *more* parallel than a typical tor relay.) To get an accurate benchmark, you could run one tor instance per machine, or, at the very least, run the client on a slow machine, and everything else on a fast machine, so that the client's multithreaded crypto is the limiting factor. But this seems like a lot of work, and I'm not sure how much accuracy you'll gain. As a first step, you could minimise the number of tor instances, which might make multithreading improvements easier to measure. You'll find the basic-min network helpful for this: ./src/test/test-network.sh --flavour basic-min Then check if you're using ~100% of all cores when you push large amounts (100MB+) of data through the network using #14175 (when it's done!) If you're not using 100%, then you'll be able to see any multithreaded improvements when you run the test again. If you are seeing 100% usage already, get more cores or more machines, and re-run the tests. Let me know how you go with this. You could also modify tor to use single-hop connections, then measure single-hop bandwidth, by making a 1-hop connection and pushing data through it. There won't be as much client crypto as the 3 or 4-hop scenario; and you'll still have the client and destination on the one machine, unlike the single relay real-world scenario. But it could be closer to real-world multithreaded performance, as you'll only be measuring 2n threads. (Ideally, you want to measure n threads.) You must *never* use a tor binary built like this on the public tor network, as it has no anonymity. To make tor use 1-hop circuits for everything, change DEFAULT_ROUTE_LEN to 1 in or.h End dire warning about loss of anonymity. Of course, 1-hop circuits might hide some subtle multithreading bugs, as there's less crypto happening overall. So please test the correctness of your code with DEFAULT_ROUTE_LEN 3 as well. Give me 8 hours or so to work on #14175, I'll try and get it into a usable state. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com pgp ABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7 signature.asc Description: Message signed with OpenPGP using GPGMail ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Env variables for chutney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 One more thing for right now: how should I do benchmarks with chutney. Should I measure the averages of how long it takes to complete the make test-network command? - - Cory -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJVmDe/AAoJEB7DuCIJauCgpQQQALOaJ2uJsSWpRIKzQZ4E8bR0 XJV6cY5oUpypvGx0JpvPvfmbKVJey98WwLw1tbyMHBKiq+w+oTdkCyUNiHKBuTTd 694iM7GzPeJFgI4NGbVlPQ+fQmysB8ujK/D4pWiywBm1+Y81p6g4mdBrvvYSy/bF E9PEor3HqYgp24R8FNzXCz6J5/yLKFJV7MHCsKKiPM5xqemULR9rMQTyY71OjTTV OOcVcmatNSaqZvXYkPUvNrHqeLPQ4Ys/bSmqAUXii64kTqlOQpwbmlLk+6j6Ysbw x0W5Ke1BiFkoQQF61U7uUnpnKRwHbzzc4M69wnYm0OmlLD9iEnaQAj3fUCg1c+O+ nLwt5MIT7G/X0aeM/MXdxthmjoCoCYtVeNk9hSk3adSm1E7nUKjws2Uf6H0nsLmC HCAUk6tpAvI+Rl/IE+xwatTWr0TeisELtpcudu+R2qEIsgdi5D6xkoAQ3qTP+Zuy wNjdxNOKhB/KrxJLVeeCTtKrIawpOjKxc4jVV0yOt40jj+b952DKjLOvbzG09f7r 2rf0mw/btvcJOcCMbEggtyZ9PMBdKt9i3zE0Ihofz8o9ILI8YuiklyuxK+zejtFB EPDv64c7dPX0rE7y1jJ+tqfZZFjhV7FaRscNDmo/fw9T78IzDc7gCZY/lxpETjZr tHo7BBNDtFgIRncjenRY =zkhF -END PGP SIGNATURE- ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Env variables for chutney
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/04/2015 12:11 AM, teor wrote: > > The CHUTNEY_PATH variable needs to point to a directory containing a chutney executable. > > So you seem to have it right the first time. What do you get when you run: > ls -l ~/tor/chutney/ > > I would expect to see an executable script called "chutney" listed in that directory, along with the other chutney distribution files. The test-network.sh script is complaining that the "chutney" script is missing. > I got it :-) Thanks! -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJVmDV0AAoJEB7DuCIJauCgWKcP/RgIpxwwZ8bb4bp6zcC5Mg4t 489IYkSxnmUQb3+s1jlKPlY3Lb2f5jkvLshWcsKvUzCR7JVVYGw1ZJ+criPD+n0C /hEY5gRfiBectMY482BLETeoO+BTv52Wdxy/wWkMmzYzGeknsK2DmWZPEGVCwfcv NYcwkLb8IUFsHWeNfjacqzcXw2kDwrdzoBwpZzNnBLOh3ZdC2ZvtvwCt0GgT/V/2 IDzouczOiQ9IsWQIJQU2gWWH5+69V+RfuP49Y2LJV0k9Cp9X/9Effr390b6j5yy6 TlFJptvRQ5XMBeQ8ds1ivJXKFxm3u3BPfpL2GNWzTEb8XNhw9strlS0fXF/cYlxH lr0/9TDANros/zcxphi6p4fJWeMwVJDGmYUpHOB2lDCapR35hWbd3NnLaznFK5v3 ktHAaycDn0kOep9nEF9aYkKZDHmG1RxwZUqQzKl4tmftaShTBfboL65PiMtPY4C3 jINfm1jA7brgVMTTnCAFSBgmXWt64vRCDl0mb1AqEe8xJwcPdwYNSmSuMy8tLEtO CQF7/LumZeIqlU3CKZhTi/0FGlTxLrrAqeHb8HFoRvU4UkekWvqthpGOnD0TAGlo uW0FmmVB47KFI7Jcq89wkPY7Dbh65QQ2QZoZIHDFnZlDHzyL62SdcNhwWfvV9r6m EK2MM1QyPH15CMBfS6rh =Dm36 -END PGP SIGNATURE- ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Roster introduction
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 > Main things accomplished so far: * Setup the basic website at: > http://www.tor-roster.org/ Thanks for doing this! A few comments: - - the page seems to be AS name centric - first column of the table is AS name - only clickable column in the table is AS name - - the AS column suggests that all relays in a family run in one AS only - - the family page of a specific family shows the AS name as title, this is a bit confusing as one might think that the page shows all families/relays in the selected AS after clicking an as name on the start page Family identifier - - by looking at your page I assume your family identifier is the fingerprint of the oldest relay in the family, does that mean that an operator has to start over at 0 with collecting "points" once he kills his oldest relay? -BEGIN PGP SIGNATURE- iQIcBAEBCgAGBQJVl5wiAAoJEFv7XvVCELh0c1kP/1C5YbWsv8pEoHuTiISs8UzI ah7ZtlBO27YNLlPj2xdDDjkEb46kf8l+AatWnk6u1z5XjiGGnc3T3kLRRfVphj4B S2b55QwuUldXJtI59aOcfNOkHdOIBWKCGLBZKkI7lgsIcsE1YAEjSgKMBQ/Q4FTb yzXnEla5bZa0daAcPkPkPwm43DQSygVrnuKqdxOglQFkvFLTB03qiEW9pa0yAhts 2qEVQyXo14sfehu2c0NoyWREnttbgrjIVZqHAM7yVRqYl5AxTZwMA8s0IohGdFTE ZRakg8uOzjhgJuXmL67VnowrVBiDLirezbTZxHLpqSBzQwyClBCbY4qgt0gmYeZ8 LMNRDYLE4r7bK9kweLhg+iJbsD0JQJUOS6H5l0LraDAVfHKtiH+dgRqwtGI0xSLh zjY+aAXUc1fpULgO1BwXHuTGh0uhXFRN8A++Xmki7DtIKm6LP40NjfiCg9nTgXkf Gv1bCzUxh1feCoJXWR4dt9oMUwcnpxPO2S16MQ4mwC0VGPedhg2D10/Ltfe8BzkZ hFodw8MQyWjTT2Xz78BI4sXGfbldYIgng0bkfvWXJzNhJ4M35sTEPUvQyLbj7szg n6cZZ/aOzMlg33sgSaS2UG0UzCa5XUtqSdmkeNuc7rrGr4/+GvV93ZXofLJnvaYZ kFz7aLxyS8kr3LR9aM7O =v2Lf -END PGP SIGNATURE- ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Env variables for chutney
> On 4 Jul 2015, at 05:37 , Cory Pruce wrote: > > > Signed PGP part > Hey Tim, > > I see that I may need to set some environment variables as told by the > git readme but the error message is on a different env var: > > cory@Nulix ~/tor/tor =) make test-network > make all-am > make[1]: Entering directory `/home/cory/tor/tor' > make[1]: Leaving directory `/home/cory/tor/tor' > ./src/test/test-network.sh > test-network.sh: missing 'chutney' in CHUTNEY_PATH (~/tor/chutney/) > make: *** [test-network] Error 1 > > > I also tried modifying my .bashrc so that the CHUTNEY_PATH pointed to > the executable but no luck. Do I need to set the chutney path? The CHUTNEY_PATH variable needs to point to a directory containing a chutney executable. So you seem to have it right the first time. What do you get when you run: ls -l ~/tor/chutney/ I would expect to see an executable script called "chutney" listed in that directory, along with the other chutney distribution files. The test-network.sh script is complaining that the "chutney" script is missing. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com pgp ABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7 signature.asc Description: Message signed with OpenPGP using GPGMail ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
