[tor-relays] Metrics for assessing EFF's Tor relay challenge?
Hi Christian, other tor relay fans, I'm looking for some volunteers, hopefully including Christian, to work on metrics and visualization of impact from new relays. We're working with EFF to do another "Tor relay challenge" [*], to both help raise awareness of the value of Tor, and encourage many people to run relays -- probably non-exit relays for the most part, since that's the easiest for normal volunteers to step up and do. You can read about the first round from several years ago here: https://www.eff.org/torchallenge To make it succeed, the challenge for us here is to figure out what to measure to track progress, and then measure it and graph it for everybody. I'm figuring that like last time, EFF will collect a list of fingerprints of relays that signed up "because of the challenge". One of the main pushes we're aiming for this year is longevity: it's easy to sign up a relay for two weeks and then stop. We want to emphasize consistency and encourage having the relays up for many months. So what are the things we'd want to track? - Number of relays signed up that are Running, over time. - Total bandwidth history of these running relays, over time. - Maybe a graph showing the total number of bytes ever contributed by these relays? That would impress people perhaps. - Total consensus weight of these running relays, over time. - Something emphasizing duration -- e.g. the total consensus weight of the subset of the relays that have been in the consensus for 90% of the past month, 2 months, 6 months, etc. Are there better ideas here I hope? We'll want to be cognizant that if we're in the first week of the challenge, the 2 month graph will be empty and thus look sad. - Something comparing the above numbers to the total numbers. Given how huge some of the relays are lately, it would be easily to visualize the new contribution as a tiny irrelevant fraction, which could be disheartening to new relay operators even if their relays will actually become a big deal with some patience. What are some strategies for making this work right? E.g. a layer graph showing y layered on top of x where y is the new contribution, rather than a percentage-of-total graph that shows approximately 0%. We could also imagine more niche categories. For example, if we're hoping to get people to sign up relays at universities, we could imagine that the folks running the challenge give us a list of fingerprints of relays that self-identify as being at universities, and then we do up the same set of graphs with that subset of relays. So, Christian, others, how much of this is possible as-is or with some limited tweaking, with Globe and related scripts? I am hoping the answer is most of it. :) I also cc Karsten because a lot of this overlaps with the metrics scripts, but I am expecting Karsten to push back against the idea of integrating these measurements more with the metrics project. Any other ideas for what to measure to help people know whether their contribution is being worthwhile? [*] Please don't take this mail as any official announcement, or timeline, or any of that. At this point we need to collect people to help make this happen, not collect news stories. Thanks! --Roger ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] log: Error binding network socket to 203.217.31.172: Cannot assign requested address
>From here: Mar 27 16:50:35.000 [warn] Error binding network socket to 203.217.31.172: Cannot assign requested address to here: Mar 27 23:56:46.000 [warn] Error binding network socket to 203.217.31.172: Cannot assign requested address I got 828 of those messages. Why no socket number, or more information in the logs to debug this? Although it only happened in that time frame above - not happening any more. TIA Zenaan ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Metrics for assessing EFF's Tor relay challenge?
On 27/03/14 11:57, Roger Dingledine wrote: > Hi Christian, other tor relay fans, > > I'm looking for some volunteers, hopefully including Christian, to work > on metrics and visualization of impact from new relays. > > We're working with EFF to do another "Tor relay challenge" [*], to both > help raise awareness of the value of Tor, and encourage many people to > run relays -- probably non-exit relays for the most part, since that's > the easiest for normal volunteers to step up and do. > > You can read about the first round from several years ago here: > https://www.eff.org/torchallenge > > To make it succeed, the challenge for us here is to figure out what to > measure to track progress, and then measure it and graph it for everybody. > > I'm figuring that like last time, EFF will collect a list of fingerprints > of relays that signed up "because of the challenge". > > One of the main pushes we're aiming for this year is longevity: it's > easy to sign up a relay for two weeks and then stop. We want to emphasize > consistency and encourage having the relays up for many months. Before going through your list of things we'd want to track below, let's first talk about our options to turn a list of fingerprints into fancy graphs: 1. Write a new metrics-web module and put graphs on the metrics website. This means parsing relay descriptors and storing certain per-relay statistics for all relays. That gives us maximum flexibility in the kinds of statistics, but is also most expensive in terms of developer hours. I don't want to do this. 2. Extend Globe to show details pages for multiple relays. This requires us to move to the server-based Globe-node, because the poor browser shouldn't download graph data for all relays, but the server should return a single graph for all relays. It's also unclear if the new graphs will be of general interest for Globe users, and if the rest of the Globe details will be confusing to people interested in the relay challenge. Probably not a great idea, but I'm not sure. 3. Extend Onionoo to return aggregate graph data for a given set of fingerprints. Seems useful. But has the big disadvantage that Onionoo would suddenly have to create responses dynamically. I'm worried about creating a new performance bottleneck there, and this is certainly not possible with poor overloaded yatei. 4. Write a new little tool that fetches Onionoo documents once (or twice) per day for all relays participating in the relay challenge and that produces graph data. That new tool could probably re-use some Compass code for the backend and some Globe code for the frontend. Graphs could be integrated directly into EFF's website. This is currently my favorite approach. Note for 2--4: Onionoo currently only gives out data for relays that have been running in the past 7 days. I'd have to extend it to give out all data for a list of fingerprints, regardless of when relays were running the last time. That's 2--3 days of coding and testing for me. It's also potentially creating a bottleneck, so we should first have a replacement for yatei. > So what are the things we'd want to track? > > - Number of relays signed up that are Running, over time. We can do something here with Onionoo's new uptime documents. > - Total bandwidth history of these running relays, over time. We can sum up data from bandwidth documents for this. > - Maybe a graph showing the total number of bytes ever contributed > by these relays? That would impress people perhaps. Sure, same data as above. > - Total consensus weight of these running relays, over time. We only have total consensus weight *fraction*, but yes. > - Something emphasizing duration -- e.g. the total consensus weight of > the subset of the relays that have been in the consensus for 90% of > the past month, 2 months, 6 months, etc. Are there better ideas here > I hope? We'll want to be cognizant that if we're in the first week > of the challenge, the 2 month graph will be empty and thus look sad. Not sure what the 90% part is for, but yes, graphs with total consensus weight fraction are doable. Regarding the sad-looking 2 month graph, we can easily define the data when the challenge starts and not show graphs until they make sense. Note that the current intervals for most data are 1 week, 1 month, 3 months, 1 year, and 5 years. > - Something comparing the above numbers to the total numbers. Given how > huge some of the relays are lately, it would be easily to visualize > the new contribution as a tiny irrelevant fraction, which could be > disheartening to new relay operators even if their relays will actually > become a big deal with some patience. What are some strategies for > making this work right? E.g. a layer graph showing y layered on top of > x where y is the new contribution, rather than a percentage-of-total > graph that shows approximately 0%. Absolute contributions to consensus weight are not availabl
[tor-relays] the obfsproxy bundle, and decrease in number of users to my bridges
A repost, as I didn't get any answers when I posted in another thread: Where can I download an obfsproxy bridge bundle nowadays? It seams to have disappeared. I use both win and linux(but not debian/ubuntu). I have found dual information pages about setting up obfsproxy+tor in Debian/Ubuntu Here: https://www.torproject.org/projects/obfsproxy-instructions.html.en And here: https://www.torproject.org/projects/obfsproxy-debian-instructions.html.en Why dual instructions for Debian/Ubuntu? But no instructions for other OS:es, and no direct download links with compiled packages/programs here: https://www.torproject.org/dist/ How do you suppose that people will be able to help setting up obfsproxy bridges, when there isn't any way to get instructions or packages/programs? Previously I ran an obfsproxy bridge and saw the amount of traffic get lower and lower, at the end it was only about 2-10 MB of traffic per week and with very few users. Some weeks I go no users so I stopped wasting my electricity and stopped running the obfs-bridge. But now I would like to try again after a couple of months pause. I'm interested to see if the Internet-blocking in Turkey might have caused some more traffic. I also used to run a normal bridge up until a couple of weeks ago but saw the traffic get as low as around 50MB per week, while normally I used to get around 1-2GB per week so I shut it down too. I asked myself: "What's the point of wasting energy and at the same time making it easier to hack into my network, when I'm not even helping the world?". So I shut it down too. Perhaps some changes in the tor network after the release of Tor 0.2.4 has reduced the number of bridge and obfsproxy users globally? Due to the constant flow of bugs in the different tor-related software and bundles, my first guess is that the problem is with changes in Tor plus related software. A look at the bug tracker makes me wonder if there are more things in the Tor related software that do work or don't work??? But perhaps more bridges are blocked nowadays by the filters that various countries apply to the Internet? Or perhaps less countries block their Internet users so less people need to use bridges? Or perhaps a lot of new bridge operators have arisen, making every bridge receive less traffic? My ISP isn't known for blocking anything and I've never heard of any government blocking of Tor in the EU, so the problem is probably not with my ISP nor the EU. I have always been running the latest stable release of the bundles, so It doesn't have anything to do with old versions. So my questions: Where can I find a obfsproxy bridge bundle for rpm-linux and for windows? Is there any good explanation to why the traffic to my normal bridge and my obfsproxy bridge got lower and lower after the release of tor 0.2.4 as stable? ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Tor-Plugin for munin
> Everybody likes graphs, right? > > I tried some of the available munin plugins, but none of those were > really working anymore. So I sat down and began writing my own, using > stem, which is really awesome. > > These are really early stages, but it already supports autoconf > determination and suggestions. If you want to take a look, it is > hosted on github[1]. > > At the moment it supports a connection graph, getting its data from > orconn-status. More graphs are possible, but not yet implemented. > Ideas are welcome. Neat, thanks Martin! Added munin-tor to our examples... https://stem.torproject.org/tutorials/double_double_toil_and_trouble.html Few quick thoughts about the Stem bits... > with Controller.from_port(port=port) as controller: > controller.authenticate() > if controller.is_authenticated(): > print('yes') > else: > print('no (Authentication failed)') The is_authenticated() is unnecessary. The authenticate() call will raise an AuthenticationFailure if unsuccessful... https://stem.torproject.org/api/connection.html#stem.connection.authenticate > response = controller.get_info('orconn-status') You might want to do the following instead if you'd rather not risk exceptions... response = controller.get_info('orconn-status', None) if response is None: return > states = {'NEW': 0, 'LAUNCHED': 0, 'CONNECTED': 0, 'FAILED': 0, 'CLOSED': 0} Up to you but you could also use Stem's ORStatus enum. >>> import stem >>> dict((state, 0) for state in stem.ORStatus) {'NEW': 0, 'CONNECTED': 0, 'CLOSED': 0, 'LAUNCHED': 0, 'FAILED': 0} > response = controller.get_info('traffic/read') > print('read.value {}'.format(response)) > response = controller.get_info('traffic/written') > print('written.value {}'.format(response)) Again, this could potentially raise exceptions unless you provide get_info() a default value. Cheers! -Damian ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Metrics for assessing EFF's Tor relay challenge?
On Thu, Mar 27, 2014 at 3:25 PM, Karsten Loesing wrote: > Before going through your list of things we'd want to track below, let's > first talk about our options to turn a list of fingerprints into fancy > graphs: Would it be possible to also have a "Top 10 countries with the most Tor relays" graph? -- Runa A. Sandvik ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Tor-Plugin for munin
On 27.03.2014 17:57, Damian Johnson wrote: >> Everybody likes graphs, right? >> >> I tried some of the available munin plugins, but none of those were >> really working anymore. So I sat down and began writing my own, using >> stem, which is really awesome. >> >> These are really early stages, but it already supports autoconf >> determination and suggestions. If you want to take a look, it is >> hosted on github[1]. >> >> At the moment it supports a connection graph, getting its data from >> orconn-status. More graphs are possible, but not yet implemented. >> Ideas are welcome. > > Neat, thanks Martin! Added munin-tor to our examples... > > https://stem.torproject.org/tutorials/double_double_toil_and_trouble.html > > > Few quick thoughts about the Stem bits... > >> with Controller.from_port(port=port) as controller: >> controller.authenticate() >> if controller.is_authenticated(): >> print('yes') >> else: >> print('no (Authentication failed)') > > The is_authenticated() is unnecessary. The authenticate() call will > raise an AuthenticationFailure if unsuccessful... > > https://stem.torproject.org/api/connection.html#stem.connection.authenticate > >> response = controller.get_info('orconn-status') > > You might want to do the following instead if you'd rather not risk > exceptions... > > response = controller.get_info('orconn-status', None) > > if response is None: > return > >> states = {'NEW': 0, 'LAUNCHED': 0, 'CONNECTED': 0, 'FAILED': 0, 'CLOSED': 0} > > Up to you but you could also use Stem's ORStatus enum. > import stem dict((state, 0) for state in stem.ORStatus) > {'NEW': 0, 'CONNECTED': 0, 'CLOSED': 0, 'LAUNCHED': 0, 'FAILED': 0} > >> response = controller.get_info('traffic/read') >> print('read.value {}'.format(response)) >> response = controller.get_info('traffic/written') >> print('written.value {}'.format(response)) > > Again, this could potentially raise exceptions unless you provide > get_info() a default value. > > Cheers! -Damian > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays > Hey Damian, thanks for your input. I'll gladly work on those issues in the next few days. Martin ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] the obfsproxy bundle, and decrease in number of users to my bridges
Viking God writes: > A repost, as I didn't get any answers when I posted in another thread: > > Where can I download an obfsproxy bridge bundle nowadays? It seams to > have disappeared. I use both win and linux(but not debian/ubuntu). > > I have found dual information pages about setting up obfsproxy+tor in > Debian/Ubuntu Here: > https://www.torproject.org/projects/obfsproxy-instructions.html.en > And here: > https://www.torproject.org/projects/obfsproxy-debian-instructions.html.en > > Why dual instructions for Debian/Ubuntu? > But no instructions for other OS:es, and no direct download links with > compiled packages/programs here: > https://www.torproject.org/dist/ > > How do you suppose that people will be able to help setting up obfsproxy > bridges, when there isn't any way to get instructions or packages/programs? > Greetings Viking God, I'm afraid that you are correct, and we indeed don't have any up-to-date obfsproxy bridge bundles. The main reason for this is that we lack manpower and making proper bundles takes a while. It took lots of time and effort to build PT-supporting TBBs (check out tbb-3.6!). Making bridge bundles is no easier and probably requires extra UI work so that users can configure their PTs. Also, because we have many tech-savy people running bridges in powerful Unix systems, it's easy for us to claim that spending all that time just to support a few Windows boxes under home Internet connections might not be worth the time and effort. Also also, we are moving away from Vidalia (see the latest TBB releases), and I'm not sure how future bridge bundles are going to work. That is, whether they will use Vidalia or tor-launcher will do that too. That said, you are absolutely right that this is not optimal and by doing so we are ruling out many potential bridge operators. We should get back on track with this project. BTW, if you want to help with this, you might want to join the PT meeting we are doing tomorrow at 17:00 UTC in #tor-dev. And there is also a TBB meeting at 18:00 UTC, so it's a very good opportunity to get up to speed with what needs to be done. > Previously I ran an obfsproxy bridge and saw the amount of traffic get > lower and lower, at the end it was only about 2-10 MB of traffic per > week and with very few users. Some weeks I go no users so I stopped > wasting my electricity and stopped running the obfs-bridge. But now I > would like to try again after a couple of months pause. I'm interested > to see if the Internet-blocking in Turkey might have caused some more > traffic. > > I also used to run a normal bridge up until a couple of weeks ago but > saw the traffic get as low as around 50MB per week, while normally I > used to get around 1-2GB per week so I shut it down too. I asked myself: > "What's the point of wasting energy and at the same time making it > easier to hack into my network, when I'm not even helping the world?". > So I shut it down too. > > Perhaps some changes in the tor network after the release of Tor 0.2.4 > has reduced the number of bridge and obfsproxy users globally? Due to > the constant flow of bugs in the different tor-related software and > bundles, my first guess is that the problem is with changes in Tor plus > related software. A look at the bug tracker makes me wonder if there are > more things in the Tor related software that do work or don't work??? > But perhaps more bridges are blocked nowadays by the filters that > various countries apply to the Internet? > Or perhaps less countries block their Internet users so less people need > to use bridges? > Or perhaps a lot of new bridge operators have arisen, making every > bridge receive less traffic? > My ISP isn't known for blocking anything and I've never heard of any > government blocking of Tor in the EU, so the problem is probably not > with my ISP nor the EU. > I have always been running the latest stable release of the bundles, so > It doesn't have anything to do with old versions. > > So my questions: > Where can I find a obfsproxy bridge bundle for rpm-linux and for windows? > Is there any good explanation to why the traffic to my normal bridge and > my obfsproxy bridge got lower and lower after the release of tor 0.2.4 > as stable? > I don't have a definite explanation on why your traffic diminished like that. As a possible scenario, BridgeDB has been experiencing some technical difficulties lately (for example, the CAPTCHA is stupidly hard to solve), so the number of people getting bridges of BridgeDB might not be that great. Most clients are probably concentrated on the bridges that are included in the bundles, and the bridges that are passed through the support team. I still don't know how much traffic BridgeDB bridges yet, and whether you are going to get more clients if you run a bridge now. Hopefully you will. If not, you might want to send your bridge credentials to the support team, so that they can personally pass your bridge to users. ___
[tor-relays] Tor download signatures
OK, perhaps I have missed "the how" and "which" somewhere, but which signature am I supposed to verify the new Tor 0.2.5.3 tarball against? I tried the ones mentioned on Tor signing page and none seem to stick. A typical message is: # gpg --verify tor-0.2.5.3-alpha.tar.gz{.asc,} gpg: Signature made Sun 23 Mar 2014 02:40:49 AM UTC using RSA key ID 8D29319A gpg: Good signature from "Nick Mathewson " gpg: aka "Nick Mathewson " gpg: aka "Nick Mathewson " gpg: aka "[jpeg image of size 3369]" gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: B35B F85B F194 89D0 4E28 C33C 2119 4EBB 1657 33EA Subkey fingerprint: EF00 F369 1387 FCC5 8CD6 8E13 9103 97D8 8D29 319A ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] Avoiding sinkholes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Heya List I currently run a VPS which hosts both my mailserver and my tor relay / exit. Recently I sent an email from this mailserver and had it bounce back. It seems the receiving mailserver subscribes to the spambot list CBL (http://cbl.abuseat.org) and denied it because my IP address was on that list. It's on that list since at some point a botnet talking through tor to its C&C server used my exit node to do so - The C&C server has since been replaced with a sinkhole. That was logged, my server was deemed infected and bam, I'm blacklisted. The site that did the blacklisting kindly has a good description of what happended (including the sinkhole IP address) and allowed an automatic delisting. I'm able to update my exit policy so it doesn't happen again, however I'd like a somewhat more proactive approach. So my question is - Does anyone know of a publicly available list of sinkholes created for botnets? If such a list exists I can dynamically update either my exit policy or firewall appropriately. Has anyone implemented such a system already? (obviosuly this only works for sinkholed botnets - but if anyone knows how to stop all botnets I'm all ears) Cheers Ramo -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (GNU/Linux) iQEcBAEBAgAGBQJTNQpGAAoJEAXQWoW8lug/mdkH/jqK5ndFWbVrRnNV7a8IqgYl 4iiR6TUYLjGEdcz8VDZ+cOkW0uCQkOvD6RWl/kWHKmA4iy7alFgbKE4Lkcg4QgBB 7EgNYww3zBbj1NX5rtRN7POge2n4ns7Y7whw0qbvHXE0ur74iLyy5H3hHZeWoosU g6t9mCsMpEpGAvbkzkDxo+idAdxYe+JiB3iaAAEUDtzeStOG5RJ/qrg8JM+U7ofA bWPehBV8+V8E//4G/XLsePCciBN0071ylg+YoGcxDpM97WHvmQKbi5VU3KfQJFTB AFaQ+7ib1BNhe7KEC7V0Iha6Yu/BTNhDRA/0i1C3pad32HQsfGvep261yq+KBjc= =Wemh -END PGP SIGNATURE- ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Avoiding sinkholes
Hi Ramo, Thanks for running an exit! You will run into similar trouble again and again, unfortunately. My suggestion: Get a second IP, or even better, don't share the same VPS. -- Moritz Bartl https://www.torservers.net/ ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Metrics for assessing EFF's Tor relay challenge?
On 27/03/14 19:51, Runa A. Sandvik wrote: > On Thu, Mar 27, 2014 at 3:25 PM, Karsten Loesing > wrote: >> Before going through your list of things we'd want to track below, let's >> first talk about our options to turn a list of fingerprints into fancy >> graphs: > > Would it be possible to also have a "Top 10 countries with the most > Tor relays" graph? Hi Runa! Hmm hmm hmm---yes! Onionoo's details documents contain country information, and it shouldn't be too hard to combine them with uptime or bandwidth information to make per-country graphs. (Wow, your question made me rethink how we resolve relay/bridge IP addresses to country codes for statistics. I was always thinking that we need to remember the full history of country codes that a relay/bridge IP address was resolved to, because a relay/bridge could be moved to another country, or a new IP-to-country database might change its mind about which country it is in. But that doesn't really matter for statistics where we're mostly interested in the big picture. We can probably just use whatever country code we learned last and apply that to the full history of the relay/bridge. Guess I should resume working on per-country graphs for the metrics website soon, for both relays and bridges. Thanks!) (Disclaimer: it's pre-second coffee time!) All the best, Karsten ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Avoiding sinkholes
On Fri, Mar 28, 2014 at 08:36:06AM +0300, r...@goodvikings.com wrote: > It's on that list since at some point a botnet talking through tor to >its C&C server used my exit node to do so Actually, it could easily have been a computer security researcher who used Tor to access that address, not realizing the collateral damage he was triggering. A growing number of malware researchers and antivirus companies use Tor to reach various parts of the Internet, because otherwise the bad guys recognize their IP address and special-case them. As Moritz says, this is alas not an easy game to win. Not long ago I learned that the .mil domain refuses to hear any packets from my computer, which runs one of the directory authorities (and it's not even an exit relay!). That meant my postfix became convinced that all mails to or from .mil addresses were spam, since their name doesn't resolve. Bad news for the Navy researchers who are signed up to, say, the petsymposium.org mailing lists. The real fun is going to start when these blacklists try to bully us by blacklisting the whole /24 nearby, in hopes that our neighbors will lean on us to cut it out. I still enjoy rereading http://paulgraham.com/spamhausblacklist.html as linked from https://www.torproject.org/docs/faq-abuse#TypicalAbuses --Roger ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays