Re: [tor-relays] On the way to more diversity
On 12.04.2013 19:40, Steve Snyder wrote:
> I thinking the sticking point is documentation.
>
> From what repository do I get the obfs3 code? How do I build obfs3?
> How do I specify its use in my Tor config file (i.e. config syntax)?
>
> If obfs3 use is so important, then why is there so little
> documentation for it on Tor's website?
There will be a time when obfs4 or something else that isn't even
defined or on the horizon yet will become important. Especially with
Internet censorship, things like that can change quickly. The obfs2
blocking of China only happened recently.
obfs3 needed to be designed, specified, implemented, tested, and is now
in the latest obfsproxy release available on Debian testing (and looks
like it is now also available from deb.torproject.org, but I'm not
certain about that).
Documentation is linked from the main Obfsproxy website,
https://www.torproject.org/projects/obfsproxy.html.en ("Installation
Instructions"). If you have suggestions on what to improve on the
website, in general or about obfsproxy, file a ticket. In the end, this
is a community effort!
--
Moritz Bartl
https://www.torservers.net/
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
On 12.04.2013 19:16, Matt Joyce wrote: > It would help a lot if we used versioning and stopped sending almost > unchanged data constantly and instead only providing the changes I doubt that this is easy to do in a privacy-preserving way. You don't want to be able to discriminate relays based on what diffs/what amount of data they pull, right? -- Moritz Bartl https://www.torservers.net/ ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
> Bittorrent may be an exception to the above but the performance cost > would be at the clients end and for one bittorrent is hardly a realtime > protocol a little delay making each connection would not make much > difference, two it performs poorly if you insist on running it over tor > anyway and thirdly the average consumer desktop system is not exactly > lacking spare CPU cycles due to the bursty nature of their workloads > they statistically tend to spend the majority of their time idling. As opposed to torrenting via exits, I'm amazed more people haven't moved their torrenting to operate entirely within anonymous networks. The performance of I2P and Tor (and I guess Phantom during tests) is actually quite usable so long as the user excercises content selection and patience. And regarding this thread, the freedom of operators and users from complaints regarding takedown would be invaluable. I'm not sure if these networks could scale to handle the node count and chatter, but I think the bit about spare cycles is right and might even act as a natural limiter to that sort of traffic. For instance, my tests show that establishing connections to many onions in parallel is quite costly and prone to failure without available headroom. But once connected, things are ok. Perhaps torrenters simply can't put aside their 'must download it all right now' mentality which keeps them away from our nodes. Or there is a major influx waiting to happen upon some future enlightenment whether they're misguided or not. ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
In some work I've done, limitations would follow as such... a) Advertising non-desire for traffic (exit policy) is the same as packet filtering with the same rules locally. b) You can filter whatever you want at any inspection level you want, for whatever reason, or random/no reason, ***so long as you are doing it in a user agnostic global policy fashion.*** ie: - you may block torproject IP address or any other place for everyone because you think spacemen live there. You may not block it for just user X unless provided for under other policy. - you may scrape email content and block all viruses and even all talk of puppy dogs, so long as it is as a global policy and not just for user X. - you may sink all bittorrent for bandwidth. - you may sink content/protocol you think is too hot/unproven legally, or just whatever you want as global policy. - you may place bandwidth caps, block protocols, throttle, reset, cache, analyze, report, aggregate, rank, etc as matter of global policy and operations. It was not what you block or permit or where or why or how, but when you delve into the affairs of a particular user with singular intent. That is the hot area. Most particularly concerning the written content of their traffic, consumer profile, etc. Best example of this is locking an account because you took a ticket that it was 'abusing' something somewhere somehow in a protocol/traffic fashion, leeching, spam, etc... but you have no global policy for it, and you still sunk them. That would get you in trouble customer contract wise. That's why TOU's are so broad. But touching any ticket involving humanity ... hatemail, threats, bickering, chatroom, trolling, assholeness, crime, etc would be *strictly and always* returned with 'contact your authorities' and zero inspection or blocking in such regard. It's an anecdote, and certainly common best practice in many areas. Consult your area experts as needed. ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
> tor could easily be made to efficiently use a similar mechanism, if it > doesn't already in order to perform the lookups to compute the answer to > "What is the subset of exit nodes allowing exit to IP addr X on port Y?" The answer may lie with the client polling some exits and computing the answer to its needs locally. I just posted a blurb about this to tor-dev. Anyone interested may want to follow up and add on over there. ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] On the way to more diversity
On 05/04/13 12:34, Philipp Winter wrote: > On Thu, Apr 04, 2013 at 06:37:51AM +0200, Andreas Krey wrote: >> And do obfs3 bridge help that are run on IPs >> also used for regular relays? > Right now, it is important to get more obfs3 bridges for China since obfs2 no > longer works [0]. In general, it would be better to run bridges and relays on > separate IP addresses to defend against censors who simply blacklist all IP > addresses listed in the consensus. At least in China, however, it is currently > possible to run both on just one IP address since the GFW blocks Tor relays > based on IP:port tuples. I thinking the sticking point is documentation. >From what repository do I get the obfs3 code? How do I build obfs3? How do I >specify its use in my Tor config file (i.e. config syntax)? If obfs3 use is so important, then why is there so little documentation for it on Tor's website? ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] On the way to more diversity
On 05/04/13 12:34, Philipp Winter wrote: > On Thu, Apr 04, 2013 at 06:37:51AM +0200, Andreas Krey wrote: >> And do obfs3 bridge help that are run on IPs >> also used for regular relays? > Right now, it is important to get more obfs3 bridges for China since obfs2 no > longer works [0]. In general, it would be better to run bridges and relays on > separate IP addresses to defend against censors who simply blacklist all IP > addresses listed in the consensus. At least in China, however, it is currently > possible to run both on just one IP address since the GFW blocks Tor relays > based on IP:port tuples. > > Cheers, > Philipp > > [0] https://trac.torproject.org/projects/tor/ticket/8591 > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays I don't really know much about obsfproxy only heard about it a bit on here but if the above is correct perhaps someone could direct me to some useful info on how to go about setting up such, I'd be happy to add the service on my 1Gbps relay at least to trial it out and then if demand is high enough I'll look to acquire another address. signature.asc Description: OpenPGP digital signature ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
On 12/04/13 15:03, Moritz Bartl wrote: > On 12.04.2013 13:33, Matt Joyce wrote: I assume you mean firewall-based blocking? You could have simply rejected those IPs via ExitPolicy (see "man tor"). That's a clear-cut way to tell the network you don't accept connections to those IPs, and no risk of being labeled a BadExit. >>> The latter. I dont know if it complicates routing decisions in the Tor >>> network to have lots of ip address exceptions at the exits... >> It possibly does slightly but we are talking about the sort of >> "complication" that a modern CPU can resolve in timescales short enough >> I doubt it would make that much difference especially if hash tables >> were used > That's not the issue here. You don't want to make the "routing table" > (consensus and descriptors) that every Tor client needs to have too > large. Users in developing countries, on mobile connections or generally > instable connections already have issues. > It would help a lot if we used versioning and stopped sending almost unchanged data constantly and instead only providing the changes they are after all text files and there is a whole range of open source projects out there with the functionality to manage versioning and incremental updates for text files, the tradeoff of course would be in the slightly increased disk space requirements for dirmirriors. That said I'm not arguing that we *should* do this merely that it is possible and there are technical options to accommodate it if it were required by enough of the community to pose a potential issue. Basically my point is that the it's possible the question therefore more one of whether we should, I don't particularly see the need and there are arguments against it I don't like censorship at the best of times which is why I run the exits I have I am just countering the technical arguments which I believe are potentially solvable if there was consensus that it is warranted and therefore are not good reasons to dismiss the idea. As for the unstable connections issue, yeah that I can understand that said tor uses TCP only and TCP is *known* to perform at less than optimum in the event of unstable connections with random drops, it will get the data there but it will also do it very slowly interpreting dropped packets as congestion. Radio communications frequently cause TCP to have problems which is why the FAQ for almost any high bandwidth TCP application will invariably suggest things like "Use a hardwired connection if possible not wireless". I'm not dismissing it as an issue of course but no TCP application is going to work well if the underlying network infrastructure is poor and I am not sure how that is going to be solved without A) Replacing TCP with something else (But what?) or B) Major investment in those networks which is unfortunately unlikely until there is a large enough user base to create the demand to make someone with capital actually pay the slightest bit of attention because they generally are only interested when they can generate a profit. signature.asc Description: OpenPGP digital signature ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
On 12.04.2013 13:33, Matt Joyce wrote: >>> I assume you mean firewall-based blocking? You could have simply rejected >>> those IPs via ExitPolicy (see "man tor"). That's a clear-cut way to tell the >>> network you don't accept connections to those IPs, and no risk of being >>> labeled a BadExit. >> The latter. I dont know if it complicates routing decisions in the Tor >> network to have lots of ip address exceptions at the exits... > It possibly does slightly but we are talking about the sort of > "complication" that a modern CPU can resolve in timescales short enough > I doubt it would make that much difference especially if hash tables > were used That's not the issue here. You don't want to make the "routing table" (consensus and descriptors) that every Tor client needs to have too large. Users in developing countries, on mobile connections or generally instable connections already have issues. -- Moritz Bartl https://www.torservers.net/ ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] big spike in cpu usage
On 08/04/13 18:41, Andreas Krey wrote: > On Mon, 08 Apr 2013 08:47:56 +, Sebastian Hahn wrote: > ... >> Now, it's entirely possible I'm missing something big here; or that the >> code changed and now does something different; or that it used to do >> something different, etc. Andreas, can you please explain more? > At least the original change explains different: > > +--- ReleaseNotes - > | > Changes in version 0.0.2pre20 - 2004-01-30 > | ... > | > | - I've split the TotalBandwidth option into BandwidthRate (how many > | bytes per second you want to allow, long-term) and > | BandwidthBurst (how many bytes you will allow at once before the cap > | kicks in). This better token bucket approach lets you, say, set > | BandwidthRate to 10KB/s and BandwidthBurst to 10MB, allowing good > | performance while not exceeding your monthly bandwidth quota. > +- > > ..which is pretty much my usage scenario, just with smaller numbers. > > And the code looks likewise. We have the global_*_buckets that are > initialized from *BandwidthBurst, and get incremented regularly by > *BandwidthRate (divide by increment frequency; TokenBucketRefillInterval) > and then capped to the *BandwidthBurst. > > Thus *BandwidthBurst ist the total amount of unused traffic we can > save up to later fire with more than *BandwidthRate. No 'per second'. > > (The interesting part is that the global_*_bucket are ints; > much more than the 1 GB default could behave strangely.) > > Andreas > Wouldn't using the AccountingMax and AccountingStart configuration options be more suitable if the objective is to enforce a bandwidth limitation over timeframes of a day/week/month instead of trying to do it this way which limits burst durations, using the accounting options it would seem to me that it's more flexable to the needs of the network as the accounting options were designed for this purpose. In particular I note the following (src tor manpage) emphasis is mine: AccountingMax N bytes|KBytes|MBytes|GBytes|TBytes Never send more than the specified number of bytes in a given accounting period, or receive more than that number in the period. For example, with AccountingMax set to 1 GByte, a server could send 900 MBytes and receive 800 MBytes and continue running. It will only hibernate once one of the two reaches 1 GByte. When the number of bytes gets low, Tor will stop accepting new connections and circuits. When the number of bytes is exhausted, Tor will hibernate until some time in the next accounting period. To prevent all servers from waking at the same time, Tor will also wait until a random point in each period before waking up. If you have bandwidth cost issues, ***enabling hibernation is preferable to setting a low bandwidth, since it provides users with a collection of fast servers that are up some of the time, which is more useful than a set of slow servers that are always "available".*** AccountingStart day|week|month [day] HH:MM Specify how long accounting periods last. If month is given, each accounting period runs from the time HH:MM on the dayth day of one month to the same day and time of the next. (The day must be between 1 and 28.) If week is given, each accounting period runs from the time HH:MM of the dayth day of one week to the same day and time of the next week, with Monday as day 1 and Sunday as day 7. If day is given, each accounting period runs from the time HH:MM each day to the same time on the next day. All times are local, and given in 24-hour time. (Default: "month 1 0:00") Essentially this way you can give tor a "budget" for the month for example to not to exceed but there are no constraints on when or how long it can burst thus while capacity remains in your package subject to the limits you have set the network can use the capacity whenever it is required by the needs of the network. Before using this however do check whether your provider counts upload download or both, if they count both you would need to set AccountingMax to half the stated value in order to ensure you remain bellow the limit. signature.asc Description: OpenPGP digital signature ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
On 11/04/13 20:00, Moritz Bartl wrote: > On 11.04.2013 12:15, t...@caber.nl wrote: >> If we want to avoid the packet-dropping problem: We could also reject >> the IP-addresses of those sites with torrc. What is your opinion about >> that Moritz? And, would it ok for the authorities and users with little >> bandwith if I reject ~100 ip-adresses? (Not that I am going to) > Apart from the last section of my answer to bartels: Yes, listing 100+ > IPs in the exit policy is not very nice for the Tor network either. :( > > I guess the simple answer would be: > > 1) Allow everything (except port 25, which is reasonable to block) > 2) If you don't want the DMCA spam notices, use the reduced exit policy. > It's not ideal if everyone does it however that's primarily due to the distribution mechanism, for now it is sending the *entire* file every time over HTTP there are however ways this could be distributed better if the community of exit operators were to decide such was necessary and the size of the descriptors thus began to increase such as versioning and distributing diffs to clients a conversation along the lines of: Client: Request descriptor for relay with $FINGERPRINT have revision X. DirMirror: Descriptor for relay $FINGERPRINT diff revision X current revision Y data follows. This would increase the necessary storage space on each mirror a little using a storage system like git, hg or svn use code from these could probably be reused, or even just use hg as a dependency and make a local repo with no daemon but unless everyone is changing their descriptors in major ways on a regular basis I suspect it would be quite small especially if older copies were expired after a reasonable period of time maybe 30 days and clients with older versions are just sent the full current version so only clients that use the network very infrequently would be actually downloading the entire policy. We are talking plain text files here too so I cant see the storage issue being a massive one given that HDD space is inexpensive now and even inexpensive VPS servers typically offer hundreds of GB of it, the main issue would be bandwidth which honestly I suspect would be lower or at worst similar if we only distributed diffs to the majority of clients. signature.asc Description: OpenPGP digital signature ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
On 04/12/2013 11:35 AM, Troy Arnold wrote: On Fri, Apr 12, 2013 at 11:00:42AM +0200, bartels wrote: On 04/12/2013 10:06 AM, Moritz Bartl wrote: On 11.04.2013 22:17, bartels wrote: I don't see the legal issue, though. Maybe it is there, but I don't see how rejecting sites via Exit Policy ;) would trigger any one of (1) through (5). Yes, rejecting via exit policy should not, but direct filtering/tampering via iptables might. How do you figure that? Where's the legal difference? Rejecting via exit policy means that those packets/traffic never reach your relay because the rest of the network won't select your relay as part of the circuit. Rejecting via iptables means those packets reach your machine but never leave. Therefor, you are making a judgement about which traffic is abusive or illegal. In some jurisdictions this has, by some twisted logic, been interpreted to mean that the operator is giving tacit approval for anything that has not been rejected. This is even more clear-cut if you are rejecting specific hosts rather than all traffic on a given set of ports. I find Klingon easier to understand than the perverse logic of lawyers. But there is no arguing with jurisdiction. So, how can isps get away with blocking port 25? Just curious. And/or offering a deliberately corrupted dns? It really is spelled out in the doc that Moritz linked: https://trac.torproject.org/projects/tor/wiki/doc/TorExitGuidelines Okay. In any case it *is* mean to tell the network that you'll relay certain traffic but then in fact not pass it on. Nobody likes a liar :) Apparently, I gave the impression that I am in favor of exit relays rejecting or dropping packets. I am not. Exit policy, or any other tor policy is good. My only concern is abuse and the best way to deal with it. Thank for the feedback. - bartels ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
On 09/04/13 20:46, krishna e bera wrote: > On Tue, 9 Apr 2013 22:59:06 +0600 > Roman Mamedov wrote: > >> On Tue, 9 Apr 2013 12:50:09 -0400 >> krishna e bera wrote: >> >>> So at the risk of being labelled a BadExit (or at best a non-net-neutral >>> exit) i >>> blocked all of ThePirateBay's ip addresses from my exit node for a >>> while. >> I assume you mean firewall-based blocking? You could have simply rejected >> those IPs via ExitPolicy (see "man tor"). That's a clear-cut way to tell the >> network you don't accept connections to those IPs, and no risk of being >> labeled a BadExit. > The latter. I dont know if it complicates routing decisions in the Tor > network to have lots of ip address exceptions at the exits... > > > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays It possibly does slightly but we are talking about the sort of "complication" that a modern CPU can resolve in timescales short enough I doubt it would make that much difference especially if hash tables were used if ipset match can handle hash table lookups containing over 20,000 CIDR ranges while processing 75,000 packets per second I suspect tor could easily be made to efficiently use a similar mechanism, if it doesn't already in order to perform the lookups to compute the answer to "What is the subset of exit nodes allowing exit to IP addr X on port Y?" to come close to the same volume of lookups as the above setup with just over 1k exit nodes the client would need to be building 70+ circuits per second assuming a separate lookup table was used for every node, two factors make me think it's unlikely to go anywhere near this load and if it did that it would only be for brief periods of time as opposed to constant as in the former example, those being that I don't see many exit nodes ever having policies 20k entries long and I can't see that many clients needing to connect to 70+ IP address per second at least not on any continual basis perhaps short bursts (Opening a browser and restoring a few dozen tabs at once or something similar). Bittorrent may be an exception to the above but the performance cost would be at the clients end and for one bittorrent is hardly a realtime protocol a little delay making each connection would not make much difference, two it performs poorly if you insist on running it over tor anyway and thirdly the average consumer desktop system is not exactly lacking spare CPU cycles due to the bursty nature of their workloads they statistically tend to spend the majority of their time idling. Maybe someone can correct me if I missed something glaringly obvious but I certainly don't think anyone should be reluctant to add any exit policy entries they deem necessary at least not for this reason. signature.asc Description: OpenPGP digital signature ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
On Fri, Apr 12, 2013 at 11:00:42AM +0200, bartels wrote: > On 04/12/2013 10:06 AM, Moritz Bartl wrote: > >On 11.04.2013 22:17, bartels wrote: > >>I don't see the legal issue, though. Maybe it is there, but I don't see > >>how rejecting sites via Exit Policy ;) would trigger any one of (1) > >>through (5). > >Yes, rejecting via exit policy should not, but direct > >filtering/tampering via iptables might. > > > > How do you figure that? Where's the legal difference? Rejecting via exit policy means that those packets/traffic never reach your relay because the rest of the network won't select your relay as part of the circuit. Rejecting via iptables means those packets reach your machine but never leave. Therefor, you are making a judgement about which traffic is abusive or illegal. In some jurisdictions this has, by some twisted logic, been interpreted to mean that the operator is giving tacit approval for anything that has not been rejected. This is even more clear-cut if you are rejecting specific hosts rather than all traffic on a given set of ports. It really is spelled out in the doc that Moritz linked: https://trac.torproject.org/projects/tor/wiki/doc/TorExitGuidelines In any case it *is* mean to tell the network that you'll relay certain traffic but then in fact not pass it on. Nobody likes a liar :) -troy ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
On 12.04.2013 11:00, bartels wrote: >>> I don't see the legal issue, though. Maybe it is there, but I don't see >>> how rejecting sites via Exit Policy ;) would trigger any one of (1) >>> through (5). >> Yes, rejecting via exit policy should not, but direct >> filtering/tampering via iptables might. > How do you figure that? Where's the legal difference? (2) the transmission, routing, provision of connections, or storage is carried out through an automatic technical process without selection of the material by the service provider (5) the material is transmitted through the system or network without modification of its content. One could argue that dropping packets is "modification of content", as well as "selection of material". By specifying an exit policy, you don't get to see the data in the first place, so no modification or selection happens. -- Moritz Bartl https://www.torservers.net/ ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
On 04/12/2013 10:06 AM, Moritz Bartl wrote: On 11.04.2013 22:17, bartels wrote: I don't see the legal issue, though. Maybe it is there, but I don't see how rejecting sites via Exit Policy ;) would trigger any one of (1) through (5). Yes, rejecting via exit policy should not, but direct filtering/tampering via iptables might. How do you figure that? Where's the legal difference? - bartels ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] BitTorrent complaint
On 11.04.2013 22:17, bartels wrote: > I don't see the legal issue, though. Maybe it is there, but I don't see > how rejecting sites via Exit Policy ;) would trigger any one of (1) > through (5). Yes, rejecting via exit policy should not, but direct filtering/tampering via iptables might. -- Moritz Bartl https://www.torservers.net/ ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
