Re: [tor-relays] Reminder: don't run transparent proxies at exits
> Date: Mon, 12 Jan 2015 01:04:58 -0500 > From: grarpamp > To: tor-relays@lists.torproject.org > >>> On Fri, Jan 9, 2015 at 10:26 PM, Drake Wilson wrote: >>> eric gisse wrote: >>> Plus the logic starts to get warped when you wonder "So do you BadExit >>> every node that runs on an ISP that caches traffic?" >>> > > As far as http caching, it would be relatively fine IF the cache > truly did good practice, and IF the site truly did good design > for the cache to follow. However those two necessary truths > are often false, whether by AND or XOR context. So to be > true, a cache shouldn't be deployed, but in the interest of > bandwidth they are, more commonly at small end-tier user > access ISPs (including exits) for that purpose. > > I'd suggest best practice is for > - users to use https to bypass > - caches to insert their tagline in http headers so > users can bitch to the owner. > - Tor exits? Well, they're volunteer paid diversity, so which is > more valuable to you? The IF's above, or TCP truth at > potential cost to diversity? > > I prefer TCP truth, but if I was a constrained operator > I'd do my best research into setting up a quality cache. > Provided caching images of ill repute on disk were not > an overriding concern. > I can imagine two strategies to avoid caching images on-disk: 1. Use an in-memory cache 2. Don't cache images While these strategies might not technically/legally offer much more protection (IANAL): 1. If the cache disappears as soon as the machine shuts down, it's much harder to prove the possession of anything illegal. (However, the proxy headers, live imaging, or an insecure/subverted server might give away what was being cached.) 2. And if images aren't being cached, while textual material could still be illegal, it's less likely to be specifically targeted by law enforcement. In my opinion, if you use HTTP over the internet, you are essentially consenting to caching, inspection, or worse. And most people know that by now. And if you use HTTP over Tor, you should be much more aware of this. teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR C3C57B23 349825DE 929A1DEF C3531C25 A32287ED___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
On Fri, Jan 9, 2015 at 10:26 PM, Drake Wilson wrote: > eric gisse wrote: >> Plus the logic starts to get warped when you wonder "So do you BadExit >> every node that runs on an ISP that caches traffic?" >> >> What about ISP's (and openDNS) that NXDOMAIN trap to insert advertising? > > These, I think, are more general points that have not adequately been > resolved anywhere, though I think the vague consensus has been that the > latter merits a BadExit at the moment. Indeed the basic idea of "exits An external NX ad trap is a bit tertiary since the exit is truly representing its view of the net. As far as http caching, it would be relatively fine IF the cache truly did good practice, and IF the site truly did good design for the cache to follow. However those two necessary truths are often false, whether by AND or XOR context. So to be true, a cache shouldn't be deployed, but in the interest of bandwidth they are, more commonly at small end-tier user access ISPs (including exits) for that purpose. I'd suggest best practice is for - users to use https to bypass - caches to insert their tagline in http headers so users can bitch to the owner. - Tor exits? Well, they're volunteer paid diversity, so which is more valuable to you? The IF's above, or TCP truth at potential cost to diversity? I prefer TCP truth, but if I was a constrained operator I'd do my best research into setting up a quality cache. Provided caching images of ill repute on disk were not an overriding concern. Last, the badexit projects should probably try to assess the current state of caching quality in order to further suggest practices for operators. ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
Yes :( 1) Blanket caching on port 80 is mostly fine, but not completely due to squid dropping/erroring on non-http traffic. Not acceptable. 2) I've been unable to find a way to pass non-http traffic in a reliable way. 3) netfilter inspection to determine protocol ends with the layer7 filter project which appears to dead. I might be able to nudge the inspection capabilities up to current 3.x kernels but that's iffy, but either way it isn't currently functional or even functional-adjacent. 4) A *gobsmackingly large* (40k+) amount of CLOSE_WAIT connections to squid gets setup over a period few hours, and I'm at a bit of a loss as to why. I've never had this problem before with previous personal or professional usage of squid... This breaks the fuck out of everything anyway after a few hours. I am unmoved by the philosophical considerations. But there are serious practical ones that I didn't fully think through. So, the puppet manifest for squid gets put away again and the iptables rule is removed until I can solve the above problems. I'm keeping DNS caching though. DNS gets to be pretty slow, as seen from this segment of the bind statistics output: 283152 queries with RTT < 10ms 1724419 queries with RTT 10-100ms 1006967 queries with RTT 100-500ms 60675 queries with RTT 500-800ms 48536 queries with RTT 800-1600ms 2961 queries with RTT > 1600ms I assume everyone's ok with that, right? :p On Sat, Jan 10, 2015 at 4:11 AM, Nusenu wrote: > > >> On Fri, Jan 9, 2015 at 6:29 PM, Nusenu >>> Are you saying you are routing exit traffic through a transparent >>> squid http proxy? >>> >>> If that is the case, please do not interfere with exit traffic in >>> any way. > > eric gisse: >> Why? > > Is your exit breaking non-HTTP protocolls on destination port 80? > > > > > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
> On Fri, Jan 9, 2015 at 6:29 PM, Nusenu >> Are you saying you are routing exit traffic through a transparent >> squid http proxy? >> >> If that is the case, please do not interfere with exit traffic in >> any way. eric gisse: > Why? Is your exit breaking non-HTTP protocolls on destination port 80? ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
Drake Wilson wrote: > But the TCP specification doesn't. Nor is the Tor client signaling > to you that they want an HTTP connection and not a raw TCP connection. > Whether they happen to be passing octets over it that correspond to an > HTTP stream is irrelevant. Or alternatively, let me put the distinction this way: "Opera-tor." "Could you please find me the number for Pythagoras' Pizza Palace?" "Sure, let me get out the copy of the phone book at my desk. It's 555-6283." "Thanks." ... "Opera-tor." "Could you please connect me to 555-6283?" "Sure." *beep beep* "Pythagoras' Pizza Palace? I'd like six Scalene Specials delivered for J. Random User-Agent." "No problem, we'll get that to you in 30 minutes." ... "Opera-tor." "Could you please connect me to 555-6283?" "Sure." *beep beep* "Pythagoras' Pizza Palace? My client just called me from jail! You _do_ remember what 'six Scalene Specials' was supposed to be code for, right?" "Oh, this is actually the operator. I had the right kind of spare, fresh pizza lying around already, so I figured..." "What?!" "Don't worry! I didn't do anything funny to it! It's all good!" "!!!" ---> Drake Wilson ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
On 2015-01-09 19:21, eric gisse wrote: What about ISP's (and openDNS) that NXDOMAIN trap to insert advertising? Just a quick point, OpenDNS doesn't do that anymore. https://www.opendns.com/no-more-ads/ (Others do, and it's still a terrible idea there, but OpenDNS has seen the light and/or found a business model that doesn't involve selling their users as product) -- Dave Warren http://www.hireahit.com/ http://ca.linkedin.com/in/davejwarren ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
eric gisse wrote: > Plus the logic starts to get warped when you wonder "So do you BadExit > every node that runs on an ISP that caches traffic?" > > What about ISP's (and openDNS) that NXDOMAIN trap to insert advertising? These, I think, are more general points that have not adequately been resolved anywhere, though I think the vague consensus has been that the latter merits a BadExit at the moment. Indeed the basic idea of "exits should behave transparently and not manipulate traffic" relies on "there exists a relevant public Internet which is more or less consistent in its behavior as far as not manipulating traffic". To the extent that that becomes less true, the notion of behaving exits becomes more detached from the practical situation. ---> Drake Wilson ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
That's my point. The logic applies to either both or none. Plus the logic starts to get warped when you wonder "So do you BadExit every node that runs on an ISP that caches traffic?" What about ISP's (and openDNS) that NXDOMAIN trap to insert advertising? Regarding 'cached evidence', logs are short term for diagnostic purposes and shredded. Nothing identifiable is logged, which would be annoying to get to due to LUKS encryption of the entire filesystem. The exit node burns through about 20mb/s continuously. Fundamentals of tor design and sheer volume of noise make this a toughie. On Fri, Jan 9, 2015 at 8:35 PM, Zack Weinberg wrote: > On Fri, Jan 9, 2015 at 9:18 PM, cacahuatl wrote: >> If you're caching exit traffic and a very naughty person uses your exit, >> you've potentially cached "evidence" (to be seized). > > That logic applies equally to DNS; indeed, it is why the CMU Tor exit > *doesn't* run a DNS cache. > > (It talks to CMU's DNS servers, which do cache, but for the entire network.) > > (If you can't trust your network provider's DNS resolver, the tradeoff > may be different.) > > zw > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
eric gisse wrote: > This isn't exactly a convincing argument. > > The HTTP specification explicitly supports caching. But the TCP specification doesn't. Nor is the Tor client signaling to you that they want an HTTP connection and not a raw TCP connection. Whether they happen to be passing octets over it that correspond to an HTTP stream is irrelevant. ---> Drake Wilson ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
This isn't exactly a convincing argument. The HTTP specification explicitly supports caching. On a protocol level, this is quite acceptable and standard. The method I am using is precisely what ISP's do in scenarios where they want to maximize their bandwidth. On Fri, Jan 9, 2015 at 8:12 PM, Drake Wilson wrote: > eric gisse wrote: >> Why? People say 'DO NOT MESS WITH TRAFFIC' but in the same breath they >> say 'BUT USE A CACHING DNS RESOLVER'. > > Because the interface level at which exit traffic proper occurs is TCP, > and the interface contract for the client is that the TCP stream will be > direct to the intended destination. The interface level at which > Tor-traversing DNS requests occur is DNS, and the interface contract for > the client is that the DNS request will be resolved in some way that > reflects the consensus public DNS on the Internet. Using a DNS cache is > consistent with being expected to terminate DNS. Using an HTTP cache is > not consistent with being expected to terminate TCP. Reblocking at the > TCP level presumably happens, for instance, and is not considered "messing > with traffic" because it's not specified that Tor passes arbitrary IP > packets, only TCP (and I'm not sure it even requires _full_ TCP other than > the bidirectional octet streams; I forget whether the urgent marker is > passed through, for instance). > > So it's not inconsistent to hear those for exit operators WRT Tor's design. > If you think the design is flawed, that's a separate matter. > >---> Drake Wilson > > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
On Fri, Jan 9, 2015 at 9:18 PM, cacahuatl wrote: > If you're caching exit traffic and a very naughty person uses your exit, > you've potentially cached "evidence" (to be seized). That logic applies equally to DNS; indeed, it is why the CMU Tor exit *doesn't* run a DNS cache. (It talks to CMU's DNS servers, which do cache, but for the entire network.) (If you can't trust your network provider's DNS resolver, the tradeoff may be different.) zw ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
eric gisse wrote: > Why? People say 'DO NOT MESS WITH TRAFFIC' but in the same breath they > say 'BUT USE A CACHING DNS RESOLVER'. Because the interface level at which exit traffic proper occurs is TCP, and the interface contract for the client is that the TCP stream will be direct to the intended destination. The interface level at which Tor-traversing DNS requests occur is DNS, and the interface contract for the client is that the DNS request will be resolved in some way that reflects the consensus public DNS on the Internet. Using a DNS cache is consistent with being expected to terminate DNS. Using an HTTP cache is not consistent with being expected to terminate TCP. Reblocking at the TCP level presumably happens, for instance, and is not considered "messing with traffic" because it's not specified that Tor passes arbitrary IP packets, only TCP (and I'm not sure it even requires _full_ TCP other than the bidirectional octet streams; I forget whether the urgent marker is passed through, for instance). So it's not inconsistent to hear those for exit operators WRT Tor's design. If you think the design is flawed, that's a separate matter. ---> Drake Wilson ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
If you're caching exit traffic and a very naughty person uses your exit, you've potentially cached "evidence" (to be seized). Also likely has interesting legal questions, eg. 'if you're actually storing the content, then do you "possess" it?' ymmv with jurisdiction and ianal. eric gisse: > Why? People say 'DO NOT MESS WITH TRAFFIC' but in the same breath they > say 'BUT USE A CACHING DNS RESOLVER'. > > This is an internally inconsistent attitude, and is not consistent > with how large scale operations function either. Tools like varnish, > CDN's, memcache, dns caching, etc are all common - and best - > practices. > > If there's a practical consideration I am missing, that's different. > > > > On Fri, Jan 9, 2015 at 6:29 PM, Nusenu > wrote: >> hi, >> >> eric gisse: >>> I even threw on a squid proxy on regular http and that's caching >>> something like 5-10% of all requests and overall http bandwidth. >> >> Are you saying you are routing exit traffic through a transparent squid >> http proxy? >> >> If that is the case, please do not interfere with exit traffic in any way. >> >> ___ >> tor-relays mailing list >> tor-relays@lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays > ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
Why? People say 'DO NOT MESS WITH TRAFFIC' but in the same breath they say 'BUT USE A CACHING DNS RESOLVER'. This is an internally inconsistent attitude, and is not consistent with how large scale operations function either. Tools like varnish, CDN's, memcache, dns caching, etc are all common - and best - practices. If there's a practical consideration I am missing, that's different. On Fri, Jan 9, 2015 at 6:29 PM, Nusenu wrote: > hi, > > eric gisse: >> I even threw on a squid proxy on regular http and that's caching >> something like 5-10% of all requests and overall http bandwidth. > > Are you saying you are routing exit traffic through a transparent squid > http proxy? > > If that is the case, please do not interfere with exit traffic in any way. > > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Reminder: don't run transparent proxies at exits
hi, eric gisse: > I even threw on a squid proxy on regular http and that's caching > something like 5-10% of all requests and overall http bandwidth. Are you saying you are routing exit traffic through a transparent squid http proxy? If that is the case, please do not interfere with exit traffic in any way. ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
