Re: [tor-relays] DDOS alerts from my provider
"Rafo \(r4fo.com\) via tor-relays" wrote: > More specifically, I?m running a middle relay on Debian 12 > On Tue, 09 Jul 2024 13:46:51 +0200 > li...@for-privacy.net wrote On Montag, 8. Juli > 2024 19:34:51 CEST Rafo (r4fo.com) via tor-relays wrote: > But this week I?ve > received 2 DDoS alerts from my provider > (Netcup), both are ~3 gigabits. > They seem to be coming from other Tor > relays.I?m running an Invidious like > instance on my server (which uses > around 600 megabits) but I have a 2.5 > gigabit port. So I configured my Tor > relay to use 300-400 megabits.I?m not > sure where that 3 gigabit of data > comes from.I have lowered my advertised > bandwidth to 100 megabits, would > that be enough to prevent these kind of > issues?Kind regards,Rafo Reducing the advertised bandwidth does not help. > ;-) In general, one tor instance will rarely reach 100 megabits. There is > little you can do on the server against targeted DDoS. But you can stop IPs > with a lot of connections to your tor daemon us ing dynamic exit police? or dyn. IP/nftable rules?. For targeted help, you should specify the type of relay you have and your OS. https://gitlab.torproject.org/tpo/community/support/-/issues/40093 ?https://github.com/artikel10/surgeprotector ?https://forum.torproject.org/t/is-tor-network-resistant-to-tcp-syn-flood-dos-attacks-from-outside-of-tor/12690/4 -- ?_? Ciao Marco! Debian GNU/Linux It's free software and it gives you freedom!___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays > Depending upon the features of the packet filter(s) available to you on your operating system, there may be one or more options that can help defend your relay. For example, all of the main FOSS *BSD systems today have some version of the pf packet filter subsystem available as well as others. One option available in pf for TCP is synproxy. This excerpt from the pf.conf(5) man page in FreeBSD tells how it works. --- SYN PROXY By default, pf(4) passes packets that are part of a tcp(4) handshake between the endpoints. The synproxy state option can be used to cause pf(4) itself to complete the handshake with the active endpoint, perform a handshake with the passive endpoint, and then forward packets between the endpoints. No packets are sent to the passive endpoint before the active endpoint has completed the handshake, hence so-called SYN floods with spoofed source addresses will not reach the passive endpoint, as the sender can't complete the handshake. The proxy is transparent to both endpoints, they each see a single connection from/to the other endpoint. pf(4) chooses random initial sequence numbers for both handshakes. Once the handshakes are completed, the sequence number modulators (see previous section) are used to translate further packets of the connection. synproxy state includes modulate state. Rules with synproxy will not work if pf(4) operates on a bridge(4). Example: pass in proto tcp from any to any port www synproxy state --- My understanding is that LINUX systems do not have pf, but rather have a less flexible filter called iptables. Whether iptables or any other packet filter that may be available on LINUX systems has synproxy or a similar feature I do not know, but I'm posting this to make novice *BSD users who run tor relays aware of it. IMHO, synproxy or its equivalents in other filters (if they exist) should always be applied to filter rules for at least the ports that tor listens on and are exposed to the outside world (e.g., ORPort and DirPort). Of course, although synproxy helps defend a tor relay (or a web server or data base or whatever), it doesn't stop what arrives at your ISP from outside. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army."
Re: [tor-relays] preventing DDoS is more than just network filtering
Chris wrote: > > > > > > > > On 11/10/2022 2:38 AM, Scott Bennett > wrote: > >cite="mid:202211100738.2aa7cw7d026...@sdf.org"> > Toralf F?rster class="moz-txt-link-rfc2396E" > href="mailto:toralf.foers...@gmx.de";><toralf.foers...@gmx.de> wrote: > > > > On 11/8/22 10:57, Chris wrote: > > > The main reason is that a simple > SYN flood can quickly fill up your > conntrack table and then legitimate packets are quietly dropped and you > won't see any problems thinking everything is perfect with your server > unless you dig into your system logs. > > > > Hhm, my system log doesn't show any problems, maybe due to (or > regardless of?): > CONFIG_SYN_COOKIES=y > ? I surmise that the above is a LINUXism that is approximately equivalent to a pf rule using synproxy. > > > > On FreeBSD 12.3 I use pf and have gone back to using synproxy on the > "pass in" statements for the ORPort and DirPort, but I doubt it has actually > made any difference I should clarify my statement above by stating that the SYN packets still have to be received from my ISP before the rule can be applied, so yes, a SYN flood attack can still tie up my Internet connection, but that does not appear to be the kind of attacks that my relay was experiencing. Specifying synproxy on the "pass in" rules for tor means that the kernel simply drops any pending connection that fails to complete the SYN-SYNACK handshake within a short time instead of passing it on to tor to deal with; IOW, no incoming connections are passed to tor unless they complete that handshake first. The second reason I made that statement was that all the attacks I have seen in recent months have tied up my inbound (and sometimes outbound) data capacity for some time, and the next appearance of a set of heartbeat messages from tor show an increase in the INTRODUCE2 rejections of 2,000 to 3000 or occasionally more. I suspect the "occasionally more" cases occur when two of the bot attacks hit my relay at the same or overlapping times. All of the above was true before I began using synproxy again and appears to be the case still. If you have seen SYN flood attacks, then that is grounds enough for me to continue to leave it in the rules for tor indefinitely. The cost to the system for using synproxy is too small to be detected, but the potential for sparing cost to tor appears to be significant. > > The quote about SYN Flood is > actually from my post which went only to toralf and wasn't > displayed on the group. My bad. To explain further, I didn't > say the current attack includes SYN floods, what I meant was Ah. I see. > whenever we have some conntrack rules in our iptables, it's > prudent to have some rate limiting rules before it, because if > the attacker knows we rely on conntrack and intends to do some Not being a LINUX user, I am unaware of what "conntrack" does. pf has a "keep" flag that tells it to keep state for each connection, but many years ago pf was changed to keep state anyway, whether one tells it to or not, so nowadays it is effectively a comment. I don't know of any method by which one can tell pf *not* to keep state. > damage, the attacker can easily flood our conntrack table with > SYN flood and then we start dropping legitimate packets > without notice. However you're correct, currently there are no > SYN floods. face="Arial">? Understood. Thank you for the clarification. > > > >cite="mid:202211100738.2aa7cw7d026...@sdf.org"> > because the only attacks I've seen > so far were coming > via other relays and triggered tor's rejections of INTRODUCE2 cells by the > thousands. Instead, what has been very effective has been to increase the > NumCPUs count drastically. > > You're correct yet again. The > number of CPUs make a huge difference. Tor automatically > detects up to 16 CPUs if you have them. Anything above that, > Tor can't see. I've never tried adding it to my torrc though, > it might see more if you tell it to look for them. It only looks for the number of CPU threads actually available if you don't specify a value for NumCPUs. You can put any natural number there that you want, unless there's some upper limit I don't know about, e.g., 255. > On my relays which
Re: [tor-relays] preventing DDoS is more than just network filtering
Toralf F?rster wrote: > On 11/8/22 10:57, Chris wrote: > > The main reason is that a simple SYN flood can quickly fill up your > > conntrack table and then legitimate packets are quietly dropped and you > > won't see any problems thinking everything is perfect with your server > > unless you dig into your system logs. > > Hhm, my system log doesn't show any problems, maybe due to (or > regardless of?): > CONFIG_SYN_COOKIES=y > ? On FreeBSD 12.3 I use pf and have gone back to using synproxy on the "pass in" statements for the ORPort and DirPort, but I doubt it has actually made any difference because the only attacks I've seen so far were coming via other relays and triggered tor's rejections of INTRODUCE2 cells by the thousands. Instead, what has been very effective has been to increase the NumCPUs count drastically. On a non-hyperthreaded quad-core CPU I now have it set as "NumCPUs 20". Each worker thread uses almost no CPU time, but haveing enough of them waiting to grab an onionskin off the queue instantly seems to stop all messages about cells, onionskins, or connections being dropped. During an attack I often saw all workers in top(1) screen updates with "NumCPUs 16", so I increased to 20 for the next restart, but I hadn't gotten any of the aforementioned error/warn messages at 16. Unfortunately, I have yet to see what happens at 20 because before the next restart Comcast made a change that blocks me from running a relay. :-( I intend to find out very soon whether I can afford to switch to their business network right away, so that I might resume running my relay or will have to wait until things happen next summer that should free up some of my limited income first. > Nevertheless, I updated the Readme to explain my point of view [1] [2]. > > [1] https://github.com/toralf/torutils#block-ddos-traffic > [2] https://github.com/toralf/torutils#rule-set Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] actively maintaining 'recommended versions' list (was: Tor 0.4.6.x is unsupported, please upgrade)
Roger Dingledine wrote: > On Sat, Oct 08, 2022 at 02:54:41PM +0200, nusenu wrote: > > dir auths still recommend running tor 0.4.6.x versions today, so relay > > operators never got any indicator > > on RS or in their logs - this is a missed opportunity. > > Good catch -- I've just started the process of getting the directory > authorities to un-recommend the 0.4.6 versions. > > And you're right that it is a missed opportunity for those fingerprints > that started the process of getting bumped out a few days ago. > > But (a) new relays that show up on 0.4.6, i.e. that don't have > fingerprints that are in the reject list, will see the warning about > versions, > > and (b) I'm going to hold off on applying the big set of 0.4.6 reject > lines on moria1, so now moria1 should be issuing a warning about versions, > rather than a warning about a rejected fingerprint, to affected relays. It seems worth pointing out that one directory authority was reporting some hours ago that it was running 0.4.7.9, which also is not in the lists of client-versions or server-versions announced in the hourly consensus files. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] a characteristic of recent attacks?
Scott Bennett wrote: > The vast majority of recent surges of unusual activity on my humble > relay, which I take as likely being attacks, result in large changes in > the hour-to-hour statistic shown below in these lines extracted from the > hourly heartbeat message groups. > > Aug 31 19:02:28.549 [notice] Heartbeat: DoS mitigation since startup: 15 > circuits killed with too many cells, 0 circuits rejected, 0 marked addresses, > 14 same address concurrent connections rejected, 0 connections rejected, 5 > single hop clients refused, 1706742 INTRODUCE2 rejected. > Aug 31 20:02:28.546 [notice] Heartbeat: DoS mitigation since startup: 15 > circuits killed with too many cells, 0 circuits rejected, 0 marked addresses, > 14 same address concurrent connections rejected, 0 connections rejected, 5 > single hop clients refused, 1706742 INTRODUCE2 rejected. > Aug 31 21:02:28.549 [notice] Heartbeat: DoS mitigation since startup: 15 > circuits killed with too many cells, 0 circuits rejected, 0 marked addresses, > 14 same address concurrent connections rejected, 0 connections rejected, 5 > single hop clients refused, 1706742 INTRODUCE2 rejected. > Aug 31 22:02:28.544 [notice] Heartbeat: DoS mitigation since startup: 15 > circuits killed with too many cells, 0 circuits rejected, 0 marked addresses, > 14 same address concurrent connections rejected, 0 connections rejected, 5 > single hop clients refused, 1770970 INTRODUCE2 rejected. > Aug 31 23:02:28.556 [notice] Heartbeat: DoS mitigation since startup: 16 > circuits killed with too many cells, 0 circuits rejected, 0 marked addresses, > 14 same address concurrent connections rejected, 0 connections rejected, 5 > single hop clients refused, 2013266 INTRODUCE2 rejected. > > The most recent such surge appears to have ended less than an hour ago. > Note that there was no change in the count of "INTRODUCE2 rejected" for > many hours leading up to the onset of unusual activity, though I've only > shown three prior hours' worth. Then there is an increase of 63227 in ^ > this count during the first hour and another 41296 during the second ^ Obviously, the above figures should have been 64228 and 242296. Either way, they seem like an awful lot of bungled hidden service access attempts to occur within an hour, so it's either a bug in hidden services (which would not be unheard of) or it's a deliberate attack. > hour. Often during these periods the input appears to be maxed out, and > sometimes the output rate is still higher by several hundred KB/s. > My question is, do other relay operators whose relays are being > attacked see the same phenomenon? In addition, if someone knows of an > effective way to turn such things aside at less cost than be simply > leaving them to tor to deal with, I'd love to know about it, too, though > I suspect there may be no such method. > Thanks in advance for any relevant information! Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] a characteristic of recent attacks?
The vast majority of recent surges of unusual activity on my humble
relay, which I take as likely being attacks, result in large changes in
the hour-to-hour statistic shown below in these lines extracted from the
hourly heartbeat message groups.
Aug 31 19:02:28.549 [notice] Heartbeat: DoS mitigation since startup: 15
circuits killed with too many cells, 0 circuits rejected, 0 marked addresses,
14 same address concurrent connections rejected, 0 connections rejected, 5
single hop clients refused, 1706742 INTRODUCE2 rejected.
Aug 31 20:02:28.546 [notice] Heartbeat: DoS mitigation since startup: 15
circuits killed with too many cells, 0 circuits rejected, 0 marked addresses,
14 same address concurrent connections rejected, 0 connections rejected, 5
single hop clients refused, 1706742 INTRODUCE2 rejected.
Aug 31 21:02:28.549 [notice] Heartbeat: DoS mitigation since startup: 15
circuits killed with too many cells, 0 circuits rejected, 0 marked addresses,
14 same address concurrent connections rejected, 0 connections rejected, 5
single hop clients refused, 1706742 INTRODUCE2 rejected.
Aug 31 22:02:28.544 [notice] Heartbeat: DoS mitigation since startup: 15
circuits killed with too many cells, 0 circuits rejected, 0 marked addresses,
14 same address concurrent connections rejected, 0 connections rejected, 5
single hop clients refused, 1770970 INTRODUCE2 rejected.
Aug 31 23:02:28.556 [notice] Heartbeat: DoS mitigation since startup: 16
circuits killed with too many cells, 0 circuits rejected, 0 marked addresses,
14 same address concurrent connections rejected, 0 connections rejected, 5
single hop clients refused, 2013266 INTRODUCE2 rejected.
The most recent such surge appears to have ended less than an hour ago.
Note that there was no change in the count of "INTRODUCE2 rejected" for
many hours leading up to the onset of unusual activity, though I've only
shown three prior hours' worth. Then there is an increase of 63227 in
this count during the first hour and another 41296 during the second
hour. Often during these periods the input appears to be maxed out, and
sometimes the output rate is still higher by several hundred KB/s.
My question is, do other relay operators whose relays are being
attacked see the same phenomenon? In addition, if someone knows of an
effective way to turn such things aside at less cost than be simply
leaving them to tor to deal with, I'd love to know about it, too, though
I suspect there may be no such method.
Thanks in advance for any relevant information!
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Flags gone after restart
andrew reid wrote: > Hey there, I am running a tor relay off an old samsung phone. When I first > started the relay, my observed bandwidth was around 4.5MB/s. It was running > for around 7 days and had the stable and HSdir flag. After a few problems > with IP6 and being overloaded (thought this was a problem on my end, not a > DDoS attack) I restarted the phone and edited the config file to take away > IPv6. It has now been running for 7 days again but my observed bandwidth is > only 1.6MB/s now and I still haven't gotten the stable and HSdir flag back. > How long would this take or is it just a problem on my end ? and for the > bandwidth, my connection is still the same and hasn't changed. Is this just > something that takes time to come back like the flags. This is my first > relay so I would like to understand a bit more. Thank you Stable, AFAIK, still depends upon something like the average uptime of the current instance and the previous instance relative to the corresponding averages among all other relays. Assignment of the Stable flag is made to relays above a certain percentile rank of those averages. Fast used to depend upon the maximum throughput speed allowed by the Bandwidth* and RelayBandwith* entries in the torrc file. More than some minimum bandwidth was required for assignment of the flag to a relay. HSDir used to depend upon its torrc option not being set to 0. That option was removed some time ago and is now apparently solely under the control of the Authority relays. *HOWEVER*, both a randomization factor appears to have been added three or four years ago to the Authority relays' algorithm used to decide whether to award Fast and to award HSDir. Now either flag comes and goes like birds landing on tree limbs and later departing, often for no reason obvious to humans. My relay has many times been up for one or more months. During those times Fast and HSDir have been repeatedly assigned to it and lifted from it and perhaps reassigned to it, often only a few hours apart. The only consistencies I have seen are that 1) the first time a tor relay is assigned the HSDir flag is after at least 96 hours of uptime in the current instance, though it may not happen until much later than 96 hours and 2) HSDir does not appear ever to be assigned unless Fast is also assigned. AFAIK, the tor project has never offered an explanation for the addition/intrusion of this randomization factor. Frankly, I think it is a destabilizing factor to the tor relay network and doubly so for hidden services activities, and it must add to overall tor traffic to have to restock the hidden service directory servers so often. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Overload (dropped ntor) due to DDoS??
li...@for-privacy.net wrote: > On Friday, August 5, 2022 1:11:27 AM CEST s7r wrote: > > Richard Menedetter wrote: > > > > I have a non exit relay running on a root server (4 AMD Epyc cores, 8 GB > > > RAM, 2.5 GBit/s Ethernet) I have limited tor to numcpus 2, > Why? Do you have other services on the server? Otherwise, omit num CPU. Let > the tor daemon use all CPU's for crypto stuff. The OP has a EPYC chip with four cores and presumably eight CPU threads, so he should specify at least "NumCPUs 8" as a bare minimum. "NumCPUs 16", "NumCPUs 24", or "NumCPUs 32" would be better. These onion skin decryption threads use hardly any actual CPU time throughout the course of a busy day, but having many extras available for work can mean drastically lower times for a decryption task to remain in the work queue and therefore a much greater chance of getting through a high connection count period without dropping any due to the high load. I've never seen an overload message like the one the OP showed, and my system is ridiculously slower than the OP's EPYC. When I last started tor 41 days ago, it was started with "NumCPUs 12", up from 10 in the previous iteration. I've since changed torrc to "NumCPUs 16" because of the connection and circuit counts during the recent troubles, so the next time I start it it will be with four onion skin decryptors per core. This is on an ancient QX9650 (Core 2 Extreme quad-core, so only one CPU thread per core) bumped up to 3.3 GHz. I hope that will be sufficient to get it through such attacks by making my Internet connection speed, rather than my tor config, be the weak spot. During the recent events I've seen tor's main thread take up 15% to 16% of CPU time on one core, while each of the worker threads consumed far less than 1% apiece. FWIW, mprime runs nice'd to 20 on all four cores, as well. Also, I normally renice tor to -6 about 15 seconds after starting it, to keep it mostly prioritized higher than anything I might be doing interactively and also other stuff running all the time (e.g., top(1), privoxy(1)). > > > > relaybandwidthburst 15 MB, hardwareaccel 1, maxadvertisedbandwidth 10 MB, > > > maxmeminqueues 3GB > > > > Thanks for running a relay! > > > > didn't you also use RelayBandwidthRate along with RelayBandwidthBurst ? > > > > > > > > > > Usually it takes less than 1 CPU core, and like 1 GB of RAM. During the couple of hours of the attack the other day my tor instance never seemed to use more than ~2700 MB. > > > But recently my relay is foten shown as obverloaded. > > > I have these LOG entries: > > > Tor[814]: General overload -> Ntor dropped (290376) fraction 5.3451% is > > > above threshold of 0.5000% > > > > You are not the only one, it's an ongoing DoS attack on the network, > > targeting onion services. > > > > > > > > > > Is this due to DDoS attacks or a misconfigration on my side? > > > > > > Besides the question above about RelayBandwidthRate I don't see anything > > wrong. > > > > > > > Is there something that I can do to aleviate this issue? > > Again, I suggest increasing NumCPUs by several times to try to avoid the overload message. > > > > Nope, there is nothing you can do, unfortunately. Tor has some defenses > > against DoS and will blacklist / mark the abusing addresses, etc. as > > much as it can. But as you know DoS is a never ending battle, usually > > won by having "larger pipe", and it's something hard to tickle in an > > environment where anonymity is the grounding law. > > > > What you can do is maintain your relay up and running in good shape with > > the latest version of Tor until this "attack" gets through. As I said, I > > guess most of relays are getting this at present times. The DoS "attack" > > is not targeted at your relay, what you are seeing is just a side effect > > of someone creating large amounts of circuits (heavy usage of Tor) which > > is reflected network-wide anyways. > Sometimes 100.000-1.000.000 connections from one IP! > I block the worst with 2 nftables egress rules. > > toralf has developed some smarter ddos scripts: > https://github.com/toralf/torutils Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Loss of Guard and HS Dir flags
Eddie wrote: > Necro alert !! > > Again, the same server has exhibited the same thing.? It has lost the > Guard, HS Dir, and Long-Lived (Stable) flags, despite being up for > almost 5 months with no reduction in Bandwidth. > > Cheers. > > > On 10/18/2021 9:42 PM, Eddie wrote: > > On 10/18/2021 1:03 AM, Georg Koppen wrote: > >> Eddie: > >>> On 10/13/2021 11:29 PM, Eddie wrote: > >>>> I currently run 3 relays, across different servers and today I noticed > >>>> that one has now lost it's Guard and HS Dir flags.? What's surprising > >>>> is that this particular relay has the highest Bandwidth and Consensus > >>>> Weight of all 3 and has not been restarted for over a month. > >>>> > >>>> The stats for all 3 can be found with:? OhNoAnotherRelay > >>>> > >>>> I know that just running the relay is the important part and caring > >>>> about what flags it has is a side issue, but I'm just interested in > >>>> seeing why this has happened. > >>>> > >>>> Cheers. > >>> Now the relay has lost the Long-Lived Circuits flag despite showing an > >>> up-time of 34 days. > >> You mean the Stable flag? Seems to be back now at least. I've filed a > >> ticket[1], though, to look a bit closer at what is going on but am not > >> sure how fast we get to that (help is welcomed!). > > And the Stable flag has gone again. > >>> There's something strange going on here. > >> Maybe. :) Yes, there definitely is, as I have been pointing out repeatedly on this list over the last several years. The Authority servers *used* to deal out these flags rationally and consistently before that time. Since then they have acted capriciously w.r.t. to those flags. BTW, you omitted the Fast flag, which has also been subject to such mishandling. > >> > >> Georg > >> > >> [1] https://gitlab.torproject.org/tpo/network-health/team/-/issues/128 > >> Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Sanity check on NumCPUs
Roger Dingledine wrote: > On Wed, May 25, 2022 at 07:31:41PM -0500, Thoughts wrote: > > For a non-exit relay, is "NumCPUs 2" still the recommended maximum??? > > Running on a quad core and recently saw a message indicating I had > > insufficient CPU power to support the desired number of connections... > > I would suggest leaving NumCPUs alone, and let Tor use as many cores as > you have. Setting it to 2, when you have more cores than that, will > limit how much Tor can scale. Does tor default to using the number of cores? Or the number of logical CPUs? > > And that message you saw, which I assume was "Your computer is too slow > to handle this many circuit creation requests!", is exactly about the > number of cores that are allocated to handling incoming create cells. > > There are three main pieces to Tor's crypto, and two of them (TLS and AES) > are bottlenecked on the main thread, but one of them (circuit handshakes) > *does* parallelize pretty well. > > So if your Tor is complaining about not being able to keep up with > circuit creation requests... consider removing the artificial limitation > of setting NumCPUs lower than it wants to be. :) I am running tor at present on an ancient machine with a QX9650 CPU, so the CPU and LCPU counts are equal, both 4. If I let it default or set NumCPUs 4, then at some point tor complains that it has to drop circuits due to timeouts. The chip is ancient, but quite speedy for its time, and I've bumped the multiplier from 9 to 10, so it runs at 3.333 GHz. I have found through gradual increments of NumCPUs that 10 are not quite enough to cover all situations that arise over a lengthy uptime, so several weeks ago I increased it once again, this time to 12. Thus far tor seems happy with that number. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Loss of Stable flag
Arlen Yaroslav via tor-relays wrote: > My relay Deepsky (09A70E396DE93F54D4541BBB0EC8E2B23761F34F) has not been > receiving the 'Stable' flag from the DA consensus. Up until recently (the > past month or so) it had been assigned the full complement of flags. Nothing > has been changed on the server and I am not aware of any issues. Would any of > the directory authority operators be able to shed light on why some of the > DAs are not assigning it the flag? > The authority relays have exhibited very unstable behavior for, I think, nearly two years now, as I have pointed out several times on this list. For about three months now, there have been signs of improvement, but the situation is still not terribly stable or predictable. The two flags that appear to me to have been most affected are HSDir and Stable, but Fast has also come and gone occasionally since the instability began. I don't run a Guard relay, so I don't know about that flag's stability, but I would imagine that it would be affected by instability in the Stable and Fast flags. All three flags can come and go, seemingly at random, without regard to uptime of a relay or stability of its own characteristics and network connections. Until a large majority of the authority relays are running versions of tor in which the problematic code has either been fixed or backed out, I expect the problems to continue. AFAIK, the tor project staff has only recently decided that there might be some sort of a problem that they should look into, so the improvements I first began seeing a very few months ago have likely been accidental. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] [Looking for feedback] An easier way to declare families
Nick Mathewson wrote: > On Sat, Nov 6, 2021 at 10:36 AM Nick Mathewson wrote: > > > Hello, relay operators! > > > > I'm hoping to get some feedback from relay operators, particularly > > those who use the MyFamily option, about the best way to deploy > > proposal 321. You can read the proposal at: > > > > > > https://gitlab.torproject.org/tpo/core/torspec/-/blob/master/proposals/321-happy-families.md > > > > The main idea of this proposal is to provide an easier way for > > relays to declare that they are in the same family. With proposal > > 321, each relay in the same family uses a cryptographic certificate > > to prove that it is in the same family as the others. > > > > One small clarification, since I know not everybody will read the proposal: > this proposal is not only for making families easier to declare: it will > also make them far more efficient to implement by saving significant > amounts of bandwidth for client directory downloads. So even if everybody > thinks that the current MyFamily configuration mechanism is perfectly easy > and hassle-free (ha), we'd still probably want to implement something like > this in order to make the network more efficient. Because the obvious incentive for cheaters is in the direction of trying to get clients' route selectors to choose routes through more than a single relay operated by a given cheater, rather than the other way around, this looks to me like a "solution" in search of a problem unless I am missing some special scenario. Can someone enlighten me as to why this mechanism would be needed? I.e., when and why would a cheater want his relay(s) to be included in *any* families at all? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Relay consensus weight drop
Scott Bennett wrote: > Georg Koppen top-posted: > > > Georg Koppen: > > > mikefloyd2: > > >> Following further decline of my server's consensus weight, I looked > > >> around a bit. > > >> I found some more servers that show similar behavior in the graphs shown > > >> on the Metrics website: > > >> * > > >> https://metrics.torproject.org/rs.html#details/01729F10A81DDD8A92D770B2133082EB56C75E26 > > >> * > > >> https://metrics.torproject.org/rs.html#details/010B7728454411F485CE29D4C79A14534151C2C4 > > >> * > > >> https://metrics.torproject.org/rs.html#details/1E5136DDC52FAE1219208F0A6BADB0BA62587EE6 > > >> * > > >> https://metrics.torproject.org/rs.html#details/899D1D12BBF6ED445048CB302C0F14B6D15ADCA2 > > >> * > > >> https://metrics.torproject.org/rs.html#details/00DCAEAE3E54C32809E7F7CC4BF2A6FC68FC552F > > > > > > Alright, that's useful, thanks! I've added those links to our ticket[1] > > > in case we need further data in order to track the problem down and fix > > > it. > > > > > > As a general note: I asked the operator of bastet to switch back to > > > Torflow as soon as possible, so that Torflow-backed bandwidth > > > authorities have a majority again in our network. That should mitigate > > > the immediate problem and get all your relays back to appropriate > > > consensus weights (unless there are sbws-unrelated bugs preventing > > > that). Additionally, it gives us time to track down this bug and > > > properly fix it. > > > > We think we found the problem and fixed it. Again, thanks to everyone > > who reported this problem. After some testing we decided it would be > > save if bastet's bandwidth authority switched over to sbws again. This > > happened with the 1600 UTC vote. Please keep your eyes open in case > > there are more critical bugs lurking and if you find any issue do not > > hesitate to report it. > > > I also note that once my relay's weighting returned to its previously > normal range, > the HSDir flag returned intermittently for a day or two for the first time > since late > last year and has been present continuously (AFAIK) ever since. That allows > my relay to > provide a bit more utility to the network once again. > And now I see that I wrote too soon. A new consensus arrived, and in it my relay is once again missing the HSDir flag. So we're back to the SOS again. Sigh. Scott Bennett ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Relay consensus weight drop
Georg Koppen top-posted: > Georg Koppen: > > mikefloyd2: > >> Following further decline of my server's consensus weight, I looked around > >> a bit. > >> I found some more servers that show similar behavior in the graphs shown > >> on the Metrics website: > >> * > >> https://metrics.torproject.org/rs.html#details/01729F10A81DDD8A92D770B2133082EB56C75E26 > >> * > >> https://metrics.torproject.org/rs.html#details/010B7728454411F485CE29D4C79A14534151C2C4 > >> * > >> https://metrics.torproject.org/rs.html#details/1E5136DDC52FAE1219208F0A6BADB0BA62587EE6 > >> * > >> https://metrics.torproject.org/rs.html#details/899D1D12BBF6ED445048CB302C0F14B6D15ADCA2 > >> * > >> https://metrics.torproject.org/rs.html#details/00DCAEAE3E54C32809E7F7CC4BF2A6FC68FC552F > > > > Alright, that's useful, thanks! I've added those links to our ticket[1] > > in case we need further data in order to track the problem down and fix it. > > > > As a general note: I asked the operator of bastet to switch back to > > Torflow as soon as possible, so that Torflow-backed bandwidth > > authorities have a majority again in our network. That should mitigate > > the immediate problem and get all your relays back to appropriate > > consensus weights (unless there are sbws-unrelated bugs preventing > > that). Additionally, it gives us time to track down this bug and > > properly fix it. > > We think we found the problem and fixed it. Again, thanks to everyone > who reported this problem. After some testing we decided it would be > save if bastet's bandwidth authority switched over to sbws again. This > happened with the 1600 UTC vote. Please keep your eyes open in case > there are more critical bugs lurking and if you find any issue do not > hesitate to report it. > I also note that once my relay's weighting returned to its previously normal range, the HSDir flag returned intermittently for a day or two for the first time since late last year and has been present continuously (AFAIK) ever since. That allows my relay to provide a bit more utility to the network once again. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Relay consensus weight drop
Claus Jacobsen wrote:
> Mike is not the only one - I've been seeing the exact same thing.
> My relay is
> https://metrics.torproject.org/rs.html#details/A2F5DF163132CF8FFC1F6343135D3397CA85CF89
>
I posted an alarm about this on 3 June, but there were no followups.
https://lists.torproject.org/pipermail/tor-relays/2021-June/019679.html
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] current HSDir flag requirements
I wrote:
> Logforme wrote:
>
> >
> > On 2021-05-26 08:18:32, "Scott Bennett" wrote:
> > >I interpret that as meaning that one
> > >or more criteria being used by one or more authorities has changed,
> > What I have noticed on my relay is that the "Consensus Weight" is
> > fluctuating.
> > CW is too complicated for my tiny brain but I believe the measurements
> > from the Bandwidth Authorities is involved. The BWAuths are spread
> > around the world and depending on current internet conditions they get
> > different speed values to your relay. But can it cause massive swings in
> > CW?
>
> Yes. My relay is on a residential service connection and its "bandwidth"
> rating from the Authority relays typically oscillates between ~30 and ~120, so
> proportionally the swings are often fairly wild.
A new, more extreme problem has emerged in the last two or three days. My
relay's "Bandwidth" in the consensus documents dropped suddenly to 14, then 4
for
about 18 hours, and finally to 3, where it has remained ever since. This is
still
the first time I can remember it showing up in the consensus with a value lower
than "Bandwidth=20 Unmeasured=1".
On Wednesday I took a quick look at the then current consensus file and
found
the following, regarding single-digit "Bandwidth" values.
Script started on Wed Jun 2 10:36:48 2021
hellas# grep '^w Bandwidth=' /var/db/tor/cached-consensus | wc -l
6777
hellas# grep '^w Bandwidth=.$' /var/db/tor/cached-consensus | wc -l
610
hellas# grep '^w Bandwidth=1$' /var/db/tor/cached-consensus | wc -l
359
hellas# ^1^2
grep '^w Bandwidth=2$' /var/db/tor/cached-consensus | wc -l
53
hellas# ^2^3
grep '^w Bandwidth=3$' /var/db/tor/cached-consensus | wc -l
32
hellas# ^3^4
grep '^w Bandwidth=4$' /var/db/tor/cached-consensus | wc -l
25
hellas# ^4^5
grep '^w Bandwidth=5$' /var/db/tor/cached-consensus | wc -l
31
hellas# ^5^6
grep '^w Bandwidth=6$' /var/db/tor/cached-consensus | wc -l
29
hellas# ^6^7
grep '^w Bandwidth=7$' /var/db/tor/cached-consensus | wc -l
27
hellas# ^7^8
grep '^w Bandwidth=8$' /var/db/tor/cached-consensus | wc -l
32
hellas# ^8^9
grep '^w Bandwidth=9$' /var/db/tor/cached-consensus | wc -l
22
hellas# exit
exit
Script done on Wed Jun 2 10:39:46 2021
As you can see, 9% of relays in that consensus had single-digit "Bandwidth"
values assigned to them by the Authority relays, and the sizable majority
(almost
59%) of those were dumped into the lowest bin ("Bandwidth=1").
I frequently observe my relay chugging along at 330 KB/s to 355 KB/s and
occasionally at more than 400 KB/s. It appears that the Authority relays no
longer consider that worthwhile, so is there any reason that I should continue
to
run tor as a relay? Or should I reconfigure it to run as a client only and stop
needing to pay atention to it? I started running it around sixteen years ago,
but
if it's no longer going to be used, maybe I should shut it down instead of
letting
it just add clutter to the consensus..
>
> > Maybe the BWAuths have changed their measurement technique during the
> > last couple of months?
>
> Well, I first noticed it late last year, IIRC. The measurement technique
> will, of course, often give deceptive results. For example, if the connection
> supports ~350 KB/s and the relay has little traffic at the time measurement
> begins, the result should be fairly close to the true value. OTOH, if the
> relay is handling 200 KB/s of traffic for other circuits at the time
> measurement
> begins, then the result should be at most only ~150 KB/s, which is far from
> the
> true value.
> >
> > > A further question I would like to raise is why do the Authority relays
> > >use different criteria from one to another for the automatic assignment of
> > >flags? Should they not be consistent in using the same rules?
> > >
> > I agree that it is confusing that 2 auths don't assign the HSDir flag
> > according to the spec.
> > I have no explanation apart from that AFAIK moria1 is run by Roger
> > Dingledine and I guess he runs a lot of beta and test stuff.
> > moria1 publishes 2 HSDir "Flag Threshold" values (hsdir-wfu and
>
> Yeah, I saw that, but don't know quite what to make of it.
>
> > hsdir-tk) that no other auth publishes which leads me to believe moria1
> > runs another version of the auth software that handles the HSDir flag
> > differently. That don't explain bastet though.
>
> And it only accounts for t
Re: [tor-relays] current HSDir flag requirements
Logforme wrote: > > On 2021-05-26 08:18:32, "Scott Bennett" wrote: > >I interpret that as meaning that one > >or more criteria being used by one or more authorities has changed, > What I have noticed on my relay is that the "Consensus Weight" is > fluctuating. > CW is too complicated for my tiny brain but I believe the measurements > from the Bandwidth Authorities is involved. The BWAuths are spread > around the world and depending on current internet conditions they get > different speed values to your relay. But can it cause massive swings in > CW? Yes. My relay is on a residential service connection and its "bandwidth" rating from the Authority relays typically oscillates between ~30 and ~120, so proportionally the swings are often fairly wild. > Maybe the BWAuths have changed their measurement technique during the > last couple of months? Well, I first noticed it late last year, IIRC. The measurement technique will, of course, often give deceptive results. For example, if the connection supports ~350 KB/s and the relay has little traffic at the time measurement begins, the result should be fairly close to the true value. OTOH, if the relay is handling 200 KB/s of traffic for other circuits at the time measurement begins, then the result should be at most only ~150 KB/s, which is far from the true value. > > > A further question I would like to raise is why do the Authority relays > >use different criteria from one to another for the automatic assignment of > >flags? Should they not be consistent in using the same rules? > > > I agree that it is confusing that 2 auths don't assign the HSDir flag > according to the spec. > I have no explanation apart from that AFAIK moria1 is run by Roger > Dingledine and I guess he runs a lot of beta and test stuff. > moria1 publishes 2 HSDir "Flag Threshold" values (hsdir-wfu and Yeah, I saw that, but don't know quite what to make of it. > hsdir-tk) that no other auth publishes which leads me to believe moria1 > runs another version of the auth software that handles the HSDir flag > differently. That don't explain bastet though. And it only accounts for two Authority relays, whereas you said five are refusing to assign HSDir to my relay, which, as you pointed out, may depend upon network conditions between those Authority relays and my relay at the time and have nothing at all to do with my relay or how much traffic my relay could handle or might actually be handling at the time. > > It's fun to speculate :) > I would rather not be kept in the dark. It should not be like trying to get information on what the criminals who rule over us are up to. The problems outlined above would be mitigated somewhat if the measurements were filtered somehow, which could be as simple a filter as a boxcar moving average. Yes, I know that for many purposes a rectangular window gives lousy results, but for the purpose of understanding relays' capacities over time as having values that usually change slowly if at all a boxcar moving average should be plenty good enough. An exponential moving average would probably also be fine. The point of using a filter for the measurements would be to minimize the temporary interference of transient network conditions affecting the measurement process and corrupting some of the results at some times but not at others. Of course, measurements outside some number of standard deviations from the mean for a relay could be discarded, as well. At present, it is difficult to separate the deficiencies of the measurement method from the network realities in trying to interpret the measurements. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] current HSDir flag requirements
Logforme wrote: > On 2021-05-22 11:31:12, "Scott Bennett" wrote: > > > What are all the current requirements for a relay to get a > >HSDir flag? 96 (97?) hours of uptime and what else? > > Can someone tell me what my relay, MYCROFTsOtherChild, is > >missing for a HSDir flag? > > First, thank you for your reply. > From the spec: > https://github.com/torproject/torspec/blob/master/dir-spec.txt > > "HSDir" -- A router is a v2 hidden service directory if it stores and > serves v2 hidden service descriptors, has the Stable and Fast flag, > and the > authority believes that it's been up for at least 96 hours (or the > current > value of MinUptimeHidServDirectoryV2). > > "Fast" -- A router is 'Fast' if it is active, and its bandwidth is > either in > the top 7/8ths for known active routers or at least 100KB/s. Up until last night when I had to reboot my system twice a few hours apart, tor had been running for more than 12 days. It had both Fast and Stable flags, but no HSDir flag at any time since it started running. > > If you look at the concensus health > (https://consensus-health.torproject.org/), go to the bottom and enter > your relay nickname you will see what each of the authorities think > about your relay. Thank you for the link. I've bookmarked it for future reference, although the page is a bit difficult to interpret when seen with lynx(1). > moria1 and bastet are very stingy about handing out HSDir. I don't think > any of my relays ever has gotten it from them. > Of the rest, the authorities that think your relay is "Fast" has also > awarded it HSDir. Unfortunately 3 don't think the relay is "Fast" so no > HSDir. And with only a 4/9 vote the relay don't get the HSDir flag. > I don't notice any change in the specification you cited above, but I think something *has* changed. Up until about seven months ago, IIRC, it had always gotten the HSDir flag by 97 hours of uptime. Then the flag began to be both assigned and revoked at random times. That continued for a bit over two months, I think, before the flag disappeared forever. I interpret that as meaning that one or more criteria being used by one or more authorities has changed, so I am trying to find out what actually changed in case there is something I can do to return my relay to HSDir service. A further question I would like to raise is why do the Authority relays use different criteria from one to another for the automatic assignment of flags? Should they not be consistent in using the same rules? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] current HSDir flag requirements
What are all the current requirements for a relay to get a HSDir flag? 96 (97?) hours of uptime and what else? Can someone tell me what my relay, MYCROFTsOtherChild, is missing for a HSDir flag? Thanks in advance for any relevant information. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] New Test for Measuring the Accuracy of Tor Relays' Advertised Bandwidths
Georg Koppen wrote: > Some of you might remember the speed test Rob had been running during > August 2019 to investigate the accuracy of the advertised bandwidth a > relay is reporting in its server descriptor.[1] Back then this involved, > for each relay, downloading several large data streams for a period of > about 20 seconds to get a better estimation of the relay's true capacity > (See: [1] and follow-up emails for the exact details of the experiment). > > We plan to do this speed test again in about 1 week and relay operators > can opt-out of it (again) if they want to. We still believe that the > overhead of this speed test is in line with regular usage, but if anyone > doesn't want to participate that's fine. Let us know if so (by replying > to this thread or off-list, if preferred) and we remove your relay(s) > from the list to scan. > > Thanks, > Georg > > [1] > https://lists.torproject.org/pipermail/tor-relays/2019-July/017535.htmlff. > Okay, I read that, but I do not see where you offer any method to account for other traffic that the relay being tested may be dealing with at the time you are testing it. IOW, your testing procedure would be fine if only the tor network, or at least each relay while it were being tested, were entirely idle w.r.t. the traffic of other users. This is the same defect that the authorities' testing procedures suffer from. Do you have a solution to offer? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] how does one file a problem report?
nusenu wrote: > Scott Bennett: > > Would someone please point me to the > > relevant information? Bugzilla seems to be much easier to use. > > If you want to report a bug in tor: > > go to core -> tor -> issues -> New issue > > https://gitlab.torproject.org/tpo/core/tor/-/issues/new?issue%5Bassignee_id%5D=&issue%5Bmilestone_id%5D= > Thank you. It is now filed as issue #40383: daily abort in tor 0.4.5.7, 0.4.6.1-alpha, and 0.4.6.2-alpha. This is occurring on FreeBSD 11.4-STABLE r369587 every day between 6 p.m. and 7 p.m. CDT, though the exact time varies. I have added a crontab entry to restart tor daily at 7:01 p.m. CDT to deal with it, but it is definitely a critical bug. And, yes, bugzilla is definitely an easier system to use to report bugs. It even works with lynx(1) as a browser. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] how does one file a problem report?
I went through the process to get an account at gitlab.torproject.org in order to file a problem report for a very irksome and tiresome daily abort in tor 0.4.5.7 and 0.4.6.1-alpha under FreeBSD 11.4. However, I do not find anywhere on that web site for filing a PR, nor do I see any instructions as to how to file a PR. Would someone please point me to the relevant information? Bugzilla seems to be much easier to use. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Help with FreeBSD relays
xplato wrote: > I am running two relays and the error message is the same for both: > > Mar 30 08:13:01 freebsd kernel: pod 1745 (tor) , jid 0 , uid 256, was killed: > out of swap space > Oh. Then Fabian may be right. Assuming that you already have what should be an adequate amount of swap space available, then this could be due to one of the cluster of memory management bugs introduced into the FreeBSD kernel in 11.2-RELEASE and remains in 12.x and very likely will be in the upcoming 13.0. > If I run > # dd if=/dev/zero of=/usr/swap0 bs=1m count=512 > Note that, while once upon a time 512 MB was a large amount of swap space, in modern times it is almost trivial and inconsequential. > #chmod 0600 /usr/swap0 > > #swapon -aL > > Will that fix the error above? It might alleviate it for a short time, but if the problem is due to those bugs, it likely will make little or no difference. The reason for that is that the message is partly erroneous; i.e., it is correct that the OOM killer has killed the process, but it is incorrect that it was out of swap space. Having watched those bugs in action for several years now, what I can tell you is that a lot of pagefixing is going on, but very little pagefreeing happens later. Processes being killed with that error message is just one symptom, and it can be a real problem, for example, if xorg is running and gets killed, leaving the console inaccessible. Another symptom is that, one by one, processes stop doing anything because they get swapped out due to the shortage of page frames on the free list. The kernel will not begin to page processes back in unless there is at least ~410 MB on the free list, so the system ends up with nothing running, not even shells, because everything is marked as swapped out. If what is happening to tor on your system, then increasing swap space likely will have no effect because swap space is not really where the shortage exists. The shortage is on the free list. There are some things that you can do in 11.4 that will minimize the situations where the memory management problems take over the system. You can set a sysctl tunable that may help a bit. Unfortunately, vm.max_wired no longer does anything and is a red herring. You can try to limit kernel memory by setting vm.kmem_size_max to some value considerably less than the size of real memory on your system. Although the system does not honor this limit either, it may still have a minor influence on how much the kernel uses. I think I set mine to 4 GB on an 8 GB machine. This should be set in /boot/loader.conf. In /etc/sysctl.conf there are several variables that should each help a little more. If you use ZFS, you can try limiting the size of the ARC by setting vfs.zfs.arc_max. After setting that, you may see the ARC grow to as much as ~200 MB more than the size you set the limit to, but it doesn't really go beyond that, so it does work after a fashion. Just allow for that extra couple of hundred megabytes or so. Next is vm.v_free_min, which on my system defaults to 65536, and I have increased that to 98304. Then there is this very important one: vm.pageout_wakeup_thresh=112640. Its default value is only 14124, a far cry from the ~410 MB needed on the free list for the kernel to begin paging a swapped process back into memory. (112640 pages are 440 MB, so it gives a tiny bit of leeway to the pagedaemon to get to work before the free list gets too low.) Lastly, set vm.pageout_oom_seq=10240 to prevent the OOM killer from killing your processes. This value is the number of complete passes through memory the pagedaemon must make in its attempt to free enough memory to satisfy the current demand for free page frames before it calls the OOM killer. Setting the value that high means that the pagedaemon never will get through that may passes, so the OOM killer never gets called. After setting this one you may occasionally see the pagedaemon using all of one core's CPU time for a while, possibly a *long* while, but it should protect your processes from being killed due to the collection of memory management bugs. With all of the variables mentioned above set to better values you may still see the system slowly grind down to an idle state. This can happen due to the kernel prioritizing the keeping of unused file system buffers in memory over swapping processes in to get actual work done. In such a case, manual intervention is required to free up page frames. For example, if you confine your ccache directory trees to a UFS file system, the system will quickly accumulate a lot of buffers it doesn't want to let go of. The same holds true for portmaster's $WRKDIRPREFIX, where a "portmaster -a" to update your ports will tie up a large number of buffers. buildworld and buildkernel are also culprits. The file system buffers can be forcibly freed up, thereby freeing page frames occupied by the file system buffers, by unmounting the UFS
Re: [tor-relays] Help with FreeBSD relays
Fabian Keil wrote: > xplato wrote on 2021-03-30: > > > I am a bit of a noob here so please bear with me. I ran a relay using > > Ubuntu with very few issues however I decide to add an additional relay > > and decided to use FreeBSD. They will only run for around 18 hours and > > then they shut down. I have adjust the torrc file every way I know how > > and increased the Max vnodes thinking this may have been my issue. I can > > post the sysrc and torrc if needed. Anyone that might help me figure > > this out I would be grateful otherwise I am going to reluctantly move > > them both back to Ubuntu. > > What do you mean by "they" and "shutting down"? It exits on signal 6. See the tail end of the log excerpt I included in my posting about 0.4.6.1-alpha being very noisy. N.B. 0.4.5.7 also has the same bug. I have browsed a bit in the consensus documents, looking for relays running those versions of tor. What I found was that the ones running those versions on LINUX systems often have much longer uptimes, but I didn't find any FreeBSD systems running them with uptimes much over one day, which is consistent with them failing in short order on FreeBSD systems. I did not look for those versions on the other BSDs, but such systems are generally much rarer than those on FreeBSD, except perhaps for MacOS. > > Seeing the torrc and /etc/sysctl.conf indeed could be useful. Possibly, but probably not. Take a look at the crash messages that tor usually writes to its notice-level log, which look like they have something to do with authority code, but we, of course, do not run authority relays. I think I did have one crash that did not write that batch of messages to syslog, but usually it does. > > Please also check the system and tor logs to see if there are > any relevant log messages. I posted such in my previous two postings. > > If you don't already, you may want to start monitoring tor's > and the system's resource usage. > xplato's relay may differ from mine, but mine isn't using much these days because the authorities have a) stopped giving mine an HSDir flag for the last two months or so after giving it seemingly randomly for months before that, and b) been giving or withholding a Fast flag seemingly also at random. This is on a relay that often runs at 300 KB/s to 400+ KB/s, both in and out. The whole mess makes me wonder whether it's worth my bothering to maintain and run a relay anymore. Can you tell us why the consensus documents on Sunday showed *all* authorities with Bandwidth=20 Unmeasured=1 for some hours? What caused such an alarming situation? Scott ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Help with FreeBSD relays
xplato wrote: > I am a bit of a noob here so please bear with me. I ran a relay using Ubuntu > with very few issues however I decide to add an additional relay and decided > to use FreeBSD. They will only run for around 18 hours and then they shut > down. I have adjust the torrc file every way I know how and increased the Max > vnodes thinking this may have been my issue. I can post the sysrc and torrc > if needed. Anyone that might help me figure this out I would be grateful > otherwise I am going to reluctantly move them both back to Ubuntu. > Thanks for attempting to run a relay in a diversifying environment. Unfortunately, your timing has left you high and dry. What is happening is not your fault. Both 0.4.5.7 and 0.4.6.1-alpha have a fatal bug. I don't know which version of tor introduced it because I have been running 0.4.5.2-alpha until a few days ago, when I "upgraded" to 0.4.6.1-alpha. That crashed in under 24 hours twice, so I switched to the allegedly "stable" version, 0.4.5.7 only to have it crash the same way in about 12 or 13 hours. Also, 0.4.6.1-alpha was filling its notice-level log file with many bug messages the whole time it was running. I posted a message on this list about it, including the last few hours of the log up through the crash messages. At the crash, the console log (also in /var/log/messages) had a message like this. Mar 27 18:29:26 hellas kernel: pid 17047 (tor), jid 0, uid 256: exited on signal 6 After the second crash I made the switch mentioned above. It has been crashing ever since, and I am getting very tired of discovering that it has been down for several hours. There have been no responses to my earlier posting, but your query is on the same topic, if a different Subject: line, so I am replying. This sort of thing has not hit me in tor in many years, and I hope the tor project developers will fix it ASAP. My suggestion is that you hold off a bit and try again once the bug has been squashed. Alternatively, if you a different BSD system (i.e., DragonflyBSD, NetBSD, or OpenBSD), running your relay on that system would help diversify the systems running relays as much as, or more than, running it on FreeBSD. IOW, it depends on how urgently you want to get a relay up and running vs. your level of patience to wait for a bug fix. Like I wrote above, this situation hardly ever happens, so if you can be patient, it may be fixed within a few days. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] 0.4.6.1-alpha sure is noisy
k the most important question again here. What are the new requirements for an HSDir flag? Before last fall, a minimum of 96 or 97 hours of uptime was the only requirements for a relay to be assigned the HSDir flag, but that is clearly not all that is necessary now. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] current requirements for Fast and HSDir flags
What are the current requirements to get an HSDir flag assigned to a relay? MYCROFTsOtherChild has, at last heartbeat message, been up for 35 days and 7 hours. To the best of my knowledge, it has not had an HSDir flag in the consensus at any time since it was last started more than a month ago. What are the current requirements to get a Fast flag assigned to a relay? MYCROFTsOtherChild often runs busily at 330 KB/s to 350 KB/s. Sometimes since it was last started over a month ago it has had a Fast flag in the consensus, and at other times it has not. The Fast flag seems to come and go like the wayward breezes. The Bandwidth= value in the consensus entry for MYCROFTsOtherChild has varied from somewhere in the 30s to the 110s, AFAI have seen, but is always subject to the delayed effects of the presence or absence of the Fast flag, of course, and prior to the currently running instance has also been subject to the presence or absence of the HSDir flag. The authorities have seemed to be acting more or less at random when assigning flags for the consensus for many months. Does the tor project envision any timeline for a return to rational, predictable behavior on the part of the authorities? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] anyone else getting sync floods from russia?
niftybunny wrote: > Glad to hear its nothing personal. Putin still loves me ?? > > Thats Perl? I have no clue what it does. > > We already changed the timers on the TCP connections and we have scripts > running which are blocking IPs who will send us x connections. Right now > they changed tactics and for me it looks like SYNC flood from datacenter IP > ranges and a few 100 IPs which undermine the easy blocking. Everything over > 2,5 million TCP connections and the servers are more or less overloaded and I > now learned that 3 million TCP connections is the point where the servers are > dead as dead can be. > > For a one time attack I would congratulate them but now daily it really is > starting to suck. I also suxx that we have a direct 10G connection to the > largest Russia ISP so they can DDOS us even faster ? > Do you have pf available as a packet filter? pf's synproxy is designed to mitigate that sort of thing, when it is used. IIRC, it doesn't pass a connection on to the application until all the SYN/ACK handshaking is completed. It may also enforce an early timeout on waiting for the next step after the initial response, but I really don't recall because I haven't used it in many years. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Is my relay broken? No stable, hsdir or guard flags
raltul...@posteo.org wrote: > > > Am 30.01.2021 05:59 schrieb Scott Bennett: > > Roger Dingledine wrote: > >> > >> > It is a fallback relay - is it still useful as such? > >> > >> It is still useful yes. > >> > > Well, at least there is that, thank goodness. And at least I > > finally > > have a plausible explanation for the authorities' seemingly bizarre > > behavior > > over the past many moons in which they appear to be awarding, then > > withholding, > > then awarding, then withholding various flags, including HSDir, Stable, > > and > > even Fast, at random when my relay has been up and unfiddled with the > > whole time > > from what I have been able to see, and without any apparent rhyme or > > reason. I > > had given up trying to figure out what the authorities were doing, > > much less why, > > and therefore had stopped giving a dam about it. > > That situation, in combination with the involvement in the tor > > project of > > very vocal persons lacking apparent comprehension of the basic > > relationships > > between hardware, firmware, supervisory (i.e., privileged) software, > > and > > application software w.r.t. security matters, led me to stop providing > > a relay > > for about a year. I finally decided to return it to service, but have > > mostly > > stopped reading the overly chatty tor-relays list and any worry over > > additional > > security issues w.r.t. tor. I take care of my relay the best I know > > how with > > one exception on the authority of Roger Dingledine and leave it at > > that. > > Because I don't like accepting such things on authority, rather than > > proof or at > > least damned good evidence and reason, I mostly stopped caring, which > > is a real > > drag because I once was convinced that tor was a truly good effort to > > provide > > what should have been designed into the Internet in the first place. > > Unfortunately, there appear to be still no viable alternatives > > available. > > > > It's been another week and the flags are all back. I'm not aware of > anything I did > to loose them for a month and I din't do anything to get them back. For > me that > just just means I'll take your advice and start caring less ?\_(?)_/?. > > Thanks for your help everyone and I hope the underlying problem get > fixed soon. > The following popped up almost an hour and a half ago in my relay's log file. Feb 13 07:27:54.947 [notice] The current consensus has no exit nodes. Tor can only build internal paths, such as paths to onion services. Two milliseconds later it said, Feb 13 07:27:54.949 [notice] Our directory information is no longer up-to-date enough to build circuits: We need more microdescriptors: we have 0/6708, and can only build 0% of likely paths. (We have 0% of guards bw, 0% of midpoint bw, and 0% of end bw (no exits in consensus, using mid) = 0% of path bw.) Feb 13 07:27:54.949 [notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 0/6708, and can only build 0% of likely paths. (We have 0% of guards bw, 0% of midpoint bw, and 0% of end bw (no exits in consensus, using mid) = 0% of path bw.) Is it telling the truth? Was there really an hour when the consensus contained no Exit flags in the entire file? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Is my relay broken? No stable, hsdir or guard flags
Toralf F?rster wrote:
> On 1/30/21 5:59 AM, Scott Bennett wrote:
> > Or, as an alternative to the above proposal, newly awakened
> > authorities'
> > votes regarding time-dependent flags should be ignored by other authorities
> > until the newly awakened have been awake at least, say, ten days?
>
> I do wonder if this helps if all authorities suffer from the same
> issue/architectural fla/DDos/bug ?
>
Well, at least some of us currently see that a majority of the authorities
*do* suffer from the bug that Roger described. That bug does seem to provide
some impairment to the performance of the tor network by causing loads to be
distributed based upon faulty information in consensus documents. The network
is presumably large enough now that such impairment is likely to be difficult
to detect on the larger scales, but is being noticed by individual relay
operators
and our observations posted here.
There was a time when the rule stated was that the different authorities
must
run different versions of tor in hopes of avoiding that sort of thing, but of
course, tor versions are still susceptible to including the same bugs in many
versions. I doubt that there is a perfect solution, but authorities can be
configured with many overrides for standard operating parameters already.
Either
of the modifications that I proposed could well be implemented with a way for
the
project to temporarily override it in emergencies that would otherwise leave
over
half of the authority relays in a non-voting state simultaneously, for example.
Also, note that the HSDir flag is withheld until a relay completes a
minimum
of 96 hours of uptime. Does that restriction currently apply to authorities as
well? If so, then it provides an example of how applying time-dependent limits
to the privileges of authorities should work.
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Is my relay broken? No stable, hsdir or guard flags
Roger Dingledine wrote: > On Fri, Jan 29, 2021 at 11:32:44AM +0100, raltul...@posteo.org wrote: > > - At the beginning of January the relay seems to have lost the guard flag > > - A week ago I checked and noticed that the relay had also lost the stable > > flag despite having an uptime of >2 months at that point > > - A week ago I rebooted the server but the situations hasn't changed - flags > > are still gone > > Some of the directory authorities have restarted many times in the past > week, and each restart impacts their view of whether other relays are > stable. In theory it should impact all the views equally (it's like > there was a blip in the matrix but it was an equal blip for everybody), > but in practice, maybe the math doesn't make it actually equal. IOW, authority relays should not receive an Authority flag until they have sufficient uptime to judge correctly? Say, ten days or more? > > > - metrics.torproject graphs show that the server has been transmitting data > > the entire time - so it doesn't seem like I missed some downtime > > The graphs on relay-search are visualizing data that is self-reported by > the relay. So from the relay's perspective there was no downtime, but > that doesn't give us much hint about whether the directory authorities > found the relay consistently reachable. > > Another hint I find useful is to look at the individual votes from > the directory authorities. One way to do that is to go to the bottom of > https://consensus-health.torproject.org/#relayinfo and put in your > nickname. Or, as an alternative to the above proposal, newly awakened authorities' votes regarding time-dependent flags should be ignored by other authorities until the newly awakened have been awake at least, say, ten days? That would seem a minimum time appropriate for them to give valid judgments on the matter of time-dependent flags. > > > - Serverlogs show no problems > > - Relay has been running continuously for almost 4 years and only gets > > rebooted for kernel/tor upgrades -> so uptime and MTBF should not be a > > problem > > > > Is there a problem on my side? > > Is there anything I can do or check? > > One part that I would look into more is the IPv6 connectivity. Maybe > that address is intermittent? If it is, then the relay would mostly > continue to work as normal (because clients mostly use IPv4), but the > subset of the dir auths that checks IPv6 reachability would consider > that instability to be a short downtime. Until IPv6 addresses are used by other relays to forward traffic, this seems a totally inappropriate criterion for determining reachability. > > > It is a fallback relay - is it still useful as such? > > It is still useful yes. > Well, at least there is that, thank goodness. And at least I finally have a plausible explanation for the authorities' seemingly bizarre behavior over the past many moons in which they appear to be awarding, then withholding, then awarding, then withholding various flags, including HSDir, Stable, and even Fast, at random when my relay has been up and unfiddled with the whole time from what I have been able to see, and without any apparent rhyme or reason. I had given up trying to figure out what the authorities were doing, much less why, and therefore had stopped giving a dam about it. That situation, in combination with the involvement in the tor project of very vocal persons lacking apparent comprehension of the basic relationships between hardware, firmware, supervisory (i.e., privileged) software, and application software w.r.t. security matters, led me to stop providing a relay for about a year. I finally decided to return it to service, but have mostly stopped reading the overly chatty tor-relays list and any worry over additional security issues w.r.t. tor. I take care of my relay the best I know how with one exception on the authority of Roger Dingledine and leave it at that. Because I don't like accepting such things on authority, rather than proof or at least damned good evidence and reason, I mostly stopped caring, which is a real drag because I once was convinced that tor was a truly good effort to provide what should have been designed into the Internet in the first place. Unfortunately, there appear to be still no viable alternatives available. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined mili
Re: [tor-relays] >23% Tor exit relay capacity found to be malicious - call for support for proposal to limit large scale attacks
nusenu wrote: > > Pascal Terjan: > > I am not convinced it would help large scale attacks. > > Running 50 relays is not much and it each was providing 0.49% of > > capacity that would give them 24.5%... > > I would expect that an attacker would create more relays than that and > > unless there is a good way to find out this is a single entity, they > > will all be well below 0.5% > > > Yes, they will try to circumvent thresholds by pretending to not be a group. > The good thing is that this requires additional resources and time on the > attacker side to hide > the fact that they are adding many relays without triggering certain > detections. > Your proposed method of delaying the problem would impose a labor burden on the tor project as well and would be slow to react to changes. Why would an automated solution not work? For example, if the directory authorities calculate the traffic percentages every hour or so or even every several hours, then why not just remove a Guard or Exit flag from any guard or exit exceeding the publicized percentage? That would be a fast reaction and would not depend upon multiple human actions. You might also implement a "repeat offender" policy, whereby if the authorities lifted a relay's Exit flag more than n times within a month, a BadExit flag would be applied in addition, which then (and only then) would require the operator to contact the tor project about it. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] DoS attacks are real (probably)
Alex Xu wrote:
> Quoting Felix (2017-12-11 17:07:30), as excerpted
> > Hi Alex
> >
> > Great points.
> >
> > > conntrack -L -p tcp --dport 9001 | awk '{print $5}' | sort | uniq -c
> > > | sort -n
> >
> > On FreeBSD one can do:
> >
>
> yeah, the optimal rule would ban "bad IPs" after some threshold of
> connections, like "if one IP makes >1 conn/sec for at least 1 minute ban
> for 1 hour" or something. I'm hoping to fix the underlying issue in Tor
> so that low-bandwidth attacks like these are less effective.
FWIW, the method that Felix posted should also work in DragonflyBSD
and NetBSD. It may also work in OpenBSD, but the caveat is that the OpenBSD
project has continued to develop its implementation of pf, so I don't know
whether Felix's solution still works in OpenBSD. The other three BSDs' pf
support has not been synchronized with that of the originating project
(OpenBSD) for many years. Perhaps an OpenBSD tor relay operator can comment
here on this matter.
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] DoS attacks on multiple relays
"x9p" wrote: > > Furthermore, nodes that think they're being smart > > and protecting their nodes and the network by filtering > > out who can access their OR ports... aren't... because > > I strongly agree with grarpamp. Sorry to read that. > > Relay operators, do NOT block the IP addresses of the following guards in > your firewall: > > [list deleted --SB] > They have been added to a file that gets included into a list of relay addresses (i.e., protected addresses) and will get the "free pass". This, BTW, is basically a temporary hack. If a frequently generated, automatically downloadable-by-relays list of otherwise unpublished relay source addresses becomes available, I'll axe this file because I'd much rather have a file that is kept up to date automatically. Note, however, that if a facility existed in tor for a relay to fetch a list from the tor project, there would have been no reason to publish them on any email list. Also, FWIW, they were not in my block list anyway, but now they won't be blocked from my relay's ORPort or DirPort by the block list even if they do get onto it. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] ISP is aking me to send a selfie holding my identity card
grarpamp wrote: > > Sent them their stuff they demanded and 2 days later my > > account was restored and they are ?fine? with abuse. > > Because doing KYC and Gestapo on operators over "bad" traffic > that you're already not responsible for somehow magically > turns it into "good" traffic that you are responsible for. > Lol, what a joke. What they're fine with is abusing rights > with a boot from above. > > > So, nothing to worry about. > > The entire thing is something to worry about. On that issue, we are in complete agreement, I think. What it means is that the people in those countries are already being denied their rights. Their governments are using a different method or set of methods from the ones the U.S. government is using, but it still equates to tyranny. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] DoS attacks on multiple relays
grarpamp wrote:
> The advantages for tor users against censorship, of operators making
> some fraction of all exit nodes exit from different addresses than
> those addresses present in consensus, have been explained many
> times on this list.
There are also advantages for tor users in keeping the hosts on which
relays run secured against attacks, which helps to keep the tor network
functioning. Having connections arrive from addresses not associable with
tor relays by the recipients of those connections hinders connectivity
among relays in the consensus, as has been explained many times on this
list. I call attention to the fact that the tor project has already decided
against Exit node secrecy by regularly publishing the results of a script
(exitmap) that reveals such addresses.
>
> It's also been noted that in at least one instance, some exit operators
> looking at constrained / creative hosting issues have chosen that
> as an enabling option too.
However, the tor project has publicly promised to identify publicly
all Exit source addresses, so that system administrators everywhere who wish
to block tor Exits to their networks may do so without blocking all tor
relays. The readily downloadable exitmap results allegedly fulfill that
promise.
Are you now alleging that, in at least one instance, the tor project
has surreptitiously broken its promise by omitting that instance's
address(es) from the exitmap file?
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] DoS attacks on multiple relays
teor wrote: > > > On 6 Dec 2017, at 19:13, Scott Bennett wrote: > > > > null wrote: > > > >> @ x9p: > >> > >>> [stuff deleted --SB] > >>> > >> relays. Since our relays are all in the same declared family, it is > >> very difficult to see how this traffic is legitimate. If it's valid > > > > But it could be legitimate. As has been discussed here previously, > > they may be connections from a relay that actually is in the consensus, > > but either a) uses an OutboundBindAddress or b) is on a LAN that has > > multiple connections to the Internet. > > Relays try very hard to have at most one connection to each other relay. Good. That tendency must have improved a lot over the years, then. > (And only two relays are allowed in the consensus per IPv4 address.) Yes. > Clients try to make one connection to one or two guards. Or however many are specified in torrc. Sometimes one needs, say, three to five guards for better capacity or lower latency or better reliability. > > So it's far more likely to be a collection of Tor clients in a network with > only a few public IPv4 addresses. (There are entire countries and large > networks that only have a few allocated IPv4 addresses.) An excellent point. Thank you. > > Or, it might be a bug in Tor, a misconfiguration, or a denial of service > attack. We'd like to know more, so we can find out and fix it. > If it's a LAN with many users and few external IPv4 addresses (or, perhaps, even IPv6 addresses if NATed to the outside via only a few), then there really is nothing to fix. > > (Snip) > > A script similar to the one used to reveal and make available the > > otherwise unidentified source IP addresses of exits could be run by the > > project to gather the hidden addresses of currently running relays, and > > a list of such addresses could be made available on a compromise basis, > > e.g., by having a relay at the project that would serve those lists only > > over tunneled directory connections *from relays*, were it not for > > obstinacy. > > Such a list could then be included into our packet filters' "free pass" > > lists without putting the list up on the project's web site like the exit > > list is. > > Outbound addresses aren't secret, because they are used for connections. Roger has claimed here that some of them are indeed secret in the sense that their owners do *not* want them to be published, one possible reason for which being that they do not want their relays blocked successfully by governments, e.g., China, Iran. (How hiding the source address of a published relay would evade the Great Firewall escapes me somehow, except perhaps for hidden services based inside China that might be reached via those hidden source addresses. Given that most source addresses of relays *are* published, the chances of getting a circuit into China seem rather slim anyway.) I guess tor project members are not all in agreement over such matters. > > Anyone is free to volunteer to create and maintain a list of outbound > relay addresses. It is technically feasible: it requires a few thousand Tor > connections per day, one via each relay, to a relay that reports the Ideally, per hour, but that is why it should only be done by one site. Note that Exit relays might be skippable because their outbound addresses are already identified by one site, namely, the tor project, and published. IOW, only entry/middle and non-Exit exit nodes need be tested, which would shrink the list by several hundred to a thousand or so. > remote address of each inbound relay connection. > > It just needs to be done safely, in a way that doesn't collect client > addresses, and avoids attaching a timestamp or order to relay connections. Why would client addresses ever be involved? What would be gathered are the addresses from which *relays* connect to other relays (N.B. *not* to destinations). The only timestamps that I see might be relevant would be the starting and ending times for each script run, so that an administrator's own script(s) for incorporating those addresses into his "free pass" list might easily discern out-of-date script output files from current script output files. > > > A torrc option to fetch the list whenever updates were available > > could default to not fetching the list, so relays whose operators who do > > not use packet filter defenses would not automatically fetch the hidden > > address list of non-exit relays. > > If someone created a list, and showed that it had value to other relay > operators, then it might gain support, and be supported by Tor, > just
Re: [tor-relays] DoS attacks on multiple relays
null wrote:
> @ x9p:
>
> > # netstat -tupan | grep ESTABLISHED | grep /tor | awk '{print $5}' | awk
> > -F: '{print $1}' | awk -F. '{print $1"."$2"."$3}' | sort | uniq -c | sort
> > | egrep -v ' 1 | 2 | 3 '
> >
> > with this information in hand, double the max of it (mine was 10
> > connections from 188.214.30.0/24):
> >
> > 10 188.214.30
> >
> > iptables -A INPUT -i eth0 -p tcp -m connlimit --connlimit-above 20
> > --connlimit-mask 24 -j REJECT --reject-with tcp-reset
>
> Thank you! This was extremely helpful.
>
> In our case we found a handful of IPs that had *thousands* of
> concurrent connections on several of our relays. The offending IPs
> were not in the consensus. After restarting the Tor service, these
> suspect connections come back rapidly, again across several of our
> relays. Since our relays are all in the same declared family, it is
> very difficult to see how this traffic is legitimate. If it's valid
But it could be legitimate. As has been discussed here previously,
they may be connections from a relay that actually is in the consensus,
but either a) uses an OutboundBindAddress or b) is on a LAN that has
multiple connections to the Internet. Those addresses (of either kind)
are not published by tor, and the tor project has deferred to the alleged
desires of some unknown portion of the operators of such relays to have
those addresses remain secret at the expense of those of us who wish both
to protect our systems, especially those systems that support the tor
relay network, by the judicious use of packet filters and to give a free
pass through those packet filters to currently running relays to access
our relays' ORPorts and DirPorts. (I currently have such a "free pass"
system for relays' IP addresses that appear in the most recently fetched
consensus or the most recently fetched list of exit IP addresses. It
also includes a handful of addresses mentioned on this list as being
relay addresses that might not appear in the consensus.)
No one on this list, BTW, has yet given a rationale for prioritizing
the allegedly desired secrecy of those relay operators over our efforts
to help to secure our tor relays and our tor relays' hosts. However, a
couple of people have resorted to name-calling and absurd accusations of
"censorship". (Now, tell me again, *why* is it that I've run a tor relay
most of the time for over a decade? To practice censorship? Such stunning
illogic is rather appalling.)
A script similar to the one used to reveal and make available the
otherwise unidentified source IP addresses of exits could be run by the
project to gather the hidden addresses of currently running relays, and
a list of such addresses could be made available on a compromise basis,
e.g., by having a relay at the project that would serve those lists only
over tunneled directory connections *from relays*, were it not for obstinacy.
Such a list could then be included into our packet filters' "free pass"
lists without putting the list up on the project's web site like the exit
list is. A torrc option to fetch the list whenever updates were available
could default to not fetching the list, so relays whose operators who do
not use packet filter defenses would not automatically fetch the hidden
address list of non-exit relays.
> Tor clients, they are behaving very strangely, and in either case we
> need to limit their impact. As such we've implemented connlimits by
> /24 as suggested (with a much higher limit to err on the side of not
> rejecting valid traffic). We can already see that this has improved
> our situation.
>
And it will likely get you roundly denounced by tor project members
and certain other individuals on this list. :-( You will also see your
Fast and HSDir flags come and go at random, depending upon how many
authorities creating testing circuits to reach and test your node(s)
go through a node that used a hidden outbound address as the source
address that fails your filter to connect to your node.
Nevertheless, without knowing whether the sources from which you
claim many connections are made to your relay(s) are themselves relays,
you cannot know whether they are legitimate. For example, suppose a
high-capacity relay is serving as the hop prior to your relay in many
circuits, quite possibly from large numbers of users in many different
countries. All you see is that a high-capacity relay has a lot of
circuits created by clients that have chosen that relay, followed by
yours, in their circuits. If you run an exit, you don't know whether
that relay was the entry relay or the middle relay for any particular
circuit. There may well be some of each.
>
> @ Scott Bennett:
>
> >
Re: [tor-relays] DoS attacks on multiple relays
null wrote: > We're experiencing what looks like a DoS attack on multiple relays in our > family: > > https://atlas.torproject.org/#search/family:CBEAE10CBBB86C51059246B2EF92EB2CB4E111BC > > The relays are currently running Tor 0.3.1.9 on Linux kernel 4.4.0 > (although when the problem started the relays were running Tor 0.3.1.8). > > The attack knocked 3 of 6 relays offline overnight. By the time we looked > at logs, the Tor service had stopped and this was the last line in the log: > > "Tor[xyz]: Failing because we have 16351 connections already. Please read > doc/TUNING for guidance." > > The attack is still ongoing. When it's happening, the number of connections > rises very rapidly, until the attack succeeds in stopping the service. > > $ ss -s > Total: 15855 (kernel 0) > TCP: 24520 (estab 23969, closed 305, orphaned 31, synrecv 0, timewait > 261/0), ports 0 > > Transport Total IPIPv6 > * 0 - - > RAW 0 0 0 > UDP 8 4 4 > TCP 24215 24213 2 > INET 24223 24217 6 > FRAG 0 0 0 > > ... and only a few seconds later: > > $ ss -s > Total: 12120 (kernel 0) > TCP: 27389 (estab 20026, closed 1906, orphaned 45, synrecv 0, timewait > 1587/0), ports 0 > > Transport Total IPIPv6 > * 0 - - > RAW 0 0 0 > UDP 8 4 4 > TCP 25483 25481 2 > INET 25491 25485 6 > FRAG 0 0 0 > > That's obviously much larger than the normal number of connections, more > than we've ever seen, and seems like more connections than would be needed > for a relay. > What you are seeing is most likely the same phenomenon brought up on this list repeatedly over at least the last decade or so. That phenomenon is providing HSDir service, or perhaps a rendez-vous point, for a popular hidden service. As soon as your node is associated with that hidden service and that association begins to be distributed by the HSDir population to clients looking for that hidden service, the number of connections to your node will increase fairly rapidly to a level corresponding to that hidden service's level of popularity. If you don't like it, you can set HidServDirectoryV2 0 which will stop clients from trying to get hidden service descriptors from your node, which will eliminate most of the bursts of connections you're seeing, but will not prevent your node from being a rendez-vous point because every tor relay is expected to provide that function as part of the relay protocols. > We have file descriptors (/proc/sys/fs/file-max) set to 64000, but it looks > like Tor sets MAX_FILEDESCRIPTORS to 16384 per /etc/init.d/tor: > > elif [ "$system_max" -gt "4" ] ; then > MAX_FILEDESCRIPTORS=16384 > > Surely that is high enough for normal service? If by normal you mean "low traffic", then yes, it's probably enough. However, that's really not very high in a general sense. Consider also that some installed packages place high demands upon the supply of file descriptors. (E.g., I gather you do not have a graphics port/package called piglit installed on your system, which recommends at least 5 be available for its runs, so I have kern.maxfiles="5" in /boot/loader.conf on my FreeBSD system. I don't think I can recall tor ever handling many more than 5000 (i.e., 10% of that figure) at one time on my low-traffic node.) The faster, larger-capacity tor nodes often have considerably higher settings to keep tor from exhausting the fd limits on those hosts. > > We haven't started looking into where the traffic is coming from or other > characteristics. We are wondering if: 1) this is a known attack, 2) if > other operators are experiencing it, 3) if there are any ideas for > mitigating it, and 4) if any additional information would be helpful. > Other than refusing to be a hidden service directory server, there is probably nothing to be done about it. Adjust your settings accordingly, along with your expectations. :-) Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Detecting Network Attack [re: exit synflooded]
grarpamp wrote: > On Fri, Nov 24, 2017 at 6:23 PM, wrote: > > Was anyone else's exit being synflooded yesterday and today? > > There could be a combined monitoring array deployed > among all nodes that might start to answer these questions. > And further alert on all sorts of interesting network attacks > launched at Tor. Software exploits... segmentation / modulation > attacks... etc. This is perimeter security 101, and Tor > currently has no fence, no guard dogs, not even a janitor > you can ask the next day. That's beyond bad. > And big oppurtunity for new research project. Not that long ago, grarpamp, you were vehemently--to the point of hysterics--opposed to fences, even with gate passes for authorized entry. Now you complain about the lack fences (without mention of gate passes, however), guard dogs, etc. What changed your mind? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Testers needed for Nyx beta release
tor wrote: > > Adjusted the message to specify installing for whatever > > interpreter version they're running... > > https://gitweb.torproject.org/nyx.git/commit/?id=93f20f9 > > Damian: there's now a missing ')' at the end of nyx/__init__.py line 78 > causing a syntax error. > > Scott: thanks for the tip about the different sqlite3 versions, although my > system seems to actually still use Python 2.7.14 by default. Try "pkg install python36". If nyx is written for python 3, then running it with python 2 might not be such a good idea. It may work for now, but nothing guarantees that it will continue to do so. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Testers needed for Nyx beta release
Damian Johnson wrote: > > Is FreeBSD supported? I've run into a few hurdles, and overcame several, but > > can't get over the last one. > > Hi. Yup, FreeBSD is supported *but* I don't have a system to test on, > so I rely on user reports like this to provide comparability. > > > I had to do: > > > > sudo pkg install py27-sqlite3 Given that nyx is supposedly now written in python 3, perhaps a better choice would be one of py3[456]-sqlite3, according to which python 3 version is installed, instead of mixing python versions. > > Perfect! I'll change Nyx so when sqlite3 is unavailable on FreeBSD it > suggests for folks to run that. See note above. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Using ContactInfo to publish additional relay properties in a standardized way (opt-in)
"Iain R. Learmonth" wrote: > Hi, > > On 14/10/17 23:01, nusenu wrote a proposal. > > The ContactInfo field doesn't need to be overloaded again. ): > > See proposal 234 [0] for adding bitcoin/zcash/etc addresses. > > Maybe we can work out new fields for extrainfo documents [1] that could > contain some of this information. ExtraInfo documents are also one possibility for communicating OutboundBindAddress values, as well. > > I do think this is a useful thing to look at, but I just don't think > that ContactInfo is the right way to go about it. I concur. > > [0]: > https://gitweb.torproject.org/torspec.git/tree/proposals/234-remittance-addresses.txt > [1]: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n799 > Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] spurios warning about using the nickname instead of the key
teor wrote: > > > On 12 Oct 2017, at 13:21, Toralf F?rster wrote: > > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA256 > > > >> On 10/11/2017 10:08 AM, Dylan Issa wrote: > >> Did you set MyFamily using nicknames of the key? > >> Because if nickname, change it to key is basically what it?s saying. > > > > > > No, > > > > both Tor processes share this common config file: > > > > grep -i famil /etc/tor/torrc*/* > > > > /etc/tor/torrc.d/00_common:MyFamily > > 1AF72E8906E6C49481A791A6F8F84F8DFEBBB2BA,6EABEBF38CE7E3DF672C4DB01383606FE3EB2215 Doessn't each fingerprint need to start with a $ character? Otherwise it's just a really long nickname, right? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Feedback wanted: letter to my university's library
Kenneth Freeman wrote: > > > On 10/03/2017 11:31 PM, Scott Bennett wrote: > > > They have refused to let me speak with those making the decisions about > > what is provided on their public computers, much less to make an organized > > presentation to them. I was told that the decisions about software on the > > computers are made by the library board, not even by the IT staff. What is > > a good approach to get better results? I am at a loss as to how to get the > > library to emerge from the stone age into the age of the Cheka, much less > > that of the NSA, FSB, search engine profilers, botnets, packet sniffers, > > spyware, etc. > > One might think that providing the Tor browser would be a no-brainer, > but that's not the case in the Boise Public Library system. The Here, assuming that they have living brains may be unwarranted. > bureaucratic inertia is a very real thing, so good luck getting them to > install relays and exits too! First things first. > I have never asked them to do any such thing. All I've asked for was the clients. The answer has simply been "No" with no explanation whatsoever provided. Sorry to hear that the Boise library is also in the Dark Ages. :--( Be careful not to get burned at the stake. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] AU Relays and data retention
teor wrote: > > > On 4 Oct 2017, at 22:52, teor wrote: > > > > But I'm not a lawyer, so you should get your own lawyer. > > Or run a relay outside Australia. > > Or run an exit, because exits never know client IP addresses. > All they know is the destination. And internet destinations are > excluded from Australia's retention regime. > That might not be good enough. An Exit node can also be an entry point to the tor network. An Exit node can also even be a Guard node. Also, an exit node need not be an Exit node, depending upon the particular ExitPolicy involved. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Feedback wanted: letter to my university's library
William Denton wrote: > On 4 October 2017, Scott Bennett wrote: > > > Let me give an example. I have for at least ten years asked my local > > public library to provide a) a secure shell client, b) a secure web browser > > for ordinary use where anonymity is not a concern, c) a secure FTP client, > > and d) the TBB for use by those who desire anonymity. They have always > > refused to budge. They run an unsecurable OS on their public computers. > > They > > provide only Internet Explorer for web access. I'm unsure whether they > > still > > allow any FTP access at all. As you can imagine, they have severely limited > > the usefulness of their computers to the library patrons they claim to > > serve. > > I could not, for example, submit my on-line application to renew my flight > > instructor certificate via the library's computers. * I missed a beat here. The procedure for renewing a flight instructor certificate on-line includes an FAA requirement to "digitally sign" the web- based application for renewal. The procedure is a farce that bears no resemblance to what the security community understands to be a digital signature. That also means that the FAA may *not* be in compliance with the federal government's own standard http://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf) The fact that the FAA's system is not in compliance with the above referenced federal standard means that the FAA may possibly be in violation of the Computer Security Act of 1987 and/or the Information Technology Reform Act of 1996. But it was recommended to me by [identity withheld] that I *not* contact the FAA to point out this problem to them in hopes of getting them to correct it because they *allegedly* might revoke my instructor certificate for not "properly" representing the FAA's view of things. IOW, representing the NIST's [correct] view of things could get me punished by the FAA. I stress here that I do not know whether that recommendation was accurate in its claim, but I think it clearly illuminates the climate of fear and distrust that exists toward all levels of government in the USA these days. If simply posting this here gets my CFI revoked, I will (attempt to) let you know. (Actually, I'm not terribly worried, but I have to admit to the possibility.) > > They have refused to let me speak with those making the decisions about > > what is provided on their public computers, much less to make an organized > > presentation to them. I was told that the decisions about software on the > > computers are made by the library board, not even by the IT staff. What is > > a good approach to get better results? > > I fear there is nothing you can do. If they're like that, it's not going to > change until there's a new chief librarian or head of library IT. Public > libraries can be terrible for problems like this. When the right person is > in > the right job, they can move fast and experiment, but that's rare. When a > library thinks offering only IE is the right thing to do, Tor must terrify > them. I was afraid that would be the response a presumably honest, IT-aware librarian might give, but I didn't know until now. Sigh. Thanks for the clear answer. :-( FWIW, my guess is that the board is way too clueless to be terrified, but rather that they simply are so hostile to any change, especially when proposed by someone not a library employee, that they simply cannot permit it, regardless of any other considerations. That's, again, only my guess, but I'm somewhat attached to it by experience. :-> > > But if you can't speak to the public library board there's a problem much > bigger > than what they run on their computers! That is just not right. Public My thoughts exactly. > libraries have to be responsible to their public. Could your city councillor This is Illinois. "Governmental bodies" and "responsible to their public" are incompatible sentencemates here. Please try your luck again. (Hint: land (,re}development deals are often viewed favorably.) This is the state that requires budgets to be balanced, but where lack of *any* budget for nearly three fiscal years was not considered a breach of the state constitution. > help? The local newspaper? My city councilcritter has generally been unreceptive to my suggestions on all issues I have ever discussed with him. The local newspaper was bought up long ago by one of the media oligarchs. It is marginally useful for local news only, but not at all worth its price. Most people don't bother with it, so even if the handful of local reporting staff and editor were agreeable, it would likely matter not a whit. Much the
Re: [tor-relays] Feedback wanted: letter to my university's library
Alison Macrina wrote: > Scott Bennet> If he discovers that neither his campus library nor the > university as a > > whole is already officially running at least one relay, this may be a better > > way to teach them. If, rather than going for a relay, which is quite likely > > to scare them until they understand more and better about tor, AJ were > > instead to campaign to get the library to install the tor browser bundle > > onto its publicly available computers, that alone would be a terrific > > coup and might engender a great deal of student support for tor on campus > > over time. (The library would, of course, need to find a way to lock down > > the settings of the installed bundle, so that it couldn't be turned into > > a relay by users, but that should not be difficult to do.) If he succeeded > > in getting the tor browser bundle added to the library's most likely tightly > > limited list of applications available on its public machines, he could then > > wait a while to see what the staff members thought of it. If they decided > > after watching it in use for a while that it was a good thing to have made > > available to their users, you might then approach another department that > > operates a student computer lab to try to get TBB installed there. If the > > library employees liked it, they might give the prospective department a > > positive recommendation. If AJ played it right and it usually turned out > > well, he might eventually cover much of the campus with TBB installations. > > In any case, getting the TBB installed would educate far more people about > > anonymity and privacy issues than merely getting a relay installed that most > > people would never be aware of. > > This is a great idea, and the slides I shared in my last email could > help get this conversation started (the slides cover Tor Browser as well > as relays and other Tor stuff). If AJ is interested I can connect him > with other libraries I've worked with that have installed Tor Browser on > all of their public computers. > I, for one, am very happy to know that Alison and her organization are making those materials available. They have the potential to assist many people like AJ in making the public more aware of the issues and of the tools available to help it protect/recover its privacy and anonymity. Alison, do you also have materials on using HTTPS where available instead of HTTP? The dangers inherent in allowing Java or JavaScript to be enabled in one's web browser? Cookies? Tools like the HTTPSeverywhere and NoScript plug-ins for Firefox? The reasons for avoiding the use of telnet clients and which tools to use instead for remote logins? If not, they would make great additions, particularly pages that explain how to convince librarians about these matters? Let me give an example. I have for at least ten years asked my local public library to provide a) a secure shell client, b) a secure web browser for ordinary use where anonymity is not a concern, c) a secure FTP client, and d) the TBB for use by those who desire anonymity. They have always refused to budge. They run an unsecurable OS on their public computers. They provide only Internet Explorer for web access. I'm unsure whether they still allow any FTP access at all. As you can imagine, they have severely limited the usefulness of their computers to the library patrons they claim to serve. I could not, for example, submit my on-line application to renew my flight instructor certificate via the library's computers. They have refused to let me speak with those making the decisions about what is provided on their public computers, much less to make an organized presentation to them. I was told that the decisions about software on the computers are made by the library board, not even by the IT staff. What is a good approach to get better results? I am at a loss as to how to get the library to emerge from the stone age into the age of the Cheka, much less that of the NSA, FSB, search engine profilers, botnets, packet sniffers, spyware, etc. Disclaimer: I confess that I have no idea how prevalent my public library's attitudes and policies are among public libraries in the U.S. today, so I can't make any claims about widespread need for the sort of materials I'm asking about. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good
Re: [tor-relays] About relay size
teor wrote: > > > On 3 Oct 2017, at 08:52, Scott Bennett wrote: > > > > teor wrote: > > > >> > >> On 3 Oct 2017, at 03:07, Scott Bennett wrote: > >> > >>>>> In the meantime, I think it would be great to have IPv6-only relays, to > >>>>> avoid this kind of NAT-related issues. > >>>> > >>>> We'd love to make this happen, but the anonymity implications > >>>> of mixed IPv4-only and IPv6-only (non-clique) networks need > >>>> further research. Search the list archives for details. > >>>> > >>>Couldn't that be taken care of in the tor client code? For example, a > >>> client, having chosen a path through which an IPv6-only relay, could > >>> extend > >>> the path by one hop to tunnel through a node with both types of interface > >>> published? > >> > >> Yes, clients choose paths, and could choose them using these kinds of > >> restrictions. But current tor relay versions won't extend to other relays > >> over IPv6. Because we don't understand the anonymity implications of > >> restricting the next relay in the path based on the previous relay. Which > >> is why we need further research. > > > > Here's a procedure: if the next hop/destination does not use a protocol > > in common with the client/current hop, a dual-protocoled node must be > > interposed; else use the originally selected hop/destination directly. > > But is this procedure safe? Why might it not be safe? > And is it an efficient use of resources to add extra hops? Well, the additional hops ought to increase the safety. Wouldn't that have to be considered in an efficiency calculation? > (That is, is there a more efficient 3-hop algorithm that's just as safe?) > We need research to answer these questions. It seems to me that a) using a small number (i.e., 3) of hops reduces safety and b) a constant number of hops also reduces safety. Added hops to make an unpredictable path length ought to counter the alleged safety issues in choosing hops from a smaller pool. Also, unless the pool of dual-protocol hops is much smaller than the number of relays available, say, ten years ago, I don't see a reason to worry. (We were thrilled to have 900 - 1300 up at any one time.) > > > The client-to-first-hop situation is analogous to using a set of entry > > guards > > today, so that much should be okay. What do IPv6-only clients currently do? > > Choose a relay with an IPv6 ORPort. > (Or configure a bridge with an IPv6 ORPort or pluggable transport.) > > IPv6-only clients need to be manually configured using "ClientUseIPv4 0", > because this feature is still experimental. We think it will be safer when all > dual-stack clients use some IPv6 guards. > Dual-protocol clients connecting to dual-protocol entry nodes ought to choose randomly which protocol to use. Similarly, a dual-protocol relay extending to another dual-protocol relay ought to choose randomly which protocol to use for the connection. Both clients and relays would have the option of retrying the attempt via the other protocol in the event a failed attempt to connect. This ought to have been implemented already, but can still be done soon. > > Allowing IPv6 destinations today limits exit-hop selections to dual- > > protocol-capable exit nodes > > No, IPv4-only exits are still useful and used, because tor clients typically > send DNS names to exits. IPv6 DNS records typically have IPv4 as well, > so any exit will work. (There's a ticket for better handling of DNS that > only resolves to IPv6.) > > And in the rare case of address literals, the client can choose a capable > exit. (I think there's a ticket for this, too.) Okay, but those circuits are incidental to the circuit that a user wants, which is limited to dual-protocol-capable nodes. > > > which is like using an "ExitNodesIPv6" (if there > > were such a thing) line in torrc with a long and growing list of nodes. > > There are flags on SOCKSPort for IP versions, including PreferIPv6, which > is the default. > > > How > > long would that list have to be for the warning on the man page under the > > ExitNodes statement definition to become unimportant? How many were there > > when IPv6 destinations were first allowed? > > The situation isn't analogous. > > For safety, IPv6 exit destinations were turned off by default until we had > enough > IPv6 exits. Now they are on by default. I repeat: how many were there when IPv6 destinations were firs
Re: [tor-relays] About relay size
Roman Mamedov wrote: > On Tue, 3 Oct 2017 09:53:46 -0400 > teor wrote: > > > > For interposing dual-protocoled nodes along the way, how many do there > > > have to be for it to become "not too limiting"? > > > > This is one of the questions we need researchers to answer. > > I can't help but feel you are overcomplicating this. > > Clients create a circuit by randomly picking 3 nodes out of the all-nodes > pile, right? If all 3 happen to be IPv6-capable, then the circuit can go over > IPv6 and all is fine. If some of the 3 happen to be IPv6-only while others are > IPv4-only, the whole selection can be thrown away and repeated. > > That way IPv6-only relays could get some usage on a totally random basis, with > no compromises and no restraining "of the next hop based on the previous one", > not hurting anonymity. Clients just need to know which nodes are IPv4-only, > IPv6-only or dual-stack, to not attempt unworkable combinations, discarding > them instead. Looks good to me, Roman. I like it better than what I suggested, though adding some variability to path length had some appeal, too. Your way is certainly simpler. If tor ever starts to support TCP over UDP and/or TCP over SCTP, that will be a similar situation for a while, where some relays can do both and others are limited to one protocol or the other. > > And as there are more and more dual-stack or IPv6-only relays, the "throw > away" step will be needed less and less often. > Yet the same process could be applied to the TCP-over-non-TCP situation quite easily, too. I think I like your way even better now. My memory is a bit hazy on this, but I seem to recall watching a video file of Roger Dingledine giving a talk somewhere about tor long ago in which he commented happily that the tor network had grown to where it usually had at least 300 nodes up and running at all times of day. If 300 were enough for path selection before, aren't they still enough? Aren't there at least 300 IPv6-only nodes and at least 300 dual-protocol nodes now? And, of course, there must have been a time when the network was smaller than 300... Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] About relay size
teor wrote: > > >>>> A related question is can a relay with only an IPv4 address > >>>> published currently set an IPv6 OutboundBindAddress? > >>> > >>> Yes. This is useful for IPv6 exits without a fixed IPv6 ORPort address. > >>> > >> That's okay, but what if the node is an entry-and-middle node only? > >> > > Hmm. On second thought, it's *not* okay because it means that such a > > node cannot be a middle node because it could only connect to the IPv6 > > universe. Or the man page is wrong about OutboundBindAddress. Or there > > is something else amiss. > > "This option may be used twice, once with an IPv4 address and once with > an IPv6 address" > I see the problem. I'm running 0.3.1.6-rc at present, 0.3.2.1-alpha has been installed since I last started tor, and the timestamp on the man page says "Oct 2 2012". :-( Now I have to track down why the man page updates have not been getting installed. Sigh. Sorry for that bit of noise. The other questions remain, however, including the one above. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] About relay size
Scott Bennett wrote: > teor wrote: > > > > > On 3 Oct 2017, at 03:07, Scott Bennett wrote: > > > > >>> In the meantime, I think it would be great to have IPv6-only relays, to > > >>> avoid this kind of NAT-related issues. > > >> > > >> We'd love to make this happen, but the anonymity implications > > >> of mixed IPv4-only and IPv6-only (non-clique) networks need > > >> further research. Search the list archives for details. > > >> > > > Couldn't that be taken care of in the tor client code? For example, a > > > client, having chosen a path through which an IPv6-only relay, could > > > extend > > > the path by one hop to tunnel through a node with both types of interface > > > published? > > > > Yes, clients choose paths, and could choose them using these kinds of > > restrictions. But current tor relay versions won't extend to other relays > > over IPv6. Because we don't understand the anonymity implications of > > restricting the next relay in the path based on the previous relay. Which > > is why we need further research. > > Here's a procedure: if the next hop/destination does not use a protocol > in common with the client/current hop, a dual-protocoled node must be > interposed; else use the originally selected hop/destination directly. > The client-to-first-hop situation is analogous to using a set of entry guards > today, so that much should be okay. What do IPv6-only clients currently do? > Allowing IPv6 destinations today limits exit-hop selections to dual- > protocol-capable exit nodes, which is like using an "ExitNodesIPv6" (if there > were such a thing) line in torrc with a long and growing list of nodes. How > long would that list have to be for the warning on the man page under the > ExitNodes statement definition to become unimportant? How many were there > when IPv6 destinations were first allowed? > For interposing dual-protocoled nodes along the way, how many do there > have to be for it to become "not too limiting"? > > > > > A related question is can a relay with only an IPv4 address > > > published currently set an IPv6 OutboundBindAddress? > > > > Yes. This is useful for IPv6 exits without a fixed IPv6 ORPort address. > > > That's okay, but what if the node is an entry-and-middle node only? > Hmm. On second thought, it's *not* okay because it means that such a node cannot be a middle node because it could only connect to the IPv6 universe. Or the man page is wrong about OutboundBindAddress. Or there is something else amiss. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] About relay size
teor wrote: > > On 3 Oct 2017, at 03:07, Scott Bennett wrote: > > >>> In the meantime, I think it would be great to have IPv6-only relays, to > >>> avoid this kind of NAT-related issues. > >> > >> We'd love to make this happen, but the anonymity implications > >> of mixed IPv4-only and IPv6-only (non-clique) networks need > >> further research. Search the list archives for details. > >> > > Couldn't that be taken care of in the tor client code? For example, a > > client, having chosen a path through which an IPv6-only relay, could extend > > the path by one hop to tunnel through a node with both types of interface > > published? > > Yes, clients choose paths, and could choose them using these kinds of > restrictions. But current tor relay versions won't extend to other relays > over IPv6. Because we don't understand the anonymity implications of > restricting the next relay in the path based on the previous relay. Which > is why we need further research. Here's a procedure: if the next hop/destination does not use a protocol in common with the client/current hop, a dual-protocoled node must be interposed; else use the originally selected hop/destination directly. The client-to-first-hop situation is analogous to using a set of entry guards today, so that much should be okay. What do IPv6-only clients currently do? Allowing IPv6 destinations today limits exit-hop selections to dual- protocol-capable exit nodes, which is like using an "ExitNodesIPv6" (if there were such a thing) line in torrc with a long and growing list of nodes. How long would that list have to be for the warning on the man page under the ExitNodes statement definition to become unimportant? How many were there when IPv6 destinations were first allowed? For interposing dual-protocoled nodes along the way, how many do there have to be for it to become "not too limiting"? > > > A related question is can a relay with only an IPv4 address > > published currently set an IPv6 OutboundBindAddress? > > Yes. This is useful for IPv6 exits without a fixed IPv6 ORPort address. > That's okay, but what if the node is an entry-and-middle node only? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Feedback wanted: letter to my university's library
Alison Macrina wrote: > Hi AJ, > > Thank you for supporting Tor! I think it's a great idea to try to work > with your university library to run a relay. I run the Library Freedom > Project which helps libraries understand and use privacy tools > (libraryfreedomproject.org). I can give you some advice based on my > experience. > > William Denton: > > On 1 October 2017, AJ Jordan wrote: > > > >> However I've just started college at the University of Rochester, > >> which obviously presents a great opportunity to set up a relay on a > >> really great network. I'm planning to reach out to the library with > >> the following email and would love some feedback: > > > > Scott Bennett had excellent advice, > > +1 > > > Academic libraries can be very experimental in some of their work, but > > they are generally risk-averse. (This is good, because they're in the > > business of preserving knowledge and cultural artifacts for decades and > > centuries.) There is, I'm afraid, close to zero chance they would let a > > non-employee student run a server on their network---and running Tor, > > even a non-exit relay, makes the chances even lower. > > > > However, don't give up. I suggest thinking about this as a long-term > > project that could get you involved with the library, faculty and campus > > IT. There must be people on campus interested in privacy issues, who > > know about Tor, and perhaps who have been thinking about running a > > relay. These people could be librarians or they could be professors or > > grad students in political science, communcations, journalism, computer > > science, privacy studies, etc. Find out who they are and approach > > them! Perhaps there is a student club interested in the same > > issues---if not, you could start one. Students and student groups > > advocating for a Tor relay or exit, while demonstrating the importance > > of Tor and how it fits in with the library's and university's mission, > > would very much help the project be successful. > > William's advice is good. You definitely need to begin by building a > relationship with the library. Don't be discouraged by the amount of > work this may take; the payoff might end up being a cultural shift > wherein the library, university IT, and CS departments all work on this > as a project together! > > You'll want to approach the library by showing that Tor is an excellent > way to uphold the values of librarianship, which are privacy, > intellectual freedom, and access. Really, be explicit about it; don't > assume that they'll just get why you think it matters. Here's something > I wrote about intellectual freedom + Tor Browser a while ago and you can > borrow the arguments I've made: > https://www.scribd.com/document/272919852/Alison-Macrina-The-Tor-Browser-and-Intellectual-Freedom-in-the-Digital-Age > > As William said, libraries are mostly risk-averse, so you also need to > be ready to answer their questions about legal and technical concerns. > LFP has collected some resources to help with all of that here: > https://github.com/LibraryFreedom/tor-exit-package/blob/master/resources.md. > > Before you email the university librarian, I'd start by talking to some > of the regular academic librarians about your ideas and gauge their > responses. Ask them if they've heard of the Library Freedom Project and > feel free to send them any of our resources. See if they think the > administration would be receptive to you offering a presentation about > Tor to library staff (even better if you can make it open to students > and faculty, too, because that can get you more support). You are > welcome to adapt these slides for that presentation: > https://libraryfreedomproject.org/allabouttor/. Make sure to show them > this academic library that has used their Tor relay as a teaching tool > for students: > https://boingboing.net/2016/03/16/first-ever-tor-node-in-a-canad.html. > I second all of the above by both Bill and Allison, and I am grateful that they were able to express better and in far greater detail much of what I was trying to say, while adding their own ideas in similar detail. Reading Bill's and Allison's followups stimulated an idea of another, and possibly better, approach that I hope AJ will consider and that may even be more likely to be approved and have greater effect in the long run. If he discovers that neither his campus library nor the university as a whole is already officially running at least one relay, this may be a better way to teach them. If, rather than going for a relay, which is quite li
Re: [tor-relays] About relay size
teor wrote: > > > On 2 Oct 2017, at 16:54, Santiago wrote: > > > >> El 02/10/17 a las 13:19, Scott Bennett escribi?: > >> grarpamp wrote: > >> > >>>> On Mon, Oct 2, 2017 at 3:53 AM, Santiago wrote: > >> ? > >> > >> Huh? What kind of ISP NATs its customers' connections? Your ISP > >> should be assigning your machine/router a legitimate, unique IPv4 address. > >> The assignment is often, even usually, a temporary assignment via DHCP, > >> but it should not be a private address. If NAT is a factor, that should > >> happen at the boundary of your own private network, not at an ISP's > >> facility. > > > > It seems that a French ISP was also planning to share an IPv4 address > > per four costumers. > > > > ? > >> ... One typical problem with running tor > >> on a NATed machine behind such a device is that the NAT table grows until > >> all > >> of the real memory on the device has been consumed and there is no more > >> room > >> for new NAT entries. > > > > I am not currently able to replace the modem/router my ISP provides. But > > I'd plan to give it away in the future. > > > > In the meantime, I think it would be great to have IPv6-only relays, to > > avoid this kind of NAT-related issues. > > We'd love to make this happen, but the anonymity implications > of mixed IPv4-only and IPv6-only (non-clique) networks need > further research. Search the list archives for details. > teor, Couldn't that be taken care of in the tor client code? For example, a client, having chosen a path through which an IPv6-only relay, could extend the path by one hop to tunnel through a node with both types of interface published? A related question is can a relay with only an IPv4 address published currently set an IPv6 OutboundBindAddress? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] About relay size
grarpamp wrote: > >> Or instead of router mode, try bridge mode feeding into any old pc running > > Noting that even some crappy hardware will still fall over when put in its > so called "bridge" mode, which should just be some packet buffering > between the wires and their encodings, but it's obviously still looking > at the traffic above layer2. So you may still need to swap out hardware. > Absolutely. Another reason to avoid electronics store routers for tor (or many other things) is the information in recently exposed documents that the CIA started invading those devices and, where possible, "upgrading" their firmware as standard practice at least a decade ago. > > because there is secondary storage (HDD and/or SSD), paging > > is available if the routing functions' memory needs grow larger than the > > Sure, but there's no free substitute for RAM, and you probably don't want > packets burning a hole in your SSD. Add more RAM if not maxed out. My point was intended to be only that having a regular computer handle the routing means it doesn't have to die if available RAM be exhausted, i.e., not an argument for speed, but rather for survival under unusual loads. Now that I'm more awake than when I wrote that, though, I realize I don't recall whether routing and NAT tables and mbufs are page-fixed or pageable anyway. :-( It's still better to have a router that you own and the CIA [probably] doesn't. > disable swap, boot USB, set read-only, use small ramdisks for write paths. > If used RAM for a used PC isn't in budget or isn't enough, then maybe > spindle, but it won't be as fast. And eventually CPU or interrupts or i/o > get swamped. Then you put a newer PC that can hold proper amounts > of RAM, CPU, etc. Very true. The device need also not be dedicated to just those functions. Many people would prefer to stick a heavily used relay on their border gateway machine to keep its traffic load off their LANs anyway. Also, if FreeBSD is used, kernel memory for routing tables, NAT tables if used, and mbufs should be allocated from 4 MB superpages, allowing the routing to run very fast. And with an electronics store router, you don't have the kernel configuration information available to look at, whereas you do have that and all the rest as well if you install the OS yourself. Let's also not omit the ability to apply security fixes as they become available, where the store-bought boxes would be running obsolete and unsafe OS in their firmware, probably by the time the store sold them. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] About relay size
grarpamp wrote: > On Mon, Oct 2, 2017 at 3:53 AM, Santiago wrote: > >> And you can only have 2 tor instances per public IPv4 address. > > > > Why? Is there any place where I can find this kind of info? > > Read the archives of this list linked at the bottom of every message. > As an operator you'll find lots more interesting subjects there too. > > > Maybe it's another issue, but I have recently tried to run a second > > relay behind the same IPv4 address than my first relay, and the > > connection quality strongly diminished. I suppose my ISP equipment was > > not able to handle the two relays on NAT, but I would need to > > investigate further. Huh? What kind of ISP NATs its customers' connections? Your ISP should be assigning your machine/router a legitimate, unique IPv4 address. The assignment is often, even usually, a temporary assignment via DHCP, but it should not be a private address. If NAT is a factor, that should happen at the boundary of your own private network, not at an ISP's facility. > > Lots of hardware for use in the home, whether ISP provided or bought > from wherever by the user, has been known to fall over under load, > cable / dsl / fiber modems, whether in bridge or router modes, wifi, etc. > > For tor you need to test with tens to hundreds of TCP connections > or more in parallel. The simple online "speedtests" don't do that. > One way is to load up increasing numbers of opensource Unix iso's, > conference videos, whatever... into whatever torrent client and watch > the stats. If upon passing the expected / required number of connections, > it starts falling significantly off the maximum recorded speed, never recovers > when unloaded, locks up, reboots, melts / smokes / combusts, etc... > then try another brand. I'll second the above comments. Most of those little router boxes are running some form of LINUX or FreeBSD as an embedded configuration, which includes swapping and paging being disabled due to the absence of secondary storage. All of them have limited RAM. One typical problem with running tor on a NATed machine behind such a device is that the NAT table grows until all of the real memory on the device has been consumed and there is no more room for new NAT entries. > > Or instead of router mode, try bridge mode feeding into any old pc running > [Free]BSD / Linux to do the functions of routing, wifi, firewall, nat, > dhcp, dns, > etc... this may often perform better and give more flexibility. Yes, and because there is secondary storage (HDD and/or SSD), paging is available if the routing functions' memory needs grow larger than the available real memory. Home electronics store routers cannot hold a candle to a full OS with a decent packet filter. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Feedback wanted: letter to my university's library
"Tor Node Admin @ SechsNullDrei.org" wrote: > Hi AJ, > > First, thank you for supporting Tor! > > Second, you're smart to contact the library, as IT would immediately shut > down the idea as they don't want to receive more abuse emails than they > already do (I know we did when I worked in academia). An additional > resource you may wish to research is the https://libraryfreedomproject.org/ > by Alison Mecrina. The resources available on this site speak directly to > librarians on such issues. Good luck! > AJ and any others in similar situations, I would add a few comments, beginning with the style of communications with your university library. When you wish to ask for some special consideration, find out whom you need to talk with about your request, and then make an appointment to visit his/her office in person. Sending an email message out of the blue and beginning it with "Heya!" is utterly inappropriate and is not at all likely to get you anywhere. You must act--and write--like an adult, and formality in your initial written communications is essential. They are not going to let someone acting like an ill-mannered adolescent run a server on their department's equipment. If you come across as a responsible adult who makes a good impression and provides a good case for satisfying the request, a case that looks good from the university's point of view, which is to say, convincing them that it would benefit the university in some way, then you may get somewhere with them. If you convince them, however, be prepared to find out that the staff has decided to run the relay itself. The department may well have to run it past the university's legal department. Because you are not acting as a university employee, either or both departments may not be willing to make the university liable for possible consequences of a non-employee's actions. I know that, as a systems programmer at the university here, I would never have let anyone out of our group, much less outside of our department, do anything like running a relay on a machine under my responsibility. I wouldn't have risked having to clean up someone else's mess, and my boss would have been apoplectic at the idea. The security issues would have overwhelmed everything. OTOH, you might still get lucky. It would definitely be worth your time to find out whether the university (e.g., through its computer center or library) or some other individual, department, or college office is already running one or more relays. Faculty members at some universities have been known to get special arrangements to run tor relays, even exits, on university- owned equipment. At some schools, faculty members would not likely even be questioned about it. Note, for example, that many individual faculty members and sometimes even (paid) graduate assistants run web servers to publicize their research and results. Note that any networking staff will want to know how much of the available network capacity your networking program (in this case, tor) would require if the decision were to allow it. Asking to run an exit is almost certain to subject your request to legal department scrutiny, so you might consider running a middle/entry node first for some time, say six or twelve months, before asking to upgrade your relay to an exit relay. That would give time to establish your skill at managing a relay, responding well and quickly when problems occur, to become personally known to the staff, and so forth. Once you've established a good reputation with them, they are more likely to oblige you. In any case, best wishes for your attempt. Please let us know whether you pull it off and, if so, what you did that succeeded. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Individual Operator Exit Probability Threshold
dawuud wrote: > > > I think it is worth remembering that there isn't evidence there is a > > global passive adversary at the moment, even if certain agencies and > > organizations clearly aspire to be one. > > Quite so. It is well established that these so called agencies do not > aspire to be passive. Or perhaps you simply typed the word "passive" by > shear force of habit and instead meant to convey "suffiently global > adversary". :) Curiouser and curiouser! So, dawuud, you imply that habit has a velocity vector(!) and that it changes over distance(!!) in one or more dimensions, and thus exerts a shear force on...what? A typist's fingers and hand placed within the volume where shear is present? Strong enough, perhaps, to roll them up and maybe even break a wrist? Well, I guess one lives and learns! (Hint: you misplaced a homonym.:) Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] abuse email for non-exit relay (masergy)
teor wrote: > > > On 22 Sep 2017, at 23:03, relay 000 wrote: > > > >> Someone is using the hidden service rendezvous protocol to ask non-exit > >> relays to scan non-tor IP addresses. > > > > wow, people can misuse my *non*-exit relay to scan (aka send a TCP SYN > > packet) other systems on the internet? > > Yes. > > But please don't worry. Receiving unsolicited TCP connections is a > normal part of running a server on the Internet. And anyone who sends > unsolicited spammy emails in response lacks a sense of irony. > > Here's how the Tor rendezvous protocol can be used like that: > > People can pretend that they are a client or onion service that's > connected to a particular relay address. > > And then they can ask your relay to extend to that pretend relay > address. There's no requirement that the relay is in the consensus that > your relay has. And so your relay tries to establish a TLS connection, > may or may not succeed, but definitely fails at the authentication step. > > And then it tells the client it failed. Without providing much info at > all. So it's pretty useless, honestly. > > The alternative would be to require that every relay used in the > rendezvous protocol is in the consensus. But which consensus? > * the consensus that the client has > * the consensus that the service has > * the consensus that the relay extending to the intro point has > * the consensus that the relay extending to the rend point has > > It gets complicated fast. > There's another, more obvious reason, I think, than hidden services. Consider what happens during relay startup. The initializing relay attempts to build a number of circuits that connect back to itself for reachability and data rate testing, yet its descriptor may well not be in any relay's cached-descriptor* files, much less in either consensus document. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] bug messages in my relay's notices.log
teor wrote: > > > On 22 Sep 2017, at 21:24, Scott Bennett wrote: > > > > nusenu wrote: > > > >> https://trac.torproject.org/projects/tor/ticket/23551 > >> > > Thanks for the pointer. I'm glad it has been reported, but I still > > have no sense of what in tor is malfunctioning because the compression > > has failed. > > It's ok, we're also somewhat clueless. > But we're good at tracking down weird bugs. > Just give us some time. > > > Are user cells lost? Do user connections get broken when > > this happens? > > This error occurs during directory document decompression using zstd. > > There's a loop that calls out to a function that decompresses some > data. The loop is meant to terminate when decompression finishes. > But if decompression doesn't make any progress, it terminates with > an error. (Otherwise, we'd just keep spinning endlessly, wondering > when the decompressor was going to cough up something useful.) > > The relay discards the received document, and tries another mirror. > At the moment, most mirrors don't support zstd, so the relay will > likely get a zlib-compressed document, decompress it successfully, > and continue merrily on its way. And the users don't even notice. Oh. Okay. > > Although, I wonder if we should do more to guard against network-wide > breakage like this. It would be awkward if all relays ran into an > error like this at the same time. > > Fortunately, zstd isn't available for all platforms and architectures. > So as long as we steer clear of a platform monoculture, we'll be fine. > (That said, this particular bug is on BSD. But people should still run > relays on BSD, so we get a good OS mix.) Are the BSDs the only systems that have zstd available? > > > Has the error been reported to the project(s) that develop > > the compression libraries? > > Well, it's kinda rude to go straight to blaming zstd. > First we have to work out what the issue is, and whose code it's in. I guess I misunderstood the traceback. Apologies. > > Although, I have to say that I wouldn't surprise me if we have a zstd > version incompatibility here. Or maybe truncated input data? Here's what I have. Sep 23 00:36:33.957 [notice] Tor 0.3.1.6-rc (git-efc306c59aa9ee1a) running on FreeBSD with Libevent 2.1.8-stable, OpenSSL 1.0.2l, Zlib 1.2.8, Liblzma 5.2.2, and Libzstd 1.3.1. > > Truncated input data could definitely explain why zstd can't make > any progress, but isn't returning an error code. In that case, it's > our fault for not giving zstd enough data to work with. (And not > failing the loop quietly?) Now, that's interesting because last night my Comcast connection was down and back up a few times beginning around 11:30 p.m. until it went down just before 1:30 a.m. and stayed down for about an hour and a half. During that time, of course, I called Comcast to get whatever information (as usual, not much) I could about the outage. They said their crew was out working on it and that it affected 327(?) customers in the area, so I surmised something was gradually failing (such nighttime down and back up sequences have been becoming more frequent recently) and after my complaints a couple of days earlier, they figured out what it was and were replacing it. The error messages started last night after they were done and the line was working again. If there's some leftover file that's dirty from having been cut off during transmission that continues to trigger the error messages from time to time, I'll get rid of it if someone can tell me what to look for. > > Because if it were corrupt data, it really should return an error: > http://facebook.github.io/zstd/zstd_manual.html#Chapter9 Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] bug messages in my relay's notices.log
nusenu wrote:
> https://trac.torproject.org/projects/tor/ticket/23551
>
Thanks for the pointer. I'm glad it has been reported, but I still
have no sense of what in tor is malfunctioning because the compression
has failed. Are user cells lost? Do user connections get broken when
this happens? Has the error been reported to the project(s) that develop
the compression libraries?
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] bug messages in my relay's notices.log
I've gotten a couple of these sets of messages in the last few hours. Sep 22 05:13:17.366 [warn] tor_bug_occurred_: Bug: src/common/compress.c:576: tor_compress_process: Non-fatal assertion !((rv == TOR_COMPRESS_OK) && *in_len == in_len_orig && *out_len == out_len_orig) failed. (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: Non-fatal assertion !((rv == TOR_COMPRESS_OK) && *in_len == in_len_orig && *out_len == out_len_orig) failed in tor_compress_process at src/common/compress.c:576. Stack trace: (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x11a63fc at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x11abbd4 at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x11ab0fa at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x11ab540 at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x114ecfd at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x1126c43 at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x106db94 at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x1c1e8564e at /usr/local/lib/libevent-2.1.so.6 (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x1c1e80a1e at /usr/local/lib/libevent-2.1.so.6 (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x106fa55 at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x1071e89 at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x106d899 at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) Sep 22 05:13:17.366 [warn] Bug: 0x106d791 <_start+417> at /usr/local/bin/tor (on Tor 0.3.1.6-rc efc306c59aa9ee1a) My outdated system is running FreeBSD 10.3-STABLE. I've never seen these messages before that I can remember. What causes them? What, if anything, should I do about them? And what fallout do they cause for users? Thanks in advance for any relevant information! Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Some Dir Authorities blocked
Andreas Krey wrote: > On Sun, 17 Sep 2017 08:13:43 +0000, Scott Bennett wrote: > ... > > connections to other relays somewhere, those of us using packet filters > > could > > include the rest of the missing addresses in aid of the connectivity you > > want. > > I really don't see what the point is in this filtering. Any attacker > can just fire up its own relay and attack from there once its address > in the consensus. > Attackers/probers, at least of my system, do not appear to be aware of tor, or of most other applications, for that matter. They assault mainly any open ports they find. The IP addresses from which such attacks connect are either direct attackers or are systems whose security is inadequately attended to and have been commandeered by attackers. The purpose of the filter rules is to deny the attackers access to *anything* on my system to the degree to which they can be identified. The source IP addresses belonging to tor relays are a special case because attacks might exit through them or might be running on those systems, but tor connectivity must be maintained. So the rules are a compromise. They allow inbound connections only to the ORPort and the DirPort from all addresses known to belong to tor relays in order to maintain that connectivity. Any such attacks on the ORPort or DirPort from those addresses are a) likely to be rare, if only because the time delays in going through tor to attack tor (or anything else) slow the attacker's automation to a degree usually not accepted by the attacker's software and b) also rare because the tor relay operator community tends to be significantly more security-conscious than the vastly broader community of Internet users at large and are therefore far less likely to allow stuff on their systems running relays to engage in such attacks in the first place. Addresses in the list of offending addresses that are not also in the list of tor relay addresses are blocked from attacking tor or any other services on my system. The problem here stems from allowing secret addresses to belong to tor relays inasmuch as connections from those secret addresses cannot be protected in the fashion described above. FWIW, I had composed two or three nights ago a fairly detailed response to teor's arguments against what I had previously posted, when my Comcast connection went down several times in rapid succession, destroying my SSH session. Unfortunately, virecover on SDF's servers does not actually produce usable recovery files, so the message, which was almost ready to be sent, was lost. I managed to copy many blocks of text from several pages of the screen buffer into a file on my system, but their order is scrambled. As soon as I can spare the time to reconstruct my arguments and proposed solutions, I will do so, but at the moment I have urgent personal matters to attend to for a few days. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Some Dir Authorities blocked
Roger Dingledine wrote:
> On Sat, Sep 16, 2017 at 11:44:41PM +, dawuud wrote:
> > > Your only option would be to ask your ISP to uncensor the internet,
> > > unfortunately. Tor requires that all relays are able to contact all
> > > other relays, and those which cannot participate in the network.
> >
> > I think you meant to say:
> > "Tor requires that all relays are able to contact all directory authorities"
>
> Actually, no, we want it to be the case that all relays can reach all
That does not contradict what dawuud wrote, as can be clearly seen.
What you want and what tor currently requires are not the same thing at all.
> relays. The less true that becomes -- that is, the less clique-like
> the network topology becomes -- the more complicated the anonymity
> measurements become, and that is potentially quite bad.
That is why OutboundBindAddress values need to be published somewhere.
Those of us who use packet filters in defense of our systems and/or our LANs
can allow connections to our relays' ORPorts from otherwise blocked addresses
if we know which of those blocked addresses have tor relays that may be trying
to connect. Without that information, the only way you will ever get what you
want is by turning away volunteers.
Roger, you may recall our earlier discussion about this subject on this
list. Since that time, I have used the information you provided then to
automate the inclusion of the exit relay address list, and I have spot checked
some of those addresses to make sure they were indeed landing in the TorRelays
table used by my pf rules. I have not been able to tell whether or how much
the exemption of exit addresses that were not published by the relays
themselves has improved anything, but the change was made. If you could get
tor to publish any otherwise unpublished addresses it uses to make outbound
connections to other relays somewhere, those of us using packet filters could
include the rest of the missing addresses in aid of the connectivity you want.
One possible place to publish them might be the extra-info documents, which
would avoid any alteration of the directory entry format. We relay operators
could then use DownloadExtraInfo to get those documents, from which we could
extract the OutboundBindAddress values and merge them in with what we already
compile. However, the network load involved in downloading extra-info for all
relays on a frequent basis could be avoided if that processing were done on a
tor project machine and the address list posted on the project web site as is
currently done for the exit relay list.
However you decide the addresses should be made available, it must be
included in any effort to make possible the full connectivity you envision.
>
> > Tor certainly does NOT require all relays can contact all relays. In
> > fact, the network is *very* partitioned... but as of the past few
> > months I haven't put any energy into proving this; although I do have
> > some mostly finished twisted python code to make all two hop tor
> > circuits and records circuit build failures and circuit build timeouts.
>
> This is a great research area that it would be good to see some
> attention for.
>
> In particular, if there are specific relays that are doing especially
> poorly for connectivity, we should work with them to try to get them to
> fix it, and if it's unfixable, downgrade their weights and/or boot them
> from the network.
1) It's not necessarily a relay operator's fault.
2) At least one problem can be fixed by supplying the missing data, as
described above.
>
> See also
> https://trac.torproject.org/projects/tor/ticket/12131
> ("Measure connectivity patterns between relays")
> and
> https://trac.torproject.org/projects/tor/ticket/19068
> ("Write and run a clique reachability test")
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] HOW-TO: Simple DNS resolver for tor exit operators
Igor Mitrofanov wrote:
> If it's important enough to do on a single relay, it's important
> enough to do it across the entire network. I bet there are, and will
> always be, plenty of exit node operators not reading this email list,
> or not planning to do anything, or not configuring everything
> properly, etc.
>
I was not presenting an argument for or against anything. I was merely
pointing out to other FreeBSD users that a basic and generalized caching tool
existed in the base system and was very easy to use. Obviously, that piece
of information is irrelevant to anyone on another OS, but FreeBSD systems have
it already installed, and many of their operators are likely to be unaware of
it. I was running FreeBSD for four or five years before I stumbled across it
myself. The system works just fine without it, but putting it into use has the
potential to speed up various system functions, name-to-address resolution
being one of those.
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] HOW-TO: Simple DNS resolver for tor exit operators
Ralph Seichter wrote: > On 12.09.17 23:43, Roman Mamedov wrote: > > > > I take it you're being ironic? > > > > Guess I failed at doing that well, if you had to clarify. (Or maybe > > you didn't read my entire message.) > > I did read it. Just the pitfalls of non-verbal communication, and I'm > also not a native English speaker. ;-) > > > Running your own authoritative nameservers is laudable as well, but the > > current discussion is about recursive resolvers. You know, the likes of > > 8.8.8.8 and the ones your ISP runs for their clients "to reduce traffic". > > If you read *my* messages in this thread, you'll find that I am fully > aware of this. I even mentioned Google's infamous resolver by IP. :-) > I came across one ISP so far which does not provide resolvers for their > customers but points resolv.conf to Google's servers. Not good. > > > Note that 'dnsmasq' won't do, that's just a caching proxy to a fixed > > set of a few upstream DNS resolvers; you need 'unbound' which IS a full > > independent DNS resolver itself. > > Yeah, I use Unbound and BIND myself, with the former of course being > much more frugal in terms of resource requirements. Easy to set up, too. > I'd like to add a note here for FreeBSD users. In addition to unbound or any of the other resolvers available in the ports tree, DNS queries for name-to-address resolution can be further reduced by a small caching utility that is in the base system, called nscd(8). Check the man pages for nscd.conf(5) and nsswitch.conf(5) to see how easily you can configure its use. nscd can also cache other, non-DNS queries' results as well (e.g., NIS). After setting up nsswitch.conf and nscd.conf (just a few lines each), remember to add a line that says, "nscd_enable=YES", to /etc/rc.conf and then (as root) give the following command. # service nscd start Note that the rc.conf entry will take care of starting nscd(8) after a reboot. The command shown above is only necessary when starting nscd at other times. nscd's caching service gets inserted between the resolver(3) and its queries of local DNS caches or distant name servers, and it is quite fast, but it serves only the machine it runs on. Further, it maintains per-user caches for each type of data. Any user can flush his cache of one type of data or all types of data. root also has the option of flushing all of the per-user caches by type of data or all types of data. Here is an example of an nscd configuration (nscd.conf). threads 4 enable-cache passwd yes # enable-cache group yes enable-cache hosts yes enable-cache services yes enable-cache protocols yes enable-cache rpc yes enable-cache networks yes suggested-size hosts 2111 keep-hot-count hosts 4096 positive-policy hosts lfu suggested-size services 1123 And here is nsswitch.conf to go with the above. group: files group_compat: nis hosts: cache files dns networks: cache files passwd: cache files passwd_compat: nis shells: cache files services: compat services_compat: nis protocols: cache files rpc: cache files Note that the only lines in each that pertain to the current discussion are the lines that refer to hosts. The rest are for caches of other data. As you can see, configuring this high-speed, local-service-only caching daemon is trivially easy and brief and requires installation of *no* other software. It can be used with or without a caching name server or other caching resolver software. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Rate setting in tor
teor wrote:
> Also, these settings apply to *Relayed* traffic only.
>
> If your tor is acting as a client, it will happily exceed these limits.
> As well as SOCKSPort traffic, client traffic also includes directory
> document downloads, and any other traffic that originates at your relay.
Directory fetches used to be counted as part of the relay operations
traffic. When did that change, please?
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Force OpenSSL AES-NI usage on a VPS without the AES CPU flag passthrough
nusenu wrote: > Roman Mamedov: > > But what if you need to > > force-enable it. Turns out the syntax working for that is simply: > > > > OPENSSL_ia32cap="+0x202" > > > Do you know what platforms and openssl versions support this? > And is the same available for libressl, which is now the default SSL package on a growing number of systems? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] OS diversity of tor relays (was Re: Relay uptime versus outdated Tor version)
l, and Windows was an endless source of maddening frustration. IOW, I feel like I got to know and understand tor on FreeBSD and only then began running a relay. Most Windows users are unlikely to have job experience doing system administration of any kind. They may know how to use its GUI and a number of applications well enough for what they do, but they generally are not programmers or sysadmins. I'm not especially surprised that a much smaller proportion of relays today is running on Windows systems. Just to understand tor's man page requires some technical understanding of operating systems and how they work internally, as well as how they look from outside. The average Windows user probably doesn't have the background to feel confident about diving in to setting up a relay. The tor browser bundle once defaulted to configuring and running a relay, although there was a check box that could be unchecked to prevent an instance from being configured as a relay. Times have changed. :-) My claim still stands. If someone would like to run a properly designed survey to disprove it, be my guest. I would be most interested in the results. In the meantime, I repeat my point that not having a tor browser for an operating system is not the way to go if you want more relays on that OS. People use whatever OS they use and for their own reasons. If you want them to run a relay on their machines in addition to what they normally run, they are not likely to switch OS just to try out tor, much less to fulfill your wish. If you want them to become familiar with tor to the point where they might run a relay, give them a browser they can use on their own system. IOW, target the people who are already using those OS, with the caveat that the technically ignorant are far less likely to tackle running a relay (the older label in the documentation being "server", which probably deters even more people from running relays because of feeling it would be too hard or over their heads). Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] OS diversity of tor relays (was Re: Relay uptime versus outdated Tor version)
Ybslik wrote: > I have been following the emails with intrigue. I run a > > [details omitted --SB] > > My relay is called Ybslik - it's been up and down due to my inability to > understand how to configure things within the terminal. > > But with the help from StackExchange I have been able to complete some > tasks. > > My small relay is now working ok and I hope adding to the Diversity of Tor. > Thank you for running a relay. Duncan has already pointed out that choosing one of the two most heavily represented relay OS reduces, rather than increases, relay OS diversity. I thank you for your intent, even though you apparently misunderstood what was meant by diversity. In any case, I commend your bravery and energy in taking up a system new to you to run tor, but I also recommend you study security issues and features closely in any OS with which you are not yet intimately familiar but choose for running a relay. Having written that, if you someday choose to tackle learning yet another OS, please consider running a relay on one of the underrepresented relay OS. See, for example and in merely alphabetical order, Bitrig, DragonflyBSD, FreeBSD, Illumos, MINIX, MirOS, OpenBSD, NetBSD, NextBSD, OpenSolaris, TrueOS (bleeding-edge FreeBSD specially packaged for novices and the only slightly involved:^), or any other OS on which tor builds properly yet is currently poorly represented among existing tor relays. If I'm not mistaken, among the four largest BSD projects (i.e., FreeBSD, DragonflyBSD, NetBSD, OpenBSD), NetBSD is the most weakly represented among tor relays, which is too bad (IMO) because it is a very high-performance system that runs on damned nearly every kind of device with some kind of CPU chip in it, which makes it terrific for increasing hardware architectural diversity among relays, too. DragonflyBSD is probably second rarest of those four in the tor relay population. Learning these OS can have other, peripheral benefits, too. For example, you will find packet filters, file systems, scheduling and dispatching algorithms, I/O subsystems, virtual memory models, virtual machine support, etc. that are not found in Windows or LINUX. That is not to say that they are necessarily better than what you might find in LINUX, but they demonstrate a lot of different, creative, and interesting ideas about how to do things that you don't get exposed to by restricting your experience to the most heavily used OS. IMO, though, they are almost always better than what is available to choose from in Windows. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] OS diversity of tor relays (was Re: Relay uptime versus outdated Tor version)
either use portmaster or poudriere to build them and install the newly built versions. A lot of people do something similar for updating the base system from source in the other BSDs. The FreeBSD developers are in the process of setting up large blocks of the base OS as packages that can be automatically updated just like any packages built from ports, although I don't think that's quite ready for prime time yet. NetBSD may already have this capability; I just don't recall. Relay operators do not come into existence like virtual particle pairs from a vacuum. We're talking about humans, and those come from real-life, individual contexts. If you want to recruit them, then understanding and accepting that fact are critical to doing it successfully. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] OS diversity of tor relays (was Re: Relay uptime versus outdated Tor version)
Duncan wrote: > In theory hot-patching kernels is a great idea. > > However, they're technically not loading a new kernel. Something like > kexec in theory lets one load a new kernel. > > Furthermore, these hot-patching programs usually only support Linux. If > we want to increase the diversity of the Tor network, as we most > certainly should, then we need more BSD relays, so these hot-patching > programs don't cut it. > The tor project has made the point that OS diversity is important, but it has failed to show the courage of its conviction. It commits great effort to maintain a "safe" tor browser for the OS for which tor relays currently abound, yet still offers no version of that browser to entice *BSD, Solaris, MINIX, or other OS users to run tor relays. Instead, such users are apparently expected either to use clearly unsafe browsers or to run VMs of other than their native OS to run a safe browser. The tor community is thus very lucky for what diversity of relay OS currently exists. I've pointed this problem out several times, but to the best of my memory, none of the tor developers has ever responded on this issue. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] New Here
Duncan wrote: > > Have been running a mirror for a couple of years and haven't had much > > time to dedicate to and exit node but have finally committed to it. > > Found a provider that doesn't seem to have a cap on data and it will > > cost me about five euros a month. > > > > Question I have is two fold. First I find that atlas.torproject.org > > sometime says the node is down but its running and connected. Secondly > > how dose the bandwidth throttling work. I have set it to 300kbits with > > 500kbit burst but I can see the in/out traffic staying over 1mb for > > large periods of time and bursting to 7/8mb. Its not a problem just > > curios. > > For bandwidth throttling, the option "BandwidthRate" applies to both > incoming and outgoing traffic. That is, setting BandwidthRate to 300 > Kbytes will limit incoming traffic to 300 kilobytes/s and outgoing > traffic to 300 kilobytes/s. It's possible that whatever is measuring the > in/out traffic is combining both these values (hence the 1megabit for > the 500kbit value). The option "BandwidthBurst" behaves in a similar > manner. From the sequence of postings in this thread so far, I note a couple of things. One is that the OP said he was limiting to 300 kilo*bits*/s, while his torrc (as posted) said 300 kilo*bytes*/s. I've already deleted his message, so I'm making this statement based upon a possibly faulty recollection that he wrote *Relay*Bandwidth, rather than Bandwidth, so any of his own traffic through tor (i.e., his use of tor as a client) is *not* counted or limited by his torrc. Likewise regarding RelayBandwidthBurst vs. BandwidthBurst, which could very well account for the overage observed. > > The manual explains each option in the torrc reasonably fully: > https://www.torproject.org/docs/tor-manual.html.en > (Alternatively, the command 'man tor' should display a virtual manual on > the computer.) > > I'm not sure why you'd be seeing the latter burst but I suspect the > reason is due to an ill-configured torrc. It's probably worth checking > the options again after reading the manual and reloading the Tor > service. See possible answer above. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] 100K circuit request per minute for hours killed my relay
Vort wrote: > > Your message prompted me to check logs, and on one relay I see the > > following: > > Similar thing for me: > > Jul 19 00:08:27.000 [notice] Circuit handshake stats since last time: > 3571/3571 TAP, 41180/41180 NTor. > Jul 19 06:08:27.000 [notice] Circuit handshake stats since last time: > 2054/2054 TAP, 29181/29181 NTor. > Jul 19 12:08:28.000 [notice] Circuit handshake stats since last time: > 2773/2773 TAP, 26497/26497 NTor. > Jul 19 18:08:28.000 [notice] Circuit handshake stats since last time: > 3970/3970 TAP, 31344/31344 NTor. > Jul 20 00:08:28.000 [notice] Circuit handshake stats since last time: > 4096/4096 TAP, 41730/41730 NTor. > Jul 20 06:08:28.000 [notice] Circuit handshake stats since last time: > 18285/18285 TAP, 54102/54102 NTor. > Jul 20 12:08:28.000 [notice] Circuit handshake stats since last time: > 61136/61386 TAP, 378196/378339 NTor. > Jul 20 18:08:29.000 [notice] Circuit handshake stats since last time: > 73297/73688 TAP, 566708/566892 NTor. > Jul 21 00:08:29.000 [notice] Circuit handshake stats since last time: > 67165/67830 TAP, 572685/572851 NTor. > Jul 21 06:08:29.000 [notice] Circuit handshake stats since last time: > 31988/32138 TAP, 521455/521536 NTor. > Jul 21 12:08:29.000 [notice] Circuit handshake stats since last time: > 5523/5523 TAP, 222378/222432 NTor. > > Also there are too much "[warn] assign_to_cpuworker failed. Ignoring." lines > in the logs. > This sort of thing has been going on for many years. I used to refer to it as "mobbing". As nearly as I was ever able to determine, the behavior is an unintended consequence of hidden services. I found that I could greatly reduce the frequency of occurrence, but *not* to zero, by setting HidServDirectoryV2 0 in my torrc file. My tentative conclusion was that the majority of these events are cases in which a relay has been selected as an HSDir to which a hidden service descriptor has been posted for a very popular hidden service, so by refusing to be a hidden service directory mirror, those cases can be eliminated. I never had a very satisfying hypothesis to explain the remaining minority of cases. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] IPv6 to IPv4 tor exit relays would fix many daily tor-problems
grarpamp wrote: > > We don't know how to give users good anonymity when some relays can't > > connect to other relays. This would happen if we allowed IPv4-only relays > > and IPv6-only relays in the same network. > > With "IPv6 only" relays available in the consensus the answer may be... > when their count is the same as when IPv4 relays were at the same count, > what was being stated and roughly understood about tor's anonymity back then? > And is it much different from today. And given respective traffic loadings, > etc. Also, is there a problem with having IPv6-only exit service where a relay is accessable via IPv4 for clients and other relays? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] [SOLVED] published descriptor missing from consensus
Roger Dingledine wrote: > On Fri, Jun 09, 2017 at 01:05:43AM -0500, Scott Bennett wrote: > > > I think you will find this is not an uncommon configuration among > > > high-bandwidth relays. > > > > I will check further into the procedure for which Roger posted a URL > > to see whether it will indeed give me a list of such addresses that I can > > use. If so, I will certainly begin running it, scraping the addresses, > > and merging them with the addresses already going into table creation. > > It will not do what you want. It is only for exit addresses. > Yes, I saw that after I took a look. However, that shouldn't really be a problem because the Exit addresses are the ones that could have gotten into the block list simply by doing their job. A non-Exit on the block list got their by its operator's own bad behavior. If someone writes to me and promises never to do it again, I'll remove his address from the block list, but on the condition the promise is kept. Likewise if someone writes to me and says his relay used to run as an Exit, but now just runs as entry/middle, I'll also remove his address, again conditionally, because his address may have gotten into the block list when it was an Exit doing its job. Again, the published address would still get a pass anytime it appears in a consensus. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] [SOLVED] published descriptor missing from consensus
teor wrote: > > > On 9 Jun 2017, at 08:30, Scott Bennett wrote: > > > > Roman Mamedov wrote: > > > >> On Thu, 08 Jun 2017 09:43:00 -0500 > >> Scott Bennett wrote: > >> > >>> As noted more than once previously, the pf rules *pass* all traffic > >>> from relay addresses *first*, so that traffic has already gone on to tor > >>> before the block list is applied. > >> > >> There are most likely some relays which use a different IP for outgoing > >> connections than what is listed in the consensus, due to multiple IPs or > >> provider multihoming. Your scheme does not seem to account for that, so > >> those > >> connections may fail. In effect you will be leaving the Tor network > >> permanently semi-broken by running a relay while employing such filtering. > > This also excludes any direct client connections to your relay. For an Exit, > clients should only connect if UseEntryGuards is 0 (the default is 1, except > for Tor2Web and Single Onion Service clients). It only excludes client connections from IP addresses with a history of offending behavior against my system. That exclusion is an express aim of the packet filter rules I posted. By excluding their access to anything on my system, I keep them from causing me further trouble. Unfortunately, if such troublemaking IP addresses also happen to host relays, then I have to let them have access to the ORPort and DirPort, but I can still exclude them from everything else. Bad behavior should be expected to have consequences. Clients at non-offending IP addresses are allowed to connect without interference. > > It also excludes connections from relays that come up and down frequently. Well, laying aside the fact that we don't want them to be up and down frequently, my TorRelays table is regenerated every 15 minutes from the most recently retrieved consensus. Consensuses are rarely fetched more than once during the same hour, but even if they are and regardless of the time during the clock hour the consensus is retrieved, only a few minutes will pass before the table is produced fresh and loaded into pf. That is much faster than a new consensus is propagated to all relays or clients anyway. > > > You're right. I recall spending a lot of time trying to find a way to > > get those addresses, so that I could include them in the TorRelays table or > > some other arrangement to let their traffic through. Unfortunately, tor > > doesn't publish them anywhere that I could find nor did anything else back > > at the time I set these rules and tables up. However, I figured most relays > > probably have only a single WAN interface, so the rules would probably cause > > few failures. > > My relays have multiple IP addresses on a single WAN interface. They all use > OutboundBindAddress to separate OR and Dir traffic from outbound traffic. And > they make up about 1% of Tor guard/middle bandwidth. So, although the traffic is not separated on that machine, you use the source addresses to allow a router to separate the traffic as it leaves your network? Or is it indeed separated on that machine via routing table entries? > > I think you will find this is not an uncommon configuration among > high-bandwidth relays. I will check further into the procedure for which Roger posted a URL to see whether it will indeed give me a list of such addresses that I can use. If so, I will certainly begin running it, scraping the addresses, and merging them with the addresses already going into table creation. > > Some directory authorities also have multiple IP addresses. See above. > > > Also, clients don't give up after something like that, but > > rather continue to try more circuits, so the end user may experience a short > > delay, but won't actually go unserved in such cases. > > What actually happens depends on a number of factors: > * whether the other relay has successfully connected to your relay, > * whether both relays think the connection is canonical, > * whether either relay has a large exponential backoff on retries. > > So in some cases, clients will be unable to connect to your exit via some Although I allow exits to a small list of places, my relay does not allow general exiting in a manner that would qualify it for an Exit flag on its entry in the consensus, so in that sense, there is no exit here. > middle relays. This reduces your exit traffic, and also reduces the number of > different circuit paths available to clients. (Using a wide variety of paths > is one of the building blocks of Tor's anonymity.) My daily exit traffic is ordinarily zero. Non-zero days are so rare
Re: [tor-relays] How to detect Tor exit IP addresses (was Re: [SOLVED] published descriptor missing from consensus)
Roger Dingledine wrote: > On Thu, Jun 08, 2017 at 05:30:37PM -0500, Scott Bennett wrote: > > Consider another case. Users have often complained that running a tor > > relay results in their IP addresses being blocked by all manner of services > > around the Internet. The providers of those services say they have suffered > > attacks originating from tor relays. The project's response was to create > > an automatically, frequently updated list of IP addresses of exit relays and > > make that list available for download by anyone wishing to block traffic > > from > > tor exits, while allowing traffic from all other relays. That list of > > addresses suffers the same problem of not including alternative IP addresses > > for those relays. Even worse, troublesome connections from those > > alternative > > addresses *can* be traced back, in some cases, to the exit relay. Once > > those > > services have identified the offending traffic as coming from a machine they > > had been promised by the tor project would be in the downloadable list of > > exit relay addresses, they may decide that they had been deceived by the tor > > project, which could lead to many bad things in the future. > > I think we might have to agree to disagree about a lot of these topics, > but I wanted to correct this one. > > The bulk exit list: > https://check.torproject.org/cgi-bin/TorBulkExitList.py > along with TorDNSEL is designed to handle exactly this situation, and > it does it pretty well. > Okay, I'll take your word for it, but I'm curious to know the source of the addresses collected if tor doesn't report them. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] [SOLVED] published descriptor missing from consensus
Roman Mamedov wrote: Hi Roman! > On Thu, 08 Jun 2017 09:43:00 -0500 > Scott Bennett wrote: > > > As noted more than once previously, the pf rules *pass* all traffic > > from relay addresses *first*, so that traffic has already gone on to tor > > before the block list is applied. > > There are most likely some relays which use a different IP for outgoing > connections than what is listed in the consensus, due to multiple IPs or > provider multihoming. Your scheme does not seem to account for that, so those > connections may fail. In effect you will be leaving the Tor network > permanently semi-broken by running a relay while employing such filtering. You're right. I recall spending a lot of time trying to find a way to get those addresses, so that I could include them in the TorRelays table or some other arrangement to let their traffic through. Unfortunately, tor doesn't publish them anywhere that I could find nor did anything else back at the time I set these rules and tables up. However, I figured most relays probably have only a single WAN interface, so the rules would probably cause few failures. Also, clients don't give up after something like that, but rather continue to try more circuits, so the end user may experience a short delay, but won't actually go unserved in such cases. Consider another case. Users have often complained that running a tor relay results in their IP addresses being blocked by all manner of services around the Internet. The providers of those services say they have suffered attacks originating from tor relays. The project's response was to create an automatically, frequently updated list of IP addresses of exit relays and make that list available for download by anyone wishing to block traffic from tor exits, while allowing traffic from all other relays. That list of addresses suffers the same problem of not including alternative IP addresses for those relays. Even worse, troublesome connections from those alternative addresses *can* be traced back, in some cases, to the exit relay. Once those services have identified the offending traffic as coming from a machine they had been promised by the tor project would be in the downloadable list of exit relay addresses, they may decide that they had been deceived by the tor project, which could lead to many bad things in the future. > > In any case I don't think there is any reasonable threat scenario against > which > you must protect by not just allowing all connections from anywhere to > ORPort/DirPort of a Tor relay. > That is something that each system administrator has to determine for the system(s) under his care. My determination for my system apparently differs from your evaluation of your situation. What originally prompted me to look into using a packet filter to block traffic from identifiable sources of trouble was the frequency of attacks on my system. It was so intense at times that even the kernel was complaining in console messages that it was limiting RSTs to 200/s. These messages were happening so fast that syslogd was reducing them in most cases with messages like "last line repeated 1353 times" (not an exact quotation, but I'm sure you've seen syslogd's message flood response before). At first, setting net.inet.tcp.blackhole=2 net.inet.udp.blackhole=1 seemed to take care of the problem. However, after a couple of years, the messages resumed. China, Malaysia, Nigeria, and a few other countries were being especially persistent and aggravating in the frequency of their attacks. (Just in case, I've since added "net.inet.sctp.blackhole=2" to the above list.) Although providing a tor relay and directory is currently the only thing my system offers to benefit other people, I decided that, if I ever chose to offer other services, I would want to deny those services to the offending parties, too, which is why those block rules apply to all ports on my system and not only to tor's ports. The pass rules apply to traffic destined to tor's ports. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] [SOLVED] published descriptor missing from consensus
tor wrote:
> > I tried disabling pf and restarting tor. To my surprise, the authorities
> > connected to my relay successfully and distributed its information in
> > subsequent consensuses!
>
> Haha. This is the least surprising thing I've read all week.
Then either you weren't paying attention or you don't know pf. And
you have to explain why correcting the openssl library versions eliminates
the problem. In short, you didn't follow the chain of evidence presented.
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] [SOLVED] published descriptor missing from consensus
teor wrote: > > > On 8 Jun 2017, at 18:44, Scott Bennett wrote: > > > > Roger Dingledine wrote: > > > > Or at least partially solved, at any rate. The problem is solved, > > but the mystery remains. > > > >> On Sun, Jun 04, 2017 at 07:14:06PM -0500, Scott Bennett wrote: > >>> Which versions are the Running votes coming from versus the > >>> non-Running? > >> > >> You can see the votes at > >> https://www.seul.org/~arma/moria1-v3-status-votes > >> > >>> I have a few commands in a crontab entry that extract relay IP > >>> addresses > >>> from the most recently received consensus, sort them, and load them into a > >>> table in pf. They run every 15 minutes. Anything coming from the > >>> addresses > >>> in the table is immediately passed. Anything not passed gets checked > >>> against > >>> a much larger table of probers, attempted intruders, etc. and is blocked > >>> if it > >>> matches a table entry. > >> > >> Well heck. Sounds like we have a winner here. > >> > >> Turn off your weird censorship infrastructure, confirm that things > >> start working after a while, and now you have something to debug. :) > >> > > The misuse of the word "censorship" aside, I tried disabling pf and > > restarting tor. To my surprise, the authorities connected to my relay > > successfully and distributed its information in subsequent consensuses! > > I do not know why, which is the remaining mystery. However, having to > > turn off one's firewall in order to run tor is not a solution to the > > problem. > > If your firewall is blocking connections that tor needs to make, turning > off that rule is precisely the solution to the problem. As noted more than once previously, the pf rules *pass* all traffic from relay addresses *first*, so that traffic has already gone on to tor before the block list is applied. As also noted previously, the rules had not (and still have not) been changed in a long time. They were working just fine for years, and then suddenly they were not. Now they are working just fine again since tor got rebuilt, so they are not the problem. > > > What I think I know is that the mismatched openssl stub and library versions > > somehow interact with pf and pre-0.3.0.7 tor authorities in some mysterious > > manner that causes connections from those authorities to my relay to fail, > > but connections from 0.3.0.7 authorities to work just fine. Having the stub > > version and the library version be identical apparently avoids the erroneous > > interaction(s). > > It's possible. But to confirm, you'd need to provide the sections of > the firewall log that show what happens when a directory authority > tries to check the reachability of your relay. That stuff isn't logged because it would soon exhaust the available disk space with garbage. > > (As an aside, I'd encourage you not to keep logs at this level on an > ongoing basis, they would contain client IP addresses and connection > times.) That's one reason; the other is the space and kernel overhead requirements. > > Or perhaps your firewall collected enough state to want to block tor > while it was running, and lost that state when you turned it off and > on again. I don't understand how an application could directly interfere with the packet filter's operation. I've previously posted the rules. As I described, it appears to be a problem of openssl misbehaving, apparently due to mismatched stub and library versions, though I don't understand how openssl trouble could occur with pf enabled and not occur with pf disabled. In any case, rebuilding tor so that the stub and library versions match has eliminated the problem. tor now plays nicely with the authorities, regardless of whether pf is enabled. Now, having said that, I should mention that tor does mess around with /dev/bpf, at least during startup, which is one reason why tor has to have root privileges during startup. I've never looked into exactly what it does with /dev/bpf, however, so I have no idea whether that could be involved and caused to malfunction somehow by openssl problems. > > You have a complex enough system that there are probably more > possibilities neither of us have thought of. Complex? Compared to what? You've already seen the handful of rules involved, which includes some near duplication of ORPort rules to provide the same protection from miscreants to the DirPort that is given to the ORPort. In a nutshell, the rules look a
Re: [tor-relays] [SOLVED] published descriptor missing from consensus
Roger Dingledine wrote: Or at least partially solved, at any rate. The problem is solved, but the mystery remains. > On Sun, Jun 04, 2017 at 07:14:06PM -0500, Scott Bennett wrote: > > Which versions are the Running votes coming from versus the > > non-Running? > > You can see the votes at > https://www.seul.org/~arma/moria1-v3-status-votes > > > I have a few commands in a crontab entry that extract relay IP > > addresses > > from the most recently received consensus, sort them, and load them into a > > table in pf. They run every 15 minutes. Anything coming from the addresses > > in the table is immediately passed. Anything not passed gets checked > > against > > a much larger table of probers, attempted intruders, etc. and is blocked if > > it > > matches a table entry. > > Well heck. Sounds like we have a winner here. > > Turn off your weird censorship infrastructure, confirm that things > start working after a while, and now you have something to debug. :) > The misuse of the word "censorship" aside, I tried disabling pf and restarting tor. To my surprise, the authorities connected to my relay successfully and distributed its information in subsequent consensuses! I do not know why, which is the remaining mystery. However, having to turn off one's firewall in order to run tor is not a solution to the problem. But, while going back through the logs, I noticed the following among tor's startup messages. Jun 05 23:25:46.849 [warn] OpenSSL version from headers does not match the version we're running with. If you get weird crashes, that might be why. (Compiled with 100020bf: OpenSSL 1.0.2k 26 Jan 2017; running with 100020cf: OpenSSL 1.0.2l 25 May 2017). Jun 05 23:25:46.895 [notice] Tor 0.3.0.7 (git-cfd9c1bdc0582656) running on FreeBSD with Libevent 2.1.8-stable, OpenSSL 1.0.2l and Zlib 1.2.8. The date of 25 May 2017 means that my last run of 0.3.0.6 was using an older OpenSSL version, but 0.3.0.7 was using the newer one, although it had been compiled with the earlier version shown in the first message. I'm not sure how that might interact badly with pf at all. However, I rebuilt 0.3.0.7, stopped the running copy, enabled pf, and restarted tor to run from the rebuilt version. Enough authorities must be happy with it because they are now distributing its information in the consensus and also its most recent descriptor. In summary, the problem is resolved, but the mystery remains unsolved. What I think I know is that the mismatched openssl stub and library versions somehow interact with pf and pre-0.3.0.7 tor authorities in some mysterious manner that causes connections from those authorities to my relay to fail, but connections from 0.3.0.7 authorities to work just fine. Having the stub version and the library version be identical apparently avoids the erroneous interaction(s). Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] published descriptor missing from consensus
Roger Dingledine wrote: > On Sun, Jun 04, 2017 at 07:14:06PM -0500, Scott Bennett wrote: > > Which versions are the Running votes coming from versus the > > non-Running? > > You can see the votes at > https://www.seul.org/~arma/moria1-v3-status-votes Very interesting indeed. The only two Running votes are from tor26 and maatuska, which are the only Authority relays running 0.3.0.7. All the other authorities are running 0.2.9.9, 0.2.9.10, or 0.3.0.7-dev. (!) I'm not quite sure what to make of that, except that it appears that 0.3.0.7 is compatible with itself. > > > I have a few commands in a crontab entry that extract relay IP > > addresses > > from the most recently received consensus, sort them, and load them into a > > table in pf. They run every 15 minutes. Anything coming from the addresses > > in the table is immediately passed. Anything not passed gets checked > > against > > a much larger table of probers, attempted intruders, etc. and is blocked if > > it > > matches a table entry. > > Well heck. Sounds like we have a winner here. How so? If all traffic from the relay addresses is passed? I suppose I should have mentioned, because I use pf, that the rules use the "quick" flag: pass in quick on nfe0 proto tcp from to nfe0 port 32323 flags S/SA label "ORPort from relays" pass in quick on nfe0 proto tcp from to nfe0 port 32326 flags S/SA label "DirPort from relays" block drop in quick on nfe0 from to any pass in quick on nfe0 proto tcp from any to nfe0 port 32323 flags S/SA label "ORPort from others" pass in quick on nfe0 proto tcp from any to nfe0 port 32326 flags S/SA label "DirPort from others" IOW, the rules act in a normal, logical sequence, rather than in the reversed order, remaindered way that is pf's standard method of operation. (I haven't the foggiest idea why the OpenBSD folks designed pf that way.) Note that TorRelays is the table generated by the crontab job, and Crackers is the table of offending addresses that I block. By passing all relay traffic first, it never hits the third rule. The fourth and fifth rules let all traffic from non-offending clients through, as well as any traffic from new relays that do not appear in the most recently downloaded consensus or in the Crackers table. > > Turn off your weird censorship infrastructure, confirm that things I must strenuously object. First, I am not a government, thus I have no censorship powers. Second, all computer operators/sysadmins have the right to defend their systems. If you think that blocking attackers is weird, then I suggest that that is your issue, not mine. > start working after a while, and now you have something to debug. :) As I noted before, my setup has been working for many years without problems. It is only with the change to 0.3.0.7 from 0.3.0.6 that the problem in question has appeared. Convince me that that is irrelevant, please. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] published descriptor missing from consensus
Roger Dingledine wrote: > On Sun, Jun 04, 2017 at 12:30:20AM -0500, Scott Bennett wrote: > > Late Wednesday afternoon, I restarted my relay (MYCROFTsOtherChild), > > which changed it from 0.3.0.6 to 0.3.0.7. That was the only change I made. > > It went through a normal startup and published its descriptor. After a few > > hours, tor noticed that its descriptor was still not in the latest consensus > > Interesting mystery! You always have the most exciting mysteries. :) A lifelong problem, I'm afraid. I started bumping into compiler and library bugs at 16 or 17 that nobody had reported. It was all unintentional on my part. BTW, some time back, I noticed that the uptime instability in heartbeat messages returned, or perhaps it's a new one. It reports the first 3600-second heartbeat period at 0:59 hours, and each one thereafter is at n:59 hours. If the system is under some particularly heavy load, I've seen the time lengthen to where the messages show up as late as n:04 hours. > > I just instrumented moria1 to be more detailed on why it doesn't find > each relay reachable, and here's what I found: > > Jun 04 18:12:44.147 [info] dirserv_single_reachability_test(): Testing > reachability of MYCROFTsOtherChild at 73.246.41.113:32323. > Jun 04 18:13:47.147 [info] connection_handle_write_impl(): in-progress > connect to 73.246.41.113:32323 failed. Removing. (Connection timed out) > > Jun 04 18:34:04.205 [info] dirserv_single_reachability_test(): Testing > reachability of MYCROFTsOtherChild at 73.246.41.113:32323. > Jun 04 18:35:07.205 [info] connection_handle_write_impl(): in-progress > connect to 73.246.41.113:32323 failed. Removing. (Connection timed out) > > So it would appear that it's trying to make a TCP connection, and > after 63 seconds, it decides it's not going to work. > > It would seem that 6 of the 8 directory authorities are not voting the > Running flag, so I guess they are seeing something similar (or would be > if they hacked their logs up to display it). Which versions are the Running votes coming from versus the non-Running? > > This is weird, because when I telnet to your IP:port, it connects > easily. And when I set your IP:port as my bridge address, my Tor client > bootstraps fine. > > So I am left wondering if there's something different about how Tor > requests that the system launch a TCP connection, or if Comcast or > your system is somehow filtering (or not being able to handle) certain > connection attempts. > I have a few commands in a crontab entry that extract relay IP addresses from the most recently received consensus, sort them, and load them into a table in pf. They run every 15 minutes. Anything coming from the addresses in the table is immediately passed. Anything not passed gets checked against a much larger table of probers, attempted intruders, etc. and is blocked if it matches a table entry. Anything left over gets passed. This setup has been in place for many years without problems. It does leave open the possibility that a relay that has started recently and is not yet listed in the most recently downloaded consensus could get failures until it does show up, but that would be very temporary. Also, authorities should basically always be in the relays list that is checked first. Again, 0.3.0.6 was working fine up until I shut it down. The restart was then as 0.3.0.7 and has not "worked" yet, although I'm still using the client functions without problems. Also, my relay had no problem connecting to itself during its reachability and "bandwidth" testing. Have significant numbers of other relays vanished from the consensus after changing to 0.3.0.7? Roger, thanks very much for taking a look at this problem. I've been running this relay for almost ten years, and I would like to continue to do so, even though it doesn't normally get a lot of traffic anymore. If there's anything you would like me to try, please let me know. Hmmm...a dim memory has blossomed while I've been typing here. Some years ago there was a problem with a version of openssl that couldn't talk to itself. That time I could see lots of connection attempts with no connections becoming established, however. In that situation, tor was unable to connect to itself during reachability testing, so it never published a descriptor and continued to try to connect until I shut it down. FWIW, the latest heartbeat messages were: Jun 04 18:35:21.761 [notice] Heartbeat: It seems like we are not in the cached consensus. Jun 04 18:35:21.762 [notice] Heartbeat: Tor's uptime is 3 days 4:59 hours, with 2 circuits open. I've sent 19.71 MB and received 181.87 MB. Jun 04 18:35:21.762 [notice] Average packaged cell fullness: 48.0
[tor-relays] published descriptor missing from consensus
This time I'm stumped. I confess I have not followed this list for
a long time until the last few days, so it's possible that I've run afoul
of something already discussed.
Late Wednesday afternoon, I restarted my relay (MYCROFTsOtherChild),
which changed it from 0.3.0.6 to 0.3.0.7. That was the only change I made.
It went through a normal startup and published its descriptor. After a few
hours, tor noticed that its descriptor was still not in the latest consensus,
so it republished it. After several more hours had passed with each
consensus lacking my relay's descriptor, I made a small change to an
ExitPolicy line in torrc. In the meantime, I had read some recent postings
to this list and had noted some remarks regarding mismatched RSA and ED25519
keys, so I renamed /var/db/tor/keys and restarted tor again, whereupon it
generated new keys and appeared to proceed normally.
Once again a new descriptor was published and has since been republished
and updated with no apparent problem beyond some republications being told by
one or more authorities that the descriptor was rejected as "not new".
Nevertheless, my relay's descriptor still has not appeared in a consensus
since the last time it ran 0.3.0.6. If an authority operator would be so kind
as to check the authority's logs for any errors pertaining to my relay with
either its old keys or its new keys and let me know what they say, I would be
grateful.
Thanks for any help or suggestions of things to try.
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *xor* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] New router, new IP address
Rick wrote: > So I saw the Craigslist post in tor-talk and checked to find that I'm at > full speed with them. Since I'm an exit relay (residential, US, 15 > months) and have been experiencing the CL Dial-Up Retro for a few > months, I was surprised. It turns out that I have a new IP address so > for the moment I'm a shiny new exit. > Two days ago I installed a new router. Nothing special: $45 Linksys but > selected specifically for use with DD-WRT. ROM burn and set-up was done > completely offline. It took maybe a minute to do the router hot swap and > all went well. IP address change never crossed my mind. After all, I was > down for four days back in the winter (ice storm) and hoping for a new > address when we came back up. No luck. > Is this normal? Or at least not unusual? It seems a bit spooky to me. I > thought the address was tied to the cable modem (ISP's equipment) and > they couldn't care less about my LAN. :-\ > Many ISPs use the MAC address of the interface(s) on your side of the cable/DSL modem as the tie to a particular IP address assigned in a DHCP lease. If you replace the NIC in your computer or replace your computer with a router, then that address gets a different IP address assigned, possibly because the old address is still assigned in an unexpired lease. BTW, I don't recommend running a relay behind an electronics store- grade of router. They usually don't have enough memory to keep track of hundreds of NAT table entries or other functions necessary for serving an active relay. As long as you aren't stuck using a Micro$lop OS, you would be far better off using your choice of your OS's packet filters to enable your computer to serve as the gateway/router for your LAN. Using DD-WRT instead of the Linksys proprietary firmware is a good move, though, if only because Linksys firmware tends to be rather buggy at handling DHCP leases correctly and out-of-memory conditions gracefully. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] [tor-dev] Hidden service policies
Thomas White wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > > Sorry, wrong answer. If you block connections from other relays, > > you break the tor network. I don't recall offhand whether that > > sort of breakage might earn your relay either an Invalid flag or > > being simply dropped from the consensus. > > For a single relay to my knowledge, it shouldn't do. There are many > reasons some relays can't connect to each other so it doesn't "break" > Tor as an alternative route is simply found. > Yes, tor, like many other Internet operations, has some ability to route around breakage in its network. However, each time it is necessary to find a way around it, a cost to the network is incurred in the form of wasted processing time over many pieces of equipment, wasted traffic, and likely wasted end-user time. > > > Are you suggesting that the mobbing attacks on HSDIR relays are the > > actions of botnets? If so, then you are suggesting that the > > problem of mobbing of HSDIR relays is probably insoluble because it > > would not be the symptom of a bug in tor. :-( > > The question is botnet CnC's, the proposal has nothing to do with > solving the botnet CnC problem and I am also stating Tor is not the > one who needs to tackle them right at this moment, the budget and Agreed. > resources are just not there. However creating a system where > operators start blacklisting hidden services is extremely bad for > anonymity both for the hidden service and the user. > Also agreed. I was referring to the as yet unsolved problem of HSDIR mobbing, which I have long thought was due to a bug somewhere in tor, just as there used to be a problem with DirPort mobbing. The DirPort mobbing bug was eventually found and fixed a long time ago, but the HSDIR mobbing still hasn't been. But now you have given me the idea that perhaps HSDIR mobbing is actually due to other software applying a malicious attack upon tor relays that have the HSDIR flag. IOW, I wasn't arguing with you, just commenting about this other problem in light of what you had written. > To answer the rest of your question, I am not a developer. I am > somebody who cares about anonymity and that is why I run the 2nd > largest server cluster on the Tor network from my own pocket. > Filtering or proposing to blacklist anything is not acceptable in my > view. Whatever solutions individuals care to launch to protect their > relay is their own responsibility, but actively developing something > by the core developers to blacklist hidden service is a completely > despicable idea. To elaborate only on the legal side of things, if I > can easily block hidden services passing through my relays or if I am > the RV point for one the government can then serve me a notice AFAICT, the introduction point and the rendez-vous point are about the only places you might be able to block them, though by doing so, you would again be introducing a form of breakage. If your relay were at any other points in the hidden service protocol, you wouldn't have any way of distinguishing it from any other middle node along a tor circuit. But I would need to reread the protocol specification in detail again see whether you could actually deny service even at the invitation and rendez-vous points. > ordering me to block it, this I have already run through my solicitor > and there no escaping that fact unfortunately. > > Also note, botnets in this sense are not the topic. The proposal is an > easy mechanism to censor hidden services and let it not be portrayed > as anything other than that. I can see why 90% of people opposed his > "coin taint" idea and 75% wanted him to leave the bitcoin foundation. > If Tor did introduce such measures, I would be swiftly leaving Tor's > ranks and withdrawing all support (both all 25 relays/exits/guards, > and financial) from it. > > So to state clearly: > > Should Tor Project develop a system to filter hidden services? > > I'll let people decide that for themselves. But my opinion, is that > doing so defies the point of a hidden service and people who push for > it should be ashamed of themselves. > Also fully agreed. To develop such a system would require weakening or breaking the current level of protection offered to users, as well as being a special gift to the NSA and its peers in other countries. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *or* bennett at freeshell.org * ** * "A well re
Re: [tor-relays] [tor-dev] Hidden service policies
Thomas White wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Mike Hearn, > > Simple. If you start filtering anything at all, regardless of what it > is (yes, even if you filter child porn or fraud sites) then I will > block any connection of your relays to mine (which are exits and > guards totally 4Gbps). There are uses for preventing some connections Sorry, wrong answer. If you block connections from other relays, you break the tor network. I don't recall offhand whether that sort of breakage might earn your relay either an Invalid flag or being simply dropped from the consensus. > like if you are legally required to then I guess the tradeoff of some > inconvenience for a handful of relays, but still providing high-speed > access to Tor for most people and sites is worth it. When you begin to > do it as a proactive censorship event is when I will be firmly against > you. > > The moment people censor things because it is illegal, immoral or > "terrorist" is the moment that person accepts responsibility for the > traffic that passes through their nodes and is an active attempt by > them to filter what people can access. Freedom isn't free unless it is > totally free and a selective reading policy through Tor is not just a > bad idea as stated below, I find it outright insulting to me and > everyone else who cares about the free and open internet. The fact > somebody has the audacity to come to a project like Tor and propose > blacklisting mechanisms is jaw-dropping. > > In addition, botnets using Tor actually improve the security of the > network. Generally the more traffic there is, the harder it is to > conduct statistical attacks against the users. Now of course it is not > the most politic thing to say or the most popular, but it's the truth. Are you suggesting that the mobbing attacks on HSDIR relays are the actions of botnets? If so, then you are suggesting that the problem of mobbing of HSDIR relays is probably insoluble because it would not be the symptom of a bug in tor. :-( > We don't need to stop x y or z using Tor, we need to get more people > using Tor regardless of their purpose. Botnets are the result of > design/security flaws and not something within the scope of Tor > Project to address. Wrong again. See multitudinous previous threads regarding bittorrent over tor. Let me give you an example of appropriate filtering. My system logs frequent attacks/probes that I consider illegitimate. I enter the source addresses of those probes into a pf table of addresses from which SYN packets for any protocol or port get dropped with no response. However, there is a cron job that runs every 30 minutes that takes all the relay IP addresses in the most recently downloaded consensus and puts them into another pf table. This latter table is used by pf rules to bypass the check described above, but only for relays attempting to connect to my relay's ORPort or DirPort. This prevents the sort of breakage you threaten to cause because currently active relays will still be able to relay through my relay, although if they are also in the table described first, then they will have no *other* type of access to my system. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *or* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Speed of my relay not correct on global list
Matthew Harrold wrote: > > I run a small relay, alias "Empire64" >(I tried an exit once, but my IP > was >then banned at several places, so >back to only a relay) on Win Xp. > > PSA: XP is out of date, and no longer supported by Microsoft. Please > update, if you can't afford MS licensing please consider a Linux OS. Ahem. I think that should be amended to say, "...please consider a currently supported OS", which would include the BSDs (e.g., DragonflyBSD, FreeBSD, OS X, NetBSD, OpenBSD, primarily) and Solaris at the least, as well as LINUX. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *or* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] CPU usage
Roman Mamedov wrote: > On Mon, 7 Jul 2014 23:30:22 -0700 > "Asa Rossoff" wrote: > > > With hyperthreading, I think 4 would be optimal? > > Yes, 4 can be set, but I remember reading somewhere that Tor doesn't scale > well beyond NumCPUs 2 (and poorly even to 2). See my previous note in this thread. Note that NumCPUs specifies the number of [logical] CPUs available for simultaneous onionskin decryption threads to make use of. Those worker threads are not used for any other work, AFAIK. > > One way to increase utilization further would be to run a second instance of > Tor. However as this server is also used for other tasks, this could begin > starving them of CPU time via the hyperthreading concurrency even while Tor > nominally runs at lower nice levels. > It seems to me that an Atom is a tad underpowered for that sort of scheduling. One instance is enough. If the OP wishes to make better use of his network throughput capacity with tor, then he needs to get a more powerful machine to do it. Another matter to consider is that very high-throughput relays need to run on operating systems that support superpages (a.k.a. hugepages in LINUX) to reduce the large percentage of time wasted in processor stalls due to TLB thrashing. Microsoft only supports that when set up manually, IIRC. LINUX used to be likewise, but I seem to recall reading somewhere that its developers eventually wised up and made them automatic, like in FreeBSD. I have no idea whether NetBSD or OpenBSD supports superpages at all. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *or* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] CPU usage
"Asa Rossoff" wrote: > On 7 July 2014, 04:49 UTC, Roman Mamedov wrote: > > On Mon, 07 Jul 2014 21:31:02 +0100 > > kingqueen wrote: > > > > > [stuff deleted --SB] > > > > Set "NumCPUs 2" in your torrc to make it try utilizing the second core > > too > > (this won't help much, but at least somewhat). > > With hyperthreading, I think 4 would be optimal? Probably, yes. > Also, I'm not a regular user of Linux, but doesn't top report load averages > as a sum of all CPUs/cores/virtual cores... e.g., in this case, if the > described CPU were at capacity, top would report 4.00? No. top reports the average number of processes in the system in the ready-to-run state (more or less equivalent to the "dispatchable" state). This state include not only those currently dispatched, but also those that could be dispatched were enough otherwise idle [logical] CPUs available. > > If nothing else, this could be nicer to other running applications, but if > tor was assuming one CPU/core previously, would actually increase the amount > of CPU available to tor. > On a high throughput node, it should make some difference, although the degree of difference depends upon the number of onion skins to be decrypted per unit of time. As has been stated here on multitudinous occasions, the real hangup on spreading the workload of tor across extra CPUs is that openssl is not a multithreaded library. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *or* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] bitcoin adopt a node idea
grarpamp wrote: > On Thu, Jun 26, 2014 at 10:44 AM, David Hill wrote: > > On Thu, Jun 26, 2014 at 12:35:00AM -0500, Scott Bennett wrote: > >> ja...@icetor.is wrote: > >> > >> > http://www.coindesk.com/adopt-node-project-aims-bolster-bitcoin-network-security/ > >> > > >> Assuming that the relevant bitcoin programs could be taught to talk > >> SOCKS, then it seems that tor hidden services would, in principle if not > >> in performance, be an ideal solution. Running those bitcoin "full" nodes > >> as hidden services might well make them less vulnerable to being shut > > bitcoind works fine with tor and has some onion full nodes. Ah, very good, then. Thanks for the info. > > >> Performance of hidden > >> services, however, are severely constrained by the hidden services > >> protocol, > >> which can slow connection times enough to make one consider USnail as a > >> possible alternative, and the need for circuits of 2n-1 relays, which makes > >> access even slower than normal tor circuits of n relays. > > Performance of hidden services is actually rather good. ymmv. In my experience, it can take a minute or more to get a circuit connected because of the delays involved in the whole rendezvous procedure. Then, due to the length of the circuit, the data transfer rate is often fairly poor. Occasionally, the 5- or 7-hop circuit comprises only fast nodes, so the transfer rate is high, but usually the greater number of hops drastically reduces the probability of getting a fast circuit. > > > I am using btcd, an alternative full-node implementation written in > > golang. Find it at https://github.com/conformal/btcd. It has built in > > proxy support. The wallet, btcwallet, is separate. It also has proxy Excellent. > > support, so that you may connect to btcd over tor or as a tor hidden > > service. That can be found at https://github.com/conformal/btcwallet. > > > > bitcoind nodes are a nice target to look for wallets. But with btcd, I > > run that at home while btcwallet runs on my encrypted laptop which > > connects to btcd over tor. There is no wallet on my btcd node machine. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *or* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] bitcoin adopt a node idea
ja...@icetor.is wrote:
> Sorry perhaps I didn't explain well enough. What I was pointing to was
> that tor could benefit from the idea of cheaply crowd sponsored relays
> that use ansible, chef or puppet to spin up for a month. That the
> article is about bitcoin is merely coincidental.
No, I got that okay. I was making a tangential point. The
juxtaposition of the two projects seemed worth a comment in support
of hidden service ideas. And, yes, the notion of crowd-sponsorship
of relays is good, although perhaps a feature to encourage longer-term
availability might be better.
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *or* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] bitcoin adopt a node idea
ja...@icetor.is wrote:
> This seems pretty damn similiar to something we should be offering for
> Tor relays, possibly even exits and bridges (if they only run for a
> month at a time). Possibly co-ordinated through the EFF?
>
> http://www.coindesk.com/adopt-node-project-aims-bolster-bitcoin-network-security/
>
Assuming that the relevant bitcoin programs could be taught to talk
SOCKS, then it seems that tor hidden services would, in principle if not
in performance, be an ideal solution. Running those bitcoin "full" nodes
as hidden services might well make them less vulnerable to being shut
down by currency counterfeiters (e.g., the Federal Reserve and the central
banks of other states, U.S. Dept. of the Treasury). Performance of hidden
services, however, are severely constrained by the hidden services protocol,
which can slow connection times enough to make one consider USnail as a
possible alternative, and the need for circuits of 2n-1 relays, which makes
access even slower than normal tor circuits of n relays.
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *or* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] MaxAdvertiseBandwith
jchase wrote:
> I have a non-exit relay running Tor 0.2.25 on a Pi. Consensus weight 37.
> exit-policy: reject *:* Now I? back from vacation and my relay has 1600
> incoming conncts and arm warns me that my bandwidth is too small to
> support so many incomers. Would I kindly restrict my reject policy or
> limit my MaxAdvertiseBandwidth. Which I believe is no longer
> supported(?). Can anyone help me cut down on the traffic and the notices?
Looks a lot like the relay mobbing problem again. Try adding
HidServDirectoryV2 0
to your torrc file. That appears to make the problem stop or at least
become very infrequent with far fewer inbound connections.
The cause of the problem really needs to be tracked down, probably
by someone very familiar with the hidden services code in tor. Otherwise
most of the HSDir relays may eventually disappear as more and more people
are forced to turn off the hidden services directory mirror function.
Scott Bennett, Comm. ASMELG, CFIAG
**
* Internet: bennett at sdf.org *or* bennett at freeshell.org *
**
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
*-- Gov. John Hancock, New York Journal, 28 January 1790 *
**
___
tor-relays mailing list
tor-relays@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] SSH scans from Tor exit
I wrote: > What do you suggest I missed in the documentation? > Exit policies. I wrote that in my earlier message. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *or* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] SSH scans from Tor exit
"s...@sky-ip.org" wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > On 4/29/2014 1:31 AM, I wrote: > > One VPS company has just asserted that SSH scans are being run from > > my Tor exit rather than another process on the VPS. Is this > > happening to anyone else? Does anyone know what can be done to stop > > it? > > > > Robert > > > > > > ___ tor-relays mailing > > list tor-relays@lists.torproject.org > > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays > > > > > Could you explain with more details? Your question is not totally clear. I thought his question was very clear. > > If your VPS is being SSH brute forced there are many ways to protect: > - - make hostbased authentication or use keys instead of password-based > authentication > - - install fail2ban to ban IPs after "x" wrong passwords > - - make sure you put a very strong password, seriously > - - disable root login via ssh > - - if you have a VPS made with KVM you can disable SSH access at all > and use the javaconsole from the VPS panel? He stated that a VPS company (I've quoted his statement above yours, so please go back ad read it again) complained that the attacks were emanating *from his tor exit*. The VPS company is very unlikely to be moved by your suggestions. The second matter that was clear was that he has been running a tor relay without having read the documentation. If he wants to restrict what exits from his node, then he needs to read about exit policies in particular, but he also ought to read the rest of the documentation as well. More generally, people really should not be running an exit in ignorance. The tor project has done a commendable job of providing a well documented product. The documentation was intended to be read, not ignored, by those wishing to run tor, whether as a client only or as a relay. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *or* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Recommended reject lines for relays affected by Heartbleed
Andrea Shepard wrote: > On Thu, Apr 17, 2014 at 08:58:46PM +0200, Lars Kumbier wrote: > > I'm supposedly running one of the still affected tor-relays and since my > > relay is also a guard, I'm in the latest blocklist[1] (pre-upgrade > > fingerprint). I did upgrade the system on April 9th to openssl > > 1.0.1-4ubuntu5.12 - base system is an ubuntu 12.04. > > > > According to the changelog[2], this should have fixed the heartbleed > > issue and according to this scanner[3], it should be as well. I did > > create new keys anyway, but just to be sure: Is the host[4] still > > affected as given in the blocklist? > > > > Best, > > Lars > > __ > > [1] > > https://atlas.torproject.org/#details/9AB511B6894566C1CF56043CE60077D213CF1A1A > > [2] https://launchpad.net/ubuntu/+source/openssl/1.0.1-4ubuntu5.12 > > [3] https://filippo.io/Heartbleed/#tor.kumbier.it > > [4] tor running on 5.9.165.90:443 > > A router at that IP with identity 9AB511B6894566C1CF56043CE60077D213CF1A1A > tested positive for Heartbleed several times, most recently at > 2014-04-17 10:19:18, before testing negative at 2014-04-17 18:51:46 (all > times UTC). If you rotate the key you should be fine, but that key is > potentially exposed. > No, I don't think that is sufficient. Not only must the onion keypair be replaced, but also the relay's identity keypair. Once the authorities have been told to reject the identity key with the fingerprint shown above, that relay will no longer be included in the consensus, nor will its published descriptor be distributed by them. The reason for rejecting the identity keys as well is that the identity secret key may just as easily have been leaked as the onion secret key. So, Lars, either destroy or rename all of your existing keys for tor, both secret and public, and then restart tor. It will not find existing keys during its startup phase and will therefore generate brand-new keys. After checking for reachability, it will publish a new descriptor. Within a couple of hours, the authorities will begin including the new relay in the consensus and distributing the descriptor. IOW, get rid of *all* the old keys, restart tor, and tor will handle the rest for you. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *or* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] NHS UK blocking Tor?
Chris Whittleston wrote: > Can someone else running a relay from their home connection confirm that > they get an 'Access denied' error from http://www.nhs.uk? I've checked with > someone using the same ISP in the flat above me and they seem able to > access the site just fine, as can I via mobile internet so I'm down to > suspecting that they are blocking all Tor relay IPs. This is the exact > error I get: > Access DeniedYou don't have permission to access "http://www.nhs.uk/"; on > this server. > > Reference #18.1f7f1002.1397514736.1fe2170c > The reference seems to change each time I visit. If this does turn out to > be them blocking Tor - advice on how to approach contacting them to resolve > this would be appreciated. > I closed the tab without noting the Reference #, so I can't comment, but yes, it did deny access. My relay is MYCROFTsOtherChild, and it and I are in Illinois. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at sdf.org *or* bennett at freeshell.org * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * ** ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
