[tor-relays] Dutch Relays
Hello, Emerald Onion is looking for co-location and IP transit opportunities in the Netherlands for deploying new exit relays. We have our own ASN, v4 and v6 IP space. Priorities are: - Free or cheap transit - Free or cheap cross-connects - Free or cheap colo - Amsterdam Internet Exchange - Netherlands Internet Exchange Working with collectives, or starting a new one, is interesting too. Please share your thoughts! -- yawnbox https://disobey.net/@yawnbox https://digitalcourage.social/@EmeraldOnion OpenPGP_0x4D3394A723A57C0A.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] obfs4 bridge management
Hello, I've has published a shell script for deploying and managing obfs4 bridges, I hope its useful to others. https://github.com/emeraldonion/bridge-management/ Feedback welcome! -- yawnbox https://disobey.net/@yawnbox https://digitalcourage.social/@EmeraldOnion OpenPGP_0x4D3394A723A57C0A.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] inet_csk_bind_conflict
I am happy to report that we have upgraded all our relays to Tor 0.4.8.0-alpha-dev and for the pst 8 days since the upgrade the bind conflict has ceased. No firewall rules are being used. No sysctl settings helped. -- Christopher Sheats (yawnbox) Executive Director Emerald Onion Signal: +1 206.739.3390 Website: https://emeraldonion.org/ Mastodon: https://digitalcourage.social/@EmeraldOnion/ > On Dec 12, 2022, at 1:18 PM, Anders Trier Olesen > wrote: > > > It is surprising, isn't it? It certainly feels like calling connect > > without first binding to an address should have the same effect as > > manually binding to an address and then calling connect, especially if > > the address you bind to is the same as the kernel would have chosen > > automatically. It seems like it might be a bug, but I'm not qualified to > > judge that. > Yes, I'm starting to think so too. And strange that Cloudflare doesn't > mention stumbling upon this problem in their blogpost on running out of > ephemeral ports. [1] > If I find the time, I'll make an attempt at understanding exactly what is > going on in the kernel. > > > If I am interpreting your results correctly, it means that either of the > > two extremes is safe > Yes. That is what I think too. > > > Anyway, thank your for the insight. I apologize if I was inconsiderate > > in my prior reply. > Likewise! > > Best regards > Anders Trier Olesen > > [1] > https://blog.cloudflare.com/how-to-stop-running-out-of-ephemeral-ports-and-start-to-love-long-lived-connections/ > > On Mon, Dec 12, 2022 at 4:16 PM David Fifield <mailto:da...@bamsoftware.com>> wrote: >> On Mon, Dec 12, 2022 at 12:39:50AM +0100, Anders Trier Olesen wrote: >> > I wrote some tests[1] which showed behaviour I did not expect. >> > IP_BIND_ADDRESS_NO_PORT seems to work as it should, but calling bind >> > without it >> > enabled turns out to be even worse than I thought. >> > This is what I think is happening: A successful bind() on a socket without >> > IP_BIND_ADDRESS_NO_PORT enabled, with or without an explicit port >> > configured, >> > makes the assigned (or supplied) port unavailable for new connect()s (on >> > different sockets), no matter the destination. I.e if you exhaust the >> > entire >> > net.ipv4.ip_local_port_range with bind() (no matter what IP you bind to!), >> > connect() will stop working - no matter what IP you attempt to connect to. >> > You >> > can work around this by manually doing a bind() (with or without an >> > explicit >> > port, but without IP_BIND_ADDRESS_NO_PORT) on the socket before connect(). >> > >> > What blows my mind is that after running test2, you cannot connect to >> > anything >> > without manually doing a bind() beforehand (as shown by test1 and test3 >> > above)! >> > This also means that after running test2, software like ssh stops working: >> > >> > When using IP_BIND_ADDRESS_NO_PORT, we don't have this problem (1 5 6 can >> > be >> > run in any order): >> >> Thank you for preparing that experiment. It's really valuable, and it >> looks a lot like what I was seeing on the Snowflake bridge: calls to >> connect would fail with EADDRNOTAVAIL unless first bound concretely to a >> port number. IP_BIND_ADDRESS_NO_PORT causes bind not to set a concrete >> port number, so in that respect it's the same as calling connect without >> calling bind first. >> >> It is surprising, isn't it? It certainly feels like calling connect >> without first binding to an address should have the same effect as >> manually binding to an address and then calling connect, especially if >> the address you bind to is the same as the kernel would have chosen >> automatically. It seems like it might be a bug, but I'm not qualified to >> judge that. >> >> If I am interpreting your results correctly, it means that either of the >> two extremes is safe: either everything that needs to bind to a source >> address should call bind with IP_BIND_ADDRESS_NO_PORT, or else >> everything (whether it needs a specific source address or not) should >> call bind *without* IP_BIND_ADDRESS_NO_PORT. (The latter situation is >> what we've arrived at on the Snowflake bridge.) The middle ground, where >> some connections use IP_BIND_ADDRESS_NO_PORT and some do not, is what >> causes trouble, because connections that do not use >> IP_BIND_ADDRESS_NO_PORT somehow "poison" the ephemeral port pool for >> connections
Re: [tor-relays] inet_csk_bind_conflict
> May I ask what your set up is? > Are you running your relays on separate VMs on the main system or are > you using a different set up like having all IP addresses on the same OS > and using OutboundBindAddress , routing, etc... to separate them? If I > know more, I might be able to make a script specific to your set up. Thank you. Yes, of course. Ubuntu server 22.04 runs on bare metal. Ansible-relayor manages 20 exit relays on each system. Netplan has each IP individually listed (sub-divided as a /25 per server from within a dedicated /24, similarly for v6 addresses). I believe an available IP is randomly picked by ansible-relayor and used statically in each torrc file. Here is an example torrc: # ansible-relayor generated torrc configuration file # Note: manual changes will be OVERWRITTEN on the next ansible-playbook run OfflineMasterKey 1 RunAsDaemon 0 Log notice syslog OutboundBindAddress 23.129.64.130 SocksPort 0 User _tor-23.129.64.130_443 DataDirectory /var/lib/tor-instances/23.129.64.130_443 ORPort 23.129.64.130:443 ORPort [2620:18c:0:192::130]:443 OutboundBindAddress [2620:18c:0:192::130] DirPort 23.129.64.130:80 Address 23.129.64.130 SyslogIdentityTag 23.129.64.130_443 ControlSocket /var/run/tor-instances/23.129.64.130_443/control GroupWritable RelaxDirModeCheck Nickname ageis ContactInfo url:emeraldonion.org proof:uri-rsa ciissversion:2 t...@emeraldonion.org Sandbox 1 NoExec 1 # we are an exit relay! ExitRelay 1 IPv6Exit 1 DirPort [2620:18c:0:192::130]:80 NoAdvertise DirPortFrontPage /etc/tor/instances/tor-exit-notice.html ExitPolicy reject 23.129.64.128/25:*,reject6 [2613:18c:0:192::]/64:*,accept *:*,accept6 *:* MyFamily # end of torrc -- Christopher Sheats (yawnbox) Executive Director Emerald Onion Signal: +1 206.739.3390 Website: https://emeraldonion.org/ Mastodon: https://digitalcourage.social/@EmeraldOnion/ > On Dec 4, 2022, at 10:08 PM, Chris wrote: > > Sorry to hear it wasn't much help. Even though the additions I suggested > didn't help they certainly couldn't cause any harm and can't be > responsible for the drops in traffic. > > As for the torutils scripts, I'm sure toralf would be able to better > investigate that but I have a feeling you have a certain set up that > might not have worked with the script. May I ask what your set up is? > Are you running your relays on separate VMs on the main system or are > you using a different set up like having all IP addresses on the same OS > and using OutboundBindAddress , routing, etc... to separate them? If I > know more, I might be able to make a script specific to your set up. > > On 12/3/2022 2:07 PM, Christopher Sheats wrote: >> Hello, >> >> Thank you for this information. After 24-hours of testing, these >> configurations brought Tor to a halt. >> >> At first I started with the sysctl modifications. After a few hours >> with just that, there was no improvement in ~75% >> inet_csk_bind_conflict utilization. I then installed Torutils for both >> IPv4 and IPv6. After only a couple of hours, Tor dropped to below 15 >> Mbps across both servers (40 relays). 16 hours later, Tor dropped >> below 2 Mbps. >> >> I've removed all of these new settings and restarted. >> >> -- >> Christopher Sheats (yawnbox) >> Executive Director >> Emerald Onion >> Signal: +1 206.739.3390 >> Website: https://emeraldonion.org/ >> Mastodon: https://digitalcourage.social/@EmeraldOnion/ >> >> >> >> >>> On Dec 2, 2022, at 7:30 AM, Chris wrote: >>> >>> Hi, >>> >>> As I'm sure you've already gathered, your system is maxing out trying to >>> deal with all the connection requests. When inet_csk_get_port is called >>> and the port is found to be occupied then inet_csk_bind_conflict is >>> called to resolve the conflict. So in normal circumstances you shouldn't >>> see it in perf top much less at 79%. There are two ways to deal with it, >>> and each method should be complimented by the other. One way is to try >>> to increase the number of ports and reduce the wait time which you have >>> somehow tried. I would add the following: >>> >>> net.ipv4.tcp_fin_timeout = 20 >>> >>> net.ipv4.tcp_max_tw_buckets = 1200 >>> >>> net.ipv4.tcp_keepalive_time = 1200 >>> >>> net.ipv4.tcp_syncookies = 1 >>> >>> net.ipv4.tcp_max_syn_backlog = 8192 >>> >>> The complimentary method to the above is to lower the number of >>> connection requests by removing the frivolous connection requests out of >>> the equation using a few iptables rules. >>> >>>
Re: [tor-relays] inet_csk_bind_conflict
server1:~$ ss -s Total: 454644 TCP: 465840 (estab 368011, closed 36634, orphaned 7619, timewait 11466) Transport Total IPIPv6 RAW 0 0 0 UDP 48480 TCP 42920641381515391 INET 42925441386315391 FRAG 0 0 0 81% inet_csk_bind_conflict server2:~$ ss -s Total: 460089 TCP: 477026 (estab 367786, closed 42817, orphaned 7456, timewait 17239) Transport Total IPIPv6 RAW 0 0 0 UDP 71710 TCP 43420941823515974 INET 43428041830615974 FRAG 1 1 0 80% inet_csk_bind_conflict (total combined throughput at the time of measurement was ~650 Mbps symmetrical per transit provider metrics, this low throughput volume is common when inet_csk_bind_conflict is this high) Re OutboundBindAddress - yes, for both v4 and v6 Re kernel version - 5.15.0-56-generic (jammy). Foundation for Applied Privacy recommended that we try the nightly repo which apparently includes the IP_BIND_ADDRESS_NO_PORT change. However that merge request mentions a workaround of modifying net.ipv4.ip_local_port_range, which we've already performed. -- Christopher Sheats (yawnbox) Executive Director Emerald Onion Signal: +1 206.739.3390 Website: https://emeraldonion.org/ Mastodon: https://digitalcourage.social/@EmeraldOnion/ > On Dec 3, 2022, at 3:02 AM, Anders Trier Olesen > wrote: > > Hi Christopher > > How many open connections do you have? (`ss -s`) > Do you happen to use OutboundBindAddress in your torrc? > > What I think we need is for the Tor developers to include this PR in a > release: https://gitlab.torproject.org/tpo/core/tor/-/merge_requests/579 > Once that has happened, I think the problem should go away, as long as you > run a recent enough Linux kernel that supports IP_BIND_ADDRESS_NO_PORT (since > Linux 4.2). > > - Anders > > > > > fre. 2. dec. 2022 kl. 09.24 skrev Christopher Sheats > mailto:yawn...@emeraldonion.org>>: >> Hello tor-relays, >> >> We are using Ubuntu server currently for our exit relays. Occasionally, exit >> throughput will drop from ~4Gbps down to ~200Mbps and the only observable >> data point that we have is a significant increase in inet_csk_bind_conflict, >> as seen via 'perf top', where it will hit 85% [kernel] utilization. >> >> A while back we thought we solved with with two /etc/sysctl.conf settings: >> net.ipv4.ip_local_port_range = 1024 65535 >> net.ipv4.tcp_tw_reuse = 1 >> >> However we are still experiencing this problem. >> >> Both of our (currently, two) relay servers suffer from the same problem, at >> the same time. They are AMD Epyc 7402P bare-metal servers each with 96GB >> RAM, each has 20 exit relays on them. This issue persists after upgrading to >> 0.4.7.11. >> >> Screenshots of perf top are shared here: >> https://digitalcourage.social/@EmeraldOnion/109440197076214023 >> >> Does anyone have experience troubleshooting and/or fixing this problem? >> >> Cheers, >> >> -- >> Christopher Sheats (yawnbox) >> Executive Director >> Emerald Onion >> Signal: +1 206.739.3390 >> Website: https://emeraldonion.org/ >> Mastodon: https://digitalcourage.social/@EmeraldOnion/ >> >> >> >> >> ___ >> tor-relays mailing list >> tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org> >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays signature.asc Description: Message signed with OpenPGP ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] inet_csk_bind_conflict
Hello, Thank you for this information. After 24-hours of testing, these configurations brought Tor to a halt. At first I started with the sysctl modifications. After a few hours with just that, there was no improvement in ~75% inet_csk_bind_conflict utilization. I then installed Torutils for both IPv4 and IPv6. After only a couple of hours, Tor dropped to below 15 Mbps across both servers (40 relays). 16 hours later, Tor dropped below 2 Mbps. I've removed all of these new settings and restarted. -- Christopher Sheats (yawnbox) Executive Director Emerald Onion Signal: +1 206.739.3390 Website: https://emeraldonion.org/ Mastodon: https://digitalcourage.social/@EmeraldOnion/ > On Dec 2, 2022, at 7:30 AM, Chris wrote: > > Hi, > > As I'm sure you've already gathered, your system is maxing out trying to > deal with all the connection requests. When inet_csk_get_port is called > and the port is found to be occupied then inet_csk_bind_conflict is > called to resolve the conflict. So in normal circumstances you shouldn't > see it in perf top much less at 79%. There are two ways to deal with it, > and each method should be complimented by the other. One way is to try > to increase the number of ports and reduce the wait time which you have > somehow tried. I would add the following: > > net.ipv4.tcp_fin_timeout = 20 > > net.ipv4.tcp_max_tw_buckets = 1200 > > net.ipv4.tcp_keepalive_time = 1200 > > net.ipv4.tcp_syncookies = 1 > > net.ipv4.tcp_max_syn_backlog = 8192 > > The complimentary method to the above is to lower the number of > connection requests by removing the frivolous connection requests out of > the equation using a few iptables rules. > > I'm assuming the increased load you're experiencing is due to the > current DDos attacks and I'm not sure if you're using anything to > mitigate that but you should consider it. > > You may find something useful at the following links > > [1](https://github.com/Enkidu-6/tor-ddos) > > [2](https://github.com/toralf/torutils) > > [background](https://gitlab.torproject.org/tpo/community/support/-/issues/40093) > > Cheers. > > On 12/1/2022 3:35 PM, Christopher Sheats wrote: >> Hello tor-relays, >> >> We are using Ubuntu server currently for our exit relays. >> Occasionally, exit throughput will drop from ~4Gbps down to ~200Mbps >> and the only observable data point that we have is a significant >> increase in inet_csk_bind_conflict, as seen via 'perf top', where it >> will hit 85% [kernel] utilization. >> >> A while back we thought we solved with with two /etc/sysctl.conf settings: >> net.ipv4.ip_local_port_range = 1024 65535 >> net.ipv4.tcp_tw_reuse = 1 >> >> However we are still experiencing this problem. >> >> Both of our (currently, two) relay servers suffer from the same >> problem, at the same time. They are AMD Epyc 7402P bare-metal servers >> each with 96GB RAM, each has 20 exit relays on them. This issue >> persists after upgrading to 0.4.7.11. >> >> Screenshots of perf top are shared >> here: https://digitalcourage.social/@EmeraldOnion/109440197076214023 >> >> Does anyone have experience troubleshooting and/or fixing this problem? >> >> Cheers, >> >> -- >> Christopher Sheats (yawnbox) >> Executive Director >> Emerald Onion >> Signal: +1 206.739.3390 >> Website: https://emeraldonion.org/ >> Mastodon: https://digitalcourage.social/@EmeraldOnion/ >> >> >> >> >> >> ___ >> tor-relays mailing list >> tor-relays@lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays signature.asc Description: Message signed with OpenPGP ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] inet_csk_bind_conflict
Hello tor-relays, We are using Ubuntu server currently for our exit relays. Occasionally, exit throughput will drop from ~4Gbps down to ~200Mbps and the only observable data point that we have is a significant increase in inet_csk_bind_conflict, as seen via 'perf top', where it will hit 85% [kernel] utilization. A while back we thought we solved with with two /etc/sysctl.conf settings: net.ipv4.ip_local_port_range = 1024 65535 net.ipv4.tcp_tw_reuse = 1 However we are still experiencing this problem. Both of our (currently, two) relay servers suffer from the same problem, at the same time. They are AMD Epyc 7402P bare-metal servers each with 96GB RAM, each has 20 exit relays on them. This issue persists after upgrading to 0.4.7.11. Screenshots of perf top are shared here: https://digitalcourage.social/@EmeraldOnion/109440197076214023 Does anyone have experience troubleshooting and/or fixing this problem? Cheers, -- Christopher Sheats (yawnbox) Executive Director Emerald Onion Signal: +1 206.739.3390 Website: https://emeraldonion.org/ Mastodon: https://digitalcourage.social/@EmeraldOnion/ signature.asc Description: Message signed with OpenPGP ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] problem with new obfs4 bridge
; at '[::]:443' Mar 26 11:38:37.000 [notice] Bootstrapped 5% (conn): Connecting to a relay Mar 26 11:38:37.000 [notice] Opening Socks listener on /run/tor/socks Mar 26 11:38:37.000 [notice] Opened Socks listener connection (ready) on /run/tor/socks Mar 26 11:38:37.000 [notice] Opening Control listener on /run/tor/control Mar 26 11:38:37.000 [notice] Opened Control listener connection (ready) on /run/tor/control Mar 26 11:38:37.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay Mar 26 11:38:37.000 [notice] Bootstrapped 14% (handshake): Handshaking with a relay Mar 26 11:38:37.000 [notice] Bootstrapped 15% (handshake_done): Handshake with a relay done Mar 26 11:38:37.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits Mar 26 11:38:37.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits Mar 26 11:38:37.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit Mar 26 11:38:38.000 [notice] Bootstrapped 100% (done): Done Mar 26 11:38:38.000 [notice] Now checking whether IPv4 ORPort 23.129.64.194:80 is reachable... (this may take up to 20 minutes -- look for log messages indicating success) Mar 26 11:38:38.000 [notice] Now checking whether IPv6 ORPort [2620:18c:0:192::194]:80 is reachable... (this may take up to 20 minutes -- look for log messages indicating success) Mar 26 11:38:39.000 [notice] Self-testing indicates your ORPort [2620:18c:0:192::194]:80 is reachable from the outside. Excellent. Mar 26 11:38:39.000 [notice] Self-testing indicates your ORPort 23.129.64.194:80 is reachable from the outside. Excellent. Publishing server descriptor. Mar 26 11:40:32.000 [notice] Performing bandwidth self-test...done. A prior torrc config set the IPs explicitly, but had the same result: ServerTransportListenAddr obfs4 23.129.64.194:443 ServerTransportListenAddr obfs4 [2620:18c:0:192::194]:443 I can provide debug logs as necessary. Possibly of note, our firewall does not use connection tracking and applies the same rules as our exit relays which use the same ports. Pro-active note: The bridge shares the same 23.129.64.0/24 subnet as Emerald Onion's Tor exit relays, so there is no risk of a user entering and exiting our physical network (see "2.2. Path selection and constraints"): https://github.com/torproject/torspec/blob/master/path-spec.txt Cheers, -- Christopher Sheats Executive Director for Emerald Onion Email: yawn...@emeraldonion.org Phone: +1 206-739-3390 Web: https://emeraldonion.org/ ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] DDoS’d offline
0 13:26:23 -0700 23.129.64.183 1810 mbps 2019-10-30 13:26:23 -0700 23.129.64.187 1786 mbps 2019-10-30 13:26:24 -0700 23.129.64.184 1870 mbps 2019-10-30 13:26:24 -0700 23.129.64.189 1796 mbps 2019-10-30 13:26:24 -0700 23.129.64.182 1868 mbps 2019-10-30 13:26:24 -0700 23.129.64.186 1872 mbps 2019-10-30 13:26:24 -0700 23.129.64.190 1751 mbps 2019-10-30 13:26:24 -0700 23.129.64.192 1828 mbps 2019-10-30 13:26:24 -0700 23.129.64.194 1891 mbps 2019-10-30 13:26:24 -0700 23.129.64.196 1801 mbps 2019-10-30 13:26:24 -0700 23.129.64.195 1814 mbps 2019-10-30 13:26:24 -0700 23.129.64.201 1849 mbps 2019-10-30 13:26:24 -0700 23.129.64.203 1894 mbps 2019-10-30 13:26:24 -0700 23.129.64.206 1885 mbps 2019-10-30 13:26:24 -0700 23.129.64.208 1801 mbps 2019-10-30 13:26:24 -0700 23.129.64.210 1840 mbps 2019-10-30 13:26:24 -0700 23.129.64.211 1853 mbps 2019-10-30 13:26:24 -0700 23.129.64.212 1776 mbps 2019-10-30 13:26:24 -0700 23.129.64.213 1818 mbps 2019-10-30 13:26:24 -0700 23.129.64.216 1945 mbps 2019-10-30 13:26:24 -0700 23.129.64.215 1729 mbps -- Christopher Sheats Executive Director for Emerald Onion Email: yawn...@emeraldonion.org Office (& Signal): +1-206-739-3390 Website: https://emeraldonion.org/ From: tor-relays on behalf of Christopher Sheats Sent: Wednesday, October 30, 2019 5:26:21 PM To: tor-relays@lists.torproject.org Subject: [tor-relays] DDoS’d offline fyi https://twitter.com/EmeraldOnion/status/1189668679752900608 Calyx appears to have been hit also, but not offline https://twitter.com/calyxinstitute/status/1189693027192840192 -- Christopher Sheats Executive Director for Emerald Onion Email: yawn...@emeraldonion.org Office (& Signal): +1-206-739-3390 Website: https://emeraldonion.org/ ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] DDoS’d offline
fyi https://twitter.com/EmeraldOnion/status/1189668679752900608 Calyx appears to have been hit also, but not offline https://twitter.com/calyxinstitute/status/1189693027192840192 -- Christopher Sheats Executive Director for Emerald Onion Email: yawn...@emeraldonion.org Office (& Signal): +1-206-739-3390 Website: https://emeraldonion.org/ ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Emerald Onion's new relays
Hello, Just a report-back: https://twitter.com/EmeraldOnion/status/1160694122069422080 "It took us 128 days to go from 1Gbps to 10Gbps advertised bandwidth with the progressive deployment of up to 56 relays (April 2 to August 8, 2019). In terms of actual bandwidth, we have a sustained throughout of roughly 5-6Gbps. We can sustain up to 15Gbps and burst to 20Gbps." Currently we're the fifth largest exit family at 1.2220% consensus weight, 1247.67 MiB/s advertised bandwidth, and 5.5546% exit probability. It costs us $1,600 per month, in donations, to manage this service. As you may know, we are all volunteers. There was some additional feedback that we got from Nusenu regarding capping bandwidth of individual Tor processes, and using a second IP for exit traffic. We are still trying to determine the per-process max bandwidth on our two hardware platforms, but we are planning on enabling these caps for quality of the service. When we originally deployed many new relays using Ansible-Relayor, we used a second IP per process. However, due to routing troubleshooting and relays not showing up on Relay Search, we removed the second IPs. We are still using only one per relay. We also found it challenging to manage this many relay IPs with Ansible-Relayor. Ansible-Relayor uses sequential IPs based on the listing from ifconfig. This presents a challenge because it is difficult to setup forward and reverse DNS in a predictable way. The second issue we have with running a secondary IP per Tor process is system load. Having more IPs opens more sockets, and we are already putting a lot of load on these multi-core servers. The third issue is that when people block our IPs, they block the scope. Should we use a secondary IP per relay if the IP is in the same scope? Should we then use two /24 scopes, and wouldn't both scopes end up getting blocked? If we were to use a second /24 for relays, how will Ansible-Relayor know to use a second IP scope for exiting? IPv4 dependency is a real burden. We would like to see Tor Project help Tor network operators more directly, both financially and securing IPv4 scopes for nonprofit organizations to take ownership of. The latter is needed until we can stop using IPv4 completely and operate only with IPv6. It is discouraging to see so many small and large network operators not using IPv6. Why is this such a problem? Tor Project, please increase your #IPv6 awareness/outreach similar to how ARIN and the other RIRs try very hard to do. Cheers, -- Christopher Sheats Executive Director for Emerald Onion Email: yawn...@emeraldonion.org Office (& Signal): +1-206-739-3390 Website: https://emeraldonion.org/ -Original Message- From: tor-relays On Behalf Of grarpamp Sent: Thursday, April 4, 2019 12:45 AM To: tor-relays@lists.torproject.org Subject: Re: [tor-relays] Emerald Onion's new relays On 4/4/19, Conrad Rockenhaus wrote: >> when ISPs are ordered to BGP blackhole some exit IP addresses > I've been assigning a second set of IP addresses to my servers in case > I want to run another instance of Tor. Would it be more prudent to use > that second set of IP addresses as an OutboundBindAddressExit instead > and use different ports as a better practice? ISP traffic filtering sinks, from the tor browser perspective, affecting traffic exiting a relay out through its exitpolicy to clearnet, can be... - dst based "sink traffic to there", appears as "cnn.com down", a minor issue, depending on scope of the sink. - src based "sink traffic from there", appears as "Internet down", a major issue, depending on scope of the sink. Unlike websites, and unless they're tied playing [geo]politics, ISP's really don't like to keep these sinks in place for a long time. Relay management such as OS updates, ssh, wget could get blocked if those addresses are in consensus. Then there is relay-to-relay traffic types that don't "exit", but can still get found and blocked. And the OR IP must be obviously not be blocked, else depending on scope, the relay won't receive traffic to move out any horizon. Tor should still allow config of 2 tor instances on one IP. If IP's are "free", and if operator survey says the exit functions are getting knocked off the tor network more often than entire OR's, try putting out the OutboundBindAddressExit on IP for sacrifice, instead of burning entire OR's which could otherwise be used more quietly as middle relays etc. An operators own cost, management, and ISP relationships may show running more relays is better IP, or net traffic pushed, wise than enduring a few sinks now and then. Probably every situation is different. Or try both and see. Common options from the manpage... Address address ORPort [address:]PORT|auto [flags]
[tor-relays] Emerald Onion's new relays
Hello, Beginning this Wednesday, 2019-Apr-03, Emerald Onion will be starting 40 new uncapped and unfiltered exit relays using HardenedBSD in Seattle. We will turn on 10 new relays per day and be monitoring performance. In tandem, we will be publishing our updated edge routing and Tor relay configurations on our Github. We'll also publish an updated transparency report (spoiler: zero new requests). This work is part of our efforts to saturate our new unmetered 10Gbps transit link in the Westin data center. There we peer with the Seattle Internet Exchange with a separate 10Gbps connection. We'll be using our existing IP scopes: 2620:18C::/36 23.129.64.0/24 We are in the process of creating an RPKI ROA for our prefixes, and we are still rebuilding our DNS resolver (thanks nusenu for the feedback). These things will be completed before we turn on the new relays. If you donated to Emerald Onion (a 501(c)(3)) nonprofit), thank you! We'll be naming the relays accordingly. We will still accept new donations for relay naming rights-- email me directly for more information. Lastly, and If you're unaware, I spoke about our work at DEFCON 26: https://www.youtube.com/watch?v=cs6a1i4Owic Cheers, -- Christopher Sheats Executive Director for Emerald Onion Email: yawn...@emeraldonion.org Office (& Signal): +1-206-739-3390 Website: https://emeraldonion.org/ ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] TAO on tor ops
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, On 07/03/2014 02:13 PM, o...@riseup.net wrote: > Do we have to assume to get targeted by TAO? > Should we dump hardware ordered online? > > > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays I think that this is a valid risk to look into. Given the history/information in the Snowden docs, I assume that it is not possible / costs too much to perform active, local surveillance on even a majority of Tor relays. I also have doubts that local surveillance on relays is at all beneficial. However, I think there is greater risk for exit routers, particularly ones that routinely receive the guard flag. Looking forward to further discussion. - -- Christopher Sheats yawn...@gmail.com GnuPG: 8397 7B9F D8BA 3EE5 71EF FDF3 C761 02B0 A531 D73D -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTtg0dAAoJEMdhArClMdc9no4P/2CGQHkutSX05CrwQW2gB4B4 yyJQaJnVB8ulImefweuGhr3RMiSscv/jhQUHhU2qGWXgLfgf2uOJYLyyB/BkWVyu NU8waDJpNNg9061vullthrGlrHP29MczLtvUTFzua72BWU9a5I9vLYoxOdoJFQiH GM7vg7aXVioahEgRoLE/nQKoBNFoSxhRmNupQyi41mZXY/eKxkcgt2ebmHZv7ymi k2cFEU+nBLl6SZGJ+HZ6korfH9UZsm5bla0sm/lmOdTNIKJGEbx8RtzkEFDPvP5C hOO6OM7dPLAlaegOk2pwFRBS1TkE6hpT0dWEGlUslXrASc6Z0HD4/cNxqwLW9/Z+ ZKLINtW59jYsu5SppnWN1UOBnHtqaXG62JLqKz984a7xpCaSyJ9Zvu3Hz55VdDUL r2ngbI5yPyc4ephNm1iW5acaiDpBRpiJSHphjsPHMAb9t5kYm/L9+yg4GNB+9dgg WpXoChwqVtxjBJ42MVNuYPzgiVpbSrQTF8uYL6OxyrYUgEFhItXNUSAUEPc6CvI6 FZeJvx5IoAXe+zCirlZfe012lY+CEf3P5SD6tTuWLTsv8UIrIhDvnmhgD7C1xSE1 R0nPqU/ohisvpfJDOSYVm1Ot/T08K3usErTX6Vur1jl4uimKuAJUpHxAUbWm2MMo niwyf332rXwqZ4bN9pxY =VF2E -END PGP SIGNATURE- ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Why is UFW bllocking allowed TOR traffic?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Jeff On 06/22/2014 12:43 PM, Jeff Odell wrote: > I was monitoring UFW today and noticed that it was periodically blocking > allowed TOR traffic. any ideas why from those with more experience than I? > > > toradmin@IrvineTorExit:~$ sudo ufw status > Status: active > > To Action From > -- -- > 22 ALLOW Anywhere > 9001/tcp ALLOW Anywhere > 9030/tcp ALLOW Anywhere > 80 ALLOW Anywhere > 22 (v6)ALLOW Anywhere (v6) > 9001/tcp (v6) ALLOW Anywhere (v6) > 9030/tcp (v6) ALLOW Anywhere (v6) > 80 (v6)ALLOW Anywhere (v6) > > > toradmin@IrvineTorExit:~$ sudo tail -f /var/log/syslog | grep DPT=9001 > > Jun 22 15:38:12 IrvineTorExit kernel: [ 2159.246977] [UFW BLOCK] IN=eth0 OUT= MAC=04:01:1b:5e:9a:01:28:8a:1c:64:cf:f0:08:00 SRC=92.108.200.200 DST=188.226.199.250 LEN=52 TOS=0x00 PREC=0x00 TTL=120 ID=10392 DF PROTO=TCP SPT=52000 DPT=9001 WINDOW=16652 RES=0x00 ACK URGP=0 > Jun 22 15:38:12 IrvineTorExit kernel: [ 2159.246988] [UFW BLOCK] IN=eth0 OUT= MAC=04:01:1b:5e:9a:01:28:8a:1c:64:cf:f0:08:00 SRC=92.108.200.200 DST=188.226.199.250 LEN=52 TOS=0x00 PREC=0x00 TTL=120 ID=10396 DF PROTO=TCP SPT=52000 DPT=9001 WINDOW=16652 RES=0x00 ACK URGP=0 > > Regards, > Jeff > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays I see a considerable amount of these in my logs (Ubuntu 14.04 server, UFW). Some time ago I asked about this on IRC with no resolve. I'm afraid of it affecting Tor users (I don't know if it is), and I'm afraid of these logs being created and stored on my exit relay. Because I have received no answer, yet need to protect my relay with a manageable firewall, I took the advice of someone on IRC and disabled my UFW logs (my exit relay isn't used for anything else, and UFW will keep doing it's job, while protecting the privacy of Tor users). You can do this too via: sudo ufw logging off By the way, you may wish to "limit" port 22 instead, to prevent SSH brute force attacks. sudo ufw limit 22/tcp (I don't allow 22/udp) hope this helps a little. - -- Christopher Sheats yawn...@gmail.com GnuPG: 8397 7B9F D8BA 3EE5 71EF FDF3 C761 02B0 A531 D73D -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTtJnGAAoJEMdhArClMdc9eSUP/2XrazjtRQm1Z9rZrGOnWwe1 pJpVuLcsVq34yFIz9xonRnV6DohF+p4Ra1Umq/hnxxJU9X2LOcF44sekhVxguwIO +LJ/hIE+FGuR1U0nlJiILiLO8vrwYUdfNcZ4EpOO4ZgSe1lG2gC2efeFdYZbXREO xqmdunUv6bAHpOoWYrWPwG7R0dTQU9Zzf9HbJrjjY+ubQepHr9Wj+FDNp0iRXZYw V+VFhdGk2FWODQrpbPfX0G7+uf2itM4ONBf76DNbyudefA5E091YjuTUiQhuIal1 Kdb59YsUME0Nxc8apl0WTUbOW0DmCJtIsYKgPlyNoz9/6R7Bi9VTqLWcsrz8xRGa Z09j1/bzpQ8Cp6HWG92RpfCQfA3KUYKN2jUh/IeQRZfZtIc+viCHNys76PRv209T hHgjLiNfzWv2PYKoko/ZrB5ZH8OvG8fWtIY2cinc/1rSBobAD88/oWn39EIVeUHu JfYXBmc3WhYghGPbl4y4bczuKtdcItldLH8RAABTDZ8bFpxqgA1vRbT7oyFOuU+V iZtbY3EB7CUkN9X8E7DbQoLQxMDXEE36RJ5hnJLe68VE5wMQx8vGwFzoOG125d23 xB8CYIkp+VB5bUDaTD5JHghEmeKH+RKGLpX+ICBy+Bp6/AK4WjXg9I4zERrLAQWL KdYf6bA/ZUwLrFCZYI6o =04+T -END PGP SIGNATURE- ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
