[tor-dev] DirAuth usage and 503 try again later
Sebastian, Thank you for comments. First of all, sorry if torpy hurt in some way Tor Network. It was unintentionally. In any case, it seems to me that if there was some high-level description of logic for official tor client, it would be very useful. >First, I found this string in the code: "Hardcoded into each Tor client >is the information about 10 beefy Tor nodes run by trusted volunteers". >The word beefy is definitely wrong here. The nodes are not particularly >powerful, which is why we have the fallback dir design for >bootstrapping. At first glance, it seemed that the AuthDirs were the most trusted and reliable place for obtaining consensus. Now I'm understand more. >The code counts Serge as a directory authority which signs the >consensus, and checks that over half of the dirauths signed it. But >Serge is only the bridge authority and never signs the consensus, so >torpy will reject some consensuses that are indeed valid. Yep, here you right. Thanks for pointing out. >Once this >happens, torpy goes into a deathly loop of "consensus invalid, >trying again". There are no timeouts, backoffs, or failures noted. Not really, because torpy has only 3 retries for getting consensus. But probably you are right because user code probably can do retry calling torpy in a loop. So that will always try download network_status... If you have some sort of statistic about increasing traffic we can compare that with time when was consensus signed by 4 signers which enough for tor but not enough for torpy. >The code frequently throws exceptions, but when an exception occurs >it just continues doing what it was doing before. It has absolutely >no regards to constrain its resources when using the Tor network. What kind of constraints can you advise? >The logic that if a network_status document was already downloaded that >is used rather than trying to download a new one does not work. It works. But probably not in optimal way. It caches network_status only. >I have >a network_status document, but the dirauths are contacted anyway. >Perhaps descriptors are not cached to disk and downloaded on every new >start of the application? Exactly. Descriptors and network_status diff every hour was asking always from AuthDirs. >New consensuses never seem to be downloaded from guards, only from >dirauths. Thanks for pointing out. I looked more deeply into tor client sources. So basically if we have network_status we can use guard nodes to ask network_status and descriptors from them. Otherwise using fallback dirs to download network_status. I've implemented such logic in last commit. >There are probably more things suboptimal that I missed here. If you find more please let me know. It really helpful. >Generally, I think torpy needs to implement the following quickly if it >wants to stop hurting the network. This is in order of priority, but I >think _ALL_ (maybe more) are needed before torpy stops being an abuser >of the network: > >- Stop automatically retrying on failure, without backoff I've added delays and backoff between retries. >- Cache failures to disk to ensure a newly started torpy_cli does not > request the same resources again that the previous instance failed to > get. That will be on the list. But probably even if there is a loop level above and without this feature but with backoff it will be delays like: 3 sec, 5, 7, 9; 3, 5, 7, 9. Seems ok? >- Fix consensus validation logic to work the same way as tor cli (maybe > as easy as removing Serge) Done. Only auth dirs with V3_DIRINFO flag will be counted. It wasn't obvious =( >- use microdescs/consensus, cache descriptors On the list. Moreover, I've switched to using fallback dirs instead of auth dirs and to guards if torpy has "reasonable" live consensus. > Defenses are probably necessary to implement even if >torpy can be fixed very quickly, because the older versions of torpy >are out there and I assume will continue to be used. Hopefully that >point is wrong? I believe that old versions doesn't work any more because them could not connect to auth dirs. Users getting 503 many times, so they will update client. I hope. Thank you very much. And sorry again. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] DirAuth usage and 503 try again later
Good day. Is there any chance that torpy (https://github.com/torpyorg/torpy) was triggered this issue https://gitlab.torproject.org/tpo/core/tor/-/issues/33018 ? Some wary facts: - Torpy using old fashion consensus (not mircodesc) - When consensus not present in cache (first time usage) it downloads consensus from random directory authorities only. - Before August 2020 it was using plain HTTP requests to DirAuths. Now it creates "CREATE_FAST" circuits to DirAuths (is that right way by the way?) From other side: - Torpy store consensus on disk (so whenever client restart it must not download full consensus again) - It will try download consensus after time which sets by valid_time field from consensus which more than 1 hour (so it's not so often) - Torpy try get consensus by "diff" feature (so it's minimize traffic) Still may be some of this features not working well in some conditions. Which could cause a lot of consensus downloads in Jan 2020... Or may be you know more info about this situation? Do you have some recommendations for tor client implementation? Can you explain in several paragraphs what behavior of original tor client is? As far as I understand when first time original tor starts it tries download consensus from fallback dirs not from DA? Is this key point? There is one more issue https://gitlab.torproject.org/tpo/core/tor/-/issues/40239 which I'm not understand correctly. Let's imagine it's first run of tor client and that time coincidentally coincided with DA voting. That means client will not be able to download consensus? That is strange decision. Or do you mean clients must download consensus from fallback dirs which never in "voting" process? ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] New python Tor client implementation
Hi Damian, Sorry for the late reply. There are two things why I didn't use Stem. 1. A number of python implementations I've found on the internet had broken dependencies or didn't work at all. So, I decided to create a fully functional Tor client with the bare minimum of dependencies. 2. Up until now I thought Stem was just a library to control the Tor process. As it said at front of https://stem.torproject.org: "With it you can use Tor's control protocol to script against the Tor process". Unfortunately API docs didn't help me to see that certain Tor primitives were indeed implemented. >If you'd care to integrate any of this functionality into Stem I'd be >delighted to work with you. I didn't yet think about integration. It's always a matter of spare time. Anyway, you can freely use the code or send me bugs. In turn, I can help you to resolve the issues. Cheers. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] New python Tor client implementation
Hello all, Recently I finished a pure python implementation of the Tor client. It's called torpy (https://github.com/torpyorg/torpy). It offers handy API, supports v2 hidden services with "basic" and "stealth" authorization protocol. Works with python 3.6+.It has no dependencies on the original C Tor client and Stem. For more information please take a look at README on github. Here is a quick example of how to use the library:```pythonfrom torpy import TorClient hostname = 'ifconfig.me' # or onion-services as well, for example 'http://facebookcorewwwi.onion'tor = TorClient() # Choose random guard node and create 3-hops circuitwith tor.create_circuit(3) as circuit: # Create tor stream to host with circuit.create_stream((hostname, 80)) as stream: # Now we can communicate with host stream.send(b'GET / HTTP/1.0\r\nHost: %s\r\n\r\n' % hostname.encode()) recv = stream.recv(1024)``` Please list torpy project at https://trac.torproject.org/projects/tor/wiki/doc/ListOfTorImplementations It would be nice if you try the client. I look forward to any feedback. Cheers. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Adding Single Onion Service support to Bitcoin Core
Core is preparing to support this new tor feature. Any helpful suggestions would be appreciated. :) https://github.com/bitcoin/bitcoin/issues/9836 https://github.com/bitcoin/bitcoin/pull/10161 Thanks for all the great onion work! James ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
