Re: Suspicious Circuits
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
> A) This explains why it is trying the old introduction points, and it
> explains why it's building a long circuit trying each one in turn. But
> why is it trying the same introduction point more than once?
Uhhhm, right. The problem is that introduction points are not removed on
failure, which they should be. It's quite unusual that introduction
points fail (and most people won't notice that), but okay, we should
better fix it. My first guess is that this problem was introduced in
0.2.0.7-alpha (ChangeLog entry: "- Hidden services were choosing
introduction points uniquely by hexdigest, but when constructing the
hidden service descriptor they merely wrote the (potentially ambiguous)
nickname.") and that hex-encoded identities are compared with nicknames
when removing introduction points. But it's quite hard to tell without
running it. On the other hand, this change was also backported to
0.1.2.18 which did not have this problem. Hmm.
Let's find it out. :) As discussed on #tor yesterday, I attached a patch
to this message containing some more logging statements. Could you,
Kyle, apply it and run the same configuration as you did last time? You
should also enforce the same timing problem, because without it's
unlikely that intro points fail and respond with a NAK message.
Thanks for that! :)
> B) Do you think it's possible to reduce the "30 second delay" to make
> this sort of behavior happen less often? It would be nice to have hidden
> services launch more 'immediately'. But on more thought, I think 30
> seconds may already be a bare minimum, if we consider users on crappy
> connections setting up hidden services. Hm.
Good question. 30 seconds are not really much compared to the overall
performance of hidden services. How important is it that hidden services
are more immediately available after setting them up? This is only done
once in a while. Does this affect user experience so much? I think that
the behavior in Kyle's case is a really rare event, compared to the
normal use cases for hidden services. So, I would not change it in favor
of fewer and more accurate descriptor publications.
- --Karsten
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHak1D0M+WPffBEmURAhe+AJwMWLUlZCiroiU7BBxIwL5vkd813ACfc4Q8
AKAxnBr9Apq13uhz5gubZIM=
=jY7A
-END PGP SIGNATURE-
Index: /home/karsten/diss/tor/tor-0_2_0_12_alpha/src/or/rendclient.c
===
--- /home/karsten/diss/tor/tor-0_2_0_12_alpha/src/or/rendclient.c
(revision 12873)
+++ /home/karsten/diss/tor/tor-0_2_0_12_alpha/src/or/rendclient.c
(working copy)
@@ -337,11 +337,21 @@
return 0;
}
+ log_info(LD_REND, "Removing failed intro point for service %s. "
+"We still have a valid descriptor with %d intro points.",
+escaped_safe_str(query), ent->parsed->n_intro_points);
if (ent->parsed->intro_point_extend_info) {
+log_info(LD_REND, "The descriptor has its intro points stored as extend "
+ "infos rather than nicknames. Failed intro point is %s",
+ hex_str(failed_intro->identity_digest, DIGEST_LEN));
for (i=0; i < ent->parsed->n_intro_points; ++i) {
+ log_info(LD_REND, "Comparing with intro point %s...",
+ hex_str(ent->parsed->intro_point_extend_info[i]->identity_digest,
+ DIGEST_LEN));
if (!memcmp(failed_intro->identity_digest,
ent->parsed->intro_point_extend_info[i]->identity_digest,
DIGEST_LEN)) {
+log_info(LD_REND, "Found a match! Removing intro point.");
tor_assert(!strcmp(ent->parsed->intro_points[i],
ent->parsed->intro_point_extend_info[i]->nickname));
tor_free(ent->parsed->intro_points[i]);
@@ -352,11 +362,19 @@
ent->parsed->intro_point_extend_info[i] =
ent->parsed->intro_point_extend_info[ent->parsed->n_intro_points];
break;
+ } else {
+log_info(LD_REND, "No match.");
}
}
} else {
+log_info(LD_REND, "The descriptor has its intro points stored as "
+ "nicknames rather than extend infos. Failed intro "
+ "point is %s", failed_intro->nickname);
for (i=0; i < ent->parsed->n_intro_points; ++i) {
+log_info(LD_REND, "Comparing with intro point %s...",
+ ent->parsed->intro_points[i]);
if (!strcasecmp(ent->parsed->intro_points[i], failed_intro->nickname)) {
+log_info(LD_REND, "Found a match! Removing intro point.");
tor_free(ent->parsed->intro_points[i]);
ent->parsed->intro_points[i] =
ent->parsed->intro_points[--ent->parsed->n_intro_points];
@@ -361,9 +379,13 @@
ent->parsed->intro_points[i] =
ent->parsed->intro_points[--ent->parsed-
Re: Suspicious Circuits
On Wed, Dec 19, 2007 at 03:35:02PM +0100, Karsten Loesing wrote: > Subsequently, at 16:07:12 you restarted Tor and made it establish new > introduction points at "otherator2", "crelm", "bytebutlerfive" and > publish a new descriptor containing these introduction points at > 16:07:53. Again, the delay of 41 seconds is intentional. But---and this > is the problem---when accessing your service at 16:07:25, Tor downloaded > the first descriptor without being able to know that it's obsolete. So, > Tor tried to connect to "Slowpoke" and the other introduction points > which were not acting as introduction points for your service any more. > That's why you get those NAKs which lead to re-extending the failed > introduction circuits which is also normal behavior. > > Hence, there is not a problem in the Tor code. Hi Karsten, A) This explains why it is trying the old introduction points, and it explains why it's building a long circuit trying each one in turn. But why is it trying the same introduction point more than once? B) Do you think it's possible to reduce the "30 second delay" to make this sort of behavior happen less often? It would be nice to have hidden services launch more 'immediately'. But on more thought, I think 30 seconds may already be a bare minimum, if we consider users on crappy connections setting up hidden services. Hm. Thanks, --Roger
Re: Suspicious Circuits
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Roger Dingledine wrote: > On Sun, Dec 09, 2007 at 09:19:53PM -0800, Kyle Williams wrote: >> I've been having problems getting to hidden services the last couple of >> days. >> I noticed something odd in Vidalia the other day, but it was gone before I >> could take a screenshot. >> However this evening, I was having a lot of problems with .onion addresses, >> and Vidalia was showing several (more than 6) nodes in a circuit almost >> every time I tried to reach any hidden service, including my own. > > Exciting. Looks like a bug of some sort. No, I don't think it's a bug. - From the log file that you, Kyle, gave me yesterday I can see that you started Tor at 16:04:51 which established introduction points at "Slowpoke", "dpujtk", and "server" and published a descriptor for your service at 16:05:27. The delay of 36 seconds comes from the fact that Tor waits at least 30 seconds for a descriptor to be stable before publishing it. Then you made a connection attempt at 16:06:26 which succeeded at 16:06:53 and another attempt at 16:07:00 succeeding at 16:07:02. Everything fine so far. Subsequently, at 16:07:12 you restarted Tor and made it establish new introduction points at "otherator2", "crelm", "bytebutlerfive" and publish a new descriptor containing these introduction points at 16:07:53. Again, the delay of 41 seconds is intentional. But---and this is the problem---when accessing your service at 16:07:25, Tor downloaded the first descriptor without being able to know that it's obsolete. So, Tor tried to connect to "Slowpoke" and the other introduction points which were not acting as introduction points for your service any more. That's why you get those NAKs which lead to re-extending the failed introduction circuits which is also normal behavior. Hence, there is not a problem in the Tor code. In general, when performing tests, you should give Tor a little bit more time to stabilize, especially for hidden services. You should also consider not to run both server and client on the same Tor instance. If the problem persists even when waiting for some more time, please report! Timing is everything! :) - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHaSwW0M+WPffBEmURAhRlAJ90qOxY2wYA6Vq/sw0VCMzn75zRZgCeJoZH MUzuCbvRrIRN/4705ieI+s4= =Uad9 -END PGP SIGNATURE-
RE: Suspicious Circuits
0.1.2.18 is what I meant, hehe. - Kyle -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Roger Dingledine Sent: Monday, December 10, 2007 10:24 AM To: or-talk@freehaven.net Subject: Re: Suspicious Circuits On Sun, Dec 09, 2007 at 09:19:53PM -0800, Kyle Williams wrote: > I've been having problems getting to hidden services the last couple of > days. > I noticed something odd in Vidalia the other day, but it was gone before I > could take a screenshot. > However this evening, I was having a lot of problems with .onion addresses, > and Vidalia was showing several (more than 6) nodes in a circuit almost > every time I tried to reach any hidden service, including my own. Exciting. Looks like a bug of some sort. > These circuits have more than 3 nodes, and as shown in the picture, has the > same node more than once in the circuit. Correct me if I'm wrong, but I > thought circuits are not allowed to have the same node more than once. We make an exception for client-side introduction circuits: the first three hops ensure your anonymity, and after that we try one random node at a time out of the set of introduction points listed in the hidden service descriptor. If one of the intro points doesn't cooperate (for example because the hidden service is no longer connected to that introduction point, so they *can't* help), then we move on to the next. We keep reusing the same circuit (just extending it another hop each time) so we don't waste a lot of time making new circuits. But we're *supposed* to be marking each intro point down when it isn't helpful, so the fact that we're extending back to it (repeatedly sometimes, based on your jpgs!) is definitely a bug of some sort. Can you get me a debug-level log from when this happens? Also, you mentioned that the bug doesn't happen on 0.1.2.8-alpha. What about 0.1.2.18? Thanks, --Roger
Re: Suspicious Circuits
On Sun, Dec 09, 2007 at 09:19:53PM -0800, Kyle Williams wrote: > I've been having problems getting to hidden services the last couple of > days. > I noticed something odd in Vidalia the other day, but it was gone before I > could take a screenshot. > However this evening, I was having a lot of problems with .onion addresses, > and Vidalia was showing several (more than 6) nodes in a circuit almost > every time I tried to reach any hidden service, including my own. Exciting. Looks like a bug of some sort. > These circuits have more than 3 nodes, and as shown in the picture, has the > same node more than once in the circuit. Correct me if I'm wrong, but I > thought circuits are not allowed to have the same node more than once. We make an exception for client-side introduction circuits: the first three hops ensure your anonymity, and after that we try one random node at a time out of the set of introduction points listed in the hidden service descriptor. If one of the intro points doesn't cooperate (for example because the hidden service is no longer connected to that introduction point, so they *can't* help), then we move on to the next. We keep reusing the same circuit (just extending it another hop each time) so we don't waste a lot of time making new circuits. But we're *supposed* to be marking each intro point down when it isn't helpful, so the fact that we're extending back to it (repeatedly sometimes, based on your jpgs!) is definitely a bug of some sort. Can you get me a debug-level log from when this happens? Also, you mentioned that the bug doesn't happen on 0.1.2.8-alpha. What about 0.1.2.18? Thanks, --Roger
RE: Suspicious Circuits
I researched this some more, and believe it to be a bug with the 0.2.0.12-alpha version being used for hidden services. I switched it to 0.1.2.8-alpha and have not been able to reproduce the problem. This makes me happy knowing this probably wasn't some sort of attack on hidden services. It was really weird seeing the same node twice in a circuit though. Bug hunting...fun times. - Kyle -Original Message- From: Kyle Williams [mailto:[EMAIL PROTECTED] Sent: Sunday, December 09, 2007 9:20 PM To: 'or-talk@freehaven.net' Subject: Suspicious Circuits I've been having problems getting to hidden services the last couple of days. I noticed something odd in Vidalia the other day, but it was gone before I could take a screenshot. However this evening, I was having a lot of problems with .onion addresses, and Vidalia was showing several (more than 6) nodes in a circuit almost every time I tried to reach any hidden service, including my own. Here are some cropped screenshots of the circuits as shown in Vidalia. http://www.janusvm.com/pub/bad-1.jpg http://www.janusvm.com/pub/bad-2.jpg http://www.janusvm.com/pub/bad-3.jpg http://www.janusvm.com/pub/bad-4.jpg http://www.janusvm.com/pub/bad-5.jpg http://www.janusvm.com/pub/bad-6.jpg<--- VERY INTERESTING These circuits have more than 3 nodes, and as shown in the picture, has the same node more than once in the circuit. Correct me if I'm wrong, but I thought circuits are not allowed to have the same node more than once. I noticed three nodes in a set, and three different sets acting this way. = $443BAA7BE006A904179DD35013F788F1DDD275E5 - askatasuna 216.195.133.27 $847B1F850344D7876491A54892F904934E4EB85D - tor26 86.59.21.38 $99BDCC9E80D4E77E2357B77142E4023CE0D12B5A - Qba20070825pl 195.34.208.22 = $CCD030D151A5BAC14D49C77386EC33FF99EAE580 - paperoga 213.203.146.95 $B2CF35C7DF36E7FFC60CCC67D3189FE09E1E4E4A - univac 217.230.243.147 $08101AD124C3B10E2F1F18DF2B51F4901E385170 - SEC 192.42.113.248 = $376FF360B98C07F84E90D3A26831223440C11062 - chaoscitytor 85.25.52.40 $A0DD5DC19A0ED1692EB6663684D04A2ABD3D491B - shadow 24.29.193.226 $3AD690A220A316B08FBFBACF8757C92DA0033B57 - mushin 18.152.2.242 = What I found even more disturbing, 'tor26' (bad-6.jpg) seemed to be participating in whatever was going on. Isn't this a DA!? So does anyone have a clue as to what is going on? Is this an attack on hidden services?? I added the following line to my torrc configs and everything seems to work well now. ExcludeNodes $443BAA7BE006A904179DD35013F788F1DDD275E5,$847B1F850344D7876491A54892F904934 E4EB85D,$99BDCC9E80D4E77E2357B77142E4023CE0D12B5A,$3AD690A220A316B08FBFBACF8 757C92DA0033B57,$B2CF35C7DF36E7FFC60CCC67D3189FE09E1E4E4A,$08101AD124C3B10E2 F1F18DF2B51F4901E385170,$376FF360B98C07F84E90D3A26831223440C11062,$A0DD5DC19 A0ED1692EB6663684D04A2ABD3D491B,$CCD030D151A5BAC14D49C77386EC33FF99EAE580 - Kyle
Suspicious Circuits
I've been having problems getting to hidden services the last couple of days. I noticed something odd in Vidalia the other day, but it was gone before I could take a screenshot. However this evening, I was having a lot of problems with .onion addresses, and Vidalia was showing several (more than 6) nodes in a circuit almost every time I tried to reach any hidden service, including my own. Here are some cropped screenshots of the circuits as shown in Vidalia. http://www.janusvm.com/pub/bad-1.jpg http://www.janusvm.com/pub/bad-2.jpg http://www.janusvm.com/pub/bad-3.jpg http://www.janusvm.com/pub/bad-4.jpg http://www.janusvm.com/pub/bad-5.jpg http://www.janusvm.com/pub/bad-6.jpg<--- VERY INTERESTING These circuits have more than 3 nodes, and as shown in the picture, has the same node more than once in the circuit. Correct me if I'm wrong, but I thought circuits are not allowed to have the same node more than once. I noticed three nodes in a set, and three different sets acting this way. = $443BAA7BE006A904179DD35013F788F1DDD275E5 - askatasuna 216.195.133.27 $847B1F850344D7876491A54892F904934E4EB85D - tor26 86.59.21.38 $99BDCC9E80D4E77E2357B77142E4023CE0D12B5A - Qba20070825pl 195.34.208.22 = $CCD030D151A5BAC14D49C77386EC33FF99EAE580 - paperoga 213.203.146.95 $B2CF35C7DF36E7FFC60CCC67D3189FE09E1E4E4A - univac 217.230.243.147 $08101AD124C3B10E2F1F18DF2B51F4901E385170 - SEC 192.42.113.248 = $376FF360B98C07F84E90D3A26831223440C11062 - chaoscitytor 85.25.52.40 $A0DD5DC19A0ED1692EB6663684D04A2ABD3D491B - shadow 24.29.193.226 $3AD690A220A316B08FBFBACF8757C92DA0033B57 - mushin 18.152.2.242 = What I found even more disturbing, 'tor26' (bad-6.jpg) seemed to be participating in whatever was going on. Isn't this a DA!? So does anyone have a clue as to what is going on? Is this an attack on hidden services?? I added the following line to my torrc configs and everything seems to work well now. ExcludeNodes $443BAA7BE006A904179DD35013F788F1DDD275E5,$847B1F850344D7876491A54892F904934 E4EB85D,$99BDCC9E80D4E77E2357B77142E4023CE0D12B5A,$3AD690A220A316B08FBFBACF8 757C92DA0033B57,$B2CF35C7DF36E7FFC60CCC67D3189FE09E1E4E4A,$08101AD124C3B10E2 F1F18DF2B51F4901E385170,$376FF360B98C07F84E90D3A26831223440C11062,$A0DD5DC19 A0ED1692EB6663684D04A2ABD3D491B,$CCD030D151A5BAC14D49C77386EC33FF99EAE580 - Kyle
