Re: [tor-relays] Undiagnosable Crashes in Relays
> Date: Wed, 25 Mar 2015 17:21:50 +0800 > From: Vincent Yu > > On Wed, Mar 25, 2015 at 4:26 PM, skyhighatrist > wrote: >> I am wondering if anyone has had their relay randomly crash in the >> past >> week or so. Three of mine (I run 6, nonexits) have fallen over. One of >> them ~5 days ago, one of them ~4 days ago, and one of them earlier >> today >> . > > This also happened to my two relays about three weeks ago. They are: > > https://globe.torproject.org/#/relay/C309A31AD772FFDD0805C9FECB6D4748A7CBF684 > https://globe.torproject.org/#/relay/18BE989663CF3351F73D33C672BB1C985E0EA5D0 > > They are both middle/guard relays (about 200 Mbps each) on the 0.2.6 > branch and are on the same dedicated server. They went down at > different times, and as far as I can remember, there was nothing > notable in the Tor and system logs. No issues prior to this over the > past 12 months. I haven't had time to investigate this. > This also happened to my EC2 relay a few weeks ago, but I never had time to investigate. It's quite possible it was a bad nightly tor version, or some sort of misconfiguration on my end, but the timing matches. I didn't manage to get it to come back up in the limited time I had to try and fix it. https://globe.torproject.org/#/relay/425E55A8FA145ACFC01FA58CD4E6F46DD7762AAB (No record, it's been down for a while.) teor teor2345 at gmail dot com pgp 0xABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR C3C57B23 349825DE 929A1DEF C3531C25 A32287ED signature.asc Description: Message signed with OpenPGP using GPGMail ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Undiagnosable Crashes in Relays
My apologies for my lack of memory... It's still fuzzy, but Roger helped jog the the gerbil into action. There was a line regarding "assert failure" in my logs. I could not get Tor to start again until I followed these instructions: https://trac.torproject.org/projects/tor/ticket/13111 My secret onion keys had 0 values, so I removed them and restarted Tor and it went on it's merry way. It's quite possible the cause of my crash was completely different than Vincent and skyhighatrist, as neither of them indicated they had to do that to get them back up and running. Matt Speak Freely ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Undiagnosable Crashes in Relays
On Wed, Mar 25, 2015 at 08:26:20AM +, skyhighatrist wrote: > I have no idea why they all fell over, the last thing in the logs was > the usual "current status" output with some traffic measuring, seemingly > immediately afterwards, the process killed itself for no reason. You might also enjoy https://www.torproject.org/docs/faq#TorCrash My first guess is that the out-of-memory killer killed it. --Roger ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Undiagnosable Crashes in Relays
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 I'll do an upgrade across all my relays tonight, hopefully that resolves the issue in the future. Interesting that SpeakFreely's relay was affected given its on the patched version... On 25/03/15 13:53, Speak Freely wrote: > Vincent Yuis Tor 0.2.6.5-rc on Linux skyhighartistis Tor > 0.2.4.24 on Linux my affected relay Tor 0.2.5.11 on Linux > > Cool. > > > > Matt Speak Freely > > > Nick Mathewson: >> What version of Tor did these relays run? Is it possible that >> one of the crash bugs fixed in 0.2.5.11 is to blame? >> > ___ tor-relays mailing > list tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays > -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQIcBAEBCAAGBQJVEs2ZAAoJEMRV9IW1w5addBwP+QFv2btGjYUE/E30+5w4aySn CiP9Xqep3xEHRKCfxE+zbAb+KIOLyJXfKQo5wF9UvOHr0WSD1UQHzaurwwnbgvyv ylvDkyRj7w2aYTfZiMi2aXlLuH5KKgaDPm72g9T8pL8rUwMfdMtK969wyayXOgJS Cm7klx+R4j4iK74bA5mselYeTgqfGBbHEvTk13HpII/WJfI3e87ilaqN/w8KyUnl T+0pEAJdLJML61/OIsarH6ydCePLNaE6Dbgy0BAI0vLWqeQILrIW8S0MpU1xq4zp 0GAFPa5qiuXDCYqOt5KSzqATUjJV9HwJqfbpLwNO58dVdpeDSDJaZNGOzkuuXM0V xDmKQ6ztolUkhJYhcwexhR+FjuiUXZW0i6kl9OMWdbYEyYg0X0J/kL7w6yx40BwP 8ikd9hcxXFYui8jURfqNw+iR8e7HrymvvgL7R5zny8qGmHGdwfxgjVBjrwVV+Zwv uWrElZpnQjx8ollpCjPaAumb/I2Wc6rcS0G7IysZKhWkLmlrrSGh3fJH2+L7aDIV EVDDCrrmz10aAJyNhVBlwGumLJv0qnWJiqJSremFIC3ZGEAMXb/zUKegz/lQyxE2 k3HnR+sklhchVq9CL9XKq/YbbwdSXAZhYQqpmFPwzz5rdBcaPCk8bMp/O/WmILCA Mjid9oC8CbRMV1Q/OzEE =go/q -END PGP SIGNATURE- ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Undiagnosable Crashes in Relays
Vincent Yu is Tor 0.2.6.5-rc on Linux skyhighartist is Tor 0.2.4.24 on Linux my affected relay Tor 0.2.5.11 on Linux Cool. Matt Speak Freely Nick Mathewson: > What version of Tor did these relays run? Is it possible that one of > the crash bugs fixed in 0.2.5.11 is to blame? > ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Undiagnosable Crashes in Relays
On Wed, Mar 25, 2015 at 4:26 AM, skyhighatrist wrote: > Hello List, > I am wondering if anyone has had their relay randomly crash in the past > week or so. Three of mine (I run 6, nonexits) have fallen over. One of > them ~5 days ago, one of them ~4 days ago, and one of them earlier today > . > > The logs indicate everything was operating normally, and the only reason > I noticed they had crashed was when I was checking what bandwidth > globe.torproject was measuring for them it said some were down. > > The affected relays are as follows: > AC9803701F9EE18194D40B38E47CE4C68CF2F567 > 73067CD4ADD8A294BDA913DF45B63190A52B5F9F > D76252B1A6E9F01FC6772CFFB651056A2B54F92B > > I have no idea why they all fell over, the last thing in the logs was > the usual "current status" output with some traffic measuring, seemingly > immediately afterwards, the process killed itself for no reason. What version of Tor did these relays run? Is it possible that one of the crash bugs fixed in 0.2.5.11 is to blame? -- Nick ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Undiagnosable Crashes in Relays
As promised, heres my utterly hideous monitoring script for checking/restarting relays. Its a work in progress, any advice/comments would be greatly appreciated. https://github.com/0x27/relaycheck Hopefully we can get to the bottom of this wierd relay-collapse problem, its a bit of a nuisance! On 25/03/15 12:53, Speak Freely wrote: > One of my relays went down a few weeks ago, and I didn't notice until a > few days ago. > > https://atlas.torproject.org/#details/4CA46581FB3C82102565B02C1ECB6DD38EF6654A > > I did find what caused it, but thus far I cannot remember what it was. > If I remember, I'll post again. > > ___ > tor-relays mailing list > tor-relays@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays > 0xB5C3969D.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Undiagnosable Crashes in Relays
One of my relays went down a few weeks ago, and I didn't notice until a few days ago. https://atlas.torproject.org/#details/4CA46581FB3C82102565B02C1ECB6DD38EF6654A I did find what caused it, but thus far I cannot remember what it was. If I remember, I'll post again. ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Re: [tor-relays] Undiagnosable Crashes in Relays
On Wed, Mar 25, 2015 at 4:26 PM, skyhighatrist wrote: I am wondering if anyone has had their relay randomly crash in the past week or so. Three of mine (I run 6, nonexits) have fallen over. One of them ~5 days ago, one of them ~4 days ago, and one of them earlier today . This also happened to my two relays about three weeks ago. They are: https://globe.torproject.org/#/relay/C309A31AD772FFDD0805C9FECB6D4748A7CBF684 https://globe.torproject.org/#/relay/18BE989663CF3351F73D33C672BB1C985E0EA5D0 They are both middle/guard relays (about 200 Mbps each) on the 0.2.6 branch and are on the same dedicated server. They went down at different times, and as far as I can remember, there was nothing notable in the Tor and system logs. No issues prior to this over the past 12 months. I haven't had time to investigate this. Vincent ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
[tor-relays] Undiagnosable Crashes in Relays
Hello List, I am wondering if anyone has had their relay randomly crash in the past week or so. Three of mine (I run 6, nonexits) have fallen over. One of them ~5 days ago, one of them ~4 days ago, and one of them earlier today . The logs indicate everything was operating normally, and the only reason I noticed they had crashed was when I was checking what bandwidth globe.torproject was measuring for them it said some were down. The affected relays are as follows: AC9803701F9EE18194D40B38E47CE4C68CF2F567 73067CD4ADD8A294BDA913DF45B63190A52B5F9F D76252B1A6E9F01FC6772CFFB651056A2B54F92B I have no idea why they all fell over, the last thing in the logs was the usual "current status" output with some traffic measuring, seemingly immediately afterwards, the process killed itself for no reason. All of the affected relays are on DigitalOcean droplets. I intend to launch more on EC2 in the near future, however, would like to know if anyone else has experienced relay instability issues over the past few days incase its something nasty like a denial of service attempt or something. Nothing on the boxes indicates any kind of compromise, just the usual bruteforce activity from the usual actors. I have decided to start writing a "Shitty Monitoring System" to copy logs down to my local box every hour or so and HUP tor at the same time as a kind of countermeasure incase its simply a resource use issue or something, however, would love to know if anyone else has been having similar issues lately. Regards, .d 0xB5C3969D.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature ___ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
