Re: Arm Release 1.4.0
Hi, I've uploaded a new tarball to: http://www.atagar.com/transfer/tmp/arm_bsdTest3.tar.bz2 http://www.atagar.com/transfer/tmp/arm_bsdTest3.tar.bz2.asc Besides a modified version of Febian's patch to autodetect FreeBSD jails it most notably includes... - A replacement for the connection test function (which was a pita in my humble opinion). The new script [1] provides the resolver runtimes, a check if all the resolvers match, and a better method of dumping the connection results. If you modify the bsd resolvers then this should provide a nice sanity check that it's working as expected. - I forgot to account for the dns resolution exits do on behalf of the clients. The resolvers need to include UDP connections so, on *nix, they're now: - netstat -np | grep "ESTABLISHED /" - sockstat | egrep "\s*.*ESTABLISHED" - lsof -nPi | egrep "^\s*.*((UDP.*)|(\(ESTABLISHED\)))" - ss -nptu | grep "ESTAB.*\"\"," I'm guessing, for the FreeBSD resolvers, that sockstats already works and procstat just needs the 'grep TCP' to be removed (or maybe replaced with 'egrep "(TCP|UDP)"'). Is that right? > The connection doesn't leave the system because its a socks > connection with both the source and the destination address > located on the same system. Hm. Sounds like basic client connections (ie, things like firefox connecting to tor via the SocksPort). However, I tried running TBB and arm didn't list any of those connections. This is what I'd expect since the connection resolution is only fetching tor connections. Am I missing something here? Regardless, I made a couple changes to address issues that have been brought up (socks connections and listing external addresses for private ip range connections - see lines 332-334 and 363-364 in src/interface/connPanel.py [2]). But without a working repro case I can't promises that this'll do the trick. > With ^ added to the pattern it seems to work Great, it's happy with that on Linux as well so I'm now using: lsof -nPi | egrep "^\s*.*((UDP.*)|(\(ESTABLISHED\)))" and including it among FreeBSD resolvers as the last fallback. > lsof also seems to be rather slow (on FreeBSD): Yikes, that's quite the difference. It's pretty bad on Linux too (ss is worse, but netstat and sockstat tend to run around 20% faster). > I intend to look into creating a FreeBSD port around Christmas. Awesome, thank you! Cheers! -Damian [1] https://svn.torproject.org/svn/arm/trunk/src/test.py [2] https://svn.torproject.org/svn/arm/trunk/src/interface/connPanel.py *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
> ... Damian, how > many connections are on the node that you successfully see the conn list ? I don't recall. It was on amunet and I'll retest this once the relay's back up to speed (we recently switched ISPs so it'll take a few weeks for the BW authorities to send more traffic our way again). I'm sorry, btw, for not applying the patch yet. There was an issue in that it would introduce a couple unnecessary system calls every time the path prefix was fetched, but the trivial fix (caching the results) would mean potentially having the wrong jail path if the connection singleton changes. While addressing this I found a couple other issues I'm also trying to address (unrelated to the patch) so it'll probably be a few more days before I have another tarball to be tested. Cheers! -Damian *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
On Sun, 12 Dec 2010, Fabian Keil wrote: Damian Johnson wrote: John mentioned that for him connection resolution doesn't work in the new arm tarball (arm_bsdTest2.tar.bz2). Hans, Fabian: can either of you confirm, and if so what sort of issue is the log indicating? I can't confirm this. I thought we had already determined that this was because of my 4k+ connections. I'm sorry I don't have another node to test on ... Damian, how many connections are on the node that you successfully see the conn list ? *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
Damian Johnson wrote: > John mentioned that for him connection resolution doesn't work in the > new arm tarball (arm_bsdTest2.tar.bz2). Hans, Fabian: can either of > you confirm, and if so what sort of issue is the log indicating? I can't confirm this. > Also, there was interest mentioned earlier in a BSD port. Anyone > interested in taking up that gauntlet? :P I intend to look into creating a FreeBSD port around Christmas. Fabian signature.asc Description: PGP signature
Re: Arm Release 1.4.0
Damian Johnson wrote: > > The connection doesn't leave the system because its a socks > > connection with both the source and the destination address > > located on the same system. > > Makes sense. So if both the source and destination are private IPs > (10.*, 172.16.* - 172.31.*, 192.168.*) then skip the private -> > external translation for the display, right? Right. > I'm still trying to get my head around the rest of this bit. From an > UI perspective doing location1 -> location2 -> location3 entries will > be pretty difficult. If it's the varying number of columns that would cause difficulties, you could keep the colums constant with lines like: location1 (location2) --> location3 ... > Also, I've never run into this use case so I'm > not quite sure what you're describing. Oh well, maybe things will be > clearer after a bit more coffee. You get pretty much the same use case, but without jails, when running Tor in a local network behind a port-forwarding gateway. I would expect this to be rather common for bridges. > > I missed procstat, though. > > Actually, neither of the BSD resolvers were added (that was the Linux > sockstat entry). I'll add them both. > > > The psutil library [1] uses lsof as its default connection resolver > for FreeBSD so I'd like to get a working fallback for it. However, > from Hans' feedback earlier it sounds like there's issues with jails > so I'll use it as a last resort. Unless I misunderstood Hans, he was mainly concerned about having to install a dependency that offers nothing the base system doesn't already provide anyway. > The command I'm now using for Linux is: > lsof -nPi -p | grep ".*(ESTABLISHED)" With ^ added to the pattern it seems to work: f...@r500 ~/git/arm $lsof -nPi -p 4775 | grep "^tor.*(ESTABLISHED)" tor 4775_tor4uIPv4 0xfe00471633a80t0 TCP 10.0.0.2:35771->[scrubbed]:9001 (ESTABLISHED) tor 4775_tor 14uIPv4 0xfe0047c710000t0 TCP 10.0.0.2:15999->[scrubbed]:443 (ESTABLISHED) tor 4775_tor 15uIPv4 0xfe0047221af80t0 TCP 10.0.0.2:9051->10.0.0.2:20692 (ESTABLISHED) Otherwise arm itself shows up multiple times, too: f...@r500 ~/git/arm $lsof -nPi -p 4775 | grep "tor.*(ESTABLISHED)" tor 4775_tor4uIPv4 0xfe00471633a80t0 TCP 10.0.0.2:35771->[scrubbed]:9001 (ESTABLISHED) tor 4775_tor 14uIPv4 0xfe0047c710000t0 TCP 10.0.0.2:15999->[scrubbed]:443 (ESTABLISHED) tor 4775_tor 15uIPv4 0xfe0047221af80t0 TCP 10.0.0.2:9051->10.0.0.2:20692 (ESTABLISHED) python9179_tor3uIPv4 0xfe00472210000t0 TCP 10.0.0.2:20692->10.0.0.2:9051 (ESTABLISHED) python9179_tor3uIPv4 0xfe00472210000t0 TCP 10.0.0.2:20692->10.0.0.2:9051 (ESTABLISHED) python9179_tor3uIPv4 0xfe00472210000t0 TCP 10.0.0.2:20692->10.0.0.2:9051 (ESTABLISHED) python9179_tor3uIPv4 0xfe00472210000t0 TCP 10.0.0.2:20692->10.0.0.2:9051 (ESTABLISHED) python9179_tor3uIPv4 0xfe00472210000t0 TCP 10.0.0.2:20692->10.0.0.2:9051 (ESTABLISHED) python9179_tor3uIPv4 0xfe00472210000t0 TCP 10.0.0.2:20692->10.0.0.2:9051 (ESTABLISHED) Showing the same connection six times in a row makes little sense to me, but then again, I'm not familiar with lsof. lsof also seems to be rather slow (on FreeBSD): f...@r500 ~/git/arm $time lsof -nPi -p 4775 | grep "^tor.*(ESTABLISHED)" > /dev/null real0m0.338s user0m0.159s sys 0m0.183s f...@r500 ~/git/arm $time sockstat -4c | grep 'tor *4775' > /dev/null real0m0.010s user0m0.015s sys 0m0.005s > However, like sockstat it probably omits the ESTABLISHED tags from > results. If so, this will need a workaround. On Linux the following > looks right: > lsof -nPi -sTCP:ESTABLISHED -p | grep "" | grep TCP > > Does this to the trick? For some reason there was a UDP connection > included despite the -sTCP part, hence the extra grep. The lsof man > warns that "State names vary with UNIX dialects" so it might expect > a different keyword. That works too, but without the ^, arm itself shows up again. > If there aren't any concerns between the resolvers then we should > default to whatever performs the best. When you run at the debug level > (arm -e 1) it provides the runtime for system calls (sockstat, > procstat, lsof, etc) so they should be easy to compare. procstat and sockstat seem to perform comparable, but sockstat has the advantage that it will work if arm is run as neither the tor user nor root. Fabian signature.asc Description: PGP signature
Re: Arm Release 1.4.0
Fantastic - thanks Fabian! The patch looks perfect. I'll apply it either after work today or tomorrow morning. > but I think the same should be done with connections on the SocksPort Yup, sounds like a bug. Until recently arm had just been for relay usage so I probably missed this when writing the connection panel. I'll change it when applying the patch. > The connection doesn't leave the system because its a socks > connection with both the source and the destination address > located on the same system. Makes sense. So if both the source and destination are private IPs (10.*, 172.16.* - 172.31.*, 192.168.*) then skip the private -> external translation for the display, right? I'm still trying to get my head around the rest of this bit. From an UI perspective doing location1 -> location2 -> location3 entries will be pretty difficult. Also, I've never run into this use case so I'm not quite sure what you're describing. Oh well, maybe things will be clearer after a bit more coffee. > Maybe it should be mentioning in the log message when the torrc > can't be found? Good point. I'll change that too. > I missed procstat, though. Actually, neither of the BSD resolvers were added (that was the Linux sockstat entry). I'll add them both. The psutil library [1] uses lsof as its default connection resolver for FreeBSD so I'd like to get a working fallback for it. However, from Hans' feedback earlier it sounds like there's issues with jails so I'll use it as a last resort. The command I'm now using for Linux is: lsof -nPi -p | grep ".*(ESTABLISHED)" However, like sockstat it probably omits the ESTABLISHED tags from results. If so, this will need a workaround. On Linux the following looks right: lsof -nPi -sTCP:ESTABLISHED -p | grep "" | grep TCP Does this to the trick? For some reason there was a UDP connection included despite the -sTCP part, hence the extra grep. The lsof man warns that "State names vary with UNIX dialects" so it might expect a different keyword. If there aren't any concerns between the resolvers then we should default to whatever performs the best. When you run at the debug level (arm -e 1) it provides the runtime for system calls (sockstat, procstat, lsof, etc) so they should be easy to compare. Cheers! -Damian [1] https://code.google.com/p/psutil/ *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
Damian Johnson wrote:
> Hazaa, many thanks for the patches! Committed with the exception of
> sockstat2 (see below).
>
> http://www.atagar.com/transfer/tmp/arm_bsdTest2.tar.bz2
>
> > One unrelated problem I noticed is that Arm tends to show local
> > connections as Outbound.
>
> Netstat, lsof, etc doesn't include a notion of the directionality of a
> connection, so I'm using the local port to determine if it's inbound
> or outbound. If it matches the ORPort or DirPort then it's inbound,
> otherwise it's outbound (line 323 of the connPanel.py [1]). Do you
> know a smarter way of handling this?
I agree that a connection on the local ORPort or DirPort should be
listed as inbound, but I think the same should be done with connections
on the SocksPort. Currently they are listed as outbound with the local
Tor address replaced:
[public gateway IP]:9050 --> 10.0.0.1:58305 (??) UNKNOWN UNKNOWN 3.4s
(OUTBOUND)
> I'm familiar with Linux's chroot jail environments (where this works),
> but not that details of what the bsd counterpart does.
I think those connections would be listed as outbound on other
systems, too.
> > Given that the connection doesn't leave the system, replacing
> > the Tor jail IP address with the public IP address of the gateway
> > is a bit confusing.
>
> Sorry, I'm not following. Why isn't the tor connection leaving the
> system? I'm using the results of 'GETINFO address' which tends to be a
> lot more helpful than showing the ip on the local network (though I
> can include an option to display the local address instead if you'd
> like).
The connection doesn't leave the system because its a socks
connection with both the source and the destination address
located on the same system.
I think an option to display the real address would be useful.
It probably would also make sense to not show the local port
when replacing the address.
The outbound connection:
f...@r500 ~ $sudo pfctl -ss | grep 31515
all tcp 10.0.0.2:31515 -> 192.168.0.106:52509 -> [some Tor server address]:9001
ESTABLISHED:ESTABLISHED
Is listed as:
[public gateway IP]:31515 --> [some Tor server address]:9001 (us)
[fingerprint] [relay name] 14.6m (OUTBOUND)
but given that the local Tor address is behind two NAT layers
it's highly unlikely that the Tor server will see the connection
as coming from source port 31515.
Maybe it would make even more sense to show the connection
similar to the way pfctl does. I'm thinking of something like:
10.0.0.2:31515 --> [public Tor IP address][:port if known] --> [Tor server
address]:9001 (us) [fingerprint] [relay name] 14.6m (OUTBOUND)
> > Also, when running Arm outside the Tor jail, the Tor
> > configuration file isn't found.
>
> See the "features.pathPrefix" entry in the sample armrc [2]. It's
> specifically for jail environments (arm will otherwise also be failing
> to find tor's state, log file, and some other resources used to
> prepopulate data). If you have a suggestion for an automatic method
> for determining the jail path then I'm all ears.
Using features.pathPrefix works for me, thanks for mentioning it.
Maybe it should be mentioning in the log message when the torrc
can't be found?
> > so arm is trying to read a torrc on the host in the location it knows
> > which is displayed from the jail, but is ignoring the jail flag.
>
> I'm attempting to read the torrc from the location Tor reports (via
> 'GETINFO config-file'), using the features.pathPrefix as... well, a
> path prefix. I'm not familiar with a method of getting the jail path
> for Linux jails. Is this information available for bsd jails?
It is, if you are outside the FreeBSD jail.
> I'm happy to help with a patch to autodetect for bsd jails if you have
> a suggestion for how.
The attached patch seems to work for me.
> > -- sockstatsockstat -4 | egrep '\s*' | grep -v '*:*'
> > +- sockstatsockstat -4c | grep ' *'
>
> Aren't the *:* connections unestablished? From the output given earlier:
> _tor tor4397 8 udp4 172.27.72.202:53 *:*
> _tor tor4397 9 tcp4 172.27.72.202:9051*:*
> _tor tor4397 12 tcp4 172.27.72.202:54011 [scrubbed]:9001
>
> that seemed to be what it meant. Arm might choke on the asterisks (it
> expects numeric port numbers), but including with the commit...
Actually the problem was egrep not understanding "\s" as meta character.
sockstat's -c flag lets it only show the connected sockets, so the last
grep is no longer necessary.
> > + userInput = raw_input("Enter query (
> > PROCESS_NAME [PID]): ").split()
> > + elif userInput[0] == "sockstat": userInput[0] = CMD_SOCKSTAT
>
> Nice catch!
I missed procstat, though.
> > +RUN_BSD_SOCKSTAT_2 = "sockstat -4c | awk '($2 == \"%s\" && $3 == %s)
> > {print $6 \" \" $7}'"
>
> Is there any advantage to having both sockstat commands? This will
> probably just confuse users so I'm holding off on this one for now.
No real advantage, but as I already had the patch an
Re: Arm Release 1.4.0
> Yes, and maybe just ignore them. IMHO there's no point in seeing these. Hm, I'm still a little confused what jails have to do with the tor SocksPort (this should only concern client tor usage). Maybe Fabian can clarify the proper handling for BSD use cases. > Well, I personally do not like anything reading files in a jail in that > way and can do without this particular feature. When starting up arm tries to read tor's log file and bandwidth entries from its state file. If arm doesn't know about the jail then it won't be able to prepopulate these values either. Besides that arm should be happy without direct access to tor's files. Is there a reason reading jailed files directly is discouraged? John mentioned that for him connection resolution doesn't work in the new arm tarball (arm_bsdTest2.tar.bz2). Hans, Fabian: can either of you confirm, and if so what sort of issue is the log indicating? Also, there was interest mentioned earlier in a BSD port. Anyone interested in taking up that gauntlet? :P Cheers! -Damian *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
On Mon, Dec 06, 2010 at 06:26:10PM -0800, Damian Johnson wrote: > > if (localPort == ORPort or localPort == DirPort): > # treat as an inbound connection with the external ip > # this is part of arm's current behavior > elif (localPort == SocksPort and OS == FreeBSD): > # treat as an inbound connection with the internal ip (ie, what's > reported by sockstat) > # this is new to fix the issue you mention Yes, and maybe just ignore them. IMHO there's no point in seeing these. > else: > # treat as an outbound connection > # again, part of arm's current behavior > > ... right? If not, could you please clarify? > > > 'GETINFO config-file' will show the path to torrc from within the jail. > > So arm tries to read: > > /path/to/torrc > > The location from the host though would be > > /path/to/jail/path/to/torrc > > > > Reading the file in that way, I believe, is not a good idea. > > > > All this only applies for systems running Tor in a jail and arm from the > > host. > > Arm works nicely with Tor if both are running on the same host or > > inside a jail on FreeBSD. > > If you set "features.pathPrefix /path/to/jail" in your armrc then it'll read: > /path/to/jail/path/to/torrc > > like you wanted. Again, if you have an approach for arm to detect that > (a) Tor exists in a BSD jail and (b) what its path is then I'd be > happy to make it work by default instead. > Well, I personally do not like anything reading files in a jail in that way and can do without this particular feature. And if, setting 'features.pathPrefix /path/to/jail' will get the desired results. Thanks Damian, thanks Fabian Regards P.S. wonder where the list sentinels will start to grumble on this ... *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
> This IP serves as the internal adress to the jail when > called from a local subnet, and may show multiple connections to the > SocksPort, > usually IP:9050. Sorry, I'm not sure if I'm following. You're saying that the check should essentially be: if (localPort == ORPort or localPort == DirPort): # treat as an inbound connection with the external ip # this is part of arm's current behavior elif (localPort == SocksPort and OS == FreeBSD): # treat as an inbound connection with the internal ip (ie, what's reported by sockstat) # this is new to fix the issue you mention else: # treat as an outbound connection # again, part of arm's current behavior ... right? If not, could you please clarify? > 'GETINFO config-file' will show the path to torrc from within the jail. > So arm tries to read: > /path/to/torrc > The location from the host though would be > /path/to/jail/path/to/torrc > > Reading the file in that way, I believe, is not a good idea. > > All this only applies for systems running Tor in a jail and arm from the > host. > Arm works nicely with Tor if both are running on the same host or > inside a jail on FreeBSD. If you set "features.pathPrefix /path/to/jail" in your armrc then it'll read: /path/to/jail/path/to/torrc like you wanted. Again, if you have an approach for arm to detect that (a) Tor exists in a BSD jail and (b) what its path is then I'd be happy to make it work by default instead. Cheers! -Damian *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
On Mon, Dec 06, 2010 at 10:25:39AM -0800, Damian Johnson wrote: > Hazaa, many thanks for the patches! Committed with the exception of > sockstat2 (see below). > > http://www.atagar.com/transfer/tmp/arm_bsdTest2.tar.bz2 > > > One unrelated problem I noticed is that Arm tends to show local > > connections as Outbound. > > Netstat, lsof, etc doesn't include a notion of the directionality of a > connection, so I'm using the local port to determine if it's inbound > or outbound. If it matches the ORPort or DirPort then it's inbound, > otherwise it's outbound (line 323 of the connPanel.py [1]). Do you > know a smarter way of handling this? > > I'm familiar with Linux's chroot jail environments (where this works), > but not that details of what the bsd counterpart does. > > > Given that the connection doesn't leave the system, replacing > > the Tor jail IP address with the public IP address of the gateway > > is a bit confusing. > > Sorry, I'm not following. Why isn't the tor connection leaving the > system? I'm using the results of 'GETINFO address' which tends to be a > lot more helpful than showing the ip on the local network (though I > can include an option to display the local address instead if you'd > like). FreeBSD jails resemble linux jails mainly by name :), and most probably have an own IP somewhere within RFC 1918. This IP serves as the internal adress to the jail when called from a local subnet, and may show multiple connections to the SocksPort, usually IP:9050. This is, what it looks like: [Host's public gateway IP address scrubbed]:9050 -->0.0s (OUTBOUND) And what it 'SHOULD NEITHER' but with proper IP look like: [Jail's private IP address scrubbed]:9050 -->0.0s (OUTBOUND) These connections are 'inbound' to the jail's SocksPort from the host or a private subnet. > > > Also, when running Arm outside the Tor jail, the Tor > > configuration file isn't found. > > See the "features.pathPrefix" entry in the sample armrc [2]. It's > specifically for jail environments (arm will otherwise also be failing > to find tor's state, log file, and some other resources used to > prepopulate data). If you have a suggestion for an automatic method > for determining the jail path then I'm all ears. > > > so arm is trying to read a torrc on the host in the location it knows > > which is displayed from the jail, but is ignoring the jail flag. > > I'm attempting to read the torrc from the location Tor reports (via > 'GETINFO config-file'), using the features.pathPrefix as... well, a > path prefix. I'm not familiar with a method of getting the jail path > for Linux jails. Is this information available for bsd jails? > > I'm happy to help with a patch to autodetect for bsd jails if you have > a suggestion for how. > 'GETINFO config-file' will show the path to torrc from within the jail. So arm tries to read: /path/to/torrc The location from the host though would be /path/to/jail/path/to/torrc Reading the file in that way, I believe, is not a good idea. All this only applies for systems running Tor in a jail and arm from the host. Arm works nicely with Tor if both are running on the same host or inside a jail on FreeBSD. *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
Hazaa, many thanks for the patches! Committed with the exception of
sockstat2 (see below).
http://www.atagar.com/transfer/tmp/arm_bsdTest2.tar.bz2
> One unrelated problem I noticed is that Arm tends to show local
> connections as Outbound.
Netstat, lsof, etc doesn't include a notion of the directionality of a
connection, so I'm using the local port to determine if it's inbound
or outbound. If it matches the ORPort or DirPort then it's inbound,
otherwise it's outbound (line 323 of the connPanel.py [1]). Do you
know a smarter way of handling this?
I'm familiar with Linux's chroot jail environments (where this works),
but not that details of what the bsd counterpart does.
> Given that the connection doesn't leave the system, replacing
> the Tor jail IP address with the public IP address of the gateway
> is a bit confusing.
Sorry, I'm not following. Why isn't the tor connection leaving the
system? I'm using the results of 'GETINFO address' which tends to be a
lot more helpful than showing the ip on the local network (though I
can include an option to display the local address instead if you'd
like).
> Also, when running Arm outside the Tor jail, the Tor
> configuration file isn't found.
See the "features.pathPrefix" entry in the sample armrc [2]. It's
specifically for jail environments (arm will otherwise also be failing
to find tor's state, log file, and some other resources used to
prepopulate data). If you have a suggestion for an automatic method
for determining the jail path then I'm all ears.
> so arm is trying to read a torrc on the host in the location it knows
> which is displayed from the jail, but is ignoring the jail flag.
I'm attempting to read the torrc from the location Tor reports (via
'GETINFO config-file'), using the features.pathPrefix as... well, a
path prefix. I'm not familiar with a method of getting the jail path
for Linux jails. Is this information available for bsd jails?
I'm happy to help with a patch to autodetect for bsd jails if you have
a suggestion for how.
> Thanks, too. BTW where is the git repo, please ?
Arm is in svn (repository links are available on the site [3]). If
others start hacking on arm with me and prefer git then I have no
objection to moving. Until then, however, svn is what I'm most
familiar with.
> 15:08:52 [ARM_WARN] Unable to resolve tor pid, abandoning connection listing
Ick!
Many thanks for the patches! Lacking the pid will mess with most of
the system commands so this is *very* helpful.
> 15:08:51 [ARM_DEBUG] system call: netstat -npl | grep 127.0.0.1:9051
> (runtime: 0.00)
> ###
That was from the failed pid resolution. I attempted to use the
PidFile, pidof, netstat, and ps to resolve the pid (see getPid in
torTools.py [4]) though as the patch points out these all fail on
FreeBSD.
> -- sockstatsockstat -4 | egrep '\s*' | grep -v '*:*'
> +- sockstatsockstat -4c | grep ' *'
Aren't the *:* connections unestablished? From the output given earlier:
_tor tor4397 8 udp4 172.27.72.202:53 *:*
_tor tor4397 9 tcp4 172.27.72.202:9051*:*
_tor tor4397 12 tcp4 172.27.72.202:54011 [scrubbed]:9001
that seemed to be what it meant. Arm might choke on the asterisks (it
expects numeric port numbers), but including with the commit...
> + userInput = raw_input("Enter query (
> PROCESS_NAME [PID]): ").split()
> + elif userInput[0] == "sockstat": userInput[0] = CMD_SOCKSTAT
Nice catch!
> +RUN_BSD_SOCKSTAT_2 = "sockstat -4c | awk '($2 == \"%s\" && $3 == %s) {print
> $6 \" \" $7}'"
Is there any advantage to having both sockstat commands? This will
probably just confuse users so I'm holding off on this one for now.
> + 5. "pgrep -x tor"
This works beautifully under Linux too, so I'm making it the secondary
resolver (after the PidFile and before pidof), this way bootstrapping
on BSD won't churn through system calls that are doomed to fail.
> + 5. "sockstat -4l -P tcp -p %i | grep tor" %
This one's BSD only due to the -4 flag, so keeping it as the last fallback.
> + # XXX: both issues could be solved by filtering for the
> + # control port IP address instead of the process name.
By the control port IP address? If there's multiple tor instances on
the same system then they'd all have the same IP.
Again, many thanks for the patches! -Damian
[1] https://svn.torproject.org/svn/arm/trunk/src/interface/connPanel.py
[2] https://svn.torproject.org/svn/arm/trunk/armrc.sample
[3] http://www.atagar.com/arm/
[4] https://svn.torproject.org/svn/arm/trunk/src/util/torTools.py
***
To unsubscribe, send an e-mail to majord...@torproject.org with
unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
Hans Schnehl wrote:
> On Sun, Dec 05, 2010 at 01:02:16PM +0100, Fabian Keil wrote:
> > Damian Johnson wrote:
> >
> > > Hi all. I've checked in the resolver fixes (thank Fabian and Hans!)
> > > and a test tarball is available at:
> > >
> > > http://www.atagar.com/transfer/tmp/arm_bsdTest.tar.bz2
> > > http://www.atagar.com/transfer/tmp/arm_bsdTest.tar.bz2.asc
> >
> > Thanks.
> Thanks, too. BTW where is the git repo, please ?
I don't think there is an official git repository.
> To Fabian, instead of adding another patch:
> Running 'pgrep tor' might show multiple results, anything containing the
> string
> 'tor' will present it's pid.
Good point.
> On 7 it's 'pgrep -x' to get an exact match of the argument ,see man pgrep(1).
It's -x on CURRENT, too. Patch attached.
I also attached one to get the pid through sockstat. I think changing
it to additionally filter for the control port address would get the
most reliable results.
Fabian
From cdb53450e64c38a5792b10328790dee49be32371 Mon Sep 17 00:00:00 2001
From: Fabian Keil
Date: Sun, 5 Dec 2010 15:09:26 +0100
Subject: [PATCH 1/2] Use pgrep's -x flag. Pointed out by Hans Schnehl
---
src/util/torTools.py |4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/util/torTools.py b/src/util/torTools.py
index 077c2e1..056b066 100644
--- a/src/util/torTools.py
+++ b/src/util/torTools.py
@@ -96,7 +96,7 @@ def getPid(controlPort=9051, pidFilePath=None):
2. "pidof tor"
3. "netstat -npl | grep 127.0.0.1:%s" %
4. "ps -o pid -C tor"
- 5. "pgrep tor"
+ 5. "pgrep -x tor"
If pidof or ps provide multiple tor instances then their results are
discarded (since only netstat can differentiate using the control port). This
@@ -155,7 +155,7 @@ def getPid(controlPort=9051, pidFilePath=None):
# - tor is running under a different name
# - there are multiple instances of tor
try:
-results = sysTools.call("pgrep tor")
+results = sysTools.call("pgrep -x tor")
if len(results) == 1 and len(results[0].split()) == 1:
pid = results[0].strip()
if pid.isdigit(): return pid
--
1.7.3.2
From fc8f0f7be7590a6bde2c2efaaa84f36be5af1136 Mon Sep 17 00:00:00 2001
From: Fabian Keil
Date: Sun, 5 Dec 2010 15:24:50 +0100
Subject: [PATCH 2/2] Add sockstat as another way to get the tor pid.
---
src/util/torTools.py | 17 -
1 files changed, 16 insertions(+), 1 deletions(-)
diff --git a/src/util/torTools.py b/src/util/torTools.py
index 056b066..1a36f71 100644
--- a/src/util/torTools.py
+++ b/src/util/torTools.py
@@ -96,7 +96,8 @@ def getPid(controlPort=9051, pidFilePath=None):
2. "pidof tor"
3. "netstat -npl | grep 127.0.0.1:%s" %
4. "ps -o pid -C tor"
- 5. "pgrep -x tor"
+ 5. "sockstat -4l -P tcp -p %i | grep tor" %
+ 6. "pgrep -x tor"
If pidof or ps provide multiple tor instances then their results are
discarded (since only netstat can differentiate using the control port). This
@@ -151,6 +152,20 @@ def getPid(controlPort=9051, pidFilePath=None):
if pid.isdigit(): return pid
except IOError: pass
+ # attempts to resolve using sockstat, failing if:
+ # - tor is running under a different name
+ # - there are multiple instances of Tor, using the
+ # same control port on different addresses.
+ #
+ # XXX: both issues could be solved by filtering for the
+ # control port IP address instead of the process name.
+ try:
+results = sysTools.call("sockstat -4l -P tcp -p %i | grep tor" % controlPort)
+if len(results) == 1 and len(results[0].split()) == 7:
+ pid = results[0].split()[2]
+ if pid.isdigit(): return pid
+ except IOError: pass
+
# attempts to resolve using pgrep, failing if:
# - tor is running under a different name
# - there are multiple instances of tor
--
1.7.3.2
signature.asc
Description: PGP signature
Re: Arm Release 1.4.0
On Sat, Dec 04, 2010 at 07:49:58PM -0800, Damian Johnson wrote:
> ...
>
> For BSD I'm using the following commands:
> sockstat -4 | egrep '\s*' | grep -v '*:*'
> procstat -f 'pgrep ' | grep '' | grep -v '0.0.0.0:0'
>
> The purpose of these greps is to only get established connections and
> to filter by the pid (in case there's multiple tor instances).
>
> To test please do the following:
> 1. Run arm at its debug runlevel ("arm -e 1")
15:08:52 [ARM_INFO] Terminal color support detected and enabled
15:08:52 [ARM_DEBUG] GETINFO events/names (runtime: 0.0002) [1 duplicate hidden]
15:08:52 [ARM_DEBUG] GETCONF ORPort (cache fetch) [19 duplicates hidden]
15:08:52 [ARM_DEBUG] GETINFO accounting/enabled (runtime: 0.0001)
15:08:52 [ARM_DEBUG] GETINFO desc/id/Unknown (runtime: 0.0001)
15:08:52 [ARM_DEBUG] GETINFO fingerprint (cache fetch) [2 duplicates hidden]
15:08:52 [ARM_WARN] Unable to resolve tor pid, abandoning connection listing
15:08:52 [ARM_DEBUG] GETINFO config/names (runtime: 0.0012)
15:08:52 [ARM_DEBUG] GETINFO config-text (runtime: 0.0016)
---
15:08:51 [ARM_DEBUG] system call: sockstat -4c | grep 'tor' (runtime: 0.00)
15:08:51 [ARM_DEBUG] GETINFO ns/id/Unknown (runtime: 0.0002)
15:08:51 [ARM_DEBUG] GETINFO fingerprint (runtime: 0.0002)
15:08:51 [ARM_DEBUG] GETINFO address (runtime: 0.0003)
15:08:51 [ARM_DEBUG] GETINFO status/version/current (runtime: 0.0002)
15:08:51 [ARM_DEBUG] GETINFO version (cache fetch)
15:08:51 [ARM_INFO] Operating System: FreeBSD, Connection Resolvers: sockstat
(bsd), procstat (bsd)
15:08:51 [ARM_WARN] Unable to load torrc (no such file or directory:
'/path/to/torrc')
### this is the path on the host, tor is running in a jail
15:08:51 [ARM_DEBUG] GETINFO config-file (runtime: 0.0002)
15:08:51 [ARM_DEBUG] system call: ps -o pid -C tor (runtime: 0.00)
15:08:51 [ARM_DEBUG] system call: netstat -npl | grep 127.0.0.1:9051 (runtime:
0.00)
###
15:08:51 [ARM_INFO] system call (failed): pidof tor (error: 'pidof' is
unavailable)
15:08:51 [ARM_INFO] Loaded configuration descriptions from
'/tmp/arm/torConfigDescriptions.txt' (runtime: 0.000)
15:08:51 [ARM_DEBUG] GETINFO version (runtime: 0.0002)
15:08:45 [ARM_NOTICE] No configuration found at '/home/user/.armrc', using
defaults
>
> 2. Arm defaults to the bsd resolvers if os.uname() matches "FreeBSD".
> Please look for an entry like the following in your logs (it'll be
> near the start):
See above.
> 3. Go to arm's second page and press "u" to bring up the resolver
> options. Regardless of the OS detected it will have:
> * auto
> * netstat
> * ss
> * lsof
> * sockstat
> * sockstat (bsd)
> * procstat (bsd)
To make it short: the sockstat (bsd) option works on FreeBSD 7.4-PRERELEASE.
This is nice. See above for log.
>
>
> That said, from the link you provided it looks like this isn't
> necessary. What arm currently attempts is:
> lsof -nPi | grep "\s*.*(ESTABLISHED)"
>
> If FreeBSD has an equivalents for the flags (n = prevent dns lookups,
> P = show port numbers (not names), i = ip only) then please show me
> what its output looks like and I'd be happy to add it as another
> fallback resolver.
>
> What order should sockstat, procstat, and lsof be attempted in? Are
> any of them preferable due to jails over the others?
I would avoid lsof.
***
To unsubscribe, send an e-mail to majord...@torproject.org with
unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
Answering to Fabian's mail, which came a minute before I was about send a more verbose answer. On Sun, Dec 05, 2010 at 01:02:16PM +0100, Fabian Keil wrote: > Damian Johnson wrote: > > > Hi all. I've checked in the resolver fixes (thank Fabian and Hans!) > > and a test tarball is available at: > > > > http://www.atagar.com/transfer/tmp/arm_bsdTest.tar.bz2 > > http://www.atagar.com/transfer/tmp/arm_bsdTest.tar.bz2.asc > > Thanks. Thanks, too. BTW where is the git repo, please ? > > > For BSD I'm using the following commands: > > sockstat -4 | egrep '\s*' | grep -v '*:*' > > procstat -f 'pgrep ' | grep '' | grep -v '0.0.0.0:0' > > Neither of those commands work for me. To let Arm figure out > Tor's pid I still need the pgrep patch I mailed you yesterday. It did work fine under FreeBSD 7.4-PRERELEASE with sockstat the way you suggested. I modified in src/util/connections.py again to RUN_BSD_SOCKSTAT = "sockstat -4c | grep '%s%s' " (See Fabian's patch.) This works fine, too. Just one cycle less. Neither of the other options though did succeed in displaying a connection list. > > Attached are the patches I used to get it working on FreeBSD 9.0 CURRENT. > I also rebased the sockstat+awk patch, but sockstat+grep probably works > reliably enough. > > One unrelated problem I noticed is that Arm tends to show local > connections as Outbound. A connection from the Privoxy jail to > the Tor jail: > > _tor tor2750 25 tcp4 10.0.0.2:9050 10.0.0.1:20528 > > shows up as: > > [public gateway IP address scrubbed]:9050 -->0.0s (OUTBOUND) Can verify this, running arm on a host connecting to a jail's tor controlport results in this. > > Given that the connection doesn't leave the system, replacing > the Tor jail IP address with the public IP address of the gateway > is a bit confusing. > > Also, when running Arm outside the Tor jail, the Tor > configuration file isn't found. An old torrc located in the host in the same path as the jail, I wondered where these odd configs came from. Removing the hosts torrc gave ### Unable to load the torrc ### so arm is trying to read a torrc on the host in the location it knows which is displayed from the jail, but is ignoring the jail flag. Sorry for not getting deeper in this right now. To Fabian, instead of adding another patch: Running 'pgrep tor' might show multiple results, anything containing the string 'tor' will present it's pid. On 7 it's 'pgrep -x' to get an exact match of the argument ,see man pgrep(1). Thanks for the patches ! To all : The working connection list looks great ! *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
Damian Johnson wrote:
> Hi all. I've checked in the resolver fixes (thank Fabian and Hans!)
> and a test tarball is available at:
>
> http://www.atagar.com/transfer/tmp/arm_bsdTest.tar.bz2
> http://www.atagar.com/transfer/tmp/arm_bsdTest.tar.bz2.asc
Thanks.
> For BSD I'm using the following commands:
> sockstat -4 | egrep '\s*' | grep -v '*:*'
> procstat -f 'pgrep ' | grep '' | grep -v '0.0.0.0:0'
Neither of those commands work for me. To let Arm figure out
Tor's pid I still need the pgrep patch I mailed you yesterday.
Attached are the patches I used to get it working on FreeBSD 9.0 CURRENT.
I also rebased the sockstat+awk patch, but sockstat+grep probably works
reliably enough.
One unrelated problem I noticed is that Arm tends to show local
connections as Outbound. A connection from the Privoxy jail to
the Tor jail:
_tor tor2750 25 tcp4 10.0.0.2:9050 10.0.0.1:20528
shows up as:
[public gateway IP address scrubbed]:9050 -->0.0s (OUTBOUND)
Given that the connection doesn't leave the system, replacing
the Tor jail IP address with the public IP address of the gateway
is a bit confusing.
Also, when running Arm outside the Tor jail, the Tor
configuration file isn't found.
Fabian
From 48734e94a8205754f793b19d1db77bd72e2a305b Mon Sep 17 00:00:00 2001
From: Fabian Keil
Date: Sat, 4 Dec 2010 17:36:53 +0100
Subject: [PATCH 1/4] Add pgrep as another way to get the tor pid.
---
src/util/torTools.py | 11 +++
1 files changed, 11 insertions(+), 0 deletions(-)
diff --git a/src/util/torTools.py b/src/util/torTools.py
index d18869b..077c2e1 100644
--- a/src/util/torTools.py
+++ b/src/util/torTools.py
@@ -96,6 +96,7 @@ def getPid(controlPort=9051, pidFilePath=None):
2. "pidof tor"
3. "netstat -npl | grep 127.0.0.1:%s" %
4. "ps -o pid -C tor"
+ 5. "pgrep tor"
If pidof or ps provide multiple tor instances then their results are
discarded (since only netstat can differentiate using the control port). This
@@ -150,6 +151,16 @@ def getPid(controlPort=9051, pidFilePath=None):
if pid.isdigit(): return pid
except IOError: pass
+ # attempts to resolve using pgrep, failing if:
+ # - tor is running under a different name
+ # - there are multiple instances of tor
+ try:
+results = sysTools.call("pgrep tor")
+if len(results) == 1 and len(results[0].split()) == 1:
+ pid = results[0].strip()
+ if pid.isdigit(): return pid
+ except IOError: pass
+
return None
def getConn():
--
1.7.3.2
From 1bd8015476dd5c3613e3384ca373d9f096aabae4 Mon Sep 17 00:00:00 2001
From: Fabian Keil
Date: Sun, 5 Dec 2010 11:49:35 +0100
Subject: [PATCH 2/4] Shorten RUN_BSD_SOCKSTAT and get it working.
---
src/util/connections.py |4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/util/connections.py b/src/util/connections.py
index 2487afb..7824f4f 100644
--- a/src/util/connections.py
+++ b/src/util/connections.py
@@ -12,7 +12,7 @@ all queries dump its stderr (directing it to /dev/null).
FreeBSD lacks support for the needed netstat flags and has a completely
different program for 'ss'. However, there's a couple other options (thanks to
Fabian Keil and Hans Schnehl):
-- sockstatsockstat -4 | egrep '\s*' | grep -v '*:*'
+- sockstatsockstat -4c | grep ' *'
- procstatprocstat -f 'pgrep ' | grep '' | grep -v '0.0.0.0:0'
"""
@@ -62,7 +62,7 @@ RUN_LSOF = "lsof -nPi | grep \"%s\s*%s.*(ESTABLISHED)\""
# *note: this isn't available by default under ubuntu
RUN_SOCKSTAT = "sockstat | egrep \"%s\s*%s.*ESTABLISHED\""
-RUN_BSD_SOCKSTAT = "sockstat -4 | egrep '%s\s*%s' | grep -v '*:*'"
+RUN_BSD_SOCKSTAT = "sockstat -4c | grep '%s *%s'"
RUN_BSD_PROCSTAT = "procstat -f 'pgrep %s' | grep '%s' | grep -v '0.0.0.0:0'"
RESOLVERS = [] # connection resolvers available via the singleton constructor
--
1.7.3.2
From 12d267f497870ecff1b37bddb1aeb44e9ba01994 Mon Sep 17 00:00:00 2001
From: Fabian Keil
Date: Sun, 5 Dec 2010 12:07:36 +0100
Subject: [PATCH 3/4] Get RUN_BSD_PROCSTAT working.
---
src/util/connections.py |6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/util/connections.py b/src/util/connections.py
index 7824f4f..062562e 100644
--- a/src/util/connections.py
+++ b/src/util/connections.py
@@ -13,7 +13,7 @@ FreeBSD lacks support for the needed netstat flags and has a completely
different program for 'ss'. However, there's a couple other options (thanks to
Fabian Keil and Hans Schnehl):
- sockstatsockstat -4c | grep ' *'
-- procstatprocstat -f 'pgrep ' | grep '' | grep -v '0.0.0.0:0'
+- procstatprocstat -f | grep TCP | grep -v 0.0.0.0:0
"""
import os
@@ -63,7 +63,7 @@ RUN_LSOF = "lsof -nPi | grep \"%s\s*%s.*(ESTABLISHED)\""
RUN_SOCKSTAT = "sockstat | egrep \"%s\s*%s.*ESTABLISHED\""
RUN_BSD_SOCKSTAT = "sockstat -4c | grep '%s *%s'"
-RUN_BSD_PROCSTAT = "procstat -f 'pgrep %s' | grep '%s' | grep -v '0.0.0.0:0'"
+RUN_BSD_PROCSTAT = "procst
Re: Arm Release 1.4.0
Hi all. I've checked in the resolver fixes (thank Fabian and Hans!)
and a test tarball is available at:
http://www.atagar.com/transfer/tmp/arm_bsdTest.tar.bz2
http://www.atagar.com/transfer/tmp/arm_bsdTest.tar.bz2.asc
I've added both of the commands you mentioned as fallback resolvers.
On Ubuntu procstat doesn't exist but sockstat does, so I've also added
handling for its Linux version too:
ata...@fenrir:~$ sockstat | egrep "tor\s*3475.*ESTABLISHED"
atagar tor29672tcp4 192.168.0.3:36511:9001
ESTABLISHED
atagar tor29672tcp4 192.168.0.3:45906:9001
ESTABLISHED
atagar tor29672tcp4 192.168.0.3:60202:4430
ESTABLISHED
atagar tor29672tcp4 192.168.0.3:34600:9001
ESTABLISHED
...
For BSD I'm using the following commands:
sockstat -4 | egrep '\s*' | grep -v '*:*'
procstat -f 'pgrep ' | grep '' | grep -v '0.0.0.0:0'
The purpose of these greps is to only get established connections and
to filter by the pid (in case there's multiple tor instances).
To test please do the following:
1. Run arm at its debug runlevel ("arm -e 1")
2. Arm defaults to the bsd resolvers if os.uname() matches "FreeBSD".
Please look for an entry like the following in your logs (it'll be
near the start):
18:50:00 [ARM_INFO] Operating System: FreeBSD, Connection Resolvers:
sockstat (bsd), procstat (bsd)
if it doesn't match that then let me know what it says.
3. Go to arm's second page and press "u" to bring up the resolver
options. Regardless of the OS detected it will have:
* auto
* netstat
* ss
* lsof
* sockstat
* sockstat (bsd)
* procstat (bsd)
4. Select one of the bsd options. If it doesn't work then the results
will be unchanged (ie, if options were already listed then they'll
still be listed so don't trust that, please still check the following
step for both).
5. Go back to the log. You'll see an entry like:
18:16:31 [ARM_DEBUG] system call: sockstat -4 | egrep 'tor\s*29672' |
grep -v '*:*' (runtime: 0.03)
Problems come in two flavors. Either the command resulted in an error, like:
18:15:40 [ARM_INFO] system call (failed): procstat -f 'pgrep tor' |
grep '29672' | grep -v '0.0.0.0:0' (error: 'procstat' is unavailable)
in which case please let me know what the error is, or it doesn't
provide any results:
18:15:55 [ARM_INFO] No results found using: sockstat -4 | egrep
'tor\s*29672' | grep -v '*:*'
in which case please try the command on its own and let me know how I
screwed up the greps.
> after polling for lsof and a foreach loop, doesn't work ?
> ...
> would work ... especially if the system in question is doing little (or
> nothing) other than Tor ...
Sorry, I'm not following. Why is a for-each loop necessary? Process
and connection resolutions are the most costly lookups arm currently
does. A netstat resolution, for instance, takes around 30-40 ms on my
system and is performed every five seconds. Arm tries to be a
lightweight application so running multiple system commands to do each
lookup isn't really an option.
That said, from the link you provided it looks like this isn't
necessary. What arm currently attempts is:
lsof -nPi | grep "\s*.*(ESTABLISHED)"
If FreeBSD has an equivalents for the flags (n = prevent dns lookups,
P = show port numbers (not names), i = ip only) then please show me
what its output looks like and I'd be happy to add it as another
fallback resolver.
What order should sockstat, procstat, and lsof be attempted in? Are
any of them preferable due to jails over the others?
> However if lsof is the only way to do it right
Nope, not necessary at all. But the more connection resolvers
available for FreeBSD the more likely arm will be able to fall back to
something that works.
Cheers! -Damian
***
To unsubscribe, send an e-mail to majord...@torproject.org with
unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
On Fri, 3 Dec 2010, Hans Schnehl wrote: specifically: ps -Al after polling for lsof and a foreach loop, doesn't work ? I know it's not elegant, but it appeared to me that: lsof + ps -Al would work ... especially if the system in question is doing little (or nothing) other than Tor ... I suppose it would, but some (like me) tend to run tor in rather minimal jails. lsof isn't exactly a small application, so it might just make sense using the base system's sockstat. At least that was the idea. Ok, I agree with that sentiment - I always run Tor in a jail and would like to not be required to install lsof. However if lsof is the only way to do it right, I will accept that in order to run Arm... Thanks. *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
On Fri, Dec 03, 2010 at 09:35:39PM +, John Case wrote: > > On Fri, 3 Dec 2010, Hans Schnehl wrote: > > > Sorry for jumping in , but please notice the above command might not > > not work on all versions of FBSD, at least it doesn't on a 7-Stable jail. > > Not working refers to Fabian's use of procstat. > > Maybe the following just produces a similar sufficient output: > > > > _...@ato# id > > uid=256(_tor) gid=256(_tor) groups=256(_tor) > > _...@ato# sockstat -4 | grep tor > > _tor tor4397 7 tcp4 172.27.72.202:9050*:* > > _tor tor4397 8 udp4 172.27.72.202:53 *:* > > _tor tor4397 9 tcp4 172.27.72.202:9051*:* > > _tor tor4397 12 tcp4 172.27.72.202:54011 [scrubbed]:9001 > > _tor tor4397 15 tcp4 172.27.72.202:59374 [scrubbed]:9001 > > _tor tor4397 19 tcp4 172.27.72.202:59673 [scrubbed]:9001 > > _tor tor4397 20 tcp4 172.27.72.202:51946 [scrubbed]:443 > > _tor tor4397 22 tcp4 172.27.72.202:60344 [scrubbed]:9001 > > > > for *not* displaying listening ports just use > > _...@ato# sockstat -4 | grep tor| sed '/\*/d' > > _tor tor4397 4 tcp4 172.27.72.202:52420 [scrubbed]:443 > > _tor tor4397 12 tcp4 172.27.72.202:54011 [scrubbed]:9001 > > _tor tor4397 13 tcp4 172.27.72.202:51736 [scrubbed]:443 > > > Wait, so the method detailed here: > > http://lists.freebsd.org/pipermail/freebsd-questions/2007-November/162970.html > > specifically: > > ps -Al > > after polling for lsof and a foreach loop, doesn't work ? > > I know it's not elegant, but it appeared to me that: > > lsof + ps -Al > > would work ... especially if the system in question is doing little (or > nothing) other than Tor ... I suppose it would, but some (like me) tend to run tor in rather minimal jails. lsof isn't exactly a small application, so it might just make sense using the base system's sockstat. At least that was the idea. Up to the one porting arm to FBSD, I guess. *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
On Fri, 3 Dec 2010, Hans Schnehl wrote: Sorry for jumping in , but please notice the above command might not not work on all versions of FBSD, at least it doesn't on a 7-Stable jail. Maybe the following just produces a similar sufficient output: _...@ato# id uid=256(_tor) gid=256(_tor) groups=256(_tor) _...@ato# sockstat -4 | grep tor _tor tor4397 7 tcp4 172.27.72.202:9050*:* _tor tor4397 8 udp4 172.27.72.202:53 *:* _tor tor4397 9 tcp4 172.27.72.202:9051*:* _tor tor4397 12 tcp4 172.27.72.202:54011 [scrubbed]:9001 _tor tor4397 15 tcp4 172.27.72.202:59374 [scrubbed]:9001 _tor tor4397 19 tcp4 172.27.72.202:59673 [scrubbed]:9001 _tor tor4397 20 tcp4 172.27.72.202:51946 [scrubbed]:443 _tor tor4397 22 tcp4 172.27.72.202:60344 [scrubbed]:9001 for *not* displaying listening ports just use _...@ato# sockstat -4 | grep tor| sed '/\*/d' _tor tor4397 4 tcp4 172.27.72.202:52420 [scrubbed]:443 _tor tor4397 12 tcp4 172.27.72.202:54011 [scrubbed]:9001 _tor tor4397 13 tcp4 172.27.72.202:51736 [scrubbed]:443 Wait, so the method detailed here: http://lists.freebsd.org/pipermail/freebsd-questions/2007-November/162970.html specifically: ps -Al after polling for lsof and a foreach loop, doesn't work ? I know it's not elegant, but it appeared to me that: lsof + ps -Al would work ... especially if the system in question is doing little (or nothing) other than Tor ... *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
On Fri, Dec 03, 2010 at 08:34:46PM +0100, Fabian Keil wrote: > Damian Johnson wrote: > > > The lsof command issued by arm [1] is: > > lsof -nPi | grep "\s*.*(ESTABLISHED)" > > What I'll need from you is the following: > > - A command that, when executed as the tor user, produces connection > > results filtered to tor's connections. > > - Example output. > > tor-jail# uname -or > FreeBSD 9.0-CURRENT > tor-jail# su -m _tor -c /bin/csh > tor-jail# id > uid=256(_tor) gid=256(_tor) groups=256(_tor) > tor-jail# procstat -f `pgrep tor` | egrep 'TCP|UDP|PID' > PID COMM FD T V FLAGSREF OFFSET PRO NAME > 3561 tor 4 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:22370 > 3561 tor 5 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 0.0.0.0:0 > 3561 tor 6 s - rw---n-- 2 0 TCP 10.0.0.2:9040 > 0.0.0.0:0 > 3561 tor 7 s - rw---n-- 2 0 UDP 10.0.0.2:53 0.0.0.0:0 > 3561 tor 8 s - rw---n-- 2 0 TCP 10.0.0.2:9051 > 0.0.0.0:0 > 3561 tor14 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:44381 > 3561 tor15 s - rw---n-- 2 0 TCP 10.0.0.2:33734 > [scrubbed]:443 > 3561 tor16 s - rw---n-- 2 0 TCP 10.0.0.2:47704 > [scrubbed]:9001 > 3561 tor17 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:46343 > 3561 tor18 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:64196 > 3561 tor19 s - rw---n-- 2 0 TCP 10.0.0.2:18856 > [scrubbed]:443 > 3561 tor20 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:20385 > 3561 tor22 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:27541 > 3561 tor23 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:21877 > (Public IP addresses scrubbed) Sorry for jumping in , but please notice the above command might not not work on all versions of FBSD, at least it doesn't on a 7-Stable jail. Maybe the following just produces a similar sufficient output: _...@ato# id uid=256(_tor) gid=256(_tor) groups=256(_tor) _...@ato# sockstat -4 | grep tor _tor tor4397 7 tcp4 172.27.72.202:9050*:* _tor tor4397 8 udp4 172.27.72.202:53 *:* _tor tor4397 9 tcp4 172.27.72.202:9051*:* _tor tor4397 12 tcp4 172.27.72.202:54011 [scrubbed]:9001 _tor tor4397 15 tcp4 172.27.72.202:59374 [scrubbed]:9001 _tor tor4397 19 tcp4 172.27.72.202:59673 [scrubbed]:9001 _tor tor4397 20 tcp4 172.27.72.202:51946 [scrubbed]:443 _tor tor4397 22 tcp4 172.27.72.202:60344 [scrubbed]:9001 for *not* displaying listening ports just use _...@ato# sockstat -4 | grep tor| sed '/\*/d' _tor tor4397 4 tcp4 172.27.72.202:52420 [scrubbed]:443 _tor tor4397 12 tcp4 172.27.72.202:54011 [scrubbed]:9001 _tor tor4397 13 tcp4 172.27.72.202:51736 [scrubbed]:443 > > - Be available to test a potential fix. > > > > If you're up for that then I'm glad to have the help! Lets take > > further discussion of this off the list. I don't think this is > > generally of interest to the rest of the tor community. -Damian > > It's at least interesting to a part of the rest of the tor community. It certainly is ! > Fabian *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
Perfect! I'll try to provide a fix for you to test later today or tomorrow. > I intent to try Arm in the future. Are you aware of anyone working on a port? Nope. Jesse just finished an ebuild for Gentoo: https://bugs.gentoo.org/show_bug.cgi?id=341731 and I'm working with Peter on a deb. But thus far no one has volunteered to do a bsd port. On Fri, Dec 3, 2010 at 11:34 AM, Fabian Keil wrote: > Damian Johnson wrote: > >> The lsof command issued by arm [1] is: >> lsof -nPi | grep "\s*.*(ESTABLISHED)" >> >> I'd be happy to work with you to provide a fix, if you'd like. Once >> upon a time I tried to use VMs to troubleshoot FreeBSD and Gentoo >> issues (thus far they're the only platforms to give arm any trouble). >> However, either VirtualBox, those OSes, or the combination of the two >> made this a colossal pain in the ass. Trying to wrangle even the most >> basic functionality out of those systems chewed up dozens of hours so >> that's definitely *not* a road I'm going down again. >> >> What I'll need from you is the following: >> - A command that, when executed as the tor user, produces connection >> results filtered to tor's connections. >> - Example output. > > tor-jail# uname -or > FreeBSD 9.0-CURRENT > tor-jail# su -m _tor -c /bin/csh > tor-jail# id > uid=256(_tor) gid=256(_tor) groups=256(_tor) > tor-jail# procstat -f `pgrep tor` | egrep 'TCP|UDP|PID' > PID COMM FD T V FLAGS REF OFFSET PRO NAME > 3561 tor 4 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:22370 > 3561 tor 5 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 0.0.0.0:0 > 3561 tor 6 s - rw---n-- 2 0 TCP 10.0.0.2:9040 > 0.0.0.0:0 > 3561 tor 7 s - rw---n-- 2 0 UDP 10.0.0.2:53 0.0.0.0:0 > 3561 tor 8 s - rw---n-- 2 0 TCP 10.0.0.2:9051 > 0.0.0.0:0 > 3561 tor 14 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:44381 > 3561 tor 15 s - rw---n-- 2 0 TCP 10.0.0.2:33734 > [scrubbed]:443 > 3561 tor 16 s - rw---n-- 2 0 TCP 10.0.0.2:47704 > [scrubbed]:9001 > 3561 tor 17 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:46343 > 3561 tor 18 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:64196 > 3561 tor 19 s - rw---n-- 2 0 TCP 10.0.0.2:18856 > [scrubbed]:443 > 3561 tor 20 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:20385 > 3561 tor 22 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:27541 > 3561 tor 23 s - rw---n-- 2 0 TCP 10.0.0.2:9050 > 10.0.0.1:21877 > (Public IP addresses scrubbed) > >> - Be available to test a potential fix. >> >> If you're up for that then I'm glad to have the help! Lets take >> further discussion of this off the list. I don't think this is >> generally of interest to the rest of the tor community. -Damian > > It's at least interesting to a part of the rest of the tor community. > I intent to try Arm in the future. Are you aware of anyone working on a port? > > Fabian > *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
On Fri, 3 Dec 2010, Fabian Keil wrote: - Be available to test a potential fix. If you're up for that then I'm glad to have the help! Lets take further discussion of this off the list. I don't think this is generally of interest to the rest of the tor community. -Damian It's at least interesting to a part of the rest of the tor community. I intent to try Arm in the future. Are you aware of anyone working on a port? I have neither seen a port, nor seen discussion of one... *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
Damian Johnson wrote: > The lsof command issued by arm [1] is: > lsof -nPi | grep "\s*.*(ESTABLISHED)" > > I'd be happy to work with you to provide a fix, if you'd like. Once > upon a time I tried to use VMs to troubleshoot FreeBSD and Gentoo > issues (thus far they're the only platforms to give arm any trouble). > However, either VirtualBox, those OSes, or the combination of the two > made this a colossal pain in the ass. Trying to wrangle even the most > basic functionality out of those systems chewed up dozens of hours so > that's definitely *not* a road I'm going down again. > > What I'll need from you is the following: > - A command that, when executed as the tor user, produces connection > results filtered to tor's connections. > - Example output. tor-jail# uname -or FreeBSD 9.0-CURRENT tor-jail# su -m _tor -c /bin/csh tor-jail# id uid=256(_tor) gid=256(_tor) groups=256(_tor) tor-jail# procstat -f `pgrep tor` | egrep 'TCP|UDP|PID' PID COMM FD T V FLAGSREF OFFSET PRO NAME 3561 tor 4 s - rw---n-- 2 0 TCP 10.0.0.2:9050 10.0.0.1:22370 3561 tor 5 s - rw---n-- 2 0 TCP 10.0.0.2:9050 0.0.0.0:0 3561 tor 6 s - rw---n-- 2 0 TCP 10.0.0.2:9040 0.0.0.0:0 3561 tor 7 s - rw---n-- 2 0 UDP 10.0.0.2:53 0.0.0.0:0 3561 tor 8 s - rw---n-- 2 0 TCP 10.0.0.2:9051 0.0.0.0:0 3561 tor14 s - rw---n-- 2 0 TCP 10.0.0.2:9050 10.0.0.1:44381 3561 tor15 s - rw---n-- 2 0 TCP 10.0.0.2:33734 [scrubbed]:443 3561 tor16 s - rw---n-- 2 0 TCP 10.0.0.2:47704 [scrubbed]:9001 3561 tor17 s - rw---n-- 2 0 TCP 10.0.0.2:9050 10.0.0.1:46343 3561 tor18 s - rw---n-- 2 0 TCP 10.0.0.2:9050 10.0.0.1:64196 3561 tor19 s - rw---n-- 2 0 TCP 10.0.0.2:18856 [scrubbed]:443 3561 tor20 s - rw---n-- 2 0 TCP 10.0.0.2:9050 10.0.0.1:20385 3561 tor22 s - rw---n-- 2 0 TCP 10.0.0.2:9050 10.0.0.1:27541 3561 tor23 s - rw---n-- 2 0 TCP 10.0.0.2:9050 10.0.0.1:21877 (Public IP addresses scrubbed) > - Be available to test a potential fix. > > If you're up for that then I'm glad to have the help! Lets take > further discussion of this off the list. I don't think this is > generally of interest to the rest of the tor community. -Damian It's at least interesting to a part of the rest of the tor community. I intent to try Arm in the future. Are you aware of anyone working on a port? Fabian signature.asc Description: PGP signature
Re: Arm Release 1.4.0
The lsof command issued by arm [1] is: lsof -nPi | grep "\s*.*(ESTABLISHED)" I'd be happy to work with you to provide a fix, if you'd like. Once upon a time I tried to use VMs to troubleshoot FreeBSD and Gentoo issues (thus far they're the only platforms to give arm any trouble). However, either VirtualBox, those OSes, or the combination of the two made this a colossal pain in the ass. Trying to wrangle even the most basic functionality out of those systems chewed up dozens of hours so that's definitely *not* a road I'm going down again. What I'll need from you is the following: - A command that, when executed as the tor user, produces connection results filtered to tor's connections. - Example output. - Be available to test a potential fix. If you're up for that then I'm glad to have the help! Lets take further discussion of this off the list. I don't think this is generally of interest to the rest of the tor community. -Damian [1] https://svn.torproject.org/svn/arm/trunk/src/util/connections.py On Wed, Dec 1, 2010 at 10:34 PM, John Case wrote: > > On Wed, 1 Dec 2010, Damian Johnson wrote: > >> Arm should work just fine under BSD with the exception of the >> connection listing. >> >> The problem there is that FreeBSD's netstat lacks the flag to list the >> pids associated with connections (so I can't narrow the list to tor >> connections), ss is a completely different program (a spreadsheet >> application instead of connection resolver), and lsof either had >> similar issues, though I don't recall exactly what. If you know a >> method of getting the connections for a given process under FreeBSD >> then I'm all ears. :) > > > Right - I've been familiar with the limitation, and the reason for the > limitation, for the lifetime of your project. > > I run Arm very well on FreeBSD, but I'd really love to have the connection > listing ... > > Can you use this: > > http://lists.freebsd.org/pipermail/freebsd-questions/2007-November/162970.html > > to cross reference, and get what you need ? > *** > To unsubscribe, send an e-mail to majord...@torproject.org with > unsubscribe or-talk in the body. http://archives.seul.org/or/talk/ > *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
On Wed, 1 Dec 2010, Damian Johnson wrote: Arm should work just fine under BSD with the exception of the connection listing. The problem there is that FreeBSD's netstat lacks the flag to list the pids associated with connections (so I can't narrow the list to tor connections), ss is a completely different program (a spreadsheet application instead of connection resolver), and lsof either had similar issues, though I don't recall exactly what. If you know a method of getting the connections for a given process under FreeBSD then I'm all ears. :) Right - I've been familiar with the limitation, and the reason for the limitation, for the lifetime of your project. I run Arm very well on FreeBSD, but I'd really love to have the connection listing ... Can you use this: http://lists.freebsd.org/pipermail/freebsd-questions/2007-November/162970.html to cross reference, and get what you need ? *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
Arm should work just fine under BSD with the exception of the connection listing. The problem there is that FreeBSD's netstat lacks the flag to list the pids associated with connections (so I can't narrow the list to tor connections), ss is a completely different program (a spreadsheet application instead of connection resolver), and lsof either had similar issues, though I don't recall exactly what. If you know a method of getting the connections for a given process under FreeBSD then I'm all ears. :) That said, everything else (bandwidth graph, logging, configuration editing, etc) should work just fine. -Damian On Tue, Nov 30, 2010 at 10:56 PM, John Case wrote: > > Hi Damian, > > On Tue, 30 Nov 2010, Damian Johnson wrote: > >> Hi. After over a year it's about time that I announced an arm release >> so here it is! What's new since August of 2009 [1], you ask? Lots. The >> project has been under very active development, continuing to add >> usability improvements to make relay operation nicer and less error >> prone. If you're really curious what I've been up to this last year >> then it's all available in the change log [2]. > > > Any news on getting all of Arms functions to work under FreeBSD ? > > Thanks. > *** > To unsubscribe, send an e-mail to majord...@torproject.org with > unsubscribe or-talk in the body. http://archives.seul.org/or/talk/ > *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Arm Release 1.4.0
Hi Damian, On Tue, 30 Nov 2010, Damian Johnson wrote: Hi. After over a year it's about time that I announced an arm release so here it is! What's new since August of 2009 [1], you ask? Lots. The project has been under very active development, continuing to add usability improvements to make relay operation nicer and less error prone. If you're really curious what I've been up to this last year then it's all available in the change log [2]. Any news on getting all of Arms functions to work under FreeBSD ? Thanks. *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
