Re: [tor-dev] Hidden Service Scaling
On 09/10/13 01:16, Matthew Finkel wrote: >> So, before I start trying to implement a prototype, I thought I would >> set out my ideas here to check they are reasonable (I have also been >> discussing this a bit on #tor-dev). The goal of this is two fold, to >> reduce the probability of failure of a hidden service and to increase >> hidden service scalability. >> > > These are excellent goals. It would be even better if you made a > stronger statement about hidden service failure. Something closer to > "increase hidden service availablity", but I won't bikeshed on the > wording. I agree, that is clearer. >> I think what I am planning distils down to two main changes. Firstly, >> when a OP initialises a hidden service, currently if you start a hidden >> service using an existing keypair and address, the new OP's introduction >> points replace the existing introduction points [2]. This does provide >> some redundancy (if slow), but no load balancing. > > So an interesting thing to note about this hack is that it does provide > *some* load balancing. Not much, but some. The reason for this is > because Tor clients cache hidden service descriptors so that they don't > need to refetch every time they want to connect to it. > >> My current plan is to change this such that if the OP has an existing >> public/private keypair and address, it would attempt to lookup the >> existing introduction points (probably over a Tor circuit). If found, it >> then establishes introduction circuits to those Tor servers. >> >> Then comes the second problem, following the above, the introduction >> point would then disconnect from any other connected OP using the same >> public key (unsure why as a reason is not given in the rend-spec). This >> would need to change such that an introduction point can talk to more >> than one instance of the hidden service. >> > > It's important to think about the current design based on the assumption > that a hidden service is a single node. Any modifications to this > assumption will change the behavior of the various components. The only interactions I currently believe can be affected are the Hidden Service instance <-> Introduction point(s) and Hidden Service instance <-> directory server. I need to go and read more about the latter, as I don't have all the information yet. >> These two changes combined should help with the two goals. Reliability >> is improved by having multiple OP's providing the service, and having >> all of these accessible from the introduction points. Scalability is >> also improved, as you are not limited to one OP (as described above, >> currently you can also have +1 but only one will receive most of the >> traffic, and fail over is slow). > > Do you see any disadvantages to this design? So, care needs to be taken around the interaction between the hidden service instances, and the introduction points. If each instance just makes one circuit, then this reveals the number of instances. There is also uncertainty around the replacement of failing introduction points. New ones have to be chosen, but as the service instances do not directly communicate, there could be some interesting behaviour unless this is done carefully. I am also unsure how the lack of direct communication between the hidden service instances could affect the usability of this. I think what would be good to do is take some large, open source, distributed web applications and look at how/how not to set them up using various possible implementations of distributed hidden services. >> I am aware that there are several undefined parts of the above >> description, e.g. how does a introduction point choose what circuit to >> use? but at the moment I am more interested in the wider picture. It >> would be good to get some feedback on this. >> >> 1: https://blog.torproject.org/blog/hidden-services-need-some-love >> 2: >> http://tor.stackexchange.com/questions/13/can-a-hidden-service-be-hosted-by-multiple-instances-of-tor/24#24 > > This is a good start! Some important criteria you might also think > about include how much you trust each component/node and which nodes do > you want to be responsible for deciding where connections are routed. > Also seriously think about how something like a botnet that uses hidden > services might impact the reliability of your design (crazy idea, I > know). I assume the characteristics of this are: 1 or more hidden service instances, connected to by very large numbers of clients, sending and reviving small amounts of information? signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Hidden Service Scaling
On Wed, Oct 09, 2013 at 09:58:07AM +0100, Christopher Baines wrote: > On 09/10/13 01:16, Matthew Finkel wrote: > >> Then comes the second problem, following the above, the introduction > >> point would then disconnect from any other connected OP using the same > >> public key (unsure why as a reason is not given in the rend-spec). This > >> would need to change such that an introduction point can talk to more > >> than one instance of the hidden service. > >> > > > > It's important to think about the current design based on the assumption > > that a hidden service is a single node. Any modifications to this > > assumption will change the behavior of the various components. > > The only interactions I currently believe can be affected are the Hidden > Service instance <-> Introduction point(s) and Hidden Service instance > <-> directory server. I need to go and read more about the latter, as I > don't have all the information yet. Indeed. Lots of issues there. > > >> These two changes combined should help with the two goals. Reliability > >> is improved by having multiple OP's providing the service, and having > >> all of these accessible from the introduction points. Scalability is > >> also improved, as you are not limited to one OP (as described above, > >> currently you can also have +1 but only one will receive most of the > >> traffic, and fail over is slow). > > > > Do you see any disadvantages to this design? > > So, care needs to be taken around the interaction between the hidden > service instances, and the introduction points. If each instance just > makes one circuit, then this reveals the number of instances. You said something similar in response to Nick, specifically you said I believe that to mask the state and possibly number of instances, you would have to at least have some of the introduction points connecting to multiple instances. I didn't understand why you said this in either place. Someone would have to know they had a complete list of introduction points to know the number of instances, but that would depend on how HS descriptors are created, stored, and distributed. From whom is this being hidden? You didn't state the adversary. Is it HS directory servers, intro point operators, potential clients of a hidden service? I don't see why any of these necessarily learns the state or number of instances simply because each intro point is chosen by a single instance (ignoring coincidental collisions if these choices are not coordinated). Also, in your response to Nick you said that not having instances share intro points in some way would place an upper bound on the number of instances. True, but if the number of available intro points >> likely number of instances, this is a nonissue. And come to think of it, not true: if the instances are sometimes choosing the same intro points then this does not bound the number of instances possible (ignoring the number of HSes or instances for which a single intro point can serve as intro point at one time). Also, above you said "If each instance just makes one circuit". Did you mean if there is a single intro point per instance? Hard to say specifically without exploring more, but in general I would be more worried about what is revealed because circuits are built to common intro points by different instances and the intro points can recognize and manage these, e.g., dropping redundant ones than I would be because the number of intro points puts an upper bound on instances. HTH, Paul ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Hidden Service Scaling
On 09/10/13 11:41, Paul Syverson wrote: These two changes combined should help with the two goals. Reliability is improved by having multiple OP's providing the service, and having all of these accessible from the introduction points. Scalability is also improved, as you are not limited to one OP (as described above, currently you can also have +1 but only one will receive most of the traffic, and fail over is slow). >>> >>> Do you see any disadvantages to this design? >> >> So, care needs to be taken around the interaction between the hidden >> service instances, and the introduction points. If each instance just >> makes one circuit, then this reveals the number of instances. > > You said something similar in response to Nick, specifically you said > >I believe that to mask the state and possibly number of instances, >you would have to at least have some of the introduction points >connecting to multiple instances. > > I didn't understand why you said this in either place. Someone would > have to know they had a complete list of introduction points to > know the number of instances, but that would depend on how HS descriptors > are created, stored, and distributed. From whom is this being hidden? > You didn't state the adversary. Is it HS directory servers, intro point > operators, potential clients of a hidden service? I don't see why > any of these necessarily learns the state or number of instances > simply because each intro point is chosen by a single instance > (ignoring coincidental collisions if these choices are not coordinated). To clarify, I was interpreting the goal as only the service operator should know the number of instances. In particular, the adversary here is the introduction point. If hidden service instances only ever create one circuit to each introduction point, each introduction point knows the number of instances of every service it is an introduction point for, as this is the same as the number of circuits for that service. > Also, in your response to Nick you said that not having instances share > intro points in some way would place an upper bound on the number of > instances. True, but if the number of available intro points >> likely > number of instances, this is a nonissue. I don't really follow your reasoning. > And come to think of it, > not true: if the instances are sometimes choosing the same intro points > then this does not bound the number of instances possible (ignoring > the number of HSes or instances for which a single intro point can > serve as intro point at one time). Ok, but I was assuming the current behaviour of Tor, which I believe prevents instances using some of the same introduction points. > Also, above you said "If each instance just makes one circuit". Did > you mean if there is a single intro point per instance? No, as you could have one instance that makes say 3 circuits to just one introduction point. This can help, as it can hide the number of instances from the introduction point. > Hard to say specifically without exploring more, but in general I > would be more worried about what is revealed because circuits are > built to common intro points by different instances and the intro > points can recognize and manage these, e.g., dropping redundant ones > than I would be because the number of intro points puts an upper > bound on instances. I don't quite understand the last part, but regarding introduction points handling more that one circuit for the same service. I think that having this helps possibly hide information (like the number of instances). This does depend on also allowing one instance to use multiple circuits, otherwise, some information would be given away. I might try creating a wiki page on the Tor wiki to collect all of the information in this thread, as it might be a nice reference for discussion. signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [tor-commits] [tor/master] Clarify who learns about ContactInfo.
On Wed, Oct 09, 2013 at 01:45:26PM +, ni...@torproject.org wrote: > diff --git a/src/config/torrc.sample.in b/src/config/torrc.sample.in > index c667efc..78013c2 100644 > --- a/src/config/torrc.sample.in > +++ b/src/config/torrc.sample.in > @@ -120,9 +120,12 @@ > ## is per month) > #AccountingStart month 3 15:00 > > -## Contact info to be published in the directory, so we can contact you > -## if your relay is misconfigured or something else goes wrong. Google > -## indexes this, so spammers might also collect it. > +## Administrative contact information for this relay or bridge. This line > +## can be used to contact you if your relay or bridge is misconfigured or > +## something else goes wrong. Note that we archive and publish all > +## descriptors containing these lines and that Google indexes them, so > +## spammers might also collect them. You may want to obscure the fact that > +## it's an email address and/or generate a new address for this purpose. > #ContactInfo Random Person > ## You might also include your PGP or GPG fingerprint if you have one: > #ContactInfo 0x Random Person Hi Nick, Karsten, You've just changed the torrc.sample.in file, which will cause everybody who uses the deb and upgrades from Tor 0.2.4.17-rc to have to manually evaluate/adjust/replace their /etc/tor/torrc file. Are you sure you want to do this? It's not the end of the world here since the torrc.sample was last updated for Tor 0.2.4.3-alpha, so people upgrading from 0.2.3.x debs will already be affected. But I wonder if it's really worth hassling every deb user who will be upgrading from 0.2.4.x? In any case we should update the ## Last updated 12 September 2012 for Tor 0.2.4.3-alpha. line at the top of the file, so we can accurately predict who will be affected by changes like this in the future. --Roger ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [tor-commits] [tor/master] Clarify who learns about ContactInfo.
On Wed, Oct 9, 2013 at 10:42 AM, Roger Dingledine wrote: > On Wed, Oct 09, 2013 at 01:45:26PM +, ni...@torproject.org wrote: >> diff --git a/src/config/torrc.sample.in b/src/config/torrc.sample.in >> index c667efc..78013c2 100644 >> --- a/src/config/torrc.sample.in >> +++ b/src/config/torrc.sample.in >> @@ -120,9 +120,12 @@ >> ## is per month) >> #AccountingStart month 3 15:00 >> >> -## Contact info to be published in the directory, so we can contact you >> -## if your relay is misconfigured or something else goes wrong. Google >> -## indexes this, so spammers might also collect it. >> +## Administrative contact information for this relay or bridge. This line >> +## can be used to contact you if your relay or bridge is misconfigured or >> +## something else goes wrong. Note that we archive and publish all >> +## descriptors containing these lines and that Google indexes them, so >> +## spammers might also collect them. You may want to obscure the fact that >> +## it's an email address and/or generate a new address for this purpose. >> #ContactInfo Random Person >> ## You might also include your PGP or GPG fingerprint if you have one: >> #ContactInfo 0x Random Person > > Hi Nick, Karsten, > > You've just changed the torrc.sample.in file, which will cause everybody > who uses the deb and upgrades from Tor 0.2.4.17-rc to have to manually > evaluate/adjust/replace their /etc/tor/torrc file. > > Are you sure you want to do this? > > It's not the end of the world here since the torrc.sample was last > updated for Tor 0.2.4.3-alpha, so people upgrading from 0.2.3.x debs > will already be affected. > > But I wonder if it's really worth hassling every deb user who will > be upgrading from 0.2.4.x? > > In any case we should update the > ## Last updated 12 September 2012 for Tor 0.2.4.3-alpha. > line at the top of the file, so we can accurately predict who will be > affected by changes like this in the future. I'm fine either way, up to and including reverting the change for 0.2.4 and keeping it in 0.2.5. What do you think, Karsten? -- Nick ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [tor-commits] [tor/master] Clarify who learns about ContactInfo.
On 10/9/13 4:45 PM, Nick Mathewson wrote: > On Wed, Oct 9, 2013 at 10:42 AM, Roger Dingledine wrote: >> On Wed, Oct 09, 2013 at 01:45:26PM +, ni...@torproject.org wrote: >>> diff --git a/src/config/torrc.sample.in b/src/config/torrc.sample.in >>> index c667efc..78013c2 100644 >>> --- a/src/config/torrc.sample.in >>> +++ b/src/config/torrc.sample.in >>> @@ -120,9 +120,12 @@ >>> ## is per month) >>> #AccountingStart month 3 15:00 >>> >>> -## Contact info to be published in the directory, so we can contact you >>> -## if your relay is misconfigured or something else goes wrong. Google >>> -## indexes this, so spammers might also collect it. >>> +## Administrative contact information for this relay or bridge. This line >>> +## can be used to contact you if your relay or bridge is misconfigured or >>> +## something else goes wrong. Note that we archive and publish all >>> +## descriptors containing these lines and that Google indexes them, so >>> +## spammers might also collect them. You may want to obscure the fact that >>> +## it's an email address and/or generate a new address for this purpose. >>> #ContactInfo Random Person >>> ## You might also include your PGP or GPG fingerprint if you have one: >>> #ContactInfo 0x Random Person >> >> Hi Nick, Karsten, >> >> You've just changed the torrc.sample.in file, which will cause everybody >> who uses the deb and upgrades from Tor 0.2.4.17-rc to have to manually >> evaluate/adjust/replace their /etc/tor/torrc file. >> >> Are you sure you want to do this? >> >> It's not the end of the world here since the torrc.sample was last >> updated for Tor 0.2.4.3-alpha, so people upgrading from 0.2.3.x debs >> will already be affected. >> >> But I wonder if it's really worth hassling every deb user who will >> be upgrading from 0.2.4.x? >> >> In any case we should update the >> ## Last updated 12 September 2012 for Tor 0.2.4.3-alpha. >> line at the top of the file, so we can accurately predict who will be >> affected by changes like this in the future. > > I'm fine either way, up to and including reverting the change for > 0.2.4 and keeping it in 0.2.5. > > What do you think, Karsten? Oh! Probably not worth hassling 0.2.4 deb users. It looks like many people upgraded to 0.2.4.17-rc in September. There are about 1000 of 4400 relays running 0.2.4 these days [0]. How about we partially a) revert the change to torrc.sample.in for 0.2.4 (the man page change can stay, right?) and b) update the "Last updated" part in the 0.2.5 torrc.sample.in? Nick, please find https://gitweb.torproject.org/karsten/tor.git/shortlog/refs/heads/task-9854-2 for the change in b). I'm not entirely sure how to do the change in a) without messing up your backporting logic. Thanks! All the best, Karsten [0] https://metrics.torproject.org/network.html?graph=versions&start=2013-01-01&end=2013-10-09#versions ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Pluggable Transport TBB Build
Hi David! Thanks for the detailed response. My thoughts are inline. On Sun, Oct 6, 2013 at 3:30 PM, David Fifield wrote: > On Sat, Oct 05, 2013 at 11:08:40AM -0400, Kevin P Dyer wrote: >> Sorry I missed the most recent Pluggable Transport bi-weekly meeting. >> >> In regards to the PTTBB build, I plan to make progress towards a >> streamlined build process. >> >> Platforms I want to get the PTTBB build working on: >> 1) CentOS 5.9 (32-bit) >> 2) CentOS 5.9 (64-bit) >> 3) Windows 7 >> 4) OSX 10.8 >> >> For each build platform I'm going to: >> 1) Create a VirtualBox image. >> 2) Construct a script that takes the latest TBB release and VirtualBox >> image as input, and outputs a PTTBB zip file. The output PTTBB zip >> file will include all of the "deployed" pluggable transports. >> >> The build process must be deterministic and reproducible. > > Thank you for taking an interest in this. Two things: > 1. There already exist build scripts and VM instructions, which we use > to build the PT TBB. If you want to build a bundle including FTE, > your best bet is to start by modifying those scripts. > 2. The Tor Project already has a very nice reproducible build system, > not yet used for the PT TBB. I want to start using it for the PT > bundles in the nearish future. > > For the scripts we use to build the PT TBB now, please see > https://gitweb.torproject.org/pluggable-transports/bundle.git > https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/Makefile > https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/bundle-gnulinux.txt > https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/bundle-macosx.txt > https://gitweb.torproject.org/pluggable-transports/bundle.git/blob/HEAD:/bundle-windows.txt > They work much the way you describe: you boot a VM, and run a "make" > command. The makefile unzips the vanilla bundle, builds the pluggable > transports and copies them into the bundle, then zips it up again. This is a good start and goes a long way towards automating the build process. However, I'm looking for "Press a button, make a sandwich, have all builds (for all platforms) ready when I come back."-type of automation. It appears that the current procedure requires booting up four systems (virtual or physical) manually, then manually running the build script in each instance. I've had success fully-automating the build using vagrant for the linux builds. (Execute "vagrant up," wait a few minutes, then we have i386/amd64 PTTBB .tar.gz files.) I only encountered two issues [1,2]. OSX is next. How may I share this work on https://gitweb.torproject.org/? > At the time we started making PT TBBs, Tor's reproducible build system > was not finished. I think the new system has great advantages for PT TBB > builds, so I want to start building them that way. That is the subject > of this ticket: > https://trac.torproject.org/projects/tor/ticket/9444 > > About the reproducible build system, please see this blog post and its > linked documentation: > https://blog.torproject.org/blog/deterministic-builds-part-two-technical-details > You make a good point about the need for reproducibility. This existing > system took six months of work working around nontrivial problems (see > the blog post), and it's super nice. It's not something you want to > reinvent by yourself. I agree. We should definitely work towards using that system long term. >> I was hoping to do this on Amazon Web Services, to avoid the licensing >> issues. However, AWS does not support OSX. > > A huge advantage of the reproducible build system is that it requires > neither a Windows license nor an OS X install. Everything is > cross-compiled from Ubuntu. From a practical perspective, I care about > this feature even more than reproducibility. The need to boot up an > actual physical Mac is one reason why PT builds have lagged behind > (#9391). Cross-compilation would certainly be a nice feature to have. However, for testing the product of a cross-compiled build, I'll need to have a development environment with each of the target platforms. I'm unwilling to have a bunch of physical machines for testing. Hence, I'm eager for full virtualization, which also helps with full-automation of the build. I've looked a bit more into the OSX EULA. Even though running OSX as a guest on Windows/Linux seems prevalent, it is against the EULA [4]. However, the EULA, since 10.7, states (c.f., sec. 2.B.iii) that you can spin up virtual instances on Mac hardware. So, I guess it will be up to each of us to figure out our ideal configuration, but the only legal way to do full virtualization requires Mac hardware. Lame. >> I anticipate this will be a non-trivial effort to get working >> correctly. So, it would be great if you could help me adjust my plan >> to minimize headache! > > As you see, it's a bit muddled because we are in a transition between > two build systems. What I recommend is first doing a
Re: [tor-dev] [tor-commits] [tor/master] Clarify who learns about ContactInfo.
On Wed, Oct 9, 2013 at 11:49 AM, Karsten Loesing wrote: > On 10/9/13 4:45 PM, Nick Mathewson wrote: >> On Wed, Oct 9, 2013 at 10:42 AM, Roger Dingledine wrote: >>> On Wed, Oct 09, 2013 at 01:45:26PM +, ni...@torproject.org wrote: diff --git a/src/config/torrc.sample.in b/src/config/torrc.sample.in index c667efc..78013c2 100644 --- a/src/config/torrc.sample.in +++ b/src/config/torrc.sample.in @@ -120,9 +120,12 @@ ## is per month) #AccountingStart month 3 15:00 -## Contact info to be published in the directory, so we can contact you -## if your relay is misconfigured or something else goes wrong. Google -## indexes this, so spammers might also collect it. +## Administrative contact information for this relay or bridge. This line +## can be used to contact you if your relay or bridge is misconfigured or +## something else goes wrong. Note that we archive and publish all +## descriptors containing these lines and that Google indexes them, so +## spammers might also collect them. You may want to obscure the fact that +## it's an email address and/or generate a new address for this purpose. #ContactInfo Random Person ## You might also include your PGP or GPG fingerprint if you have one: #ContactInfo 0x Random Person >>> >>> Hi Nick, Karsten, >>> >>> You've just changed the torrc.sample.in file, which will cause everybody >>> who uses the deb and upgrades from Tor 0.2.4.17-rc to have to manually >>> evaluate/adjust/replace their /etc/tor/torrc file. >>> >>> Are you sure you want to do this? >>> >>> It's not the end of the world here since the torrc.sample was last >>> updated for Tor 0.2.4.3-alpha, so people upgrading from 0.2.3.x debs >>> will already be affected. >>> >>> But I wonder if it's really worth hassling every deb user who will >>> be upgrading from 0.2.4.x? >>> >>> In any case we should update the >>> ## Last updated 12 September 2012 for Tor 0.2.4.3-alpha. >>> line at the top of the file, so we can accurately predict who will be >>> affected by changes like this in the future. >> >> I'm fine either way, up to and including reverting the change for >> 0.2.4 and keeping it in 0.2.5. >> >> What do you think, Karsten? > > Oh! Probably not worth hassling 0.2.4 deb users. It looks like many > people upgraded to 0.2.4.17-rc in September. There are about 1000 of > 4400 relays running 0.2.4 these days [0]. > > How about we partially a) revert the change to torrc.sample.in for 0.2.4 > (the man page change can stay, right?) and b) update the "Last updated" > part in the 0.2.5 torrc.sample.in? > > Nick, please find > https://gitweb.torproject.org/karsten/tor.git/shortlog/refs/heads/task-9854-2 > for the change in b). I'm not entirely sure how to do the change in a) > without messing up your backporting logic. Merging b. To do a, I did: 541 cd src/tor-024 545 git revert --no-commit 66a04a6ac334775dc396025e0c15fa49eca138a7 549 git reset doc/tor.1.txt 551 git checkout doc/tor.1.txt 554 git revert --continue 556 git push 557 cd ../tor 562 git log master..origin/maint-0.2.4 565 git merge -s ours origin/maint-0.2.4 566 git push (The omitted commands are obsessive "git status" invocations, and screw-ups.) ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Reminder: Attentive Otter IM Bundle Meeting in 1h
Hello hello, This is your roughly 60-minute reminder that we'll be comparing proposals for the IM browser bundle on #tor-dev in about an hour. For your reference, and to brush up on the research that everyone valiantly did, the analyses are: *[Purple](https://lists.torproject.org/pipermail/tor-dev/2013-October/005544.html) *[xmpp-client](https://lists.torproject.org/pipermail/tor-dev/2013-October/005546.html) *[Instantbird](https://lists.torproject.org/pipermail/tor-dev/2013-October/00.html) That's all for now. -Tom ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Pluggable Transport TBB Build
On Wed, Oct 09, 2013 at 12:02:20PM -0400, Kevin P Dyer wrote: > On Sun, Oct 6, 2013 at 3:30 PM, David Fifield wrote: > This is a good start and goes a long way towards automating the build > process. However, I'm looking for "Press a button, make a sandwich, > have all builds (for all platforms) ready when I come back."-type of > automation. It appears that the current procedure requires booting up > four systems (virtual or physical) manually, then manually running the > build script in each instance. > > I've had success fully-automating the build using vagrant for the > linux builds. (Execute "vagrant up," wait a few minutes, then we have > i386/amd64 PTTBB .tar.gz files.) I only encountered two issues [1,2]. > OSX is next. How may I share this work on > https://gitweb.torproject.org/? My point is that the "push a button" technique is already handled by the Gitian build. (Even though it's more like "make 20 sandwiches.") I just want to caution you against redoing a lot of work that has already been done. Did you know that the Gitian build has a Vagrant configuration? https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/HEAD:/vagrant Many of these battles have already been fought. Also understand my motives: in the future, I don't want the building of PT bundles to fall post-hoc to pluggable transport authors. I want them to be made as a side effect of the vanilla bundle builds, so that nor I nor others not interested in making bundles will have to make them. So while improvements to the bundle.git system are welcome, I eventually want to get rid of it completely. Pushing a git repo somewhere is a good way to share your changes. > > A huge advantage of the reproducible build system is that it requires > > neither a Windows license nor an OS X install. Everything is > > cross-compiled from Ubuntu. From a practical perspective, I care about > > this feature even more than reproducibility. The need to boot up an > > actual physical Mac is one reason why PT builds have lagged behind > > (#9391). > > Cross-compilation would certainly be a nice feature to have. However, > for testing the product of a cross-compiled build, I'll need to have a > development environment with each of the target platforms. I'm > unwilling to have a bunch of physical machines for testing. Hence, I'm > eager for full virtualization, which also helps with full-automation > of the build. > > I've looked a bit more into the OSX EULA. Even though running OSX as a > guest on Windows/Linux seems prevalent, it is against the EULA [4]. > However, the EULA, since 10.7, states (c.f., sec. 2.B.iii) that you > can spin up virtual instances on Mac hardware. So, I guess it will be > up to each of us to figure out our ideal configuration, but the only > legal way to do full virtualization requires Mac hardware. Lame. Virtualizing OS X is problematic. There are many OS X testers, however, on the tor-qa list, to which new bundles are announced before they are published. David Fifield ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Reminder: Attentive Otter IM Bundle Meeting in 1h
On 10/09/2013 06:04 PM, Tom Lowenthal wrote: > Hello hello, > > This is your roughly 60-minute reminder that we'll be comparing > proposals for the IM browser bundle on #tor-dev in about an hour. For > your reference, and to brush up on the research that everyone > valiantly did, the analyses are: > > *[Purple](https://lists.torproject.org/pipermail/tor-dev/2013-October/005544.html) > *[xmpp-client](https://lists.torproject.org/pipermail/tor-dev/2013-October/005546.html) > *[Instantbird](https://lists.torproject.org/pipermail/tor-dev/2013-October/00.html) > > That's all for now. > -Tom It's rescheduled and happening in one hour. Jurre -- Developer at https://www.useotrproject.org/ ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Hidden Service Scaling
On Wed, Oct 09, 2013 at 09:58:07AM +0100, Christopher Baines wrote: > On 09/10/13 01:16, Matthew Finkel wrote: > >> Then comes the second problem, following the above, the introduction > >> point would then disconnect from any other connected OP using the same > >> public key (unsure why as a reason is not given in the rend-spec). This > >> would need to change such that an introduction point can talk to more > >> than one instance of the hidden service. > >> > > > > It's important to think about the current design based on the assumption > > that a hidden service is a single node. Any modifications to this > > assumption will change the behavior of the various components. > > The only interactions I currently believe can be affected are the Hidden > Service instance <-> Introduction point(s) and Hidden Service instance > <-> directory server. I need to go and read more about the latter, as I > don't have all the information yet. Also, to be fair, one of the devs has already started working on upgrading various components of hidden services [3][4]. You may also want to read through these so you have an idea of what are some future plans. Also, keep in mind that that the current design may not work well for this (scaling) use case. Perhaps also thinking about modifications to the current design that are backwards compatible will help. > >> These two changes combined should help with the two goals. Reliability > >> is improved by having multiple OP's providing the service, and having > >> all of these accessible from the introduction points. Scalability is > >> also improved, as you are not limited to one OP (as described above, > >> currently you can also have +1 but only one will receive most of the > >> traffic, and fail over is slow). > > > > Do you see any disadvantages to this design? > > So, care needs to be taken around the interaction between the hidden > service instances, and the introduction points. If each instance just > makes one circuit, then this reveals the number of instances. Does it? > There is also uncertainty around the replacement of failing introduction > points. New ones have to be chosen, but as the service instances do not > directly communicate, there could be some interesting behaviour unless > this is done carefully. Is there a reason they shouldn't communicate with each other? > I am also unsure how the lack of direct communication between the hidden > service instances could affect the usability of this. I think what would > be good to do is take some large, open source, distributed web > applications and look at how/how not to set them up using various > possible implementations of distributed hidden services. > > >> I am aware that there are several undefined parts of the above > >> description, e.g. how does a introduction point choose what circuit to > >> use? but at the moment I am more interested in the wider picture. It > >> would be good to get some feedback on this. > >> > >> 1: https://blog.torproject.org/blog/hidden-services-need-some-love > >> 2: > >> http://tor.stackexchange.com/questions/13/can-a-hidden-service-be-hosted-by-multiple-instances-of-tor/24#24 > > > > This is a good start! Some important criteria you might also think > > about include how much you trust each component/node and which nodes do > > you want to be responsible for deciding where connections are routed. > > Also seriously think about how something like a botnet that uses hidden > > services might impact the reliability of your design (crazy idea, I > > know). > > I assume the characteristics of this are: 1 or more hidden service > instances, connected to by very large numbers of clients, sending and > reviving small amounts of information? Perhaps, but just think about the load an intro point can handle and sustain. If Introduction Points are where load balacing takes place, then does this affect the difficulty of attacking a hidden service? (for some undefined definition of 'attack'.) [3] https://lists.torproject.org/pipermail/tor-dev/2013-October/005534.html [4] https://lists.torproject.org/pipermail/tor-dev/2013-October/005536.html ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Reminder: Attentive Otter IM Bundle Meeting in 1h
How about a specific time? with the time zone indicated. thanks. On Wed, Oct 9, 2013 at 10:11 AM, Jurre van Bergen wrote: > On 10/09/2013 06:04 PM, Tom Lowenthal wrote: > > Hello hello, > > > > This is your roughly 60-minute reminder that we'll be comparing > > proposals for the IM browser bundle on #tor-dev in about an hour. For > > your reference, and to brush up on the research that everyone > > valiantly did, the analyses are: > > > > *[Purple]( > https://lists.torproject.org/pipermail/tor-dev/2013-October/005544.html) > > *[xmpp-client]( > https://lists.torproject.org/pipermail/tor-dev/2013-October/005546.html) > > *[Instantbird]( > https://lists.torproject.org/pipermail/tor-dev/2013-October/00.html) > > > > That's all for now. > > -Tom > It's rescheduled and happening in one hour. > > Jurre > > -- > Developer at https://www.useotrproject.org/ > > ___ > tor-dev mailing list > tor-dev@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Reminder: Attentive Otter IM Bundle Meeting in 1h
On 2013-10-09 2:05 PM, Andrew F wrote: > How about a specific time? with the time zone indicated. > thanks. > It's starting right now in #tor-dev. That's 18:00 UTC -- Best Regards, Justin Bull E09D 38DE 8FB7 5745 2044 A0F4 1A2B DEAA 68FD B34C signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Pluggable Transport TBB Build
On Wed, Oct 9, 2013 at 12:22 PM, David Fifield wrote: > On Wed, Oct 09, 2013 at 12:02:20PM -0400, Kevin P Dyer wrote: >> On Sun, Oct 6, 2013 at 3:30 PM, David Fifield wrote: >> This is a good start and goes a long way towards automating the build >> process. However, I'm looking for "Press a button, make a sandwich, >> have all builds (for all platforms) ready when I come back."-type of >> automation. It appears that the current procedure requires booting up >> four systems (virtual or physical) manually, then manually running the >> build script in each instance. >> >> I've had success fully-automating the build using vagrant for the >> linux builds. (Execute "vagrant up," wait a few minutes, then we have >> i386/amd64 PTTBB .tar.gz files.) I only encountered two issues [1,2]. >> OSX is next. How may I share this work on >> https://gitweb.torproject.org/? > > My point is that the "push a button" technique is already handled by the > Gitian build. (Even though it's more like "make 20 sandwiches.") I just > want to caution you against redoing a lot of work that has already been > done. Did you know that the Gitian build has a Vagrant configuration? > https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/HEAD:/vagrant > Many of these battles have already been fought. > > Also understand my motives: in the future, I don't want the building of > PT bundles to fall post-hoc to pluggable transport authors. I want them > to be made as a side effect of the vanilla bundle builds, so that nor I > nor others not interested in making bundles will have to make them. So > while improvements to the bundle.git system are welcome, I eventually > want to get rid of it completely. I conflated two issues: challenges in setting up my personal development environment and challenges in the formal Tor build process. I'm personally motivated to get fully-automated PTTBB builds working to make my development life easier. I'll make less mistakes if I can make a change, then easily test those changes across multiple platforms. -Kevin ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Attentive Otter IM bundle update
We had a meeting as scheduled at 1100h Pacific. We broadly agreed that: * we don't want to use Pidgin/libpurple * we have a mild preference for Instantbird. We have some open questions about Instantbird: 1. What sort of attack surface does Instantbird present, compared to -- say -- a browser. 2. What's the timeline for Instantbird implementing OTR, and how are they going about it. Our current plan is to go for a more thorough investigation of Instantbird, and work out what we'd need to do to take Instantbird and produce Tor Messenger, and what roadblocks stand in our way. We were unable to find a lead for this project during the meeting. Afterwards, Arlo volunteered to put together such a plan, and possibly even execute it. Arlo has committed to putting that plan together by the end of the weekend. If we allow Monday morning to read and digest Arlo's work, we should reconvene some time on Monday afternoon, Tuesday, or Wednesday next week. [Let me know when you're free.][1] I've decided to send out updates to both tor-dev and tor-assistants, as well as to the individual addresses of folks who are interested. I'm committed to using the string "Otter" and "Attentive" in the subject line. If you want to get those emails, please [edit the wiki page][2] to add that info. I've only put my info there for now, so please edit it if you want to get personal attention in future emails. What you should do now: 1. Please [tell me about your availability][1] to meet next week. 2. Please add your name, preferred email, and IRC nick to the appropriate section of the [Attentive Otter wiki page][2]. [1]: http://doodle.com/g7xq4e2x8pgf7vda [2]: https://trac.torproject.org/projects/tor/wiki/org/sponsors/Otter/Attentive On 9 October 2013 09:04, Tom Lowenthal wrote: > Hello hello, > > This is your roughly 60-minute reminder that we'll be comparing > proposals for the IM browser bundle on #tor-dev in about an hour. For > your reference, and to brush up on the research that everyone > valiantly did, the analyses are: > > *[Purple](https://lists.torproject.org/pipermail/tor-dev/2013-October/005544.html) > *[xmpp-client](https://lists.torproject.org/pipermail/tor-dev/2013-October/005546.html) > *[Instantbird](https://lists.torproject.org/pipermail/tor-dev/2013-October/00.html) > > That's all for now. > -Tom ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Attentive Otter: Analysis of Instantbird/Thunderbird
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 So the lack of OTR support in Instantbird is nearly a dealbreaker for me, as it makes it a bit more likely than a rogue exit could intercept a user's communications. Though this depends in part on SSL/TLS support and whether a user *actually enables* it in their settings. Would the plan be to create and test a reliable OTR patch for Instantbird? Pidgin's big issue before was DNS leaks. How is this addressed by Instantbird? (okay, there are a few big issues with Pidgin but...) I like Instantbird's UI, but we should come up with a plan to set proper defaults. ~Griffin Mike Perry & Sukhbir Singh wrote: > - No OTR support yet > + OTR support tickets: > https://bugzilla.instantbird.org/show_bug.cgi?id=877 > https://bugzilla.mozilla.org/show_bug.cgi?id=779052 > + For a stopgap/prototype: We can use the js-ctypes wrapper of libotr > along with the message observer API > + Example observer API use w/ rot13: > http://hg.instantbird.org/addons/file/tip/rot13 > + JS-Ctypes wrapper for native libotr: > http://gitorious.org/fireotr/fireotr/blobs/master/chrome/content/otr_wrapper.js > + The ctypes wrapper can be converted to an XPCOM wrapper later. > + According to sshagarwal #maildev on irc.mozilla.org, Mozilla is > also working towards implementing all of the primitives needed for OTR (and OTR > itself) in NSS. These are listed in this comment: > https://bugzilla.mozilla.org/show_bug.cgi?id=779052#c17 > + We could also rely on the ctypes wrapper until native support is > available, and possibly skip an XPCOM libotr wrapper entirely. - -- "Cypherpunks write code not flame wars." --Jurre van Bergen #Foucault / PGP: 0xAE792C97 / OTR: sa...@jabber.ccc.de My posts are my own, not my employer's. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQIcBAEBAgAGBQJSVcVCAAoJEOMx/SmueSyX/T4QALZ3WzbxxNtOj/FbDciYb3t5 B793kSDIrezwZ/lhYtfgAxP9TKGZ2K5yCBf5CtAPWoBSp4vXmscdfkI+1jE2aU6E rBCTgjDpw+r27hrmU6rm78awhzrg6jpTrBpiMFuEKmCT0ZREdt/1xj9cJCLYjUId 50fmz7uv6/8MeLvy+yUJTYMbxwLfTbLRMWY7OzipJTozuot3+13I6qh4zh9N5qnq qahxu2cpE+oe8MuWSHdGv14pHziKs+r9ebIh4WKcrP4dRq4hixagSRbS078XN9fR BEV+JFsHrtLXLjvfaxxBZor4lhoK3Zt1GmgPFOB+LiMZh5X9LdsVlLSha04+084z JmMkYVruPYAJo0uaxOQU0pTTKJmzSwxzB3xebw+oGBzrLP5rdO85oOOv6ND+mOFv EymTo6exsbQiFg7bmFuT2pF9npJsqpKuNEM+Pinrxx1JZrzPTBBD0wDMZs6jh7Z3 vE7cyFYI5qIfv5uAyDQyF1UENX8L7HIQIf9N4TmagksdEsM/wAn9w8ZswSsUwUHO xOzaJwC/4NnUQODlI/YYzI+9E6vaXJ2RxWtiCeCGHSbDkXiAdUTPps6se4I/jdGW t2G6DPFM+k9BZDA8Wz+ulH10rlaYOPuhTmBHvuf+cXZpZRyLpse4bTqCcSOAfL8p PZbvZukOi83RQ4ThJv0C =5/Gh -END PGP SIGNATURE- ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [30-minute reminder] Cute Otters hidden services and publishing meeting at 1500h Pacific in #tor-dev
See you soon. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [ACTION REQUESTED] Attentive Otter IM bundle: meeting scheduling; team signup [due Friday]
Please read your other mail or the [wiki][2] for a status update. In addition, I would like you to do the following two things: 1. [Tell me about your availability][1] to meet next week. 2. Add your name, preferred email, and IRC nick to the appropriate section of the [Attentive Otter wiki page][2]. For maximum effectiveness, you should before about 1530h Friday, since that's when I'll send out the meeting invite. If you haven't filled the Doodle out, I won't account for your availability. If you haven't added your details, you'll need to read `tor-dev` or `tor-assistants` to find out when the meeting is. [1]: http://doodle.com/g7xq4e2x8pgf7vda [2]: https://trac.torproject.org/projects/tor/wiki/org/sponsors/Otter/Attentive ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Design for an exit relay scanner: feedback appreciated
I am working on a Python-based exit relay scanner which should detect malicious and misbehaving exits. The design should have a reasonable balance between being fast/parallel and stressing the network as little as possible. I came up with the following three steps: 1. Spawn a "parent" Tor process to get an up-to-date consensus. 2.1 For every selected exit relay, spawn a lightweight Tor process. 2.2 The consensus is copied from the "parent" process to the lightweight process' data directory. That way, the consensus has to be downloaded only once. 2.3 Every lightweight Tor process has the following configuration: --- SOCKSPort auto ControlPort 0 __DisablePredictedCircuits 1 UseEntryGuards 0 FetchServerDescriptors 0 DataDirectory ExitNodes --- Entry guards are not used to distribute the load. Predicted circuits are disabled to prevent expensive creation of circuits which would not be used anyway. In addition, I am considering adding "EntryNodes" or "Bridge" to concentrate the first hop's load on machines under my control. 3. torsocks is then used to establish decoy connections over the respective exit relay. After that, the process is terminated. Any thoughts on how to further improve the design or ideas for a better one? Cheers, Philipp ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [ACTION REQUESTED] Cute Otters hidden services publishing meeting request &c. [due Friday Morning]
Hello cute pals. We didn't have a meeting today due to scheduling. If you think you want to work on making point-click-publish hidden services a reality, and hidden services better in general, then this is what you should do: 1. [Note down when you're free next week.][1] 2. Add your name, IRC nick and email to the [Cute wiki page][2]. 3. Read [Sina's proposal][3]. I'm planning to read [this Doodle][1] and send out updates on Friday morning (Pacific). If you haven't filled it out by then, I won't be able to account for your preferences. Please fill out the doodle only if you think that I need to consider your availability before scheduling the meeting. If you just want to spectate, no RSVP is needed, you can just lurk in #tor-dev. Please make sure that you only fill out times that you can actually make it. Future emails about Cute Otters will be sent `to` the email addresses listed on the [Cute wiki page][2], `cc`'ing `tor-dev` and `tor-assistants`. If you want to get messages "to" you, you should add yourself to the team listing on the [Cute wiki page][2]. Sina has produced an [outline of the publishing system][3] we're planning to build. You should read it, help him improve it, and come to the meeting next week prepared. That's all for now, -Tom [1]: http://doodle.com/wqr9phdck23mac7u [2]: https://trac.torproject.org/projects/tor/wiki/org/sponsors/Otter/Cute [3]: https://lists.torproject.org/pipermail/tor-dev/2013-October/005558.html ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Pushed HS proposals on git
Hey Nick, I pushed the recent HS proposals to my torspec git repo. You can find them on branch 'hs_experimentation' at https://git.torproject.org/user/asn/torspec.git . My next step would be to write the anti-enumeration crypto part of xxx-hs-id-keys-and-onion-leaking.txt (as I started doing with c8b7e40). Feel free to work on them. Maybe send me an IRC message before you start touching them so that we don't do change the same areas of the spec. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
