Re: [tor-dev] Relay Database: Existing Schemas?

[Kostas Jakeliunas]

On Thu, Apr 16, 2015 at 4:53 PM, Karsten Loesing  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 15/04/15 21:18, nusenu wrote:
>> Hi,
>>
>> I'm planing to store relay data in a database for analysis. I
>> assume others have done so as well, so before going ahead and
>> designing a db schema I'd like to make sure I didn't miss
>> pre-existing db schemas one could build on.
>>
>> Data to be stored: - (most) descriptor fields - everything that
>> onionoo provides in a details record (geoip, asn, rdns, tordnsel,
>> cw, ...) - historic records
>>
>> I didn't find something matching so far, so I'll go ahead, but if
>> you know of other existing relay db schemas I'd like to hear about
>> it.
>>
>> thanks, nusenu
>>
>>
>>
>>
>> "GSoC2013: Searchable Tor descriptor archive" (Kostas Jakeliunas)
>> https://www.google-melange.com/gsoc/project/details/google/gsoc2013/wfn/
>>
>>
> 5866452879933440
>> https://lists.torproject.org/pipermail/tor-dev/2013-May/004923.html
>>
>>
> https://lists.torproject.org/pipermail/tor-dev/2013-September/005357.htm
>> l https://github.com/wfn/torsearch (btw, someone knows the license
>> of this?)
>
> Cc'ing Kostas for this question.

Hi nusenu,

I've been going through old mail, and on 2015-04-16 you asked about
about a license (see above).

Just added a LICENSE file - can't hurt (standard BSD 3-clause).

If you're still by any chance collating (ha) and/or want to talk about
schema design for descriptors (I personally would not lose hope for
RDBMSes for large datasets - not until one gets into *actually* big
data - say, terabytes at least, or more - but of course it gets
nuanced real fast).

--

Kostas.

0x0e5dce45 @ pgp.mit.edu

>
>>> This is true: the summary/details documents (just like in Onionoo
>>>  proper) deal with the *last* known info about relays.
>>
>>
>> ernie
>> https://gitweb.torproject.org/metrics-db.git/plain/doc/manual.pdf
>> (didn't find db/tordir.sql mentioned in the pdf)
>
> That file lives here now:
>
> https://gitweb.torproject.org/metrics-web.git/tree/modules/legacy/db/tordir.sql
>
> A better schema might be the following one though.  It's smaller, but
> it's better documented:
>
> https://gitweb.torproject.org/exonerator.git/tree/db/exonerator.sql
>
>> "Instructions for setting up relay descriptor database"
>> https://lists.torproject.org/pipermail/tor-dev/2010-March/001783.html
>
> That's
>>
> five years old.  I'd say ignore that one.
>
>> "Set up descriptor database for other researchers"
>> https://trac.torproject.org/projects/tor/ticket/1643
>
> Also five years old.  Better ignore.
>
> Hope that helps.
>
> All the best,
> Karsten
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1
> Comment: GPGTools - http://gpgtools.org
>
> iQEcBAEBAgAGBQJVL9rcAAoJEJD5dJfVqbCrFZgIAIEv/Yi4sNoa8clYVAxuk0Sh
> FFbRDT0kLs19t/DgTwUtB6jD4Lh0akMc806AaIFgfCdL+QwcG0llBfZnSsrbszoH
> Xoi226PRx9lPITrA7KYds4PUZfqIqg3ECpNsKNa4PLB7SlQdNfJQ1wDngcwu2CrF
> Hk+zHbu0gfSkfZRBqxt5aJLTFXR0aBYybF4d6sPJ4OW5Al2U8r9DYysXc0xALvwq
> bvEDFctV1wkDgA3mP3guRrXImXYT1AQPFFlz0TR1eBruuSJBiPKIv7Fs/ocns4aR
> OhxIEaKBaAO+HkvyxDcZ1ukXldR13s3MUPD0XvvZ8xQRCBZpNMygqTMi6pIjTN4=
> =a0Nb
> -END PGP SIGNATURE-
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

Re: [tor-dev] stopping the censoring of tor users.

[blacklight .]

i wonder if you could recommend me what i can do to make such a thing, or
someone who can, you see i am an it student and im eager to learn how i can
make these kind of things and contribute to the tor project, but i need
help from other people to accomplish this.

This email has been sent from a virus-free computer protected by Avast.
www.avast.com

<#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On 26 February 2016 at 14:53, Michael Rogers 
wrote:

> As far as I can tell, this would work, and you could do it without any
> changes to the Tor network. Just set up a hidden service where the
> service is an open proxy.
>
> It wouldn't be transparent to clients, however - they'd need to do some
> proxychains-style juggling to connect to their local onion proxy, then
> connect through that to your hidden service, then connect through that
> to the internet. You could of course create a library to do the extra
> work on the client side - but the point is, it wouldn't work for
> unmodified clients.
>
> But perhaps the bigger problem with hidden exit nodes is that when
> someone does something illegal through your exit node and the police
> come to your door, you can't point to the public list of exit nodes and
> say "It wasn't me, I'm just an exit node". The best you can do is point
> to the hidden exit nodes library and say "It might not have been me,
> anyone could be an exit node".
>
> Cheers,
> Michael
>
> On 25/02/16 22:06, blacklight . wrote:
> > About the issue of exit nodes needing to know to which bridge they need
> > to connect to,  could we not make a system that similair to hidden
> > services, so that the nodes can connect to them without knowing the
> > actulle ip adress? If we could design an automatic system in which flash
> > proxies could be configered like that, then it might work i think, what
> > are your thoughts?
> >
> > Op 25 feb. 2016 22:37 schreef "Thom Wiggers"  > >:
> >
> > You may be interested in the following from the FAQ:
> >
> > https://www.torproject.org/docs/faq.html.en#HideExits
> >
> > You should hide the list of Tor relays, so people can't block the
> exits.
> >
> > There are a few reasons we don't:
> >
> > a) We can't help but make the information available, since Tor
> > clients need to use it to pick their paths. So if the "blockers"
> > want it, they can get it anyway. Further, even if we didn't tell
> > clients about the list of relays directly, somebody could still make
> > a lot of connections through Tor to a test site and build a list of
> > the addresses they see.
> > b) If people want to block us, we believe that they should be
> > allowed to do so. Obviously, we would prefer for everybody to allow
> > Tor users to connect to them, but people have the right to decide
> > who their services should allow connections from, and if they want
> > to block anonymous users, they can.
> > c) Being blockable also has tactical advantages: it may be a
> > persuasive response to website maintainers who feel threatened by
> > Tor. Giving them the option may inspire them to stop and think about
> > whether they really want to eliminate private access to their
> > system, and if not, what other options they might have. The time
> > they might otherwise have spent blocking Tor, they may instead spend
> > rethinking their overall approach to privacy and anonymity.
> >
> > On 25/02/16 20:04, blacklight . wrote:
> >> hello there! i don't know if this mailing list works but i thought
> >> of giving it a try.
> >>
> >> i was lately reading an article
> >> (
> http://www.pcworld.com/article/3037180/security/tor-users-increasingly-treated-like-second-class-web-citizens.html
> )
> >>  and it was about tor users getting blocked from accessing alot of
> >> website. but after giving this some thought i think i came up with
> >> a possible solution to the problem :there is a thing called
> >> bridges, they are used to access the tor network without your isp
> >> knowing that you use tor, but if you can use those proxies to
> >> enter the network, it might also be possible to exit the network
> >> with them. But then we face a second challenge, the exit nodes
> >> have to be configured in such a way that it will relay traffic to
> >> such a bridge, so the exit node owners also need to know the ip of
> >> the bridge. While this doesn't seem difficult to do, it can become
> >> difficult. You see if the bridges are published on a public
> >> list(like normal bridges are) then the blocking sites in question
> >> will be able to block those address too. While this also posses a
> >> problem, a possible solution could be found in something called
> >> 

Re: [tor-dev] stopping the censoring of tor users.

[Michael Rogers]

As far as I can tell, this would work, and you could do it without any
changes to the Tor network. Just set up a hidden service where the
service is an open proxy.

It wouldn't be transparent to clients, however - they'd need to do some
proxychains-style juggling to connect to their local onion proxy, then
connect through that to your hidden service, then connect through that
to the internet. You could of course create a library to do the extra
work on the client side - but the point is, it wouldn't work for
unmodified clients.

But perhaps the bigger problem with hidden exit nodes is that when
someone does something illegal through your exit node and the police
come to your door, you can't point to the public list of exit nodes and
say "It wasn't me, I'm just an exit node". The best you can do is point
to the hidden exit nodes library and say "It might not have been me,
anyone could be an exit node".

Cheers,
Michael

On 25/02/16 22:06, blacklight . wrote:
> About the issue of exit nodes needing to know to which bridge they need
> to connect to,  could we not make a system that similair to hidden
> services, so that the nodes can connect to them without knowing the
> actulle ip adress? If we could design an automatic system in which flash
> proxies could be configered like that, then it might work i think, what
> are your thoughts?
> 
> Op 25 feb. 2016 22:37 schreef "Thom Wiggers"  >:
> 
> You may be interested in the following from the FAQ:
> 
> https://www.torproject.org/docs/faq.html.en#HideExits
> 
> You should hide the list of Tor relays, so people can't block the exits.
> 
> There are a few reasons we don't:
> 
> a) We can't help but make the information available, since Tor
> clients need to use it to pick their paths. So if the "blockers"
> want it, they can get it anyway. Further, even if we didn't tell
> clients about the list of relays directly, somebody could still make
> a lot of connections through Tor to a test site and build a list of
> the addresses they see.
> b) If people want to block us, we believe that they should be
> allowed to do so. Obviously, we would prefer for everybody to allow
> Tor users to connect to them, but people have the right to decide
> who their services should allow connections from, and if they want
> to block anonymous users, they can.
> c) Being blockable also has tactical advantages: it may be a
> persuasive response to website maintainers who feel threatened by
> Tor. Giving them the option may inspire them to stop and think about
> whether they really want to eliminate private access to their
> system, and if not, what other options they might have. The time
> they might otherwise have spent blocking Tor, they may instead spend
> rethinking their overall approach to privacy and anonymity.
> 
> On 25/02/16 20:04, blacklight . wrote:
>> hello there! i don't know if this mailing list works but i thought
>> of giving it a try.
>>
>> i was lately reading an article
>> 
>> (http://www.pcworld.com/article/3037180/security/tor-users-increasingly-treated-like-second-class-web-citizens.html)
>>  and it was about tor users getting blocked from accessing alot of
>> website. but after giving this some thought i think i came up with
>> a possible solution to the problem :there is a thing called
>> bridges, they are used to access the tor network without your isp
>> knowing that you use tor, but if you can use those proxies to
>> enter the network, it might also be possible to exit the network
>> with them. But then we face a second challenge, the exit nodes
>> have to be configured in such a way that it will relay traffic to
>> such a bridge, so the exit node owners also need to know the ip of
>> the bridge. While this doesn't seem difficult to do, it can become
>> difficult. You see if the bridges are published on a public
>> list(like normal bridges are) then the blocking sites in question
>> will be able to block those address too. While this also posses a
>> problem, a possible solution could be found in something called
>> flashproxies, flashproxies are bridges with a really short life
>> span, they are created and destroyed fairly swiftly, when this is
>> done in a rapid pace, they become really hard to block because the
>> ip changes all the time. So if the exit nodes can be configured to
>> make use of such flash proxies, then the problem could be solved.
>> I Must admit that not an expert on this or anything, and it needs
>> alot of more thought, but it could work. so i was wondering if
>> there are any experts who could help me with thinking out this
>> subject and maybe confirm if this idea could work.
>>
>>
>> greetings, blacklight
>>
>>
>> ___
>>