[tor-dev] [GSOC16] Fingerprint Central - Status report n°6
Hi everyone, Here is my sixth status report for my GSOC project. Repository: https://github.com/plaperdr/fp-central Beta: https://fpcentral.irisa.fr 1- Development of the "Acceptable" feature When a user sends his fingerprint to the server, he can now see for each attribute if the collected value is "acceptable" or not (i.e., if it is an expected one). If the server returns that a value is not an acceptable one, this means that it is something to investigate. Does it come from a simple misconfiguration of the browser or could it be a genuine difference between TBB browsers that should be fixed? Right now, it is a simple Yes/No answer but I'll add more feedback so that a user understands the information that is presented. 2- i18n support with documentation I have finished the FR version of the website. Not everything is localized since some libraries I use for the front-end are only in English but the majority of FP Central is now translatable. I have written a small guide if anyone wants to contribute (https://github.com/plaperdr/fp-central/tree/master/translations). The process is pretty simple: you generate a file that references all the strings that need to be translated and you complete it with your translations. 3- Multiple bug fixes and improvements I fixed multiple bugs in the customStats page that were caused by the way I structured data in the page and it should be much smoother now. I'm sure other bugs are lurking in the code so do not hesitate to visit the beta website and report to me any bugs you may find. In terms of features, I have normally developed everything I wrote in my proposal. So my focus for the final stretch of the coding period is to clean the code, fix bugs, improve the website and write even more documentation. I'll also start planning the post-GSOC period to see what interesting features could be added in the future and to make sure that FP Central is ready for prime time for users and developers. Have a great week-end, Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [GSOC16] Fingerprint Central - Status report n°5
Hi everyone, Here is my fifth status report for my GSOC project. Repository: https://github.com/plaperdr/fp-central Beta: https://fpcentral.irisa.fr 1 - Support of TBB versions for statistics through tags I have added the selection of TBB versions for the custom statistics page. In order to do that, I designed and developed the tag system that I planned to develop much later for the project. For those who do not know what tags are, it is a way to classify fingerprints and put them into different categories. By analyzing collected fingerprints, the system can assign one or more tags to them. After several iterations, I think I found a good balance between expressiveness and simplicity. You can find the documentation on how to add your own tag to the system here: https://github.com/plaperdr/fp-central/blob/master/fingerprint/tags/README.md I have added a command to refresh and recompute all tags in case a new one is added to the system or if the logic of one tag has been changed. Right now, the tag system is used to tag each fingerprint with the corresponding TBB version but it can be used later on for many different things like identifying the OS or the type of the browser. 2 - Better and more complete custom statistics page. The custom statistics page is now much more complete than before. In addition to the already-existing pie chart, I have added an HTML table which gives the complete list of data from the chosen attributes. You can sort the data and download it in several different formats. It is a handy feature if you want to put the fingerprints into a different software or if you want to copy/paste them for a thorough bug report. I've pushed the latest updates today to the beta server so that you can try them right now and send me feedback on them. My goal for the next two weeks is to iron things out in the beta and fix bugs. I still need to make sure that only TBB fingerprints are stored in the database. Feature-wise, it seems that I'm getting near the end on the list of features I wrote in my proposal. The only real big one that is left is to have a page to see if your complete TBB fingerprint is "acceptable" (right now, only several attributes are checked). Finally, for the final stretch in the second half of august, I'm planning on completing the documentation and the french localization while making sure that deployment and updates of the site are as easy as possible. Have a great week-end, Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [GSOC16] Fingerprint Central - Status report n°4
Hi everyone, Here is my fourth status report for my GSOC project. https://github.com/plaperdr/fp-central 1 - The beta for FP Central is live! You can visit it at this address: https://fpcentral.irisa.fr/ Don't hesitate to send us feedback so that we can improve the website to make it the best we can! I'm already fixing and tweaking several things on my end. Right now, I'm updating the website once a week so don't hesitate to come back to see newly added features! 2 - I've added two statistics pages on the website. I'm still tweaking how the data is being shown to the user but the goal is that anyone can select the attributes they are interested in and get the distribution for a given time period. I'm planning on adding new ways to visualize and download data so that developers can easily get access to what they want. My goal for the next two weeks is to improve the statistics page so that it works well and add support of TBB versions. I'll also make sure that only TBB fingerprints are stored in the database so that the current stats are not polluted by other browsers. Have a great week-end, Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Fingerprint Central - Testers needed!
Hi everyone, As part of my GSOC project, the beta for Fingerprint Central is finally live! Here is the link: https://fpcentral.irisa.fr For the moment, we are interested in testing the core feature of the project which is the collection of fingerprints. Other features are in development and will be added when they are ready (notably, the statistics page). There are two main pages available now: - "View my fingerprint": On this page, you can execute the tests that have been added in Fingerprint Central. If you have JavaScript enabled, you can use the Dashboard to see the generated data and review them before you send it to our server. The "%" column indicates the percentage of collected fingerprints that share the same value as you. - "Tor": This page will give you several advice related to Tor guidelines and will try to detect the level of your security slider. We are interested in feedback and bug reports! If you feel something is missing or is not right, tell us so that we can improve the website. Unfortunately, we don't have data to correctly kickstart the database so the percentage shown to the first users won't be really interesting. Thanks in advance for your help! Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [GSOC16] Fingerprint Central - Status report n°3
Hi everyone, Here is my third status report for my GSOC project. https://github.com/plaperdr/fp-central 1 - Following my last report, I've added the following elements to Fingerprint Central: - A page that detects the current level of the security slider of the Tor browser - A page that looks if the current fingerprint is "acceptable" compared to the Tor guidelines. This feature will evolve in the future to be more complete and to provide more feedback to users. - Support of users without JavaScript - Improvements of the code of the main FP page so that it reduces server load (by storing locally the information of the current fingerprint) I've also added support of Promises so that computation-heavy tests can be run without blocking the whole collection process. The added test on the AudioContext API is the first one to make use of that feature. 2 - We will launch a beta next week of FP Central to get early feedback on the project. Any ideas and suggestions will be welcome so that the website will be the best it can be for both users and developers. The server is online but I'm still making some final tweaks and bug fixes so that the launch will be smooth. I'll send an email early next week to this mailing list to inform everyone when it'll be ready. My goal in the next two weeks is to launch the beta of the website and start working on a great statistics page along with a more complete version of the "acceptable fingerprint" page. Have a great week-end, Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [GSOC16] Fingerprint Central - Cookies and localStorage
Hello everyone, I know the next status report is not due until next week but I wanted to get some feedback on how I use cookies and localStorage on FP Central. Right now, due the very short lifespan of cookies in the Tor browser, I don't use cookies as an identification mechanism but as an expiration/anti-pollution mechanism. Here is how it works: - For users without JavaScript, I put a cookie the first time a fingerprint is sent. This means that in the same session, no new fingerprint from the same user will be stored if he or she wants to see his or her own fingerprint again. If the user generates a new identity, he will be able to store a new fingerprint since no cookie will be present. - For users with JavaScript, I add a cookie when the test suite is run and I use localStorage to store data generated during the collection process. When the user sees his complete fingerprint with the percentage for each attribute, all the values are stored in localStorage. If the user gets back to the FP page page, it won't have to contact the server and the fingerprint will appear instantaneously by getting the stored copy from localStorage. The presence of data in localStorage prevents the user from sending his fingerprint a second time. The presence of the cookie acts as an expiration mechanism. If the cookie is still present, I consider the data in localStorage to be valid and the user cannot perform the collection process again. If the cookie has expired, I remove data present in localStorage and allow the user to execute the whole process again. I don't know if my approach is a good one but I wanted to use what is available to me in the browser to prevent too many fingerprints from being stored and to lessen as much as possible the server load. If anyone has suggestions on my use of cookies and localStorage, I'll gladly take them. Now, what I'm not really sure is: what about users who want to play with their browser and see the modifications done to their fingerprint? The way the site is built now prevents that. There are no buttons to force the collection process again. I see two alternatives here: add a "Force fingerprinting" button even though the database could be polluted or add a sort of "playground" webpage that is not connected to the database where the user can rerun the test suite as many times as he wants. Hopefully, if everything goes well on my end, I'll be able to launch a beta version of the website next week. We could see from there if everything is working well and if the limitations imposed on users are not too important. Have a great week-end, Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [GSOC16] Fingerprint Central - Status report n°2
On 06/24/2016 02:27 PM, Nicolas Vigier wrote:
> On Fri, 24 Jun 2016, Pierre Laperdrix wrote:
>
>>
>>
>> On 06/23/2016 05:19 PM, Nicolas Vigier wrote:
>>> On Fri, 17 Jun 2016, Pierre Laperdrix wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> Here is my second status report for my GSOC project.
>>>> A little reminder that the repo is located on GitHub:
>>>> https://github.com/plaperdr/fp-central
>>>
>>> I have looked at this quickly, and the system to define the attribute
>>> tests seems nice. Is there an option at the end of the tests to download
>>> a file containing all the attributes collected?
>>>
>>
>> I don't know if this is what you are looking for but I've just added a
>> way to download a JSON file containing all the information in the
>> fingerprint. Here is the commit:
>> https://github.com/plaperdr/fp-central/commit/6f09ea5b88cf850f7be14af950e928327f0ded6c
>
> Thanks!
>
>>>> 1 - I have progressed faster than I expected in the last two weeks. Here
>>>> is everything that I have done:
>>>> - Storage of fingerprints in a MongoDB database
>>>> - Adding a small API to get statistics on stored variables
>>>> - Adding support of hashed variables for faster stats computation
>>>> - Adding collection of new attributes and support of HTTP headers
>>>> - Adding support of translation with the start of a French version
>>>>
>>>> 2 - I also started development of a page to tell if a user has an
>>>> "acceptable" fingerprint or not (I haven't pushed the code to GitHub
>>>> yet). So far, I'm verifying that the screen resolution is in the correct
>>>> bounds (i.e. not fullscreen) and that there are no plugins in the
>>>> browser. If anyone has any idea that I could implement to help users
>>>> have a less recognizable fingerprint, I'll be happy to add it. I have
>>>> also added steps to follow to help people better configure their browser.
>>>
>>> This "acceptable" fingerprint page is a good idea. However, unless I
>>> misunderstood your latest commit, it seems to be done as a separate
>>> thing from the attributes tests. Is there a reason for not using the
>>> collected attributes to check if the fingerprint is acceptable, rather
>>> than reimplementing the same tests separately?
>>
>> Right now, it is done as a separate thing because I'm only focusing on
>> two key attributes for the moment. It could be integrated in the main
>> collection page but I like the fact that it is separated. Also, I don't
>> want to deceive the user because lots of automatic things are running
>> and he or she doesn't have the control on it. That's why I put three
>> buttons on the main FP page to trigger different actions because I want
>> the user to understand and control what is happening. And so, one
>> additional reason behind the separate page is to avoid a mega page with
>> everything on it. Each page has its own purpose (and also for
>> maintainability, it is a plus).
>> I don't know if that makes sense but I can change how it is done if the
>> approach is not good.
>
> Ah, then it looks like the goal of this page is something different than
> what I was thinking. The feature I was thinking was telling if a browser
> looks exactly the same as the current version of Tor Browser, based on
> all the attributes that we can collect. But your page is something else
> that you do before running the fingerprint tests?
>
>>
>>>
>>> I think one way to do it would be to have a directory with a list
>>> of .json files containing attributes and their values, one file for each
>>> supported version/slider-setting/platform. And if the browser is
>>> matching one of the .json files, then it is considered good. The .json
>>> files would not include attributes such as screen.width or screen.heigh,
>>> but it could include other attributes indicating if they are rounded.
>>>
>
> After thinking again about this, rather than having a set of .json
> files, with one for each supported version/slider-setting/platform
> containing a lot of duplicate information (most attributes will be the
> same in all the cases), an other way to do it would be to list in the
> fingerprint/attributes/*.json files the expected values in the different
> cases.
>
> Adding something like this to the .json files:
>
> "expected_values" : {
>"variable_1&q
Re: [tor-dev] [GSOC16] Fingerprint Central - Status report n°2
On 06/23/2016 05:19 PM, Nicolas Vigier wrote: > On Fri, 17 Jun 2016, Pierre Laperdrix wrote: > >> Hi everyone, >> >> Here is my second status report for my GSOC project. >> A little reminder that the repo is located on GitHub: >> https://github.com/plaperdr/fp-central > > I have looked at this quickly, and the system to define the attribute > tests seems nice. Is there an option at the end of the tests to download > a file containing all the attributes collected? > I don't know if this is what you are looking for but I've just added a way to download a JSON file containing all the information in the fingerprint. Here is the commit: https://github.com/plaperdr/fp-central/commit/6f09ea5b88cf850f7be14af950e928327f0ded6c >> >> 1 - I have progressed faster than I expected in the last two weeks. Here >> is everything that I have done: >> - Storage of fingerprints in a MongoDB database >> - Adding a small API to get statistics on stored variables >> - Adding support of hashed variables for faster stats computation >> - Adding collection of new attributes and support of HTTP headers >> - Adding support of translation with the start of a French version >> >> 2 - I also started development of a page to tell if a user has an >> "acceptable" fingerprint or not (I haven't pushed the code to GitHub >> yet). So far, I'm verifying that the screen resolution is in the correct >> bounds (i.e. not fullscreen) and that there are no plugins in the >> browser. If anyone has any idea that I could implement to help users >> have a less recognizable fingerprint, I'll be happy to add it. I have >> also added steps to follow to help people better configure their browser. > > This "acceptable" fingerprint page is a good idea. However, unless I > misunderstood your latest commit, it seems to be done as a separate > thing from the attributes tests. Is there a reason for not using the > collected attributes to check if the fingerprint is acceptable, rather > than reimplementing the same tests separately? Right now, it is done as a separate thing because I'm only focusing on two key attributes for the moment. It could be integrated in the main collection page but I like the fact that it is separated. Also, I don't want to deceive the user because lots of automatic things are running and he or she doesn't have the control on it. That's why I put three buttons on the main FP page to trigger different actions because I want the user to understand and control what is happening. And so, one additional reason behind the separate page is to avoid a mega page with everything on it. Each page has its own purpose (and also for maintainability, it is a plus). I don't know if that makes sense but I can change how it is done if the approach is not good. > > I think one way to do it would be to have a directory with a list > of .json files containing attributes and their values, one file for each > supported version/slider-setting/platform. And if the browser is > matching one of the .json files, then it is considered good. The .json > files would not include attributes such as screen.width or screen.heigh, > but it could include other attributes indicating if they are rounded. > I haven't thought about it that way but that could be a very good approach. My main intuition behind the "acceptable" fingerprint change was to follow the design principles of Tor and see if they are respected. Now, I check the screen size and the absence of plugins. I think comparing a browser's fingerprint with a list of "normal"/"standard" ones could be good but in order for it to work, I think we need to get some data. One reason behind that would be what happens if the user's fingerprint does not match any "normal" ones. We have to put in place some boundaries for each attribute to say this value for this attribute is varying in a range we consider acceptable. Also, if the user's fingerprint deviates a little, can the user do anything about it? Can generic actions be applied or would it be more a case by case modifications? All in all, I put this on the roadmap because I definitely think something cool can be done here but without some data first, I don't think I can implement something really relevant. Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [GSOC16] Fingerprint Central - Status report n°2
Hey, On 06/20/2016 12:55 PM, Georg Koppen wrote: > Hi! > > Pierre Laperdrix: >> Hi everyone, >> >> Here is my second status report for my GSOC project. >> A little reminder that the repo is located on GitHub: >> https://github.com/plaperdr/fp-central >> >> 1 - I have progressed faster than I expected in the last two weeks. Here >> is everything that I have done: >> - Storage of fingerprints in a MongoDB database >> - Adding a small API to get statistics on stored variables >> - Adding support of hashed variables for faster stats computation >> - Adding collection of new attributes and support of HTTP headers >> - Adding support of translation with the start of a French version >> >> 2 - I also started development of a page to tell if a user has an >> "acceptable" fingerprint or not (I haven't pushed the code to GitHub >> yet). So far, I'm verifying that the screen resolution is in the correct >> bounds (i.e. not fullscreen) and that there are no plugins in the >> browser. If anyone has any idea that I could implement to help users >> have a less recognizable fingerprint, I'll be happy to add it. I have >> also added steps to follow to help people better configure their browser. >> >> 3 - I have tried to add a webpage where I can detect the level of the >> security slider. This way, I could give recommendations to users to >> maybe try a higher security level or it would be a way to know the >> distribution of Tor users on that feature. However, it has proven to be >> much harder than anticipated. >> * For "Medium-low", I verify that MathML is disabled. >> * For "High", I verify that there are either no JavaScript or no SVG >> elements. > > I think testing SVG is the safe bet here. I guess there is (still) a > bunch of users out there that is disabling JavaScript by default and > enabling it only when needed without bothering with the security slider. > In fact, if you could detect this then it might be a good thing for the > "How to improve your fingerprint" feature. > I think I'll do both: a message for users without JavaScript and the execution of the test suite for users with it. >> * I have troubles to detect the "Medium-High" level. I tried detecting >> the support of OpenType SVG fonts but it seems that I haven't found the >> right set of instructions to detect a difference. I'm using a font that >> I modified where I'm able to display a difference depending on the level >> of the security slider but I can't detect that difference through >> JavaScript. When SVG support is present, the displayed character is >> bigger than the HTML element but I can't detect that it is out of >> bounds. If anyone has any idea to detect the "Medium-high" level of the >> security slider, I'll be very happy about it. > > Loading a script with http:// should fail doing so with https://, > however, should work. This behavior is pretty distinctive for > Medium-High and would be my first idea for detecting this mode. > I tried this morning to go a little deeper with the SVGs but with no visible progress. In a way, it is a good news because they had security in mind when they designed that feature. One document which confirms the difficulties I encountered is this documentation: https://www.microsoft.com/typography/otspec/svg.htm In the security considerations section, they say that "script execution, external references and interactivity" is disabled (i.e. embedding JavaScript directly inside the SVG glyph is not possible) and the use of " and " is prohibited. These are exactly what I tried but with no success. In the end, I'll switch to the detection of HTTP blocking. Pierre > Georg > >> My goal in the next two weeks is to finish both the "acceptable >> fingerprint" page and the "slider" page. I also want to start working on >> a complete statistics page (outside of the main fingerprinting page). >> Hopefully, in two weeks, I'll have a version that is more complete and >> from there, I'll start digging into more complicated features like >> dealing with returning users. >> >> Have a great week-end, >> Pierre >> >> >> >> ___ >> tor-dev mailing list >> tor-dev@lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev >> > > > > > ___ > tor-dev mailing list > tor-dev@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [GSOC16] Fingerprint Central - Status report n°2
Hi everyone, Here is my second status report for my GSOC project. A little reminder that the repo is located on GitHub: https://github.com/plaperdr/fp-central 1 - I have progressed faster than I expected in the last two weeks. Here is everything that I have done: - Storage of fingerprints in a MongoDB database - Adding a small API to get statistics on stored variables - Adding support of hashed variables for faster stats computation - Adding collection of new attributes and support of HTTP headers - Adding support of translation with the start of a French version 2 - I also started development of a page to tell if a user has an "acceptable" fingerprint or not (I haven't pushed the code to GitHub yet). So far, I'm verifying that the screen resolution is in the correct bounds (i.e. not fullscreen) and that there are no plugins in the browser. If anyone has any idea that I could implement to help users have a less recognizable fingerprint, I'll be happy to add it. I have also added steps to follow to help people better configure their browser. 3 - I have tried to add a webpage where I can detect the level of the security slider. This way, I could give recommendations to users to maybe try a higher security level or it would be a way to know the distribution of Tor users on that feature. However, it has proven to be much harder than anticipated. * For "Medium-low", I verify that MathML is disabled. * For "High", I verify that there are either no JavaScript or no SVG elements. * I have troubles to detect the "Medium-High" level. I tried detecting the support of OpenType SVG fonts but it seems that I haven't found the right set of instructions to detect a difference. I'm using a font that I modified where I'm able to display a difference depending on the level of the security slider but I can't detect that difference through JavaScript. When SVG support is present, the displayed character is bigger than the HTML element but I can't detect that it is out of bounds. If anyone has any idea to detect the "Medium-high" level of the security slider, I'll be very happy about it. My goal in the next two weeks is to finish both the "acceptable fingerprint" page and the "slider" page. I also want to start working on a complete statistics page (outside of the main fingerprinting page). Hopefully, in two weeks, I'll have a version that is more complete and from there, I'll start digging into more complicated features like dealing with returning users. Have a great week-end, Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [GSOC16] Fingerprint Central - Status report n°1
Hi everyone, Here is my first status report for my GSOC project. 1 - Here is the link to the GitHub repo of the project: https://github.com/plaperdr/fp-central I have chosen the name "Fingerprint Central" for the project. I may change it in the future but I'm pretty satisfied with it for the moment. If someone has a great suggestion to replace it, don't hesitate to send it to me! I have also included on the main page a checklist for anyone to follow the progression of development. 2 - I took the time to build strong foundations for my project to support both current and future features. * To support addition and removal of fingerprinting tests, I have added a system that need two components: - a definition file where one gives all the information related to the test - the source code needed to execute the test With this system, I can manage tests dynamically and I can give information to the users on what is currently being run in the test suite. * To support different types of fingerprints and browsers, I plan to integrate a system to tag fingerprints. Instead of storing fingerprints in specific databases, I found that a tag system is a flexible way to deal with any changes and when statistics will be implemented, it will be easier to get numbers on relevant categories thanks to tags. Everything is explained in greater details in the "fingerprint" folder of the repo in the Readme files. I'd love to have feedback on it to know if I'm going in the right direction and if my project seems well structured. 3 - Everything that I'm planning to use is present in the Debian repositories except for the PyMongo support of Flask. This means that it may possible to use the Tor infrastructure to host the website. I'll see when I get closer to a first stable version what could be possible on that front. What's next? My focus for the next two weeks is mainly to add new tests (both standard and specific to the Tor browser) and to add support of MongoDB. I hope to have a page that collects fingerprints, stores them in Mongo and calculate statistics on them. That's it for me this week! Have a great week-end everyone, Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [GSoC16] A website to improve Tor fingerprinting defenses
That's a very good question! The website is first and foremost aimed to improve the Tor browser (even though it will open its doors to every browsers after that). This means that, at first, we would only collect fingerprints coming from Tor browsers and not from users redirecting their network traffic through Tor. I plan on having statistics for different versions of the Tor browser so that we can follow evolution or potential regressions. Then, from the developers side, the website will be built in a way that tests can be added and removed really easily. Contrary to Panotpiclick or AmIUnique where the set of collected attributes is fixed, I'll try to make it as easy as possible to add a test to the website with a link to a ticket in the Tor bug tracker and a way to collect statistics for this specific test. I want to emphasize on that point because common attributes like the user-agent or the size of the screen that are collected in browser fingerprints are already covered by the Tor browser. However, when a new fingerprinting technique is discovered or when a new browser API is launched, it is really hard to get an insight into how much identifiable information is in there without running a test and getting concrete data. Finally, from the user side, I want to give the tools to users to understand what each collected attribute is and what to do in case his or her browser fingerprint is far from an acceptable one. In the end, the main mechanisms are very similar to Panopticlick (collection and statistics) but the set of added features aimed primarily at the Tor browser and the Tor community is what will set this website apart from others. I hope my explanations are clear enough. If you have additional questions, I'll be happy to answer them. Pierre On 04/24/2016 01:01 PM, Virgil Griffith wrote: > It's unclear to me how this would be different than standard > panopticlick with >50% of the users using TBB. But those not using TBB > with had browser statistics like the rest of the web (for example, all > of the tor2web traffic). > > -V > signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] [GSoC16] A website to improve Tor fingerprinting defenses
Hi Tor Community! My name is Pierre and I'm a second year PhD student from France working on browser fingerprinting. I'm really fortunate that my proposal for this year Google Summer of Code has been been selected. My goal for this summer is to set up a website similar to Panopticlick or AmIUnique that will collect data to improve Tor fingerprinting defenses. The collected data will be used to detect if there are differences between browsers that could ultimately lead to a user's identification. With that knowledge, developers will be able to patch the Tor browser to prevent leak of identifiable information and reinforce the anonymity of Tor users. I also plan to add details on browser fingerprinting for users who are not familiar with the subject so that they may learn about its mechanisms and the potential dangers linked to it. I'll also try to implement a page where Tor users can see how far they are from an "acceptable" fingerprint so that they may tweak their browser in order to have a fingerprint that is shared by many more users. I'll officially start coding at the end of May but in the mean time, I'll familiarize myself with the Django framework that I plan to use and I'll lock down the exact set of features that I'll include in the very first version of the project. My primary mentor is Georg and my secondary mentors are Gunes and Nicolas. I'll post bi-weekly reports to the tor-dev mailing list and to my blog to inform everyone of my progress. If you have any questions about the project, don't hesitate to ask me. The name of the project has not been found yet so feel free to send me any suggestions that you may have. You can reach me here or on IRC where my handle is SuperOctopus. Cheers! Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] GSoC'16 proposal: the Torprinter project (a Panopticlick-like website)
Hi Lunar, Thanks for the valuable feedback! It would be great to have the features that you cite into the website. For the first version, my point of view is to mainly focus on the added value for developers which is to add/remove tests easily and get the relevant data as easily as possible for them. With that, they can make decisions on what to do next inside the Tor browser. For subsequent versions, focusing on users could be really interesting if the rest proves to be solid and stable. What you described in your answer would be a more advanced version of the "How far are you from an acceptable fingerprint?" feature that I plan to integrate from the get-go. If the website detects that the user has not a recommended configuration, it could give different links with information and steps on how to fix that so that a non-tech person can understand what they will do and what they will modify. Suggestions to play with the security slider could be a great addition. I'll be honest, I don't know how much of what you propose could be done before summer's end. My timeline is a little bit rough but even if it is not done by then, I can still add it after the GSoC period ends. For the internationalization, the framework that I plan to use (either Play or Django) supports it through templating. This means that anyone can contribute to the translation without writing a line of HTML. The main file will be in English and I'll probably do the one in French at the same time. One contributor who wants to help will just have to take the English file and translate each line without having to find scattered hardcoded strings through different HTML files. Pierre On 03/22/2016 11:21 AM, Lunar wrote: > Hi Pierre! > > Thanks for this valuable proposal. :) Just a quick comment frome someone > who has experienced supporting Tor users. > > Georg Koppen: >>> - How new tests should be added: A pull request? A form where >>> submissions are reviewed by admins? A link to the Tor tracker? >> >> From a Tor perspective opening a ticket and posting the test there or >> ideally having a link to a test in the ticket that is fixing the >> fingerprinting vector seems like the preferred solution. I'd like to >> avoid the situation where tests get added to the system and we don't >> know about that dealing with users that are scared because of the new >> results. So, yes, some review should be involved here. > > It would be great if you could also include ways to guide users in > understanding the test results. From the top of my mind, it would be > good if the application would have a way to know which Tor Browser > version is being run. Then together with results, it would be good if > users would get an answer to the following questions: > > * Is this the expected result? > * If not, is there any remediation available? At which cost? >- This could be prompting users to upgrade to a new version. > Ideally include support for known tools which bundle the > Tor Browser, so the message could be “Upgrade to Tails 2.2” > for Tails users. >- Tell them to fiddle with the security slider, with a warning > that they will loose some features. > * If there's no immediate remediation available, can they do anything? >- Is the issue known at all? Can we then assist them to report the > problem in a meaningful manner? Or point them at the existing > ticket—with a warning that it's going to be tech+english. >- Should they take extra precaution? Link to some documentation. >- Do we need to collect more data? Let's guide them how. >- Maybe it's a good opportunity to ask them for some money so we can > hire more browser developers? > > I'm pretty sure the UX team could give input on good wordings and > layout. And probably on the whole thing. :) > > Have you consider any internationalization? > > If not all of this can be implemented over the summer, just keeping it > in mind in the design stage might help to add the required features > later. > > > > ___ > tor-dev mailing list > tor-dev@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] GSoC'16 proposal: the Torprinter project (a Panopticlick-like website)
Hi, Thanks for the rest of the feedback and for taking the time to read everything! I'll update my proposal accordingly. I have added some small comments below. Pierre On 03/21/2016 04:23 PM, Georg Koppen wrote: > Hi, > > here comes feedback to the remaining part of the proposal. > > Pierre Laperdrix: > > [snip] > >> Code sample >> In 2014, I developed the entire AmIUnique.org website from scratch. Its >> aim is to collect fingerprints to study the current diversity of >> fingerprints on the Internet while providing full details to users on >> this subject. It was the first time that I built a complete website from >> the design phase to its deployment online. >> One of the first challenge that I encountered was to build a script that >> would not only use state-of-the-art techniques but that could simply >> work on the widest variety of browsers. Testing a script for a recent >> version of a major browser like Chrome and Firefox is an easy task since >> they implement the latest HTML and JavaScript technologies but making >> sure that the script runs correctly on older browsers like Internet >> Explorer is another story. Juggling with a dozen different virtual >> machines was necessary to obtain a bug-free and stable version of the >> script. A small beta-test was required to make sure that everything was >> good to go for what is now the foundations of the AmIUnique website. The >> totality of the source code for AmIUnique and my other projects can be >> found on GitHub. >> A second challenge that I faced was to deal with the increasing load of >> users so that the server could return personalized statistics to >> visitors in a timely manner (less than 2/3s). By having a separate >> entity that updates statistics in real time on top of the database, I >> managed to drastically reduce the server load. With the number of Tor >> users around the world, the website needs from the get go to handle a >> high load of visitors and statistics computation and my previous >> experience on that specific task will prove useful. >> >> For the very first version of Torprinter, I plan on testing well-known >> and widespread fingerprinting techniques to make sure that there is no >> variation among Tor users. These include HTTP headers and known >> JavaScript objects. There should be no need for any Flash attributes >> since plugins are not present in the Tor browser (thus removing complex >> code in charge of correctly loading the Flash object). > > We might think about that a bit. We have a bunch of users it seems that > are still trying to go through all the hassle and are getting Flash > going in Tor Browser. It might be enough to detect them by just > enumerating available plugins. > >> For this proposal, I have also developed a special page with 7 different >> tests that are mainly targeted at the Tor browser to give an idea of >> what tests can be included that are more suited to the Tor users. >> Tests n°5, n°6 and n°7 are broader and also concerns the Firefox browser. >> You can found a working version of the script on a special webpage (need >> to scroll to make the results appear): >> https://plaperdr.github.io/torScript.html >> The script can be found here: https://plaperdr.github.io/assets/tor/tor.js >> >> Test n°1 >> Test the size of the current window - As reported by ticket n°14098 >> https://trac.torproject.org/projects/tor/ticket/14098 > > FWIW: that test is not working correctly. We cap the width and the > height at 1000px and round the window to a multiple of 200pxx100px. > I have updated the test to reflect this, I didn't have all the numbers right. It should be fixed now. I was a little bit surprised to find out that the Tor browser gives the exact size of the window with pixel precision (even if I completely understand why). On Firefox, even in windowed mode, the browser reports the full size of the screen. With a test like this, we could see if the rounding works correctly for all users and find out the percentage of users who resize their windows. >> Test n°2 >> Test the support of emoji - As reported by ticket n°18172 >> https://trac.torproject.org/projects/tor/ticket/18172 >> Test n°3 >> Analysis of the "scroll" behavior of the window - As investiagted by >> http://jcarlosnorte.com/security/2016/03/06/advanced-tor-browser-fingerprinting.html >> Test n°4 >> Test the size of current fallback font by using the canvas API to render >> some text (no need for user permission like canvas extraction) - Custom test >> Test n°5 >> Test the difference between OS on the maximum font size
Re: [tor-dev] GSoC'16 proposal: the Torprinter project (a Panopticlick-like website)
On 03/17/2016 06:02 PM, gunes acar wrote: > Hi Pierre, > > On 2016-03-16 11:58, Pierre Laperdrix wrote: >> Hi Gunes, >> >> Thanks a lot for the feedback! >> >> On 03/16/2016 03:30 PM, gunes acar wrote: >>> Hi Pierre, >>> >>> Thanks for the very well thought proposal! >>> >>> I'm curious about your ideas on the "returning device problem." EFF's >>> Panopticlick and AmIUnique.org use a combination of cookies and IP >>> address to recognize returning users - so that their fingerprints are >>> not "double-counted." >>> >>> Since these signals will not available anymore (unless the user opt-ins >>> to retain the cookie), I wonder what'd be your ideas to address this issue. >> >> This one is a really interesting question but a tricky one because we >> can't really rely on the cookies+IP combination with the Tor browser. >> My answer here is simple: it all depends on the goal we set for the website. >> > > I think the original goals were to understand the fingerprint > distribution and to measure the effect of introduced defenses (e.g. > by measuring the uniqueness/entropy before vs. after the defense). > > I agree with you that guaranteeing no double-counting may not be > possible, especially if we consider a determined attacker. A more > realistic goal could be to filter out double-submissions from benign users. > > Let me point out an idea raised in the previous discussions: > One option to enroll users for the tests was to have a link on the > about:tor page similar to "Test Tor Network Settings" link. The > fingerprint link could also include (e.g. as URL parameters) TB version, > localization and OS type to establish ground truth for the tests. > > I wonder, if the same link can be used to signal a fresh fingerprint > submission to the server. This may require keeping a boolean state (!) > on the client side which may mean "already submitted a fingerprint with > the current TB version." This state can be kept in TorButton's storage, > away from the reach of non-chrome scripts. The fingerprinting site could > then use this parameter to distinguish between fresh and recurrent > submissions. > > An alternative can be to present a fresh submission link on the > "changelog" tab, which is guaranteed to be shown once after each update > - right when we want to collect a new test from users. > > Perhaps we should be cautious about keeping any client-side state, and > be clear about the limitations of these approaches. But I feel like the > way we enroll the users can be used to prevent pollution, at least by > the well-behaving Tor users. Just wanted point out this line of thought, > no doubt you can come up with better alternatives. > > Best, > Gunes > I was so focused on basic browser mechanisms and what we could do with fingerprinting that I forgot that something can be done inside the browser. Even though storing a boolean won't totally fix the problem of someone polluting the database, if we want to analyze the distribution of fingerprints, this may be a good step forward for legitimate users who want to contribute. Then comes the question of the "recurrent" or "not fresh" submissions. If we only store the first "fresh" fingerprint, we may miss subsequent fingerprints that may be more interesting for us. So, one solution would be: - Storage of all fresh fingerprints. This would give an idea of the fingerprint distribution. - Storage of recurrent fingerprints while removing duplicates. This would give all the possible values for a specific attribute and the same device could contribute several times. I don't know if this seems a good approach or if this is too complicated. It is a hard balance to keep with privacy on one side and relevant data on the other. If we really have to identify returning users, we need some kind of ID somewhere but even that could be modified. Also, to have a ground truth given directly by the browser seems to be really good. At first, I thought about detecting the browser version through fingerprinting but when I looked at some of the changelog, some updates may not be detectable through fingerprinting (example with minor version 5.5.2 of the Tor browser). Pierre >> Do we want to learn how many different values there are for a specific >> test so that we can reduce diversity among Tor users? In that case, the >> site would not store duplicated fingerprints or it could be >> finer-grained and not store duplicated values for each test. >> >> Or do we want to go further and know the actual distribution of values >> among Tor users so that it ma
Re: [tor-dev] GSoC'16 proposal: the Torprinter project (a Panopticlick-like website)
Hi Gunes, Thanks a lot for the feedback! On 03/16/2016 03:30 PM, gunes acar wrote: > Hi Pierre, > > Thanks for the very well thought proposal! > > I'm curious about your ideas on the "returning device problem." EFF's > Panopticlick and AmIUnique.org use a combination of cookies and IP > address to recognize returning users - so that their fingerprints are > not "double-counted." > > Since these signals will not available anymore (unless the user opt-ins > to retain the cookie), I wonder what'd be your ideas to address this issue. This one is a really interesting question but a tricky one because we can't really rely on the cookies+IP combination with the Tor browser. My answer here is simple: it all depends on the goal we set for the website. Do we want to learn how many different values there are for a specific test so that we can reduce diversity among Tor users? In that case, the site would not store duplicated fingerprints or it could be finer-grained and not store duplicated values for each test. Or do we want to go further and know the actual distribution of values among Tor users so that it may guide the development of a potential defense? In this case, the site must identify returning users and it is a lot harder to do here. The only method that comes to mind and that would be accurate enough to work in this situation would be to put the test suite behind some kind of registration system. The problem is that a mandatory registration goes in the complete opposite direction of what Tor is about and it would greatly limit the number of participating users (or even render the site useless before it is even launched). A solution in the middle would be not to store duplicated fingerprints but I really don't know how much it would affect the statistics in the long run. Would it be marginal and affect like 2/3/4% of collected fingerprints or would it be a lot more and go above 20% or else? Finally, I thought about using additional means of identification like canvas fingerprinting but I don't think there would be enough diversity here to identify a browser. > > Please find other responses below. > > Best, > Gunes > > On 2016-03-15 04:46, Pierre Laperdrix wrote: >> Hi Tor Community, >> >> - How closed/private/transparent should the website be about its tests >> and the results? Should every tests be clearly indicated on the webpage >> with their own description? or should some tests stay hidden to prevent >> spreading usable tests to fingerprint Tor users? > > I think the site should be transparent about the tests it runs. Perhaps > the majority of the fingerprinting tests/code will run on the client > side and can be easily captured by anyone with necessary skills (even if > you obfuscate them). > You are right on that. It makes sense to be transparent since obfuscated JS code can be deciphered by someone with the necessary skills. Also, if tests are hidden, most Tor users would rightfully be wary on what is exactly being executed in their browser and they would simply not take the test. In that case, the impact of the website would be greatly limited. Being transparent really seems to be the right way to go here. >> - Should a statistics page exist? Should we give a read access to the >> database to every user (like in the form of a REST API or other solutions)? > > I think aggregate statistics should be available publicly but exposing > individual fingerprints publicly may not be necessary. > Like you said, aggregate statistics seem to be the best solution here. Then, I'm wondering if it would be possible to offer the complete list of values for each attribute separately from others. Then, my concern is how easy it would be to correlate separate attributes to recreate fingerprints, even partial ones. Regards, Pierre signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] GSoC'16 proposal: the Torprinter project (a Panopticlick-like website)
Hi Georg, Thanks for the feedback! On 03/17/2016 10:06 PM, Georg Koppen wrote: > Hi Pierre, > > thanks for this proposal. Gunes has already raised some good points and > I won't repeat them here. This is part one of my feedback as I need a > bit more time to think about the code example section. > > Pierre Laperdrix: >> Hi Tor Community, >> . >> >> Website features >> The main feature of the website is to collect a set of fingerprintable >> attributes on the client and calculate the distribution of values for >> each attribute like Panopticlick or AmIUnique. The set of tests would >> not only include known fingerprinting techniques but also ones developed >> specifically for the Tor browser. >> The second main feature of the website would be for Tor users to check >> how close their current fingerprint is from the ideal unique fingerprint >> that most users should share. A list of actions should be added to help >> users configure their browser to reach this ideal fingerprint. > > We might want to think about that ideal fingerprint idea a bit. I think > there is no such a thing even for Tor Browser users as we are e.g. > rounding the content window size to a multiple of 200x100 for each user. > Thus, we have at least one fingerprintable attribute where we say "you > are good if you have one out of a bunch of possible values". The same > holds for our security slider which basically partitions the Tor Browser > users. We could revisit these design decisions and I am especially > interested in getting data that is backing/not backing our decisions > regarding them. Nevertheless, I assume we won't always be able to put > users into just one bucket per attribute due to usability issues. And > this in turn makes the idea to help users configure their browser not > easier. > You are right on that. I'll switch to the idea of an "ideal" fingerprint to an "acceptable" one in my proposal. The idea to partition the dataset into categories from the security slider can be really interesting and we could try to play with some JS benchmarking since some JS optimizations are removed on higher security levels. Then, we could try to detect the security level either through fingerprinting or following the suggestion of Gunes, the browser could directly add the level as a URL parameter for the ground truth. >> The third main feature would be an API for automated tests as detailed >> by this page : >> https://people.torproject.org/~boklm/automation/tor-automation-proposals.html#helper-fingerprint >> . This would enable automatic verification of Tor protection features >> with regard to fingerprinting. When a new version is released, the >> output of specific tests will be verified to check for any >> evolution/changes/regressions from previous versions. >> The fourth main feature I'd like to include is a complete stats page >> where the user can go through every attribute and filter by OS, browser >> version and more. >> The inclusion of additional features that go beyond the core >> functionnalities of the site should be driven by the needs of the >> developers and the Tor community. >> Still, a lot of open questions remain that should be addressed during >> the bonding period to define precisely how each of these features should >> ultimately work. >> Some of these open questions include: >> - How closed/private/transparent should the website be about its tests >> and the results? Should every tests be clearly indicated on the webpage >> with their own description? or should some tests stay hidden to prevent >> spreading usable tests to fingerprint Tor users? >> - Should a statistics page exist? Should we give a read access to the >> database to every user (like in the form of a REST API or other solutions)? >> - Where the data should be stored? How long should the data be kept? If >> tests are performed by versions, should the data from an old TBB version >> be removed? Should the data be kept a week, a month or more? > > I am not sure about how long the data should be kept. It probably > depends on what kind of data we are talking about (e.g. aggregate or > not). I think, though, that data we collected with Tor Browser A should > not get deleted just because Tor Browser A+1 got released. I think, in > fact, we might want to keep that data especially if we want to give > users a guide about how to get a "better" fingerprint. But even if not > we might want to have this data to measure e.g. whether a fix for a > particular fingerprinting vector had an impact and if so, which one. > It makes sense to keep data from previous
[tor-dev] GSoC'16 proposal: the Torprinter project (a Panopticlick-like website)
Hi Tor Community, My name is Pierre and I'm really interested in participating in a GSoC project this year with the Tor organization. Since I've been working on browser fingerprinting for the past two years, I'd love to build a Panopticlick-like website to improve the fingerprinting defenses of the Tor browser. I've included below my proposal in case anyone has ideas or suggestions, especially on the technical section or on some of the open questions that I have. (It should be noted that the Torprinter name is subject to change). ** Summary - The Torprinter project: a browser fingerprinting website to improve Tor fingerprinting defenses The capabilities of browser fingerprinting as a tool to track users online has been demonstrated by Panopticlick and other research papers since 2010. The Tor community is fully aware of the problem and the Tor browser has been modified to follow the "one fingerprint for all" approach. Spoofing HTTP headers, removing plugins, including bundled fonts, preventing canvas image extraction: these are a few examples of the progress made by Tor developers to protect their users against such threat. However, due to the constant evolution of the web and its underlying technologies, it has become a true challenge to always stay ahead of the latest fingerprinting techniques. I'm deeply interested in privacy and I've been studying browser fingerprinting for the past 2 years. I've launched 18 months ago the AmIUnique.org website to investigate the latest fingerprinting techniques. Collecting data on thousands of devices is one of the keys to understand and counter the fingerprinting problem. For this Google Summer of Code project, I propose to develop the Torprinter website that will run a fingerprinting test suite and collect data from Tor browsers to help developers design and test new defenses against browser fingerprinting. The website will be similar to AmIUnique or Panopticlick for users where they will get a complete summary with statistics after the test suite has been executed. It can be used to test new fingerprinting protection as well as making sure that fingerprinting-related bugs were correctly fixed with specific regression tests. The expected long-term impact of this project is to reduce the differences between Tor users and reinforce their privacy and anonymity online. In a second step, the website could open its doors to more browsers so that it could become a platform where vendors can implement significant changes in their browsers with regards to privacy and see the impact first-hand on the website. With the strong expertise I have acquired on the fingerprinting subject and the experience I have gained by developing the AmIUnique website, I believe I'm fully qualified to see such a project through to completion. Website features The main feature of the website is to collect a set of fingerprintable attributes on the client and calculate the distribution of values for each attribute like Panopticlick or AmIUnique. The set of tests would not only include known fingerprinting techniques but also ones developed specifically for the Tor browser. The second main feature of the website would be for Tor users to check how close their current fingerprint is from the ideal unique fingerprint that most users should share. A list of actions should be added to help users configure their browser to reach this ideal fingerprint. The third main feature would be an API for automated tests as detailed by this page : https://people.torproject.org/~boklm/automation/tor-automation-proposals.html#helper-fingerprint . This would enable automatic verification of Tor protection features with regard to fingerprinting. When a new version is released, the output of specific tests will be verified to check for any evolution/changes/regressions from previous versions. The fourth main feature I'd like to include is a complete stats page where the user can go through every attribute and filter by OS, browser version and more. The inclusion of additional features that go beyond the core functionnalities of the site should be driven by the needs of the developers and the Tor community. Still, a lot of open questions remain that should be addressed during the bonding period to define precisely how each of these features should ultimately work. Some of these open questions include: - How closed/private/transparent should the website be about its tests and the results? Should every tests be clearly indicated on the webpage with their own description? or should some tests stay hidden to prevent spreading usable tests to fingerprint Tor users? - Should a statistics page exist? Should we give a read access to the database to every user (like in the form of a REST API or other solutions)? - Where the data should be stored? How long should the data be kept? If tests are performed by versions, should the data from an old TBB version be removed? Should
