I want to help, how can I add info to the maps? thanks
Santiago On Fri, Nov 20, 2015 at 11:11 AM, Hanno Schlichting < [email protected]> wrote: > Keeping this on the public list - warning: it's a bit of a brain-dump / > rant :) > > I have to admit that the main reason I took a look at the map was from a > performance / scaling perspective. Abusing the number of data points per > grid to indicate something, was the only user visible change I could > implement in a short time. Abusing point density to indicate recency may > not be the best choice. > > Today we have about 400 million grids (100x100 meter areas) as input into > the mapmaking. The way we generated the map took close to 10 hours each > day. I was able to speed it up by parallelizing more of it, but it still > takes 6 hours each night. > > If we were to use smaller grids, we'd increase the number of grids and the > processing time. If we'd use 30x30 meter grids, we'd probably end up with > roughly 5 times as many grids. You can fit 11 smaller grids into one big > grid (10000 sqm ~= 11 x 900 sqm), but from experience we know that only > about half of the smaller grids would actually have data points in them (in > cities most of them have data, on country roads most of them are empty). > > This means we'd end up with 2 billion grids and the map would take more > than 24 hours to generate, which is rather silly. > > We are now at a point, where it makes more sense to build maps showing the > locations of all the cell and wifi networks. WiFi networks also tend to be > colocated, so our 440 million WiFi networks only result in about 220 > million locations rounded to 10 meter precision. The 18 million cells > almost don't matter from a scaling perspective. > > 220 million is already lower than the number of grids we have today, with > a much better precision. For cell networks we'd really want different maps > to show different carriers and radio standards, so one could look at only > AT&T's LTE networks or only Deutsche Telekom's GSM networks. > > Those kinds of maps would actually tell you whether or not someone > collected WiFi networks or cell networks from your carrier in a specific > place. For the purpose of "should I visit this street" these maps would be > more useful. The datamap we have today only says that someone went to a > place and collected at least one data point. But it could have been a > single cell entry for a GSM network of a different carrier and no WiFi data > at all. > > Unfortunately we don't know how to build these maps at the scale we have. > All the standard mapping tools will only be performant until about 10000 > points on desktop browsers and more like 100 to 1000 points on low-cost > Android phones. Since the target users would be primarily Mozilla Stumbler > users, we have to aim for low-cost Android phones with screen sizes of > ~300x500 pixels as the main audience. > > The datamap we have is a nice eye-candy and looks cool. It shows the > general global momentum and reach of the project. But there is no way to > tweak it to really do the "should I walk over / drive this street" use-case > justice. We only keep rendering it at fairly high zoom levels, because we > can and we don't have anything better. > > If anyone wants to help and figure out how to build maps of cell or wifi > networks, we need you :) > > Hanno > > > On 17.11.2015, at 01:36, Felix Baumann <[email protected]> wrote: > > > > Hi Hanno, > > > > awesome that you are working on the map again! > > > > It's nice that you can kinda distinguish older and newer entries now. > > But newer entries now simply look like their coverage is denser now. > > (I'm not sure if I like it, but it's a way to distinguish routes and it > works) > > > > I know you are tired of the discussion about privacy, > > but I have another suggestion, now that I see the new map style: > > But what do you think about making older entries more precise than newer > ones? > > > > This would help a lot in improving the map's precision and therefore > Ichnaea > > might get a lot more precise as well, because the stumblers won't have to > > remember which streets they already stumbled and which not and can > therefore > > stumble streets which were previously unstumbled. > > > > The home of persons stumbling would still be obfuscated, because the map > > would be blurred there, so it shouldn't be much of a problem > privacy-wise. > > Only older reports were stumblers haven't been for months would be > accurate. > > > > But at least other positions where people weren't stumbling for months > would > > be precise so that the coverage could be optimized. > > > > This would improve the following issue: > > In cities/villages, stumbling one street will mark adjacent streets as > well, > > even though you didn't stumble many of the wifis in the other street. > (Often if > > you look at the map the dots aren't placed on the corresponding streets > where > > they were recorded but next to them -> between two streets) > > > > IMO the map should still be a bit obfuscated (whether a position has been > > stumbled lots of times or just once shouldn't be visible), but the > position of > > the older reports should be precise. And the dots should be small(er). > > > > This could be achieved by taking all records out of a square of 30m x > 30m (roughly > > 30ms diameter, a circle would be nicer but it's harder to calculate) and > placing > > a dot at their average position (not the middle). So if most of them are > at 5x10 > > then dot should be there instead of 15x15. > > > > This would hide lots of visits but shows an accurate position. > > > > This could be adjusted: if there are more than 100(0) visits it could be > an accuracy > > of 20x20 or 10x10. This would result in a lot more dots but shouldn't > affect privacy, > > because if the number of visits is high enough, then it gets unlikely, > that it is just > > somebody's home. (Rather a train station or a sight) > > > > One reason why it might be time to rethink the privacy strategy behind > the map: > > MLS has grown a lot since the decision has been made, that the map should > > be obfuscated. It's not that easy anymore to see someone's home on the > map. > > And there are many people that participate in it now. > > > > > > Do you think this idea preserves the stumblers' privacy enough? > > > > > > Regards, > > Felix > > _______________________________________________ > dev-geolocation mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-geolocation > _______________________________________________ dev-geolocation mailing list [email protected] https://lists.mozilla.org/listinfo/dev-geolocation
