Thanks for the informative response. I understand. I'm sorry for this mess too. I can't help but feel responsible as it was my data that resulted in this. I'll be more careful in the future.
— Sent from Mailbox On Mon, Mar 2, 2015 at 8:34 PM, Hanno Schlichting <[email protected]> wrote: > Hi. > On 28.02.2015, at 14:50 , RK Aranas <[email protected]> wrote: >> I was able to identify the source of errors for these observations. Turns >> out, those were readings taken when my phone's clock was messed up. Date on >> my phone during that time was sometime in 1970. Those resulted in wacky >> computed GPS positions. What do I need to do to help you remove the >> erroneous data? Right now, it's making it rather hard to see which spots on >> the map are not yet covered in the Metro Manila area. Thanks! > Sorry, your file has been sitting in my email inbox for a good long while > now. I haven't forgotten about it, it's just difficult to correct this. > The problem is that all we have for the map data is a table consisting of: > auto-inc id, rounded lat, rounded lon, creation date > 1, 27500, 95000, "2014-08-03" > 2, 27501, 95000, "2014-08-04" > ... > Where the lat/lon fields in that example represent 27.500 / 95.000. We only > have the creation date since about July 2014 and no modification date. For > each of those coordinates we randomly generate five points into it, to fill > out the tile a bit more. That randomness depends on a pseudo-random generator > which relies on the ordering of the auto-inc ids, to generate the exact same > pseudo-random sequence with each run. If it where truly random, we'd > regenerate and change each tile with each run, breaking all caching. > And we don't have online access to any of the old observation data, just the > data from the last day. The rest is all dumped to the filesystem (or rather > Amazon S3) in large chunks of 100k records. We can only access that data by > the months it was uploaded in. > Add to all this, that the map generation process doesn't currently support > deleting map tiles. The way it's wired up, it only uploads new data to Amazon > S3 / Cloudfront when the data in a tile has changed. But it doesn't issue a > delete when there's no more data in a tile. > This is all excuses, but there's a good number of things here which makes > correcting the map currently difficult and time consuming, so it always ends > up at the end of my backlog. > Sorry, I'll try to work on the map generation in the next weeks and see if I > can correct this. I'll have to work on the map generation to speed it up as > well, since it is now taking close to two hours to run each night. > Hanno > _______________________________________________ > dev-geolocation mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-geolocation _______________________________________________ dev-geolocation mailing list [email protected] https://lists.mozilla.org/listinfo/dev-geolocation
