2009/8/11 MilesTogoe <[email protected]>: > On 08/11/2009 10:27 AM, Jeffrey Johnson wrote: >> I concur. The largest obstacle here in infrastructure. I have recently >> completed processing and tiling 10s of TB of imagery for DoD and just moving >> that amount of data around is costly and complicated, let alone trying to >> serve it. >> >> Amazon has indicated they would be willing to help us with this kind of >> project, but I agree that a distributed registry approach makes the most >> sense in the short/medium term, and think we should focus our efforts there. > > I also think it is a good idea to distribute the server data regionally to > spread the storage requirements.
Agreed, and I think a "crowdsourced storage" approach is worth considering. Lots of people have apache running on their desktops with a not too fast internet connection but with a 1.0 or 1.5TB desktop-class disk. They'd just need to run a server side script that could also accept tile uploads from some kind of distpatcher. On the client side the dispatching could be done inside the browser, by computing some simple hash in javascript, so as to avoid asking a central server for the address of every tile. The tiles should not be mapped to servers (people's desktops) geographically but instead using a hash function so that e.g. when someone browses Germany imagery all his requests don't go to a single computer, and instead every desktop hosts a tiny bit of Germany and a tiny bit of Massachuetts. If there's enough hosts then there could be 10 or 20 computers hosting copies of each tile and the central server serving the website would just send the mapping of hash values to a subset of mirrors, which would be rotated. Cheers _______________________________________________ talk mailing list [email protected] http://openaerialmap.org/mailman/listinfo/talk_openaerialmap.org
