On Mon, Nov 2, 2009 at 11:13 AM, Schuyler Erle <[email protected]> wrote:
> Thanks, Paul. I think these are great ideas for getting an OAM network > to scale robustly. However, I debate your consideration that OAM has > foundered thus far on the storage problem; I believe that building the > community is more significant and that the technical solutions will > follow. Witness the fits and starts in the growth of OSM for example. I'm sure others have a better idea of the hitches OAM has hit to date. It sounded like client tools (getting data *in*) were a big deal this time. I gravitated to storage/access as a problem because (a) it's fun and (b) it's the obvious next bottleneck once people can easily get their data in. > I see two hitches with your technical proposal. Number one is that > traversing a distributed hash table introduces significant latency. We'd > have to test the hell out of a DHT solution to ensure performance. Indeed. I came to DHT more or less by default because I couldn't find a library solution to a more centralized napster-like solution, and I don't trust myself to create core computer-science solutions to general problems. I do think something more like Napster would work better for this problem, since we can probably guarantee uptime on a small central network of metadata systems to service the leaf nodes, in a way we couldn't guarantee uptime on a complete data hosting system. > The other hitch is that some people care about verifying a tile's > provenance. I think that this implies a set of core servers that do the > actual proxying to the source WMS servers, and then crypto-sign the > images themselves (by adding the metadata and signature to an image > metadata field or simply appending the metadata/signature combo to the > end of the image, which works with JPEG at least). Is the issue one of guaranteeing provenance (this image was definitively produced by this organization) or guaranteeing the existence of basic metadata (this image was captured on this date by this sensor, and uploaded on this data by this user)? In either case, I think that's orthogonal to storage/access and a function of the data upload/integration process (which is to say it's a non-trivial, but separable, concern). > I think that improving the likelihood of cache hits on client requests > by using DNS to take advantage of geographic locality is a really great > idea, and I hope to see this implemented in whatever topology gets > worked out. I'm quite enthusiastic about this, though trying to remain cognizant of the fact that people in Argentina do also view Australia sometimes, so that use case also has to have acceptable performance characteristics. P. _______________________________________________ talk mailing list [email protected] http://openaerialmap.org/mailman/listinfo/talk_openaerialmap.org
