On Nov 16, 2010, at 12:23 AM, Benjamin Scherrey wrote: > Just recently stumbled upon MapNik
Ben, welcome to Mapnik. > and would like to use it in our upcoming project as it appears to fit the > exact technologies we're looking for. We are a python/django/postgis/C++ shop > so it was great to see all these things seemingly brought together for us. Yes, several of the lead developers of Mapnik are also longtime django users. > There's so much stuff that MapNik connects to, however, > that I'm not sure how much of the capabilities of the underlying libraries > are well supported or easily accessed. > Would appreciate if anyone could give me any pointers in MapNik "best > practices" or, even better, sample code out there that is doing something > very similar to our fairly common use case. > > The proof-of-concept is a simple zoomable map with a vector street map > layer + superimposed raster imagery layer (orthorectified to fit the prior > layer correctly) + another superimposed grid of some specified length/width > rectangles which will all be managed in a PostGIS DB and viewed from a web > browser. The follow up would be to tile the imagery (if this isn't done > automatically) and support multiple zoom levels of imagery ala Google Maps. > Our challenge, beyond getting the base functionality working, is to also make > it highly responsive to the map viewing user. Yes, Mapnik can be the rendering engine that handles reading in data from those variety of datasources (either raster imagery from the file system with the GDAL plugin [1] or vector data from PostGIS[2]) and sending along to a web client or desktop app. The Mapnik python bindings can also be used to programmatically inspect your data, like grabbing the extent or envelope: import mapnik bbox = mapnik.Gdal(file="/path/to/geo.tiff").envelope() Ultimately, you'll need to put a server in front of Mapnik that meets the demands of your use case. mod_tile, TileStash, and tirex are the three tile-based servers I am most familiar with that excel with Mapnik. TileCache also works well but must be used with mod_wsgi daemon mode with threads=1 (e.g NOT mod_python otherwise you'll hit thread safety issues). > So my questions are: > > 1. What aspects of the above are already well supported by MapNik? Mapnik is just the backend library that handles turning abstract datasources into either rendered tiles or styled svg/pdf (or in trunk also geojson [3]). Combine it with a tile server (as listed above), a database (like PostGIS), and a front end UI (like OpenLayers or Polymaps), and you have a full app. > 2. What aspects of the above maybe aren't "built-in" to MapNik directly but > would be fairly easily implemented on top of it and that you feel MapNik is > an appropriate platform to build upon? Application logic like permissions to certain datasets, custom styles of certain users, etc. Stuff like django might handle. I've done some R&D on deeper integration between GeoDjango spatial types and Mapnik (see [4]), so you can pass - in pure python - spatial querysets to be rendered by mapnik. But, in general if you want highly scalable and performant tile services then you likely DONT want to mix your tiles with your application, so that they can be separate concerns, on separate machines. > 3. What aspect of the above does MapNik currently have no support for and > should be done using other tools? Wrangling raw data. Use the GDAL/OGR toolset (command line utilities) for prepping your GIS data, like georeferenceing and reprojecting raster datasets to your target projection. > 4. Is there any open source examples of doing major aspects of the above that > we should build upon for our initial proof of concept? Yes, but your use case to me is still pretty abstract so I can't recommend anything specific. > 5. Regarding user responsiveness, has anyone tried various techniques of > pre-loading tiles behind the scenes in AJAX, I've not messed with that - bigger moles to wack usually for performance and pre-caching commonly loaded tiles is way faster than trying to guess what direction someone is going to zoom at any second and does not cost anything in terms of http requests. That said, if you come up with a good algorithm it would be cool to check out. > storing tile URIs in a manner where memcache or an independent media server > can serve them up interdependently of the app server, etc..? Basically that is 101 tile serving. All the servers listed above use cacheable schemes. If your data does not change and you have the disk space precaching and serving statically through apache, lighttpd, nginx is the fastest method around. Just be aware that if you need zoom levels past around 16-17 and continental or global extents then that is WAY more tiles that you want to precache, so caching on-demand is what most people do, while precaching at zooms 0-5,6... > Be curious to hear any recommendations for techniques or experience as to > what is typically the bottleneck. > Depends on the data of course. How big it is and how frequently it changes, and how complex or dynamic your styling needs are. Generally however with big datasets - common in mapping - in the many GB's range, disk-boundness is a common mole. > I would expect what we're doing has been done by many before us so would > like to avoid reinventing the wheel and get a chance to spend some of this > effort making the wheel better. :-) :) > We will certainly be willing to share the results of anything we come up with > that may be useful. > "willing" ;) Actions speak louder than words. > Thanks for any pointers/advice/interest! [1] http://trac.mapnik.org/wiki/GDAL [2] http://trac.mapnik.org/wiki/PostGIS [3] http://trac.mapnik.org/wiki/MetaWriter [4] http://bitbucket.org/springmeyer/djmapnik > > -- Ben Scherrey > _______________________________________________ > Mapnik-users mailing list > [email protected] > https://lists.berlios.de/mailman/listinfo/mapnik-users _______________________________________________ Mapnik-users mailing list [email protected] https://lists.berlios.de/mailman/listinfo/mapnik-users
