Hi Marcin , Excellent stuff! And great looking maps. I'll be in touch regarding your patches no doubts, this is just a quick email to say thank you very much for your contributions!
I started work on compositing (merging) using agg:http://trac.mapnik.org/wiki/Compositing and I know that Brian Quinion has done some work as well. This would be a great feature to have in Mapnik. I'll have a look at all implementation asap. Thanks again! Artem 2009/2/26 Marcin Rudowski <[email protected]>: > Hi again > > After seeing hill shading in action, I did some research too in this > matter. I based my work on: > http://wiki.openstreetmap.org/wiki/HikingBikingMaps > and again came across some issues with Mapnik. > > Mapnik problems: > - very basic support for rasters, no opacity, no advanced merging, only > replacing already drawn background. > - no raster scaling, only nearest neighbor. I need scaling up to 64x > (12-18 zoom level) and mapnik generated only big squares > - despite my last patch, 256 color palette was still bad for slow > gradients like shading. > > I corrected, in some degree, all of above limits, and want > to share with You. > > Here are some examples of my work in action: > http://mapa.ump.waw.pl/ump-www/?zoom=11&lat=49.70878&lon=19.28666 > http://mapa.ump.waw.pl/ump-www/?zoom=15&lat=49.30126&lon=19.94725 > http://mapa.ump.waw.pl/ump-www/?zoom=13&lat=52.56388&lon=19.70792 > http://mapa.ump.waw.pl/ump-www/?zoom=11&lat=51.29269&lon=19.3567 > http://mapa.ump.waw.pl/ump-www/?zoom=11&lat=54.61966&lon=18.23952 > http://mapa.ump.waw.pl/ump-www/?zoom=11&lat=53.2975&lon=18.36724 > http://mapa.ump.waw.pl/ump-www/?zoom=14&lat=49.39029&lon=22.47346 > > I use SRTM-3 v2: > ftp://e0srp01u.ecs.nasa.gov/srtm/version2/SRTM30/ > converted to mercator projection (gdal) avoiding elipsoid->spheroid > reprojection (only latlon->mercator) with pixel size: 76.437 x 76.437 m. > Based on srtm-3 resolution, it should be 90x90, but alignment to output > resolution was needed. 76.437 is closest number to 90 from > (2*pi*6378137)/2^n set. > For Poland, this gives 16000x14000 GeoTIFF in resolution aligned > to zoom 11. I made scaled copies 50% and 25% for lower zooms, to > increase speed and avoid downscaling. I could do the same to avoid > upscaling, but already it is 250MB file, and each zoom would make it 4x > bigger then previous. > > > Patches are in attachment (hopefully this time :) ), here I will > describe in short, what is changed, and what can be done more. > > 1) Raster opacity > > Added opacity handling using existing methods. It isn't needed > for shading, but can be useful. To enable opacity, add css style opacity > to RasterSymbolizer. It works both for cairo and internal agg renderer. > > 2) Raster merging > > Added some basic merging methods. I use for hill shading which is gimp > like "merge grain": > http://docs.gimp.org/en/gimp-concepts-layer-modes.html#id2834930 > Default is normal mode, to select differently, set css "mode" to > desired value. Available modes based on given link: > - grain_merge : bg + fg - 0.5 > - grain_merge2 : bg + fg*2 - 1.0 > - multiply : fg * bg > - multiply2 : fg * bg * 2.0 > - divide : bg / fg > - divide2 : bg * 2.0 / fg > - screen : 1 - (1-fg)(1-g) > - hard_light : ..... (look ad link above) > > Agg renderer have some similar code in > agg/include/agg_pixfmt_rgba.h > but I didn't use it. I wrote my code based on existing part for alpha > blending from mapnik/include/mapnik/graphics.hpp. > This fix works only with agg renderer, I couldn't find appropriate way > to implement it using Cairo. I googled and found some references to > filters for ie. hard light, but no documentation or examples, and I'm > not sure if it exists in cairo. > This fix and previous probably handle: > http://trac.mapnik.org/ticket/157 > if someone could implement it also in cairo, or at least show me right > direction. > > 3) Raster scaling > > Added bilinear scaling algorithm. Algorithm can be selected with css > "scaling" parameter and available values are: > - fast : nearest neighbor as it was already > - bilinear : new bilinear interpolation for all 4 chanels (RGBA) > - bilinear8 : as bilinear, but only one channel assumed. This probably > needs some fix, as it doesn't handle alfa. I added it to speedup in my > case, where raster is grayscale without transparency. > > Scaling in cairo renderer could be reimplemented using matrix, but I use > mapnik-0.5.1 with my patches, so basically I concentrated on > agg renderer. > > 4) 256 color palette optimizations > > My previous patch removed obvious bug in octree. Still, generated > palette is not enough for hill shading. It looks more like 50 colors, > then 256. The problem is with prunning only on lowest level, so all left > leafs are almost on the same level. It is similar to dividing to even > ranges, regardless of density. My patch prunes only branches with lowest > pixel count from all levels, so subtrees with many leafs, remain deeper, > then one with smaller sample count. > > Some limiting was needed, to allow colors with small sample count to > remain unharmed (ie, some distinctive feature on map with not enough > pixels, to "fight" with gradients should be preserved). > I didn't find any significant increase in processing time, and > only 5-10% increase in size (due to more significant colors). Effects > are almost as good, as optimal palette without error-diffusion in > imagemagick or Gimp. > > Still there are problem with color assignment, because quite often cube > for classified color doesn't have the best color. Neighbor cubes should > be compared, but traversing octree is fast, but not free. I made some > fix, to check siblings and neighbour cubes (going up and down the > three), but it increased processing time of quantization about 40% and > effects aren't so much worse without it. It isn't included in > attachment. > > > > Example osm.xml fragment: > <Style name="raster18"> > <Rule> > <MaxScaleDenominator>1000000</MaxScaleDenominator> > <MinScaleDenominator>250000</MinScaleDenominator> > <RasterSymbolizer> > <CssParameter name="opacity">1.0</CssParameter> > <CssParameter name="scaling">bilinear</CssParameter> > <CssParameter name="mode">grain_merge</CssParameter> > </RasterSymbolizer> > </Rule> > </Style> > <Layer name="dem18" status="on" > > <StyleName>raster18</StyleName> > <Datasource> > <Parameter name="type">gdal</Parameter> > <Parameter name="file">&geotiffs;/hill_15m.tif</Parameter> > <Parameter name="format">tiff</Parameter> > </Datasource> > </Layer> > > > > Attachments: > > - mapnik.mr.palette2.patch.gz : palette fix 4) > - mapnik.mr.scale.patch.gz : scaling methods > - mapnik.mr.raster1.patch.gz : parsing RasterSymbolizer rule > - mapnik.mr.process.patch.gz : using raster params to select scaling > algorithm, opacity and merging method. > > > > > Additional notes: > > If scaling and tiling, there can be some problems with alignment if > raster resolution isn't integer multiply of output resolution and > rendered regions have margin, like in mod_tile: > A---------B---|---A-----------B > Because size and position of source raster bbox is averaged to pixel steps, > error can be different on each edge and '|' in A image could be > displaced in B image. 0.5 pix in raster could mean 16px after 32x scaling (5 > zoom steps) and this is noticeable even earlier. > > I use raster in resolution of zoom 11, so scaling is power of 2 and any > errors are same on each rendered edge. In my case, mod_tile renders > in each zoom: 9*256 x 9*256 region. This is cut with 128 margin to 8x8 > tiles, each 256x256 size. Everything is scaled by power of 2, and > everything is aligned. > > It shouldn't(?) matter if we consider no margin tile joining: > A--------AB---------B. > > > > Adding hill shading to my Mapnik using raster increased generation time > only about 2x (35tiles/s -> 17 tiles/s). > > In my case map is updated ~twice a week and only 300k are forced to > redraw and send to cache server. > > I use self-patched mapnik 0.5.1 with some 3 months old version > of mod_tile for apache. Actually I have two mod_tile modules, to handle > two separate osm.xml and two databases, which I switch on update. > > My osm.xml is divided to separate files defining layers, styles, db > settings etc. Styles are grouped by categories. In effect I have 22 xml > files that sum up to 420kB. Thank You for libxml2 features, or I would > go insane managing this in one osm.xml :) > > Regards, > Marcin Rudowski > > _______________________________________________ > Mapnik-users mailing list > [email protected] > https://lists.berlios.de/mailman/listinfo/mapnik-users > > _______________________________________________ Mapnik-users mailing list [email protected] https://lists.berlios.de/mailman/listinfo/mapnik-users
