On Wed, 2010-07-14 at 17:52 -0400, joel collins wrote:
> http://dl.dropbox.com/u/203098/mapnik.7z
>
> File is 30mb compressed, 80mb uncompressed. Today i setup mapnik on
> apache via fastcgi, and the performance is still about 10x slower than
> mapserver. I'd really like to know if i'm doing something dumb...
>
> The .index file may not have found its way into the 7z file, but i did
> generate it using shapeindex. Also, just as a sanity check, here are
> 2 example urls i'm using to query the data:
>
> Mapserver: (response time 60ms)
> http://servername/cgi-bin/mapserv.exe?map=C%3A%2FOSGeo4W%2Fapps%2Fmapserv-demo%2Fdistricts_googproj.map&transparent=true&mode=map&map_imagetype=png&mapext=-10018754.1704+4383204.9484+-9705668.1026+4696291.0162&imgext=-10018754.1704+4383204.9484+-9705668.1026+4696291.0162&map_size=256+256&imgx=128&imgy=128&imgxy=256+256
>
> Mapnik:
> http://servername/fcgi-bin/wms?LAYERS=dist_google&VERSION=1.1.1&UNITS=meters&SERVICE=WMS&REQUEST=GetMap&STYLES=&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&FORMAT=image%2Fjpeg&SRS=EPSG%3A900913&BBOX=-10018754.1704,4383204.9484,-9705668.1026,4696291.0162&WIDTH=256&HEIGHT=256
With your shapefile & WMS setup I get about 6 requests per second using
Mapnik which seems to roughly match your performance.
It looks to me that the slow performance may be because the geometries
in your shapefiles are quite detailed. If I run the commands below to
simplify the shapefile from 80MB to 2.5MB then the rendering using
google_300.shp is about 10 times faster with no obvious degradation in
the output quality.
$ shp2pgsql dist_google.shp | psql gis
$ pgsql2shp -f google_300 gis "select simplify(the_geom, 300) as
the_geom, color from dist_google where area(simplify(the_geom,300)) > 0"
I don't know whether Mapserver is better at deciding how much data
detail of the geometries to render or if it has a simpler rendering
model which is faster at filling complex polygons.
At higher zooms you may need to go back to rendering from the more
detailed shapefiles again.
Jon
> On Wed, Jul 14, 2010 at 4:16 AM, Artem Pavlenko <[email protected]> wrote:
> > Hi Joel,
> >
> > Could you post that shapefile somewhere or provide a link, pls.
> >
> > Artem
> >
> > On 14 July 2010 00:11, joel collins <[email protected]> wrote:
> >> Sorry guys, i know i've sent a lot of mail about this. I managed to
> >> get my shapefile loaded into mapserver to try to compare performance,
> >> and the difference is night and day. There is essentially no
> >> noticable lag when viewing the shapefile via mapserver, regardless of
> >> zoom level.
> >>
> >> Do you think i'm doing something wrong that is making mapnik a lot
> >> slower than mapserver? My layer is very simple, just some borders
> >> essentially. Am I better off using a tool like mapserver or geoserver
> >> rather than mapnik for simple layers without much visual pizzaz?
> >>
> >> I am frustrated because the toolsets around mapnik seem much more
> >> modern and easy for me to comprehend (cascadenick, etc), while
> >> mapserver seems more difficult to grasp.
> >>
> >> On Mon, Jul 12, 2010 at 9:42 PM, Dane Springmeyer <[email protected]> wrote:
> >>>
> >>> On Jul 12, 2010, at 6:14 PM, joel collins wrote:
> >>>
> >>>> I realize that my performance will improve using apache / mod_tile,
> >>>> but 0.5 cpu seconds to render a tiny tile is a lot! Even with a 4
> >>>> core machine this would mean that 8 uncached tile requests per second
> >>>> would max out the CPU. I have kept experimenting and found
> >>>> (obviously, in hindsight), that since my shapefile was not in the
> >>>> google projection, this conversion was being done at runtime.
> >>>
> >>> Yes, reprojection on the fly is very costly, so ensuring the layer srs
> >>> exactly matches the map srs is a critical thing.
> >>>
> >>>> after
> >>>> running ogr2ogr my render time is below 0.2 seconds for zoom level 10.
> >>>> This may have to do...
> >>>
> >>> nice.
> >>>
> >>>>
> >>>> An interesting thing i noticed was when running on windows (osgeo4w
> >>>> installation), mapnik was 2x as slow as mapnik running on an ubuntu VM
> >>>> on the same physical machine!
> >>>
> >>> Make sure you rebuild any indexes for shapefiles on each platform. If
> >>> that does not bring the speeds more closely inline then it would be
> >>> worthwhile to do a bit more digging as they should be comparable.
> >>>
> >>>>
> >>>> On Thu, Jul 8, 2010 at 6:37 PM, Dane Springmeyer <[email protected]> wrote:
> >>>>>
> >>>>> On Jul 7, 2010, at 5:16 PM, joel collins wrote:
> >>>>>
> >>>>>> New mapnik user here, i've made a lot of progress in the last few
> >>>>>> weeks but its been slow going. I am trying to create a tile overlay
> >>>>>> on google maps of some census bureau tiger data (congressional
> >>>>>> district maps in particular) and i have downloaded the associated
> >>>>>> shapefiles (containing polygons only, no markers or other data types).
> >>>>>>
> >>>>>> For a web server I am currently just using Tilelight (mapnik-utils)
> >>>>>> for development, and although i'm planning on using a different
> >>>>>> configuration in production, i'm seeing very poor performance during
> >>>>>> tile generation.
> >>>>>
> >>>>> That's because the 'liteserv.py' script is only for development, and
> >>>>> deployment targets multicore architectures only (mod_wsgi with N
> >>>>> processes = number of cores).
> >>>>>
> >>>>> Although I too find that python's wsgiref server implementation (that
> >>>>> liteserv.py) uses is very slow. You can get about double the speed with
> >>>>> other development servers that implement the WSGI standard, one being
> >>>>> werkzeug. In fact, just the other day I added support for werkzeug:
> >>>>>
> >>>>> http://bitbucket.org/springmeyer/tilelite/changeset/e2335783f42c
> >>>>>
> >>>>> But, again the faster deployment for TileLite is with mod_wsgi inside
> >>>>> Apache.
> >>>>>
> >>>>> And faster again will be a multithreaded server like mod_tile.
> >>>>>
> >>>>>> On my google maps, for example, when i scroll to a
> >>>>>> new area or change zoom levels it can often take 15 seconds for the
> >>>>>> new tiles to appear (this is especially slow because there are about a
> >>>>>> dozen simultaneous tile requests that queue up).
> >>>>>
> >>>>> That's because you are running a single-threaded, single-process
> >>>>> development server that is only for development.
> >>>>>
> >>>>>> I realize that on a
> >>>>>> multi-core production machine this wont be so bad but it still seems
> >>>>>> very slow.
> >>>>>>
> >>>>>> When I render a single tile, it still takes between 0.5-3 seconds. I
> >>>>>> have tried loading the shapes into postgres but that actually resulted
> >>>>>> in worse performance (5 seconds), although I may have configured
> >>>>>> something wrong. Some rudimentary math tells me that I wont be able
> >>>>>> to pre-render all tiles (we would like to be able to show street-level
> >>>>>> zooming in google maps, around 16 or 17), so we will need to have the
> >>>>>> ability to render tiles that haven't been pre-rendered fast enough for
> >>>>>> real time browsing. Also, using something like TileCache will help of
> >>>>>> course but at the end of the day new tiles still need to be rendered.
> >>>>>>
> >>>>>> My questions are:
> >>>>>> What kind of response times should I expect from mapnik for creating
> >>>>>> tiles from shapefiles?
> >>>>>>
> >>>>>> What can be done to speed up the rendering? Should I be downsampling
> >>>>>> the number of data points in the shape files? We like the high level
> >>>>>> of accuracy that we see in the tiger files but maybe thats what is
> >>>>>> slowing this down?
> >>>>>>
> >>>>>> Is leaving the data in shapefiles the appropriate approach? (I was
> >>>>>> planning on using postgis so i could do other queries like see which
> >>>>>> polygons touch, find what polygons are near a lat/long, etc)
> >>>>>>
> >>>>>> Thanks!
> >>>>>> _______________________________________________
> >>>>>> Mapnik-users mailing list
> >>>>>> [email protected]
> >>>>>> https://lists.berlios.de/mailman/listinfo/mapnik-users
> >>>>>
> >>>>>
> >>>
> >>>
> >> _______________________________________________
> >> Mapnik-users mailing list
> >> [email protected]
> >> https://lists.berlios.de/mailman/listinfo/mapnik-users
> >>
> >
> _______________________________________________
> Mapnik-users mailing list
> [email protected]
> https://lists.berlios.de/mailman/listinfo/mapnik-users
_______________________________________________
Mapnik-users mailing list
[email protected]
https://lists.berlios.de/mailman/listinfo/mapnik-users
