Re: [mapserver-users] Does Mapserver utilise Spatialite spatial index correctly?
Rahkonen Jukka wrote: > > Even Rouault wrote: > > > > >> I've commited improvements in GDAL trunk for both points ( > >> http://trac.osgeo.org/gdal/changeset/23944 and > >> http://trac.osgeo.org/gdal/changeset/23945 ) that make the > above request go > >> from 1.7 sec to 0.54 sec . I don't think there's any more > significant speed > >> gain to expect now (at least on OGR side). > > > Actually, I found a new improvement (committed in > > http://trac.osgeo.org/gdal/changeset/23946) that make the > query time go down > > to 0.07 sec (*) ! There was a slowness when opening the > layer due to the > > presence of an ORDER BY clause in the DATA string, that was > evaluated before > > setting the spatial filter. > > > (*) For comparison, I translated your database to Postgis, > and set-up the > > connection mapfile to use that Postgis database (still > through OGR, not the > > MapServer Postgis backend). The same request with OGR PG is > run in 0.18 sec. > > Awesome! About 25 times faster in the evening than it was in > the morning. My opinion is that because Spatialite seems to > be, at least in this case, more that two times faster backend > for Mapserver than PostGIS it should lead to some intensive > and controlled testing now. I updated the cgi-bin directory of my MS4W 3.0.4beta1 with the mapserv.exe and dll files taken from the brand new binaries from http://www.gisinternals.com/sdk/ (contains Mapserver and GDAL revisions r13144, r23972, respectively). As a result Mapserver is really serving me 10, or 25 or even 40 times faster than it did before the update measured as WMS throughput (images/minute). The times include rendering times and all the lags in the system so the difference tells exactly how the end user feels it. Greatest enhancement is with close zooms because then the apatial index bites hardest. And at least with my computer Spatialite backend is clearly faster than PostGIS with the same data and it may be even faster than shapefiles but it is too early to say that really. But this is absolutely something worth more testing. My test database is moderately sized and contains 10 points, 9 lines and 149000 polygons from OpenStreetMap data. I will myself do next trials with ten times bigger Spatialite database with something like million points, lines, and polygons. We do not have much bigger tables in our production. However, I do not believe we are changing from Oracle to SQLite/Spatialite very rapidly in our core business :) -Jukka Rahkonen- ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] Topology error prevents rendering
Hi, OpenStreetMap data tend to contain topology errors and Mapserver sends an empty map when it finds an error on one layer of a multi-layer map. Cases are like this: msOGRFileNextShape(): OGR error. TopologyException: side location conflict at 1.48639e+006 6.89386e+006 [Tue Feb 14 09:41:37 2012].113000 msDrawMap(): Image handling error. Failed to draw layer named 'landuse_layer1'. Topology errors would be best to correct in the data but I wonder if Mapserver could be somehow more robust and just skip the faulty feature or that certain layer instead of giving in totally. Sometimes it would be better to get an incomplete map than no map at all. -Jukka Rahkonen- ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] layers don´t appear on the map
Hello to all, This is my first email on the list. I have a problem, i cannot add layers to the map and i don´t why. I added anothers and led all perfect. Now I can not. Iadded in the mapfile and config_default.xml but do not appear on the map. I'm working with pmapper modifying the demo. Can everybody give me any possible errors to the layers are not being added now? Is there a variable that limits the maximum number of layers? os: windows 7 64 bits mapserver 6.02 thank you Pedro ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
RE: [mapserver-users] Does Mapserver utilise Spatialite spatial index correctly?
Great work Jukka (and Even). Thanks for pursuing this!! David. -Original Message- From: mapserver-users-boun...@lists.osgeo.org [mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Rahkonen Jukka Sent: Tuesday, February 14, 2012 2:24 AM To: 'mapserver-users@lists.osgeo.org' Subject: Re: [mapserver-users] Does Mapserver utilise Spatialite spatial index correctly? Rahkonen Jukka wrote: > > Even Rouault wrote: > > > > >> I've commited improvements in GDAL trunk for both points ( > >> http://trac.osgeo.org/gdal/changeset/23944 and > >> http://trac.osgeo.org/gdal/changeset/23945 ) that make the > above request go > >> from 1.7 sec to 0.54 sec . I don't think there's any more > significant speed > >> gain to expect now (at least on OGR side). > > > Actually, I found a new improvement (committed in > > http://trac.osgeo.org/gdal/changeset/23946) that make the > query time go down > > to 0.07 sec (*) ! There was a slowness when opening the > layer due to the > > presence of an ORDER BY clause in the DATA string, that was > evaluated before > > setting the spatial filter. > > > (*) For comparison, I translated your database to Postgis, > and set-up the > > connection mapfile to use that Postgis database (still > through OGR, not the > > MapServer Postgis backend). The same request with OGR PG is > run in 0.18 sec. > > Awesome! About 25 times faster in the evening than it was in > the morning. My opinion is that because Spatialite seems to > be, at least in this case, more that two times faster backend > for Mapserver than PostGIS it should lead to some intensive > and controlled testing now. I updated the cgi-bin directory of my MS4W 3.0.4beta1 with the mapserv.exe and dll files taken from the brand new binaries from http://www.gisinternals.com/sdk/ (contains Mapserver and GDAL revisions r13144, r23972, respectively). As a result Mapserver is really serving me 10, or 25 or even 40 times faster than it did before the update measured as WMS throughput (images/minute). The times include rendering times and all the lags in the system so the difference tells exactly how the end user feels it. Greatest enhancement is with close zooms because then the apatial index bites hardest. And at least with my computer Spatialite backend is clearly faster than PostGIS with the same data and it may be even faster than shapefiles but it is too early to say that really. But this is absolutely something worth more testing. My test database is moderately sized and contains 10 points, 9 lines and 149000 polygons from OpenStreetMap data. I will myself do next trials with ten times bigger Spatialite database with something like million points, lines, and polygons. We do not have much bigger tables in our production. However, I do not believe we are changing from Oracle to SQLite/Spatialite very rapidly in our core business :) -Jukka Rahkonen- ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Topology error prevents rendering
On 2/14/2012 3:37 AM, Rahkonen Jukka wrote: Hi, OpenStreetMap data tend to contain topology errors and Mapserver sends an empty map when it finds an error on one layer of a multi-layer map. Cases are like this: msOGRFileNextShape(): OGR error. TopologyException: side location conflict at 1.48639e+006 6.89386e+006 [Tue Feb 14 09:41:37 2012].113000 msDrawMap(): Image handling error. Failed to draw layer named 'landuse_layer1'. Topology errors would be best to correct in the data but I wonder if Mapserver could be somehow more robust and just skip the faulty feature or that certain layer instead of giving in totally. Sometimes it would be better to get an incomplete map than no map at all. -Jukka Rahkonen- Jukka, This is actually an issue in OGR. If you put the same data in a shapefile, mapserver will serve it with no problem but when you go through the OGR interface, OGR is throwing the error. It might be possible to make the OGR driver in mapserver more forgiving, but that probably depends on whether or not the OGR error is recoverable and it can continue. I would write a ticket for this issue and assign it to the OGR driver module. -Steve W ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] How to enable a spatial query with MODE=FEATUREQUERY (CGI mode)?
Hi, I have two layers (poly_layer and point_layer) and using MapServer CGI interface I wish to: a) - let user click a polygon at 'poly_layer' b) - then list attributes of 'point_layer' that are contained into the polygon choosen at (a) At my mapfile, both layers has layer->template property set and at CGI interface (web->template) I'm passing: - mode = featurequery - slayer = poly_layer - qlayer = point_layer But after the click on map, got only a "not found" error. Am I doing right use of this mode? Or for this need, is it better to use another mode? ps.: all layers are Mapinfo files and can't use PostGIS at this application. Eduardo Patto Kanegae http://www.webmapit.com | @webmapit ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Topology error prevents rendering
Hi Stephen yep. OGR does not like self-intersecting polygons. should definitely tolerate them ... Mit freundlichem Gruss / Best regards Flavio Hendry TYDAC Inc. - http://www.tydac.ch Web Mapping - http://www.mapserver.ch Swiss Maps - http://www.mapplus.ch Mit freundlichen Gruessen / Kind Regards Flavio Hendry, CEO - mailto:fla...@tydac.ch TYDAC AG - http://www.tydac.ch Geographic Information Solutions Optingenstrasse 27 -- CH-3013 Bern Tel +41 (0)31 368 0180 - Fax +41 (0)31 368 1860 Location: http://www.mapplus.ch/adr/bern/optingenstrasse/27 ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] Constraining label position
Hi, The development version of Openlayers has a new facility: smooth tile transition. When panning, the old image remains visible for 2.5 seconds under the new one, so as to make the panning process smoother. This works fine for static maps, but gives undesired effects with MapServer labels. Those are computed anew for every request, so e.g. labels for items at the corner of the map get a different position, to keep them inside the displayed map. This results in old and new labels being displayed next to one another for some time. Is there an option in MapServer to constrain the labels to always have the same position respective to the geometry they are labeling? Jan ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Constraining label position
On 2/14/2012 12:03 PM, Jan Hartmann wrote: Hi, The development version of Openlayers has a new facility: smooth tile transition. When panning, the old image remains visible for 2.5 seconds under the new one, so as to make the panning process smoother. This works fine for static maps, but gives undesired effects with MapServer labels. Those are computed anew for every request, so e.g. labels for items at the corner of the map get a different position, to keep them inside the displayed map. This results in old and new labels being displayed next to one another for some time. Is there an option in MapServer to constrain the labels to always have the same position respective to the geometry they are labeling? google: mapserver geomtransform this will let you fix the label point to the centroid of the object or some other positions. The issue is that the label point is generally calculated based on the geometry of the object as clipped to the view port. So the geometry changes based on the view clipping and this causes the label point to also change. the geomtransform works on the original data and so the label point becomes fixed. There is also a lot of label enhancements being added to trunk for the next release. You might want to look at rfc 77 [1] in additions to rfc 72 [2]. [1] http://mapserver.org/development/rfc/ms-rfc-77.html [2] http://mapserver.org/development/rfc/ms-rfc-72.html -Steve W ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Does Mapserver utilise Spatialite spatial index correctly?
Le mardi 14 février 2012 09:24:15, Rahkonen Jukka a écrit : > Rahkonen Jukka wrote: > > Even Rouault wrote: > > >> I've commited improvements in GDAL trunk for both points ( > > >> http://trac.osgeo.org/gdal/changeset/23944 and > > >> http://trac.osgeo.org/gdal/changeset/23945 ) that make the > > > > above request go > > > > >> from 1.7 sec to 0.54 sec . I don't think there's any more > > > > significant speed > > > > >> gain to expect now (at least on OGR side). > > > > > > Actually, I found a new improvement (committed in > > > http://trac.osgeo.org/gdal/changeset/23946) that make the > > > > query time go down > > > > > to 0.07 sec (*) ! There was a slowness when opening the > > > > layer due to the > > > > > presence of an ORDER BY clause in the DATA string, that was > > > > evaluated before > > > > > setting the spatial filter. > > > > > > (*) For comparison, I translated your database to Postgis, > > > > and set-up the > > > > > connection mapfile to use that Postgis database (still > > > > through OGR, not the > > > > > MapServer Postgis backend). The same request with OGR PG is > > > > run in 0.18 sec. > > > > Awesome! About 25 times faster in the evening than it was in > > the morning. My opinion is that because Spatialite seems to > > be, at least in this case, more that two times faster backend > > for Mapserver than PostGIS it should lead to some intensive > > and controlled testing now. > > I updated the cgi-bin directory of my MS4W 3.0.4beta1 with the mapserv.exe > and dll files taken from the brand new binaries from > http://www.gisinternals.com/sdk/ (contains Mapserver and GDAL revisions > r13144, r23972, respectively). > > As a result Mapserver is really serving me 10, or 25 or even 40 times > faster than it did before the update measured as WMS throughput > (images/minute). The times include rendering times and all the lags in the > system so the difference tells exactly how the end user feels it. Greatest > enhancement is with close zooms because then the apatial index bites > hardest. And at least with my computer Spatialite backend is clearly > faster than PostGIS with the same data and it may be even faster than > shapefiles but it is too early to say that really. But this is absolutely > something worth more testing. I tested a bit with shapefiles with your Berlin OSM database. Even after adding spatial index (.qix) and attribute index (.idm & .idn) on fields in where clauses, it is a bit slower than spatialite. If you remove all columns from the DBF that are not necessary for the rendering, it improves things a bit again. You can also try the new special SQL command "RESIZE table_name" for shapefiles I've committed yesterday (see http://gdal.org/ogr/drv_shapefile.html ) to adjust columns to their minimum needed size. > > My test database is moderately sized and contains 10 points, 9 > lines and 149000 polygons from OpenStreetMap data. I will myself do next > trials with ten times bigger Spatialite database with something like > million points, lines, and polygons. We do not have much bigger tables in > our production. However, I do not believe we are changing from Oracle to > SQLite/Spatialite very rapidly in our core business :) > > -Jukka Rahkonen- > > ___ > mapserver-users mailing list > mapserver-users@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] postgis raster query question
Hello All, I'm using mapserver to display some rasters as WMS layers using Openlayers. At the moment I have all my rasters in a web-readable folder and it works fine. I want to try accessing the rastes using postgis, but I'm not sure exactly how to do this. I've read through this http://mapserver.org/input/vector/postgis.html and the relevant part of PostGIS in action. My original map file (which worked) looked like this: LAYER NAME "new_york_pop" DATA"/home//htdocs/data/new_york_pop.tif" STATUS OFF TYPERASTER METADATA "wms_title" "WMS new_york Raster" "wms_srs""EPSG:900913" END END I tried changing this to the following: LAYER NAME "new_york_pop" TYPERASTER CONNECTIONTYPE POSTGIS CONNECTION "host=18.00.00.00 dbname=raster_analysis user=postgres password=mypassword port=5434" DATA "SELECT ST_AsTIFF(ST_Clip(r.rast, 1, p.geom, NULL, TRUE)) FROM us_pop_800 AS r, cities AS p WHERE ST_Intersects(r.rast, p.geom) AND p.name= 'new_york'" STATUS OFF METADATA "wms_title" "WMS new_york Raster" "wms_srs""EPSG:900913" END END The SQL query runs fine within the SQL console of Postgresql and returns a few tiffs, but it does not work on mapserver. My mapserver error log shows the following: [Tue Feb 14 12:16:29 2012].918681 msDrawMap(): Image handling error. Failed to draw layer named 'new_york_pop'. [Tue Feb 14 12:16:29 2012].917535 msDrawMap(): rendering using outputformat named png (AGG/PNG). [Tue Feb 14 12:16:29 2012].918757 msDrawRasterLayerLow(): Unable to access file. Corrupt, empty or missing file 'SELECT ST_AsTIFF(ST_Clip(r.rast, 1, p.geom, NULL, TRUE)) FROM us_pop_raster AS r, cities AS p WHERE ST_Intersects(r.rast, p.geom) AND p.name = 'new_york'' for layer 'new_york_pop'. [Tue Feb 14 12:16:29 2012].918766 msDrawMap(): Image handling error. Failed to draw layer named 'new_york_pop'. I would appreciate any suggestions for debugging this. I have connected to the postgresql database using a php file from the same location so mapserver should have the correct permissions to connect (and my port is 5434, rather than the usual 5432). Thank you, David ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] raster storage format recommendations - tiff too large
Hello, I would like to increase the speed on my air photos. I have created a uncompressed tiff, which Is fast however, the file size is much too large for my project. On the other hand the file size of MrSID is appropriate, but it is slow even when using PROCESSING "OVERSAMPLE_RATIO=1.0." Is there a happy medium between the two file formats? Thanks Mark Volz GIS Specialist ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] raster storage format recommendations - tiff too large
Hi, Do geotiffs with -co tiled=YES -co compression=JPEG -co photometric=YCBCR and then gdaladdo with --config COMPRESS_OVERVIEW JPEG --config PHOTOMETRIC_OVERVIEW YCBCR I prefer external overviews nowadays. -Jukka Rahkonen- Mark Volz wrote: > Hello, > I would like to increase the speed on my air photos. I have created a > uncompressed tiff, which Is fast however, the file size is much too large for > my project. On the other hand the file size of MrSID is appropriate, but it > is slow even when using PROCESSING "OVERSAMPLE_RATIO=1.0." > Is there a happy medium between the two file formats? > Thanks > Mark Volz > GIS Specialist ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] Mapcache Seed / libpng error?
Hello List, We've been using mapcache_seed ( mapcache SVN checkout 13002 ) to build our caches mostly successfully so far, but recently have come up against an error which has only cropped up with a certain layer which includes our text & labels www-data@server$ ./mapcache_seed -c /opt/mapserver/mapcache/oe_mapcache.xml -t 2012-02-02_text_cache_nztm -g portalgridnztm1 -z 7,8 found old lockfile /tmp/_gc_lock9-175-1233-slope_cache_nztm.lck, deleting it found old lockfile /tmp/_gc_lock7-7-9-2012-02-02_text_cache_nztm.lck, deleting it found old lockfile /tmp/_gc_lock9-136-179-terr_auth_cache_nztm.lck, deleting it found old lockfile /tmp/_gc_lock9-138-86-wards_cache_nztm.lck, deleting it seeding tile 65 25 7libpng error: IDAT: CRC error failed to setjmp(png_jmpbuf(png_ptr)) failed to load image data from metatile failed to setjmp(png_jmpbuf(png_ptr)) failed to load image data from metatile seeding tile 130 50 8 www-data@server$ or... www-data@server$ ./mapcache_seed -c /opt/mapserver/mapcache/oe_mapcache.xml -t 2012-02-02_text_cache_nztm -g portalgridnztm1 -z 6,7 found old lockfile /tmp/_gc_lock9-137-51-terr_auth_cache_nztm.lck, deleting it found old lockfile /tmp/_gc_lock9-182-1201-slope_cache_nztm.lck, deleting it found old lockfile /tmp/_gc_lock9-138-197-wards_cache_nztm.lck, deleting it seeding tile 25 15 6libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng warning: Ignoring bad adaptive filter type libpng error: IDAT: CRC error failed to setjmp(png_jmpbuf(png_ptr)) failed to load image data from metatile seeding tile 50 35 7failed to setjmp(png_jmpbuf(png_ptr)) failed to load image data from metatile www-data@server$ The next step will be to checkout the latest SVN version. But we are currently mid-build with some other layers, so are waiting for them to finish before we do that. But thought we'd fire it out to the list to see if anyone has any ideas as to what might be causing this? Below is the mapcache config: http://server/cgi-bin/mapserv601? image/png text /opt/mapserver/mapfiles/m6scenz_topo_new.map true src_text_update disk PNG portalgridnztm1 5 5 10 "labelcache_map_edge_buffer""-10"is set in mapserver WEB METADATA Cheers Andrew Please consider the environment before printing this email Warning: This electronic message together with any attachments is confidential. If you receive it in error: (i) you must not read, use, disclose, copy or retain it; (ii) please contact the sender immediately by reply email and then delete the emails. The views expressed in this email may not be those of Landcare Research New Zealand Limited. http://www.landcareresearch.co.nz ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
