Re: [mapserver-users] OpenGL on Linux (Debian/Ubunu)

[Schepers, Benjamin]

Hi Daniel,

thanks for your advice - I've done exactly the same way you as you described it 
for some years now (doing the painful "average"-resampling due to preprocessing 
the overviews with GDAL and nesrest in WMS-postprocessing). The servers and 
services are "highly optimized" and generally fast - thanks to GDAL's and 
mapserver's functionality and performance and community hints. But I wasn't 
really satisfied with final image-quality, because of the trade-of between 
performance and image-quality on WMS-postprocessing - I decided for performance.

From my point of view it still makes a difference in image quality, whether I 
use nearest or average on WMS-postprocessing, especially the aliasing-effects 
on edges of buildings, bridges, etc. would be nearly totally eliminated with 
"average"postprocessing , I did some tests!

My guess was, that resampling could be done much better by the GPU (having a 
look in the world of gaming, where some "super-mega-ultra-antialiasing" is done 
with more than 100 frames per seconds in highest resolutions.

Now there is an option to get some new servers with integrated GPUs (Nvidia 
GTX1080) and asked myself (and now the community ;-) ) whether it would make 
sense  to exercise that option...


Regards Ben




-Ursprüngliche Nachricht-
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Daniel Morissette
Gesendet: Donnerstag, 1. Juni 2017 16:05
An: mapserver-users@lists.osgeo.org
Betreff: Re: [mapserver-users] OpenGL on Linux (Debian/Ubunu)

There have been some experiments a few years ago but I don't remember how far 
the implementation went, how much real testing it got and whether it is still 
maintained. You may want to look at 
http://mapserver.org/development/rfc/ms-rfc-50.html and search for discussions 
on the dev list around 2008-2011 for the full history.

But for your specific problem, have you considered simply generating overviews 
in your source images using average resampling?  This way you would pay the 
price of (average or bilinear) resampling only once and your WMS requests could 
use nearest neighbor on the overviews to provide much better quality results 
without the processing cost.

More info about optimizing raster access is available here (in case you had not 
seen it yet):
http://mapserver.org/input/raster.html#preprocessing-rasters

Daniel


On 2017-06-01 9:35 AM, Schepers, Benjamin wrote:
> Hi list,
> 
> has anyone of you made some experience with mapserver and GPU-rendering on 
> Linux (Debian preferred)?
> 
> The use case would be to speed up raster-resampling on the fly (nearest <-> 
> average) in WMS and maybe later vector-rendering on complex symbology.
> For now raster-rendering in average-mode is three-times slower than 
> resampling with nearest neighbor (which looks worse, indeed).
> 
> Are there some pitfalls? Maybe compiling with openGL-driver, other issues, 
> etc
> 
> Would GDAL benefit from using GPU on processing very large rasters 
> (aerialimage-BigTIFFs). How would this be done? OpenGL / OpenCL?
> Any hints are welcome, I couldn't find that much info on this...
> 
> Regards
> Ben
> 
> ___
> mapserver-users mailing list
> mapserver-users@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-users
> 


--
Daniel Morissette
Mapgears Inc
T: +1 418-696-5056 #201
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/mapserver-users
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/mapserver-users

[mapserver-users] OpenGL on Linux (Debian/Ubunu)

[Schepers, Benjamin]

Hi list,

has anyone of you made some experience with mapserver and GPU-rendering on 
Linux (Debian preferred)?

The use case would be to speed up raster-resampling on the fly (nearest <-> 
average) in WMS and maybe later vector-rendering on complex symbology.
For now raster-rendering in average-mode is three-times slower than resampling 
with nearest neighbor (which looks worse, indeed).

Are there some pitfalls? Maybe compiling with openGL-driver, other issues, 
etc 

Would GDAL benefit from using GPU on processing very large rasters 
(aerialimage-BigTIFFs). How would this be done? OpenGL / OpenCL?
Any hints are welcome, I couldn't find that much info on this...

Regards
Ben

___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] geotiff 48bit with mapserver 7 wms

[Schepers, Benjamin]

Hi David, Hi list,

@David: you said "Well QGIS does show just a black image"

I assume that this is just a layer-specific contrast setting. Maybe just play 
around with stretching (layer properties -> style -> band rendering), e.g. set 
"min" to "0" and "max" to "255" (per each band) and then set "contrast 
enhancement" to "stretch to minmax"

Regards Ben



Mit freundlichen Grüßen
Im Auftrag

Benjamin Schepers


Geodaten, Stadtplanwerk, Luftbilder
Kronprinzenstraße 6
45128 Essen
Fon: +49 201 2069-232
Fax: +49 201 2069-369
schep...@rvr-online.de



Die Regionaldirektorin
Kronprinzenstraße 35
45128 Essen
Zentrale: +49 (0) 201 2069-0
Fax: +49 (0) 201 2069-500
www.metropoleruhr.de

Postfach 10 32 64
45032 Essen

Steuernummer: RVR 112/5797/0116
USt.-ldNr.: DE 173867500

Diese E-Mail koennte vertrauliche und/oder rechtlich geschuetzte Informationen 
enthalten.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich 
erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie 
diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail sind nicht 
gestattet.
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in error) 
please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.
-Ursprüngliche Nachricht-
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von David
Gesendet: Dienstag, 18. April 2017 18:01
An: 'mapserver-users@lists.osgeo.org'
Betreff: Re: [mapserver-users] geotiff 48bit with mapserver 7 wms

So, i did some testing.

The wms is not serving the images with float32 as data type. It shows the same 
issue, false colors.
The wcs is working and serving the float32 data type data.
For my use-case setting the MAXSIZE higher than the tile dimension was 
important.

Well QGIS does show just a black image, but GlobalMapper is loading the data.

Just out of curiosity, can somebody explain "WCS_SIZE" "..." to me? Is this the 
size in pixel x y for the complete TILE_INDEX?

David


-- Original Message --From: "David" 
To: "'mapserver-users@lists.osgeo.org'" 

Sent: 4/18/2017 11:07:04 AM
Subject: Re[2]: [mapserver-users] geotiff 48bit with mapserver 7 wms

>Hi Cora,
>
>do you use PROCESSING "SCALE=AUTO" with you LAYER definition?
>
>SCALE[_n]=AUTO or min,max
>This directive instructs the GDAL reader to pre-scale the incoming 
>raster data. It is primarily used to scale 16bit or floating point data 
>to the range 0-255, but can also be used to constrast stretch 8bit 
>data.
>
>http://mapserver.org/input/raster.html#special-processing-directives
>
>Cheers David
>
>
>-- Original Message --
>From: "Berger, Cora" 
>To: "'mapserver-users@lists.osgeo.org'" 
>
>Sent: 4/18/2017 10:47:42 AM
>Subject: Re: [mapserver-users] geotiff 48bit with mapserver 7 wms
>
>>Dear David, Steve, Ben, etc.
>>I am working with MapServer-WCS. I still have problems defining the 
>>correct output formats. Input to most of our WCS are 8bit Geotiffs.
>>The MapServer WCS-output is either tiff, png or jpeg with 24bit or 
>>Png;mode=8bit with 8 bit (if no output format is defined). The 
>>png;mode=8bit-format changes the imagecolours, so it isn't really 
>>useful. I can't get a colored 8bit Geotiff as output (a 
>>8bit-definition always results in greyscale images). I tried different 
>>output format-definitions, none of them worked. I can define a palette 
>>for the png-format, but unfortunately not for any other format.
>>For our WCS it is important that the output-data looks the same like 
>>the input-data. It seems that at the moment MapServer can't just pass 
>>the input-data through... This might be a GDAL issue...
>>
>>So if you find a way to define your wished output-format, please share 
>>it, maybe it helps me, too. Maybe anybody else has an idea?
>>
>>Cora
>>
>>
>>>Message: 1
>>>Date: Mon, 17 Apr 2017 22:14:28 +
>>>From: David 
>>>To: mapserver-users@lists.osgeo.org
>>>Subject: Re: [mapserver-users] geotiff 48bit with mapserver 7 wms
>>>Message-ID: 
>>>
>>>Content-Type: text/plain; format=flowed; charset=utf-8
>>>
>>>Thank you very much Steve. You helped me to understand the problem!
>>>
>>>David F.
>>>
>>>-- Original Message --
>>>From: "Stephen Woodbridge" 
>>>To: mapserver-users@lists.osgeo.org
>>>Sent: 4/14/2017 9:54:20 PM
>>>Subject: Re: [mapserver-users] geotiff 48bit with mapserver 7 wms
>>>
Yeah, technically a geotiff can be any format, but mapserver only 
supports a limit set for band widths.

A new version of Mapcache was just released and it has GDAL support 
in it  and serves 

Re: [mapserver-users] geotiff 48bit with mapserver 7 wms

[Schepers, Benjamin]

Hi,

maybe it would the better way to use the WCS-Interface? To be honest, I didn't 
use it that way by myself. But it sounds like that would be an proper scenario 
for WCS (doing some "more" with that data besides the classic "viewing").

To configure WCS shouldn't be that hard, I think:
http://mapserver.org/ogc/wcs_format.html
http://mapserver.org/de/ogc/wcs_server.html

Moreover QGIS has built-in WCS-support.

Just an idea...

Regards Ben



-Ursprüngliche Nachricht-
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von David
Gesendet: Dienstag, 18. April 2017 00:14
An: mapserver-users@lists.osgeo.org
Betreff: Re: [mapserver-users] geotiff 48bit with mapserver 7 wms

Thank you very much Steve. You helped me to understand the problem!

David F.

-- Original Message --
From: "Stephen Woodbridge" 
To: mapserver-users@lists.osgeo.org
Sent: 4/14/2017 9:54:20 PM
Subject: Re: [mapserver-users] geotiff 48bit with mapserver 7 wms

>Yeah, technically a geotiff can be any format, but mapserver only 
>supports a limit set for band widths.
>
>A new version of Mapcache was just released and it has GDAL support in 
>it  and serves WMS, but I'm not sure if it will do what you want.
>
>Sorry, I'm out of ideas on this.
>
>-Steve W
>
>On 4/14/2017 6:56 PM, David F. wrote:
>>I did not reply after my latest test, sorry.
>>I tried all tiff IMAGEMODE for my data set. None gave me 16 bit at the 
>>gis client.
>>As far as I understand gtiff is 8 bit.
>>The only IMAGEMODE for integer is signed INT16 , which is 
>>unfortunately the only supported integer data type. For RGB data 
>>unsigned integer is the only integer data type that makes sense. Just 
>>if i change the IMAGEMODE to FLOAT32 mapserver will serve more than 8 
>>bit per band.
>>I think i will give gdal and float32 bit a try nevertheless it will 
>>double the image size.
>>Maybe someone else has an idea.
>>
>>
>>
>>
>>
>>On April 14, 2017 1:43:09 PM Stephen Woodbridge 
>> wrote:
>>
>>>http://mapserver.org/mapfile/outputformat.html
>>>
>>>Why not serve them a GTiff format for qgis?
>>>
>>>You can define multiple output formats in the mapfile, and let the 
>>>browser use the 8bit png and have qgis request images in gtiff 
>>>format.
>>>
>>>-Steve
>>>
>>>On 4/13/2017 11:03 AM, David wrote:
Hi Steve,

thank you very much for your help. The Mapserver is serving the 
tiles with the correct RGB values now. The problem is, that it is 
serving
8
bit per band and not 16 bit.
I need the 16bit because i want to use the wms as a base layer for 
some further processing with qgis or another gis client. So no 
problem regarding browsers for me.

I will take a look and change the data type to float32 for the tiff 
files. At least this is an supported OUTPUTFORMAT for mapserver.
Any suggestions on this matter are highly appreciated.

Cheers,
David

-- Original Message --
From: mapserver-users-requ...@lists.osgeo.org
To: mapserver-users@lists.osgeo.org
Sent: 4/12/2017 4:00:02 PM
Subject: mapserver-users Digest, Vol 111, Issue 5

>Send mapserver-users mailing list submissions to
>  mapserver-users@lists.osgeo.org
>
>To subscribe or unsubscribe via the World Wide Web, visit
>  https://lists.osgeo.org/mailman/listinfo/mapserver-users
>or, via email, send a message with subject or body 'help' to
>  mapserver-users-requ...@lists.osgeo.org
>
>You can reach the person managing the list at
>  mapserver-users-ow...@lists.osgeo.org
>
>When replying, please edit your Subject line so it is more specific 
>than "Re: Contents of mapserver-users digest..."
>
>
>Today's Topics:
>
>1. geotiff 48bit with mapserver 7 wms (David)
>2. Re: geotiff 48bit with mapserver 7 wms (Stephen Woodbridge)
>
>
>---
>---
>
>Message: 1
>Date: Wed, 12 Apr 2017 00:22:12 +
>From: David 
>To: mapserver-users@lists.osgeo.org
>Subject: [mapserver-users] geotiff 48bit with mapserver 7 wms
>Message-ID: 
>
>Content-Type: text/plain; charset="utf-8"; Format="flowed"
>
>Hi there,
>
>iam having some trouble to serve 48 bit geotiff images with a 
>mapserver
>7 wms. I created to layers for speed purposes with scale limiters. 
>The
>lowres layer is served well and looks good, it is 24bit geotiff 
>data(8bit per channel). But if the layer changes to the highres, 
>48bit geotiff data(16bit per channel), the data is shown wrong.
>I searched for some hints regarding a similar issue but could not 
>find something.
>
>This is a screenshot of the loaded highres layer. It does not 
>matter if i load a bbox 

Re: [mapserver-users] WMS T time series data support

[Schepers, Benjamin]

Hey,

h, debugging can be an awful thing…..

Always have a look at the logs (error, etc.).

I’d suppose, that mapserver can’t find the tiles.
A valid DIMENSION-Element in GetCapabilities could look like this:
2009/2016/P1Y<-- in this case the dataset 
supports time from 2009-2016 in one-year-steps]


Are you really sure about the right relations on paths? See:
http://mapserver.org/optimization/tileindex.html#tileindex

I can’t see any “SHAPEPATH” (directory of index), which is referenced in 
MAP-section, see:
http://mapserver.org/mapfile/map.html [from mapserver-docs: … Normally the 
location should contain the path to the tile file relative to the shapepath, 
not relative to the tileindex itself. … ]


What also works for me is using absolute paths:
Please try the the absolute path to tileindex, e.g.:
TILEINDEX "/data/subfolder/index"

My index-file also contains absolute paths to tiff-files


___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] WMS T time series data support

[Schepers, Benjamin]

Hi Sushrut,

yes this should be possible.

Here are general information:
http://mapserver.org/ogc/wms_time.html

Build index with time-information for your needs (also works for wms):
http://mapserver.org/ogc/wcs_server.html#building-spatio-temporal-tile-indexes


è This is a index-shapefile including a BBOX of every tif-file, the tif-path 
within the filesystem and another column with the date of the specific file. 
The date-field needs to be created/filled manually…

I just thought of writing a python script (extending “gdaltindex” 
commandline-utility), which reads the date from tif-metadata 
('TIFFTAG_DATETIME' ) and writes it to the index-shape.


Regards
Ben

Mit freundlichen Grüßen
Im Auftrag

Benjamin Schepers


Luftbild und Geoinformationssysteme
Kronprinzenstraße 6
45128 Essen
Fon: +49 201 2069-232
Fax: +49 201 2069-369
schep...@rvr-online.de

[cid:image001.jpg@01D26B4F.354A0EC0]

Die Regionaldirektorin
Kronprinzenstraße 35
45128 Essen
Zentrale: +49 (0) 201 2069-0
Fax: +49 (0) 201 2069-500
www.metropoleruhr.de

Postfach 10 32 64
45032 Essen

Steuernummer: RVR 112/5797/0116
USt.-ldNr.: DE 173867500

Diese E-Mail koennte vertrauliche und/oder rechtlich geschuetzte Informationen 
enthalten.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich 
erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie 
diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail sind nicht 
gestattet.
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in error) 
please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Sushrut Shivaswamy
Gesendet: Dienstag, 10. Januar 2017 14:34
An: mapserver-users@lists.osgeo.org
Betreff: [mapserver-users] WMS T time series data support

Hey,

I'm trying to visualise my time series data set using map server.
I have a series of files with file names in the format 
{coveragename}_{timestamp}.tif.

I need to query a single layer composed of the data from all these files and
see the data by varying the TIME parameter in my GetMap request.

GeoServer provides a tutorial on how to setup an Image Mosaic as time series 
layer.

Is there way to get the same effect in MapServer.

Regards,
Sushrut Shivaswamy
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] mapserver-users Digest, Vol 108, Issue 12

[Schepers, Benjamin]

Hi Lowell,

thank you for your response.

< For the oblique images, we use the ground footprint layer to populate a DB 
table with the four corners of each image and a calculated center for each 
shot. That table is used as a match against the center of the current map 
extent to locate the oblique that is the "closet" as a view starting point.
OK so far... could be done in a lot of ways.

 Date: Mon, 9 Jan 2017 06:37:44 +0000
> From: "Schepers, Benjamin" <schep...@rvr-online.de>
> To: "'mapserver-users@lists.osgeo.org'"
>   <mapserver-users@lists.osgeo.org>
> Subject: [mapserver-users] Oblique aerial images with mapserver?
> Message-ID:
>   <7088A26751CB34409B159B4237D510A87C878D7C@W2K8-EXDB02.VERBAND.LOCAL>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> Hi list,
> 
> are there any suggestions or concepts out there to serve oblique 
> aerial images with mapserver / opensource-software?
> The oblique images are not mosaicked and are just indirectly 
> georeferenced. There do exist ground-footprints as shape with path to 
> each file - comparable to a tileindex.
> 
> One way I thought of, could be mosaicking each direction (N-E-S-W) and 
> add them as separate layers to the service... (I know the results look 
> a bit strange and I also don't know such software for mosaicking - 
> Google's/Bing's secret? Are there any open solutions for that?) 
> Another idea could be to use a "special tileindex mode" for each 
> direction, where just the most matching oblique will be rendered from 
> footprint-tileindex.
> Would you suggest a real 3D-solution? How to serve that?
> 
> Any more hints are welcome :)
> ... and a happy new year!
> 
> Regards
> Ben

Ben,
For the oblique images, we use the ground footprint layer to populate a DB 
table with the four corners of each image and a calculated center for each 
shot. That table is used as a match against the center of the current map 
extent to locate the oblique that is the "closet" as a view starting point.
We then use Imagemagick for three different operations to merge the oblique and 
the map image. Rotate the current map image to match the orientation of the 
oblique image. Distort (-disort Perspective) to stretch the map image to match 
the perspective of the oblique. Composite to merge the map image and oblique 
into a single image.
The navigation is not fancy, there are pan arrows provided to allow the user to 
move to the next oblique to the N,S,E,W of the current oblique and a compass 
rose to allow the user to change the oblique perspective N,S,E,W. With the pan 
arrows and compass rotated to match whatever perspective is being viewed.
We have yet to investigate the methods used in Birds-eye to see how that works. 
We expect that the plugin provided by the oblique vendor is being used.
Hope that helps.

Lowell
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

[mapserver-users] Oblique aerial images with mapserver?

[Schepers, Benjamin]

Hi list,

are there any suggestions or concepts out there to serve oblique aerial images 
with mapserver / opensource-software?
The oblique images are not mosaicked and are just indirectly  georeferenced. 
There do exist ground-footprints as shape with path to each file - comparable 
to a tileindex.

One way I thought of, could be mosaicking each direction (N-E-S-W) and add them 
as separate layers to the service... (I know the results look a bit strange and 
I also don't know such software for mosaicking - Google's/Bing's secret? Are 
there any open solutions for that?)
Another idea could be to use a "special tileindex mode" for each direction, 
where just the most matching oblique will be rendered from footprint-tileindex.
Would you suggest a real 3D-solution? How to serve that?

Any more hints are welcome :)
... and a happy new year!

Regards
Ben

Mit freundlichen Grüßen
Im Auftrag

Benjamin Schepers


Luftbild und Geoinformationssysteme
Kronprinzenstraße 6
45128 Essen
Fon: +49 201 2069-232
Fax: +49 201 2069-369
schep...@rvr-online.de

[cid:image001.jpg@01D26A49.07CD0800]

Die Regionaldirektorin
Kronprinzenstraße 35
45128 Essen
Zentrale: +49 (0) 201 2069-0
Fax: +49 (0) 201 2069-500
www.metropoleruhr.de

Postfach 10 32 64
45032 Essen

Steuernummer: RVR 112/5797/0116
USt.-ldNr.: DE 173867500

Diese E-Mail koennte vertrauliche und/oder rechtlich geschuetzte Informationen 
enthalten.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich 
erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie 
diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail sind nicht 
gestattet.
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in error) 
please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] Speeding up WFS paging with Spatialite and GeoPackage

[Schepers, Benjamin]

Hi all,

to extend Jukkas Question I'd like to use predefined queries (WFS 2.0) to break 
large datasets into smaller pieces. A common usecase could be to deliver 
predefined queries, where users could select a value from a list (simple 
example:name of city where data belongs to...) which will be used for 
filtering. In the docs I didn't find a solution to provide value-lists, just 
variablename and datatype, for example:


Is it possible to provide restrictions such as ranges or enumerations as 
documented there:
http://www.w3schools.com/xml/schema_facets.asp

And am I right that this would also be an client-side problem to evaluate those 
restrictions?


Mit freundlichen Grüßen
Im Auftrag

Benjamin Schepers


Luftbild und Geoinformationssysteme
Kronprinzenstraße 6
45128 Essen
Fon: +49 201 2069-232
Fax: +49 201 2069-500
schep...@rvr-online.de

[cid:image001.jpg@01D25AB6.936B34B0]

Die Regionaldirektorin
Kronprinzenstraße 35
45128 Essen
Zentrale: +49 (0) 201 2069-0
Fax: +49 (0) 201 2069-500
www.metropoleruhr.de

Postfach 10 32 64
45032 Essen

Steuernummer: RVR 112/5797/0116
USt.-ldNr.: DE 173867500

Diese E-Mail koennte vertrauliche und/oder rechtlich geschuetzte Informationen 
enthalten.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich 
erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie 
diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail sind nicht 
gestattet.
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in error) 
please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Rahkonen Jukka (MML)
Gesendet: Dienstag, 20. Dezember 2016 11:37
An: Mapserver-Users (mapserver-users@lists.osgeo.org)
Betreff: [mapserver-users] Speeding up WFS paging with Spatialite and GeoPackage

Hi,

It seems that because of how SQLite works it is not possible to make fast 
paging for big tables by using simple SQL with LIMIT and OFFSET.  Making a 
query as "LIMIT 100 OFFSET 100" makes SQLite to read 1000100 rows and throw 
away the first million ones. The bigger the offset the slower the query.

I guess that Mapserver must still use LIMIT and OFFSET because they work even 
with combined attribute and spatial filters. But I wonder if it could be 
possible to make a WFS 2.0 stored query for a not so uncommon use case of 
loading pages without other filters from very big tables. This stored query 
would also take count and startindex as inputs but place them into a query like

SELECT * FROM table WHERE rowid>startindex AND rowid<=(startindex+count)

An optional parameter could be used for defining some other unique, numeric and 
indexed attribute for paging. This query should work very fine with a SQLite 
table that is freshly  created for example with ogr2ogr because then rowids 
start from one and they do not have gaps. If table has been edited there may be 
missing rowids but that is not critical because query would just return 
sometimes less rows than "count".

Unfortunately by reading 
http://mapserver.org/ogc/wfs_server.html#stored-queries-wfs-2-0 it is not 
obvious for me how to write such "GetPageByRowid" query and I would appreciate 
getting some help.

-Jukka Rahkonen-





___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] New Raster MapServer Tutorial

[Schepers, Benjamin]

Hi Seth,

nice work!

To your questions:

-  PNG24/32: lossless, so no quality-parameter, just compression 
efficiency (comparable to ZIP-Compression; size vs. needed time) [FORMATOPTION 
“ZLEVEL=n”]

-  PNG8: also lossless compression, but with color-depth-reduction 
(quantization), if you have data with more than 256 colors – this can/will  be 
lossy; also holds transparency-information (look-up-table for defined colors 
can have rgba-values)

-  JPEG: IMHO the optimum for base-layers because of it’s small size 
and good performance (at least, if the base layer is not optimized with reduced 
color – png8); optimum for aerial images and “noisy” data; quality can be 
controlled as show downwards…

-  GeoTIFF: good solution, if you’d like to use the wms as 
“download-service” – images then hold complete GeoTIFF-Header (was buggy with 
older mapserver/GDAL-versions, but seems OK for now)

Transparency can be controlled in different manners (in mapfile or in original 
data), at least these, please correct me:

Mapfile:

-  LAYER-MASK (polygon-layer, outer area is transparent)

-  LAYER-OFFSITE (mapfile-NoData-value)


Data-side:

-  NoData-value (e.g. GTIFF-Header)

-  Alpha/Transparency (e.g. GTIFF – per Pixel)

-  Mask-layer inside data (separate GDAL-special pixel-mask-layer 
inside GTIFF for full transparency; useful in combination with GTIFF inside 
JPEG-compression, because JPEG can’t hold alpha-channel and NoData doesn’t work 
calculable caused by lossy compression)

If compiling GDAL and mapserver by your own, then using libjpeg-turbo instead 
of libjpeg brings some extra pieces of performance, specially when dealing with 
large amounts of image-data.

You can define/modify default-formats and add special GDAL compression-options 
with ‘ FORMATOPTION “KEY=VALUE” ’, do not forget to add format to formatlist:

http://pastebin.com/1YztvEcw


Ben

Luftbild und Geoinformationssysteme
Kronprinzenstraße 6
45128 Essen
Fon: +49 201 2069-232
Fax: +49 201 2069-500
schep...@rvr-online.de

Die Regionaldirektorin
Kronprinzenstraße 35
45128 Essen
Zentrale: +49 (0) 201 2069-0
Fax: +49 (0) 201 2069-500
www.metropoleruhr.de

Postfach 10 32 64
45032 Essen

Steuernummer: RVR 112/5797/0116
USt.-ldNr.: DE 173867500
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] GetFeatureInfo layout in plain text

[Schepers, Benjamin]

Hi @all,

thank you for that inspiration, Steve! This shows how valuable this list is...
And sorry, Yves, for giving a wrong hint; but HTML-GFI within ArcGIS 10.2+ 
works, but be careful with embedded javascript and advanced css in 
html-template, ArcGIS doesn't like that much.

Some thoughts on the Template-thing from Steve/Daniel:
Regarding to Daniels reference to wms-server-documentation 
(http://mapserver.org/ogc/wms_server.html#how-does-a-wms-work), GFI-section:
"text/html output using MapServer query templates (see Templating) specified in 
the CLASS TEMPLATE parameter (the filename has to have an .html extension). The 
MIME type returned by the Class templates defaults to text/html and can be 
controlled using the metadata “wms_feature_info_mime_type”.

It looks like there's a "natural" binding between template and html output. As 
Daniel stated, this can be overridden with setting a other " 
wms_feature_info_mime_type" - OK so far.
For example someone would now be using the template for individual text-plain 
output-styling and setting " 'wms_feature_info_mime_type' 'text/plain' ", what 
happens then, if a user requests the FeatureInfo in text/html, which is from my 
point of view the typical format? Mapserver wouldn't produce a valid html 
response anymore (because of the lack of an valid html-template), would it?
Should GFI with HTML then be forbidden or are multiple templates for different 
outputformats possible? How would this work?

Maybe I didn't fully understand Steve's hint and one of you can explain?

Thanks a lot,

Ben

-Ursprüngliche Nachricht-
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Lime, Steve D (MNIT)
Gesendet: Donnerstag, 10. November 2016 19:51
An: Yves Jacolin; mapserver-users@lists.osgeo.org
Betreff: Re: [mapserver-users] GetFeatureInfo layout in plain text

This should be possible. Templates are not limited to HTML output, any text 
based format should be able to be created. You do this through output formats. 
Here's an example for plain text:

 OUTPUTFORMAT
NAME 'plain-text'
DRIVER 'TEMPLATE'
MIMETYPE 'text/plain'
FORMATOPTION "FILE=templates/plain.tmpl"
FORMATOPTION 'ATTACHMENT=myfile.txt'
  END

The template file's extension is really irrelevant to what's being delivered 
back to the client - the mimetype sets that. I don't recall of the top of my 
head how you expose this type via WMS but I think that's pretty straight 
forward.

Steve

-Original Message-
From: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] On 
Behalf Of Yves Jacolin
Sent: Thursday, November 10, 2016 4:16 AM
To: mapserver-users@lists.osgeo.org
Subject: [mapserver-users] GetFeatureInfo layout in plain text

Hello,

I have to setup a text format for a WMS' GetFeatureInfo service. I would like 
to know if it is possible to change the layout using a template.

My first test was not working so any feedback is welcome. The documentation 
said that the user should read the templating chapter so I guess this is 
possible.

Any confirmation?

Y.
--
Responsable Formation et Support
Camptocamp France SAS
Savoie Technolac, BP 352
73377 Le Bourget du Lac, Cedex
Tel (France) : +33 4 58 48 20 43 (new !) Tel (Suisse) : +41 21 619 10 43 Mob. : 
+33 6 18 75 42 21 Fax : 04 79 70 15 81 Mail : yves.jaco...@camptocamp.com 
http://www.camptocamp.com ___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] GetFeatureInfo layout in plain text

[Schepers, Benjamin]

Hi Yves,



from my point of view this isn't possible, templating in mapserver-WMS just 
works for HTML.

You would need an wrapper-script, which does some postprocessing/transforming 
with GFI-response (text) from mapserver - could be done with 
bash/perl/python/php whatever, maybe mapscript-engine (I'm not into this) or 
within Apache-Webserver (mod_sed ;-) ).



Most structured way would be rewriting GFI-request from text-format to 
gml-format within the receiving webserver, directing this request to mapserver 
(which would create gml-response), directing mapserver-response to 
xml-processor and transform to whatever you want and then let webserver deliver 
the response to the client...



Edit:

Jörg was faster, because of having a phonecall from one of his coworkers ;-) 
and also my post was blocked by the mailing-list caused by an attachment ☹ 
,sorry, Jeff.

ArcGIS Desktop 10.x does show html-GFI, see attachment (image is clipped, it is 
a screenshot from within ArcGIS):

http://pasteboard.co/pANcOnMbD.jpg



Ben


Mit freundlichen Grüßen
Im Auftrag

Benjamin Schepers


Luftbild und Geoinformationssysteme
Kronprinzenstraße 6
45128 Essen
Fon: +49 201 2069-232
Fax: +49 201 2069-500
schep...@rvr-online.de

[cid:image001.jpg@01D23B71.BDC6EEC0]

Die Regionaldirektorin
Kronprinzenstraße 35
45128 Essen
Zentrale: +49 (0) 201 2069-0
Fax: +49 (0) 201 2069-500
www.metropoleruhr.de

Postfach 10 32 64
45032 Essen

Steuernummer: RVR 112/5797/0116
USt.-ldNr.: DE 173867500

Diese E-Mail koennte vertrauliche und/oder rechtlich geschuetzte Informationen 
enthalten.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich 
erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie 
diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail sind nicht 
gestattet.
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in error) 
please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Yves Jacolin
Gesendet: Donnerstag, 10. November 2016 13:03
An: Jörg Thomsen (WhereGroup)
Cc: mapserver-users
Betreff: Re: [mapserver-users] GetFeatureInfo layout in plain text

humm, good point, I need to check the output request from ArcGIS. The users 
said that for other service they get a table, so probably an html format.
Thanks,
Y.

2016-11-10 12:59 GMT+01:00 Jörg Thomsen (WhereGroup) 
>:
so, what do you have to style? Perhaps you could create a text-format
output with templates to create a sort of ascii-tables, you don't have
to use html-tags within the templates. But in this case the request has
to be format=text/html.

Jörg


Am 10.11.2016 um 12:39 schrieb Yves Jacolin:
> Thanks Jörg,
>
> Unfortunately I should use text format as ArcGIS desktop don't support other
> format (to be confirmed).
>
> Y.
> On Thursday, November 10, 2016 11:49:49 Jörg Thomsen wrote:
>> Hi Yves,
>>
>> text/plain is text/plain, no layout. The Mapserver-templates work with
>> html-format. Another way could be using the xml-output and styling it
>> with xslt. But you can enable the html-output using templates with
>> mapserver and it still will be possible to send text/plain requests.
>>
>> Templating ist easy and powerful, so to answer your question: yes, read
>> the templating chapter. You'll find a short introduction in the old
>> tutorial: http://demo.mapserver.org/tutorial/section3.html (example 3.1)
>>
>> Jörg
>>
>> Am 10.11.2016 um 11:16 schrieb Yves Jacolin:
>>> Hello,
>>>
>>> I have to setup a text format for a WMS' GetFeatureInfo service. I would
>>> like to know if it is possible to change the layout using a template.
>>>
>>> My first test was not working so any feedback is welcome. The
>>> documentation
>>> said that the user should read the templating chapter so I guess this is
>>> possible.
>>>
>>> Any confirmation?
>>>
>>> Y.
>>
>> Viele Grüße,
>> Jörg Thomsen
>


Viele Grüße,
Jörg Thomsen

--

Where2B Konferenz 2016
15. Dezember 2016 in Bonn
www.where2b-conference.com


---
Jörg Thomsen
WhereGroup GmbH & Co. KG
Gillweg 3
14193 Berlin
Germany

Fon: +49 (0)30 / 5130 278 74
Fax: +49 (0)30 / 89 09 53 11

joerg.thom...@wheregroup.com
www.wheregroup.com
Amtsgericht Bonn, HRA 6788
---
Komplementärin:
WhereGroup Verwaltungs GmbH
vertreten durch:
Olaf Knopp, Peter Stamm
---
Folgen Sie der 

Re: [mapserver-users] GetFeatureInfo request for multiple times on tileindex raster layer

[Schepers, Benjamin]

Hi Mike,

just my thoughts:
What would be the result of an getmap-Request in your second scenario with 
multiple discrete timestamps?
A getmap-request just can produce one single image – anything else wouldn’t 
make sense to me. So if you request two discrete timestamps, one would expect 
to get two rasters from DB/filesystem and there has to be a logic what should 
be done by mapserver with those two images (choose the topmost or the 
underlying or a merge of them – which merge method? – of those pics to get one 
of two or many)…
Maybe the latest date, as you determined, is the topmost.

My thought what getfeatureinfo logic does are equivalent to that…
Maybe I’m wrong with that, but that could just be answered by someone who can 
read the sourcecode?!

If you want to merge rasters on the fly (or their values) COMPOSITE maybe could 
help you: http://mapserver.org/de/mapfile/composite.html

Sincerely
Ben



Benjamin Schepers


Luftbild und Geoinformationssysteme
Kronprinzenstraße 35
45128 Essen
Fon: +49 201 2069-232
Fax: +49 201 2069-500
schep...@rvr-online.de

[cid:image001.jpg@01D1EEF2.E97E2630]

Die Regionaldirektorin
Kronprinzenstraße 35
45128 Essen
Zentrale: +49 (0) 201 2069-0
Fax: +49 (0) 201 2069-500
www.metropoleruhr.de

Postfach 10 32 64
45032 Essen

Steuernummer: RVR 112/5797/0116
USt.-ldNr.: DE 173867500

Diese E-Mail koennte vertrauliche und/oder rechtlich geschuetzte Informationen 
enthalten.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich 
erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie 
diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail sind nicht 
gestattet.
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in error) 
please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Michael Billmire
Gesendet: Donnerstag, 4. August 2016 21:53
An: mapserver-users@lists.osgeo.org
Betreff: [mapserver-users] GetFeatureInfo request for multiple times on 
tileindex raster layer

Hi all,

I'm serving raster layers as time-series using a PostGIS tile index 
(GetCapabilities,
 for reference).

GetFeatureInfo is working fine for single timestamp requests, e.g.:
http://spatial.mtri.org/cgi-bin/mapserv.fcgi?SERVICE=WMS=1.1.1=GetFeatureInfo=watersensing_map_LAYERS=chl_erie=chl_erie=2015-09-06=512=512=EPSG:4326=-83.55,39.45,-78.45,44.55=1_FORMAT=text/html=250=250

However, when requesting multiple discrete time stamps (e.g. 
time=2015-09-06,2016-05-23), I get multiple results (i.e. a result for each 
timestamp), BUT the pixel value returned is the same for each result, 
representing the raster with the latest date:
http://spatial.mtri.org/cgi-bin/mapserv.fcgi?SERVICE=WMS=1.1.1=GetFeatureInfo=watersensing_map_LAYERS=chl_erie=chl_erie=2015-09-06,2016-05-23=512=512=EPSG:4326=-83.55,39.45,-78.45,44.55=1_FORMAT=text/html=250=250

In my template, [value_list] returns just the single value (i.e. the value of 
the raster at the latest date), [value_0] returns that same value, and 
[value_1] returns nothing.

Is there anything obvious here that I'm doing wrong? Is this type of multiple 
timestamp pixel querying even supported?

thanks!
-Mike


--
Michael Billmire, CMS-GIS/LIS
Research Scientist
Michigan Tech Research Institute (MTRI)
3600 Green Ct. Suite 100
Ann Arbor, MI 48105

michael.billm...@mtu.edu
work: 734.913.6853
cell: 513.739.0686
fax: 734.913.6880
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] Map file, Label WRAP Function in mapbook removes character it

[Schepers, Benjamin]

Hi Carrie,

in my opinion the easiest way would be converting the data to spatialite-DB 
(you get a lot of database-functions in a file based-DB - no 
client-server-architecture, just a file; at best you use ogr2ogr or QGIS for 
conversion) and then use replace function in your SQL-QUERY

i.e. to dynamically insert pipe "|" as wrap-character it should look like that 
(fieldname is "parcel_no"):
SELECT *, replace(parcel_no, '.', '.|') AS parcel_no_wrap FROM parcel_table

If you have a shapefile and your parcel-no have a strict pattern (ie. 
2digits.3digits.4digits) then it should also be possible to use OGR generic 
"concat" and "substring"-function, which were introduced with GDAL 1.8 
(according to documentation at 
http://www.gdal.org/ogr_sql.html#ogr_sql_flist_ops and 
http://www.gdal.org/ogr_sql.html#ogr_sql_functions):

Example, also with pipe as wrap-character:
SELECT *, 
CONCAT(SUBSTR(parcel_no,1,2),'.|',SUBSTR(parcel_no,4,3),'.|',SUBSTR(parcel_no,8,4))
 AS parcel_no_wrap FROM parcel_table

In both examples the new virtual field "parcel_no_wrap" could be used for 
displaying text with wrap character...

Sincerely,
Benjamin Schepers



Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Mark Volz
Gesendet: Donnerstag, 30. Juni 2016 15:09
An: mapserver-users@lists.osgeo.org
Betreff: Re: [mapserver-users] Map file, Label WRAP Function in mapbook removes 
character it

Carrie,

Our trick is to wrap on a character that you would otherwise not normally see 
in the string.

LAYER

CLASS 
TEXT ([LastName] * [PIN])  #overrides LABELITEM
.
LABEL
WRAP "*"
END #Parcel Label
END #Parcel Class
END #Parcel Layer


Sincerely,
Mark Volz, GISP

-Original Message-
From: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] On 
Behalf Of mapserver-users-requ...@lists.osgeo.org
Sent: Wednesday, June 29, 2016 11:04 AM
To: mapserver-users@lists.osgeo.org
Subject: mapserver-users Digest, Vol 101, Issue 37

Send mapserver-users mailing list submissions to
mapserver-users@lists.osgeo.org

To subscribe or unsubscribe via the World Wide Web, visit
http://lists.osgeo.org/mailman/listinfo/mapserver-users
or, via email, send a message with subject or body 'help' to
mapserver-users-requ...@lists.osgeo.org

You can reach the person managing the list at
mapserver-users-ow...@lists.osgeo.org

When replying, please edit your Subject line so it is more specific than "Re: 
Contents of mapserver-users digest..."


Today's Topics:

   1. Map file, Label WRAP Function in mapbook removes character it
  is wrapping on. (Carrie Quast)


--

Message: 1
Date: Wed, 29 Jun 2016 15:58:59 +
From: Carrie Quast 
To: "'mapserver-users@lists.osgeo.org'"

Subject: [mapserver-users] Map file, Label WRAP Function in mapbook
removes character it is wrapping on.
Message-ID: <77b9eefcc9264bddb77bd9a85bd03...@msexchange1.bc.local>
Content-Type: text/plain; charset="utf-8"

Hello - I'm working on updating my .MAP Documents to Mapserver 7.0.1.

I'm trying to get the labels in my Parcel Layer to wrap on the decimal point 
(example 002.001.001.01.010) to make for a better a fit.  I'm using the WRAP 
function, which works wonderfully , except it removes the character I am 
wrapping on.  In this case, the point (.) In my parcel numbers.

Each of the numbers should have a decimal between it.   Any idea of how to get 
the line to wrap on a character, but also keep it in the displayed label?  The 
MAXLENGTH works well too, but still removes all decimal points.

I'm using Mapserver 7.0.1 with GeoMoose 2.9

Thank You - Carrie Quast, Brown County MN GIS Specialist

[cid:image002.png@01D1D1F5.3D5294F0]


-- next part --
An HTML attachment was scrubbed...
URL: 

-- next part --
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 5 bytes
Desc: image001.png
URL: 

-- next part --
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 31355 bytes
Desc: image002.png
URL: 


--

Subject: Digest Footer

___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

--

End of mapserver-users Digest, Vol 101, Issue 37

Re: [mapserver-users] MapServer Tile Indexes

[Schepers, Benjamin]

Hi all,

another try for troubleshooting could be building a VRT (use gdalbuildvrt - it 
produces some kind of "advanced raster-catalogue"). If you have faulty data, 
there would be a chance, that you get some error-messages, which dataset 
produces problems...

This VRT - if everything went fine - could also be the source to build a large 
TIFF-Dataset, similar to what Jukka recommended.
Some good compression-parameters for elevation-data in large GeoTIFF would be 
to use gdal_translate with "-of GTiff -ot Float32 -co BIGTIFF=YES -co 
COMPRESS=DEFLATE -co TILED=YES" (if you use GDAL 2.1 "-co NUM_THREADS=ALL_CPUS" 
will speed up this process, as it uses all cpu-cores).

You should also think about overviews for small scales. Interesting post from 
Paul Ramsey:
http://blog.cleverelephant.ca/2015/02/geotiff-compression-for-dummies.html - 
This is about compressing Aerial Images, but it points you into the right 
direction (remind my compression parameters above for DEM-data).

Such compressed dataset will be presented by mapserver amazingly fast. I've got 
a dataset with similar dimensions but 10m res which is about 100MB --> plain 
upscaling to your resolution would produce a dataset about 10-15GB, but this 
depends on your data...


Cheers
Benjamin

Luftbild und Geoinformationssysteme
Regionalverband Ruhr
Kronprinzenstraße 35
45128 Essen
Germany
luftbilder.geoportal.ruhr


-Ursprüngliche Nachricht-
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Jeff McKenna
Gesendet: Dienstag, 21. Juni 2016 13:13
An: mapserver-users@lists.osgeo.org
Betreff: Re: [mapserver-users] MapServer Tile Indexes

Hello,

MapServer should definitely be able to handle your dataset, it is not large.

If I was in your shoes, here is what I would do:

- try to see more debug information (follow the steps and examples in 
http://www.mapserver.org/optimization/debugging.html)
   -- set in your mapfile: CONFIG "CPL_DEBUG" "ON"
   -- shp2img -m my.map -o test.png -all_debug 5
   -- mapserv -nh "QUERY_STRING=map=c:/ms4w/apps/my.map=map"

- it does sound like you have a faulty GRID file.  Start from your 
single working tileindex, then add in maybe 5 more rasters into your 
index and retry; if it works, add in another 5 etc.

Sometimes the best way to solve this is the hard way.

-jeff


-- 
Jeff McKenna
MapServer Consulting and Training Services
http://www.gatewaygeomatics.com/



On 2016-06-21 6:12 AM, Wright, Simon M. wrote:
> Dear All
>
>
>
> I’m trying to use the MapServer Tile Indexes to create a map service,
> and ultimately a WMS, of some DSM data but am getting an Internal Server
> Error.
>
>
>
> The DSM data are organised as 466 5 × 5 km rasters with a resolution of
> 1 × 1 m with a total size of 69.2 GB.  The data format is Esri ASCII GRID.
>
>
>
> To test the use of Tile Indexes in MapServer I used the gdaltindex to
> create an index shapefile for just twenty of the 5 × 5 km rasters and
> created a basic map file:
>
>
>
> MAP
>
>IMAGETYPEPNG
>
>EXTENT   24 10 25 15
>
>SIZE 400 2000
>
>PROJECTION
>
>   "init=epsg:27700"
>
>END
>
>LAYER
>
>   NAME  "tiled-rasters"
>
>   STATUSON
>
>   TILEINDEX
> "/datastore/b81071f2-85b3-4e31-8506-cabe899f989a/b81071f2-85b3-4e31-8506-cabe899f989a_copy/index_file"
>
>   TILEITEM  "location"
>
>   TYPE  RASTER
>
>   PROJECTION
>
>  "init=epsg:27700"
>
>   END
>
>END
>
> END
>
>
>
>
>
> When I request an image of the map with the URL
> http://localhost:8080/cgi-bin/mapserv?map=/vagrant/maps/tiled-rasters.map=tiled-rasters=map
> I get an “Internal Server Error”.
>
>
>
> Therefore I created a second index shapefile using the gdaltindex tool
> that only contained one of the 5 × 5 km rasters and updated the map file
> to point to the second index shapefile.  When I request an image of the
> map I can see an image of the DSM data as expected.
>
>
>
> Therefore, am I asking too much of the tile indexing approach in
> MapServer to work against a massive dataset – 69.2 GB in 466 raster
> datasets – or have MapServer users been able to get tile indexing too
> work in MapServer against large data volumes and can provide some guidance.
>
>
>
> Many thanks in advance of any help that MapServer users can offer.
>
>
>
>
>
>
>
> Best wishes, Simon.
>
>

___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] MAP file INCLUDE with wildcard

[Schepers, Benjamin]

Hi,

it doesn't work out of the box with typical wildcards as *?.

Maybe this could be done with some naming-conventions and a wrapper-script 
[http://mapserver.org/cgi/wrapper.html] adding something like "cat" with 
wildcards on "include-files" before mapserver execution.
i.e 
cat ways_*.layer > all_ways.include

"all_ways.include" would be referenced in your mapfile. With this method you 
could also realize user specific mapfiles or other kinds of differentiation.

but as stated on the above link, using a wrapper script is inefficient because 
of process-spawning... and I'm sure, sooner or later exhaustive use of this 
method will also lead to mapfile-chaos ;-)


Benjamin Schepers

Luftbild und Geoinformationssysteme
luftbilder.geoportal.ruhr
schep...@rvr-online.de



Kronprinzenstraße 35
45128 Essen
Germany
www.metropoleruhr.de
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] OGR csv - no field names

[Schepers, Benjamin]

Sorry list,

the mail sleeped in my drafts-folder, I see, that you have been further ;-)

Regards
Benjamin


Benjamin Schepers

Luftbild und Geoinformationssysteme
luftbilder.geoportal.ruhr
schep...@rvr-online.de



Kronprinzenstraße 35
45128 Essen
Germany
www.metropoleruhr.de

___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] Upgrading GDAL

[Schepers, Benjamin]

Hi Robert,

imho this should work, if you build the binaries by your own - first build 
GDAL, then Mapserver from the sources.
I assume, that there aren't any ready packages for your scenario, but I'm not 
using ubuntu, please correct me, if I'm wrong.
I did it vice versa (debian), first raised Mapserver to V7, then after some 
bugfix-releases GDAL to V2 - finally worked fine.

Regards
Benjamin


Mit freundlichen Grüßen
Im Auftrag

Benjamin Schepers


Luftbild und Geoinformationssysteme
Kronprinzenstraße 35
45128 Essen
Fon: +49 201 2069-232
Fax: +49 201 2069-400
schep...@rvr-online.de



Die Regionaldirektorin
Kronprinzenstraße 35
45128 Essen
Zentrale: +49 (0) 201 2069-0
Fax: +49 (0) 201 2069-500
www.metropoleruhr.de

Postfach 10 32 64
45032 Essen

Steuernummer: RVR 112/5797/0116
USt.-ldNr.: DE 173867500

Diese E-Mail koennte vertrauliche und/oder rechtlich geschuetzte Informationen 
enthalten.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich 
erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie 
diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail sind nicht 
gestattet.
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in error) 
please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.

-Ursprüngliche Nachricht-
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Robert Sanson
Gesendet: Dienstag, 26. April 2016 04:32
An: mapserver-users@lists.osgeo.org
Betreff: [mapserver-users] Upgrading GDAL

Hi

Is it fine to upgrade GDAL separately from Mapserver? I have Mapserver 6.4 and 
GDAL v 1.10 on a Ubuntu server. I wish to take advantage of some GDAL v 2 
improvements - specifically . Will anything break?

Many thanks,

Robert L. Sanson
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] OGR csv - no field names

[Schepers, Benjamin]

Hi Ian,

maybe creating a .vrt-File helps? This works as a preprocessor/transformer - 
the impact on performance for that kind of modification shouldn't be noticeable.
Mapserver can read those vrt-Files, so this could be a solution for you.
There is an example at the gdal-csv-driver documentation:
http://www.gdal.org/drv_csv.html


Regards
Benjamin

Benjamin Schepers

Luftbild und Geoinformationssysteme
luftbilder.geoportal.ruhr
schep...@rvr-online.de



Kronprinzenstraße 35
45128 Essen
Germany
www.metropoleruhr.de


-Ursprüngliche Nachricht-
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Ian Walberg
Gesendet: Sonntag, 24. April 2016 04:50
An: mapserver-users@lists.osgeo.org
Betreff: [mapserver-users] OGR csv - no field names

Folks,

Can a csv file with no field names as the first row be used as an OGR source?

GDAL creates dummy field names, are these passed to mapserver?

OGRFeature(USA-Hilton):652
  field_1 (String) = -74.16958
  field_2 (String) =  40.73526
  field_3 (String) =  Penn Station

What do we need in the layer to make this work?

LABELITEM 'name' - this throws an error, 'field_3' does not seem to work either.
CONNECTIONTYPE OGR
CONNECTION 'Stations.csv'

Thanks

Ian
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

[mapserver-users] UTF-Grid not working via cgi/fcgi?

[Schepers, Benjamin]

Hi @all,

has someone successfully realized an WMS with UTF-Grid-Layer? 
(http://mapserver.org/development/rfc/ms-rfc-93.html)

I use Mapserver 7.0.0 (and also tried the latest master) and wasn't able to get 
any output from the web-Service as WMS-Request with format "application/json".
The application/json-response has a filesize of 0Bytes, other formats 
(jpeg/png) work well - with exactly the same request, except the format.

UTF-Grid also seems to be produced well with a simulated request against 
mapserver on the commandline-interface
("/usr/local/bin/mapserv 
QUERY_STRING="SERVICE=WMS=1.3.0=GetMap=339444,5697190,347678,5702286=EPSG:25832=1556=963=poi
 ==application/json=TRUE=PATH_TO_MAPFILE" > 
OUTPUT")

Mapserver LOG (DEBUG 5) with wms request:
[Wed Mar  2 14:47:17 2016].978816 CGI Request 1 on process 23097
[Wed Mar  2 14:47:17 2016].978943 msDrawMap(): rendering using outputformat 
named UTFGRID (UTFGRID).
[Wed Mar  2 14:47:17 2016].978951 msDrawMap(): WMS/WFS set-up and query, 0.000s
[Wed Mar  2 14:47:18 2016].19127 msDrawMap(): Layer 0 (poi), 0.040s
[Wed Mar  2 14:47:18 2016].19143 msDrawMap(): Drawing Label Cache, 0.000s
[Wed Mar  2 14:47:18 2016].19146 msDrawMap() total time: 0.040s
[Wed Mar  2 14:47:18 2016].23178 msSaveImage(stdout) total time: 0.004s
[Wed Mar  2 14:47:18 2016].23191 mapserv request processing time (msLoadMap not 
incl.): 0.044s
[Wed Mar  2 14:47:18 2016].23194 msFreeMap(): freeing map at 0x51c1bf0.

--> I would say, this looks good, but no response on the client-side!

Some further investigation showed me, that the correct output is written to the 
apache error.log!!!
I'm not a developer of web application/cgi etc, but comparing this to writing 
shell scripts (which I do a lot), it seems, that the mapserver output for 
format "application/json" goes to "STDERR" (apache error.log) and not to 
"STDOUT" (client)

System is debian7, apache 2.2, mapserver 7/master (self compiled, as well as 
the geos-/gdal-/proj-stack).
Could someone crosscheck this feature, does it need special apache-config? Is 
it a misconfiguration of apache or a mapserver/apache-bug?

Any ideas out there?

Thank You very much

Ben



Benjamin Schepers

Luftbild und Geoinformationssysteme
schep...@rvr-online.de

[cid:image001.jpg@01D17473.3BF29630]

Kronprinzenstraße 35
45128 Essen
Germany
www.metropoleruhr.de

___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] Very poor WMS tiling performance with small area, ultra-high resolution image

[Schepers, Benjamin]

Hi,

you still pointed to the right answer, strategies like overviews and tiling are 
IMHO not scale-dependent. They depend on the pixel-dimension of the image and 
you have - let's say - a not so small one ;-)

So converting to GeoTIFF with internal "Tiling" could be the right strategy  
for you (http://www.gdal.org/frmt_gtiff.html;  use "BigTIFF" if files are 
expected to be larger than 4GB) --> hint: gdal_translate with "BIGTIFF=YES" and 
"TILED=YES" ... and of course Overviews (gdaladdo; also possible with 
BigTIFF-option).

Compression for original and overviews, could be lossless "LZW" or  lossy 
"JPEG-YCBCR" for example, this would also substantially squeeze down the size 
an d depends on your needs...

I have a lot of aerial images with up to 250.000*150.000px (GSD up to 6cm) in 
that manner and they are serving really, really fast via mapserver. Not every 
GIS-software is able to read those tiled and compressed BigTiffs directly (due 
to the lack of support for this file-format), but ArcGIS and QGIS do well (both 
use GDAL) and all others could still use the WMS served from mapserver!

Regards
Ben

Luftbild und Geoinformationssysteme
schep...@rvr-online.de

[cid:image001.jpg@01D167C6.4912F620]

Kronprinzenstraße 35
45128 Essen
Germany
www.metropoleruhr.de

Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Avi Blackmore
Gesendet: Freitag, 12. Februar 2016 23:28
An: mapserver-users@lists.osgeo.org
Betreff: [mapserver-users] Very poor WMS tiling performance with small area, 
ultra-high resolution image

Hello,

We've been using Mapserver as, among other things, a WMS server for 
some time with satellite and aerial imagery, without problems.  We've been able 
to serve imagery of resolutions down to .25 meters without any significant 
performance hit.

Today, we got a sample image from a new source, which has a 
resolution of 0.06 meters.  The image is 317 megabytes in size, and has 
dimensions of 17538 x 19035 pixels.  The file size is smaller than many of the 
Landsat or other images that we have displayed in the past.  However, we're 
finding that rendering this image is extremely expensive.  In particular, with 
tiled access, a la Google Maps, the WMS requests bog down our server to the 
point where all of its memory is consumed.

I know that overviews, tiling, and such, are good strategies for 
reducing the render time on a large area, reasonably high resolution image.  
But what would be good strategies for improving rendering time on this smaller, 
ultra-high resolution image?
--
Avi Blackmore
Head Programmer/System Administrator
Satshot Inc.

___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] File GDB performance vs Shapefile Performance

[Schepers, Benjamin]

I generally agree with Steve, shapes (with spatial index) are in my experience 
the fastest - with all their disadvantages in datamanagement.
It could be reasoned by the massive use of compression of attributes in 
filegdb-format. Comparing the  filesizes, filegdbs produce significant smaller 
sizes (maybe just 10% of a shape).
I often use sqlite/spatialite as "open" alternative to filegdb which seems to 
be a bit faster than fgdb and also has the benefits of a filebased db (as 
fgdb), maybe you should give it a try?

Regards

Ben

Luftbild und Geoinformationssysteme
Kronprinzenstraße 35
45128 Essen
schep...@rvr-online.de

[cid:image001.jpg@01D15DDE.0C871FC0]

Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Lime, Steve D (MNIT)
Gesendet: Montag, 1. Februar 2016 22:27
An: Mark Volz; mapserver-users@lists.osgeo.org
Betreff: Re: [mapserver-users] File GDB performance vs Shapefile Performance

You'll really just have to test side-by-side see. I've only done anecdotal 
testing with more moderate-size layers and found file geodatabases to be about 
twice as slow as a shapefile. Maybe others have done more rigorous benchmarking.

MapServer 7.0
ESRI FileGDB API 1.2
GDAL 1.11.2

Steve

From: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] On 
Behalf Of Mark Volz
Sent: Monday, February 01, 2016 2:39 PM
To: mapserver-users@lists.osgeo.org
Subject: [mapserver-users] File GDB performance vs Shapefile Performance

Hello,

My interactive mapping application is currently using Shapefiles to store the 
contour lines.  At 1.5 GB the Shapefiles are rather large.  Instead I am 
considering using a File Geodatabase which will only take up about 500 MB.

Is there a significant performance loss between Shapefiles and File 
Geodatabases?  And are there any performance tips and tricks for using File 
Geodatabases?

Thanks

Sincerely,
Mark Volz, GISP
Lyon County GIS Coordinator


___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] limit max execution time in cgi?

[Schepers, Benjamin]

Hi Mark,

that should be a setting in Apache-WebServer (Timeout-Directive), which 
executes mapserver as cgi/fcgi.

https://httpd.apache.org/docs/2.2/en/mod/core.html#timeout



Regards

Benjamin Schepers


Luftbild und Geoinformationssysteme
schep...@rvr-online.de

[cid:image001.jpg@01D154E4.C5E74600]

www.metropoleruhr.de


___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Re: [mapserver-users] WMS GetFeatureInfo no processed template on cluster layer

[Schepers, Benjamin]

I can’t confirm your solution, but maybe we’ve got different problems…
See my post a week ago: https://github.com/mapserver/mapserver/issues/5219
Maybe someone gets an idea, whether this is an configuration error?

Regards
Ben


Mit freundlichen Grüßen
Im Auftrag

Benjamin Schepers


Luftbild und Geoinformationssysteme
Kronprinzenstraße 35
45128 Essen
Fon: +49 201 2069-232
Fax: +49 201 2069-400
schep...@rvr-online.de

[cid:image001.jpg@01D14EA4.FF3261B0]

Die Regionaldirektorin
Kronprinzenstraße 35
45128 Essen
Zentrale: +49 (0) 201 2069-0
Fax: +49 (0) 201 2069-500
www.metropoleruhr.de

Postfach 10 32 64
45032 Essen

Steuernummer: RVR 112/5797/0116
USt.-ldNr.: DE 173867500

Diese E-Mail koennte vertrauliche und/oder rechtlich geschuetzte Informationen 
enthalten.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich 
erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie 
diese Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail sind nicht 
gestattet.
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in error) 
please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material in this 
e-mail is strictly forbidden.
Von: mapserver-users [mailto:mapserver-users-boun...@lists.osgeo.org] Im 
Auftrag von Carlos Ruiz
Gesendet: Mittwoch, 13. Januar 2016 20:48
An: MapserverList OSGEO
Betreff: [mapserver-users] WMS GetFeatureInfo no processed template on cluster 
layer

Hi to all,

Maybe I'm missing something, but I cannot retreieve features' information when 
I have a CLUSTER tag within a WMS layer.

I could retrieve features' information before as text/html on an HTML template 
performing a GetFeatureInfo query, but after I have included CLUSTER for 
presentation purposes, the fields on the template are not substituted anymore. 
The field names are shown as they were written (i.e. [street_name]) on the 
template.

This can be a little bit obvious because CLUSTER can represent a lot of near 
features. However, I have tested to retrieve info over a cluster with just one 
feature and the result is the same.

Is there something that I'm missing or the query behavior on a cluster is just 
undefined ?

Thanks in advance
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users