Re: [MapServer-users] "Emergency" map server on Raspberry Pi - raster/PNG source in folder structure
Hi, On Wednesday, 12 October 2022 20:11:37 EEST Scott wrote: > This is an ideal scenario for serverless. > > Put your vector data in FlatGeoBuf (.fgb) format and your highest res > tiles as a COG tif. You can have those files on a local device or in the > cloud. > > Use leaflet with flatgeobuf, georaster-layer-for-leaflet and some modest > javascript. > > You're making this wy too difficult. No. - Distributing about 75GB of raster data to different devices, keeping it current, etc., is a nightmare. - Over half of the devices using the map data are mobile devices which neither have the necessary storage capacity, nor the flexibility to freely install stuff. - Leaflet is all well and good but there are applications already in use. Developing a leaflet-based application to replace all currently used geospatial applications is not an option. - Cloud is not an option because the primary purpose of the solution is to be public-infrastructure-independent and for emergency/off-grid use. The requirement is a local WMS/WFS server and mapserver is the minimum and apparently the easiest solution for that. -Stefan -- Sent from mobile ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [MapServer-users] "Emergency" map server on Raspberry Pi - raster/PNG source in folder structure
This is an ideal scenario for serverless. Put your vector data in FlatGeoBuf (.fgb) format and your highest res tiles as a COG tif. You can have those files on a local device or in the cloud. Use leaflet with flatgeobuf, georaster-layer-for-leaflet and some modest javascript. You're making this wy too difficult. On 10/12/22 04:55, Stefan Gofferje wrote: Hi all, Life (and COVID) hit, so I never got anywhere with updating my weather website and taking mapserver into use there. Currently I have another project - due to the geopolitical situation: I'm working on a mobile system to provide emergency/off-grid "tactical" communication and data. It's centered on a Raspberry Pi and includes things like Meshtastic and others. One of the things I would like to add is a WMS/WFS server to provide map data via local Wifi AP to connected devices. Topographical (and other) maps of whole Finland are available under CC4 from the Finnish Land Survey through the KAPSI non-profit. The maps are in 12kx12k PNGs in a folder structure by UTM grid. See here: http://kartat.kapsi.fi/files/peruskarttarasteri_jhs180/taustavari_korkeuskayrilla/1m/etrs89/png/ The data can by rsync'd down. I'm not clear how I would configure mapserver for that. Do I have to create one map with extent and such for each PNG tile or can I point mapserver to the root directory of the data and tell it to go through it recursively and get the data from the PGW sidecar files? -Stefan ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [MapServer-users] "Emergency" map server on Raspberry Pi - raster/PNG source in folder structure
I gave a talk about something similar a few years back at FOSS4G - the video is at https://vimeo.com/248675454 I used a Raspberry Pi, GeoServer and MapProxy but I can't see why MapServer would not work. It's perfectly feasible, though a little slow. A newer Pi4 would probably be fine though, especially with a SSD for the data and cache Ian On Wed, 12 Oct 2022 at 13:06, Stefan Gofferje wrote: > Hi all, > > Life (and COVID) hit, so I never got anywhere with updating my weather > website and taking mapserver into use there. > > Currently I have another project - due to the geopolitical situation: > I'm working on a mobile system to provide emergency/off-grid "tactical" > communication and data. It's centered on a Raspberry Pi and includes > things like Meshtastic and others. One of the things I would like to add > is a WMS/WFS server to provide map data via local Wifi AP to connected > devices. > > Topographical (and other) maps of whole Finland are available under CC4 > from the Finnish Land Survey through the KAPSI non-profit. The maps are > in 12kx12k PNGs in a folder structure by UTM grid. See here: > > http://kartat.kapsi.fi/files/peruskarttarasteri_jhs180/taustavari_korkeuskayrilla/1m/etrs89/png/ > The data can by rsync'd down. > I'm not clear how I would configure mapserver for that. Do I have to > create one map with extent and such for each PNG tile or can I point > mapserver to the root directory of the data and tell it to go through > it recursively and get the data from the PGW sidecar files? > > -Stefan > > -- > (o_ Stefan Gofferje| SCLT, MCP, CCSA > //\ Reg'd Linux User #247167 | VCP #2263 > V_/_ https://www.gofferje.net | https://www.saakeskus.fi > > ___ > MapServer-users mailing list > MapServer-users@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users > -- Ian Turton ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [MapServer-users] "Emergency" map server on Raspberry Pi - raster/PNG source in folder structure
You'll need to set up an index to all those PNG/PGW files. I've done this with a SHP file in the past as Sven described. There is more than on way to do this though. Let me know if you need a MapServer example, I can dig one out for you. Bobb My machine - - - PW19-S295-C024 > > > -Original Message- > > > From: Stefan Gofferje > > > Sent: Wednesday, October 12, 2022 10:53 AM > > > To: Basques, Bob (CI-StPaul) ; > > > mapserver-users@lists.osgeo.org > > > Subject: Re: [MapServer-users] "Emergency" map server on Raspberry Pi > > > - raster/PNG source in folder structure > > > > > > Think Before You Click: This email originated outside our organization. > > > > > > > > > Hi, > > > > > > On 10/12/22 17:35, Basques, Bob (CI-StPaul) wrote: > > > > This might help you get started: > > > > > > > > https://github.com/klassenjs/rpi- > > > workshop/blob/master/Workshop.org > > > > > > > > We've got 7 of these used in the filed now as standalone Webservers, > > > with Wifi access. No internet required. You just turn it on. You can > > > add > > > in a Battery and/or connect directly to a 12v power source like the > > > ignition switch for power from a vehicle, and it's always on. > > > > > > > > Real low power and depending on how big a card you use, a lot of data. > > > Our survey version has over 250k geolocated document scans available. > > > > > > > > This is a map based system, but it a whole Server Stack configuration, > > > so should give you a good head start. > > > > > > > > We ran this as a workshop at a OSGeo conference a few years back. > > > > > > Cool! Not very much detail about mapserver, though. I'm planning to > > > throw my stuff to Github too, once I got a little further. I'm a little > > > bit > > > more on the complicated side with 4G/5G and potential other IP over > > > radio being in the mix. Also looking at batman-adv for meshing and > > > testing some ideas for voice/radio over IP. I found a very nice CM4 > > > baseboard which is tiny and still has space for an 42mm NVME SSD, so > > > atm I'm planning for 256G storage in addition to the EMMC. > > > > > > -S > > > > > > -- > > > (o_ Stefan Gofferje| SCLT, MCP, CCSA > > > //\ Reg'd Linux User #247167 | VCP #2263 > > > V_/_ https://www.gofferje.net | https://www.saakeskus.fi ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [MapServer-users] "Emergency" map server on Raspberry Pi - raster/PNG source in folder structure
that's exactly what I thought;-) You can also do this with QGIS Am 12.10.2022 um 16:57 schrieb Stefan Gofferje: Hi, On 10/12/22 16:37, Sven Schroeter wrote: I don't know if I understood you correctly, but I think the solution for you would be an image catalog, right? Then you would have only one layer for the UTM maps. In the mapfile it would look like this, in the .dbf in the column "location" would be the paths to the png data. LAYER NAME "utm_data" STATUS ON MAXSCALE 8000 TILEINDEX "filesindex_png.shp" TILEITEM "location" TYPE RASTER PROCESSING "Resample=average" CLASS name "utm_data" END END Like here: https://mapserver.org/input/raster.html#rasters-and-tile-indexing ? I'm still a total newbie to mapserver and haven't had much time to tinker with it yet, unfortunately. But that looks easy enough. So, basically, I 'find' all pngs/pgws, move them from the subfolders into one folder and run gdaltindex over the folder. And that creates a shapefile which I use as data source? Let's try that. -S ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [MapServer-users] "Emergency" map server on Raspberry Pi - raster/PNG source in folder structure
Mapserver is in there, but in it's basic configuration. Maybe I can get the author to chime in on any new capabilities with the newer RPis, he's still up on them more than I am. Bobb My machine - - - PW19-S295-C024 > > > -Original Message- > > > From: Stefan Gofferje > > > Sent: Wednesday, October 12, 2022 10:53 AM > > > To: Basques, Bob (CI-StPaul) ; > > > mapserver-users@lists.osgeo.org > > > Subject: Re: [MapServer-users] "Emergency" map server on Raspberry Pi > > > - raster/PNG source in folder structure > > > > > > Think Before You Click: This email originated outside our organization. > > > > > > > > > Hi, > > > > > > On 10/12/22 17:35, Basques, Bob (CI-StPaul) wrote: > > > > This might help you get started: > > > > > > > > https://github.com/klassenjs/rpi- > > > workshop/blob/master/Workshop.org > > > > > > > > We've got 7 of these used in the filed now as standalone Webservers, > > > with Wifi access. No internet required. You just turn it on. You can > > > add > > > in a Battery and/or connect directly to a 12v power source like the > > > ignition switch for power from a vehicle, and it's always on. > > > > > > > > Real low power and depending on how big a card you use, a lot of data. > > > Our survey version has over 250k geolocated document scans available. > > > > > > > > This is a map based system, but it a whole Server Stack configuration, > > > so should give you a good head start. > > > > > > > > We ran this as a workshop at a OSGeo conference a few years back. > > > > > > Cool! Not very much detail about mapserver, though. I'm planning to > > > throw my stuff to Github too, once I got a little further. I'm a little > > > bit > > > more on the complicated side with 4G/5G and potential other IP over > > > radio being in the mix. Also looking at batman-adv for meshing and > > > testing some ideas for voice/radio over IP. I found a very nice CM4 > > > baseboard which is tiny and still has space for an 42mm NVME SSD, so > > > atm I'm planning for 256G storage in addition to the EMMC. > > > > > > -S > > > > > > -- > > > (o_ Stefan Gofferje| SCLT, MCP, CCSA > > > //\ Reg'd Linux User #247167 | VCP #2263 > > > V_/_ https://www.gofferje.net | https://www.saakeskus.fi ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [MapServer-users] "Emergency" map server on Raspberry Pi - raster/PNG source in folder structure
Hi, On 10/12/22 16:37, Sven Schroeter wrote: I don't know if I understood you correctly, but I think the solution for you would be an image catalog, right? Then you would have only one layer for the UTM maps. In the mapfile it would look like this, in the .dbf in the column "location" would be the paths to the png data. LAYER NAME "utm_data" STATUS ON MAXSCALE 8000 TILEINDEX "filesindex_png.shp" TILEITEM "location" TYPE RASTER PROCESSING "Resample=average" CLASS name "utm_data" END END Like here: https://mapserver.org/input/raster.html#rasters-and-tile-indexing ? I'm still a total newbie to mapserver and haven't had much time to tinker with it yet, unfortunately. But that looks easy enough. So, basically, I 'find' all pngs/pgws, move them from the subfolders into one folder and run gdaltindex over the folder. And that creates a shapefile which I use as data source? Let's try that. -S -- (o_ Stefan Gofferje| SCLT, MCP, CCSA //\ Reg'd Linux User #247167 | VCP #2263 V_/_ https://www.gofferje.net | https://www.saakeskus.fi ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [MapServer-users] "Emergency" map server on Raspberry Pi - raster/PNG source in folder structure
Hi, On 10/12/22 17:35, Basques, Bob (CI-StPaul) wrote: This might help you get started: https://github.com/klassenjs/rpi-workshop/blob/master/Workshop.org We've got 7 of these used in the filed now as standalone Webservers, with Wifi access. No internet required. You just turn it on. You can add in a Battery and/or connect directly to a 12v power source like the ignition switch for power from a vehicle, and it's always on. Real low power and depending on how big a card you use, a lot of data. Our survey version has over 250k geolocated document scans available. This is a map based system, but it a whole Server Stack configuration, so should give you a good head start. We ran this as a workshop at a OSGeo conference a few years back. Cool! Not very much detail about mapserver, though. I'm planning to throw my stuff to Github too, once I got a little further. I'm a little bit more on the complicated side with 4G/5G and potential other IP over radio being in the mix. Also looking at batman-adv for meshing and testing some ideas for voice/radio over IP. I found a very nice CM4 baseboard which is tiny and still has space for an 42mm NVME SSD, so atm I'm planning for 256G storage in addition to the EMMC. -S -- (o_ Stefan Gofferje| SCLT, MCP, CCSA //\ Reg'd Linux User #247167 | VCP #2263 V_/_ https://www.gofferje.net | https://www.saakeskus.fi ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [MapServer-users] "Emergency" map server on Raspberry Pi - raster/PNG source in folder structure
Hi Stefan, This might help you get started: https://github.com/klassenjs/rpi-workshop/blob/master/Workshop.org We've got 7 of these used in the filed now as standalone Webservers, with Wifi access. No internet required. You just turn it on. You can add in a Battery and/or connect directly to a 12v power source like the ignition switch for power from a vehicle, and it's always on. Real low power and depending on how big a card you use, a lot of data. Our survey version has over 250k geolocated document scans available. This is a map based system, but it a whole Server Stack configuration, so should give you a good head start. We ran this as a workshop at a OSGeo conference a few years back. Bobb My machine - - - PW19-S295-C024 > > > -Original Message- > > > From: MapServer-users > > > On Behalf Of Stefan Gofferje > > > Sent: Wednesday, October 12, 2022 6:55 AM > > > To: mapserver-users@lists.osgeo.org > > > Subject: [MapServer-users] "Emergency" map server on Raspberry Pi - > > > raster/PNG source in folder structure > > > > > > Think Before You Click: This email originated outside our organization. > > > > > > > > > Hi all, > > > > > > Life (and COVID) hit, so I never got anywhere with updating my weather > > > website and taking mapserver into use there. > > > > > > Currently I have another project - due to the geopolitical situation: > > > I'm working on a mobile system to provide emergency/off-grid "tactical" > > > communication and data. It's centered on a Raspberry Pi and includes > > > things like Meshtastic and others. One of the things I would like to add > > > is > > > a WMS/WFS server to provide map data via local Wifi AP to connected > > > devices. > > > > > > Topographical (and other) maps of whole Finland are available under CC4 > > > from the Finnish Land Survey through the KAPSI non-profit. The maps > > > are in 12kx12k PNGs in a folder structure by UTM grid. See here: > > > http://kartat.kapsi.fi/files/peruskarttarasteri_jhs180/taustavari_korkeus > > > kayrilla/1m/etrs89/png/ > > > The data can by rsync'd down. > > > I'm not clear how I would configure mapserver for that. Do I have to > > > create one map with extent and such for each PNG tile or can I point > > > mapserver to the root directory of the data and tell it to go through it > > > recursively and get the data from the PGW sidecar files? > > > > > > -Stefan > > > > > > -- > > > (o_ Stefan Gofferje| SCLT, MCP, CCSA > > > //\ Reg'd Linux User #247167 | VCP #2263 > > > V_/_ https://www.gofferje.net | https://www.saakeskus.fi > > > > > > ___ > > > MapServer-users mailing list > > > MapServer-users@lists.osgeo.org > > > https://lists.osgeo.org/mailman/listinfo/mapserver-users ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [MapServer-users] "Emergency" map server on Raspberry Pi - raster/PNG source in folder structure
Hi Stefan, I don't know if I understood you correctly, but I think the solution for you would be an image catalog, right? Then you would have only one layer for the UTM maps. In the mapfile it would look like this, in the .dbf in the column "location" would be the paths to the png data. LAYER NAME "utm_data" STATUS ON MAXSCALE 8000 TILEINDEX "filesindex_png.shp" TILEITEM "location" TYPE RASTER PROCESSING "Resample=average" CLASS name "utm_data" END END Greetings Sven Am 12.10.2022 um 12:55 schrieb Stefan Gofferje: Hi all, Life (and COVID) hit, so I never got anywhere with updating my weather website and taking mapserver into use there. Currently I have another project - due to the geopolitical situation: I'm working on a mobile system to provide emergency/off-grid "tactical" communication and data. It's centered on a Raspberry Pi and includes things like Meshtastic and others. One of the things I would like to add is a WMS/WFS server to provide map data via local Wifi AP to connected devices. Topographical (and other) maps of whole Finland are available under CC4 from the Finnish Land Survey through the KAPSI non-profit. The maps are in 12kx12k PNGs in a folder structure by UTM grid. See here: http://kartat.kapsi.fi/files/peruskarttarasteri_jhs180/taustavari_korkeuskayrilla/1m/etrs89/png/ The data can by rsync'd down. I'm not clear how I would configure mapserver for that. Do I have to create one map with extent and such for each PNG tile or can I point mapserver to the root directory of the data and tell it to go through it recursively and get the data from the PGW sidecar files? -Stefan ___ MapServer-users mailing list MapServer-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
