Re: [Geoserver-users] GeoServer without PostGIS, is it a good idea?
Hello, Peter. My own personal experience ... We started off connecting Geoserver to MS SQL Server tables to deliver KML to Google Earth. This was because SQL Server was a corporate direction, and consequently our data warehouse sits in SQL Server. I was aware that performance was worse than that of PostGIS but persisted for while, but then we had an upgrade to 64-bit SQL Server and performance halved. I then got approval to install PostGIS on the same servers that were running Geoserver. We use FME to replicate data from SQL Server to PostGIS on a nightly basis, but we are copying only the data that is required (eg. attributes required for styling and filtering) into PostGIS - so we are really just using PostGIS to hold simple 'layers' (in GIS-speak) rather than for a relational data structure. In some ways conceptually we are using PostGIS as an alternative to holding layers in shapefiles. When I explained it that way, the database people were happy. The advantage of replicating the data into the Geoserver/PostGIS system, is that you can trim it right down - it is lean and fit for purpose - so you should get good performance. Our systems have been running for about 5 years, and we never have to do anything much with PostGIS. Regards, David Collins Senior Geoscientist On Wed, Jun 24, 2015 at 9:23 PM, Peter Kovac wrote: > Dear GeoServer users, > > my company develops systems in meteorology, climatology, radiation > monitoring etc. It's mostly data collection, storing and later retrieval. > Our customers started to demand maps, recently. > > What we need is a map client serving customers' data, both rasters (e.g. > forecasts from GRIB files) and vector (e.g. various values measured by > stations from a national climatology network). We looked out for some > options and settled on a combination of a GeoServer & OpenLayers > integrated into our existing web interface. > > After several weeks of playing with GeoServer I got an impression that > GeoServer works great in combination with a local PostGIS instance. > PostGIS fits well into our scenario, too: it would contain metadata about > our rasters with time dimensions, and whole vector layers. > > The problem is we already have our own infrastructure which does all the > bookkeeping of all our datasets. E.g. we already store our raster data > somewhere in the filesystem and there is a table in our database which > tracks metadata (origin, time, etc.) about these rasters. For climatology > data, we use Oracle database and have our own schema which stores > information about weather stations (e.g. name, location) and current and > historical measured values. > > Because of this, my manager does not want to allow me to create a PostGIS > instance along with a GeoServer instance. His arguments is it's > unnecessary duplicity, it won't be consistent with our database unless we > invest heavily in syncing the two and it needs additional maintenance. His > original idea was that our system will provide data to a GeoServer > instance via some Web Service / WFS / whatever and GeoServer will just > "render" it. The problem is our system does not have a WFS interface > (yet?) and our existing interfaces are not OGC standardized. Of course, if > we implement WFS we won't need GeoServer (mostly). > > I tried to explain to him that we should treat the PostGIS store as some > kind of internal GeoServer store (just like various index files on disk - > he is a database guy and does not care about all those files created by > GeoServer - he cares about databases) and not try to mess with it in order > to speed up development (deadlines are tight, as usual). > > I need either better arguments in this debate or an alternative solution. > > I stumbled upon DataStore Development tutorial on GeoTools ( > http://docs.geotools.org/latest/tutorials/advanced/datastore.html ) which > looks promising, but I'm not quite sure if it's the right thing. I'll try > to implement a proof-of-concept to see if I'm able to convert our existing > database schema into a new Data Store type recognized by GeoSever. > > In the meantime I'd love to hear your opinion about the case. Is it common > to implement custom DataStores from existing databases or mirroring it in > PostGIS is the preferred way? Are there any other options? > > -- > Peter Kovac > IMS Programmer > MicroStep-MIS > peter.ko...@microstep-mis.sk > > > > -- > Monitor 25 network devices or servers for free with OpManager! > OpManager is web-based network management software that monitors > network devices and physical & virtual servers, alerts via email & sms > for fault. Monitor 25 devices for free with no restriction. Download now > http://ad.doubleclick.net/ddm/clk/292181274;119417398;o > ___ > Geoserver-users mailing list > Geoserver-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listi
Re: [Geoserver-users] GeoServer without PostGIS, is it a good idea?
Peter, your proposal to use a PostGIS instance on the same server as GeoServer is what I know as the Forward Cache Pattern: https://www.google.co.nz/search?q=forward%20cache%20pattern http://flylib.com/books/en/2.79.1.86/1/ Is your spatial data service read-only? How fresh does the data need to be? A forward cache is easiest to implement with read-only data where real-time updates are not required. In my experience, a forward cache is a great way to improve the quality of data from disparate sources. I always choose PostGIS as I have found it very easy to administer and it seems to run unattended without the regular human sacrifice required by Oracle. Advantages of a PostGIS forward cache: - Performance: local PostGIS is very fast, and it is much easier to diagnose and fix performance bottlenecks in the forward cache rather than in a network of sources - Indices: PostGIS can calculate indices for data sources that do not support them - Constraints: PostGIS can be configured to reject inconsistent data, for example, data in a shapefile may reference a feature in a database that no longer exists; a PostGIS forward cache constraint will tell you - Relationships: views/joins that calculate relationships between types from different sources; you can also deliver complex types with app-schema to make these relationships available to users - Security isolation, as scheduled tasks can push data into PostGIS in a DMZ server, with no need for incoming connections into your internal network - Confidentiality: data loading process can select only the subset approved for publication, reducing the risk of unwanted disclosures - Redundancy: GeoServer can keep on using PostGIS even when the Oracle or remote filesystems are down Yes, there is an additional cost in implementing a forward cache, but the benefits are substantial. I have seen several examples where insistence on using a bare corporate database (driven by managerial and DBA pressure) has resulted in poor outcomes for web service end users. If you are not permitted to use a forward cache and need to access your non-OGC back end web services, you might want to consider the unsupported webservice data store. The webservice data store was written to allow GeoServer to consume arbitrary SOAP web services as a back end for app-schema complex feature data stores (usual GeoServer OGC front end): https://www.seegrid.csiro.au/wiki/Infosrvices/GeoserverWebserviceUserGuide https://github.com/geotools/geotools/tree/master/modules/unsupported/app-schema/webservice https://github.com/geoserver/geoserver/tree/master/src/community/app-schema I do not know if there are any current users of the webservice data store. It is unsupported. You can run the unit tests by building GeoServer with the -Pwebservice profile. Kind regards, Ben. On 24/06/15 23:23, Peter Kovac wrote: > Dear GeoServer users, > > my company develops systems in meteorology, climatology, radiation > monitoring etc. It's mostly data collection, storing and later retrieval. > Our customers started to demand maps, recently. > > What we need is a map client serving customers' data, both rasters (e.g. > forecasts from GRIB files) and vector (e.g. various values measured by > stations from a national climatology network). We looked out for some > options and settled on a combination of a GeoServer & OpenLayers > integrated into our existing web interface. > > After several weeks of playing with GeoServer I got an impression that > GeoServer works great in combination with a local PostGIS instance. > PostGIS fits well into our scenario, too: it would contain metadata about > our rasters with time dimensions, and whole vector layers. > > The problem is we already have our own infrastructure which does all the > bookkeeping of all our datasets. E.g. we already store our raster data > somewhere in the filesystem and there is a table in our database which > tracks metadata (origin, time, etc.) about these rasters. For climatology > data, we use Oracle database and have our own schema which stores > information about weather stations (e.g. name, location) and current and > historical measured values. > > Because of this, my manager does not want to allow me to create a PostGIS > instance along with a GeoServer instance. His arguments is it's > unnecessary duplicity, it won't be consistent with our database unless we > invest heavily in syncing the two and it needs additional maintenance. His > original idea was that our system will provide data to a GeoServer > instance via some Web Service / WFS / whatever and GeoServer will just > "render" it. The problem is our system does not have a WFS interface > (yet?) and our existing interfaces are not OGC standardized. Of course, if > we implement WFS we won't need GeoServer (mostly). > > I tried to explain to him that we should treat the PostGIS store as some > kind of internal GeoServer store (just like various index files on disk - > he is a dat
Re: [Geoserver-users] GeoServer without PostGIS, is it a good idea?
We do have a small POSTGis instance, used largely for "project level" databases as opposed to "corporate level", but most of what we serve through geoserver is coming from the corporate Oracle database. Notice: This email and any attachments are confidential. If received in error please destroy and immediately notify us. Do not copy or disclose the contents. -- Monitor 25 network devices or servers for free with OpManager! OpManager is web-based network management software that monitors network devices and physical & virtual servers, alerts via email & sms for fault. Monitor 25 devices for free with no restriction. Download now http://ad.doubleclick.net/ddm/clk/292181274;119417398;o ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] WMS-C (tiled=true) not using GWC's metatiling: How to debug?
Hi Jens, You can check GeoServer logs which are default written to "logs/geoserver.log" which is relative to the GeoServer data directory. You can change the log settings under the "Global" option of the administration screen to higher levels. There is also a log viewer in the "About & Status" menu section called "GeoServer Logs". This provides, by default, the last 1000 lines from the current log file. Hope this helps, Chris Snider Senior Software Engineer Intelligent Software Solutions, Inc. [Description: Description: Description: cid:image001.png@01CA1F1F.CBC93990] From: Nachtigall, Jens (init) [mailto:jens.nachtig...@init.de] Sent: Wednesday, June 24, 2015 7:24 AM To: geoserver-users@lists.sourceforge.net Subject: [Geoserver-users] WMS-C (tiled=true) not using GWC's metatiling: How to debug? Hi, I ran into the problem that text labels are omitted when the labels would go over a WMS tile border. So I thought of passing tiled=true and tileorigin=minX,minY as part of the WMS request, and use Metatiling this way. I activated *all* the options as stated on http://docs.geoserver.org/latest/en/user/geowebcache/using.html#direct-integration-with-geoserver-wms Still it seems that the WMS request does not pass the request over to the integrated GWC. I looked at logs/catalina.out but this still does not give me any hint. Any advice on how I could debug this? Maybe the extent is misaligned or similar, but the log does not say anything about this. Tilesorigin is "154224,5219773" for a custom Gridset based on EPSG:25832 (used in ol3 and the gridset as minX,minY for Extent). Using /geoserver/gwc/service/wms it is working, but for /geoserver/gwc//wms the tiled and tilesorigin option seem to make no difference. The used client is OpenLayers3. Best, Jens -- Monitor 25 network devices or servers for free with OpManager! OpManager is web-based network management software that monitors network devices and physical & virtual servers, alerts via email & sms for fault. Monitor 25 devices for free with no restriction. Download now http://ad.doubleclick.net/ddm/clk/292181274;119417398;o___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] GeoServer without PostGIS, is it a good idea?
Hi Peter, GeoServer has data drivers for both MS SQL Server and Oracle, but I believe you have to install them separately as extensions from the download page. MS SQL Server: http://sourceforge.net/projects/geoserver/files/GeoServer/2.7.1/extensions/geoserver-2.7.1-sqlserver-plugin.zip Oracle: http://sourceforge.net/projects/geoserver/files/GeoServer/2.7.1/extensions/geoserver-2.7.1-oracle-plugin.zip Once you have your appropriate extension installed, you can use them to either serve the native tables, native views, or add an SQL View to the GeoServer side on the "New Layer" screen from the " On databases you can also create a new feature type by configuring a native SQL statement. Configure new SQL view..." This allows you to define a relatively simple SQL statement that GeoServer will execute against the backend for WMS/WFS results. Hope this helps. Chris Snider Senior Software Engineer Intelligent Software Solutions, Inc. -Original Message- From: Peter Kovac [mailto:peter.ko...@microstep-mis.sk] Sent: Wednesday, June 24, 2015 5:24 AM To: geoserver-users@lists.sourceforge.net Subject: [Geoserver-users] GeoServer without PostGIS, is it a good idea? Dear GeoServer users, my company develops systems in meteorology, climatology, radiation monitoring etc. It's mostly data collection, storing and later retrieval. Our customers started to demand maps, recently. What we need is a map client serving customers' data, both rasters (e.g. forecasts from GRIB files) and vector (e.g. various values measured by stations from a national climatology network). We looked out for some options and settled on a combination of a GeoServer & OpenLayers integrated into our existing web interface. After several weeks of playing with GeoServer I got an impression that GeoServer works great in combination with a local PostGIS instance. PostGIS fits well into our scenario, too: it would contain metadata about our rasters with time dimensions, and whole vector layers. The problem is we already have our own infrastructure which does all the bookkeeping of all our datasets. E.g. we already store our raster data somewhere in the filesystem and there is a table in our database which tracks metadata (origin, time, etc.) about these rasters. For climatology data, we use Oracle database and have our own schema which stores information about weather stations (e.g. name, location) and current and historical measured values. Because of this, my manager does not want to allow me to create a PostGIS instance along with a GeoServer instance. His arguments is it's unnecessary duplicity, it won't be consistent with our database unless we invest heavily in syncing the two and it needs additional maintenance. His original idea was that our system will provide data to a GeoServer instance via some Web Service / WFS / whatever and GeoServer will just "render" it. The problem is our system does not have a WFS interface (yet?) and our existing interfaces are not OGC standardized. Of course, if we implement WFS we won't need GeoServer (mostly). I tried to explain to him that we should treat the PostGIS store as some kind of internal GeoServer store (just like various index files on disk - he is a database guy and does not care about all those files created by GeoServer - he cares about databases) and not try to mess with it in order to speed up development (deadlines are tight, as usual). I need either better arguments in this debate or an alternative solution. I stumbled upon DataStore Development tutorial on GeoTools ( http://docs.geotools.org/latest/tutorials/advanced/datastore.html ) which looks promising, but I'm not quite sure if it's the right thing. I'll try to implement a proof-of-concept to see if I'm able to convert our existing database schema into a new Data Store type recognized by GeoSever. In the meantime I'd love to hear your opinion about the case. Is it common to implement custom DataStores from existing databases or mirroring it in PostGIS is the preferred way? Are there any other options? -- Peter Kovac IMS Programmer MicroStep-MIS peter.ko...@microstep-mis.sk -- Monitor 25 network devices or servers for free with OpManager! OpManager is web-based network management software that monitors network devices and physical & virtual servers, alerts via email & sms for fault. Monitor 25 devices for free with no restriction. Download now http://ad.doubleclick.net/ddm/clk/292181274;119417398;o ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users -- Monitor 25 network devices or servers for free with OpManager! OpManager is web-based network management software that monitors network devices and physical & vir
[Geoserver-users] WMS-C (tiled=true) not using GWC's metatiling: How to debug?
Hi, I ran into the problem that text labels are omitted when the labels would go over a WMS tile border. So I thought of passing tiled=true and tileorigin=minX,minY as part of the WMS request, and use Metatiling this way. I activated *all* the options as stated on http://docs.geoserver.org/latest/en/user/geowebcache/using.html#direct-integration-with-geoserver-wms Still it seems that the WMS request does not pass the request over to the integrated GWC. I looked at logs/catalina.out but this still does not give me any hint. Any advice on how I could debug this? Maybe the extent is misaligned or similar, but the log does not say anything about this. Tilesorigin is "154224,5219773" for a custom Gridset based on EPSG:25832 (used in ol3 and the gridset as minX,minY for Extent). Using /geoserver/gwc/service/wms it is working, but for /geoserver/gwc//wms the tiled and tilesorigin option seem to make no difference. The used client is OpenLayers3. Best, Jens -- Monitor 25 network devices or servers for free with OpManager! OpManager is web-based network management software that monitors network devices and physical & virtual servers, alerts via email & sms for fault. Monitor 25 devices for free with no restriction. Download now http://ad.doubleclick.net/ddm/clk/292181274;119417398;o___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
[Geoserver-users] GeoServer without PostGIS, is it a good idea?
Dear GeoServer users, my company develops systems in meteorology, climatology, radiation monitoring etc. It's mostly data collection, storing and later retrieval. Our customers started to demand maps, recently. What we need is a map client serving customers' data, both rasters (e.g. forecasts from GRIB files) and vector (e.g. various values measured by stations from a national climatology network). We looked out for some options and settled on a combination of a GeoServer & OpenLayers integrated into our existing web interface. After several weeks of playing with GeoServer I got an impression that GeoServer works great in combination with a local PostGIS instance. PostGIS fits well into our scenario, too: it would contain metadata about our rasters with time dimensions, and whole vector layers. The problem is we already have our own infrastructure which does all the bookkeeping of all our datasets. E.g. we already store our raster data somewhere in the filesystem and there is a table in our database which tracks metadata (origin, time, etc.) about these rasters. For climatology data, we use Oracle database and have our own schema which stores information about weather stations (e.g. name, location) and current and historical measured values. Because of this, my manager does not want to allow me to create a PostGIS instance along with a GeoServer instance. His arguments is it's unnecessary duplicity, it won't be consistent with our database unless we invest heavily in syncing the two and it needs additional maintenance. His original idea was that our system will provide data to a GeoServer instance via some Web Service / WFS / whatever and GeoServer will just "render" it. The problem is our system does not have a WFS interface (yet?) and our existing interfaces are not OGC standardized. Of course, if we implement WFS we won't need GeoServer (mostly). I tried to explain to him that we should treat the PostGIS store as some kind of internal GeoServer store (just like various index files on disk - he is a database guy and does not care about all those files created by GeoServer - he cares about databases) and not try to mess with it in order to speed up development (deadlines are tight, as usual). I need either better arguments in this debate or an alternative solution. I stumbled upon DataStore Development tutorial on GeoTools ( http://docs.geotools.org/latest/tutorials/advanced/datastore.html ) which looks promising, but I'm not quite sure if it's the right thing. I'll try to implement a proof-of-concept to see if I'm able to convert our existing database schema into a new Data Store type recognized by GeoSever. In the meantime I'd love to hear your opinion about the case. Is it common to implement custom DataStores from existing databases or mirroring it in PostGIS is the preferred way? Are there any other options? -- Peter Kovac IMS Programmer MicroStep-MIS peter.ko...@microstep-mis.sk -- Monitor 25 network devices or servers for free with OpManager! OpManager is web-based network management software that monitors network devices and physical & virtual servers, alerts via email & sms for fault. Monitor 25 devices for free with no restriction. Download now http://ad.doubleclick.net/ddm/clk/292181274;119417398;o ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
