[mapserver-users] GeoJSON OGR output with WFS data type issue
Hi: using:
MapServer: 6.2.0
GDAL/OGR: 1.9.2
PostgreSQL: 8.4.13
$ cat /etc/issue
Ubuntu 10.04.4 LTS \n \l
...we have a MapServer setup and configured to feed downstream
applications with (point) observation data via WFS and GeoJSON.
We have an issue where a null value in an integer field is being
returned as a value of 0 in GeoJSON. This is throwing off applications
who are processing 0 values. The layer in question is a PostGIS
connection.
Below is a trace along the OGR/MapServer workflow:
PostgreSQL/PostGIS:
FOO=# select postgis_full_version();
postgis_full_version
POSTGIS=2.0.2SVN r10398 GEOS=3.3.5-CAPI-1.7.5 PROJ=Rel. 4.8.0, 6
March 2012 GDAL=GDAL 1.9.2, released 2012/10/08 LIBXML=2.9.0 RASTER
(1 row)
FOO=# \q data_mart_ca_live;
air_temp_qa| integer
...
FOO=# select air_temp_qa from data_mart_ca_live where
data_payload_id=11177725;
air_temp_qa
-
(1 row)
Here we see that air_temp_qa (integer value) returns null in SQL.
OGR:
$ ogrinfo -al PG:dbname=FOO host=localhost user=foo password=bar
data_mart_ca_live -where data_payload_id = '11177725'
...
# column definition
air_temp_qa: Integer (0.0)
...
# actual value
air_temp_qa (Integer) = (null)
...
$ ogr2ogr -select air_temp_qa -f GeoJSON foo.json PG:dbname=FOO
host=localhost user=foo password=bar data_mart_ca_live -where
data_payload_id = '11177725'
$ cat foo.json
type: FeatureCollection,
features: [
{ type: Feature, properties: { air_temp: null, air_temp_qa:
null }, geometry: { type: Point, coordinates: [ -111.93,
60.03 ] } }
]
}
So far all is consistent in OGR with what is represented in
PostgreSQL/PostGIS.
MapServer layer definition:
LAYER
NAME ca-1.0-ascii
STATUS ON
DEBUG ON
CONNECTION dbname=FOO host=localhost user=foo password=bar
CONNECTIONTYPE POSTGIS
PROCESSING CLOSE_CONNECTION=DEFER
DATA the_geom from data_mart_ca_live using unique data_payload_id
using srid=4326
TYPE POINT
DUMP TRUE
PROJECTION
init=epsg:4326
END
METADATA
ows_authorityurl_name taxonomy
ows_authorityurl_href http://localhost/taxonomy;
ows_identifier_authority taxonomy
ows_identifier_value
/msc/observation/atmospheric/surface_weather/ca-1.0-ascii
wms_layer_group /msc/observation/atmospheric/surface_weather
ows_title DMS CA obs data with qa
ows_abstract DMS CA obs data with qa
ows_keywordlist dms,climate,test,mapserver
ows_extent -141.089000 36.392987 -52.089000 89.784987 # early out
wfs_metadataurl_format XML
gml_include_items all
gml_types auto
wms_timeextent 2013-01-11T18:39:34Z/2013-04-11T18:39:34Z
wms_timeitem instance_datetime
END
INCLUDE /home/tomk/msc-ows/trunk/services/msc/classes/dashboard.inc
END
- Test using WMS GetFeatureInfo returns air_temp_qa as an empty XML
element (which is expected/desired).
- Test using WFS GetFeature with default OUTPUTFORMAT:
$ GET
http://localhost/msc-ows?service=WFSversion=1.1.0request=GetFeaturet
ypename=ca-1.0-asciifilter=ogc:Filterogc:PropertyIsEqualToogc:Prop
ertyNamedata_payload_id/ogc:PropertyNameogc:Literal11177725/ogc:Li
teral/ogc:PropertyIsEqualTo/ogc:Filter
... returns air_temp_qa as an empty XML element (which is
expected/desired).
- Test using WFS GetFeature with OUTPUTFORMAT=GeoJSON:
$ GET
http://localhost/msc-ows?service=WFSversion=1.1.0request=GetFeaturet
ypename=ca-1.0-asciifilter=ogc:Filterogc:PropertyIsEqualToogc:Prop
ertyNamedata_payload_id/ogc:PropertyNameogc:Literal11177725/ogc:Li
teral/ogc:PropertyIsEqualTo/ogc:Filteroutputformat=GeoJSON
...returns air_temp_qa in GeoJSON with a value of 0 (i.e. air_temp_qa:
0).
So it looks like, somewhere in MapServer, a null integer value is being
cast to 0. What's weird is that this is happening only for OGR driver
requested formats, not native MapServer outputs. But in OGR proper it
works as expected.
Other notes:
- LAYER.METADATA includes gml_types auto. When commenting that out,
the value comes back as blank (which is good), but all OGC Filters
(which are perfectly valid, from a wide variety of WFS clients) fail
because it looks like, without this, the client is forced to apply a
cast or quote the ogc:Literal value given the data type.
Any suggestions are valued.
Thanks
..Tom
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] GeoJSON OGR output with WFS data type issue
Any suggestions are valued.
Try the attached patch (only compile tested by me).
I'm unsure if null values are well represented by empty strings by MapServer
in the shape-values array, but I think so from my quick inspection of the
code (that's maybe unfortunate that there's no distinction between unset/null
fields and empty strings)
Ultimately OGR_F_SetFieldString could ignore empty string passed for setting
numeric fields, but that's probably the job of the client to *not* call
SetFieldString when it doesn't intent to set a value.
Thanks
..Tom
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
diff --git a/mapogroutput.c b/mapogroutput.c
index 0ee8692..db32c3e 100644
--- a/mapogroutput.c
+++ b/mapogroutput.c
@@ -205,8 +205,10 @@ static int msOGRWriteShape( layerObj *map_layer, OGRLayerH hOGRLayer,
OGRErr eErr;
int i, out_field;
OGRwkbGeometryType eLayerGType, eFeatureGType = wkbUnknown;
+ OGRFeatureDefnH hLayerDefn;
- eLayerGType = OGR_FD_GetGeomType(OGR_L_GetLayerDefn(hOGRLayer));
+ hLayerDefn = OGR_L_GetLayerDefn( hOGRLayer );
+ eLayerGType = OGR_FD_GetGeomType(hLayerDefn);
/* */
/* Transform point geometry. */
@@ -380,7 +382,7 @@ static int msOGRWriteShape( layerObj *map_layer, OGRLayerH hOGRLayer,
/* doesn't match the layer.*/
/* */
eLayerGType =
-wkbFlatten(OGR_FD_GetGeomType(OGR_L_GetLayerDefn(hOGRLayer)));
+wkbFlatten(OGR_FD_GetGeomType(hLayerDefn));
if( hGeom != NULL )
eFeatureGType = wkbFlatten(OGR_G_GetGeometryType( hGeom ));
@@ -413,7 +415,7 @@ static int msOGRWriteShape( layerObj *map_layer, OGRLayerH hOGRLayer,
/* Consider flattening the geometry to 2D if we want 2D*/
/* output. */
/* */
- eLayerGType = OGR_FD_GetGeomType(OGR_L_GetLayerDefn(hOGRLayer));
+ eLayerGType = OGR_FD_GetGeomType(hLayerDefn);
if( hGeom != NULL )
eFeatureGType = OGR_G_GetGeometryType( hGeom );
@@ -426,7 +428,7 @@ static int msOGRWriteShape( layerObj *map_layer, OGRLayerH hOGRLayer,
/* */
/* Create the feature, and attach the geometry.*/
/* */
- hFeat = OGR_F_Create( OGR_L_GetLayerDefn( hOGRLayer ) );
+ hFeat = OGR_F_Create( hLayerDefn );
OGR_F_SetGeometryDirectly( hFeat, hGeom );
@@ -440,6 +442,18 @@ static int msOGRWriteShape( layerObj *map_layer, OGRLayerH hOGRLayer,
if( !item-visible )
continue;
+/* Avoid setting empty strings for numeric fields, so that OGR */
+/* doesn't take them as 0. */
+if( shape-values[i][0] == '\0' ) {
+ OGRFieldDefnH hFieldDefn = OGR_FD_GetFieldDefn(hLayerDefn, out_field);
+ OGRFieldType eFieldType = OGR_Fld_GetType(hFieldDefn);
+ if( eFieldType == OFTInteger || eFieldType == OFTReal )
+ {
+out_field++;
+continue;
+ }
+}
+
OGR_F_SetFieldString( hFeat, out_field++, shape-values[i] );
}
___
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] GeoJSON OGR output with WFS data type issue
-Original Message- From: Even Rouault [mailto:even.roua...@mines-paris.org] Sent: Thursday, April 11, 2013 15:33 To: mapserver-users@lists.osgeo.org Cc: Kralidis,Tom [Ontario] Subject: Re: [mapserver-users] GeoJSON OGR output with WFS data type issue Any suggestions are valued. Try the attached patch (only compile tested by me). I'm unsure if null values are well represented by empty strings by MapServer in the shape-values array, but I think so from my quick inspection of the code (that's maybe unfortunate that there's no distinction between unset/null fields and empty strings) Ultimately OGR_F_SetFieldString could ignore empty string passed for setting numeric fields, but that's probably the job of the client to *not* call SetFieldString when it doesn't intent to set a value. Even: thanks for this. Works as expected. Thanks ..Tom ___ mapserver-users mailing list mapserver-users@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/mapserver-users
