Re: [MapServer-dev] Using OGR geoparquet data in mapserver - spatial filtering works

Tue, 22 Jul 2025 04:12:00 -0700

I guess we would need to add ExecuteSQL when the processing option or connection options are set.


Michael Smith

On Jul 22, 2025, at 6:31 AM, Michael Smith <[email protected]> wrote:

Seth,

Yeah I was looking at maybe doing something like that but I didn’t see an option in https://gdal.org/en/stable/api/ogrlayer_cpp.html#_CPPv4N8OGRLayer18SetAttributeFilterEPKc that would allow changing the dialect. So there would have to be changes on the OGR side as well.

I looked into doing it via ADBC and I could get it to work but the memory/cpu load using libduckdb was substantial when doing multiple queries (eg fetching multiple bbox requests).  Using geoparquet was much less taxing.

Michael Smith

On Jul 22, 2025, at 6:20 AM, Seth G <[email protected]> wrote:


Hi Mike,

I don't think so, but coincidentally I was checking yesterday as it would allow some of the functions such as:

DATA "SELECT ObjectId, ST_Z(geometry) Z FROM mydata"

There is the CONNECTIONOPTIONS block, but this doesn't support a "--dialect" equivalent when testing. It would be nice to do something as follows:

CONNECTIONOPTIONS
    "dialect" "sqlite"
END

Or alternatively adding in:

PROCESSING 'ogr_dialect=sqlite'

The MapServer OGR code would also need to be updated. 

Seth


On Mon, Jul 21, 2025, at 5:52 PM, Michael Smith via MapServer-dev wrote:

And spatial filtering is working, it was due to BBOX sorting that it was not being very efficient in the spatial filtering.

 

Is there a method to use SQLite dialect with MapServer queries with parquet and OGR? Wanted to use SAMPLE but this is only supported via SQLite dialect.

 

Mike

 

 

-- 

Michael Smith

RSGIS Center – ERDC CRREL NH

US Army Corps

 

From: Even Rouault <[email protected]>
Date: Monday, July 21, 2025 at 10:05 AM
To: Michael Smith <[email protected]>, MapServer-dev <[email protected]>
Subject: Re: [MapServer-dev] Using OGR geoparquet data in mapserver - no spatial filtering

 

Mike,

 
Anyone know a current way to get mapserver to add spatial filtering on OGR geoparquet layers. Have been unable to get the actual sql query being used with CPL_DEBUG inside mapserver with CONFIG "CPL_DEBUG" "ON" but from the performance, it appears no spatial filtering is being done. I suspect code changes to mapogr would be needed to do this but wondering if anyone else has looked into this.


I haven't tried but I would assume this should work


 
 
Also, I tried using ADBC connection to parquet, and while it worked (technically) it really used an excessive amount of CPU and memory over using the geoparquet driver.

Yes this is "expected". I noticed that the Arrow API of duckdb tends to ingest everything in memory before paging results.

Even

-- 
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
MapServer-dev mailing list


_______________________________________________
MapServer-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/mapserver-dev

Reply via email to