Re: [gdal-dev] Faster alternative to GetFeatureCount?

[Roman Breitfuss-Schiffer]

Hi Jon!

I don't know if there's an alternative to GetFeatureCount. At least I 
couldn't find one in the API documentation. Maybe there's a workaround 
I'm not aware of...


What you could do is wrap the call of GetFeatrueCount in a function 
which is killed after some time (2 approaches are discussed here: 
https://stackoverflow.com/questions/40915527/kill-function-after-a-given-amount-of-time). 
With that approach the question remains of getting the value of 
GetFeatureCount when the function gets killed. One could assume that 
there are more features than 1 if the function takes too long. This 
assumption might be a bit risky though...


Best regards
Roman

Am 20.01.2022 um 10:00 schrieb Jon Morris:


I'm writing applications using the GDAL Python bindings and when I 
profile for performance, GetFeatureCount frequently comes out near the 
top. I'm often using it to check whether a spatial or attribute filter 
has returned any features and don't need the full count. When the 
layer contains millions of features, there would be a big performance 
improvement if we could exit the count early.


Is there a better way of doing this? I've tried using GetNextFeature 
instead, but there must be quite a lot of overhead in that function as 
it is much slower. All I need to know is if the layer has has 0, 1 or 
>1 features, I don't need the actual count. Can anyone suggest the 
fastest way of doing this in Python? I'm using GDAL 3.3.1 at the 
moment but could upgrade if there is new functionality that would help.


Thanks,

Jon

*Jon Morris*

*Software Developer*

e: ​*jon.mor...@jbarisk.com* 
t:  *+44 (0)1756 799919* 



*www.jbarisk.com* 

Facebook 


LinkedIn 


Twitter 


YouTube 



All JBA Risk Management's email messages contain confidential 
information and are intended only for the individual(s) named. If you 
are not the named addressee you should not disseminate, distribute or 
copy this e-mail.
Please notify the sender immediately by email if you have received 
this email by mistake and delete this email from your system.
JBA Risk Management Limited is registered in England, company number 
07732946, 1 Broughton Park, Old Lane North, Broughton, Skipton, North 
Yorkshire, BD23 3FD, England.



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Build GDAL for RedHat 7.9 with python bindings and non-default drivers

[Roman Breitfuss-Schiffer]

Dear all!

We are currently trying to build a GDAL/OGR installation (version >=3.1) 
for our server which operates on RedHat 7.9 (RHEL).


I have some experience with building GDAL on Ubuntu but have no working 
workflow for RHEL as the required packages and header files (e.g. 
gdal-libs, gdal-libs-devel, python-devel etc.) are not available for 
recent GDAL versions via yum.


*Problem 1: Does anyone have experience with building GDAL **(version 
>=3.1)**on RHEL?*


We want to build with Python bindings and some non-default drivers (e.g. 
GPKG (raster), JP2OpenJPEG, MBTILES, WMS/WCS). I tried out the build on 
Ubuntu with the following ./configure:


./configure
  --prefix=/usr
  --libdir=/usr/lib64
  --with-armadillo=yes
  --with-sqlite3=yes
  --with-spatialite=yes
  --with-openjpeg
  --with-geos=yes
  --with-curl=/usr/bin/curl-config
  --with-python=yes
  --with-png=internal
  --with-jpeg=internal

*Problem 2: After running the ./configure command the GPKG (raster) 
driver and the JP2OpenJPEG driver are not listed in misc. gdal formats.

*

./configure output see attachment.

Best regards
Roman

 
___

Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS)
GDAL is now configured for x86_64-pc-linux-gnu

  Installation directory:/usr
  C compiler:gcc -DHAVE_AVX_AT_COMPILE_TIME 
-DHAVE_SSSE3_AT_COMPILE_TIME -DHAVE_SSE_AT_COMPILE_TIME -g -O2
  C++ compiler:  g++ -DHAVE_AVX_AT_COMPILE_TIME 
-DHAVE_SSSE3_AT_COMPILE_TIME -DHAVE_SSE_AT_COMPILE_TIME -g -O2
  C++14 support: no

  LIBTOOL support:   yes

  Armadillo support: yes
  Blosc support: no
  CFITSIO support:   external
  crypto/openssl support:yes
  cryptopp support:  no
  cURL support (wms/wcs/...):yes
  DDS support:   no
  DODS support:  no
  ECW support:   no
  Expat support: yes
  EXR support:   no
  FGDB support:  no
  FreeXL support:yes
  GEORASTER support: no
  GEOS support:  yes
  Google libkml support: yes
  GRASS support: no
  GTA support:   no
  HDF4 support:  yes
  HDF5 support:  yes
  JXL support:   no
  HDFS support:  no
  HEIF support:  no
  INFORMIX DataBlade support:no
  Ingres support:no
  JP2Lura support:   no
  JPEG 12 bit:   yes
  JPEG-in-TIFF 12 bit:   no
  JPEG JasPer support:   no
  JPEG-Lossless/CharLS:  yes
  Kakadu support:no
  Kea support:   no
  LERC support:  internal
  libbrunsli support:no
  libdeflate support:no
  LIBGEOTIFF support:external
  LIBGIF support:external
  LIBJPEG support:   internal
  LIBLZMA support:   no
  LIBPNG support:internal
  LIBTIFF support:   external (BigTIFF=yes)
  libxml2 support:   yes
  LIBZ support:  external
  LZ4 support:   no
  MDB support:   no
  MongoCXX v3 support:   no
  MongoDB support:   no
  MrSID/MG4 Lidar support:   no
  MrSID support: no
  MSG support:   no
  MySQL support: no
  NetCDF has netcdf_mem.h:   yes
  NetCDF support:yes
  OCI support:   no
  ODBC support:  yes
  OGDI support:  yes
  OpenCL support:no
  OpenJPEG support:  yes
  PCIDSK support:internal
  PCRaster support:  internal
  PCRE support:  yes
  PCRE2 support: no
  PDFium support:no
  Podofo support:no
  Poppler support:   no
  PostgreSQL support:yes
  QHull support: external
  Rasdaman support:  no
  RasterLite2 support:   no
  RDB support:   no
  SFCGAL support:
  SOSI support:  no
  SpatiaLite support:yes
  SQLite support:yes
  Teigha (DWG and DGNv8):no
  TileDB support:no
  userfaultfd support:   yes
  WebP support:  yes
  Xerces-C support:  yes
  ZSTD support:  yes


  misc. gdal formats:aaigrid adrg aigrid airsar arg blx bmp bsb cals 
ceos ceos2 coasp cosar ctg dimap dted elas envisat ers esric fit gff gsg gxf 
hf2 idrisi ilwis ingr iris iso8211 jaxapalsar jdem kmlsuperoverlay l1b leveller 
map mrf msgn ngsgeoid nitf northwood pds prf r raw rmf rs2 safe saga sdts 
sentinel2 sgi sigdem srtmhgt stacit stacta terragen tga til tsx usgsdem xpm xyz 
zarr zmap rik ozi eeda plmosaic rda wcs wms wmts daas ogcapi rasterlite mbtiles 
grib pdf
  misc. ogr formats: arcgen avc cad csv dgn dxf edigeo flatgeobuf 
geoconcept georss gml gmt gpsbabel gpx gtm jml mapml mvt ntf openfilegdb pgdump 
rec s57 selafin shape svg sxf tiger vdv wasp idrisi pds sdts amigocloud carto 
cl

Re: [gdal-dev] GDALWMS - error: TCP connection reset by peer

[Roman Breitfuss-Schiffer]

Dear Jukka!

Thanks for your answer! I think I'm gonna follow that road you suggested.

We have been thinking a lot and I was just wondering if there could be 
some GDAL configuration which could cause that behaviour? For example 
the value for  in the WMS XML file? Or if more 
applications are accessing the same WMS XML at the same time which is 
theoretically possible in our case.


Best regards
Roman

Am 18.11.2021 um 12:12 schrieb Rahkonen Jukka (MML):


Hi,

It seems that the WMS server is closing the door and there is nothing 
else to do on the GDAL side except to try again later. Contact the WMS 
service maintainer and report your troubles. They may be able to 
provide you a more reliable service especially if you are ready to pay 
for it.


-Jukka Rahkonen-

*Lähettäjä:* gdal-dev  *Puolesta 
*Roman Breitfuss-Schiffer

*Lähetetty:* torstai 18. marraskuuta 2021 11.59
*Vastaanottaja:* gdal-dev@lists.osgeo.org
*Aihe:* [gdal-dev] GDALWMS - error: TCP connection reset by peer

Dear all!

We are running a Python application an a linux distribution (RedHat) 
which creates clips in MBTILES format from a WMS. We are using the 
Python-API and are basically doing something like this:


src_ds = gdal.Warp(
    tmp_file,
    ds_in,
    format='GTiff',
    outputBounds=[bbox[0], bbox[2], bbox[1], bbox[3]],
    dstSRS=output_srs,
)
ds_out = gdal.Translate(
    out_filepath,
    src_ds,
    format='MBTILES',
    creationOptions=creation_options,
)

The application throughs an error from time to time since a few days. 
We couldn't figure out a pattern yet and therefore we are also not 
able to reproduce the error. The error is the following:


Mon Nov 15 13:48:13 2021: CPLError: GDALWMS: Unable to download block 
2765, 602.

URL: TCP connection reset by peer
  HTTP status code: 0, error: TCP connection reset by peer.
Add the HTTP status code to  to ignore this error 
(see http://www.gdal.org/frmt_wms.html).
Mon Nov 15 13:48:13 2021: CPLError: /wms_xml/wms_config.xml, band 1: 
IReadBlock failed at X offset 2764, Y offset 601
Mon Nov 15 13:48:13 2021: CPLError: GetBlockRef failed at X block 
offset 2764, Y block offset 601


The exact same data gets processed successfully in one run and not 
successfully in another run. We didn't change the code nor the Python 
or GDAL version. Hence, we are kind of puzzled as the error comes and 
goes seemingly at random.


Does anyone of you have any hints?

Thanks & best regards!
Roman

___
Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS)


--
___
Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS)
Veronikagasse 38/20, 1170 Wien
+43 664 5547678
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDALWMS - error: TCP connection reset by peer

[Roman Breitfuss-Schiffer]

Dear all!

We are running a Python application an a linux distribution (RedHat) 
which creates clips in MBTILES format from a WMS. We are using the 
Python-API and are basically doing something like this:


src_ds = gdal.Warp(
    tmp_file,
    ds_in,
    format='GTiff',
    outputBounds=[bbox[0], bbox[2], bbox[1], bbox[3]],
    dstSRS=output_srs,
)
ds_out = gdal.Translate(
    out_filepath,
    src_ds,
    format='MBTILES',
    creationOptions=creation_options,
)

The application throughs an error from time to time since a few days. We 
couldn't figure out a pattern yet and therefore we are also not able to 
reproduce the error. The error is the following:


Mon Nov 15 13:48:13 2021: CPLError: GDALWMS: Unable to download block 
2765, 602.

URL: TCP connection reset by peer
  HTTP status code: 0, error: TCP connection reset by peer.
Add the HTTP status code to  to ignore this error 
(see http://www.gdal.org/frmt_wms.html).
Mon Nov 15 13:48:13 2021: CPLError: /wms_xml/wms_config.xml, band 1: 
IReadBlock failed at X offset 2764, Y offset 601
Mon Nov 15 13:48:13 2021: CPLError: GetBlockRef failed at X block offset 
2764, Y block offset 601


The exact same data gets processed successfully in one run and not 
successfully in another run. We didn't change the code nor the Python or 
GDAL version. Hence, we are kind of puzzled as the error comes and goes 
seemingly at random.


Does anyone of you have any hints?

Thanks & best regards!
Roman

___
Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS)
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev