[gdal-dev] Request for advice in unexpected slowdown working with LVBAG

My c++ utility-app collects Kadaster BAG data in postgresql tables for use in "populatie service". It has functioned for several years, with a typical runtime of 40 minutes. In the process of rounding this off, I updated the gdal binaries from 3.03 to 3.53, as well as added some threaded workloads

Re: [gdal-dev] Proposed RFC 8 amendment regarding (prohibited use of) generative AI tools

Some reading . On Wed, Oct 9, 2024 at 9:11 AM ElPaso via gdal-dev wrote: > Il 09/10/24 00:55, Greg Troxel via gdal-dev ha scritto: > > ElPaso via gdal-dev writes: > > > >> I have read the discussion on lwn and

Re: [gdal-dev] Proposed RFC 8 amendment regarding (prohibited use of) generative AI tools

Sorry to butt in. I have no say in this whatsoever. A few mayhaps/perby useful points: - not using GAI is not a lasting option, so rather focus on precautions and be aware of dangers. Prime danger, to me, seams copyright trolling ala Oracle. They have bought/adopted stuff in order to sue people. Of

Re: [gdal-dev] Inquire mapped column names for conversion to shapefile

teField(ogr.FieldDefn(name)) > shp_field_name = lyr_defn.GetFieldDefn(lyr_defn.GetFieldCount() - > 1).GetNameRef() > map_to_shp[name] = shp_field_name > ds = None > ogr.GetDriverByName("ESRI Shapefile").DeleteDataSource(tmpfilename) > print(map_to_shp) > >

[gdal-dev] Inquire mapped column names for conversion to shapefile

Hi everyone, I want to make a map of original (let's say postgresql/postgis) column names to those used in a shapefile after conversion; assuming that at least some column names in postgis will be over 10 characters in length, and shortening may produce conflicts. Basically I know how this is done,

Re: [gdal-dev] Possible error in orientation of converted arcs from dxf

Sun, Feb 25, 2024 at 10:13 PM Even Rouault wrote: > Please file an issue with a (as small as possible) .dxf file reproducing > the issue > > Even > Le 25/02/2024 à 22:05, Jan Heckman a écrit : > > Hi Even, all, > There might be a slight connection, but my case imo does not i

Re: [gdal-dev] Possible error in orientation of converted arcs from dxf

I'm wondering if there woulnd't be a connection with > https://github.com/OSGeo/gdal/issues/1839#issuecomment-537185826 ? > > Even > > Le 25/02/2024 à 13:32, Jan Heckman via gdal-dev a écrit : > > Hi all, > > Sorry for a somewhat long post, but here we go: >

[gdal-dev] Possible error in orientation of converted arcs from dxf

Hi all, Sorry for a somewhat long post, but here we go: I am converting a .dxf having a sequence of simple lines and arcs which form a continuous (poly)line, (correctly) ordered by EntityHandle. The arcs are converted (to shp) as well as displayed (Qgis) in the opposite direction as the simple line

Re: [gdal-dev] layer creation option shapefile when converting arcs in dxf

> Even > > Le 24/02/2024 à 12:15, Jan Heckman via gdal-dev a écrit : > > Hi All, > > I am looking for a way to influence the (density of the) linestrings > > which are produced by converting a circle segment in dxf (to a > > linestring in shapefile format); the a

[gdal-dev] layer creation option shapefile when converting arcs in dxf

Hi All, I am looking for a way to influence the (density of the) linestrings which are produced by converting a circle segment in dxf (to a linestring in shapefile format); the aim, ideally, is to limit the angle(s) between the chords produced. If not available, a setting of the number of vertices

Re: [gdal-dev] layer.GetSpatialRef() fails on linux for shapefiles

r two WGS84. Hence thinking it may > be environmental, but I don't have any real *nix knowledge to use to > diagnose the issue. > > > On 2023-09-24 22:52, Jan Heckman wrote: > > Sorry to break in, but surely, we would like to see the .prj file in > question for a simp

Re: [gdal-dev] layer.GetSpatialRef() fails on linux for shapefiles

Sorry to break in, but surely, we would like to see the .prj file in question for a simple try at reproducing? On Sun, Sep 24, 2023 at 11:37 PM Jonathan Moules via gdal-dev < gdal-dev@lists.osgeo.org> wrote: > Thanks Even. I don't have access to the machine either as the colleague > is moving to

Re: [gdal-dev] Shapefile with corrupted index: SHAPE_RESTORE_SHX=YES doesn't correctly repairs it.

ps://github.com/OSGeo/gdal/pull/7778 > [5] > > https://web.archive.org/web/20091024035556/http://geocities.com/SiliconValley/Haven/2295/howto_shapechk.html > [6] > > https://www.autodesk.com/support/technical/article/caas/sfdcarticles/sfdcarticles/Required-files-that-make-up-a-shapefile.html &g

Re: [gdal-dev] Shapefile with corrupted index: SHAPE_RESTORE_SHX=YES doesn't correctly repairs it.

reconstruction, I think. Good luck, Jan On Wed, May 17, 2023 at 12:39 PM Jan Heckman wrote: > Hi all, > > As noted in the QGis issues <https://github.com/qgis/QGIS/issues/53058>, > the shx has problems that ogr2ogr (or QGis Repair Shapefile) will not fix; > The error

Re: [gdal-dev] Shapefile with corrupted index: SHAPE_RESTORE_SHX=YES doesn't correctly repairs it.

dev < gdal-dev@lists.osgeo.org> wrote: > Il 16/05/2023 13:52, Jan Heckman ha scritto: > > Also out of curiosity, > > Are you willing to share the shapefile, preferably at least shp and dbf, > > but probably .shp alone will do and .prj helps for checking conversion > >

Re: [gdal-dev] Shapefile with corrupted index: SHAPE_RESTORE_SHX=YES doesn't correctly repairs it.

Also out of curiosity, Are you willing to share the shapefile, preferably at least shp and dbf, but probably .shp alone will do and .prj helps for checking conversion results. Also, to be sure, the expected number of rows is relevant so as not to call victory too soon. Jan On Tue, May 16, 2023 at

Re: [gdal-dev] how to export all the definitions of the relationships present in a FileGDB?

I seem to remember that the n:m relationships can be retrieved as a table. Look in the TOC. 1:n relationships are not stored in GDB as a table, but as a simple foreign key relationship as far as I remember. If you were import an n:m relationship(table) into say postgresql, you would then have to ad

Re: [gdal-dev] ogrinfo UPDATE performance request

Hi Jukka, Thanks for all the dialect info. Reading your last message about testing the dialect in use, I have to admit that I did not think about native sql (dialect) being deduced from the source type rather than from any dbms involved. That is clear to me now, of course. Would it be correct, t

Re: [gdal-dev] ogrinfo UPDATE performance request

ault dialect once I edited the geometry column name to the one > I have in PostGIS. Are you sure that it requires SQLite dialect for you? > Have you tested the SQL with psql? > > > > -Jukka Rahkonen- > > > > *Lähettäjä:* Jan Heckman > *Lähetetty:* torstai 9. kesäkuuta 20

Re: [gdal-dev] ogrinfo UPDATE performance request

Afaik as I know, slightly more involved sql on a postgresql table requires dialect=sqlite to work at all, e.g. (picked a random example using ogr2ogr instead of ogrinfo) ogr2ogr -f postgresql -dialect sqlite -append PG:"user=%user% dbname=%dbname%" -sql "select trim(coalesce(iv1,''),';') as fidstri

Re: [gdal-dev] ogr2ogr Postgres upload performance

Perhaps the spatial index update (in de DB) slows the insertion. It might be more efficient to not create the SI in the first run, then insert more rows and create the SI after all is done. I don't know how to turn off SI creation in the command line. On Mon, May 23, 2022 at 3:04 PM Alexandre Gaco

Re: [gdal-dev] What's so wrong with this nc file ...

Hi list, I am working with the LVBAG driver, for which I am truly grateful, by the way, in C++. I have a problem with OFTStringList and the corresponding function GetAsStringList() and/or the cast operator(s) to std::vector In my code they produce an empty result. switch (oField.GetType()) { ... br

Re: [gdal-dev] modify zvalue in database when inserted using zfield option

Quite possibly in the Z-coordinate of your geometry column. You can check in pgadmin or psql (or another program like omnidb) what the exact type of the postgis geometry in your table is. You should see something like geometry(LineStringZM,28992) what i.c. means a dutch projection of a linestring w

Re: [gdal-dev] unicode support in GDAL

Does the shapefile have a .cpg with UTF8 as spec? Did you open the shape (or, in fact, the .dbf) in e.g. Qgis? You might try open it (the .dbf) in Libre office calc and specify the encoding as UTF8. If there's still ? or non-Russian where it should be, you are sure that the shapefile (dbf) is not

Re: [gdal-dev] Fwd: ogr2ogr crash attempting to drop table in filegdb

However, ogrinfo does no better. It refuses to rename a table (filegdb) and crashes on drop table. Jan On Fri, Feb 5, 2021 at 11:43 AM jratike80 < jukka.rahko...@maanmittauslaitos.fi> wrote: > Hi, > > SQL in ogr2ogr is only for SELECT. For any other purpose use ogrinfo. > An example: > > ogrinfo

Re: [gdal-dev] Fwd: ogr2ogr crash attempting to drop table in filegdb

Hi Jukka, Shame on me. Had just found out. thanks, Jan On Fri, Feb 5, 2021 at 11:43 AM jratike80 < jukka.rahko...@maanmittauslaitos.fi> wrote: > Hi, > > SQL in ogr2ogr is only for SELECT. For any other purpose use ogrinfo. > An example: > > ogrinfo -sql "alter table point_p rename column test to

[gdal-dev] Fwd: ogr2ogr crash attempting to drop table in filegdb

Dear list, About the filegdb driver Attempting: ogr2ogr -f FILEGDB test.gdb bag2101.gdb -update -sql "drop table standplaats" crashed ogr2ogr (obtained from gisinternals.com, gdal-302-1928-x64-core.msi etc) with a heap corruption. (VS: Unhandled exception at 0x7FFE1C6EF099 (ntdll.dll) in ogr2o

Re: [gdal-dev] Driver maintenance - long-term solution ?

I have none of the financial/organisational knowledge that some of you have, and this is mere googling. With that being said, is https://bigdatastack.eu/european-open-source-initiative a potential? It has a participation link and explicitly includes individuals and not only academia and companies.

Re: [gdal-dev] Considering drivers removal ?

In good company when pruning the source trees: https://www.phoronix.com/scan.php?page=news_item&px=2021-Linux-Drop-Old-CPUs Apparently 68000 may not be supported any longer in linux... Just joking. Hope you don't mind... On Tue, Jan 12, 2021 at 3:46 AM Jed O. Kaplan wrote: > I for one am using t

Re: [gdal-dev] Considering drivers removal ?

Hey guys, I was just typing pretty much the same thing. So these drivers could be removed gracefully with a trace allowing gis-archeologists/archivists to use them. I can just see the papers in my imagination. Jan On Mon, Jan 11, 2021 at 1:03 AM Stephen Woodbridge < stephenwoodbridg...@gmail.com>

Re: [gdal-dev] Considering drivers removal ?

A bit of googling sometimes gives an indication. I was a little confused about SVG, not using it myself but getting quite a few references. E.g. this one in stackexchange, with a mention how often it was looked up: https://gis.stackexchange.com/questions/11476/importing-svg-into-gis Just an idea. O

Re: [gdal-dev] exporting all tables from PG schema

That's cool. Had not yet run into this, but was probably going to pretty soon. Thanks! On Tue, Dec 29, 2020 at 6:28 PM jratike80 < jukka.rahko...@maanmittauslaitos.fi> wrote: > Hi, > > I would have a try with open option LIST_ALL_TABLES=YES > https://gdal.org/drivers/vector/pg.html. > "LIST_ALL_T

Re: [gdal-dev] ogr2ogr (I)LIKE not caseINsensitive

Hi Even, I'll upgrade to get rid of the error. Thanks again. Jan On Thu, Nov 28, 2019 at 5:49 PM Even Rouault wrote: > On jeudi 28 novembre 2019 17:40:34 CET Jan Heckman wrote: > > That's > > ogr2ogr plangebieden.shp -t_srs EPSG:28992 -dim 2 -spat 45000 405000 >

Re: [gdal-dev] ogr2ogr (I)LIKE not caseINsensitive

a%' AND dossierstatus <> 'vervallen' AND Naam NOT LIKE '%parke%' AND Naam NOT LIKE '%paraplu%' AND Naam NOT LIKE '%Parke%' AND Naam NOT LIKE '%Paraplu%'" On Thu, Nov 28, 2019 at 4:58 PM Even Rouault wrote: > On jeudi 28 novembre 20

Re: [gdal-dev] ogr2ogr (I)LIKE not caseINsensitive

s, Jan On Thu, Nov 28, 2019 at 4:25 PM Even Rouault wrote: > On jeudi 28 novembre 2019 16:09:28 CET Jan Heckman wrote: > > Hi everybody, > > According to docs <https://gdal.org/user/ogr_sql_dialect.html>, LIKE and > > ILIKE should be caseinsensitive. > > However,

[gdal-dev] ogr2ogr (I)LIKE not caseINsensitive

Hi everybody, According to docs , LIKE and ILIKE should be caseinsensitive. However, they are not. ogr2ogr plangebieden.shp -t_srs EPSG:28992 -dim 2 -spat 45000 405000 14 485000 WFS:"http://afnemers.ruimtelijkeplannen.nl/afnemers2012/services"; Bestem

Re: [gdal-dev] Numeric accuracy

Hi Ari and all, why not use machine epsilon ? somewhat over 1e-16 for double precision, or an even more tolerant tolerance? Most cases of meter coordinates 0.001 is realistic. Given projection errors, a bit more is also defensible. Am I to simplistic

Re: [gdal-dev] Improve clipping of shapefiles

Can't really help you with the OGR part, however, I wrote utilities do solve this (really) fast for calculating grids based on this type of intersection. Speed is often a matter of using specific tools utilizing the characteristics of the problem, i.c. the fishnet regularities. Can you share the sh

Re: [gdal-dev] Raster calculation optimalisation

Maybe it does not compile the formula but runs parser etc. as a kind of interpreter at each function call. Googled and saw http://beltoforion.de/article.php?a=muparsersse. Did not try, so no recommendation, let alone guarantee ;) You could check by defining a test function explicitly, if it is much

Re: [gdal-dev] SQLite driver hit and miss with spatial index

Hi, Out of interest, I tried the tack of converting to shapefile with a .dbf only containing the primary index OGC_FID. This might provide an alternative for indexed access to the geometry. I see it isn't needed anymore, but anyway, for info and comparison: What worked: - shapefile record count 252

Re: [gdal-dev] Release of a GDAL 2.0.x and 2.1.x Visual Studio 2010 Solution (static and dynamic lib)

Hi Mateusz, Thanks for your effort! It is really handy for me. FYI, a few more things I came across: - In the dll post build event there are 3 copy commands (dll, lib and exp). The second and third command need a space before 'copy'. - Enclosing source and destination file macros in quotes helps. -

Re: [gdal-dev] Release of a GDAL 2.0.x and 2.1.x Visual Studio 2010 Solution (static and dynamic lib)

Hi, Great stuff here! A minor point: please add quotes to the copy path in post-build event. Often people (like me) will have spaces in the project path, e.g. C:\Users\jan\Documents\Visual Studio 2015\Projects\GDAL_2.1.x_VC-master\libgdal-2.1.1. copy $(Projectdir)$(PlatformShortName)\$(Configuratio

Re: [gdal-dev] Interoperability issues with deleted features in shapefiles

Hi Even, everyone, Sorry for not including the list - my mistake. I've experimented with larger shapefiles than 2 GB but not necessarily in combination with editing. I'll do a few tests when I get around to it. Doesn't the .shx file get rewritten anyway? There could be some time-consuming actions a

Re: [gdal-dev] Reading shapefile PRJ failing

afaik PRJ files should be pure (ascii)text. You could refer to getting rid of BOM to clean them out. Do not know an additional parameter for the file opener/reader. Jan On Sat, Jul 12, 2014 at 6:45 PM, mcco

Re: [gdal-dev] SBN shapefile indexing

, 2014 at 7:48 PM, Even Rouault wrote: > Le mercredi 18 juin 2014 18:44:17, Jan Heckman a écrit : > > Hi all, > > > > Starting from https://github.com/drwelby/hasbeen I've implemented an SBN > > shapefile indexing routine in C++. I'm happy, so far, with

[gdal-dev] SBN shapefile indexing

Hi all, Starting from https://github.com/drwelby/hasbeen I've implemented an SBN shapefile indexing routine in C++. I'm happy, so far, with the results; I've used it for a few weeks now. With some work it could be integrated into OGR. Question is, how much interest, considering pros and cons of my

Re: [gdal-dev] Asking about feature field types of Shape File (SHP) ?

Not focusing on ogr, but on shapefile: Shapefile uses .dbf (dBase III) to store attributes (except geometrical attributes which are stored in the .shp file). This format does not know about integers, only about numericals (type N or F), with a specifiable precision (how many number-positions) and n

Re: [gdal-dev] Difficulty finding all parts of shapefile polygon using Python and osgeo/ogr

t;holes". I modified my code to account for the possibility of > this case and it works great now. > > Thanks again, and best regards, > Bruce Raup > > > On Sun, May 18, 2014 at 6:19 AM, Jan Heckman wrote: > >> Bruce, >> Attached is the digging - in C++, but should he

Re: [gdal-dev] Difficulty finding all parts of shapefile polygon using Python and osgeo/ogr

Hello Bruce, The point is that the geometrycount identifies two polygons, one a small little piece sort of inside a bay of your shape, the other the rest, along with the holes. See ogr_apitutorial - all assuming you are using 1.11: *Starting with OGR 1.11,

Re: [gdal-dev] shapefile enhancements

Thanks for the info. I'll look into GeoPackage! Jan On Thu, May 1, 2014 at 8:25 PM, Even Rouault wrote: > Le mercredi 30 avril 2014 22:02:34, Jan Heckman a écrit : > > Hi, > > > > Extending shapefile capabilities: The one reason which might persuade > > any

Re: [gdal-dev] shapefile enhancements

Just sharing my thoughts here. Jan On Wed, Apr 30, 2014 at 4:30 PM, Even Rouault wrote: > Le mardi 29 avril 2014 19:46:46, Jan Heckman a écrit : > > Hi, > > It appears I have to do some homework on ogr's shapefile functions as it > > stands now. > > > > 8GB: If

Re: [gdal-dev] shapefile enhancements

e, repair things. Best regards, Jan On Mon, Apr 28, 2014 at 11:48 PM, Even Rouault wrote: > Le lundi 28 avril 2014 23:39:34, Jan Heckman a écrit : > > Hi, > > > > I've used and maintained an offshoot shapefile library (thanks, Frank), > and > > added bits of fun

[gdal-dev] shapefile enhancements

Hi, I've used and maintained an offshoot shapefile library (thanks, Frank), and added bits of functionality when needed. There are two that I think might be of wider interest. 1) possibility to have shapefiles up to 8 GB (instead of 4 GB in ogr or 2 GB elsewhere); 2) implementing deletes. ad 1) t

Re: [gdal-dev] Import .GDB into MSSQL Server database

Hey Kyle, I had missed it can now read earlier gdb! Great! Thanks for the head(s)-up!! Jan On Wed, Apr 9, 2014 at 12:52 AM, Kyle Shannon wrote: > On Tue, Apr 8, 2014 at 4:47 PM, Jan Heckman wrote: > > Depends on the geodatabase version. > > If (AG) 10, there is a ogr driver

Re: [gdal-dev] Import .GDB into MSSQL Server database

Depends on the geodatabase version. If (AG) 10, there is a ogr driver for it, so you would not need shapefiles and avoid losing long fieldnames etc. There is no driver for earlier AG/gdb versions. Maybe it's an old thread, but it

Re: [gdal-dev] Feature request: gauss and other interpolations in gdalwarp

Hello Jan, There hasn't been a lot of response to your question, so I take the liberty to give you my thoughts, for what they are worth. My first guess was that you need something like a bilateral filter. Noise (detail) reduction is combined with edg

Re: [gdal-dev] extract information from polygon shapefile based on raster grid?

Hello Chao, I may not understand your wish quite well, but I think you need some datastructure which conforms to the elements of the grid and receives addtional information derived from a number of shapes with attributes. Your output could be another shapefile, with the summed results as attribut

Re: [gdal-dev] ReadAsArray: putting data in an already existent array

Hi Diego, I haven't checked this out, but my guess is that you should make a (python)list of the arrays if you need to keep them. If you don't need to keep them, you might try to del(ete) myarray before the next ReadAsArray call. Of course there is a lower level function which accepts an address a

Re: [gdal-dev] Issues with FileGDB support

Hi, I have the gdb support working on a windows box, compiled 1.9.1, some time ago. >From memory: Remember filegdb support is a plugin. The dll needs to be in a specific gdalplugins directory. I guess that if the support is compiled correctly and the dll is not found, this would be mentioned in an

Re: [gdal-dev] More information to understand ColorTables in GeoTiffs

Robert, Afaik, you need a three tuple for the color info. See e.g. http://osgeo-org.1560.n6.nabble.com/gdal-dev-tiff-with-colortable-loses-alpha-values-td3747611.html . It has a line giving an apparently working SetColorEntry() call without also using CreateColorRamp() which just complicates things

Re: [gdal-dev] nodata in .bil format

Hi, There should certainly not be a space between Nodata and Value as in your mail below. There are two formats relevant for (at least) arcgis: .bil. In this case the keyword is NoData .flt. In this case the keyword is NoData_Value. Capitalization, afaik, is irrelevant. So you could modify the .hdr

Re: [gdal-dev] Most optimal algorithm for contour correctness?

Hi, I've tackled a similar problem. Correctness of contour is a simple matter, except for one thing. That is the issue of saddles. That issue cannot be solved "correctly" since typically the data can be interpreted in several ways. This is also described in a wikipedia article (google for it, if yo

Re: [gdal-dev] libFileGDBAPI.so not found with compiling gdal 1.9.2 on openSUSE 12.2 x64.

Hi, On windows, using visual studio (2008 and 2012), I could compile both 32 bits and 64 bits, but only the 32 bits version actually worked. I haven't tried to figure out why the 64 bits version didn't work, 32 bits was good enough for me. Hope this is an option for you as well, Jan On Fri, Jan 25