Hi, I had a try with your dataset on Windows and I had no segmentation fault. Perhaps not surprise because I do not use the same container than you. I had some encoding problem but I believe that it is not related. But perhaps you have some use for knowing that the unzipping part is not necessary, shapefile can be read directly from the zip with /vsizip/.
ogr2ogr -f GeoJSON -oo ENCODING=MS932 d1_raw.geojson /vsizip/test.zip d1 --config CPL_DEBUG ON GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW. GDAL: GDALOpen(/vsizip/test.zip, this=0000023F19871250) succeeds as ESRI Shapefile. GDAL: GDALDriver::Create(GeoJSON,d1_raw.geojson,0,0,0,Unknown,0000000000000000) Shape: DBF Codepage = SJIS for /vsizip/test.zip/d1.shp Shape: Treating as encoding 'MS932'. Shape: Cannot recode from 'MS932'. Disabling recoding GDALVectorTranslate: 5315 features written in layer 'd1' Shape: 5315 features read on layer 'd1'. GDAL: GDALClose(/vsizip/test.zip, this=0000023F19871250) GDAL: GDALClose(d1_raw.geojson, this=0000023F19876CF0) GDAL: In GDALDestroy - unloading GDAL shared library. -Jukka Rahkonen- Lähettäjä: gdal-dev <gdal-dev-boun...@lists.osgeo.org> Puolesta ?? ?? via gdal-dev Lähetetty: tiistai 12. maaliskuuta 2024 10.56 Vastaanottaja: Abel Pau <a....@creaf.uab.cat>; gdal-dev@lists.osgeo.org Aihe: Re: [gdal-dev] Segmentation fault (core dumped) occurs when running ogr2ogr command on container Hi, I tried executing with --config CPL_DEBUG ON, but it did not provide any useful information. I'm sending you the command that's causing the issue. We're using curl command to obtain the Shape file from the internet. --- 1. Obtaining the Shape file: $curl -f --output-dir ./ -O https://data.bodik.jp/dataset/d4b9a284-58c0-4559-9e48-b7c83a4275f0/resource/28aa8f13-7cf0-4dc5-9dbf-a77f58c3f2c8/download/shape.zip 1. Unzipping the ZIP file: $unzip -o -j -d ./ ./shape.zip ③ Converting from Shape format to GeoJSON format: $ogr2ogr -f GeoJSON -oo ENCODING=MS932 ./d1_raw.geojson ./d1.shp --config CPL_DEBUG ON --- These commands are executed within the convert.sh called from the Dockerfile. Best regards, From: Abel Pau <a....@creaf.uab.cat<mailto:a....@creaf.uab.cat>> Sent: Wednesday, March 6, 2024 5:53 PM To: 寛座 智大 <tomohiro.kanza....@baycurrent.co.jp<mailto:tomohiro.kanza....@baycurrent.co.jp>>; gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Subject: RE: Segmentation fault (core dumped) occurs when running ogr2ogr command on container Hi, I reduced the amount of memory requested. I could do that because I am developing the driver that failed. After my mail I discovered another error, so, perhaps, this was the real cause. 4Mb it’s so small, so I is not the cause... Perhaps you can add --config CPL_DEBUG ON at your command line to see if this give you more information about the place where it crashes. I’m sorry not being able to help you more. Perhaps you can send me the shape and the command line to see if it crashes in my computer and I can find out the problem. See you De: 寛座 智大 <tomohiro.kanza....@baycurrent.co.jp<mailto:tomohiro.kanza....@baycurrent.co.jp>> Enviado el: dimecres, 6 de març de 2024 7:20 Para: Abel Pau <a....@creaf.uab.cat<mailto:a....@creaf.uab.cat>>; gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Asunto: RE: Segmentation fault (core dumped) occurs when running ogr2ogr command on container Abel Pau, Hi. Thank you for your response. >I reduced this capacity 10 times and it worked. Does this mean you reduced the size of the Shape file? Or does it mean that you reduced the amount of memory requested? ( If it's the latter, I'd like to learn how to do it.) Shape files are typically around 4MB, so I don't consider them to be very large files. Additionally, I tried it on an AKS with a free memory size of over 20GB, but encountered the same issue. It seems unlikely that there is a lack of specifications in the execution environment. Could the issue possibly be due to a bug in the command? From: Abel Pau <a....@creaf.uab.cat<mailto:a....@creaf.uab.cat>> Sent: Monday, March 4, 2024 4:57 PM To: 寛座 智大 <tomohiro.kanza....@baycurrent.co.jp<mailto:tomohiro.kanza....@baycurrent.co.jp>> Subject: RE: Segmentation fault (core dumped) occurs when running ogr2ogr command on container Hi, I did similar tests with a driver I am implementing and I obtained exact the same behavior as yours. After a few speculations I deduced that my driver was asking for too much memory for the docker capacity. I reduced this capacity 10 times and it worked. So, perhaps, Shape or Geojson asks for too much memory for the docker capacity. I don’t know if it’s possible to make it bigger. Abel. De: gdal-dev <gdal-dev-boun...@lists.osgeo.org<mailto:gdal-dev-boun...@lists.osgeo.org>> En nombre de ?? ?? via gdal-dev Enviado el: dilluns, 4 de març de 2024 3:30 Para: gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> Asunto: [gdal-dev] Segmentation fault (core dumped) occurs when running ogr2ogr command on container I am implementing batch processing to convert Shape files, publicly available as open data, into Geojson format within a container environment. I installed gdal-bin on top of ubuntu:22.04 base image. The Dockerfile I implemented is as follows: ``` FROM ubuntu:22.04 ENV DEBIAN_FRONTEND=noninteractive RUN apt update RUN apt -y install apt-transport-https jq unzip openjdk-17-jdk-headless curl # Install gdal-bin # Reference: https://mothergeo-py.readthedocs.io/en/latest/development/how-to/gdal-ubuntu-pkg.html#how-to-install-gdal-ogr-packages-on-ubuntu RUN apt -y install software-properties-common RUN add-apt-repository -y ppa:ubuntugis/ppa RUN apt update RUN apt -y install gdal-bin COPY ./convert.sh /convert.sh CMD ["bash","convert.sh"] ``` When executing the ogr2ogr command within convert.sh, the following error occurs: ``` Shape: DBF Codepage = SJIS for /d1.shp Shape: Treating as encoding 'MS932'. GDAL: GDALOpen(/d1.shp, this=0x55f1edd77f30) succeeds as ESRI Shapefile. GDAL: QuietDelete(/d1_raw.geojson) invoking Delete() GDAL: GDALDriver::Create(GeoJSON,/d1_raw.geojson,0,0,0,Unknown,(nil)) GDALVectorTranslate: 5315 features written in layer 'd1' Shape: 5315 features read on layer 'd1'. GDAL: GDALClose(/d1.shp, this=0x55f1edd77f30) GDAL: GDALClose(/d1_raw.geojson, this=0x55f1edd75fd0) GDAL: In GDALDestroy - unloading GDAL shared library. Segmentation fault (core dumped) ``` I checked if convert.sh execution is possible on the VM (host OS running Docker). The error seems to occur only when executed within the container. How should I resolve this? ※Environment Host OS: ubuntu 22.04 Docker Deamon: 25.0.3 gdal-bin: 3.6.4
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
