Re: [gdal-dev] Process Sentinel-5p netCDF to tiff files but overlapping appears clearly between 2 files?

2021-11-09 Thread Pham Huu, Bang 
Did someone have the same issue to process Sentinel 5p to tiff file with gdal 
as well?

** The overlapping image on QGIS looks like below https://imgur.com/uu6pS5n

** The two extracted tiff files can be found from
https://drive.google.com/drive/folders/1lOXyJXhhzmfs5e_WTsKPOj96CHEEIyhP?usp=sharing


Best,


From: Pham Huu, Bang
Sent: Monday, November 8, 2021 07:09
To: gdal-dev@lists.osgeo.org 
Subject: Process Sentinel-5p netCDF to tiff files but overlapping appears 
clearly between 2 files?

Hello,

I've used 2 Sentinel-5p scenes in hdf5 / netCDF to geoTiff by gdal
version 2.2.4 like example below:

First, I extract the lat and lon coordinates from one HDF5 file

 gdal_translate -of VRT
HDF5:"S5P_OFFL_L2__AER_AI_20210501T110005_20210501T124135_18388_01_010400_20210503T005051.nc"://PRODUCT/latitude
lat.vrt
 gdal_translate -of VRT
HDF5:"S5P_OFFL_L2__AER_AI_20210501T110005_20210501T124135_18388_01_010400_20210503T005051.nc"://PRODUCT/longitude
lon.vrt

Then, I create the product.vrt like below


  
0
1
0
1
1
/home/PROJECTS/Sentinel-5P/test_L2_AER_AI/lon.vrt
1
/home/PROJECTS/Sentinel-5P/test_L2_AER_AI/lat.vrt
  
  

  9.9692099683869e+36
  7.78603750784e+35
  -7.209698677063
  2.6750402750247e+36


  HDF5:S5P_OFFL_L2__AER_AI_20210501T110005_20210501T124135_18388_01_010400_20210503T005051.nc://PRODUCT/aerosol_index_340_380
  1
  
  
  

  


And finally, I extract the data inside the HDF5 to geoTiff and project
it to EPSG:4326 CRS.

gdalwarp -geoloc -t_srs EPSG:4326 -srcnodata 9.96921e+36f
product.vrt aerosol_index_340_380.tiff -tr 0.069 0.069 -tap -of GTiff

I did that for 2 files which are neighbor to extract the wanted
sub_dataset to tiff files and then opened them on QGIS.

** The image looks like below https://imgur.com/uu6pS5n

** The two extracted tiff files can be found from
https://drive.google.com/drive/folders/1lOXyJXhhzmfs5e_WTsKPOj96CHEEIyhP?usp=sharing

I wonder, why there are clear borders (the line in the middle) between
the 2 overlapping files?

Thanks,

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Strange behavior opening CSVs

2021-11-09 Thread Simon Eves 
That makes sense. Thanks. Our plan is to trap CSV files before invoking the
GDAL identification.

The second part does seem like a bug, though.

On Mon, Nov 8, 2021 at 11:47 PM Rahkonen Jukka (MML) <
jukka.rahko...@maanmittauslaitos.fi> wrote:

> Hi,
>
>
>
> The trial with gdalinfo makes GDAL to try XYZ
> https://gdal.org/drivers/raster/xyz.html?highlight=xyz. It is not a bad
> guess at all. Ogrinfo tries CSV and finds the x, y, and val fields. I can’t
> say anything about the second part.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* gdal-dev  *Puolesta *Simon
> Eves
> *Lähetetty:* tiistai 9. marraskuuta 2021 2.21
> *Vastaanottaja:* gdal dev 
> *Aihe:* [gdal-dev] Strange behavior opening CSVs
>
>
>
> I am currently implementing a better file identifier, and during the
> course of development, I unintentionally let GDAL try to GDALOpenEx() the
> attached trivial CSV file.
>
>
>
> First, it reported "Ungridded dataset: At line 3, change of Y direction"
> (as a warning to the registered error-handler) and the returned
> OGRDataSource* was null.
>
>
>
> When I tried again, it then claimed that I was trying to open the same
> file recursively. implying that it had, in fact, opened it, even though it
> never returned a valid handle, so I wouldn't have been able to close it
> again even if I'd wanted to.
>
>
>
> I can reproduce the first part with gdalinfo.
>
>
>
> This is with GDAL 3.2.2 on Ubuntu 20.04 with GCC 9.
>
>
>
> I'll poke at the GDAL code myself when I have a bit more time.
>
>
>
> --
>
> 
>
> *Simon Eves*
>
> Senior Graphics Engineer, Rendering Group
> 100 Montgomery St (5th Floor), San Francisco, CA 94104, USA
>
> Email: simon.e...@omnisci.com | Cell:
>
> +1 (415) 902-1996
>
>
>


-- 

Simon Eves
Senior Graphics Engineer, Rendering Group
100 Montgomery St (5th Floor), San Francisco, CA 94104, USA


Email: simon.e...@omnisci.com | Cell:  +1 (415) 902-1996
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Gdal2tiles, Python 3.9 and multiprocessing

2021-11-09 Thread Rahkonen Jukka (MML) 
Hi,

Have a look at this gis.stackexchange question 
https://gis.stackexchange.com/questions/415925/error-in-gdal2tiles-with-processes.
Question is about using gdal2tiles with QGIS but the issue is reproduced also 
with OSGeo4w. Is there something changed in multiprocessing between Python 
versions 3.7 and 3.9, or is something else failing with errors:

RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:

if __name__ == '__main__':
freeze_support()
...

The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\multiprocessing\spawn.py", 
line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
  File "C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\multiprocessing\spawn.py", 
line 125, in _main
prepare(preparation_data)
  File "C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\multiprocessing\spawn.py", 
line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
  File "C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\multiprocessing\spawn.py", 
line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
  File "C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\runpy.py", line 268, in 
run_path
return _run_module_code(code, init_globals, run_name,
  File "C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\runpy.py", line 97, in 
_run_module_code
_run_code(code, mod_globals, init_globals,
  File "C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\runpy.py", line 87, in 
_run_code
exec(code, run_globals)
  File "C:\PROGRA~1\QGIS32~1.0\apps\Python39\Scripts\gdal2tiles.py", line 11, 
in 
sys.exit(main(sys.argv))
  File 
"C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\site-packages\osgeo_utils\gdal2tiles.py",
 line 3257, in main
multi_threaded_tiling(input_file, output_folder, options)
  File 
"C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\site-packages\osgeo_utils\gdal2tiles.py",
 line 3206, in multi_threaded_tiling
pool = Pool(processes=nb_processes)
  File "C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\multiprocessing\context.py", 
line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
  File "C:\PROGRA~1\QGIS32~1.0\apps\Python39\lib\multiprocessing\pool.py", line 
212, in __init__
self._repopulate_pool()

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev