Re: [gdal-dev] can't handle grib file
Even, On Thu, Aug 23, 2012 at 8:36 PM, Even Rouault wrote: > > > So it looks like the information is actually there, but the gdal driver > > can't handle it. Any hint? > > Margherita, > > There's a possibility that your data file is of a type that isn't > understood by > the underlying library that GDAL uses to decode GRIB files. Or perhaps GDAL > isn't using it correctly with that particular file. Difficult to know. > > GDAL uses the g2clib library (v1.0.4), that has evolved ( > http://www.nco.ncep.noaa.gov/pmb/docs/grib2/download/g2clib.changes ) > since it > was integrated in GDAL. So perhaps a newer version of it better supports > your > data file. (but merging a newer version to GDAL might be non trivial, > since a > lot of fixes were done in GDAL copy (to address cross-platform portability, > etc..)). > > Thank you for your hints. I understand that grib is a pretty much flexible type of file and in some cases flexibility is difficult to handle. If you devs think it's worthy, I'd file a ticket about it and I can send you the grib file if you wish to make tests on it. Please let me know. Thanks, -- Dr. Margherita Di Leo ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] FWTools outdated on GDAL/OGR download page
Le 24/08/2012 07:24, Stefan Keller a écrit : Hi, The FWTools mentioned on the bottom of the download page of GDAL/OGR Binaries [1] leads to releases where the latest build dates back to 2007. I think there should be a note that FWTools are outdated and not maintained any more - unless I'm wrong. Hi, There is already a note : The latest FWtools version for Windows, 2.4.7, dates back to a pre-1.6 GDAL version. In order to benefit from the latest and greatest, you can refer to the other binary builds mentionned above. Jean-Claude ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] About sqlite tileindex and Mapserver
Hi, There is a trick with using Spatialite tileindex with Mapserver. I have no idea anymore how I came to this solution but most probably I was reading something about it from the internet. Try odd trick is that you cannot user Spatialite directly as tileindex but you must first create a LAYER from Spatialite and use that as tileindex. Let's take an example. In the following "z:\tindex\aerial_images.sqlite" is the Spatialite db and tile locations are in the table "aerial_images" LAYER NAME "sqlite_tileindex" STATUS OFF TYPE POLYGON CONNECTIONTYPE OGR CONNECTION "z:\tindex\aerial_images.sqlite" DATA "aerial_images" END Now for the real aerial image layer the tileindex is read from the LAYER that was just created LAYER NAME "aerial_imagery" STATUS ON TILEINDEX "sqlite_tileindex" -Jukka Rahkonen- ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] About sqlite tileindex and Mapserver
Jukka, Perhaps you can document this on the Wiki as a Spatialite example in addition to the example for Oracle (https://github.com/mapserver/mapserver/wiki/MapServer-TILEINDEXes-with-Dat abase-RASTERS) Mike -- Michael Smith US Army Corps Remote Sensing GIS/Center On 8/24/12 7:12 AM, "Rahkonen Jukka" wrote: >Hi, > >There is a trick with using Spatialite tileindex with Mapserver. I have >no idea anymore how I came to this solution but most probably I was >reading something about it from the internet. > >Try odd trick is that you cannot user Spatialite directly as tileindex >but you must first create a LAYER from Spatialite and use that as >tileindex. Let's take an example. In the following >"z:\tindex\aerial_images.sqlite" is the Spatialite db and tile locations >are in the table "aerial_images" > >LAYER >NAME "sqlite_tileindex" >STATUS OFF >TYPE POLYGON >CONNECTIONTYPE OGR >CONNECTION "z:\tindex\aerial_images.sqlite" >DATA "aerial_images" >END > >Now for the real aerial image layer the tileindex is read from the LAYER >that was just created > >LAYER >NAME "aerial_imagery" >STATUS ON >TILEINDEX "sqlite_tileindex" > > >-Jukka Rahkonen- >___ >gdal-dev mailing list >gdal-dev@lists.osgeo.org >http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] About sqlite tileindex and Mapserver
Hi, It is documented in http://mapserver.org/mapfile/layer.html . TILEINDEX [filename|layername] Name of the tileindex file or layer. A tileindex is similar to an ArcInfo library index. The tileindex contains polygon features for each tile. The item that contains the location of the tiled data is given using the TILEITEM parameter. When a file is used as the tileindex for shapefile or raster layers, the tileindex should be a shapefile. For CONNECTIONTYPE OGR layers, any OGR supported datasource can be a tileindex. Normally the location should contain the path to the tile file relative to the shapepath, not relative to the tileindex itself. If the DATA parameter contains a value then it is added to the end of the location. When a tileindex layer is used, it works similarly to directly referring to a file, but any supported feature source can be used (ie. postgres, oracle). Perhaps this tileindex section could have direct links to some tileindex layer examples rather than having them only in wiki? -Jukka Rahkonen- Smith, Michael wrote: > Jukka, > > Perhaps you can document this on the Wiki as a Spatialite example in > addition to the example for Oracle > (https://github.com/mapserver/mapserver/wiki/MapServer-TILEINDEXes- > with-Dat > abase-RASTERS) > > Mike > > -- > Michael Smith > > US Army Corps > Remote Sensing GIS/Center > > > > On 8/24/12 7:12 AM, "Rahkonen Jukka" > wrote: > > >Hi, > > > >There is a trick with using Spatialite tileindex with Mapserver. I have > >no idea anymore how I came to this solution but most probably I was > >reading something about it from the internet. > > > >Try odd trick is that you cannot user Spatialite directly as tileindex > >but you must first create a LAYER from Spatialite and use that as > >tileindex. Let's take an example. In the following > >"z:\tindex\aerial_images.sqlite" is the Spatialite db and tile > >locations are in the table "aerial_images" > > > >LAYER > >NAME "sqlite_tileindex" > >STATUS OFF > >TYPE POLYGON > >CONNECTIONTYPE OGR > >CONNECTION "z:\tindex\aerial_images.sqlite" > >DATA "aerial_images" > >END > > > >Now for the real aerial image layer the tileindex is read from the > >LAYER that was just created > > > >LAYER > >NAME "aerial_imagery" > >STATUS ON > >TILEINDEX "sqlite_tileindex" > > > > > >-Jukka Rahkonen- > >___ > >gdal-dev mailing list > >gdal-dev@lists.osgeo.org > >http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Large raster (ecw) identify very long
Hi, Even was missing a proper sample image in ECW format. This one comes originally from the geotorrent.org site which obviously disappeared at the same time than ER Mapper. Anyway, I have been trying to keep the image available. Image is a mosaic made from Landsat landcover scenes (bands 7-4-2) and it is in public domain. http://laillisettorrentit.net/index.php?page=torrent-details&id=d2c502c1aec3cce0180d92b31721597cbb523751 -Jukka Rahkonen- ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] About sqlite tileindex and Mapserver
On 12-08-24 8:22 AM, Smith, Michael ERDC-RDE-CRREL-NH wrote: > Jukka, > > Perhaps you can document this on the Wiki as a Spatialite example in > addition to the example for Oracle > (https://github.com/mapserver/mapserver/wiki/MapServer-TILEINDEXes-with-Dat > abase-RASTERS) > > Mike > I would side with keeping the various examples in the wiki (you can even add screengrabs of tables and index bounding rectangles there directly in your wiki page). -jeff -- Jeff McKenna MapServer Consulting and Training Services http://www.gatewaygeomatics.com/ ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] [Qgis-developer] Large raster (ecw) identify very long
Le mercredi 22 août 2012 20:13:39, Radim Blazek a écrit : > Even, > thanks for exhaustive explanation and testing. > > On Wed, Aug 22, 2012 at 2:37 PM, Even Rouault > > wrote: > >> I found in GDAL ecwdataset.cpp that it is treating single row > > > >> requests in IRasterIO in a special way: > > I tried the following Python script that must be representative of how > > QGIS must do picking (I guess it does a RasterIO(, x, y, 1, 1, ... > > 1, 1) ) > > Yes. > > > I suppose your workaround in QGIS will be to read 1x2 pixel or something > > like that. > > Yes, I have used 2x2. Radim, I've tested the big ECW (1.8 GB, 141970 x 141970) provided by Jukka and could indeed reproduce a noticable performance issue when simulating picking with my tiny GDAL python script. In QGIS, the performance was still OK however, a few hundreds of milliseconds for each picking. It is just when you do a lot of iterations that you begin to notice the slowness, or if the dataset is really huge. I've fixed that in http://trac.osgeo.org/gdal/ticket/4790 in trunk and 1.9 branch. So it will be in GDAL 1.9.2. Actually, as I think that the ECW driver is generally available through a plugin (and not built-in in GDAL DLL), this could be available sooner if the ECW plugin is rebuilt with the fixed sources. Best regards, Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] [Qgis-developer] Large raster (ecw) identify very long
On Fri, Aug 24, 2012 at 1:45 PM, Even Rouault wrote: > I've fixed that in http://trac.osgeo.org/gdal/ticket/4790 in trunk and 1.9 > branch. So it will be in GDAL 1.9.2. Actually, as I think that the ECW driver > is generally available through a plugin (and not built-in in GDAL DLL), this > could be available sooner if the ECW plugin is rebuilt with the fixed sources. Folks, I have created a calendar item for me on the weekend to rebuild the 1.9 ECW plugin for OSGeo4W. Hopefully I'll get it done soon. Best regards, -- ---+-- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] SQL Server Driver
Hello, I've been using SQL Server Driver for a few days now, and I noticed it was very slow, whatever the parameters, such as GT, I was using. I need to insert millions of rows in a table. To give you an idea, it takes around 1 minute to import 10k records with the driver On the other hand, if I write my own routine which reads a MapInfo file and does batch insert into the database, I can insert over 125k records per minute. I tried reading the same TAB file with ogr2ogr but writing to PostGis and noticed the speed was about the same than my routine, which is enough for my needs. Would it be possible that the SQL Server Driver does not do bulk insert? Thanks for the help. Jimmy Duchesne ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] can't handle grib file
Ciao Years ago to handling easily grib file i was using CDO " http://www.nersc.gov/users/software/vis-analytics/climate-data-operators-cdo/ " . it can also convert grib to geoTiff. ciao On 24 August 2012 02:05, Margherita Di Leo wrote: > Even, > > On Thu, Aug 23, 2012 at 8:36 PM, Even Rouault > wrote: >> >> >> > So it looks like the information is actually there, but the gdal driver >> > can't handle it. Any hint? >> >> Margherita, >> >> There's a possibility that your data file is of a type that isn't >> understood by >> the underlying library that GDAL uses to decode GRIB files. Or perhaps >> GDAL >> isn't using it correctly with that particular file. Difficult to know. >> >> GDAL uses the g2clib library (v1.0.4), that has evolved ( >> http://www.nco.ncep.noaa.gov/pmb/docs/grib2/download/g2clib.changes ) >> since it >> was integrated in GDAL. So perhaps a newer version of it better supports >> your >> data file. (but merging a newer version to GDAL might be non trivial, >> since a >> lot of fixes were done in GDAL copy (to address cross-platform >> portability, >> etc..)). >> > Thank you for your hints. I understand that grib is a pretty much flexible > type of file and in some cases flexibility is difficult to handle. If you > devs think it's worthy, I'd file a ticket about it and I can send you the > grib file if you wish to make tests on it. Please let me know. > > Thanks, > > -- > Dr. Margherita Di Leo > > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/gdal-dev -- Giuseppe Amatulli Web: www.spatial-ecology.net ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] SQL Server Driver
Hi, Assuming you refer to the MSSQL driver the driver should provide fast data retrieval, but hasn't yet been optimized to provide fast data upload. Currently the MSSQL driver use WKT for the geometries when submitting data to the server, while when retrieving data we use the native SqlGeometry/SqlGeography serialization format. Not sure what you mean by "do not do bulk insert". How your own routine is looking like? Is this a .NET based program (using SqlGeometry by default)? Best regards, Tamas 2012/8/25 Duchesne, Jimmy > Hello, > > ** ** > > I’ve been using SQL Server Driver for a few days now, and I noticed it was > very slow, whatever the parameters, such as GT, I was using. > > ** ** > > I need to insert millions of rows in a table. > > ** ** > > To give you an idea, it takes around 1 minute to import 10k records with > the driver > > ** ** > > On the other hand, if I write my own routine which reads a MapInfo file > and does batch insert into the database, I can insert over 125k records per > minute. > > ** ** > > I tried reading the same TAB file with ogr2ogr but writing to PostGis and > noticed the speed was about the same than my routine, which is enough for > my needs. > > ** ** > > Would it be possible that the SQL Server Driver does not do bulk insert?** > ** > > ** ** > > Thanks for the help. > > ** ** > > *Jimmy Duchesne *** > > ** ** > > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] SQL Server Driver
Jimmy, >From an examination of MSSQLSpatialTableLayer::CreateFeature() it appears a regular INSERT statement is done for each feature written to the db rather than any sort of bulk mechanism. I could be missing something of course. Hopefully Tamas will provide a more experienced answer. Best regards, Frank On Fri, Aug 24, 2012 at 3:03 PM, Duchesne, Jimmy wrote: > Hello, > > > > I’ve been using SQL Server Driver for a few days now, and I noticed it was > very slow, whatever the parameters, such as GT, I was using. > > > > I need to insert millions of rows in a table. > > > > To give you an idea, it takes around 1 minute to import 10k records with the > driver > > > > On the other hand, if I write my own routine which reads a MapInfo file and > does batch insert into the database, I can insert over 125k records per > minute. > > > > I tried reading the same TAB file with ogr2ogr but writing to PostGis and > noticed the speed was about the same than my routine, which is enough for my > needs. > > > > Would it be possible that the SQL Server Driver does not do bulk insert? > > > > Thanks for the help. > > > > Jimmy Duchesne > > > > > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/gdal-dev -- ---+-- I set the clouds in motion - turn up | Frank Warmerdam, warmer...@pobox.com light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush| Geospatial Software Developer ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] can't handle grib file
On Fri, Aug 24, 2012 at 7:27 PM, Giuseppe Amatulli wrote: > Ciao > Years ago to handling easily grib file i was using > CDO " > http://www.nersc.gov/users/software/vis-analytics/climate-data-operators-cdo/ > " . > it can also convert grib to geoTiff. I doubt cdo can convert to geotiff (in fact I am sure of ir, as I was having a discussion with the devs about it some months ago). Perhaps you are confusing with other software (or a really old versions used to do that). However, it can convert to netcdf which is well supported in gdal/qgis. The proper (current) page for cdo is https://code.zmaw.de/projects/cdo/ cheers Etienne > ciao > > On 24 August 2012 02:05, Margherita Di Leo wrote: >> Even, >> >> On Thu, Aug 23, 2012 at 8:36 PM, Even Rouault >> wrote: >>> >>> >>> > So it looks like the information is actually there, but the gdal driver >>> > can't handle it. Any hint? >>> >>> Margherita, >>> >>> There's a possibility that your data file is of a type that isn't >>> understood by >>> the underlying library that GDAL uses to decode GRIB files. Or perhaps >>> GDAL >>> isn't using it correctly with that particular file. Difficult to know. >>> >>> GDAL uses the g2clib library (v1.0.4), that has evolved ( >>> http://www.nco.ncep.noaa.gov/pmb/docs/grib2/download/g2clib.changes ) >>> since it >>> was integrated in GDAL. So perhaps a newer version of it better supports >>> your >>> data file. (but merging a newer version to GDAL might be non trivial, >>> since a >>> lot of fixes were done in GDAL copy (to address cross-platform >>> portability, >>> etc..)). >>> >> Thank you for your hints. I understand that grib is a pretty much flexible >> type of file and in some cases flexibility is difficult to handle. If you >> devs think it's worthy, I'd file a ticket about it and I can send you the >> grib file if you wish to make tests on it. Please let me know. >> >> Thanks, >> >> -- >> Dr. Margherita Di Leo >> >> ___ >> gdal-dev mailing list >> gdal-dev@lists.osgeo.org >> http://lists.osgeo.org/mailman/listinfo/gdal-dev > > > > -- > Giuseppe Amatulli > Web: www.spatial-ecology.net > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
