[gdal-dev] Re: Resquest for comments (RFC 36)

[Jukka Rahkonen]

Ivan Lucena  pmldnet.com> writes:

> 
> Hi All,
> 
> RFC 36: http://trac.osgeo.org/gdal/wiki/rfc36_open_by_drivername
> 
> The goal of RFC 36 is to save processing time by telling GDALOpen what driver
to use, avoiding the
> driver-probing mechanism (Where all drivers are asked to checked if they
recognize the file).

You can add to the motivation of RFC also the case when a file format is
supported by several drivers. I do not know if there are others than JPEG2000
but for that GDAL has JPEG2000, JP2KAK, JP2ECW, JP2MrSID and JP2OpenJPEG
drivers. If several drivers are configured the only way to control which one is
used is to use GDAL_SKIP. Being able to tell which driver to use instead of
which one(s) not to use feels much better for me.

I can easily imagine that someone making benchmark with different GDAL JPEG2000
drivers could get totally wrong results if the new driver:file opening system
for some reason fails and GDAL takes automatically the first capable JPEG2000
driver from the list without warning the user. Wouldn't it be better to throw an
error by default?

-Jukka Rahkonen-

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Resquest for comments (RFC 36)

[Even Rouault]

Le mardi 04 octobre 2011 04:52:09, Daniel Morissette a écrit :
> On 11-10-03 10:04 PM, Frank Warmerdam wrote:
> > I believe that is because it is hard to know if HFA:test.tif was really
> > something other than a request for the HFA driver.  It could actually
> > be a file named "HFA:test.tif

Daniel's suggestion to not fallback to the regular Open() that loops over the 
driver, once we've recognized the driver name, could make sense actually. 
Because Ivan's code is not called if the "drivername:filename" is a regular 
filename ( see if ( oOpenInfo.bStatOK == false && strstr( pszFilename, ":" ) != 
NULL  )  ).

> > " or it could be that another driver happens
> > to use the HFA: prefix to mean something.

That would be pretty confusing and should be banned. I hope there's no such 
case in the current code base !

> Um... the best I can say then is -0 in the hope that we can find a
> better delimiter or syntax. Unfortunately I just can't come up with a
> better option at the moment.

There's already another GDALOpenInternal() that I pushed some time ago for the 
purpose of the NITF driver (that needs to restrict the opening of JPEG2000 
datastream to the JPEG2000 capable driver that are PAM capable). That 
GDALOpenInternal() accepts a list of driver names as an additional argument. 

That could be an alternative to Ivan's solution. The downside of this approach 
is that as it is a new API, you need to retrofit all callers, such as gdalinfo, 
gdal_translate, and all the other gdal utilities/scripts etc.., to add a new 
user option to specify the driver name(s).

So I don't see any obvious winner between the 2 approachs (embedding the 
driver name in the string vs providing a new argument to GDALOpenXXX() )

> 
> > I'm not convinced that this is necessary.  I have had a hope for
> > some time to provide a standard way of binding up open options,
> > including the driver name, for OGR as demonstrated by a
> > 
> > request like:
> >@driver=ingres,dbname=test
> 
> Unfortunately "@driver=ingres,dbname=test.tif" is also a valid filename,
> so we'd have the same problem as with "HFA:test.tif" above.
> 
> I also think it is important to be consistent between GDAL and OGR for
> something like this.
> 
> Once again, I'm not trying to kill the idea, just pointing the flaws in
> the hope that the someone in the group will find a more predictable
> mechanism.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Resquest for comments (RFC 36)

[Even Rouault]

Le mardi 04 octobre 2011 05:55:27, Chaitanya kumar CH a écrit :
> What happens when a user tries to open a file named "hfa:example_file.img"
> the current way when there is also a file name "example_file.img"?

That should be fine. Ivan's code is only triggered :

if ( oOpenInfo.bStatOK == false && strstr( pszFilename, ":" ) != NULL  ) 

So if "hfa:example_file.img" is a regular file, it will be opened through the 
usual open mechanism.

> 
> What if we try to reorder the driver list according to the file extension?

That was one of the previous Ivan's attempt (actually the reordering was done 
according to the provided driver name, not file extension which is not a 100% 
reliable way of guessing the driver), but there are multithread concurrency 
problems with this solution that would oblige to hold a mutex around pfn-
>Open(), which would hurt scalability.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Resquest for comments (RFC 36)

[Chaitanya kumar CH]

What happens when a user tries to open a file named "hfa:example_file.img"
the current way when there is also a file name "example_file.img"?

What if we try to reorder the driver list according to the file extension?

On Tue, Oct 4, 2011 at 8:22 AM, Daniel Morissette
wrote:

> On 11-10-03 10:04 PM, Frank Warmerdam wrote:
>
>>
>> I believe that is because it is hard to know if HFA:test.tif was really
>> something other than a request for the HFA driver.  It could actually
>> be a file named "HFA:test.tif" or it could be that another driver happens
>> to use the HFA: prefix to mean something.   I believe this is an
>> effort to avoid failure in these cases.
>>
>>
> Um... the best I can say then is -0 in the hope that we can find a better
> delimiter or syntax. Unfortunately I just can't come up with a better option
> at the moment.
>
>
>
>> I'm not convinced that this is necessary.  I have had a hope for
>> some time to provide a standard way of binding up open options,
>> including the driver name, for OGR as demonstrated by a
>> request like:
>>
>>   @driver=ingres,dbname=test
>>
>>
> Unfortunately "@driver=ingres,dbname=test.**tif" is also a valid filename,
> so we'd have the same problem as with "HFA:test.tif" above.
>
> I also think it is important to be consistent between GDAL and OGR for
> something like this.
>
> Once again, I'm not trying to kill the idea, just pointing the flaws in the
> hope that the someone in the group will find a more predictable mechanism.
>
>
> --
> Daniel Morissette
> http://www.mapgears.com/
> Provider of Professional MapServer Support since 2000
>
> __**_
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/**mailman/listinfo/gdal-dev
>



-- 
Best regards,
Chaitanya kumar CH.

+91-9494447584
17.2416N 80.1426E
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Resquest for comments (RFC 36)

[Daniel Morissette]

On 11-10-03 10:04 PM, Frank Warmerdam wrote:


I believe that is because it is hard to know if HFA:test.tif was really
something other than a request for the HFA driver.  It could actually
be a file named "HFA:test.tif" or it could be that another driver happens
to use the HFA: prefix to mean something.   I believe this is an
effort to avoid failure in these cases.



Um... the best I can say then is -0 in the hope that we can find a 
better delimiter or syntax. Unfortunately I just can't come up with a 
better option at the moment.




I'm not convinced that this is necessary.  I have had a hope for
some time to provide a standard way of binding up open options,
including the driver name, for OGR as demonstrated by a
request like:

   @driver=ingres,dbname=test



Unfortunately "@driver=ingres,dbname=test.tif" is also a valid filename, 
so we'd have the same problem as with "HFA:test.tif" above.


I also think it is important to be consistent between GDAL and OGR for 
something like this.


Once again, I'm not trying to kill the idea, just pointing the flaws in 
the hope that the someone in the group will find a more predictable 
mechanism.


--
Daniel Morissette
http://www.mapgears.com/
Provider of Professional MapServer Support since 2000

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Resquest for comments (RFC 36)

[Frank Warmerdam]

On Mon, Oct 3, 2011 at 6:14 PM, Daniel Morissette
 wrote:
> I like the idea, especially since it can increase performance in some cases.
> However unless I'm mistaken the proposed patch seems to fall back on the
> regular test-open loop when the GDALOpenInternal() call with the explicit
> driver name failed, no matter why it failed. e.g. I specify the invalid
> "hfa:test.tif" open string, then I should get a NULL return value and not a
> handle to a TIFF file.

Daniel,

I believe that is because it is hard to know if HFA:test.tif was really
something other than a request for the HFA driver.  It could actually
be a file named "HFA:test.tif" or it could be that another driver happens
to use the HFA: prefix to mean something.   I believe this is an
effort to avoid failure in these cases.

> The patch would need to be reworked for this, but I think that if the
> GetDriverByName() call succeeds (i.e. the driver name prefix is recognized
> by GDAL) and the call to poDriver->pfnOpen() fails then we should return
> NULL to the application code (and not fall back on the test-open loop).
> OTOH, if the GetDriverByName() lookup failed, then we should fall back on
> the test-open loop.
>
> Also, if we implement this mechanism in GDAL, then we'd have to provide it
> in OGR as well to be consistent.

I'm not convinced that this is necessary.  I have had a hope for
some time to provide a standard way of binding up open options,
including the driver name, for OGR as demonstrated by a
request like:

  @driver=ingres,dbname=test

One objection I had previously to Ivan's proposal is that I'm
interested in pursuing this general purpose mechanism for
many open options in OGR and in GDAL.  But since I haven't
gotten to it for so long there is no point in holding up Ivan's
fairly straight forward change.  Nevertheless, I'm hesitant to
push his change to OGR unnecessarily.

Best regards,
-- 
---+--
I set the clouds in motion - turn up   | Frank Warmerdam, warmer...@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Software Developer
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Resquest for comments (RFC 36)

[Daniel Morissette]

I like the idea, especially since it can increase performance in some 
cases. However unless I'm mistaken the proposed patch seems to fall back 
on the regular test-open loop when the GDALOpenInternal() call with the 
explicit driver name failed, no matter why it failed. e.g. I specify the 
invalid "hfa:test.tif" open string, then I should get a NULL return 
value and not a handle to a TIFF file.


The patch would need to be reworked for this, but I think that if the 
GetDriverByName() call succeeds (i.e. the driver name prefix is 
recognized by GDAL) and the call to poDriver->pfnOpen() fails then we 
should return NULL to the application code (and not fall back on the 
test-open loop). OTOH, if the GetDriverByName() lookup failed, then we 
should fall back on the test-open loop.


Also, if we implement this mechanism in GDAL, then we'd have to provide 
it in OGR as well to be consistent.


Daniel


On 11-10-03 08:02 PM, Ivan Lucena wrote:

Hi All,

RFC 36: http://trac.osgeo.org/gdal/wiki/rfc36_open_by_drivername

The goal of RFC 36 is to save processing time by telling GDALOpen what driver 
to use, avoiding the driver-probing mechanism (Where all drivers are asked to 
checked if they recognize the file).

It works by adding the driver name before the file-name as in 
GDALOpen("hfa:example_file.img").

*It is optional*. It does not invalidate the current simple filename entry that most of 
the drivers use nor the more complex "filename" string formats used by a few 
drivers (with things like server-name, port-number, table-name).

It is intended to speed up applications or scripts that need to process a large 
number of datasets when it already knows what driver to use.

Command line use should not be affect considerably in terms of speed but it 
adds the option to avoid the wrong driver from taking precedence from the 
intended one (#3043), ex.:

$ gdalinfo ntif:/autotest/gdrivers/data/rgb.ntf"

If you want to try it, a patch that implements this proposed change is 
available on ticket 3043 http://trac.osgeo.org/gdal/ticket/3043

Please take a look and send us your comments.

Regards,

Ivan
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev



--
Daniel Morissette
http://www.mapgears.com/
Provider of Professional MapServer Support since 2000

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Resquest for comments (RFC 36)

[Brian Case]

Ivan

Makes me wonder what kind of performance gain you could see with
mapserver on a layer with a large number of images to open.

Seems to me that a tileindex would be a good application of this.

Brian




On Mon, 2011-10-03 at 19:02 -0500, Ivan Lucena wrote:
> Hi All,
> 
> RFC 36: http://trac.osgeo.org/gdal/wiki/rfc36_open_by_drivername
> 
> The goal of RFC 36 is to save processing time by telling GDALOpen what driver 
> to use, avoiding the driver-probing mechanism (Where all drivers are asked to 
> checked if they recognize the file).
> 
> It works by adding the driver name before the file-name as in 
> GDALOpen("hfa:example_file.img").
> 
> *It is optional*. It does not invalidate the current simple filename entry 
> that most of the drivers use nor the more complex "filename" string formats 
> used by a few drivers (with things like server-name, port-number, table-name).
> 
> It is intended to speed up applications or scripts that need to process a 
> large number of datasets when it already knows what driver to use.
> 
> Command line use should not be affect considerably in terms of speed but it 
> adds the option to avoid the wrong driver from taking precedence from the 
> intended one (#3043), ex.:
> 
> $ gdalinfo ntif:/autotest/gdrivers/data/rgb.ntf"
> 
> If you want to try it, a patch that implements this proposed change is 
> available on ticket 3043 http://trac.osgeo.org/gdal/ticket/3043
> 
> Please take a look and send us your comments.
> 
> Regards,
> 
> Ivan
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Resquest for comments (RFC 36)

[Ivan Lucena]

Hi All,

RFC 36: http://trac.osgeo.org/gdal/wiki/rfc36_open_by_drivername

The goal of RFC 36 is to save processing time by telling GDALOpen what driver 
to use, avoiding the driver-probing mechanism (Where all drivers are asked to 
checked if they recognize the file).

It works by adding the driver name before the file-name as in 
GDALOpen("hfa:example_file.img").

*It is optional*. It does not invalidate the current simple filename entry that 
most of the drivers use nor the more complex "filename" string formats used by 
a few drivers (with things like server-name, port-number, table-name).

It is intended to speed up applications or scripts that need to process a large 
number of datasets when it already knows what driver to use.

Command line use should not be affect considerably in terms of speed but it 
adds the option to avoid the wrong driver from taking precedence from the 
intended one (#3043), ex.:

$ gdalinfo ntif:/autotest/gdrivers/data/rgb.ntf"

If you want to try it, a patch that implements this proposed change is 
available on ticket 3043 http://trac.osgeo.org/gdal/ticket/3043

Please take a look and send us your comments.

Regards,

Ivan
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] ogr, vrt - connecting to arcsde errors only with srcsql

[Even Rouault]

Le lundi 03 octobre 2011 19:17:25, Duarte Carreira a écrit :
> Hi there.
> 
> I'm having trouble using a SrcSQL tag in a vrt that connects to an ArcSDE
> feature class.
> 
> If I use SrcLayer everything works.
> 
> But if I remove SrcLayer, and add a SrcSQL with the simplest of queries
> (SELECT * FROM USER.TABLE1) I get errors in ogrinfo:
> 
> ERROR 1: SQL Expression Parsing Error: syntax error
> ERROR 1: SQL statement failed, or returned no layer result:
> SELECT * FROM GDBMAN.SREGA_PERIMETROS_REGA
> ERROR 1: SQL Expression Parsing Error: syntax error
> ERROR 1: SQL statement failed, or returned no layer result:
> SELECT * FROM GDBMAN.SREGA_PERIMETROS_REGA
> FAILURE:
> Unable to open datasource `sde_Regadio_SQL.vrt' with the following drivers.
> 
> Any idea why this is?

I guess it is because of the . in the table name. You should surround the 
table name by simple quote character.

SELECT * FROM 'USER.TABLE1'

> 
> Thanks,
> Duarte
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] ogr, vrt - connecting to arcsde errors only with srcsql

[Duarte Carreira]

Hi there.

I'm having trouble using a SrcSQL tag in a vrt that connects to an ArcSDE 
feature class.

If I use SrcLayer everything works.

But if I remove SrcLayer, and add a SrcSQL with the simplest of queries (SELECT 
* FROM USER.TABLE1) I get errors in ogrinfo:

ERROR 1: SQL Expression Parsing Error: syntax error
ERROR 1: SQL statement failed, or returned no layer result:
SELECT * FROM GDBMAN.SREGA_PERIMETROS_REGA
ERROR 1: SQL Expression Parsing Error: syntax error
ERROR 1: SQL statement failed, or returned no layer result:
SELECT * FROM GDBMAN.SREGA_PERIMETROS_REGA
FAILURE:
Unable to open datasource `sde_Regadio_SQL.vrt' with the following drivers.

Any idea why this is?

Thanks,
Duarte
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Re: ogr2ogr (1.8.2) error cannot fetch a layer

[Jukka Rahkonen]

hdam99  gmail.com  gmail.com> writes:

> 
> Hi,
> 
> I'm using ogr2ogr to get data from a oracle view (11g2) to a gml (3.1) file. 
> The gml file was created as expected even I got the ERROR1: ORA-04043:
> object blah does not exist errors.
> 
> Why did I get those errors even though they are in the database?
> 
> There are two views that I didn't have any data.  One of them has additional
> error "FAILURE: Couldn't fetch requested layer 'AIRWAY'!  
> 
> What would be the cause and would be there any work around?
hdam99  gmail.com  gmail.com> writes:

> 
> Hi,
> 
> I'm using ogr2ogr to get data from a oracle view (11g2) to a gml (3.1) file. 
> The gml file was created as expected even I got the ERROR1: ORA-04043:
> object blah does not exist errors.
> 
> Why did I get those errors even though they are in the database?
> 
> There are two views that I didn't have any data.  One of them has additional
> error "FAILURE: Couldn't fetch requested layer 'AIRWAY'!  
> 
> What would be the cause and would be there any work around?

ORA-error is coming from the database but I fear with this information not much
more can be said. For having something to start with, could you provide us

- The CREATE VIEW you have used
- What you have in the USER_SDO_GEOM_COLUMNS about the new view
- What does ogrinfo list about the view?
- Is the Oracle user who created the view the same as the ogr2ogr user? If
not,what rights does ogr2ogr user have for the view?


> Thanks - H
> 
> --
> View this message in context:
http://osgeo-org.1803224.n2.nabble.com/ogr2ogr-1-8-2-error-cannot-fetch-a-layer-tp6854910p6854910.html
> Sent from the GDAL - Dev mailing list archive at Nabble.com.
> 




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] ogr2ogr (1.8.2) error cannot fetch a layer

[hda...@gmail.com]

Hi,

I'm using ogr2ogr to get data from a oracle view (11g2) to a gml (3.1) file. 
The gml file was created as expected even I got the ERROR1: ORA-04043:
object blah does not exist errors.

Why did I get those errors even though they are in the database?

There are two views that I didn't have any data.  One of them has additional
error "FAILURE: Couldn't fetch requested layer 'AIRWAY'!  

What would be the cause and would be there any work around?

Thanks - H

--
View this message in context: 
http://osgeo-org.1803224.n2.nabble.com/ogr2ogr-1-8-2-error-cannot-fetch-a-layer-tp6854910p6854910.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Re: Question regarding GDAL .dll files for C#

[Tamas Szekeres]

Hi Zhenyu Lu,

You need not install the plugin dll-s if you don't require the corresponding
functionality.
With regards to the dependent dll-s, you can use the -dev packages from
http://www.gisinternals.com/sdk/ to compile your own versions using the
following steps:

1. Download the -dev package according to your compiler version.
2. Extract the SDK files in a specific directory (like C:\builds)
3. Check out the gdal sources into this directory.
4. Edit makefile and comment out the unwanted dependecies, like:

#GDAL_POSTGIS = YES

5. Open a Visual studio command prompt, and type:

nmake gdal GDAL_DIR=[your gdal directory]


Best regards,

Tamas



2011/10/3 Zhenyu Lu 

> Hi Tamas,
>
> Sorry for sending you email so late at night. I noticed that you're in
> charge of maintaining the GDAL binary and SDK packages. I would say that
> you've done a fatastic job so far for making so many GDAL binary and SDK
> packages. I've downloaded two .msi files which worked great. So I'm thinking
> that your knowledge might offer huge help for me.
> I would like to ask you several questions regarding the binary and SDK
> packages you've released.
> I used to use the .dll files of GDAL 1.5.2 before, which were compiled by
> myself using VS2005. Now I wrote a tool for students to use for their labs.
> I suddenly realized that all the computers of my uniersity have installed
> Windows 7 64bit operating system, and my previous compiled .dll files do not
> work anymore.
> So I downloaded the gdal-18-1400-x64-core.msi and installed it on
> my computer. Compared with the previous .dll files, the newly created .dll
> files has much larger size (maybe much more functions). I think it is mainly
> because I must include some other .dll files (hdf5dll, geos_c, libmysql,
> pdflib, and xerces-c_2_8.dll) to make it work. However, using my previously
> compiled .dll file, I only need to include the gdal1.5.dll itself. So my
> previous implementation has the file size of < 5 MB, but now the
> implementation grows to more than 20 MB which make it hard to distribute.
> Now I would like to ask is it possible for you to create .dll files or the
> installation file to only include the gdal18.dll file without the other
> plugin .dll files? Or is there a way for me to create such .dll files?
> Hopefully I have made my questions straightforward.
> Thank you so much if you could offer me some suggestions.
>
> -- Zhenyu Lu
> SUNY-ESF
>
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev