Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[iomeneandrei]

Hi all,
for what is worth also for me the simplicity of FWTOOLS wins.

Best regards,

andrea

-
Andrea Borruso


email: aborr...@tin.it
website: http://blog.spaziogis.it
my 2.0 life: http://aborruso.spaziogis.it
feed: http://feeds2.feedburner.com/Tanto
38° 7' 48" N, 13° 21' 9" E

-- 
View this message in context: 
http://osgeo-org.1803224.n2.nabble.com/gdal-dev-FWTools-and-GDAL-1-7-0-tp5887121p5891689.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

On 1/5/11 3:41 AM, iomeneandrei wrote:

for what is worth also for me the simplicity of FWTOOLS wins.


FWTools is nice, but I think with OSgeo4win, not really important.

However, for simplicity, a one-click installer for just GDAL/OGR for 
Windows, complete with command line tools and ready for use with the 
python bindings (and others language bindings?) would be great.


I've found it painful to find appropriate Windows executables every time 
I've need to upgrade to the latest.


-Chris


--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/5 Christopher Barker 

> However, for simplicity, a one-click installer for just GDAL/OGR for
> Windows, complete with command line tools and ready for use with the python
> bindings (and others language bindings?) would be great.
>

I've found it painful to find appropriate Windows executables every time
> I've need to upgrade to the latest.
>
>
Supporting multiple vesions (development/stable branches/releases, x32/x64,
multiple MSVC CRT dependencies) is quite a difficult task in a single
installer. With regards to the development version it would also be
reasonable to provide a build quite frequently to be in sync with the latest
changes in trunk. These are the main reasons I've originally set up
http://vbkto.dyndns.org/sdk/ to provide most of the resonable combinations
as daily built packages.

I also wanted to include these files in an installer (or multiple
installers) but at the moment I don't see the real benefit of this over
extracting a single zip package, since these libraries don't require
significant preparation (like regkey entries) during the deployment. Any
further consideration in this topic would be helpful, as I may have missed
something that would also be important by a windows user.


Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Daniel Morissette]

On 11-01-05 12:44 PM, Tamas Szekeres wrote:


I also wanted to include these files in an installer (or multiple
installers) but at the moment I don't see the real benefit of this over
extracting a single zip package, since these libraries don't require
significant preparation (like regkey entries) during the deployment. Any
further consideration in this topic would be helpful, as I may have
missed something that would also be important by a windows user.



Tamas,

I agree with you, but it seems that an [OK] button (even if it doesn't 
do anything) makes Windows users feel so much better.  :)


Daniel
--
Daniel Morissette
http://www.mapgears.com/
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

On 1/5/11 9:44 AM, Tamas Szekeres wrote:


Supporting multiple vesions (development/stable branches/releases,
x32/x64, multiple MSVC CRT dependencies) is quite a difficult task in a
single installer.


yes, a Major pain. I don't know that we need a single installer, but 
there is a lot to support.



These are the main reasons I've originally set
up http://vbkto.dyndns.org/sdk/ to provide most of the resonable
combinations as daily built packages.


Actually, that is a GREAT resource. At the moment, I can't remember why 
it's been difficult for me to use, but a few thoughts:


First -- I use GDAL from Python and with the command line tools, so 
that's my perspective.


1) It would be nice to have binaries for the latest release front and 
center at the main GDAL site -- having to poke around to find Tamas's 
site is not a big deal, but not always obvious.


2) A standard install location would be good. As I've messed with this 
each time, I never know where stuff should go -- maybe installers would 
help with that


3) If there is a standard install location, then "easy_install gdal" (or 
setup.py build, or...) could work for the python bindings.


Another option is to have a binary installer for the python bindings 
that includes gdal and the gdal utilities -- that would be great for 
users like me, but I don't know how common my use case is. In that case, 
you'd want to support a few recent pythons versions, the python.org 
binaries: 2.6, 2.7, 3.1 (maybe 2.5 too).


One of the tricks here is which numpy to support, etc. numpy has been 
pretty good with binary compatibility lately (except for one mistake 
recently that was corrected)


However, I DON'T want gdal to give me  Python -- I use Python for too 
many other things for that.


4) it might be nice if the install location for the utilities got put on 
the user's PATH -- I don't know how hard that is in an installer -- 
Windows really sucks in that regard.



I also wanted to include these files in an installer (or multiple
installers) but at the moment I don't see the real benefit of this over
extracting a single zip package, since these libraries don't require
significant preparation (like regkey entries) during the deployment.


An installer would better enforce a standard install location. You could 
also have a custom DOS box with a menu entry that set up the environment 
for the command line tools (maybe only PATH?), provide an uninstaller, 
and of course, give the Windows users a nice warm and fuzzy feeling.


With regard to Python, an installer could see what Python the user has 
and install the bindings for that version. Not that I have any idea how 
to build that!


Inno Setup is a very nice free installer, by the way.

-Chris



--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/5 Christopher Barker 

>
> 1) It would be nice to have binaries for the latest release front and
> center at the main GDAL site -- having to poke around to find Tamas's site
> is not a big deal, but not always obvious.
>
>
Chris,

With regards to the comment above, while I'm not sure about the objectives
but I don't think the GDAL site would intend to be a hosting provider of
various binary packages, the most reasonable thing is to put the references
pointing the user to the correct locations which has already been done, see
the "Downloads" section at http://www.gdal.org/

2) A standard install location would be good. As I've messed with this each
> time, I never know where stuff should go -- maybe installers would help with
> that
>
>
This doesn't seem to be decisive requirement to me. We may also create a
definite location in the hard drive to host such files (which can be
remembered later). Or some other folks may prefer installing these files
along with their applications or keep such files in separate - project
specific - directories. We may also have a requirement to deploy and run
these applications from portable locations (ie. from CD or a flash drive).
Another issue of an installer may be due to a single product key along with
the setup which would prevent from installing multiple versions side by side
in the same environment.


> 3) If there is a standard install location, then "easy_install gdal" (or
> setup.py build, or...) could work for the python bindings.
>
>
I admit I don't have enough knowledge about the 'magic' tricks related to
python-ish way installing applications. I expect that most 'magic' are
implemented by copying the files at certain locations, setting some
environment variables or regkey entries. However I might also consider
running a custom application with gdal not necessarily be the responsibility
of a GDAL package. You might also want to install python from a separate
installer (either ActivePython, python.org whatever) and run the application
direcly from a command prompt where the environments are set to run the gdal
applications properly. Most of these packages provide the required .bat file
to setup the environment this way.


> Another option is to have a binary installer for the python bindings that
> includes gdal and the gdal utilities -- that would be great for users like
> me, but I don't know how common my use case is. In that case, you'd want to
> support a few recent pythons versions, the python.org binaries: 2.6, 2.7,
> 3.1 (maybe 2.5 too).
>
>
I don't know much about this either. This may however be doable for those
guys who know the Python packaging approach well enough. I don't think eiter
of the packages referred at
http://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries would support this
feature though.



> One of the tricks here is which numpy to support, etc. numpy has been
> pretty good with binary compatibility lately (except for one mistake
> recently that was corrected)
>
>
Not sure how this be related to a GDAL binary distribution, as far as I
remember numpy can be installed to the Python deployment directly.



> However, I DON'T want gdal to give me  Python -- I use Python for too many
> other things for that.
>
>
Yes, adding more runtime environments to a simple GDAL package makes it more
heavy weighted. Would also be reasonable to include the Perl environment or
a .NET framework installer for example to make it more complete. However, in
many cases it's more reasonable to let the application (using the GDAL
binaries) decide how to make a proper installer to run their application
smoothly.



> 4) it might be nice if the install location for the utilities got put on
> the user's PATH -- I don't know how hard that is in an installer -- Windows
> really sucks in that regard.
>
>
I don't think it would be beneficial in most cases. This could easily break
other applications (using the dll-s with the same names) to fail
unexpectedly. I would prefer to keep the applications isolated (setting the
environment variables specifically to the process and not the user/system)
as much as possible.

An installer would better enforce a standard install location. You could
> also have a custom DOS box with a menu entry that set up the environment for
> the command line tools (maybe only PATH?), provide an uninstaller, and of
> course, give the Windows users a nice warm and fuzzy feeling.
>
>
The 'nice warm and fuzzy feeling' is a good objective indeed, setting up an
entry on the start menu instead of a running the batch file directly may
also be an advantage. While I'm not sure starting a DOS prompt would
validate an installer by it's own, I can see this requirement to be valid in
most cases.



> With regard to Python, an installer could see what Python the user has and
> install the bindings for that version. Not that I have any idea how to build
> that!
>
>
Agreed, but I have no idea either.


> Inno Setup is a very nice free installer, by the way.
>

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/5 Daniel Morissette 

> I agree with you, but it seems that an [OK] button (even if it doesn't do
> anything) makes Windows users feel so much better.  :)
>
>
Daniel,

:-)   And sometimes we wonder what a heck is being done behind an OK button
on Windows which takes so long (or lasts forever ;-)

Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

It may well be that GDAL has too many different use cases to even have a 
"standard" install, but...


On 1/5/11 1:37 PM, Tamas Szekeres wrote:

2011/1/5 Christopher Barker 


With regards to the comment above, while I'm not sure about the
objectives but I don't think the GDAL site would intend to be a hosting
provider of various binary packages,


Well, many (most?) open source packages have "official" binaries hosted 
on its site. It's pretty common to go to a project's site and expect to 
find a way to download binaries right then and there.



2) A standard install location would be good. As I've messed with
this each time, I never know where stuff should go -- maybe
installers would help with that

This doesn't seem to be decisive requirement to me.


It's not a strong requirement, but standard defaults do make things 
easier for everyone.



Or some other folks may prefer installing these files
along with their applications or keep such files in separate - project
specific - directories.


Well, users should certainly be able to do something custom if they 
want. This is all about use-cases -- if you are building a custom app 
linked against GDAL, then you probably want to control where you put things.


However, if you are interested in the command line tools, and using GDAL 
via Python or Perl or ??, then it makes it easier to have a standard 
location.


> Another issue of an installer may be due to a single product key

along with the setup which would prevent from installing multiple
versions side by side in the same environment.


Surely there are ways to accommodate that? though "dll hell" is in the 
lexicon for a reason!



3) If there is a standard install location, then "easy_install gdal"
(or setup.py build, or...) could work for the python bindings.



I admit I don't have enough knowledge about the 'magic' tricks related
to python-ish way installing applications.


That's the thing -- there is no magic here. If you are building a python 
extension, you need to tell the build system where its dependencies lie. 
If you are installing a pre-built python extension, then the 
dependencies need to be in a known place (or maybe on the right PATHS -- 
Windows is pretty ugly this way). Which is why a "standard" install 
location would be a good idea.



I might also consider
running a custom application with gdal not necessarily be the
responsibility of a GDAL package.


well, that depends on whether you consider Python bindings a "custom 
application". In any case, I think it helps third party packages to have 
standard default install locations.


Oh for *nix -- this would be easier if we just could just put stuff in 
/usr/local/...



You might also want to install python
from a separate installer (either ActivePython, python.org
 whatever)


True -- but it is very much a standard for third party packages to 
provide binaries for the python.org python build. Again, I'm not 
suggesting that folks should be prevented (or even discouraged) from 
doing various sorts of custom installs, just that there should be 
defaults, so that it's clear an easy for a newbie to know what to to to 
get things to "just work".



Another option is to have a binary installer for the python bindings
that includes gdal and the gdal utilities -- that would be great for
users like me, but I don't know how common my use case is. In that
case, you'd want to support a few recent pythons versions, the
python.org  binaries: 2.6, 2.7, 3.1 (maybe 2.5 too).

I don't know much about this either. This may however be doable for
those guys who know the Python packaging approach well enough.


It's not that hard (at lest once GDAL is built), but it is work.


I don't
think eiter of the packages referred at
http://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries would support
this feature though.


Agreed -- that's my point!


One of the tricks here is which numpy to support, etc. numpy has
been pretty good with binary compatibility lately (except for one
mistake recently that was corrected)


Not sure how this be related to a GDAL binary distribution, as far as I
remember numpy can be installed to the Python deployment directly.


yes, but the Python bindings are built against a particular numpy. 
That's OK for version so numpy that are binary compatible, but it 
potentially fragile. Note that with the new extended buffer interface, 
it should be possible to build GDAL with full numpy support, but not 
have to compile against numpy. But I think that's only good for 2.7 and 3.*



However, I DON'T want gdal to give me  Python -- I use Python for
too many other things for that.

Yes, adding more runtime environments to a simple GDAL package makes it
more heavy weighted.


right -- I think we're talking about lightweight, GDAL only packages here.


in many cases it's more reasonable to let the
application (using the GDAL binaries) decide 

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Frank Warmerdam]

Folks,

Wow, what a lot of discussion.

From my perspective, I'm pleased with Tamas' provided binaries as a source
for developers who want a GDAL SDK for a particular compiler version and
choice of win32/win64.

I like Jurgen's idea of building a stock GDAL binaries package with an
installer (similar to FWTools) from OSGeo4W packages.

Like some others, I'm also confused by the "correct" way of providing
python binaries.  I'm inclined to provide an included python in a self
installer similar to FWTools, but I'd be pleased if the hardcore
Pythonistas can get the magic ways of producing python binaries for any
Python version working.

Is there someone that wants to take on producing official GDAL binaries
from OSGeo4W with a self installer sort of similar to FWTools, likely
using the mechanism Jurgen described?  I'm interested, but interest hasn't
turned into action over quite a few months of interest.

Best regards,
--
---+--
I set the clouds in motion - turn up   | Frank Warmerdam, warmer...@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush| Geospatial Programmer for Rent

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

Specifically regarding Python programmers who primarily or exclusively use
Windows:

 

The majority of popular Python packages are delivered to Windows users by
one of two mechanisms:

 

1.A stand-alone installation program (either a .exe or .msi file) that
the user just downloads and runs, often built with the Python "distutils"
technology.

2.The easy_install mechanism, in which the user starts a shell and types
"easy_install x".

 

Most Windows users are not familiar with the UNIX way of doing things, in
which the package is compiled and installed from a script. Most Windows
users do not even have a C compiler on their machine.

 

As evidence for the popularity of #1, just look at the installation
statistics for popular packages such as scipy or numpy. For example, the
Python 2.6 .exe installer for scipy 0.8.0 currently has 28417 downloads,
while the source code (.tar.gz file) currently has 14403 downloads. As UNIX
users will be downloading the source code and not the Windows executable,
the 14403 is likely to be a lot of UNIX users and few Windows users. As you
can see, the vast majority of Windows scipy users prefer to install it via
the GUI installation program rather than the source code.

 

Right now, to install the latest GDAL for Python on Windows, the user has to
download a zip file from http://vbkto.dyndns.org/sdk/, drill in to find the
Python files, copy them to C:\PythonXX\Lib\site-packages, drill into find
the GDAL binaries and related files, copy them to a directory such as
C:\GDAL, set the PATH and GDAL environment variables, and so on. And there
are no obvious instructions anywhere about how to do this. Python
programmers are programmers after all, so they can generally figure that out
and accomplish it. But it is not what they are used to doing if they are a
Windows programmer. This is why Chris basically says that he has to remember
/ relearn how to do it every time he upgrades GDAL.

 

It would be really helpful to Windows Python programmers who want to use
GDAL-probably a large number of potential GDAL users-for the GDAL team to
offer installation via one or both of the mechanisms I mentioned above.

 

Best,

Jason

 

From: gdal-dev-boun...@lists.osgeo.org
[mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Tamas Szekeres
Sent: Wednesday, January 05, 2011 4:37 PM
To: Christopher Barker
Cc: gdal-dev@lists.osgeo.org
Subject: Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

 

 

2011/1/5 Christopher Barker 


1) It would be nice to have binaries for the latest release front and center
at the main GDAL site -- having to poke around to find Tamas's site is not a
big deal, but not always obvious.


Chris,

With regards to the comment above, while I'm not sure about the objectives
but I don't think the GDAL site would intend to be a hosting provider of
various binary packages, the most reasonable thing is to put the references
pointing the user to the correct locations which has already been done, see
the "Downloads" section at http://www.gdal.org/

2) A standard install location would be good. As I've messed with this each
time, I never know where stuff should go -- maybe installers would help with
that


This doesn't seem to be decisive requirement to me. We may also create a
definite location in the hard drive to host such files (which can be
remembered later). Or some other folks may prefer installing these files
along with their applications or keep such files in separate - project
specific - directories. We may also have a requirement to deploy and run
these applications from portable locations (ie. from CD or a flash drive).
Another issue of an installer may be due to a single product key along with
the setup which would prevent from installing multiple versions side by side
in the same environment.
 

3) If there is a standard install location, then "easy_install gdal" (or
setup.py build, or...) could work for the python bindings.


I admit I don't have enough knowledge about the 'magic' tricks related to
python-ish way installing applications. I expect that most 'magic' are
implemented by copying the files at certain locations, setting some
environment variables or regkey entries. However I might also consider
running a custom application with gdal not necessarily be the responsibility
of a GDAL package. You might also want to install python from a separate
installer (either ActivePython, python.org whatever) and run the application
direcly from a command prompt where the environments are set to run the gdal
applications properly. Most of these packages provide the required .bat file
to setup the environment this way.
 

Another option is to have a binary installer for the python bindings that
includes gdal and the gdal utilities -- that would be great for users like
me, but I don't know how common my use case is. In that case, you'd want to
support a few 

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/5 Jason Roberts 

>
> Right now, to install the latest GDAL for Python on Windows, the user has
> to download a zip file from http://vbkto.dyndns.org/sdk/, drill in to find
> the Python files, copy them to C:\PythonXX\Lib\site-packages, drill into
> find the GDAL binaries and related files, copy them to a directory such as
> C:\GDAL, set the PATH and GDAL environment variables, and so on. And there
> are no obvious instructions anywhere about how to do this. Python
> programmers are programmers after all, so they can generally figure that out
> and accomplish it. But it is not what they are used to doing if they are a
> Windows programmer. This is why Chris basically says that he has to remember
> / relearn how to do it every time he upgrades GDAL.
>
>
>
> It would be really helpful to Windows Python programmers who want to use
> GDAL—probably a large number of potential GDAL users—for the GDAL team to
> offer installation via one or both of the mechanisms I mentioned above.
>
>

Jason,

Maybe this is due to my lack of knowledge in Python, but why is it a
requirement to place the files to such specific locations to run a python
app with gdal on Windows? As far as I remember setting up the following
environment variables properly (in a command prompt) does the right thing
from arbitrary locations (at least when running the autotest suite for
GDAL):

PROJ_LIB
GDAL_DRIVER_PATH
GDAL_DATA
PYTHONPATH
PATH

Certainly one should find out the location of the python.exe which might
also be included in PATH for convenience.

Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

Chris,

Good points below, but having the compiled gdal binaries (and the binaries
of the dependent libraries) in hand, which is the right way to install those
files on Windows? (Assuming we don't provide python.exe and the related
files in the package)? I mean which install actions should be done in
detail?

Best regards,

Tamas



2011/1/5 Christopher Barker 

> It may well be that GDAL has too many different use cases to even have a
> "standard" install, but...
>
>
> On 1/5/11 1:37 PM, Tamas Szekeres wrote:
>
>> 2011/1/5 Christopher Barker >1) It would be nice to have binaries for the latest release front
>>and center at the main GDAL site -- having to poke around to find
>>Tamas's site is not a big deal, but not always obvious.
>>
>
>  With regards to the comment above, while I'm not sure about the
>> objectives but I don't think the GDAL site would intend to be a hosting
>> provider of various binary packages,
>>
>
> Well, many (most?) open source packages have "official" binaries hosted on
> its site. It's pretty common to go to a project's site and expect to find a
> way to download binaries right then and there.
>
>
> 2) A standard install location would be good. As I've messed with
>>this each time, I never know where stuff should go -- maybe
>>installers would help with that
>>
>> This doesn't seem to be decisive requirement to me.
>>
>
> It's not a strong requirement, but standard defaults do make things easier
> for everyone.
>
>
>  Or some other folks may prefer installing these files
>> along with their applications or keep such files in separate - project
>> specific - directories.
>>
>
> Well, users should certainly be able to do something custom if they want.
> This is all about use-cases -- if you are building a custom app linked
> against GDAL, then you probably want to control where you put things.
>
> However, if you are interested in the command line tools, and using GDAL
> via Python or Perl or ??, then it makes it easier to have a standard
> location.
>
>
> > Another issue of an installer may be due to a single product key
>
>> along with the setup which would prevent from installing multiple
>> versions side by side in the same environment.
>>
>
> Surely there are ways to accommodate that? though "dll hell" is in the
> lexicon for a reason!
>
>
> 3) If there is a standard install location, then "easy_install gdal"
>>(or setup.py build, or...) could work for the python bindings.
>>
>
>  I admit I don't have enough knowledge about the 'magic' tricks related
>> to python-ish way installing applications.
>>
>
> That's the thing -- there is no magic here. If you are building a python
> extension, you need to tell the build system where its dependencies lie. If
> you are installing a pre-built python extension, then the dependencies need
> to be in a known place (or maybe on the right PATHS -- Windows is pretty
> ugly this way). Which is why a "standard" install location would be a good
> idea.
>
>
>  I might also consider
>> running a custom application with gdal not necessarily be the
>> responsibility of a GDAL package.
>>
>
> well, that depends on whether you consider Python bindings a "custom
> application". In any case, I think it helps third party packages to have
> standard default install locations.
>
> Oh for *nix -- this would be easier if we just could just put stuff in
> /usr/local/...
>
>  You might also want to install python
>> from a separate installer (either ActivePython, python.org
>>  whatever)
>>
>
> True -- but it is very much a standard for third party packages to provide
> binaries for the python.org python build. Again, I'm not suggesting that
> folks should be prevented (or even discouraged) from doing various sorts of
> custom installs, just that there should be defaults, so that it's clear an
> easy for a newbie to know what to to to get things to "just work".
>
> Another option is to have a binary installer for the python bindings
>>that includes gdal and the gdal utilities -- that would be great for
>>users like me, but I don't know how common my use case is. In that
>>case, you'd want to support a few recent pythons versions, the
>>python.org  binaries: 2.6, 2.7, 3.1 (maybe 2.5
>> too).
>>
>>
>> I don't know much about this either. This may however be doable for
>> those guys who know the Python packaging approach well enough.
>>
>
> It's not that hard (at lest once GDAL is built), but it is work.
>
>
>  I don't
>> think eiter of the packages referred at
>> http://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries would support
>> this feature though.
>>
>
> Agreed -- that's my point!
>
>
> One of the tricks here is which numpy to support, etc. numpy has
>>been pretty good with binary compatibility lately (except for one
>>mistake recently that was corrected)
>>
>>
>> Not sure how this be related to a GDAL binary distribution, as far as I
>> remember numpy c

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

Tamas,

 

I have a decent amount of experience programming with Python (6 years,
develop my own extension modules, etc) and I have never seen a package that
recommends you install it to a directory of your choice and then modify the
PYTHONPATH environment variable. The standard installation location is the
"site-packages" directory. See http://www.python.org/dev/peps/pep-0250/ for
some background on that.

 

The GDAL team certainly could publish instructions that said "copy the
release-1500-dev\gdal\swig\python\*.py and the osgeo subdirectory directory
to a location of your choice and modify set the PYTHONPATH environment
variable to include that location" but it would be very unusual. GDAL would
likely be the only package that the user encounters with those instructions.
The PYTHONPATH environment variable would likely not even be defined
already. (Although if they have ArcGIS, it would be, because ArcGIS includes
some Python extensions that live in the ArcGIS installation dir.)

 

So, in sum, you could make PYTHONPATH your official solution but it would be
weird. The more traditional route would be to modify the GDAL build scripts
to invoke the existing setup.py with the arguments "bdist --formats=wininst"
to build a Windows installation package (see
http://docs.python.org/distutils/builtdist.html). Ideally, it would also
include a post-install script that could install the GDAL binaries somewhere
(such as a subdir inside \PythonXX\Lib\site-packages\gdal) and set PATH,
PROJ_LIB, GDAL_DRIVER_PATH, and GDAL_DATA to the correct things. I could
potentially help you develop that. It looks like most of what is needed is
already there.

 

Jason

 

From: Tamas Szekeres [mailto:szeker...@gmail.com] 
Sent: Wednesday, January 05, 2011 6:12 PM
To: Jason Roberts
Cc: gdal-dev@lists.osgeo.org; Christopher Barker
Subject: Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

 

 

2011/1/5 Jason Roberts 

 

Right now, to install the latest GDAL for Python on Windows, the user has to
download a zip file from http://vbkto.dyndns.org/sdk/, drill in to find the
Python files, copy them to C:\PythonXX\Lib\site-packages, drill into find
the GDAL binaries and related files, copy them to a directory such as
C:\GDAL, set the PATH and GDAL environment variables, and so on. And there
are no obvious instructions anywhere about how to do this. Python
programmers are programmers after all, so they can generally figure that out
and accomplish it. But it is not what they are used to doing if they are a
Windows programmer. This is why Chris basically says that he has to remember
/ relearn how to do it every time he upgrades GDAL.

 

It would be really helpful to Windows Python programmers who want to use
GDAL-probably a large number of potential GDAL users-for the GDAL team to
offer installation via one or both of the mechanisms I mentioned above.



Jason,

Maybe this is due to my lack of knowledge in Python, but why is it a
requirement to place the files to such specific locations to run a python
app with gdal on Windows? As far as I remember setting up the following
environment variables properly (in a command prompt) does the right thing
from arbitrary locations (at least when running the autotest suite for
GDAL):

PROJ_LIB
GDAL_DRIVER_PATH
GDAL_DATA
PYTHONPATH
PATH

Certainly one should find out the location of the python.exe which might
also be included in PATH for convenience.

Best regards,

Tamas

 

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

On 1/5/11 2:32 PM, Frank Warmerdam wrote:

 From my perspective, I'm pleased with Tamas' provided binaries as a source
for developers who want a GDAL SDK for a particular compiler version and
choice of win32/win64.


yes, those are great!


Like some others, I'm also confused by the "correct" way of providing
python binaries. I'm inclined to provide an included python in a self
installer similar to FWTools,


This is really nice for folks that aren't using Python for other things, 
but if GDAL is only part of your Python tools stack, not so great.



but I'd be pleased if the hardcore
Pythonistas can get the magic ways of producing python binaries for any
Python version working.


I think it's possible, though I'm not sure about "any"! maybe "the most 
common"



Is there someone that wants to take on producing official GDAL binaries
from OSGeo4W


I'm confused about what "from OSGeo4w" means, and I don't know that I 
can "take it on", but I've spent enough time on this thread already, I 
should help out!


On 1/5/11 2:43 PM, Jason Roberts wrote:

The majority of popular Python packages are delivered to Windows users
by one of two mechanisms:

1.A stand-alone installation program (either a .exe or .msi file) that
the user just downloads and runs, often built with the Python
“distutils” technology.

2.The easy_install mechanism, in which the user starts a shell and types
“easy_install x”.


absolutely true. It's also not that hard to do one of those, once you've 
got the other. Once you've got the .msi building, building a binary egg 
is pretty easy, and then you just have to make sure pypi has the right 
links in place so that people get it with easy_install.


In short, we have one problem to solve, not two.


Right now, to install the latest GDAL for Python on Windows, the user
has to download a zip file from http://vbkto.dyndns.org/sdk/, drill in
to find the Python files, copy them to C:\PythonXX\Lib\site-packages,
drill into find the GDAL binaries and related files, copy them to a
directory such as C:\GDAL, set the PATH and GDAL environment variables,
and so on. And there are no obvious instructions anywhere about how to
do this. Python programmers are programmers after all, so they can
generally figure that out and accomplish it. But it is not what they are
used to doing if they are a Windows programmer. This is why Chris
basically says that he has to remember / relearn how to do it every time
he upgrades GDAL.


Good summary -- yes, that's pretty much it.

Which is why I'm saying that if we had a "standard" way that GDAL 
binaries got installed, it would be a lot easier. Then it would be:


1) install GDAL binaries
2) either:
  a) download the source for the python bindings and "setup.py build"
or
  b) run "easy_install gdal" (which would download the source and build 
the binary)



It would be really helpful to Windows Python programmers who want to use
GDAL—probably a large number of potential GDAL users—for the GDAL team
to offer installation via one or both of the mechanisms I mentioned above.


yup.

On 1/5/11 3:12 PM, Tamas Szekeres wrote:

Maybe this is due to my lack of knowledge in Python, but why is it a
requirement to place the files to such specific locations to run a
python app with gdal on Windows?


well, it's not the only way to do it, but it is the most common way.


As far as I remember setting up the
following environment variables properly (in a command prompt) does the
right thing from arbitrary locations (at least when running the autotest
suite for GDAL):

PROJ_LIB
GDAL_DRIVER_PATH
GDAL_DATA
PYTHONPATH
PATH


well -- first of all, setting environment variables on Windows is a 
PITA, second of all, is setting all that up easier than what Jason 
described?, and third, PYTHONPATH is a problem, particularly if you have 
more than one version of python installed.


So I think Jason's way is a better option, but it really would be nice 
to do an installer.


so what to do?

This is all complicated because GDAL is many things:
 - a lib that can be used to build your own C/C++, etc. apps
 - a set of command line utilities
 - a python package
 - a package for various other languages
 - a set up utilities written in Python, also.

Being a Pythonista, I'll focus on the Python bits

Question 1:

  Do we want:
a) a binary package for the python bindings that is completely 
stand-alone -- i.e. has its own copy of everything GDAL that it needs, 
and does not share the dlls with anythign else.


b) (a) but sharing the dlls with other things that may need gdal? (other 
language bindings, mapserver, what have you)


c) (a) or (b) plus the python command line tools

d) (a), (b), or (c) plus the compiled command line tools.


As I write this, I realize that maybe using or following OSGeo4w is the 
way to go -- put the gdal binaries wherever OSGeo4w puts them, and 
building the python bindings against that. Essentially letting OSGeo4w 
establish the standard.


To Tamas:

I wonder if there is

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jürgen E . Fischer]

Hi Frank,

On Wed, 05. Jan 2011 at 17:32:42 -0500, Frank Warmerdam wrote:
> Is there someone that wants to take on producing official GDAL binaries
> from OSGeo4W with a self installer sort of similar to FWTools, likely
> using the mechanism Jurgen described?  I'm interested, but interest hasn't
> turned into action over quite a few months of interest.

A starting point could be [1].  It's result is [2].


Jürgen 

[1] http://qgis.org/~jef/creatensis.tar.bz2
[2] http://qgis.org/~jef/GDAL-OSGeo4W-1.7.3-Setup.exe

-- 
Jürgen E. Fischer norBIT GmbH   Tel. +49-4931-918175-20
Dipl.-Inf. (FH)   Rheinstraße 13Fax. +49-4931-918175-50
Software Engineer D-26506 Norden   http://www.norbit.de

-- 
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Ari Jolma]

On 01/06/2011 01:18 AM, Tamas Szekeres wrote:

Chris,

Good points below, but having the compiled gdal binaries (and the 
binaries of the dependent libraries) in hand, which is the right way 
to install those files on Windows? (Assuming we don't provide 
python.exe and the related files in the package)? I mean which install 
actions should be done in detail?





2011/1/5 Christopher Barker:

"""
Windows binaries built in MinGW are available at:

http://map.hut.fi/files/Geoinformatica/win32/

The Geoinformatica-yy-mm-dd.zip contains GDAL (usually a
development version), Perl-GDAL, Perl, and many other things.
"""

good for MinGW users, I suppose -- I remember them not working for
me, tough I can't recall how or why not. They also suffer from
perhaps trying to be too much (though if it all worked, I wouldn't
care, I have a fast network and large hard drive)



I found it impossible to deliver a Windows binary solution, which would 
depend on the three main packages (Perl, GTK+, and GDAL) to be retrieved 
as binaries from some standard location - that's why I bundle them all 
together into a quite large (32MB) package.


When I started there was only ActivePerl for Windows that was usable, 
but it turned out impossible for me to use because they use MS 
compilers. Now there is Strawberry Perl, which I might try again (or pay 
someone to try). It is good because it uses MinGW.


There are binary installers for GTK+ for Windows, which install them 
into a shared location - but there are problems still and some pieces 
are missing that I need (Glade for example). The official GTK+ site has 
a long list of zips containing binaries. Despite the shared nature of 
GTK+, many projects include own binaries of them in Windows due to 
version mismatches etc.


GDAL is available but again typically as MS compiler builds - which 
should not be a problem in theory because the bindings use it through 
the C API. I've tried to use those a couple of times without luck 
(compiling the bindings in MinGW was the problem). Maybe I should try 
again using binaries from Tamas' site.


I agree that there could be a one main site for GDAL Windows binaries 
(something like http://www.gtk.org/download-windows.html). Tamas' site 
looks good but I'd like to have dev packages also (the SDK packages 
there look old) - just the header files should be enough.


Then there could also be a more end-user type Windows installation 
package project for GDAL, which installs the stuff into some standard 
location (c:\Program Files\share?). For GTK+ that's provided by some 
folks at http://sourceforge.net/projects/gtk-win/


It would be neat to be able to just put to the Geoinformatica Windows 
installer subinstallations, which go and get Perl, GTK+ and GDAL for the 
user if she does not yet have them or has too old ones. In fact 
Geoinformatica should be simply installable through Perl's cpan 
application - but that does not even work in Linux yet :( (the reason is 
that the GDAL Perl module installation script is not intelligent enough 
and the version of the module in CPAN is horribly old and I really suck 
at managing that)


Best regards,

Ari

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/6 Ari Jolma 

>
> GDAL is available but again typically as MS compiler builds - which should
> not be a problem in theory because the bindings use it through the C API.
> I've tried to use those a couple of times without luck (compiling the
> bindings in MinGW was the problem). Maybe I should try again using binaries
> from Tamas' site.
>
> I agree that there could be a one main site for GDAL Windows binaries
> (something like http://www.gtk.org/download-windows.html). Tamas' site
> looks good but I'd like to have dev packages also (the SDK packages there
> look old) - just the header files should be enough.
>
>
Ari,

The SDK packages from http://vbkto.dyndns.org/sdk/ are exactly the same
which have been used to compile the daily builds, so it should be up to
date. The only thing may have to be done is to download the required version
of the gdal sources in the root folder, because not all of the versions
included in the package in order to keep the size as small as possible.

BTW: What is the desired practice to install the gdal files + bindings along
with a pre-installed perl runtime on Windows? Something like we have been
discussing for python in this thread, do we have some desired install
locations, environment settings or packaging conventions?


Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jan Hartmann]

Three months ago I put an announcement on the MapServer-Users list about 
a shell script I wrote to compile the whole MapServer-GDAL-PostGIS-PLR 
suite, with and without PHP/Python, in a separate user directory using 
only two commands. Separate installations can coexist in separate 
directory trees and all parameters of all programs can be adapted for 
eacht installation. Apache, PHP, PostGIS and Python are installed for 
each installation separately. The binaries can be transported to 
computers with the same system libraries. Since then,I tried it out on 
dozens of Virtual Machines in a Cloud virtual environment on Suse, 
Fedora and (prefered) Debian virtual machines, and it works for me 
without glitches. It's something like a low-level replacement for 
FWTools (actually, I wrote it when I heard from Frank that FWTools 
wasn't supported at the moment). It's Linux-only, although porting it to 
Windows should be possible with a msys/cygwin shell, even for native 
compiles.


Since then I got no (zero) replies, so I don't know whether it doesn't 
work, isn't interesting, or people just didn't look at it. If it's worth 
your time and answers some of the questions from this discussion, please 
have a look and let me know what you think of it. And that includes 
"doesn't work, not interesting, not adaptable enough, downright stupid". 
Here is the MS-Users posting:


http://lists.osgeo.org/pipermail/mapserver-users/2010-October/066898.html

Cheers,

Jan

On 01/05/11 23:32, Frank Warmerdam wrote:

Folks,

Wow, what a lot of discussion.

From my perspective, I'm pleased with Tamas' provided binaries as a 
source

for developers who want a GDAL SDK for a particular compiler version and
choice of win32/win64.

I like Jurgen's idea of building a stock GDAL binaries package with an
installer (similar to FWTools) from OSGeo4W packages.

Like some others, I'm also confused by the "correct" way of providing
python binaries.  I'm inclined to provide an included python in a self
installer similar to FWTools, but I'd be pleased if the hardcore
Pythonistas can get the magic ways of producing python binaries for any
Python version working.

Is there someone that wants to take on producing official GDAL binaries
from OSGeo4W with a self installer sort of similar to FWTools, likely
using the mechanism Jurgen described?  I'm interested, but interest 
hasn't

turned into action over quite a few months of interest.

Best regards,
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

Jason,

I appreciate the expertise for all of you along with this thread, I could
already gather quite some useful information from here for this reason. I
must mention that my programming practice in Python can be considered as
zero, this is the main reason that my issues may have trivial solutions for
the hardcore pythonists but not trivial to me. Apologies for this
inconvenience :-)

Getting back to the original topic, you mention that the gdal binaries
should be installed somewhere an set PATH, GDAL_DATA, PROJ_LIB and
GDAL_DRIVER_PATH as a systemwide setting. This is where the problems of mine
are starting. Modifying the PATH globally is a bad practice in 99% of the
cases. The only case I'm aware of which may not be a problem when we make
sure that only one version of such files (dll-s and executables) will ever
be installed to a particular system. But this is not the case with the gdal
binaries as I would expect at least a development or a stable version (and
their x86/x64 variants) to coexist which should be used by the same user.
The same problem may arise when we would like to install multiple versions
to the site packages directory, how the versions of the files are maintained
by the python runtime? In this regard I could mention something like what
have been done with the .NET framework with the multiple versions of the
packages installed simultaneously in the global assembly cache. The
individual packages may specify the required minimum version of the referred
packages loaded by the .NET runtime. Is this something that can be done with
the python environment as well?

(As opposed to this, the dumb solution of having a starting script to open a
command prompt (and setting PYTHONPATH properly) would ensure multiple
versions to be used at the same time, since those settings are applyed to
the cmd process solely.)

Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/6 Christopher Barker 

>
> On 1/5/11 3:12 PM, Tamas Szekeres wrote:
>
>> As far as I remember setting up the
>> following environment variables properly (in a command prompt) does the
>> right thing from arbitrary locations (at least when running the autotest
>> suite for GDAL):
>>
>> PROJ_LIB
>> GDAL_DRIVER_PATH
>> GDAL_DATA
>> PYTHONPATH
>> PATH
>>
>
> well -- first of all, setting environment variables on Windows is a PITA,
> second of all, is setting all that up easier than what Jason described?, and
> third, PYTHONPATH is a problem, particularly if you have more than one
> version of python installed.
>
>


Chris,

I did mention to set the environment for the cmd process solely, and not by
a systemwide setting. In this regard it doesn't matter how many python
runtime is installed on a particular system, the only issue is to refer to
the desired one (by setting the PATH to the python executable or using
absolute path when running python) from the command prompt.




> As I write this, I realize that maybe using or following OSGeo4w is the way
> to go -- put the gdal binaries wherever OSGeo4w puts them, and building the
> python bindings against that. Essentially letting OSGeo4w establish the
> standard.
>
>
The main issue I'm encountering with OSGeo4w is the problem supporting
multiple versions side by side which may trigger some problem for the
packages, for example:

1. If we would provide both x32 and x64 binaries how to ensure that each dll
or executable would load the corresponding version (by setting the PATH,
GDAL_DIVER_PATH in a different way whatever).
2. Most packages are from different sources so when compiling my package (ie
mapserver) I should rely on the dependent packages (like gdal) to compile
mapserver and then submit my package to the OSGeo4w repository. It wouldn't
be a best pactice to use my version of gdal at compile time, and let the
user rely on another version at run time. When upgrading gdal we should make
sure to upgrade the compilation of mapserver or provide having the older
version of gdal side by side which confuses the user.
3. Similar issue may exist with the multiple CRT dependencies privided by
the packages, for example when I would package my vesion of mapserver
compiled with MSVC2008 but we have only gdal-dev packages compiled with
MSVC2003 out there.


I wonder if there is some confusion about how the python bindings are used.
> You've referred to "setting up an application", making me think that you're
> talking about how to install an application, written in Python, that uses
> GDAL. IN that case I think you've got it right.
>
> However, what I'm thinking of is python developers, casual scripters. In
> this case you want to install python-gdal, and be able to fire up the
> interpreter and type "from osgeo import gdal" and away you go. For that use
> case, Jason has described the install process quite well.
>
>
Hmmm.. only for that reason fireing up the interpreter from a command prompt
(where PYTHONPATH has already been set) would probably work as well. But
indeed we may speak about different things, you may probably want to install
a single version of the gdal packages throughout the system, while I'd be
curious about the desired approach to host multiple versions side by side.


Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Ari Jolma]

On 01/06/2011 01:38 PM, Tamas Szekeres wrote:


2011/1/6 Ari Jolma mailto:ari.jo...@gmail.com>>


GDAL is available but again typically as MS compiler builds -
which should not be a problem in theory because the bindings use
it through the C API. I've tried to use those a couple of times
without luck (compiling the bindings in MinGW was the problem).
Maybe I should try again using binaries from Tamas' site.

I agree that there could be a one main site for GDAL Windows
binaries (something like
http://www.gtk.org/download-windows.html). Tamas' site looks good
but I'd like to have dev packages also (the SDK packages there
look old) - just the header files should be enough.


Ari,

The SDK packages from http://vbkto.dyndns.org/sdk/ are exactly the 
same which have been used to compile the daily builds, so it should be 
up to date. The only thing may have to be done is to download the 
required version of the gdal sources in the root folder, because not 
all of the versions included in the package in order to keep the size 
as small as possible.


By the age I meant that the SDK packages are old releases (from 1310 to 
1600 and not trunk for example - do I understand the release names 
correctly?)




BTW: What is the desired practice to install the gdal files + bindings 
along with a pre-installed perl runtime on Windows? Something like we 
have been discussing for python in this thread, do we have some 
desired install locations, environment settings or packaging conventions?


CPAN has only sources, thus cpan application which is the standard to 
download and install perl modules expects you to have a compiler.


ActivePerl (in fact ActiveState, the company) maintains a repository of 
perl modules in binary versions, from where they can be simply 
downloaded and installed with another program. ActivePerl has tools for 
developing those binary packages. That's very similar to what Python 
has. I think ActiveState maintains its repository by itself - so if I 
just make the CPAN module intelligent enough it may end up there 
eventually. I think my Geo::Shapelib module was/is there.


I think it would not make sense to include GDAL into such a binary perl 
module package. Thus GDAL would need to be separately installable - the 
module installer could probably be made to offer install it for the user 
if it existed somewhere.


In short /me thinks the requirements are: 1) /me develop the perl 
bindings configure & make procedure better 2) we make GDAL-dev.msi 
available at an URL.


Ari




Best regards,

Tamas



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/6 Ari Jolma 

>
> By the age I meant that the SDK packages are old releases (from 1310 to
> 1600 and not trunk for example - do I understand the release names
> correctly?)
>
>
>
Ari,

Those numbers are MSVC compiler numbers (according to
http://trac.osgeo.org/gdal/browser/trunk/gdal/nmake.opt) not gdal version
numbers.

CPAN has only sources, thus cpan application which is the standard to
> download and install perl modules expects you to have a compiler.
>
> ActivePerl (in fact ActiveState, the company) maintains a repository of
> perl modules in binary versions, from where they can be simply downloaded
> and installed with another program. ActivePerl has tools for developing
> those binary packages. That's very similar to what Python has. I think
> ActiveState maintains its repository by itself - so if I just make the CPAN
> module intelligent enough it may end up there eventually. I think my
> Geo::Shapelib module was/is there.
>
> I think it would not make sense to include GDAL into such a binary perl
> module package. Thus GDAL would need to be separately installable - the
> module installer could probably be made to offer install it for the user if
> it existed somewhere.
>
>
This is also the case with python and the other bindings: the gdal dll-s and
dependencies should also be installed somewhere. Assuming we install the
gdal-perl modules in a standard location (not sure where it is), do we have
a common mechanism in the perl runtime to find the dependent dlls without
having to violate system wide settings (like the PATH environment variable)?

Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Ari Jolma]

On 01/06/2011 03:25 PM, Tamas Szekeres wrote:


Assuming we install the gdal-perl modules in a standard location (not 
sure where it is), do we have a common mechanism in the perl runtime 
to find the dependent dlls without having to violate system wide 
settings (like the PATH environment variable)?


:) I'm not completely satisfied (I'm also not sure I really understand 
the whole f* issue) with how Perl searches for dll's - especially dll 
import libs - in Windows.


But that's configure/compile time issue. In compile time a Perl GDAL dll 
(for the bindings) is created and put into some location, which is known 
to Perl and Perl knows how to find it in runtime.


Assuming I've built the binaries, then in runtime (when the GDAL module 
is called for) I think somebody (because the Perl GDAL module dll has a 
link to GDAL dll that was put there in compile time) asks for the GDAL 
dll's and they simply need to be available. Thus, if there is GDAL in 
the system (for example in c:\Program Files\Share\GDAL\bin) then that 
path needs to be in the PATH - I don't think there is a way around that. 
The PATH may be a temporary PATH set by somebody for Perl programs, but 
I think the policy should be that when you run GDAL.msi, the system 
default PATH is modified to have the path to the GDAL binaries.


Best regards,

Ari

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

Ari

The only wokaround which is satisfactory to me is to set the required
enviroment at run time before the gdal bindings are used. In csharp we can
do something like:

string path =
Environment.GetEnvironmentVariable("PATH");
if (!path.Contains(gdal_bin_path))
{
Environment.SetEnvironmentVariable("PATH",
gdal_bin_path + ";" + path);
}
Gdal.SetConfigOption("GDAL_DRIVER_PATH",
gdal_driver_path);

But it requires that loading the gdal dll is happening when the interface is
used actually (in Gdal.SetConfigOption with the code above). This is the
case with the .NET runtime,  but I'm not sure how do we stand with the other
bindings in this regard.

Futher problem with this solution is that this startup code should be placed
in the user's script and not sure how this could be implemented in the gdal
bindings itself.

Best regards,

Tamas




2011/1/6 Ari Jolma 

> On 01/06/2011 03:25 PM, Tamas Szekeres wrote:
>
>>
>> Assuming we install the gdal-perl modules in a standard location (not sure
>> where it is), do we have a common mechanism in the perl runtime to find the
>> dependent dlls without having to violate system wide settings (like the PATH
>> environment variable)?
>>
>
> :) I'm not completely satisfied (I'm also not sure I really understand the
> whole f* issue) with how Perl searches for dll's - especially dll import
> libs - in Windows.
>
> But that's configure/compile time issue. In compile time a Perl GDAL dll
> (for the bindings) is created and put into some location, which is known to
> Perl and Perl knows how to find it in runtime.
>
> Assuming I've built the binaries, then in runtime (when the GDAL module is
> called for) I think somebody (because the Perl GDAL module dll has a link to
> GDAL dll that was put there in compile time) asks for the GDAL dll's and
> they simply need to be available. Thus, if there is GDAL in the system (for
> example in c:\Program Files\Share\GDAL\bin) then that path needs to be in
> the PATH - I don't think there is a way around that. The PATH may be a
> temporary PATH set by somebody for Perl programs, but I think the policy
> should be that when you run GDAL.msi, the system default PATH is modified to
> have the path to the GDAL binaries.
>
> Best regards,
>
> Ari
>
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

e it install the GDAL DLLs
as a subdirectory of the osgeo directory, e.g.
C:\PythonXY\Lib\site-packages\osgeo\bin. Modify gdal.py to set
os.environ['PATH'] = os.environ['PATH'] + ';' gdalInstallDir to modify the
PATH to include that directory prior to importing _gdal.pyd. The PATH will
be modified for the running process only, for the duration of that process.

 

3.Same as #2 but rather than modifying gdal.py to set the PATH variable,
instead create a new Python extension module called _gdal_dll_helper.pyd.
The job of this C extension module is simply to get gdal.dll and other DLLs
loaded without resorting to modifying the system PATH which can sometimes
have weird consequences (I can explain more if needed). The extension module
would call the Windows SetDllDirectory
<http://msdn.microsoft.com/en-us/library/ms686203%28v=vs.85%29.aspx>
function, call LoadLibrary to explicitly load gdal17.dll into the current
process, then call SetDllDirectory again to set the DLL directory back to
what it was previously. Then, when gdal.py wants to load _gdal.pyd,
gdal17.dll is already loaded and the binding succeeds.

 

4.Statically link all necessary code to _gdal.pyd, _gdal_array.pyd, etc.
This would eliminate the need to store the GDAL DLLs on the machine at all,
as all code would be in the bindings .pyd DLLs. This would likely be a big
job, and probably not even possible given the variety of libraries that GDAL
leverages.

 

I know #2 and #3 sound scary but they can be done cleanly. I currently use a
variation of #3 in my own project that embeds GDAL and its Python bindings.

 

You also asked:

 

The individual packages may specify the required minimum version of the
referred packages loaded by the .NET runtime. Is this something that can be
done with the python environment as well?

 

As far as I know, there is not really a clean, standardized way to do that.
There are some innovations related to the "easy_install" thing I mentioned
earlier, but nothing has become standard in Python itself, at least with the
2.x releases.

 

Jason

 

 

From: Tamas Szekeres [mailto:szeker...@gmail.com] 
Sent: Thursday, January 06, 2011 7:24 AM
To: Jason Roberts
Cc: gdal-dev@lists.osgeo.org; Christopher Barker
Subject: Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

 

Jason,

I appreciate the expertise for all of you along with this thread, I could
already gather quite some useful information from here for this reason. I
must mention that my programming practice in Python can be considered as
zero, this is the main reason that my issues may have trivial solutions for
the hardcore pythonists but not trivial to me. Apologies for this
inconvenience :-)

Getting back to the original topic, you mention that the gdal binaries
should be installed somewhere an set PATH, GDAL_DATA, PROJ_LIB and
GDAL_DRIVER_PATH as a systemwide setting. This is where the problems of mine
are starting. Modifying the PATH globally is a bad practice in 99% of the
cases. The only case I'm aware of which may not be a problem when we make
sure that only one version of such files (dll-s and executables) will ever
be installed to a particular system. But this is not the case with the gdal
binaries as I would expect at least a development or a stable version (and
their x86/x64 variants) to coexist which should be used by the same user.
The same problem may arise when we would like to install multiple versions
to the site packages directory, how the versions of the files are maintained
by the python runtime? In this regard I could mention something like what
have been done with the .NET framework with the multiple versions of the
packages installed simultaneously in the global assembly cache. The
individual packages may specify the required minimum version of the referred
packages loaded by the .NET runtime. Is this something that can be done with
the python environment as well?

(As opposed to this, the dumb solution of having a starting script to open a
command prompt (and setting PYTHONPATH properly) would ensure multiple
versions to be used at the same time, since those settings are applyed to
the cmd process solely.)

Best regards,

Tamas



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/6 Jason Roberts 

> 2.Build an installation package as above. Have it install the GDAL
> DLLs as a subdirectory of the osgeo directory, e.g.
> C:\PythonXY\Lib\site-packages\osgeo\bin. Modify gdal.py to set
> os.environ['PATH'] = os.environ['PATH'] + ';' gdalInstallDir to modify the
> PATH to include that directory prior to importing _gdal.pyd. The PATH will
> be modified for the running process only, for the duration of that process.
>
>
>
> 3.Same as #2 but rather than modifying gdal.py to set the PATH
> variable, instead create a new Python extension module called
> _gdal_dll_helper.pyd. The job of this C extension module is simply to get
> gdal.dll and other DLLs loaded without resorting to modifying the system
> PATH which can sometimes have weird consequences (I can explain more if
> needed). The extension module would call the Windows 
> SetDllDirectoryfunction,
>  call LoadLibrary to explicitly load gdal17.dll into the current
> process, then call SetDllDirectory again to set the DLL directory back to
> what it was previously. Then, when gdal.py wants to load _gdal.pyd,
> gdal17.dll is already loaded and the binding succeeds.
> I know #2 and #3 sound scary but they can be done cleanly. I currently use
> a variation of #3 in my own project that embeds GDAL and its Python
> bindings.
>
>
>
>
Jason,

Good writeup, thanks for that. You mention #2 and #3 which is exactly the
same I have in my mind. Already posted a code for #2 related to the csharp
bindings to demonstrate this option.

#2 would be fairly transparent for the user (no additional modules
imported), the only thing we could add is how gdal.py would find out the
install location of the gdal files. This may probably set in the registry
added by the installer. Not sure how this addition would affect the gdal
binding on other platforms (if we implement this in gdal)

#3 seems to be better for gdal (doesn't require to modify the bindings)
however the user will require to use an additional module (which is not
required normally when using gdal.py)


I would personally vote on having an extension (like #3) which is imported
by gdal.py in case if it is installed. If this extension is not installed,
gdal.py would work as before. This extension would scan the registry to find
out the install location of the corresponding files (probably based on an
unique key) and perform the required actions to make the dll-s loadable.


Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

Tamas,

 

I think with either option #2 or #3, gdal.py will have to be modified in
order for everything to be invisible to the user. In #2, gdal.py would have
to modify the PATH variable. In #3, it would have to import the
_gdal_dll_helper module. In either case, it would just be a few lines of
code at the top of the file that occurs before any other GDAL extension
modules are loaded (_gdal, _gdal_array, etc). It would also need to be done
for other modules such as ogr.py, osr.py, or whatever also imports GDAL
extension modules (unless those ones also import gdal.py first).

 

So #3 is not better by virtue of not having to modify the bindings; it does
have to modify the bindings. But #3 is appealing because setting the PATH
from Python code sometimes has weird issues. For example, when I tried it in
my code, it produced a weird problem when I tried to use the "nose" Python
testing framework to run test cases that involved calling Python modules
from GDAL and ArcGIS. The problem there may ultimately be a Microsoft issue
in which the environment settings for a process are maintained by the
Microsoft C runtime library (msvcrt*.dll) and Python, ArcGIS, and GDAL may
use different versions of that library. I do not want to digress into this
here, but suffice to say, I prefer #3 to #2 because it did not have this
problem.

 

Here is an outline of tasks. We probably should put it in an RFC to keep
track.

 

1.Modify the makefiles for your SDK so that it runs
release--dev\gdal\swig\python\setup.py with the "bdist
--formats=wininst" option. This will produce an installation program such as
gdal-1.7.3.win32-py2.5.exe. This is what the user will run to install the
Python bindings together with a private copy of the GDAL DLLs used just by
those bindings. On Python 2.6 and later, we probably want "bdist
--formats=msi" to produce a .msi file rather than a .exe, because Windows
likes .msi files much better than .exes for security purposes. IIRC, Python
2.5 and earlier do not have the msi option.

 

2.Modify release--dev\gdal\swig\python\setup.py to include the GDAL
DLLs and data files in the data_files list that is passed to the setup()
function. Make sure it is only done for Windows (Python code can check
that). The goal is to have the installation program create the following
kind of installation:

 

Existing Python bindings:

 

C:\PythonXX\Lib\site-packages\osgeo\gdal.py

C:\PythonXX\Lib\site-packages\osgeo\ogr.py

C:\PythonXX\Lib\site-packages\osgeo\...

C:\PythonXX\Lib\site-packages\osgeo\_gdal.pyd

C:\PythonXX\Lib\site-packages\osgeo\_ogr.pyd

C:\PythonXX\Lib\site-packages\osgeo\...

 

New bin, data, and plugins directories, which are private to this
installation of the bindings:

 

C:\PythonXX\Lib\site-packages\osgeo\bin\gdal17.dll

C:\PythonXX\Lib\site-packages\osgeo\bin\proj.dll

C:\PythonXX\Lib\site-packages\osgeo\bin\...

 

C:\PythonXX\Lib\site-packages\osgeo\bin\gdal-data\coordinate_axis.csv

C:\PythonXX\Lib\site-packages\osgeo\bin\gdal-data\...

 

C:\PythonXX\Lib\site-packages\osgeo\bin\gdal-plugins\ogr_PG.dll

C:\PythonXX\Lib\site-packages\osgeo\bin\gdal-plugins\...

 

3.Introduce a new file
release--dev\gdal\swig\python\extensions\_gdal_dll_loader.cpp. This file
will call GetDllDirectory to find the current DLL directory setting, call
SetDllDirectory to C:\PythonXX\Lib\site-packages\osgeo\bin (or wherever the
Python instance is installed), call LoadLibrary("gdal17.dll"), and call
SetDllDirectory back to what it was before. There are more details to
consider. For example, we might want to have it call LoadLibrary first to
see if it can load the GDAL DLL from the PATH, allowing the user to use
their own GDAL DLLs without having to overwrite the ones in the Python
directory. Or we might not want to do that, with the idea that the GDAL
Python bindings must be tightly coupled to a particular version of GDAL's
DLLs.

 

4.Modify setup.py to compile _gdal_dll_loader.cpp, only for Windows.
This will result in _gdal_dll_loader.pyd alongside the other .pyd files.

 

5.Modify gdal.py, osr.py, and so on to do something like this:

 

import sys

if sys.platform == 'win32': # TODO: add check for Windows x64 as well

try:

import osgeo._gdal_dll_loader  # As part of this, Python
will call an "init" function inside. That function will do the
SetDllDirectory, etc.

except:

pass

 

6.Do whatever is necessary to ensure the GDAL_DATA etc are properly set
up inside GDAL. I can't remember if this will "just work" using directories
named gdal-data, gdal-plugins, or if it is necessary for the
_gdal_dll_loader.pyd to call some GDAL functions to make it happen.

 

Does that sound like the right direction?

 

Jason

 

From: Tamas Szekeres [mailto:szeker...@gmail.com] 
Sent: Thursday, January 06, 2011 11:07 AM
To: Jason Roberts
Cc: gdal-dev@lists.osgeo.org
Subject: 

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

Lots of stuff in this thread, I'll try to contribute what I can inline:

On 1/6/11 2:15 AM, Ari Jolma wrote:

When I started there was only ActivePerl for Windows that was usable,
but it turned out impossible for me to use because they use MS
compilers. Now there is Strawberry Perl, which I might try again (or pay
someone to try). It is good because it uses MinGW.


The situation is a bit different for Python -- the Python.org binaries 
are almost universal (and I think the ActiveState ones are compatible). 
So it's the obvious choice to build against. They are built with MS 
compilers, but you can compile extensions with MinGW (at least for 2.6 
and above). However, ideally the MS compilers are users -- there are 
freely available versions that work (not Open source though!)



Then there could also be a more end-user type Windows installation
package project for GDAL, which installs the stuff into some standard
location (c:\Program Files\share?).


I think so too, but it's a bit unclear to me what's best as far as the 
various language bindings go...given all the versions of Python (and I 
assume others) it's a bit messy.



 In fact
Geoinformatica should be simply installable through Perl's cpan
application - but that does not even work in Linux yet :( (the reason is
that the GDAL Perl module installation script is not intelligent enough
and the version of the module in CPAN is horribly old and I really suck
at managing that)


Wow! given how much folks rave about CPAN, it sounds like PyPi is better 
in this regard, at least for binary packages.


On 1/6/11 4:00 AM, Jan Hartmann wrote:

Three months ago I put an announcement on the MapServer-Users list about
a shell script I wrote to compile the whole MapServer-GDAL-PostGIS-PLR
suite, with and without PHP/Python,


Did you provide Python, to use an existing python.org build?


Since then I got no (zero) replies, so I don't know whether it doesn't
work, isn't interesting, or people just didn't look at it.


well, for what it's worth, on LInux, users can go a long way with distro 
packages and the plain old ./configure; make.. dance.


Binaries on Windows are in much higher demand.

On 1/6/11 4:24 AM, Tamas Szekeres wrote:

Modifying the PATH globally is a bad practice in 99%
of the cases.


really? for command line tools, I do it all the time! It seems the ay to 
go if you want it to just work". Maybe it makes for more dll hell?



I would expect at least a development or a
stable version (and their x86/x64 variants) to coexist which should be
used by the same user.


This is mostly an issue for developers, and you are smart enough to take 
care of yourselves... As for me, I usually have a system wide default 
for things like GDAL on the PATH, and then will have a startup script or 
something that re-sets things if I want to run test versions, or I just 
call the exe I want explicitly


Anyway, putting a binary in a "standard" location is separate from 
having an installer modify the users PATH -- that second step should be 
optional, or just leave it for the user to do manually if they want.



The same problem may arise when we would like to
install multiple versions to the site packages directory, how the
versions of the files are maintained by the python runtime?


Python is very weak in that regard, There are three ways to handl eit now:

1)a roll-your-open version selection mechanism -- wxPython and I think 
PyGTK do this.


2) virtualenv, or similar -- they set up a "virtual" python install, so 
that you can have multiple installs, each with different versions of 
packages


3) setuptools provides some mechanism for egg version selection, though 
i've never really figure out how to use it.


I think (2) is the best recommendation -- the nice thing about it is 
that the packager doesn't have to do anything special -- it's handled by 
how the user installs a package -- if they use setup.py or easy_install, 
or pip, it just works -- maybe not with an msi, though.


So this points to building binary eggs, ideally.


 The individual packages may
specify the required minimum version of the referred packages loaded by
the .NET runtime. Is this something that can be done with the python
environment as well?


That's kind of what setuptools provides, but it's pretty kludgy.


(As opposed to this, the dumb solution of having a starting script to
open a command prompt (and setting PYTHONPATH properly) would ensure
multiple versions to be used at the same time, since those settings are
applied to the cmd process solely.)


I suspect that's how virtualenv works, more or less.

On 1/6/11 5:01 AM, Tamas Szekeres wrote:

I did mention to set the environment for the cmd process solely, and not
by a systemwide setting. In this regard it doesn't matter how many
python runtime is installed on a particular system, the only issue is to
refer to the desired one (by setting the PATH to the python executable
or using absolute path when running python) from the command pr

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Ari Jolma]

Tamas,

In Perl the programmer can postpone loading the external DLL required by 
a module. Thus a similar approach is viable in Perl. But that's not how 
it's usually done. If you have a DLL in a system that is meant for 
shared use then it should be in the system PATH. Thus I believe the 
policy should be to set the PATH if GDAL is installed into a system (as 
such - not as a part of something else). Only if a package installs a 
private GDAL, it should be kept out of PATH and the package executables 
need to take care of setting the PATH or other env vars as appropriate 
(in Geoinformatica I do this - the Geoinformatica installer is basically 
a file copy and no env vars are changed).


Ari

6.1.2011 16:50, Tamas Szekeres kirjoitti:

Ari

The only wokaround which is satisfactory to me is to set the required 
enviroment at run time before the gdal bindings are used. In csharp we 
can do something like:


string path = 
Environment.GetEnvironmentVariable("PATH");

if (!path.Contains(gdal_bin_path))
{
Environment.SetEnvironmentVariable("PATH", 
gdal_bin_path + ";" + path);

}
Gdal.SetConfigOption("GDAL_DRIVER_PATH", 
gdal_driver_path);


But it requires that loading the gdal dll is happening when the 
interface is used actually (in Gdal.SetConfigOption with the code 
above). This is the case with the .NET runtime,  but I'm not sure how 
do we stand with the other bindings in this regard.


Futher problem with this solution is that this startup code should be 
placed in the user's script and not sure how this could be implemented 
in the gdal bindings itself.


Best regards,

Tamas




2011/1/6 Ari Jolma mailto:ari.jo...@gmail.com>>

On 01/06/2011 03:25 PM, Tamas Szekeres wrote:


Assuming we install the gdal-perl modules in a standard
location (not sure where it is), do we have a common mechanism
in the perl runtime to find the dependent dlls without having
to violate system wide settings (like the PATH environment
variable)?


:) I'm not completely satisfied (I'm also not sure I really
understand the whole f* issue) with how Perl searches for dll's -
especially dll import libs - in Windows.

But that's configure/compile time issue. In compile time a Perl
GDAL dll (for the bindings) is created and put into some location,
which is known to Perl and Perl knows how to find it in runtime.

Assuming I've built the binaries, then in runtime (when the GDAL
module is called for) I think somebody (because the Perl GDAL
module dll has a link to GDAL dll that was put there in compile
time) asks for the GDAL dll's and they simply need to be
available. Thus, if there is GDAL in the system (for example in
c:\Program Files\Share\GDAL\bin) then that path needs to be in the
PATH - I don't think there is a way around that. The PATH may be a
temporary PATH set by somebody for Perl programs, but I think the
policy should be that when you run GDAL.msi, the system default
PATH is modified to have the path to the GDAL binaries.

Best regards,

Ari


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org 
http://lists.osgeo.org/mailman/listinfo/gdal-dev



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

On 1/6/11 12:13 PM, Ari Jolma wrote:

 If you have a DLL in a system that is meant for
shared use then it should be in the system PATH. Thus I believe the
policy should be to set the PATH if GDAL is installed into a system


I notice on my system, the dll is "gdal17.dll"

That is, the version is part of the file name, so there shouldn't be a 
problem with different versions installed in the PATH.


WE could even use a longer filename, like *nix systems, we're not 
restricted to 8.3 anymore, are we?


gdal-1.7.1-a.dll

for instance.

Though maybe setting up the build to do that is more trouble that it's 
worth.


-Chris


--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

Chris,

Here are some comments on specific parts of your mail:

> me neither, with Python (or anything Windows, for that matter). Maybe 
> Jason knows better, but I *think* we should be OK with a standard 
> location for GDAL. But could you have:
>
> Program Files\GDAL\1.6\gdal.dll
>   and
> Program Files\GDAL\1.6\gdal.dll
>
> and have one Python binding find 1.6, and one find 1.7 -- I'm not sure 
> how to do that, but it SHOULD be doable! OH what i'd give for a symlink!

Those would be reasonable locations for GDAL to live if the GDAL team
decided to distribute the GDAL binaries using an installer that adhered to
the best practices on Windows. But putting them there would not mean they
would be automatically added to the PATH. The GDAL Python bindings would
still have to use one of the options I mentioned to load the binaries there
(e.g. modify the PATH).

Incidentally, Windows does support both symbolic and hard links, although it
is not widely known, and I'm not sure I'd recommend their use for this
particular problem.

> I think 2.5 is MSVC2003 (but that may be 2.4, my memory is fading...).

I believe both 2.4 and 2.5 used MSVC2003.

> Great summary -- thanks! And you clearly have a better grasp that I on 
> Windows dll hell.

I used to work for Microsoft, back in the day... :-)

>> Modify gdal.py to set
>> os.environ['PATH'] = os.environ['PATH'] + ';' gdalInstallDir to modify
>> the PATH to include that directory prior to importing _gdal.pyd. The
>> PATH will be modified for the running process only, for the duration of
>> that process.
>
> I didn't think that was required, if the dll is in the same place as the 
> *.pyd, but maybe it is. And it seems it would work fine. Might it mess 
> things up if someone where to do a system call from Python? Also, would 
> that support py2exe?

Unfortunately, it is not sufficient to simply put the GDAL DLLs in the same
directory as the GDAL .pyd files. When the .pyd tries to load the GDAL DLL
through "implicit" linking, Windows checks the locations described in
http://msdn.microsoft.com/en-us/library/7d83bc18%28v=vs.80%29.aspx.

You are probably thinking of the first rule in that list, "the directory
where the executable module for the current process is located", i.e. the
.exe file. In most cases, this is C:\PythonXY\python.exe. C:\PythonXY is
probably not an appropriate location for the GDAL DLLs. Plus it would not
work for other executables that host the Python interpreter, such as
C:\Program Files\ArcGIS\Bin\ArcMap.exe.

Incidentally, the Python interpreter itself is in a DLL that Python installs
in C:\Windows\system32, which is always very high up in the PATH. GDAL could
put its binaries there. But that might be very risky because GDAL ships a
lot of DLLs that other apps might also use. For example, GDAL optionally
includes xerces-c, which MATLAB and ArcGIS also use, among others. Plunking
GDAL's own copy of xerces-c into the PATH will screw up those apps.

> I wonder what is donefor theones here:
>
> http://www.lfd.uci.edu/~gohlke/pythonlibs/

I have been meaning to look at that since you originally sent it. It
probably gives an example of the kinds of stuff I mentioned. I wonder if it
is statically linked. Rgdal is one example where someone has statically
linked GDAL into their own project.

> could all that be done with some ctypes calls, rather than another 
> extension?

Yes I believe you are absolutely right. I always forget about ctypes because
I'm working on something that must be compatible back to Python 2.4 and
ctypes was not included in 2.4.

Jason


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

On 1/6/11 12:31 PM, Jason Roberts wrote:

Here are some comments on specific parts of your mail:

Program Files\GDAL\1.6\gdal.dll
   and
Program Files\GDAL\1.6\gdal.dll



Those would be reasonable locations for GDAL to live if the GDAL team
decided to distribute the GDAL binaries using an installer that adhered to
the best practices on Windows.


I *think* we're heading for a consensus to do that, but not many people 
have been on this thread.



Incidentally, Windows does support both symbolic and hard links, although it
is not widely known, and I'm not sure I'd recommend their use for this
particular problem.


I've heard that, but never seen it done.


You are probably thinking of the first rule in that list, "the directory
where the executable module for the current process is located", i.e. the
.exe file. In most cases, this is C:\PythonXY\python.exe. C:\PythonXY is
probably not an appropriate location for the GDAL DLLs. Plus it would not
work for other executables that host the Python interpreter, such as
C:\Program Files\ArcGIS\Bin\ArcMap.exe.


What about C:\PYTHON26\libs ?

or is that for the main python distro only?


Incidentally, the Python interpreter itself is in a DLL that Python installs
in C:\Windows\system32, which is always very high up in the PATH. GDAL could
put its binaries there. But that might be very risky because GDAL ships a
lot of DLLs that other apps might also use.


I agree -- and Tamas would never go for it! ;-)


I wonder what is donefor theones here:

http://www.lfd.uci.edu/~gohlke/pythonlibs/


I have been meaning to look at that since you originally sent it.


let us now what you find -- I really can't spend any more time on this 
right now!



Rgdal is one example where someone has statically
linked GDAL into their own project.


so we know it can be done -- but with all the third party libs (xerces, 
etc, etc) that could be ugly/impossible.



could all that be done with some ctypes calls, rather than another
extension?


Yes I believe you are absolutely right. I always forget about ctypes because
I'm working on something that must be compatible back to Python 2.4 and
ctypes was not included in 2.4.


I've never actually used it, but it does seem made for this sort of thing.

As for versions, I don't know if GDAL has a policy about what versions 
to support, but I'd think we could dump 2.4 (and maybe even 2.5...), and 
that means only one MS compiler to support.


Thanks for your work on this,

-Chris




--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/6 Christopher Barker 

> I notice on my system, the dll is "gdal17.dll"
>
> That is, the version is part of the file name, so there shouldn't be a
> problem with different versions installed in the PATH.
>
> WE could even use a longer filename, like *nix systems, we're not
> restricted to 8.3 anymore, are we?
>
> gdal-1.7.1-a.dll
>
>
Chris,

This is where a typical dll hell problem is starting. The application is
happy to load a common dll let's say a gdal17.dll, zlib1.dll, libcurl.dll
whatever, but is not the same at it is expected to be (for example it
depends on different CRT libraries). The issue is that you rarely get a
clear error message what the problem is, but your application is failing
with access violations in certain conditions.

To make sure about the issue I did a quick test on my devserver with a
whereis.bat containing the following script, trying to find the location of
some common dlls (used by gdal):

@for %%e in (%PATHEXT%;.dll) do @for %%i in (%1%%e) do @if NOT
"%%~$PATH:i"=="" echo %%~$PATH:i


E:\builds>whereis zlib1
C:\Program Files (x86)\Mono-1.2.6\bin\zlib1.dll

E:\builds>whereis ssleay32
C:\Program Files (x86)\Subversion\ssleay32.dll

Depending on the location of my entry in the PATH may break both
applications or these files may break mine.

Not sure this causes an issue in this particular case but, it makes the
things fragile well enough.

Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Frank Warmerdam]

On 11-01-06 04:42 PM, Christopher Barker wrote:

On 1/6/11 12:31 PM, Jason Roberts wrote:

Here are some comments on specific parts of your mail:

Program Files\GDAL\1.6\gdal.dll
and
Program Files\GDAL\1.6\gdal.dll



Those would be reasonable locations for GDAL to live if the GDAL team
decided to distribute the GDAL binaries using an installer that adhered to
the best practices on Windows.


I *think* we're heading for a consensus to do that, but not many people have
been on this thread.


Jason / Christopher,

I have no objection to using \Program Files\GDAL\.\ as a
standard location for GDAL stuff on windows.  If this is done, the GDAL
data search location should be compiled in for this location on windows,
much as it defaults to /usr/share/gdal (or /usr/local/share/gdal) on
linux.

Note that this means point releases cannot coexist on a system.  I think
that is mostly a good thing, in that upgrading to a new point release
should not change the ABI and should be a transparent change for any
applications.  That is also why we do not encode the point release values
in the DLL name.  That is the GDAL17.DLL from GDAL 1.7.3 should be a drop
in replacement for the GDAL17.DLL from GDAL 1.7.2.   If the DLL name was
more specific then applications would be forced to relink to use the new
version.

On the other hand, we do want major relases (1.7.x vs. 1.8.x) to coexist
and it is decision that applications need to make which they want to use.
So the proposed structure handles that well.

I'm going to stay out of the Python specific aspects.

Best regards,
--
---+--
I set the clouds in motion - turn up   | Frank Warmerdam, warmer...@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush| Geospatial Programmer for Rent

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

On 1/6/11 1:51 PM, Tamas Szekeres wrote:

2011/1/6 Christopher Barker 

yup -- it seems long, descriptive file names would help, but that 
doesn't seem to be the Windows way...


OS_X hard-codes the path to shared libs when linking, which causes its 
own problems, but not these ones.


I think Jason has some good ideas, though.


To make sure about the issue I did a quick test on my devserver with a
whereis.bat containing the following script, trying to find the location
of some common dlls (used by gdal):

@for %%e in (%PATHEXT%;.dll) do @for %%i in (%1%%e) do @if NOT
"%%~$PATH:i"=="" echo %%~$PATH:i


E:\builds>whereis zlib1
C:\Program Files (x86)\Mono-1.2.6\bin\zlib1.dll

E:\builds>whereis ssleay32
C:\Program Files (x86)\Subversion\ssleay32.dll


so your gdal depends on Mono's zlib and Subversion's ssleay? that does 
seem fragile!


Or do you mean that you have these somewhere for gdal, and they are ALSO 
in those other locations?


-Chris



--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

Frank,

Thanks for sharing your opinion. I do have one question that I hope you will
weigh in on. Which of the following two options seems better to you:

1. The GDAL libraries (possibly accompanied with executable programs) are
installed as a separately from the GDAL Python bindings. The libraries are
installed to \Program Files as you describe and the Python bindings are
installed in the standard Python location. A Windows Python programmer
wanting to use GDAL would perform two installs: one for the GDAL libraries,
the other for the bindings.

2. The GDAL Python bindings include a private copy of the GDAL libraries
(with no executable programs). These are installed to a private subdirectory
with the bindings. A Windows Python programmer wanting to use GDAL only
needs to perform one install: the bindings.

#1 is essentially how things are done now, just not following Windows best
practices (not using \Program files) and without any automation to ease the
process (no regularly maintained installer for the Python bindings).

I was proposing #2, basically under the argument that a Windows Python
programmer who needs to use GDAL will rarely ever need to use GDAL for some
other purpose, and therefore not want to install and think about a separate
installation of the GDAL libraries themselves. But if you don't believe
that, or think it is important that the libraries remain distinct from the
bindings in this scenario for some other reason, then we would appreciate
your opinion.

Thanks,
Jason

-Original Message-
From: gdal-dev-boun...@lists.osgeo.org
[mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Frank Warmerdam
Sent: Thursday, January 06, 2011 5:01 PM
To: gdal-dev@lists.osgeo.org
Subject: Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

On 11-01-06 04:42 PM, Christopher Barker wrote:
> On 1/6/11 12:31 PM, Jason Roberts wrote:
>> Here are some comments on specific parts of your mail:
>>> Program Files\GDAL\1.6\gdal.dll
>>> and
>>> Program Files\GDAL\1.6\gdal.dll
>
>> Those would be reasonable locations for GDAL to live if the GDAL team
>> decided to distribute the GDAL binaries using an installer that adhered
to
>> the best practices on Windows.
>
> I *think* we're heading for a consensus to do that, but not many people
have
> been on this thread.

Jason / Christopher,

I have no objection to using \Program Files\GDAL\.\ as a
standard location for GDAL stuff on windows.  If this is done, the GDAL
data search location should be compiled in for this location on windows,
much as it defaults to /usr/share/gdal (or /usr/local/share/gdal) on
linux.

Note that this means point releases cannot coexist on a system.  I think
that is mostly a good thing, in that upgrading to a new point release
should not change the ABI and should be a transparent change for any
applications.  That is also why we do not encode the point release values
in the DLL name.  That is the GDAL17.DLL from GDAL 1.7.3 should be a drop
in replacement for the GDAL17.DLL from GDAL 1.7.2.   If the DLL name was
more specific then applications would be forced to relink to use the new
version.

On the other hand, we do want major relases (1.7.x vs. 1.8.x) to coexist
and it is decision that applications need to make which they want to use.
So the proposed structure handles that well.

I'm going to stay out of the Python specific aspects.

Best regards,
-- 
---+
--
I set the clouds in motion - turn up   | Frank Warmerdam,
warmer...@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush| Geospatial Programmer for Rent

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/6 Christopher Barker 

> yup -- it seems long, descriptive file names would help, but that doesn't
> seem to be the Windows way...
>
>
It would be true, but it's not a common practice of the libraries currently
(assuming we keep the original dll name of each dependency). However we
could probably rename our custom dlls with a random name during the build
process. Pretty odd, but workable.



> so your gdal depends on Mono's zlib and Subversion's ssleay? that does seem
> fragile!
>
>
Or do you mean that you have these somewhere for gdal, and they are ALSO in
> those other locations?
>
>
I must say a common gdal build (either FWTools, OSGeo4w, ms4w whatever) may
use dll-s with the same names as many other libraries a user may install. In
my particular case mono and subversion is just a live example. but we could
find many others that use zlib libpng libcurl whatever (these are widely
used libraries in the world). I don't actually suffer from this problem
because I never rely on the PATH environment to load the GDAL dependencies
and I would evangelize against this practice when trying to find out a
reasonable setup approach here also.

Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Frank Warmerdam]

On 11-01-06 05:10 PM, Jason Roberts wrote:

Frank,

Thanks for sharing your opinion. I do have one question that I hope you will
weigh in on. Which of the following two options seems better to you:

1. The GDAL libraries (possibly accompanied with executable programs) are
installed as a separately from the GDAL Python bindings. The libraries are
installed to \Program Files as you describe and the Python bindings are
installed in the standard Python location. A Windows Python programmer
wanting to use GDAL would perform two installs: one for the GDAL libraries,
the other for the bindings.

2. The GDAL Python bindings include a private copy of the GDAL libraries
(with no executable programs). These are installed to a private subdirectory
with the bindings. A Windows Python programmer wanting to use GDAL only
needs to perform one install: the bindings.

#1 is essentially how things are done now, just not following Windows best
practices (not using \Program files) and without any automation to ease the
process (no regularly maintained installer for the Python bindings).

I was proposing #2, basically under the argument that a Windows Python
programmer who needs to use GDAL will rarely ever need to use GDAL for some
other purpose, and therefore not want to install and think about a separate
installation of the GDAL libraries themselves. But if you don't believe
that, or think it is important that the libraries remain distinct from the
bindings in this scenario for some other reason, then we would appreciate
your opinion.


Jason,

I don't have a strong position on this, but I would note that beyond
real Python programmers wanting to use the bindings, they are also needed
just to run the python commandline utilities (such as rgb2pct.py).  So
there is definately a large body of folks who would benefit from having
working python for the python utilities, and the regular commandline
executables.

Another benefit of even Python using GDAL from a standard location is
things like format plugins could be easily handled in one place.

Actually, the more I think about it, the more I prefer even the Python
bindings to use the GDAL under \Program Files\GDAL.

If the Pythonistas really want to have their own copy of GDAL then
we really don't need to have this conversation.  They can do their
own thing, in their own place and there is no need for meaningful
cooperation with the core project.

Best regards,
--
---+--
I set the clouds in motion - turn up   | Frank Warmerdam, warmer...@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush| Geospatial Programmer for Rent

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/6 Jason Roberts 


>
> So #3 is not better by virtue of not having to modify the bindings; it does
> have to modify the bindings. But #3 is appealing because setting the PATH
> from Python code sometimes has weird issues. For example, when I tried it in
> my code, it produced a weird problem when I tried to use the “nose” Python
> testing framework to run test cases that involved calling Python modules
> from GDAL and ArcGIS. The problem there may ultimately be a Microsoft issue
> in which the environment settings for a process are maintained by the
> Microsoft C runtime library (msvcrt*.dll) and Python, ArcGIS, and GDAL may
> use different versions of that library. I do not want to digress into this
> here, but suffice to say, I prefer #3 to #2 because it did not have this
> problem.
>
>
>

This is quite a common issue the getenv/_putenv CRT functions operate only
on the data structures accessible to the run-time library (CRT) and not on
the environment "segment" created for the process by the operating system.
In this regard the libraries work only on a snapshot of the variables that
have been set during the CRT startup. In this case we should probably find a
function that invokes the
SetEnvironmentVariableAPI
instead.



>
>
> 1.Modify the makefiles for your SDK so that it runs
> release--dev\gdal\swig\python\setup.py with the “bdist
> --formats=wininst” option. This will produce an installation program such as
> gdal-1.7.3.win32-py2.5.exe. This is what the user will run to install the
> Python bindings together with a private copy of the GDAL DLLs used just by
> those bindings. On Python 2.6 and later, we probably want “bdist
> --formats=msi” to produce a .msi file rather than a .exe, because Windows
> likes .msi files much better than .exes for security purposes. IIRC, Python
> 2.5 and earlier do not have the msi option.
>
>
>
This can be done fairly easily. Does this mean we would require each build
for multiple python versions?


> 2.Modify release--dev\gdal\swig\python\setup.py to include the
> GDAL DLLs and data files in the data_files list that is passed to the
> setup() function. Make sure it is only done for Windows (Python code can
> check that). The goal is to have the installation program create the
> following kind of installation:
>
>
>
Hmmm. I'm keen to avoid any local modifications in the compiled files
throughout the build system. Do we have some option to load this infromation
from an external file (which is probably loaded by setup.py if exists)? Or I
must write a custom setup2.py containing this customization.



> 3.Introduce a new file
> release--dev\gdal\swig\python\extensions\_gdal_dll_loader.cpp. This file
> will call GetDllDirectory to find the current DLL directory setting, call
> SetDllDirectory to C:\PythonXX\Lib\site-packages\osgeo\bin (or wherever the
> Python instance is installed), call LoadLibrary("gdal17.dll"), and call
> SetDllDirectory back to what it was before. There are more details to
> consider. For example, we might want to have it call LoadLibrary first to
> see if it can load the GDAL DLL from the PATH, allowing the user to use
> their own GDAL DLLs without having to overwrite the ones in the Python
> directory. Or we might *not* want to do that, with the idea that the GDAL
> Python bindings must be tightly coupled to a particular version of GDAL’s
> DLLs.
>
>
>
That seems to be quite complicated, not sure it that's working for those
libraries using LoadLibrary binding at run time. Wouldn't that be better
getting back to add the location to the PATH of the process in a loader.py
(using SetEnvironmentVariable).


> 5.Modify gdal.py, osr.py, and so on to do something like this:
>
>
>
> import sys
>
> if sys.platform == 'win32': # TODO: add check for Windows x64 as well
>
> try:
>
> import osgeo._gdal_dll_loader  # As part of this, Python
> will call an “init” function inside. That function will do the
> SetDllDirectory, etc.
>
> except:
>
> pass
>
>
Such changes can be done in the SWIG interface files easily.


>
>
> 6.Do whatever is necessary to ensure the GDAL_DATA etc are properly
> set up inside GDAL. I can’t remember if this will “just work” using
> directories named gdal-data, gdal-plugins, or if it is necessary for the
> _gdal_dll_loader.pyd to call some GDAL functions to make it happen.
>
>
>

I guess this could also be set form a python script (probably from
loader.py)


Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Joaquim Luis]

On 06-01-2011 21:42, Christopher Barker wrote:

On 1/6/11 12:31 PM, Jason Roberts wrote:

Here are some comments on specific parts of your mail:

Program Files\GDAL\1.6\gdal.dll
   and
Program Files\GDAL\1.6\gdal.dll



Those would be reasonable locations for GDAL to live if the GDAL team
decided to distribute the GDAL binaries using an installer that 
adhered to

the best practices on Windows.


I *think* we're heading for a consensus to do that, but not many 
people have been on this thread.


Hi,

I have been following this with attention and sometimes I thought I had 
something to say but than it has many python specific issues, which I'm 
not versed at all. My experience with other installers is GMT & Mirone.  
Here I simply put the gdal dll and ALL its dependencies (determined by 
careful Dependency Walker analysis) in the same directory as the other 
exes. Since this must be in the PATH, the gdal package goes along. But 
this solution normally doesn't care about the gdal exes.
Also a word about the "best practice on Windows". I really don't see 
anything not even good in that practice to put them in "Program Files". 
Having directories with blanks in their name give nothing but future 
problems when running command line programs (I have seen that happen 
many times). The GMT & Mirone installs propose as default a C:\programs 
diectory. Let me remember that "Program Files" is not a unique name. For 
example in Portugues Win versions it is called "O Meus Programas" 
(Horror). Remember also that MS used to have "Documents and Settings" 
but now on Win7 it's only "Users".



Incidentally, Windows does support both symbolic and hard links, 
although it

is not widely known, and I'm not sure I'd recommend their use for this
particular problem.


I've heard that, but never seen it done.


It works pretty well, the command is mklink and works much like on *nix 
BUT exists only on Vista and Win7.


Time to time, it has been raised also the hypothesis to put the dlls in 
windowes\system32. PLEASE, PLEASE don't even think on that. It's the 
easiest way of making the worst dll-hell. I learned that after spending 
many hours trying to understand why GDAL was not working in the 
classroom computers. It turned out to be an very old zlib1.dll put there 
by ArcS*



Joaquim


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

Frank,

Thanks for your thoughts. Based on them, I'd recommend the following two
things be created.

1. GDAL windows installation program, or at minimum, a wiki page that says
how to install the GDAL libraries and utilities (executables and Python
scripts) to \Program Files\GDAL\... Perhaps a quick compromise would be for
Tamas's build system to produce a .zip that would have everything in a
suitable directory structure and for wiki page to instruct the user "just
unzip this to \Program Files\GDAL\...".

2. GDAL Python bindings installation program. This could be easily created
using the standard Python distutils stuff I've been mentioning, and would
install the bindings to the normal Python place. The bindings would ideally
be modified to check for and explicitly load libraries from \Program
Files\GDAL\... This would eliminate the need to modify the PATH variable.

A Windows Python programmer would just install those two things and they're
done. No questions about what goes where. No need to set any environment
variables.

People like me who embed GDAL in their own application would now have a new
alternative: rather than including GDAL libraries we could instruct the user
to download an official GDAL build and install it, and then just use it from
there.

What do you think?

Jason

-Original Message-
From: Frank Warmerdam [mailto:warmer...@pobox.com] 
Sent: Thursday, January 06, 2011 5:49 PM
To: Jason Roberts
Cc: gdal-dev@lists.osgeo.org
Subject: Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

On 11-01-06 05:10 PM, Jason Roberts wrote:
> Frank,
>
> Thanks for sharing your opinion. I do have one question that I hope you
will
> weigh in on. Which of the following two options seems better to you:
>
> 1. The GDAL libraries (possibly accompanied with executable programs) are
> installed as a separately from the GDAL Python bindings. The libraries are
> installed to \Program Files as you describe and the Python bindings are
> installed in the standard Python location. A Windows Python programmer
> wanting to use GDAL would perform two installs: one for the GDAL
libraries,
> the other for the bindings.
>
> 2. The GDAL Python bindings include a private copy of the GDAL libraries
> (with no executable programs). These are installed to a private
subdirectory
> with the bindings. A Windows Python programmer wanting to use GDAL only
> needs to perform one install: the bindings.
>
> #1 is essentially how things are done now, just not following Windows best
> practices (not using \Program files) and without any automation to ease
the
> process (no regularly maintained installer for the Python bindings).
>
> I was proposing #2, basically under the argument that a Windows Python
> programmer who needs to use GDAL will rarely ever need to use GDAL for
some
> other purpose, and therefore not want to install and think about a
separate
> installation of the GDAL libraries themselves. But if you don't believe
> that, or think it is important that the libraries remain distinct from the
> bindings in this scenario for some other reason, then we would appreciate
> your opinion.

Jason,

I don't have a strong position on this, but I would note that beyond
real Python programmers wanting to use the bindings, they are also needed
just to run the python commandline utilities (such as rgb2pct.py).  So
there is definately a large body of folks who would benefit from having
working python for the python utilities, and the regular commandline
executables.

Another benefit of even Python using GDAL from a standard location is
things like format plugins could be easily handled in one place.

Actually, the more I think about it, the more I prefer even the Python
bindings to use the GDAL under \Program Files\GDAL.

If the Pythonistas really want to have their own copy of GDAL then
we really don't need to have this conversation.  They can do their
own thing, in their own place and there is no need for meaningful
cooperation with the core project.

Best regards,
-- 
---+
--
I set the clouds in motion - turn up   | Frank Warmerdam,
warmer...@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush| Geospatial Programmer for Rent


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

 

1.Modify the makefiles for your SDK so that it runs
release--dev\gdal\swig\python\setup.py with the "bdist
--formats=wininst" option. This will produce an installation program such as
gdal-1.7.3.win32-py2.5.exe. This is what the user will run to install the
Python bindings together with a private copy of the GDAL DLLs used just by
those bindings. On Python 2.6 and later, we probably want "bdist
--formats=msi" to produce a .msi file rather than a .exe, because Windows
likes .msi files much better than .exes for security purposes. IIRC, Python
2.5 and earlier do not have the msi option.

 

This can be done fairly easily. Does this mean we would require each build
for multiple python versions?

 

Yes, it would mean that. One build for Python 2.5, one for 2.6, one for 2.7,
etc. This is how it is typically done.

 

 

2.Modify release--dev\gdal\swig\python\setup.py to include the GDAL
DLLs and data files in the data_files list that is passed to the setup()
function. Make sure it is only done for Windows (Python code can check
that). The goal is to have the installation program create the following
kind of installation:

 

Hmmm. I'm keen to avoid any local modifications in the compiled files
throughout the build system. Do we have some option to load this infromation
from an external file (which is probably loaded by setup.py if exists)? Or I
must write a custom setup2.py containing this customization.



I was not saying that setup.py be forked and a special copy exist for
windows. I was saying that setup.py should internally check whether it was
being executed on Windows (check Python's sys.platform variable), and then
perform special logic. But this may be moot because it sounded like Frank
was not keen on my suggestion of enclosing a private copy of the GDAL DLLs
with the Python bindings. If we avoid my suggestion then we are left with
the idea of putting them in \Program Files\GDAL. In that case, we would not
need to modify setup.py for this purpose. 


 

3.Introduce a new file
release--dev\gdal\swig\python\extensions\_gdal_dll_loader.cpp. This file
will call GetDllDirectory to find the current DLL directory setting, call
SetDllDirectory to C:\PythonXX\Lib\site-packages\osgeo\bin (or wherever the
Python instance is installed), call LoadLibrary("gdal17.dll"), and call
SetDllDirectory back to what it was before. There are more details to
consider. For example, we might want to have it call LoadLibrary first to
see if it can load the GDAL DLL from the PATH, allowing the user to use
their own GDAL DLLs without having to overwrite the ones in the Python
directory. Or we might not want to do that, with the idea that the GDAL
Python bindings must be tightly coupled to a particular version of GDAL's
DLLs.

 

That seems to be quite complicated, not sure it that's working for those
libraries using LoadLibrary binding at run time. Wouldn't that be better
getting back to add the location to the PATH of the process in a loader.py
(using SetEnvironmentVariable).

I do not think it would be good to tamper with setting the PATH. I am not
certain, but I think Python's environment handler may call _putenv rather
than SetEnvironmentVariable, or something like that, which can cause some
problems. It might be better just to use SetDllDirectory. Also, as Chris
pointed out, the Python ctypes module can be used to call win32 APIs. This
would eliminate the need for a new _gdal_dll_loader module. Instead, the
try/except block that I described below would just call win32 directly via
ctypes. It could do the necessary SetDllDirectory/LoadLibrary/etc itself.

 

 

5.Modify gdal.py, osr.py, and so on to do something like this:

 

import sys

if sys.platform == 'win32': # TODO: add check for Windows x64 as well

try:

import osgeo._gdal_dll_loader  # As part of this, Python
will call an "init" function inside. That function will do the
SetDllDirectory, etc.

except:

pass


Such changes can be done in the SWIG interface files easily.

 

Sounds good to me.


 

 

6.Do whatever is necessary to ensure the GDAL_DATA etc are properly set
up inside GDAL. I can't remember if this will "just work" using directories
named gdal-data, gdal-plugins, or if it is necessary for the
_gdal_dll_loader.pyd to call some GDAL functions to make it happen.

 


I guess this could also be set form a python script (probably from
loader.py)

If we follow the \Program Files\GDAL suggestion, then I further suggest that
GDAL itself be modified to use that location as the default for GDAL_DATA,
GDAL_PLUGINS, etc. That way the user does not have to set ANY environment
variables for GDAL to work. That would be a major plus because, frankly,
Windows users rarely have to set environment variables.





Best regards,

Tamas

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

On 1/6/11 3:03 PM, Joaquim Luis wrote:

Also a word about the "best practice on Windows". I really don't see
anything not even good in that practice to put them in "Program Files".
Having directories with blanks in their name give nothing but future
problems when running command line programs (I have seen that happen
many times).


True, I'm still quite amazed how MS has ignored the command line for 
years. Sure, we geeks are a small fraction of Windows users, but a small 
fraction of a huge userbase is still a LOT.


Perhaps that's why the standard place for Python is NOT in "Program Files"

That being said, "Program Files" does seem to work fine for me most of 
the time.



Time to time, it has been raised also the hypothesis to put the dlls in
windowes\system32. PLEASE, PLEASE don't even think on that.


That, we do have a consensus on!

By the way, isn't there some new stuff about "side by side" installs or 
something -- that's supposed to help dll hell?


On 1/6/11 2:49 PM, Frank Warmerdam wrote:

I don't have a strong position on this, but I would note that beyond
real Python programmers wanting to use the bindings, they are also needed
just to run the python commandline utilities (such as rgb2pct.py). So
there is definately a large body of folks who would benefit from having
working python for the python utilities, and the regular commandline
executables.


yup. Also, I use gdal command line utilities and python bindings. It's 
actually really nice to know that they are using the exact same 
versions, as I can test stuff with the utilities and compare results.



Another benefit of even Python using GDAL from a standard location is
things like format plugins could be easily handled in one place.


Yes, and then Perl, and who knows what else can share the same install.


Actually, the more I think about it, the more I prefer even the Python
bindings to use the GDAL under \Program Files\GDAL.


+1 from me, too.

I winder where Howard is? But anyway, the gdal PyPi site (which I think 
Howard wrote/maintained) says:


"""
Windows

You will need the following items to complete an install of the GDAL 
Python bindings on Windows:


* GDAL Windows Binaries The basic install requires the 
gdalwin32exe###.zip (### is the version number). Other files you see in 
the directory are for various optional plugins and development 
headers/include files. After downloading the zip file, extract it to the 
directory of your choosing.

"""

** the problem here is that that zip file hasn'tbeen maintained for a while.

"""
* GDAL Python Bindings available at the Python Cheeseshop. An 
executable installer is available for both Python 2.4 or 2.5 or as a 
Python egg.

"""
** this is pretty much what Jason is suggeting - a binary egg dependent 
on a certain gdal install.


"""
As explained in the README_EXE.txt file, after unzipping the GDAL 
binaries you will need to modify your system path and variables. If 
you're not sure how to do this, read the Microsoft KnowledgeBase doc


   1. Add the installation directory bin folder to your system PATH, 
remember to put a semicolon in front of it before you add to the 
existing path.


  C:\gdalwin32-1.7\bin

   2. Create a new user or system variable with the data folder from 
your installation.


  Name : GDAL_DATA
  Path : C:\gdalwin32-1.7\data
"""

It would be nice to remove these steps. Again, I think Jason's got some 
good ideas for that.



By the way -- it is fairly common for python packages to include command 
line utilities. There are tools in distutils and setuptools to support 
that -- they get installed in a Scripts directory in the Python install. 
Should the Python-based gdal utilities go there?


-Chris





--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Joaquim Luis]

Perhaps that's why the standard place for Python is NOT in "Program 
Files"


Yes,  there is wisdom.



That being said, "Program Files" does seem to work fine for me most of 
the time.


And for the (1 - "most") times that it didn't work you knew how to fix 
it (enclosing the full path with "") but many people don't know it.


By the way, isn't there some new stuff about "side by side" installs 
or something -- that's supposed to help dll hell?


I think you are referring to the "manifests" but here also MS has 
changed their mind (aleluia). VC2010 doesn't have them anymore which IMO 
is a compelling reason to use this compiler whenever possible.


Joaquim
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Frank Warmerdam]

On 11-01-06 06:43 PM, Jason Roberts wrote:

Frank,

Thanks for your thoughts. Based on them, I'd recommend the following two
things be created.

1. GDAL windows installation program, or at minimum, a wiki page that says
how to install the GDAL libraries and utilities (executables and Python
scripts) to \Program Files\GDAL\... Perhaps a quick compromise would be for
Tamas's build system to produce a .zip that would have everything in a
suitable directory structure and for wiki page to instruct the user "just
unzip this to \Program Files\GDAL\...".

...

What do you think?


Jason,

I'm supportive, but not necessarily willing to take it as action item
for myself.

I would suggest building it as an installer .exe, perhaps using NSIS as
I did for FWTools, or perhaps the method mentioned by Jurgen produces
a nice installer.

One question not discussed is whether GDAL should be minimalist or
maximalist.  That is, do we want to include as many formats as possible
despite the fact that it drags in lots of supporting libraries?  If we
just use whatever decision OSGeo4W uses then we will have a fairly
maximalist solution which is ok I suppose but might make integration of
the resulting GDAL in other complex applications messy.

I would like to suggest production of such an installer be done in such
a way that the generating scripts live in SVN somewhere, and with
instructions for the process in the wiki so it isn't all tied to one
person (as FWTools was).

Man, I'm really tempted to leap into this myself but I have so many
other outstanding items right now.  I am willing to assist in an
effort by a small (2-3 people?) team.

Best regards,
--
---+--
I set the clouds in motion - turn up   | Frank Warmerdam, warmer...@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush| Geospatial Programmer for Rent

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/7 Jason Roberts 

> Thanks for your thoughts. Based on them, I'd recommend the following two
> things be created.
>
> 1. GDAL windows installation program, or at minimum, a wiki page that says
> how to install the GDAL libraries and utilities (executables and Python
> scripts) to \Program Files\GDAL\... Perhaps a quick compromise would be for
> Tamas's build system to produce a .zip that would have everything in a
> suitable directory structure and for wiki page to instruct the user "just
> unzip this to \Program Files\GDAL\...".
>
>
Jason,

What would be the suitable directory structure? I'm keen to provide an
installer which could place the files to the right location. By using
WIXit's could be generated automatically
by the command line tools candle.exe
and light.exe during the build process easily.

However naming the install root folder to GDAL doesn't seem to be a right
thing as we provide other packages like mapserver as well. For the sake of
simplicitly I could imagine to place everything in a common folder (like
SDKBuilds) and add a shortcut for invoking a command prompt (for starting
the commandline tools) and a shortcut to uninstall the package. I would also
mention OSGeo as the root, but I'm not sure how it will violate the files if
a similar installer will ever derived from an OSGeo4W bundle. (We may
probably warn the user not to install both versions side by side)

I may also be mention that by default referring to the programsfolder in the
installation process may select different folders depending on the
architecture (Win32/Win64) or the OS version. I don't think it would be a
good way to force everything to be in  "C:\Program Files" which contains the
x64 binaries on x64 platforms for example or it may reside in various
logical drives on a particular system. It would probably be reasonable to
use something like
SHGetSpecialFolderLocationto
retrieve the current location in the loader program.



> 2. GDAL Python bindings installation program. This could be easily created
> using the standard Python distutils stuff I've been mentioning, and would
> install the bindings to the normal Python place. The bindings would ideally
> be modified to check for and explicitly load libraries from \Program
> Files\GDAL\... This would eliminate the need to modify the PATH variable.
>
>
That's ok with me as a separate installer provided by distutils.


Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

Frank,

> I would suggest building it as an installer .exe, perhaps using NSIS as
> I did for FWTools, or perhaps the method mentioned by Jurgen produces
> a nice installer.

If possible, I would suggest the final output be a .msi (Windows Installer
package) rather than a .exe. In an effort to make the average user more
secure, Microsoft continues to clamp down on the flexibility afforded to
.exe installers, due to the trouble of disambiguating them from malicious
programs. It was a big shock when Windows Vista was introduced and it
prevented .exe installers from truly running with administrative rights even
if the logged on user was an administrator. Everybody knows about that issue
now and has learned how to modify their .exe to request those rights
(causing the user to be prompted for confirmation) but who knows what
Microsoft will do next. For that reason, I believe it would be better to
bite the bullet and adhere to their recommended installation format.

Microsoft's Visual Studio has long had an easy mechanism for building .msi
installers. It is what I would recommend for this, unless you prefer to use
a FOSS alternative, in which case I cannot advise on what would be good.
There are many. Perhaps Jürgen has a better idea.

> One question not discussed is whether GDAL should be minimalist or
> maximalist.  That is, do we want to include as many formats as possible
> despite the fact that it drags in lots of supporting libraries?  If we
> just use whatever decision OSGeo4W uses then we will have a fairly
> maximalist solution which is ok I suppose but might make integration of
> the resulting GDAL in other complex applications messy.

IMO, there are two primary goals of this packaging of GDAL: provide easy
access to just the GDAL command line utilities (separate from larger tool
suites such as FWTools and OSGeo4W), and provide the GDAL libraries in a
well-known location so they can be located by the various GDAL bindings. I
think a secondary goal is to provide GDAL libraries in a well-known location
for integration by arbitrary applications. Because the GDAL binaries will
not normally be in the PATH, only when the user is wanting to use the GDAL
utilities, the mere presence of a large number of supporting libraries will
not be an issue. As you noted, it is when it comes to integration that there
is a problem.

I don't think there's an easy answer for the minimalist/maximalist question.
I think it would be best to err on the maximimalist side, with the idea of
providing as fully-functional command line suite as possible. But if there
are particular libraries that are known to cause problems for popular
integration scenarios but are rarely used by command-line users, then leave
them out. I don't know how to identify such libraries; the GDAL team and
power users are the best source of that info. Over time, you can hopefully
ease any problems by moving things to plugins that are only loaded on
demand, allowing someone to integrate without hitting a problem unless the
plugin is loaded.  In the end, integrators are likely to be skilled
programmers and their fallback position will be to build a private copy of
GDAL with the select dependencies they need.

> I would like to suggest production of such an installer be done in such
> a way that the generating scripts live in SVN somewhere, and with
> instructions for the process in the wiki so it isn't all tied to one
> person (as FWTools was).
>
> Man, I'm really tempted to leap into this myself but I have so many
> other outstanding items right now.  I am willing to assist in an
> effort by a small (2-3 people?) team.

I would be happy offer opinions, review, and test but am not available to
lead at this point. As the developer of the primary source of Windows builds
of GDAL, I think Tamas is the logical leader for this if he is available.

Thank you for seeing this as an important priority.

Jason


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

Tamas,

 

WIX looks like a great technology for building the installation package.
I've never used it but I took a quick look and it seems to provide what is
needed.

 

As I understand it, you are pondering whether it would be better to have
GDAL in Program Files\OSGeo\GDAL or in Program Files\GDAL. Is it simply a
question aesthetics or principle, in which it seems proper to put all OSgeo
projects under Program Files\OSGeo, but there could be problems coordinating
the efforts of multiple projects to adhere to that carefully and not mash
each other's files? If that summarizes the issue, then I'd recommend going
with the more practical, less principled approach: put GDAL in Program
Files\GDAL, and OSGeo4W in Program Files\OSGeo4W. It could get messy if both
installers have to be able to create the Program Files\OSGeo directory but
not necessarily delete it when they are uninstalled, because there could be
another OSGeo project in there.

 

Along those lines, I would suggest that if GDAL plans to support
side-by-side installations of GDAL itself, then we need to contemplate
carefully whether to use Program Files\GDAL\GDAL-X.Y or Program
Files\GDAL-X.Y. I think either one would be ok, so long as whoever writes
the installer thinks out all the side-by-side installation/uninstallation
scenarios.

 

Another question, also already raised, is whether to have just two or all
three version numbers in the installation directory. I think that question
depends on whether you ever need to support bugfix releases installed
side-by-side (e.g. 1.8.0 and 1.8.1), and also whether the bindings and other
integrators will need to bind to specific bugfix releases or not. For
example, if you guarantee that all 1.8.x releases will have compatible
ABIs-that you will never introduce a change that will break the binary
interface without increasing the minor build number-then it would be ok to
just use X.Y in the directory name. But if that cannot be guaranteed, then
you need to support X.Y.Z so that bindings and other integrators can locate
the specific bugfix release that they are compatible with.

 

Following up on that point: I think we've agreed that we do not want the
user to have to set the PATH in order to have the GDAL bindings work.
Therefore the bindings must be changed to look for the GDAL libraries in the
well-known location. Because they currently use implicit linking, they are
tightly coupled to a particular version of the libraries. The user will have
to be aware that they must install a compatible pair of bindings + GDAL. The
GDAL team should decide right now about the X.Y vs. X.Y.Z compatibility
question.

 

Regarding x86 vs. x64. The GDAL installer should follow the standard
Microsoft convention. On an x64 machine, the x64 GDAL utils/libs should go
in \Program Files and the x86 utils/libs should go in \Program Files (x86).
The bindings and other integrators must be aware of x86 vs. x64 and locate
libs from the correct directory.

 

I agree that calling SHGetSpecialFolderLocation with an appropriate CSIDL is
an appropriate way to find the Program Files directory (addressing issue
noted by Joaquim and others that Program Files is localized). It is probably
ok to use the ProgramFiles and ProgramFiles(x86) environment variables if
calling the Win32 API is not easy for particular bindings.

 

Jason

 

From: Tamas Szekeres [mailto:szeker...@gmail.com] 
Sent: Friday, January 07, 2011 5:26 AM
To: Jason Roberts
Cc: Frank Warmerdam; gdal-dev@lists.osgeo.org
Subject: Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

 

 

2011/1/7 Jason Roberts 

Thanks for your thoughts. Based on them, I'd recommend the following two
things be created.

1. GDAL windows installation program, or at minimum, a wiki page that says
how to install the GDAL libraries and utilities (executables and Python
scripts) to \Program Files\GDAL\... Perhaps a quick compromise would be for
Tamas's build system to produce a .zip that would have everything in a
suitable directory structure and for wiki page to instruct the user "just
unzip this to \Program Files\GDAL\...".


Jason,

What would be the suitable directory structure? I'm keen to provide an
installer which could place the files to the right location. By using WIX
<http://wix.sourceforge.net/>  it's could be generated automatically by the
command line tools candle.exe and light.exe during the build process easily.

However naming the install root folder to GDAL doesn't seem to be a right
thing as we provide other packages like mapserver as well. For the sake of
simplicitly I could imagine to place everything in a common folder (like
SDKBuilds) and add a shortcut for invoking a command prompt (for starting
the commandline tools) and a shortcut to uninstall the package. I would also
mention OSGeo as the root, but I'm not sure how it will violate the files if
a similar installer will ever derived from an OSGeo4W bundle. (We may
pr

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Joaquim Luis]

On 07-01-2011 15:07, Jason Roberts wrote:


Tamas,

WIX looks like a great technology for building the installation 
package. I've never used it but I took a quick look and it seems to 
provide what is needed.


As I understand it, you are pondering whether it would be better to 
have GDAL in Program Files\OSGeo\GDAL or in Program Files\GDAL. Is it 
simply a question aesthetics or principle, in which it seems proper to 
put all OSgeo projects under Program Files\OSGeo, but there could be 
problems coordinating the efforts of multiple projects to adhere to 
that carefully and not mash each other's files? If that summarizes the 
issue, then I'd recommend going with the more practical, less 
principled approach: put GDAL in Program Files\GDAL, and OSGeo4W in 
Program Files\OSGeo4W. It could get messy if both installers have to 
be able to create the Program Files\OSGeo directory but not 
necessarily delete it when they are uninstalled, because there could 
be another OSGeo project in there.




Let me just recall that OSGeo4W proposes the default install directory 
as C:\OSGeo4W
This is just my opinion and I feel quite at rest because I'm able to 
build GDAL and install it wherever I wont, but I would really hate that 
an installer would force me to install in a location contrary to my 
will. In particular when that location has blanks in the name (Program 
Files).

But, as I said it's just my private view.


Joaquim
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Frank Warmerdam]

On 11-01-07 10:07 AM, Jason Roberts wrote:

put GDAL in Program Files\GDAL, and
OSGeo4W in Program Files\OSGeo4W.


Jason,

I would note that OSGeo4W installs in C:\OSGeo4W by default, and there are
no plans currently to change this.  OSGeo4W will continue to use it's
internal copy of GDAL.

> It could get messy if both installers have to

be able to create the Program Files\OSGeo directory but not necessarily delete
it when they are uninstalled, because there could be another OSGeo project in
there.


I would prefer to stick to C:\Program Files\GDAL.


Along those lines, I would suggest that if GDAL plans to support side-by-side
installations of GDAL itself, then we need to contemplate carefully whether to
use Program Files\GDAL\GDAL-X.Y or Program Files\GDAL-X.Y. I think either one
would be ok, so long as whoever writes the installer thinks out all the
side-by-side installation/uninstallation scenarios.


I prefer to just use the major.minor version as the subdirectory name.

C:\Program Files\GDAL\1.7\


Another question, also already raised, is whether to have just two or all three
version numbers in the installation directory. I think that question depends on
whether you ever need to support bugfix releases installed side-by-side (e.g.
1.8.0 and 1.8.1), and also whether the bindings and other integrators will need
to bind to specific bugfix releases or not. For example, if you guarantee that
all 1.8.x releases will have compatible ABIs—that you will never introduce a
change that will break the binary interface without increasing the minor build
number—then it would be ok to just use X.Y in the directory name. But if that
cannot be guaranteed, then you need to support X.Y.Z so that bindings and other
integrators can locate the specific bugfix release that they are compatible 
with.


It is an explicit guarantee that point releases do not change the C *or* C++
public ABI.  It is an explicit intention that a point release can be installed
over a previous release without impacting any applications using GDAL.  This
extends to the point that plugins should be compatible across point releases.


Following up on that point: I think we’ve agreed that we do not want the user
to have to set the PATH in order to have the GDAL bindings work.


I do *not* agree that installing GDAL would necessarily insert it into the
global PATH.  It might, or might not, but I'm still very concerned about
possible interference with other applications using GDAL.  I'm not saying
I will never agree to it, but I remain very cautious about interference
in the global space.  Particularly since we are likely to carry along lots
of other DLLs (zlib, xerces, etc).

If this installer is to be a product of the project, I would suggest you
write up a proposal as an RFC and we work from that.

Best regards,
--
---+--
I set the clouds in motion - turn up   | Frank Warmerdam, warmer...@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush| Geospatial Programmer for Rent

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

Joaquim,

 

I hear your pain about having the space in the name. It would probably not
be too much trouble for the installer to allow the user to change the
installation directory. That is a basic option of most installers so it
would probably not be hard to implement with a WIX-built installer.

 

But I should reiterate that a requirement of this package is that the GDAL
bindings (installed as separate packages) must be able to locate the GDAL
libraries. The one approach mentioned so far is to put the binaries in a
well-known location. If the bindings were designed to look for the libraries
in a well-known location, they would be broken if the user installed them to
a different location. Therefore it is important to pick a default location
and recommend that only very knowledgeable users select a different place.
My view is that adhering to Microsoft's best practice of using \Program
Files outweighs the convenience of using a directory off a root partition
that does not have a space in the name. 

 

An alternative idea is to store the location of the GDAL libraries in the
registry and then permitting them to be installed anywhere with no
consequences. The bindings would read the registry to determine their
location. This is also a fairly standard way to do things on Windows, but it
would be slightly more complicated for the bindings.

 

Jason

 

From: Joaquim Luis [mailto:jl...@ualg.pt] 
Sent: Friday, January 07, 2011 10:27 AM
To: Jason Roberts
Cc: 'Tamas Szekeres'; gdal-dev@lists.osgeo.org
Subject: Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

 

On 07-01-2011 15:07, Jason Roberts wrote: 

Tamas,

 

WIX looks like a great technology for building the installation package.
I've never used it but I took a quick look and it seems to provide what is
needed.

 

As I understand it, you are pondering whether it would be better to have
GDAL in Program Files\OSGeo\GDAL or in Program Files\GDAL. Is it simply a
question aesthetics or principle, in which it seems proper to put all OSgeo
projects under Program Files\OSGeo, but there could be problems coordinating
the efforts of multiple projects to adhere to that carefully and not mash
each other's files? If that summarizes the issue, then I'd recommend going
with the more practical, less principled approach: put GDAL in Program
Files\GDAL, and OSGeo4W in Program Files\OSGeo4W. It could get messy if both
installers have to be able to create the Program Files\OSGeo directory but
not necessarily delete it when they are uninstalled, because there could be
another OSGeo project in there.


Let me just recall that OSGeo4W proposes the default install directory as
C:\OSGeo4W
This is just my opinion and I feel quite at rest because I'm able to build
GDAL and install it wherever I wont, but I would really hate that an
installer would force me to install in a location contrary to my will. In
particular when that location has blanks in the name (Program Files).
But, as I said it's just my private view.


Joaquim

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Tamas Szekeres]

2011/1/7 Jason Roberts 

> WIX looks like a great technology for building the installation package.
> I’ve never used it but I took a quick look and it seems to provide what is
> needed.
>
>
>

Jason,

I've already used WIX many times and I'm very satisfied with it.


> As I understand it, you are pondering whether it would be better to have
> GDAL in Program Files\OSGeo\GDAL or in Program Files\GDAL. Is it simply a
> question aesthetics or principle, in which it seems proper to put all OSgeo
> projects under Program Files\OSGeo, but there could be problems coordinating
> the efforts of multiple projects to adhere to that carefully and not mash
> each other’s files? If that summarizes the issue, then I’d recommend going
> with the more practical, less principled approach: put GDAL in Program
> Files\GDAL, and OSGeo4W in Program Files\OSGeo4W. It could get messy if both
> installers have to be able to create the Program Files\OSGeo directory but
> not necessarily delete it when they are uninstalled, because there could be
> another OSGeo project in there.
>
>
>
> Along those lines, I would suggest that if GDAL plans to support
> side-by-side installations of GDAL itself, then we need to contemplate
> carefully whether to use Program Files\GDAL\GDAL-X.Y or Program
> Files\GDAL-X.Y. I think either one would be ok, so long as whoever writes
> the installer thinks out all the side-by-side installation/uninstallation
> scenarios.
>
>
Just by thinking about the details if we implement a smart dll loader which
could in fact work with any locations to load the dll-s why do we require to
use a specific folder as the install location? The installer may create a
registry entry or a config file in a specific place where the necessary
dependencies are installed actually. The  loader would be responsible to
query for a compatible entry and use that location when loading the
dependent dll-s. If no entries found the loader would provide an error that
a dependent setup should also be executed.



>
>
> Another question, also already raised, is whether to have just two or all
> three version numbers in the installation directory. I think that question
> depends on whether you ever need to support bugfix releases installed
> side-by-side (e.g. 1.8.0 and 1.8.1), and also whether the bindings and other
> integrators will need to bind to specific bugfix releases or not. For
> example, if you guarantee that all 1.8.x releases will have compatible
> ABIs—that you will never introduce a change that will break the binary
> interface without increasing the minor build number—then it would be ok to
> just use X.Y in the directory name. But if that cannot be guaranteed, then
> you need to support X.Y.Z so that bindings and other integrators can locate
> the specific bugfix release that they are compatible with.
>
>
>

It would not be a problem in the scope of my comments above. The user could
install multiple versions anywhere and the loader would be responsible to
find out the corresponding version at run time.


> Regarding x86 vs. x64. The GDAL installer should follow the standard
> Microsoft convention. On an x64 machine, the x64 GDAL utils/libs should go
> in \Program Files and the x86 utils/libs should go in \Program Files (x86).
> The bindings and other integrators must be aware of x86 vs. x64 and locate
> libs from the correct directory.
>
>
>
> I agree that calling SHGetSpecialFolderLocation with an appropriate CSIDL
> is an appropriate way to find the Program Files directory (addressing issue
> noted by Joaquim and others that Program Files is localized). It is probably
> ok to use the ProgramFiles and ProgramFiles(x86) environment variables if
> calling the Win32 API is not easy for particular bindings.
>
>
We could probably implement the loader in C an use SWIG to map the functions
to the various languages. The implementation would be the same for all
languages in this regard.
The only issue here is that where to place the loader.dll (providing the
core implementation) which would also rely on the Windows dll search rules.
For python you could easily say to create a pyd installed along with the
python files, but in may not be so trivial for the other languages. I'm
tending to think that this part should probably be implemented in a generic
way (which doesn't change frequently) and be installed in a common folder
available in the PATH.

Best regards,

Tamas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Joaquim Luis]

Jason,

I understand all that (though not necessarily agree). Possibly part of 
my reluctance stands also from the no-understanding of the degree of 
potential evilness attributed to PATH changes. Remember that path 
changes can be done on local profiles only or system wide but most of 
all, path changes are APPENDED to previous value of PATH not PREPEND (at 
least that's my experience with the Inno Setup) so it will not disrupt 
any other previously program that uses dll with same names. In the worst 
case what may happen is that that GDAL install would not work due to 
incompatibilities with previously installed libs. But that is essence of 
the dll-hell for which an universal solution is yet to come.


Joaquim


Joaquim,

I hear your pain about having the space in the name. It would probably 
not be too much trouble for the installer to allow the user to change 
the installation directory. That is a basic option of most installers 
so it would probably not be hard to implement with a WIX-built installer.


But I should reiterate that a requirement of this package is that the 
GDAL bindings (installed as separate packages) must be able to locate 
the GDAL libraries. The one approach mentioned so far is to put the 
binaries in a well-known location. If the bindings were designed to 
look for the libraries in a well-known location, they would be broken 
if the user installed them to a different location. Therefore it is 
important to pick a default location and recommend that only very 
knowledgeable users select a different place. My view is that adhering 
to Microsoft's best practice of using \Program Files outweighs the 
convenience of using a directory off a root partition that does not 
have a space in the name.


An alternative idea is to store the location of the GDAL libraries in 
the registry and then permitting them to be installed anywhere with no 
consequences. The bindings would read the registry to determine their 
location. This is also a fairly standard way to do things on Windows, 
but it would be slightly more complicated for the bindings.


Jason

*From:*Joaquim Luis [mailto:jl...@ualg.pt]
*Sent:* Friday, January 07, 2011 10:27 AM
*To:* Jason Roberts
*Cc:* 'Tamas Szekeres'; gdal-dev@lists.osgeo.org
*Subject:* Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

On 07-01-2011 15:07, Jason Roberts wrote:

Tamas,

WIX looks like a great technology for building the installation 
package. I've never used it but I took a quick look and it seems to 
provide what is needed.


As I understand it, you are pondering whether it would be better to 
have GDAL in Program Files\OSGeo\GDAL or in Program Files\GDAL. Is it 
simply a question aesthetics or principle, in which it seems proper to 
put all OSgeo projects under Program Files\OSGeo, but there could be 
problems coordinating the efforts of multiple projects to adhere to 
that carefully and not mash each other's files? If that summarizes the 
issue, then I'd recommend going with the more practical, less 
principled approach: put GDAL in Program Files\GDAL, and OSGeo4W in 
Program Files\OSGeo4W. It could get messy if both installers have to 
be able to create the Program Files\OSGeo directory but not 
necessarily delete it when they are uninstalled, because there could 
be another OSGeo project in there.



Let me just recall that OSGeo4W proposes the default install directory 
as C:\OSGeo4W
This is just my opinion and I feel quite at rest because I'm able to 
build GDAL and install it wherever I wont, but I would really hate 
that an installer would force me to install in a location contrary to 
my will. In particular when that location has blanks in the name 
(Program Files).

But, as I said it's just my private view.


Joaquim



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

RE: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jason Roberts]

Frank,

Thanks for the clarification of the various issues. I will start an RFC for
this next week.

Jason

-Original Message-
From: Frank Warmerdam [mailto:warmer...@pobox.com] 
Sent: Friday, January 07, 2011 10:29 AM
To: Jason Roberts
Cc: gdal-dev@lists.osgeo.org
Subject: Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

On 11-01-07 10:07 AM, Jason Roberts wrote:
>put GDAL in Program Files\GDAL, and
> OSGeo4W in Program Files\OSGeo4W.

Jason,

I would note that OSGeo4W installs in C:\OSGeo4W by default, and there are
no plans currently to change this.  OSGeo4W will continue to use it's
internal copy of GDAL.

 > It could get messy if both installers have to
> be able to create the Program Files\OSGeo directory but not necessarily
delete
> it when they are uninstalled, because there could be another OSGeo project
in
> there.

I would prefer to stick to C:\Program Files\GDAL.

> Along those lines, I would suggest that if GDAL plans to support
side-by-side
> installations of GDAL itself, then we need to contemplate carefully
whether to
> use Program Files\GDAL\GDAL-X.Y or Program Files\GDAL-X.Y. I think either
one
> would be ok, so long as whoever writes the installer thinks out all the
> side-by-side installation/uninstallation scenarios.

I prefer to just use the major.minor version as the subdirectory name.

C:\Program Files\GDAL\1.7\

> Another question, also already raised, is whether to have just two or all
three
> version numbers in the installation directory. I think that question
depends on
> whether you ever need to support bugfix releases installed side-by-side
(e.g.
> 1.8.0 and 1.8.1), and also whether the bindings and other integrators will
need
> to bind to specific bugfix releases or not. For example, if you guarantee
that
> all 1.8.x releases will have compatible ABIs-that you will never introduce
a
> change that will break the binary interface without increasing the minor
build
> number-then it would be ok to just use X.Y in the directory name. But if
that
> cannot be guaranteed, then you need to support X.Y.Z so that bindings and
other
> integrators can locate the specific bugfix release that they are
compatible with.

It is an explicit guarantee that point releases do not change the C *or* C++
public ABI.  It is an explicit intention that a point release can be
installed
over a previous release without impacting any applications using GDAL.  This
extends to the point that plugins should be compatible across point
releases.

> Following up on that point: I think we've agreed that we do not want the
user
> to have to set the PATH in order to have the GDAL bindings work.

I do *not* agree that installing GDAL would necessarily insert it into the
global PATH.  It might, or might not, but I'm still very concerned about
possible interference with other applications using GDAL.  I'm not saying
I will never agree to it, but I remain very cautious about interference
in the global space.  Particularly since we are likely to carry along lots
of other DLLs (zlib, xerces, etc).

If this installer is to be a product of the project, I would suggest you
write up a proposal as an RFC and we work from that.

Best regards,
-- 
---+
--
I set the clouds in motion - turn up   | Frank Warmerdam,
warmer...@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush| Geospatial Programmer for Rent


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

On 1/6/11 7:55 PM, Frank Warmerdam wrote:


One question not discussed is whether GDAL should be minimalist or
maximalist. That is, do we want to include as many formats as possible
despite the fact that it drags in lots of supporting libraries?


+1 for Maximalist. We want as many things as possible to "just work"


If we
just use whatever decision OSGeo4W uses then we will have a fairly
maximalist solution which is ok I suppose but might make integration of
the resulting GDAL in other complex applications messy.


People that are integrating into complex apps will likely roll their own 
anyway.



I would like to suggest production of such an installer be done in such
a way that the generating scripts live in SVN somewhere, and with
instructions for the process in the wiki so it isn't all tied to one
person (as FWTools was).


Absolutely!

On 1/6/11 8:23 PM, Jürgen E. Fischer wrote:

You get a desktop link
and a start menu entry that both opens a command line window from where you can
use GDAL


very nice.


and start python with gdal available.


what Python gets started? Can you start Python from any other command 
line and have it work, or do you need ENV VARS set up?


On 1/7/11 2:26 AM, Tamas Szekeres wrote:

However naming the install root folder to GDAL doesn't seem to be a
right thing as we provide other packages like mapserver as well.


well, either you can have GDAL in GDAL, and mapserver in MapServer. or...

> I would also mention OSGeo as the root,

If there are a number of things under the OSGeo umbrella provided by the 
same people, then OSGeo is a fine root. Note that I'd have:


Program Files\OSGeo\GDAL
Program Files\OSGeo\Mapserver


rather than dumping everything directly in OSGeo\ -- though maybe a 
shared "bin" does make sense? Oh for some standards!



but I'm not sure how it
will violate the files if a similar installer will ever derived from an
OSGeo4W bundle. (We may probably warn the user not to install both
versions side by side)


Ideally, the OSGEo4W bundle will be compatible -- that really would be 
better --i.e. a GDAL binary is a subset of an OSGeo install. But if 
that's not possible, then we'll just have to use a different name than 
OSGeo4W does.




IMO, there are two primary goals of this packaging of GDAL: provide easy
access to just the GDAL command line utilities (separate from larger tool
suites such as FWTools and OSGeo4W), and provide the GDAL libraries in a
well-known location so they can be located by the various GDAL bindings. I
think a secondary goal is to provide GDAL libraries in a well-known location
for integration by arbitrary applications.


good summary.


> For example, if you guarantee that all 1.8.x releases

will have compatible ABIs—that you will never introduce a change that
will break the binary interface


guarantee is a strong word, but that is the idea as I understand it.


Therefore the bindings must be changed to look for the GDAL libraries in
the well-known location.


(or more than one, but I think one is a good start). It seems on *nix, 
for instance, this would generally happen in the build stage (actually 
configure) -- the builder specifies where they want it installed, and 
everything is built to match that. That's a fine model to follow. I 
don't know if there is any equivalent of ./configure for Windows 
building, but a simple script could at least do this bit easily enough.



The
user will have to be aware that they must install a compatible pair of
bindings + GDAL. The GDAL team should decide right now about the X.Y vs.
X.Y.Z compatibility question.


yes -- and If we go with X.Y, then you should be able to drop an updated 
point release of GDAL in, without updating the bindings. Is that good, 
however, or are the bindings also updated, and tightly related? I think 
you could do X.Y, but have the bindings check for point version 
compatibility, so you'd get a Warning if you were using a mismatched 
binding.



Let me just recall that OSGeo4W proposes the default install directory
as C:\OSGeo4W


not a bad option either, but is that difficult/impossible/ugly with 
recent MS security policies (still on XP here...)? Ugly though it is, 
there is a lot to be said for conforming to MS recommended practice.



Following up on that point: I think we’ve agreed that we do not want
the user
to have to set the PATH in order to have the GDAL bindings work.


I do *not* agree that installing GDAL would necessarily insert it into the
global PATH.


I think we have ways of getting the bindings to work without global PATH.


It might, or might not, but I'm still very concerned about
possible interference with other applications using GDAL.


I'd want global PATH settings for command line tool usage. using it for 
finding dlls seems like a really bad idea. With that in mind, perhaps 
the command line tools should be in a different dir from the dlls, so 
that you can put the former on your global PATH without breaking any 
linking.


Whether

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Jürgen E . Fischer]

Hi Chris,

On Fri, 07. Jan 2011 at 12:03:47 -0800, Christopher Barker wrote:
> On 1/6/11 8:23 PM, Jürgen E. Fischer wrote:
>> You get a desktop link and a start menu entry that both opens a command line
>> window from where you can use GDAL

> very nice.

>> and start python with gdal available.

> what Python gets started? Can you start Python from any other command  
> line and have it work, or do you need ENV VARS set up?

The python shipped with the installer (just like in OSGeo4W).

> On 1/7/11 2:26 AM, Tamas Szekeres wrote:
>> IMO, there are two primary goals of this packaging of GDAL: provide easy
>> access to just the GDAL command line utilities (separate from larger tool
>> suites such as FWTools and OSGeo4W), and provide the GDAL libraries in a
>> well-known location so they can be located by the various GDAL bindings. I
>> think a secondary goal is to provide GDAL libraries in a well-known location
>> for integration by arbitrary applications.

> good summary.

hm, just an idea - creatensis.pl could even be generized to pick an arbitrary
OSGeo4W package and produce a NSIS installer with all the necessary
dependencies (even as CGI - I created the package on Linux).

If the GDAL package in OSGeo4W did provide a desktop and start link and start
menu link the nsi wouldn't need to create those and the extra batch file could
be dropped.  So the current customization wouldn't be needed anymore.

That also would work for meta package (eg. something like gdal-min and
gdal-max).


Jürgen

-- 
Jürgen E. Fischer norBIT GmbH   Tel. +49-4931-918175-20
Dipl.-Inf. (FH)   Rheinstraße 13Fax. +49-4931-918175-50
Software Engineer D-26506 Norden   http://www.norbit.de

-- 
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Chris Barker]

On 1/6/2011 4:25 PM, Joaquim Luis wrote:

By the way, isn't there some new stuff about "side by side" installs
or something -- that's supposed to help dll hell?


I think you are referring to the "manifests" but here also MS has
changed their mind (aleluia). VC2010 doesn't have them anymore which IMO
is a compelling reason to use this compiler whenever possible.


This is what I was talking about:

http://msdn.microsoft.com/en-us/library/aa376307%28VS.85%29.aspx

It does look like it is (or was) MS's solution to dll hell -- however, I 
remember it causing much pain for wxPython (particularly for py2exe), 
and I'm happy to defer to the experts here that seem to know that the 
cure was worse than the disease.


-Chris


--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev