Re: [gdal-dev] Considering drivers removal ?

[Howard Butler]

I am also supportive of RFC approvals before someone goes and invests the time 
to develop a driver. Otherwise it is easy to get in the spot where it's not 
merged because the community didn't want it included. A nice thing the RFC 
process would ensure is a full document of the expectations of the driver. 
Specifically, whether it depends on binary-only SDKs, what features it would 
support, and *who* is on the hook for responding to issues on it. Another 
feature of this RFC should be to clarify the deprecation and removal process 
for drivers that are no longer relevant or no longer have someone providing 
them attention.


> On Jan 28, 2021, at 1:33 PM, Tamas Szekeres  wrote:
> 
> Hi Sean,
> 
> I would also be supportive to formalize the driver submission process.
> 
> Best regards,
> 
> Tamas
> 
> 
> Sean Gillies mailto:s...@mapbox.com>> ezt írta (időpont: 
> 2021. jan. 28., Cs, 17:39):
> Hi Tamas,
> 
> Are you suggesting that a RFC be required for a new driver? I would support 
> this 100%.
> 
> On Wed, Jan 27, 2021 at 2:17 PM Tamas Szekeres  > wrote:
> David,
> 
> Up to this time the driver writers were highly welcomed to author new drivers 
> for the project and these effort didn't require a separate RFC in terms of 
> the Project Management Committee Guidelines 
>  document. Adding new drivers 
> hasn't been considered to be substantial changes or something that causes to 
> change the API or brings in backwards incompatibility issues.
> However in a real deployment situation the drivers may cause issues for the 
> developers, end users and package maintainers from several aspects like so:
> 
> 1. The drivers may depend on external libraries and we don't want to link 
> against those libraries in all cases.
> 2. The external libraries may cause license incompatibilities, ie linking 
> against that may change the license terms of the overall project.
> 3. Not all of the drivers are required in a particular solution (that applies 
> to the built in drivers, too). In a real situation the application may use 
> only a limited set of drivers and the existence of the unwanted drivers may 
> cause some performance degradation.
> 4. Implementing custom compilations (and omitting unwanted drivers from the 
> build) may be problematic for most of the users or projects utilizing gdal as 
> a sub-project.
> 
> In my opinion, the current solution of incubating new drivers is fairly 
> minimalistic, we might probably want to know what kind of format is being 
> addressed, who is the audience, how amount of user might be of interest, how 
> the licensing of the dependent project is looking like, is the dependent 
> project (if any) maintained well enough and can be compiled to each supported 
> platforms etc.
> 
> But replying to the question, I think you shoud continue the effort, but 
> consider to implement a plugin build for that. There are several existing 
> drivers are compiled as plugins at the moment, so it should not be so 
> difficult.
> 
> 
> Best regards,
> 
> Tamas
> 
> 
> David Brochart mailto:david.broch...@gmail.com>> 
> ezt írta (időpont: 2021. jan. 27., Sze, 15:29):
> I am currently trying to add a Zarr driver to GDAL (see 
> https://github.com/OSGeo/gdal/pull/3411 
> ), but from what I can see the trend 
> is to remove drivers, so I'm now wondering it I should pursue this effort. 
> I'm relatively new to GDAL, but from my point of view supporting a lot of 
> formats is part of GDAL's success, so maybe the real focus should be on 
> implementing a plugin mechanism that would allow driver development 
> independently from core GDAL. Again, I might not have enough background to 
> give valuable feedback.
> 
> Regards,
> 
> David.
> 
> On Wed, Jan 27, 2021 at 12:34 PM thomas bonfort  > wrote:
> 
> 
> On Tue, Jan 12, 2021 at 11:36 PM Even Rouault  > wrote:
> 
> The issue with esoteric/legacy drivers is not that much maintenance of the 
> actual code of the drivers, in the sense of dealing with bug reports, 
> questions, etc. (pretty sure they are none for the ones I listed). Most of 
> them must work reasonably well and be feature complete, and most 
> vulnerabilities have now been fixed. My concern is that this legacy code has 
> indirect costs on other GDAL developers and users. The psychological cost I 
> mentionned. Let's say someone want to turn on higher warning levels, and that 
> this breaks in tens of drivers. Would he have to ping every maintainer and 
> wait for them to address the issue ? Or maybe he will just give up. Similarly 
> for breaking changes in the driver API. As Sean mentionned, this is probably 
> a 
> serious obstacle to growing up the core development team.
> 
> Given that we have a relatively easy way to disable individual drivers at 
> compile time, could we expand on this mechanism to m

Re: [gdal-dev] Considering drivers removal ?

[Tamas Szekeres]

Hi Sean,

I would also be supportive to formalize the driver submission process.

Best regards,

Tamas


Sean Gillies  ezt írta (időpont: 2021. jan. 28., Cs,
17:39):

> Hi Tamas,
>
> Are you suggesting that a RFC be required for a new driver? I would
> support this 100%.
>
> On Wed, Jan 27, 2021 at 2:17 PM Tamas Szekeres 
> wrote:
>
>> David,
>>
>> Up to this time the driver writers were highly welcomed to author new
>> drivers for the project and these effort didn't require a separate RFC in
>> terms of the Project Management Committee Guidelines
>>  document. Adding new
>> drivers hasn't been considered to be substantial changes or something that
>> causes to change the API or brings in backwards incompatibility issues.
>> However in a real deployment situation the drivers may cause issues for
>> the developers, end users and package maintainers from several aspects like
>> so:
>>
>> 1. The drivers may depend on external libraries and we don't want to link
>> against those libraries in all cases.
>> 2. The external libraries may cause license incompatibilities, ie linking
>> against that may change the license terms of the overall project.
>> 3. Not all of the drivers are required in a particular solution (that
>> applies to the built in drivers, too). In a real situation the application
>> may use only a limited set of drivers and the existence of the unwanted
>> drivers may cause some performance degradation.
>> 4. Implementing custom compilations (and omitting unwanted drivers from
>> the build) may be problematic for most of the users or projects utilizing
>> gdal as a sub-project.
>>
>> In my opinion, the current solution of incubating new drivers is fairly
>> minimalistic, we might probably want to know what kind of format is being
>> addressed, who is the audience, how amount of user might be of interest,
>> how the licensing of the dependent project is looking like, is the
>> dependent project (if any) maintained well enough and can be compiled to
>> each supported platforms etc.
>>
>> But replying to the question, I think you shoud continue the effort, but
>> consider to implement a plugin build for that. There are several existing
>> drivers are compiled as plugins at the moment, so it should not be so
>> difficult.
>>
>>
>> Best regards,
>>
>> Tamas
>>
>>
>> David Brochart  ezt írta (időpont: 2021. jan.
>> 27., Sze, 15:29):
>>
>>> I am currently trying to add a Zarr driver to GDAL (see
>>> https://github.com/OSGeo/gdal/pull/3411), but from what I can see the
>>> trend is to remove drivers, so I'm now wondering it I should pursue this
>>> effort. I'm relatively new to GDAL, but from my point of view supporting a
>>> lot of formats is part of GDAL's success, so maybe the real focus should be
>>> on implementing a plugin mechanism that would allow driver development
>>> independently from core GDAL. Again, I might not have enough background to
>>> give valuable feedback.
>>>
>>> Regards,
>>>
>>> David.
>>>
>>> On Wed, Jan 27, 2021 at 12:34 PM thomas bonfort <
>>> thomas.bonf...@gmail.com> wrote:
>>>


 On Tue, Jan 12, 2021 at 11:36 PM Even Rouault <
 even.roua...@spatialys.com> wrote:

>
> The issue with esoteric/legacy drivers is not that much maintenance of
> the
> actual code of the drivers, in the sense of dealing with bug reports,
> questions, etc. (pretty sure they are none for the ones I listed).
> Most of
> them must work reasonably well and be feature complete, and most
> vulnerabilities have now been fixed. My concern is that this legacy
> code has
> indirect costs on other GDAL developers and users. The psychological
> cost I
> mentionned. Let's say someone want to turn on higher warning levels,
> and that
> this breaks in tens of drivers. Would he have to ping every maintainer
> and
> wait for them to address the issue ? Or maybe he will just give up.
> Similarly
> for breaking changes in the driver API. As Sean mentionned, this is
> probably a
> serious obstacle to growing up the core development team.
>

 Given that we have a relatively easy way to disable individual drivers
 at compile time, could we expand on this mechanism to mark problematic
 drivers as "orphaned" in this case? The code would still live in the repo
 but would be effectively disabled until someone with sufficient interest
 invests the time/money to update the problematic code and re-enable it.
 This will of course create some overhead to keep track of which drivers
 have been orphaned from one release to the next, create github
 issues/labels to list which drivers need work to be re-enabled, etc... But
 it shifts the burden of maintaining the codebase of 250 drivers from the
 core team over to the people/companies who actually need them. I'd be
 willing to contribute the scripts that could ease the process of
 or

Re: [gdal-dev] Considering drivers removal ?

[Sean Gillies via gdal-dev]

Hi Tamas,

Are you suggesting that a RFC be required for a new driver? I would support
this 100%.

On Wed, Jan 27, 2021 at 2:17 PM Tamas Szekeres  wrote:

> David,
>
> Up to this time the driver writers were highly welcomed to author new
> drivers for the project and these effort didn't require a separate RFC in
> terms of the Project Management Committee Guidelines
>  document. Adding new
> drivers hasn't been considered to be substantial changes or something that
> causes to change the API or brings in backwards incompatibility issues.
> However in a real deployment situation the drivers may cause issues for
> the developers, end users and package maintainers from several aspects like
> so:
>
> 1. The drivers may depend on external libraries and we don't want to link
> against those libraries in all cases.
> 2. The external libraries may cause license incompatibilities, ie linking
> against that may change the license terms of the overall project.
> 3. Not all of the drivers are required in a particular solution (that
> applies to the built in drivers, too). In a real situation the application
> may use only a limited set of drivers and the existence of the unwanted
> drivers may cause some performance degradation.
> 4. Implementing custom compilations (and omitting unwanted drivers from
> the build) may be problematic for most of the users or projects utilizing
> gdal as a sub-project.
>
> In my opinion, the current solution of incubating new drivers is fairly
> minimalistic, we might probably want to know what kind of format is being
> addressed, who is the audience, how amount of user might be of interest,
> how the licensing of the dependent project is looking like, is the
> dependent project (if any) maintained well enough and can be compiled to
> each supported platforms etc.
>
> But replying to the question, I think you shoud continue the effort, but
> consider to implement a plugin build for that. There are several existing
> drivers are compiled as plugins at the moment, so it should not be so
> difficult.
>
>
> Best regards,
>
> Tamas
>
>
> David Brochart  ezt írta (időpont: 2021. jan.
> 27., Sze, 15:29):
>
>> I am currently trying to add a Zarr driver to GDAL (see
>> https://github.com/OSGeo/gdal/pull/3411), but from what I can see the
>> trend is to remove drivers, so I'm now wondering it I should pursue this
>> effort. I'm relatively new to GDAL, but from my point of view supporting a
>> lot of formats is part of GDAL's success, so maybe the real focus should be
>> on implementing a plugin mechanism that would allow driver development
>> independently from core GDAL. Again, I might not have enough background to
>> give valuable feedback.
>>
>> Regards,
>>
>> David.
>>
>> On Wed, Jan 27, 2021 at 12:34 PM thomas bonfort 
>> wrote:
>>
>>>
>>>
>>> On Tue, Jan 12, 2021 at 11:36 PM Even Rouault <
>>> even.roua...@spatialys.com> wrote:
>>>

 The issue with esoteric/legacy drivers is not that much maintenance of
 the
 actual code of the drivers, in the sense of dealing with bug reports,
 questions, etc. (pretty sure they are none for the ones I listed). Most
 of
 them must work reasonably well and be feature complete, and most
 vulnerabilities have now been fixed. My concern is that this legacy
 code has
 indirect costs on other GDAL developers and users. The psychological
 cost I
 mentionned. Let's say someone want to turn on higher warning levels,
 and that
 this breaks in tens of drivers. Would he have to ping every maintainer
 and
 wait for them to address the issue ? Or maybe he will just give up.
 Similarly
 for breaking changes in the driver API. As Sean mentionned, this is
 probably a
 serious obstacle to growing up the core development team.

>>>
>>> Given that we have a relatively easy way to disable individual drivers
>>> at compile time, could we expand on this mechanism to mark problematic
>>> drivers as "orphaned" in this case? The code would still live in the repo
>>> but would be effectively disabled until someone with sufficient interest
>>> invests the time/money to update the problematic code and re-enable it.
>>> This will of course create some overhead to keep track of which drivers
>>> have been orphaned from one release to the next, create github
>>> issues/labels to list which drivers need work to be re-enabled, etc... But
>>> it shifts the burden of maintaining the codebase of 250 drivers from the
>>> core team over to the people/companies who actually need them. I'd be
>>> willing to contribute the scripts that could ease the process of
>>> orphaning/reintegrating a specific driver.
>>>
>>> Regards,
>>> Thomas
>>>
>>
-- 
Sean Gillies
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Andrew C Aitchison]

On Wed, 27 Jan 2021, gdisk.mike wrote:


Just curious -
Some time ago there was a motion for RFC 76 - OGR Python drivers. Correct
me if I'm wrong, but this sort of makes it easy to add python drivers?
Perhaps this could be a means for others to add/reclaim a driver that gets
dropped?


RFC 76 is only for vector drivers;
most of the formats being discussed are raster formats.

--
Andrew C. Aitchison Kendal, UK
and...@aitchison.me.uk
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[gdisk.mike]

Just curious -
Some time ago there was a motion for RFC 76 - OGR Python drivers. Correct
me if I'm wrong, but this sort of makes it easy to add python drivers?
Perhaps this could be a means for others to add/reclaim a driver that gets
dropped?

v/r,
Mike



On Wed, Jan 27, 2021 at 2:17 PM Tamas Szekeres  wrote:

> David,
>
> Up to this time the driver writers were highly welcomed to author new
> drivers for the project and these effort didn't require a separate RFC in
> terms of the Project Management Committee Guidelines
>  document. Adding new
> drivers hasn't been considered to be substantial changes or something that
> causes to change the API or brings in backwards incompatibility issues.
> However in a real deployment situation the drivers may cause issues for
> the developers, end users and package maintainers from several aspects like
> so:
>
> 1. The drivers may depend on external libraries and we don't want to link
> against those libraries in all cases.
> 2. The external libraries may cause license incompatibilities, ie linking
> against that may change the license terms of the overall project.
> 3. Not all of the drivers are required in a particular solution (that
> applies to the built in drivers, too). In a real situation the application
> may use only a limited set of drivers and the existence of the unwanted
> drivers may cause some performance degradation.
> 4. Implementing custom compilations (and omitting unwanted drivers from
> the build) may be problematic for most of the users or projects utilizing
> gdal as a sub-project.
>
> In my opinion, the current solution of incubating new drivers is fairly
> minimalistic, we might probably want to know what kind of format is being
> addressed, who is the audience, how amount of user might be of interest,
> how the licensing of the dependent project is looking like, is the
> dependent project (if any) maintained well enough and can be compiled to
> each supported platforms etc.
>
> But replying to the question, I think you shoud continue the effort, but
> consider to implement a plugin build for that. There are several existing
> drivers are compiled as plugins at the moment, so it should not be so
> difficult.
>
>
> Best regards,
>
> Tamas
>
>
> David Brochart  ezt írta (időpont: 2021. jan.
> 27., Sze, 15:29):
>
>> I am currently trying to add a Zarr driver to GDAL (see
>> https://github.com/OSGeo/gdal/pull/3411), but from what I can see the
>> trend is to remove drivers, so I'm now wondering it I should pursue this
>> effort. I'm relatively new to GDAL, but from my point of view supporting a
>> lot of formats is part of GDAL's success, so maybe the real focus should be
>> on implementing a plugin mechanism that would allow driver development
>> independently from core GDAL. Again, I might not have enough background to
>> give valuable feedback.
>>
>> Regards,
>>
>> David.
>>
>> On Wed, Jan 27, 2021 at 12:34 PM thomas bonfort 
>> wrote:
>>
>>>
>>>
>>> On Tue, Jan 12, 2021 at 11:36 PM Even Rouault <
>>> even.roua...@spatialys.com> wrote:
>>>

 The issue with esoteric/legacy drivers is not that much maintenance of
 the
 actual code of the drivers, in the sense of dealing with bug reports,
 questions, etc. (pretty sure they are none for the ones I listed). Most
 of
 them must work reasonably well and be feature complete, and most
 vulnerabilities have now been fixed. My concern is that this legacy
 code has
 indirect costs on other GDAL developers and users. The psychological
 cost I
 mentionned. Let's say someone want to turn on higher warning levels,
 and that
 this breaks in tens of drivers. Would he have to ping every maintainer
 and
 wait for them to address the issue ? Or maybe he will just give up.
 Similarly
 for breaking changes in the driver API. As Sean mentionned, this is
 probably a
 serious obstacle to growing up the core development team.

>>>
>>> Given that we have a relatively easy way to disable individual drivers
>>> at compile time, could we expand on this mechanism to mark problematic
>>> drivers as "orphaned" in this case? The code would still live in the repo
>>> but would be effectively disabled until someone with sufficient interest
>>> invests the time/money to update the problematic code and re-enable it.
>>> This will of course create some overhead to keep track of which drivers
>>> have been orphaned from one release to the next, create github
>>> issues/labels to list which drivers need work to be re-enabled, etc... But
>>> it shifts the burden of maintaining the codebase of 250 drivers from the
>>> core team over to the people/companies who actually need them. I'd be
>>> willing to contribute the scripts that could ease the process of
>>> orphaning/reintegrating a specific driver.
>>>
>>> Regards,
>>> Thomas
>>> ___
>>> gdal-dev mailing list
>

Re: [gdal-dev] Considering drivers removal ?

[Tamas Szekeres]

David,

Up to this time the driver writers were highly welcomed to author new
drivers for the project and these effort didn't require a separate RFC in
terms of the Project Management Committee Guidelines
 document. Adding new
drivers hasn't been considered to be substantial changes or something that
causes to change the API or brings in backwards incompatibility issues.
However in a real deployment situation the drivers may cause issues for the
developers, end users and package maintainers from several aspects like so:

1. The drivers may depend on external libraries and we don't want to link
against those libraries in all cases.
2. The external libraries may cause license incompatibilities, ie linking
against that may change the license terms of the overall project.
3. Not all of the drivers are required in a particular solution (that
applies to the built in drivers, too). In a real situation the application
may use only a limited set of drivers and the existence of the unwanted
drivers may cause some performance degradation.
4. Implementing custom compilations (and omitting unwanted drivers from the
build) may be problematic for most of the users or projects utilizing gdal
as a sub-project.

In my opinion, the current solution of incubating new drivers is fairly
minimalistic, we might probably want to know what kind of format is being
addressed, who is the audience, how amount of user might be of interest,
how the licensing of the dependent project is looking like, is the
dependent project (if any) maintained well enough and can be compiled to
each supported platforms etc.

But replying to the question, I think you shoud continue the effort, but
consider to implement a plugin build for that. There are several existing
drivers are compiled as plugins at the moment, so it should not be so
difficult.


Best regards,

Tamas


David Brochart  ezt írta (időpont: 2021. jan.
27., Sze, 15:29):

> I am currently trying to add a Zarr driver to GDAL (see
> https://github.com/OSGeo/gdal/pull/3411), but from what I can see the
> trend is to remove drivers, so I'm now wondering it I should pursue this
> effort. I'm relatively new to GDAL, but from my point of view supporting a
> lot of formats is part of GDAL's success, so maybe the real focus should be
> on implementing a plugin mechanism that would allow driver development
> independently from core GDAL. Again, I might not have enough background to
> give valuable feedback.
>
> Regards,
>
> David.
>
> On Wed, Jan 27, 2021 at 12:34 PM thomas bonfort 
> wrote:
>
>>
>>
>> On Tue, Jan 12, 2021 at 11:36 PM Even Rouault 
>> wrote:
>>
>>>
>>> The issue with esoteric/legacy drivers is not that much maintenance of
>>> the
>>> actual code of the drivers, in the sense of dealing with bug reports,
>>> questions, etc. (pretty sure they are none for the ones I listed). Most
>>> of
>>> them must work reasonably well and be feature complete, and most
>>> vulnerabilities have now been fixed. My concern is that this legacy code
>>> has
>>> indirect costs on other GDAL developers and users. The psychological
>>> cost I
>>> mentionned. Let's say someone want to turn on higher warning levels, and
>>> that
>>> this breaks in tens of drivers. Would he have to ping every maintainer
>>> and
>>> wait for them to address the issue ? Or maybe he will just give up.
>>> Similarly
>>> for breaking changes in the driver API. As Sean mentionned, this is
>>> probably a
>>> serious obstacle to growing up the core development team.
>>>
>>
>> Given that we have a relatively easy way to disable individual drivers at
>> compile time, could we expand on this mechanism to mark problematic drivers
>> as "orphaned" in this case? The code would still live in the repo but would
>> be effectively disabled until someone with sufficient interest invests the
>> time/money to update the problematic code and re-enable it.
>> This will of course create some overhead to keep track of which drivers
>> have been orphaned from one release to the next, create github
>> issues/labels to list which drivers need work to be re-enabled, etc... But
>> it shifts the burden of maintaining the codebase of 250 drivers from the
>> core team over to the people/companies who actually need them. I'd be
>> willing to contribute the scripts that could ease the process of
>> orphaning/reintegrating a specific driver.
>>
>> Regards,
>> Thomas
>> ___
>> gdal-dev mailing list
>> gdal-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Stephen Woodbridge]

I think a 4th option is a hybrid approach of moving to a more modular plug-in 
architecture that allows the core more flexibility to evolve at the same time 
by moving to a more plug-in driver allows for more independent development, 
testing and release lets the community participation. This does stop the core 
from maintaining some of the drivers also. I don’t see value in maintaining 
obsolete drivers or drivers that only a few people use if it costs us a lot to 
maintain them. 

That said figuring out the long term funding for the project core is critical. 

Unfortunately I’m only in a position to offer an opinion and not much more so 
feel free to ignore it. 

Best regards,
-Steve W

Sent from my iPhone

> On Jan 27, 2021, at 12:28 PM, Howard Butler  wrote:
> 
> 
>> 
>> but from my point of view supporting a lot of formats is part of GDAL's 
>> success,
> 
> GDAL is a 22 year old software project. It's not just that GDAL supports lots 
> of formats. It is also that the code supporting all of those formats is 
> meticulously maintained, and it maintains *good* support for all of those 
> formats. The bulk of that meticulous maintenance has not been evenly carried 
> by the individuals, organizations, and companies that have been enjoying the 
> benefits from it. GDAL maintenance as currently happen(ed) is unsustainable 
> in all of the ways you read about in handwringing think pieces on the 
> internet about open source software projects.
> 
>> so maybe the real focus should be on implementing a plugin mechanism that 
>> would allow driver development independently from core GDAL.
> 
> As I see it, the project has three potential futures:
> 
> 1) Continue the current architectural and niche trajectory. A one-stop-shop 
> for geospatial formats that is conveniently distributed in all relevant 
> platforms.
> 2) Split GDAL/OGR core from the drivers so that each can evolve and be 
> maintained at their own pace according to the attention they can attract.
> 3) Let GDAL rot as-is with low wattage community maintenance and exist as a 
> zombie gut pile of useful code that organizations continue to pull from and 
> incorporate into their own software. 
> 
> I think we as a community want status quo – #1 – all of the goodness that 
> GDAL provides by a complete implementation of the geospatial format universe 
> all in one spot. As should be becoming clear by these threads, this scenario 
> is not likely to continue due to the three jobs problem [1] I described 
> earlier in the thread. Our options to maintain this status quo is for the 
> community to provide the revenue stream for someone to do just the maintainer 
> job, effectively split the maintenance activities, or find another Even that 
> wants three jobs :)
> 
> The second scenario has the potential to make it easier to share the 
> maintenance burden, but it cleaves off what many see as GDAL's best feature – 
> universality – by making support for specific formats be a packager's or a 
> user's burden. It would limit the GDAL platform leverage that vendors 
> currently get by injecting support for their proprietary SDKs for the project 
> to carry, and the impact and station of GDAL is likely to be reduced by this 
> approach. Maybe that could be a good thing.
> 
> The third scenario is a common one. Organizations with the need and resources 
> to internally spend will continue to maintain GDAL in their (closed) 
> codebases. The software-based interoperability that GDAL provides the 
> industry will diminish, and the existing tree will reach a kind of stasis 
> with open source distributors until the bugs accumulate in frequency and 
> scope to cause it to get dropped. 
> 
> 
> Howard
> 
> [1] https://lists.osgeo.org/pipermail/gdal-dev/2021-January/053302.html
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Howard Butler]

> but from my point of view supporting a lot of formats is part of GDAL's 
> success,

GDAL is a 22 year old software project. It's not just that GDAL supports lots 
of formats. It is also that the code supporting all of those formats is 
meticulously maintained, and it maintains *good* support for all of those 
formats. The bulk of that meticulous maintenance has not been evenly carried by 
the individuals, organizations, and companies that have been enjoying the 
benefits from it. GDAL maintenance as currently happen(ed) is unsustainable in 
all of the ways you read about in handwringing think pieces on the internet 
about open source software projects.

> so maybe the real focus should be on implementing a plugin mechanism that 
> would allow driver development independently from core GDAL.

As I see it, the project has three potential futures:

1) Continue the current architectural and niche trajectory. A one-stop-shop for 
geospatial formats that is conveniently distributed in all relevant platforms.
2) Split GDAL/OGR core from the drivers so that each can evolve and be 
maintained at their own pace according to the attention they can attract.
3) Let GDAL rot as-is with low wattage community maintenance and exist as a 
zombie gut pile of useful code that organizations continue to pull from and 
incorporate into their own software. 

I think we as a community want status quo – #1 – all of the goodness that GDAL 
provides by a complete implementation of the geospatial format universe all in 
one spot. As should be becoming clear by these threads, this scenario is not 
likely to continue due to the three jobs problem [1] I described earlier in the 
thread. Our options to maintain this status quo is for the community to provide 
the revenue stream for someone to do just the maintainer job, effectively split 
the maintenance activities, or find another Even that wants three jobs :)

The second scenario has the potential to make it easier to share the 
maintenance burden, but it cleaves off what many see as GDAL's best feature – 
universality – by making support for specific formats be a packager's or a 
user's burden. It would limit the GDAL platform leverage that vendors currently 
get by injecting support for their proprietary SDKs for the project to carry, 
and the impact and station of GDAL is likely to be reduced by this approach. 
Maybe that could be a good thing.

The third scenario is a common one. Organizations with the need and resources 
to internally spend will continue to maintain GDAL in their (closed) codebases. 
The software-based interoperability that GDAL provides the industry will 
diminish, and the existing tree will reach a kind of stasis with open source 
distributors until the bugs accumulate in frequency and scope to cause it to 
get dropped. 


Howard

[1] https://lists.osgeo.org/pipermail/gdal-dev/2021-January/053302.html___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[David Brochart]

I am currently trying to add a Zarr driver to GDAL (see
https://github.com/OSGeo/gdal/pull/3411), but from what I can see the trend
is to remove drivers, so I'm now wondering it I should pursue this effort.
I'm relatively new to GDAL, but from my point of view supporting a lot of
formats is part of GDAL's success, so maybe the real focus should be on
implementing a plugin mechanism that would allow driver development
independently from core GDAL. Again, I might not have enough background to
give valuable feedback.

Regards,

David.

On Wed, Jan 27, 2021 at 12:34 PM thomas bonfort 
wrote:

>
>
> On Tue, Jan 12, 2021 at 11:36 PM Even Rouault 
> wrote:
>
>>
>> The issue with esoteric/legacy drivers is not that much maintenance of
>> the
>> actual code of the drivers, in the sense of dealing with bug reports,
>> questions, etc. (pretty sure they are none for the ones I listed). Most
>> of
>> them must work reasonably well and be feature complete, and most
>> vulnerabilities have now been fixed. My concern is that this legacy code
>> has
>> indirect costs on other GDAL developers and users. The psychological cost
>> I
>> mentionned. Let's say someone want to turn on higher warning levels, and
>> that
>> this breaks in tens of drivers. Would he have to ping every maintainer
>> and
>> wait for them to address the issue ? Or maybe he will just give up.
>> Similarly
>> for breaking changes in the driver API. As Sean mentionned, this is
>> probably a
>> serious obstacle to growing up the core development team.
>>
>
> Given that we have a relatively easy way to disable individual drivers at
> compile time, could we expand on this mechanism to mark problematic drivers
> as "orphaned" in this case? The code would still live in the repo but would
> be effectively disabled until someone with sufficient interest invests the
> time/money to update the problematic code and re-enable it.
> This will of course create some overhead to keep track of which drivers
> have been orphaned from one release to the next, create github
> issues/labels to list which drivers need work to be re-enabled, etc... But
> it shifts the burden of maintaining the codebase of 250 drivers from the
> core team over to the people/companies who actually need them. I'd be
> willing to contribute the scripts that could ease the process of
> orphaning/reintegrating a specific driver.
>
> Regards,
> Thomas
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[thomas bonfort]

On Tue, Jan 12, 2021 at 11:36 PM Even Rouault 
wrote:

>
> The issue with esoteric/legacy drivers is not that much maintenance of the
> actual code of the drivers, in the sense of dealing with bug reports,
> questions, etc. (pretty sure they are none for the ones I listed). Most of
> them must work reasonably well and be feature complete, and most
> vulnerabilities have now been fixed. My concern is that this legacy code
> has
> indirect costs on other GDAL developers and users. The psychological cost
> I
> mentionned. Let's say someone want to turn on higher warning levels, and
> that
> this breaks in tens of drivers. Would he have to ping every maintainer and
> wait for them to address the issue ? Or maybe he will just give up.
> Similarly
> for breaking changes in the driver API. As Sean mentionned, this is
> probably a
> serious obstacle to growing up the core development team.
>

Given that we have a relatively easy way to disable individual drivers at
compile time, could we expand on this mechanism to mark problematic drivers
as "orphaned" in this case? The code would still live in the repo but would
be effectively disabled until someone with sufficient interest invests the
time/money to update the problematic code and re-enable it.
This will of course create some overhead to keep track of which drivers
have been orphaned from one release to the next, create github
issues/labels to list which drivers need work to be re-enabled, etc... But
it shifts the burden of maintaining the codebase of 250 drivers from the
core team over to the people/companies who actually need them. I'd be
willing to contribute the scripts that could ease the process of
orphaning/reintegrating a specific driver.

Regards,
Thomas
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Stephen Woodbridge]

On 1/12/2021 5:36 PM, Even Rouault wrote:

On mardi 12 janvier 2021 12:56:13 CET Frank Warmerdam wrote:

On Tue, Jan 12, 2021 at 12:38 PM Howard Butler  wrote:

The only question that matters here is "Who is going to maintain it?" and
if the answer to that is "no one", it should be removed. There doesn't
need
to be any meetings because the only criteria that matters is if someone is
willing to maintain it. We should provide the list of drivers and assign
the GitHub handles that step forward to be responsible for each. If
obscure
government one-offs formats have an audience of downtrodden government
users forced to use them, they need to put their handle in and take
ownership. They then need to find the time, money, or attention to carry
things forward.

Howard / Even,

I'd be willing to commit to maintaining some of the archaic drivers that
meet the conditions I mentioned (buildable, testable in core build).  If
Even would like I can provide a sublist of those he proposed i'd be willing
to be responsible for.

NTv1: this is the perfect example of a driver of absolutely no use in 2021.
Unless I'm wrong, there was only one single public dataset for that format,
ntv1_can.dat, and it is now available as GeoTIFF in
https://github.com/OSGeo/PROJ-data/blob/master/ca_nrc/ca_nrc_ntv1_can.tif

My current plan is:

- for BPG, E00GRID, EPSILON, IGNFHeightASCIIGrid, ISG, Aeronav FAA, BNA, HTF,
OpenAir, SEG-P1, SEG-Y, SUA, X-Plane, move *now* driver code, documentation
and tests to https://github.com/OSGeo/gdal-extra-drivers which is a slightly
improved version of a cemetery repository, since it includes a build script to
create a plugin. I have no plan to maintain that repository after that initial
move (that means I won't merge pull requests unless someone else steps up for
the role) and it will likely break in the future. I'd wish we would agree to
move more drivers there. And probably most future drivers for esoteric formats
should go there.
Those drivers are ones I've authored, that received no significant
contribution from anyone else AFAIR, no-one paid development for and I suspect
are close to be unused. So hopefully no one should have bad feelings with them
going away. It was a bad taste of mine to have put them in GDAL to start with.
Why did I picked up the extra repository after all ? A tiny fraction of them
might be useful like ISG or IGNFHeightASCIIGrid in some contexts (to create
grids for PROJ), but definitely not for general purpose. So as far as I'm
concerned, I'll go through the extra step of building the extra repository or
a subset of it if I've an occasional need for them.

- proceed as I mentionned initially for other drivers I listed and no-one
steps up to maintain, with the variation of moving the code to gdal-extra-
drivers instead of just removing them (but potentially not including a build
recipee for them in the build script, if that proves to be too complex).

The issue with esoteric/legacy drivers is not that much maintenance of the
actual code of the drivers, in the sense of dealing with bug reports,
questions, etc. (pretty sure they are none for the ones I listed). Most of
them must work reasonably well and be feature complete, and most
vulnerabilities have now been fixed. My concern is that this legacy code has
indirect costs on other GDAL developers and users. The psychological cost I
mentionned. Let's say someone want to turn on higher warning levels, and that
this breaks in tens of drivers. Would he have to ping every maintainer and
wait for them to address the issue ? Or maybe he will just give up. Similarly
for breaking changes in the driver API. As Sean mentionned, this is probably a
serious obstacle to growing up the core development team.


Even


Even,

Just want to say this sounds like good plan to me, not that my input 
means a lot. I also want to say Thank You! for all your hard work 
supporting this and other projects, but answering my questions through 
the years. I've had a lot of roles in my career in open source and 
industry and can appreciate the difficult balance between compatibility 
with legacy code and the need to break free of it to move forward. It's 
hard and I never enjoyed having to make those decisions, but you have my 
respect and support whatever you decide.


Thank You! again for your efforts and support!

-Steve W
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Even Rouault]

On mardi 12 janvier 2021 12:56:13 CET Frank Warmerdam wrote:
> On Tue, Jan 12, 2021 at 12:38 PM Howard Butler  wrote:
> > The only question that matters here is "Who is going to maintain it?" and
> > if the answer to that is "no one", it should be removed. There doesn't
> > need
> > to be any meetings because the only criteria that matters is if someone is
> > willing to maintain it. We should provide the list of drivers and assign
> > the GitHub handles that step forward to be responsible for each. If
> > obscure
> > government one-offs formats have an audience of downtrodden government
> > users forced to use them, they need to put their handle in and take
> > ownership. They then need to find the time, money, or attention to carry
> > things forward.
> 
> Howard / Even,
> 
> I'd be willing to commit to maintaining some of the archaic drivers that
> meet the conditions I mentioned (buildable, testable in core build).  If
> Even would like I can provide a sublist of those he proposed i'd be willing
> to be responsible for.

NTv1: this is the perfect example of a driver of absolutely no use in 2021. 
Unless I'm wrong, there was only one single public dataset for that format, 
ntv1_can.dat, and it is now available as GeoTIFF in
https://github.com/OSGeo/PROJ-data/blob/master/ca_nrc/ca_nrc_ntv1_can.tif

My current plan is:

- for BPG, E00GRID, EPSILON, IGNFHeightASCIIGrid, ISG, Aeronav FAA, BNA, HTF, 
OpenAir, SEG-P1, SEG-Y, SUA, X-Plane, move *now* driver code, documentation  
and tests to https://github.com/OSGeo/gdal-extra-drivers which is a slightly 
improved version of a cemetery repository, since it includes a build script to 
create a plugin. I have no plan to maintain that repository after that initial 
move (that means I won't merge pull requests unless someone else steps up for 
the role) and it will likely break in the future. I'd wish we would agree to 
move more drivers there. And probably most future drivers for esoteric formats 
should go there.
Those drivers are ones I've authored, that received no significant 
contribution from anyone else AFAIR, no-one paid development for and I suspect 
are close to be unused. So hopefully no one should have bad feelings with them 
going away. It was a bad taste of mine to have put them in GDAL to start with.
Why did I picked up the extra repository after all ? A tiny fraction of them 
might be useful like ISG or IGNFHeightASCIIGrid in some contexts (to create 
grids for PROJ), but definitely not for general purpose. So as far as I'm 
concerned, I'll go through the extra step of building the extra repository or 
a subset of it if I've an occasional need for them.

- proceed as I mentionned initially for other drivers I listed and no-one 
steps up to maintain, with the variation of moving the code to gdal-extra-
drivers instead of just removing them (but potentially not including a build 
recipee for them in the build script, if that proves to be too complex).

The issue with esoteric/legacy drivers is not that much maintenance of the 
actual code of the drivers, in the sense of dealing with bug reports, 
questions, etc. (pretty sure they are none for the ones I listed). Most of 
them must work reasonably well and be feature complete, and most 
vulnerabilities have now been fixed. My concern is that this legacy code has 
indirect costs on other GDAL developers and users. The psychological cost I 
mentionned. Let's say someone want to turn on higher warning levels, and that 
this breaks in tens of drivers. Would he have to ping every maintainer and 
wait for them to address the issue ? Or maybe he will just give up. Similarly 
for breaking changes in the driver API. As Sean mentionned, this is probably a 
serious obstacle to growing up the core development team.


Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Frank Warmerdam]

On Tue, Jan 12, 2021 at 12:38 PM Howard Butler  wrote:

> The only question that matters here is "Who is going to maintain it?" and
> if the answer to that is "no one", it should be removed. There doesn't need
> to be any meetings because the only criteria that matters is if someone is
> willing to maintain it. We should provide the list of drivers and assign
> the GitHub handles that step forward to be responsible for each. If obscure
> government one-offs formats have an audience of downtrodden government
> users forced to use them, they need to put their handle in and take
> ownership. They then need to find the time, money, or attention to carry
> things forward.
>

Howard / Even,

I'd be willing to commit to maintaining some of the archaic drivers that
meet the conditions I mentioned (buildable, testable in core build).  If
Even would like I can provide a sublist of those he proposed i'd be willing
to be responsible for.

Best regards,
-- 
---+--
I set the clouds in motion - turn up   | Frank Warmerdam,
warmer...@pobox.com
light and sound - activate the windows | +1 650-701-7823
and watch the world go round - Rush| Geospatial Software Developer
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Howard Butler]

> On Jan 12, 2021, at 8:48 AM, Jonathan Gale  wrote:
> 
> Hi,
>  
> Looking at the list, we use/support SDTS Raster import. As a US government 
> format with the use case mentioned below, I’d prefer to not see the format 
> removed from the software.  We have some evidence that the format is still 
> used by some of our users.

It would be helpful for Mathworks to provide the GDAL project a list of "used" 
formats in its software. GDAL doesn't get that recon, and it can't put phone 
home capability into the software to get it either.

>  The users of the software I help develop and maintain expect a high level of 
> compatibility across releases.  A commonly discussed use case involves 
> rerunning the same code with the same data across many years to ensure 
> reproducibility of results.  While we see the need to keep code maintainable, 
> it feels important to consider this use case. 

So the open source project your commercial software product depends upon is 
supposed to worry about forever version compatibility to protect your product? 
You may not have said it this way, but this is what I hear when I read this 
statement. It is especially frustrating to hear given the many commercial 
entities like Mathworks that have leveraged GDAL for years without providing 
non-targeted financial and developer resources back to the project to carry 
this kind of maintenance burden. 

The community doesn't maintain GDAL – Even Rouault does (right now, just like 
Frank Warmerdam did before him). The community doesn't really share the burden, 
but it does share its rewards (1000 line NEWS posts of updates to the project 
every release). Even is now searching for ways to make it more maintainable 
given that the resources to sustain the current trajectory have not 
materialized. This includes culling drivers that merely add burden and evolving 
old stuff where appropriate (code, configuration, APIs).  It isn't on us to say 
whether something is or isn't a burden either – we need to step into the breach 
if we need to keep something around. 

A problem with GDAL is the breach is a 1 million line behemoth with homegrown 
test suites, build setups, software layout, and type system. It is not a 
comfortable project to make casual contributions because it requires a lot of 
investment to get up to speed. You end up having to carry the rock of 20 years 
of software (mis)behavior as you do it. For anything other than adding another 
driver, this burden ends up being too great. So while we want more maintainers 
(working free to us!), the software itself makes it very difficult for them to 
grow into the role. Attempts to relax the dough to allow that to happen are met 
with, well, this mailing list thread.  
 
> Not specific to the software I work on, I think of GDAL as a “swiss army 
> knife” of geospatial format support.  It is the FOSS way to access many 
> formats both new and old.  I do wonder whether alternatives available should 
> be considered when evaluating a given driver for removal.  It would be 
> unfortunate if there was no way to read a format anymore with available 
> software.

The old versions are still available, but I understand as a commercial product, 
you want BOTH the maintenance and bug fixing of existing software plus all of 
the goodie features of a new release. 

> Overall, we like the idea of introducing a process for this.  Thinking about 
> next steps, it is important to consider what the criterion should be as Even 
> mentioned.  How many users?  Who are these users?  How do we estimate the 
> downstream impact on users who don’t know that their format is being 
> considered for removal?  As others have mentioned, how does maintainability 
> factor in?  I hope that if there was a process to do this, that it would be 
> codified and visible.

The only question that matters here is "Who is going to maintain it?" and if 
the answer to that is "no one", it should be removed. There doesn't need to be 
any meetings because the only criteria that matters is if someone is willing to 
maintain it. We should provide the list of drivers and assign the GitHub 
handles that step forward to be responsible for each. If obscure government 
one-offs formats have an audience of downtrodden government users forced to use 
them, they need to put their handle in and take ownership. They then need to 
find the time, money, or attention to carry things forward. 

Howard

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Jonathan Gale]

Hi,

Looking at the list, we use/support SDTS Raster import. As a US government 
format with the use case mentioned below, I'd prefer to not see the format 
removed from the software.  We have some evidence that the format is still used 
by some of our users.

The users of the software I help develop and maintain expect a high level of 
compatibility across releases.  A commonly discussed use case involves 
rerunning the same code with the same data across many years to ensure 
reproducibility of results.  While we see the need to keep code maintainable, 
it feels important to consider this use case.

Not specific to the software I work on, I think of GDAL as a "swiss army knife" 
of geospatial format support.  It is the FOSS way to access many formats both 
new and old.  I do wonder whether alternatives available should be considered 
when evaluating a given driver for removal.  It would be unfortunate if there 
was no way to read a format anymore with available software.

Overall, we like the idea of introducing a process for this.  Thinking about 
next steps, it is important to consider what the criterion should be as Even 
mentioned.  How many users?  Who are these users?  How do we estimate the 
downstream impact on users who don't know that their format is being considered 
for removal?  As others have mentioned, how does maintainability factor in?  I 
hope that if there was a process to do this, that it would be codified and 
visible.

Best,
Jonathan


From: gdal-dev  On Behalf Of Even Rouault
Sent: Sunday, January 10, 2021 6:02 PM
To: gdal-dev@lists.osgeo.org
Subject: [gdal-dev] Considering drivers removal ?

Hi,

It's not spring yet, but I'm in a mood lately of axing useless things, and we
probably have tons of candidate for that in GDAL, especially in drivers.
I was going to just axe the DB2 driver
(https://github.com/OSGeo/gdal/pull/3366<https://github.com/OSGeo/gdal/pull/3366>)
 but the issue is more general.

Any idea how we can know what is used and what isn't ? A "call-home"
functionality where we would track driver usage would only be acceptable if
people enable it and have network connectivity, so we won't probably get lots
of feedback. Having a spreadsheet with the driver list and asking people to
fill it would probably also receive little feedback. So the idea I had was to
do something like the following in the Open() method of a candidate for
removal:

GDALDataset* FooDriver::Open(  )
{
if( !Identify(poOpenInfo) )
return nullptr;

if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
{
CPLError(CE_Failure, CPLE_AppDefined,
"Driver FOO is considered for removal in GDAL 3.5. You are invited "
"to convert any dataset in that format to another more common one ."
"If you need this driver in future GDAL versions, create a ticket at "
"https://github.com/OSGeo/gdal<https://github.com/OSGeo/gdal> (look first for 
an existing one first) to "
"explain how critical it is for you (but the GDAL project may still "
"remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
"configuration option / environment variable to YES");
return nullptr;
}
...
}

That is, when we detect a file to be handled by the driver, emit the above
error message and do not open the dataset, unless the user defines the
environment variable.
Similarly in the Create()/CreateCopy() methods.
If we ship this in 3.3, with a 3.5 milestone for removal, this would offer a
feedback period of one year / 2 feature versions.

Here's my own list of candidates for retirement (probably over-conservative).
Mostly based on gut feeling. None of them are particularly bad citizens, but I
have no indication that they are still used, which doesn't mean they aren't.

* Raster side:
BPG
DB2Raster
DOQ1
DOQ2
E00GRID
Epsilon
FujiBAS
GS7BG
GSAG
IDA
JDEM
JPEG2000 (Jasper): JP2OpenJPEG is a better replacement
JPEGLS
LAN
MFF
MG4Lidar ?
NDF
NTv1
SDTS Raster
SGI
XPM
ZMap

* Vector side:
AERONAVFAA
ESRI ArcObjects
ARCGEN
BNA
Cloudant
CouchDB
DB2
DODS
FMEObjects Gateway
Geomedia MDB
GMT ASCII Vectors
GTM
HTF
INGRES
MongoDB (the old one, superseded by MongoDBv3)
OpenAIR
REC
SDTS
SUA
SVG
TIGER
WALK


Anything you'd add / remove ?

What is not obvious is what would be the criterion for keeping a driver: 1,
10, 100 users asking for the driver to be kept ?
If a GDAL developer contributing to the overall good of the project needs the
preservation of a driver to be able to justify its continued involvement, I'd
tend to think it to be enough to keep it.


Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com<http://www.spatialys.com>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev<https://lists.osgeo.org/mailman/listinfo/gdal-dev>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Jan Heckman]

In good company when pruning the source trees:
https://www.phoronix.com/scan.php?page=news_item&px=2021-Linux-Drop-Old-CPUs
Apparently 68000 may not be supported any longer in linux...
Just joking. Hope you don't mind...

On Tue, Jan 12, 2021 at 3:46 AM Jed O. Kaplan 
wrote:

> I for one am using the GMT vector driver for GDAL on a regular basis,
> e.g., for integration between GRASS GIS and GMT for cartography. I am
> grateful to the developers for their continued support for this driver.
>
> Thanks, Jed
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Jed O. Kaplan]

I for one am using the GMT vector driver for GDAL on a regular basis, e.g., for 
integration between GRASS GIS and GMT for cartography. I am grateful to the 
developers for their continued support for this driver.

Thanks, Jed

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Joaquim Manuel Freire Luís]

> - GMT is an active project and some GMT developers appear on this list as 
> well. Maybe some of them happend to read this and say if GMT ASCII vectors 
> are still important for GMT. Or otherwise I can ask from the GMT forum.


Right.

The GMT raster driver is from the times GNT used a 1-d array to represent 2-D 
grids in netCDF. That was abandoned some ~15 years ago for COARS compliant nc 
grids and is a good candidate for deprecation.

The " GMT ASCII vectors" format (written under contract with NIWA) is still 
very much used (under the hood) by the GMT library and should be kept alive 
until we finish the integration with the GDAL vector side.

Joaquim




-Original Message-
From: gdal-dev  On Behalf Of jratike80
Sent: Monday, January 11, 2021 6:56 PM
To: gdal-dev@lists.osgeo.org
Subject: Re: [gdal-dev] Considering drivers removal ?

Hi,

The joy of being a Windows user is that it is so easy to use old GDAL versions 
if the binaries still happen to be on some dusty backup disk. Even the FWTools 
including GDAL 1.7.0 from 2010 seemed to work fine and include quite a many 
later removed drivers.

A few comments about the driver list:

- There are indeed questions about SVG in gis.stackexchange every now and then.
- I used LAN a lot when FWTools was young and ERDAS wrote bad GeoTIFFs.
Things are probably changed since that. 
- GMT is an active project and some GMT developers appear on this list as well. 
Maybe some of them happend to read this and say if GMT ASCII vectors are still 
important for GMT. Or otherwise I can ask from the GMT forum.
- GPS Track Maker, as far as I know, is quite popular in Brazil. However, when 
I used the software I just used GPX format for data transfer. Are there any 
Brazilian GDAL user here to comment?

When it comes to Windows binaries, there is a very valuable archive in 
https://gisinternals.com/archive.php. It would be pity if it would get lost 
some day.

-Jukka Rahkonen-



Even Rouault-2 wrote
> Hi,
> 
> It's not spring yet, but I'm in a mood lately of axing useless things, 
> and we probably have tons of candidate for that in GDAL, especially in 
> drivers.
> I was going to just axe the DB2 driver
> (https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.
> 
> Any idea how we can know what is used and what isn't ? A "call-home" 
> functionality where we would track driver usage would only be 
> acceptable if people enable it and have network connectivity, so we 
> won't probably get lots of feedback. Having a spreadsheet with the 
> driver list and asking people to fill it would probably also receive 
> little feedback. So the idea I had was to do something like the 
> following in the Open() method of a candidate for
> removal:
> 
> GDALDataset* FooDriver::Open(  )
> {
>if( !Identify(poOpenInfo) )
>   return nullptr;
> 
>if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
>{
>CPLError(CE_Failure, CPLE_AppDefined,
> "Driver FOO is considered for removal in GDAL 3.5. You are invited "
> "to convert any dataset in that format to another more common one ."
> "If you need this driver in future GDAL versions, create a ticket at "
> "https://github.com/OSGeo/gdal (look first for an existing one 
> first) to "
> "explain how critical it is for you (but the GDAL project may still "
> "remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
> "configuration option / environment variable to YES");
>return nullptr;
> }
> ...
> }
> 
> That is, when we detect a file to be handled by the driver, emit the 
> above error message and do not open the dataset, unless the user 
> defines the environment variable.
> Similarly in the Create()/CreateCopy() methods.
> If we ship this in 3.3, with a 3.5 milestone for removal, this would 
> offer a feedback period of one year / 2 feature versions.
> 
> Here's my own list of candidates for retirement (probably 
> over-conservative).
> Mostly based on gut feeling. None of them are particularly bad 
> citizens, but I have no indication that they are still used, which 
> doesn't mean they aren't.
> 
> * Raster side:
> BPG
> DB2Raster
> DOQ1
> DOQ2
> E00GRID
> Epsilon
> FujiBAS
> GS7BG
> GSAG
> IDA
> JDEM
> JPEG2000 (Jasper): JP2OpenJPEG is a better replacement JPEGLS LAN MFF 
> MG4Lidar ?
> NDF
> NTv1
> SDTS Raster
> SGI
> XPM
> ZMap
> 
> * Vector side:
> AERONAVFAA
> ESRI ArcObjects
> ARCGEN
> BNA
> Cloudant
> CouchDB
> DB2
> DODS
> FMEObjects Gateway
> Geomedia MDB
> GMT ASCII Vectors
> GTM
> HTF
> INGRES
>

Re: [gdal-dev] Considering drivers removal ?

[Frank Warmerdam]

Even,

My opinion is that old drivers which do not depend on external
libraries/services and that we have tests for should be retained until they
cause painful problems.  I would be supportive of dropping drivers for
which there is no apparent interest, and that are not buildable and
testable due to dependence on external libraries and services.  We can
always reintroduce them if someone comes forward and wants them and is
willing to help support them.

LAN, NTv1 and SDTS Raster are examples (IMHO) of relatively low value
drivers that should be retained because they are buildable, testable and
not problematic.

Best regards,
Frank


On Sun, Jan 10, 2021 at 6:02 PM Even Rouault 
wrote:

> Hi,
>
> It's not spring yet, but I'm in a mood lately of axing useless things, and
> we
> probably have tons of candidate for that in GDAL, especially in drivers.
> I was going to just axe the DB2 driver
> (https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.
>
> Any idea how we can know what is used and what isn't ? A "call-home"
> functionality where we would track driver usage would only be acceptable
> if
> people enable it and have network connectivity, so we won't probably get
> lots
> of feedback. Having a spreadsheet with the driver list and asking people
> to
> fill it would probably also receive little feedback. So the idea I had was
> to
> do something like the following in the Open() method of a candidate for
> removal:
>
> GDALDataset* FooDriver::Open(  )
> {
>if( !Identify(poOpenInfo) )
>   return nullptr;
>
>if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
>{
>CPLError(CE_Failure, CPLE_AppDefined,
> "Driver FOO is considered for removal in GDAL 3.5. You are invited "
> "to convert any dataset in that format to another more common one ."
> "If you need this driver in future GDAL versions, create a ticket at "
> "https://github.com/OSGeo/gdal (look first for an existing one first)
> to "
> "explain how critical it is for you (but the GDAL project may still "
> "remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
> "configuration option / environment variable to YES");
>return nullptr;
> }
> ...
> }
>
> That is, when we detect a file to be handled by the driver, emit the above
> error message and do not open the dataset, unless the user defines the
> environment variable.
> Similarly in the Create()/CreateCopy() methods.
> If we ship this in 3.3, with a 3.5 milestone for removal, this would offer
> a
> feedback period of one year / 2 feature versions.
>
> Here's my own list of candidates for retirement (probably
> over-conservative).
> Mostly based on gut feeling. None of them are particularly bad citizens,
> but I
> have no indication that they are still used, which doesn't mean they
> aren't.
>
> * Raster side:
> BPG
> DB2Raster
> DOQ1
> DOQ2
> E00GRID
> Epsilon
> FujiBAS
> GS7BG
> GSAG
> IDA
> JDEM
> JPEG2000 (Jasper): JP2OpenJPEG is a better replacement
> JPEGLS
> LAN
> MFF
> MG4Lidar ?
> NDF
> NTv1
> SDTS Raster
> SGI
> XPM
> ZMap
>
> * Vector side:
> AERONAVFAA
> ESRI ArcObjects
> ARCGEN
> BNA
> Cloudant
> CouchDB
> DB2
> DODS
> FMEObjects Gateway
> Geomedia MDB
> GMT ASCII Vectors
> GTM
> HTF
> INGRES
> MongoDB (the old one, superseded by MongoDBv3)
> OpenAIR
> REC
> SDTS
> SUA
> SVG
> TIGER
> WALK
>
>
> Anything you'd add / remove ?
>
> What is not obvious is what would be the criterion for keeping a driver:
> 1,
> 10, 100 users asking for the driver to be kept ?
> If a GDAL developer contributing to the overall good of the project needs
> the
> preservation of a driver to be able to justify its continued involvement,
> I'd
> tend to think it to be enough to keep it.
>
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>


-- 
---+--
I set the clouds in motion - turn up   | Frank Warmerdam,
warmer...@pobox.com
light and sound - activate the windows | +1 650-701-7823
and watch the world go round - Rush| Geospatial Software Developer
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Sean Gillies via gdal-dev]

Hi Even,

On Mon, Jan 11, 2021 at 7:44 AM Even Rouault 
wrote:

> Hi,
>
> trying to answer the different points raised up to now:
>
> - SVG: let's keep it as it is used. This is exactly the feedback I'm
> seeking
> for. I had developed this as a toy, crazy me, I won't do it anymore. No
> idea
> anyone was using it actually.


> - making those driver plugins. This would require significant work, and
> the
> purpose is to save work. Our current build infrastructure is not ready for
> easy plugification. And announcing they will be unmaintained is probably
> not
> efficient to know in advance the impact of the planned breakage. People
> don't
> read documentation. The only way to force people to react is to break them
> in
> some way.
>
> - regarding Python, I'm not sure what the question is exactly. If it was
> how
> to still enable the drivers candidate for retirement to work, then it is
> just
> gdal.SetConfigOption('GDAL_ENABLE_DRIVER_FOO', 'YES')
>

Making users explicitly turn these formats back on feels good to me. I
don't see a downside to it. We might want to consider one round of warnings
before setting this option's default to NO? Doing so would take care of the
operators of deployed applications who *do* pay attention to warnings.

Would it make any sense to retire read and write of formats on a different
schedule? The fewer SDTS files written in the next 6-12 months, the less
impact there will be when we remove the driver.

When the change is made to GDAL, I might bikeshed about the config option
name a bit.


> - once we have decided which drivers should be retired, I don't find
> moving
> them to some cemetery repository very useful. Because that would lack part
> of
> the build scripts. What would be more useful is to add a paragraph in the
> documentation that drivers FOO, BAR, BAZ were retired in GDAL 3.5. That
> way
> people know they have to download a GDAL tarball or checkout a git tag
> before
> that release, or download a past OSGeo-Live VM, or Conda package, etc...
> And
> they will get (hopefully) functional code. Much better than the cemetery
> approach.
>
> - regarding schedule:
>* GDAL 3.3, May 2021: version with drivers semi-retired
>* GDAL 3.4, November 2021: still with drivers semi-retired
>* GDAL 3.5, May 2022: retired drivers are gone
>  So we won't get much feedback from Ubuntu LTS april 2022, as at that
> date,
> the drivers will have to be retired for the 3.5 release.
>
> - Where are the costs in keeping these drivers ?
>* monetary: there is one, but I'm unable to quantify it
>* environmental: burn CPU cycles each time someone does a GDAL build
>* psychological: prevent developers from doing modernization in GDAL
> internals. When you know you have to change 250 drivers, you think twice
> before doing the change, and generally you conclude it is not worth it, or
> fallback to hacks to limit the amount of code change. We should probably
> trim
> down the list to 20 raster and vector drivers to have a real effect
> regarding
> that. For a next time :-)
>
> Even
>

Would we disable default compilation of these drivers in 3.3 as well? I'd
be in favor of that.

Thanks for writing that third bullet point. I've thought twice about
changes for exactly that reason.

Formats come and go. We can't expect the GDAL project and its maintainers
(mostly you!) to be the curators forever of all data.

-- 
Sean Gillies
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[jratike80]

Hi,

The joy of being a Windows user is that it is so easy to use old GDAL
versions if the binaries still happen to be on some dusty backup disk. Even
the FWTools including GDAL 1.7.0 from 2010 seemed to work fine and include
quite a many later removed drivers.

A few comments about the driver list:

- There are indeed questions about SVG in gis.stackexchange every now and
then.
- I used LAN a lot when FWTools was young and ERDAS wrote bad GeoTIFFs.
Things are probably changed since that. 
- GMT is an active project and some GMT developers appear on this list as
well. Maybe some of them happend to read this and say if GMT ASCII vectors
are still important for GMT. Or otherwise I can ask from the GMT forum.
- GPS Track Maker, as far as I know, is quite popular in Brazil. However,
when I used the software I just used GPX format for data transfer. Are there
any Brazilian GDAL user here to comment?

When it comes to Windows binaries, there is a very valuable archive in
https://gisinternals.com/archive.php. It would be pity if it would get lost
some day.

-Jukka Rahkonen-



Even Rouault-2 wrote
> Hi,
> 
> It's not spring yet, but I'm in a mood lately of axing useless things, and
> we 
> probably have tons of candidate for that in GDAL, especially in drivers.
> I was going to just axe the DB2 driver
> (https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.
> 
> Any idea how we can know what is used and what isn't ? A "call-home" 
> functionality where we would track driver usage would only be acceptable
> if 
> people enable it and have network connectivity, so we won't probably get
> lots 
> of feedback. Having a spreadsheet with the driver list and asking people
> to 
> fill it would probably also receive little feedback. So the idea I had was
> to 
> do something like the following in the Open() method of a candidate for 
> removal:
> 
> GDALDataset* FooDriver::Open(  )
> {
>if( !Identify(poOpenInfo) )
>   return nullptr;
> 
>if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
>{
>CPLError(CE_Failure, CPLE_AppDefined,
> "Driver FOO is considered for removal in GDAL 3.5. You are invited "
> "to convert any dataset in that format to another more common one ."
> "If you need this driver in future GDAL versions, create a ticket at "
> "https://github.com/OSGeo/gdal (look first for an existing one first)
> to "
> "explain how critical it is for you (but the GDAL project may still "
> "remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
> "configuration option / environment variable to YES");
>return nullptr;
> }
> ...
> }
> 
> That is, when we detect a file to be handled by the driver, emit the above 
> error message and do not open the dataset, unless the user defines the 
> environment variable.
> Similarly in the Create()/CreateCopy() methods.
> If we ship this in 3.3, with a 3.5 milestone for removal, this would offer
> a 
> feedback period of one year / 2 feature versions.
> 
> Here's my own list of candidates for retirement (probably
> over-conservative). 
> Mostly based on gut feeling. None of them are particularly bad citizens,
> but I 
> have no indication that they are still used, which doesn't mean they
> aren't.
> 
> * Raster side:
> BPG
> DB2Raster
> DOQ1
> DOQ2
> E00GRID
> Epsilon
> FujiBAS
> GS7BG
> GSAG
> IDA
> JDEM
> JPEG2000 (Jasper): JP2OpenJPEG is a better replacement
> JPEGLS
> LAN
> MFF
> MG4Lidar ?
> NDF
> NTv1
> SDTS Raster
> SGI
> XPM
> ZMap
> 
> * Vector side:
> AERONAVFAA
> ESRI ArcObjects
> ARCGEN
> BNA
> Cloudant
> CouchDB
> DB2
> DODS
> FMEObjects Gateway
> Geomedia MDB
> GMT ASCII Vectors
> GTM
> HTF
> INGRES
> MongoDB (the old one, superseded by MongoDBv3)
> OpenAIR
> REC
> SDTS
> SUA
> SVG
> TIGER
> WALK
> 
> 
> Anything you'd add / remove ?
> 
> What is not obvious is what would be the criterion for keeping a driver:
> 1, 
> 10, 100 users asking for the driver to be kept ?
> If a GDAL developer contributing to the overall good of the project needs
> the 
> preservation of a driver to be able to justify its continued involvement,
> I'd 
> tend to think it to be enough to keep it.
> 
> 
> Even
> 
> -- 
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> ___
> gdal-dev mailing list

> gdal-dev@.osgeo

> https://lists.osgeo.org/mailman/listinfo/gdal-dev





--
Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Greg Troxel]

Sorry,  I didn't read the first message in it entirety before digging in
to the thread.

Reading over the proposed list of removals, nothing jumps out at me as
problematic.



signature.asc
Description: PGP signature
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[David Strip]

  
  
Bearing in mind that I use none of the drivers on Even's list, I
find his suggestion and reasoning compelling. I especially agree
with his comment that the only way to get anyone's attention is to
break their workflow, if only temporarily. The main risk here is
that a program that uses gdal (eg, QGIS) might hide this from it's
users by setting the options in code. Of course, when it truly
breaks then this program will have deal with unhappy users, so the
burden will be on them, not on gdal devs (we can only hope). 

As to the "cemetery" as Even calls it, this is in line with my
thinking before I saw Even's message. GIT maintains history, so
anyone wanting an old driver can always revert back to older
versions (at the cost of losing new capabilities.) I would consider
that instead of just noting in this in the docs somewhere, an
attempt to load a removed driver will result in a message that says
"This driver was removed in GIT update " to make it easier to
track down. A  variation on this that would require a little more
work is to replace each removed driver with a stub that prints an
appropriate failure message. In most cases, this would be the same
"this was removed" message, but in the rare case that someone else
picks up maintenance of the driver, the message could be something
like  "This driver is independently maintained at ". 
  

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Greg Troxel]

Even Rouault  writes:

> - Where are the costs in keeping these drivers ?
>* monetary: there is one, but I'm unable to quantify it
>* environmental: burn CPU cycles each time someone does a GDAL build
>* psychological: prevent developers from doing modernization in GDAL 
> internals. When you know you have to change 250 drivers, you think twice 
> before doing the change, and generally you conclude it is not worth it, or 
> fallback to hacks to limit the amount of code change. We should probably trim 
> down the list to 20 raster and vector drivers to have a real effect regarding 
> that. For a next time :-)

It would probably help anchor the discussion for someone to send a list
of drivers with remove/keep as a straw proposal, not taken super
seriously, just to see roughly where we are.   Personally, I only use
maintstream things so am not too likely to be affected, but I don't have
a good grasp of the scope of the proposal.


signature.asc
Description: PGP signature
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Even Rouault]

Hi,

trying to answer the different points raised up to now:

- SVG: let's keep it as it is used. This is exactly the feedback I'm seeking 
for. I had developed this as a toy, crazy me, I won't do it anymore. No idea 
anyone was using it actually.

- making those driver plugins. This would require significant work, and the 
purpose is to save work. Our current build infrastructure is not ready for 
easy plugification. And announcing they will be unmaintained is probably not 
efficient to know in advance the impact of the planned breakage. People don't 
read documentation. The only way to force people to react is to break them in 
some way.

- regarding Python, I'm not sure what the question is exactly. If it was how 
to still enable the drivers candidate for retirement to work, then it is just 
gdal.SetConfigOption('GDAL_ENABLE_DRIVER_FOO', 'YES')

- once we have decided which drivers should be retired, I don't find moving 
them to some cemetery repository very useful. Because that would lack part of 
the build scripts. What would be more useful is to add a paragraph in the 
documentation that drivers FOO, BAR, BAZ were retired in GDAL 3.5. That way 
people know they have to download a GDAL tarball or checkout a git tag before 
that release, or download a past OSGeo-Live VM, or Conda package, etc... And 
they will get (hopefully) functional code. Much better than the cemetery 
approach.

- regarding schedule:
   * GDAL 3.3, May 2021: version with drivers semi-retired
   * GDAL 3.4, November 2021: still with drivers semi-retired
   * GDAL 3.5, May 2022: retired drivers are gone
 So we won't get much feedback from Ubuntu LTS april 2022, as at that date, 
the drivers will have to be retired for the 3.5 release.

- Where are the costs in keeping these drivers ?
   * monetary: there is one, but I'm unable to quantify it
   * environmental: burn CPU cycles each time someone does a GDAL build
   * psychological: prevent developers from doing modernization in GDAL 
internals. When you know you have to change 250 drivers, you think twice 
before doing the change, and generally you conclude it is not worth it, or 
fallback to hacks to limit the amount of code change. We should probably trim 
down the list to 20 raster and vector drivers to have a real effect regarding 
that. For a next time :-)

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Andrew C Aitchison]

On Mon, 11 Jan 2021, Even Rouault wrote:


It's not spring yet, but I'm in a mood lately of axing useless things, and
we probably have tons of candidate for that in GDAL, especially in drivers.
I was going to just axe the DB2 driver
(https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.
   ...... 

That is, when we detect a file to be handled by the driver, emit the above
error message and do not open the dataset, unless the user defines the
environment variable.
Similarly in the Create()/CreateCopy() methods.
If we ship this in 3.3, with a 3.5 milestone for removal, this would offer a
feedback period of one year / 2 feature versions.


That sounds a reasonable timetable.

The next Ubuntu LTS is due for release April 2022, so is likely to have
GDAL 3.4 and the avoidable warning, which is good. (If it had GDAL 3.5
people moving to it from a previous LTS would not see the warning ...)

Where are the costs in keeping these drivers ?
If as Kurt describes, the costs are for the user, we already have plugins
and options to not include drivers in the main binary. We could add a 3 way
switch like the kernel: builtin/plugin/not built which would make things
simple for packagers and users.

I am assuming that these are drivers that don't get much maintenance
at the moment, so the gain for GDAL developers is not having to add
new GDAL features to so many drivers ?

--
Andrew C. Aitchison Kendal, UK
and...@aitchison.me.uk
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Kurt Schwehr]

I am definitely for removing unused things that talk to servers or use
binary drivers.

I can see wanting to keep drivers that don't interact with servers for
posterity, but I think that cost is starting to get high.  A system like
you describe with a good lead time sounds good.

In the Google setup, I switched to using the set of minimum viable drivers
for business needs in our internal build.  The cost of having other drivers
was just too much. We have issues with custom filesystem support, tracking
of things that go through sandbox boundaries, and maximum binary size
(nothing like having your compiler bail from an OOM).


On Sun, Jan 10, 2021 at 3:02 PM Even Rouault 
wrote:

> Hi,
>
> It's not spring yet, but I'm in a mood lately of axing useless things, and
> we
> probably have tons of candidate for that in GDAL, especially in drivers.
> I was going to just axe the DB2 driver
> (https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.
>
> Any idea how we can know what is used and what isn't ? A "call-home"
> functionality where we would track driver usage would only be acceptable
> if
> people enable it and have network connectivity, so we won't probably get
> lots
> of feedback. Having a spreadsheet with the driver list and asking people
> to
> fill it would probably also receive little feedback. So the idea I had was
> to
> do something like the following in the Open() method of a candidate for
> removal:
>
> GDALDataset* FooDriver::Open(  )
> {
>if( !Identify(poOpenInfo) )
>   return nullptr;
>
>if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
>{
>CPLError(CE_Failure, CPLE_AppDefined,
> "Driver FOO is considered for removal in GDAL 3.5. You are invited "
> "to convert any dataset in that format to another more common one ."
> "If you need this driver in future GDAL versions, create a ticket at "
> "https://github.com/OSGeo/gdal (look first for an existing one first)
> to "
> "explain how critical it is for you (but the GDAL project may still "
> "remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
> "configuration option / environment variable to YES");
>return nullptr;
> }
> ...
> }
>
> That is, when we detect a file to be handled by the driver, emit the above
> error message and do not open the dataset, unless the user defines the
> environment variable.
> Similarly in the Create()/CreateCopy() methods.
> If we ship this in 3.3, with a 3.5 milestone for removal, this would offer
> a
> feedback period of one year / 2 feature versions.
>
> Here's my own list of candidates for retirement (probably
> over-conservative).
> Mostly based on gut feeling. None of them are particularly bad citizens,
> but I
> have no indication that they are still used, which doesn't mean they
> aren't.
>
> * Raster side:
> BPG
> DB2Raster
> DOQ1
> DOQ2
> E00GRID
> Epsilon
> FujiBAS
> GS7BG
> GSAG
> IDA
> JDEM
> JPEG2000 (Jasper): JP2OpenJPEG is a better replacement
> JPEGLS
> LAN
> MFF
> MG4Lidar ?
> NDF
> NTv1
> SDTS Raster
> SGI
> XPM
> ZMap
>
> * Vector side:
> AERONAVFAA
> ESRI ArcObjects
> ARCGEN
> BNA
> Cloudant
> CouchDB
> DB2
> DODS
> FMEObjects Gateway
> Geomedia MDB
> GMT ASCII Vectors
> GTM
> HTF
> INGRES
> MongoDB (the old one, superseded by MongoDBv3)
> OpenAIR
> REC
> SDTS
> SUA
> SVG
> TIGER
> WALK
>
>
> Anything you'd add / remove ?
>
> What is not obvious is what would be the criterion for keeping a driver:
> 1,
> 10, 100 users asking for the driver to be kept ?
> If a GDAL developer contributing to the overall good of the project needs
> the
> preservation of a driver to be able to justify its continued involvement,
> I'd
> tend to think it to be enough to keep it.
>
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Jan Heckman]

Hey guys, I was just typing pretty much the same thing.
So these drivers could be removed gracefully with a trace allowing
gis-archeologists/archivists to use them.
I can just see the papers in my imagination.
Jan

On Mon, Jan 11, 2021 at 1:03 AM Stephen Woodbridge <
stephenwoodbridg...@gmail.com> wrote:

> Even,
>
> This makes a lot of sense to me. How would you handle this in Python?
> Would it make sense to create a GDAL-removed repository and move stuff
> into it just so it is available if someone wants it. This would not be
> supported or updated by GDAL just making it available if someone
> wants/needs to fork it, something else like this?
>
> -Steve W
>
> On 1/10/2021 6:02 PM, Even Rouault wrote:
> > Hi,
> >
> > It's not spring yet, but I'm in a mood lately of axing useless things,
> and we
> > probably have tons of candidate for that in GDAL, especially in drivers.
> > I was going to just axe the DB2 driver
> > (https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.
> >
> > Any idea how we can know what is used and what isn't ? A "call-home"
> > functionality where we would track driver usage would only be acceptable
> if
> > people enable it and have network connectivity, so we won't probably get
> lots
> > of feedback. Having a spreadsheet with the driver list and asking people
> to
> > fill it would probably also receive little feedback. So the idea I had
> was to
> > do something like the following in the Open() method of a candidate for
> > removal:
> >
> > GDALDataset* FooDriver::Open(  )
> > {
> > if( !Identify(poOpenInfo) )
> >return nullptr;
> >
> > if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
> > {
> > CPLError(CE_Failure, CPLE_AppDefined,
> >  "Driver FOO is considered for removal in GDAL 3.5. You are invited "
> >  "to convert any dataset in that format to another more common one ."
> >  "If you need this driver in future GDAL versions, create a ticket
> at "
> >  "https://github.com/OSGeo/gdal (look first for an existing one
> first) to "
> >  "explain how critical it is for you (but the GDAL project may still
> "
> >  "remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
> >  "configuration option / environment variable to YES");
> > return nullptr;
> >  }
> >  ...
> > }
> >
> > That is, when we detect a file to be handled by the driver, emit the
> above
> > error message and do not open the dataset, unless the user defines the
> > environment variable.
> > Similarly in the Create()/CreateCopy() methods.
> > If we ship this in 3.3, with a 3.5 milestone for removal, this would
> offer a
> > feedback period of one year / 2 feature versions.
> >
> > Here's my own list of candidates for retirement (probably
> over-conservative).
> > Mostly based on gut feeling. None of them are particularly bad citizens,
> but I
> > have no indication that they are still used, which doesn't mean they
> aren't.
> >
> > * Raster side:
> > BPG
> > DB2Raster
> > DOQ1
> > DOQ2
> > E00GRID
> > Epsilon
> > FujiBAS
> > GS7BG
> > GSAG
> > IDA
> > JDEM
> > JPEG2000 (Jasper): JP2OpenJPEG is a better replacement
> > JPEGLS
> > LAN
> > MFF
> > MG4Lidar ?
> > NDF
> > NTv1
> > SDTS Raster
> > SGI
> > XPM
> > ZMap
> >
> > * Vector side:
> > AERONAVFAA
> > ESRI ArcObjects
> > ARCGEN
> > BNA
> > Cloudant
> > CouchDB
> > DB2
> > DODS
> > FMEObjects Gateway
> > Geomedia MDB
> > GMT ASCII Vectors
> > GTM
> > HTF
> > INGRES
> > MongoDB (the old one, superseded by MongoDBv3)
> > OpenAIR
> > REC
> > SDTS
> > SUA
> > SVG
> > TIGER
> > WALK
> >
> >
> > Anything you'd add / remove ?
> >
> > What is not obvious is what would be the criterion for keeping a driver:
> 1,
> > 10, 100 users asking for the driver to be kept ?
> > If a GDAL developer contributing to the overall good of the project
> needs the
> > preservation of a driver to be able to justify its continued
> involvement, I'd
> > tend to think it to be enough to keep it.
> >
> >
> > Even
> >
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Stephen Woodbridge]

Even,

This makes a lot of sense to me. How would you handle this in Python?
Would it make sense to create a GDAL-removed repository and move stuff 
into it just so it is available if someone wants it. This would not be 
supported or updated by GDAL just making it available if someone 
wants/needs to fork it, something else like this?


-Steve W

On 1/10/2021 6:02 PM, Even Rouault wrote:

Hi,

It's not spring yet, but I'm in a mood lately of axing useless things, and we
probably have tons of candidate for that in GDAL, especially in drivers.
I was going to just axe the DB2 driver
(https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.

Any idea how we can know what is used and what isn't ? A "call-home"
functionality where we would track driver usage would only be acceptable if
people enable it and have network connectivity, so we won't probably get lots
of feedback. Having a spreadsheet with the driver list and asking people to
fill it would probably also receive little feedback. So the idea I had was to
do something like the following in the Open() method of a candidate for
removal:

GDALDataset* FooDriver::Open(  )
{
if( !Identify(poOpenInfo) )
   return nullptr;

if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
{
CPLError(CE_Failure, CPLE_AppDefined,
 "Driver FOO is considered for removal in GDAL 3.5. You are invited "
 "to convert any dataset in that format to another more common one ."
 "If you need this driver in future GDAL versions, create a ticket at "
 "https://github.com/OSGeo/gdal (look first for an existing one first) to "
 "explain how critical it is for you (but the GDAL project may still "
 "remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
 "configuration option / environment variable to YES");
return nullptr;
 }
 ...
}

That is, when we detect a file to be handled by the driver, emit the above
error message and do not open the dataset, unless the user defines the
environment variable.
Similarly in the Create()/CreateCopy() methods.
If we ship this in 3.3, with a 3.5 milestone for removal, this would offer a
feedback period of one year / 2 feature versions.

Here's my own list of candidates for retirement (probably over-conservative).
Mostly based on gut feeling. None of them are particularly bad citizens, but I
have no indication that they are still used, which doesn't mean they aren't.

* Raster side:
BPG
DB2Raster
DOQ1
DOQ2
E00GRID
Epsilon
FujiBAS
GS7BG
GSAG
IDA
JDEM
JPEG2000 (Jasper): JP2OpenJPEG is a better replacement
JPEGLS
LAN
MFF
MG4Lidar ?
NDF
NTv1
SDTS Raster
SGI
XPM
ZMap

* Vector side:
AERONAVFAA
ESRI ArcObjects
ARCGEN
BNA
Cloudant
CouchDB
DB2
DODS
FMEObjects Gateway
Geomedia MDB
GMT ASCII Vectors
GTM
HTF
INGRES
MongoDB (the old one, superseded by MongoDBv3)
OpenAIR
REC
SDTS
SUA
SVG
TIGER
WALK


Anything you'd add / remove ?

What is not obvious is what would be the criterion for keeping a driver: 1,
10, 100 users asking for the driver to be kept ?
If a GDAL developer contributing to the overall good of the project needs the
preservation of a driver to be able to justify its continued involvement, I'd
tend to think it to be enough to keep it.


Even



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Andrew Bell]

Could you simply make them all plugins and announce that they will be
unmaintained? After a release cycle, we place such things into their own
repo. Don't know if that would work for GDAL.

On Sun, Jan 10, 2021, 6:02 PM Even Rouault 
wrote:

> Hi,
>
> It's not spring yet, but I'm in a mood lately of axing useless things, and
> we
> probably have tons of candidate for that in GDAL, especially in drivers.
> I was going to just axe the DB2 driver
> (https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.
>
> Any idea how we can know what is used and what isn't ? A "call-home"
> functionality where we would track driver usage would only be acceptable
> if
> people enable it and have network connectivity, so we won't probably get
> lots
> of feedback. Having a spreadsheet with the driver list and asking people
> to
> fill it would probably also receive little feedback. So the idea I had was
> to
> do something like the following in the Open() method of a candidate for
> removal:
>
> GDALDataset* FooDriver::Open(  )
> {
>if( !Identify(poOpenInfo) )
>   return nullptr;
>
>if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
>{
>CPLError(CE_Failure, CPLE_AppDefined,
> "Driver FOO is considered for removal in GDAL 3.5. You are invited "
> "to convert any dataset in that format to another more common one ."
> "If you need this driver in future GDAL versions, create a ticket at "
> "https://github.com/OSGeo/gdal (look first for an existing one first)
> to "
> "explain how critical it is for you (but the GDAL project may still "
> "remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
> "configuration option / environment variable to YES");
>return nullptr;
> }
> ...
> }
>
> That is, when we detect a file to be handled by the driver, emit the above
> error message and do not open the dataset, unless the user defines the
> environment variable.
> Similarly in the Create()/CreateCopy() methods.
> If we ship this in 3.3, with a 3.5 milestone for removal, this would offer
> a
> feedback period of one year / 2 feature versions.
>
> Here's my own list of candidates for retirement (probably
> over-conservative).
> Mostly based on gut feeling. None of them are particularly bad citizens,
> but I
> have no indication that they are still used, which doesn't mean they
> aren't.
>
> * Raster side:
> BPG
> DB2Raster
> DOQ1
> DOQ2
> E00GRID
> Epsilon
> FujiBAS
> GS7BG
> GSAG
> IDA
> JDEM
> JPEG2000 (Jasper): JP2OpenJPEG is a better replacement
> JPEGLS
> LAN
> MFF
> MG4Lidar ?
> NDF
> NTv1
> SDTS Raster
> SGI
> XPM
> ZMap
>
> * Vector side:
> AERONAVFAA
> ESRI ArcObjects
> ARCGEN
> BNA
> Cloudant
> CouchDB
> DB2
> DODS
> FMEObjects Gateway
> Geomedia MDB
> GMT ASCII Vectors
> GTM
> HTF
> INGRES
> MongoDB (the old one, superseded by MongoDBv3)
> OpenAIR
> REC
> SDTS
> SUA
> SVG
> TIGER
> WALK
>
>
> Anything you'd add / remove ?
>
> What is not obvious is what would be the criterion for keeping a driver:
> 1,
> 10, 100 users asking for the driver to be kept ?
> If a GDAL developer contributing to the overall good of the project needs
> the
> preservation of a driver to be able to justify its continued involvement,
> I'd
> tend to think it to be enough to keep it.
>
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Nyall Dawson]

On Mon, 11 Jan 2021 at 09:50, Jan Heckman  wrote:
>
> A bit of googling sometimes gives an indication. I was a little confused 
> about SVG, not using it myself but getting quite a few references.
> E.g. this one in stackexchange, with a mention how often it was looked up:
> https://gis.stackexchange.com/questions/11476/importing-svg-into-gis
> Just an idea. Obviously, it is not easy to decide exactly what to do with 
> this sort of information.

I've previously found the SVG driver useful for getting data out of
typically non-spatial formats for georeferencing (e.g. I once had to
convert a bunch of spatial data which had been hand drawn in visio on
top of a google map screenshot...). I'd be sorry to see this driver
go, as it's a useful last-resort aid for situations like this.

Nyall

> Jan
>
> On Mon, Jan 11, 2021 at 12:02 AM Even Rouault  
> wrote:
>>
>> Hi,
>>
>> It's not spring yet, but I'm in a mood lately of axing useless things, and we
>> probably have tons of candidate for that in GDAL, especially in drivers.
>> I was going to just axe the DB2 driver
>> (https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.
>>
>> Any idea how we can know what is used and what isn't ? A "call-home"
>> functionality where we would track driver usage would only be acceptable if
>> people enable it and have network connectivity, so we won't probably get lots
>> of feedback. Having a spreadsheet with the driver list and asking people to
>> fill it would probably also receive little feedback. So the idea I had was to
>> do something like the following in the Open() method of a candidate for
>> removal:
>>
>> GDALDataset* FooDriver::Open(  )
>> {
>>if( !Identify(poOpenInfo) )
>>   return nullptr;
>>
>>if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
>>{
>>CPLError(CE_Failure, CPLE_AppDefined,
>> "Driver FOO is considered for removal in GDAL 3.5. You are invited "
>> "to convert any dataset in that format to another more common one ."
>> "If you need this driver in future GDAL versions, create a ticket at "
>> "https://github.com/OSGeo/gdal (look first for an existing one first) to 
>> "
>> "explain how critical it is for you (but the GDAL project may still "
>> "remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
>> "configuration option / environment variable to YES");
>>return nullptr;
>> }
>> ...
>> }
>>
>> That is, when we detect a file to be handled by the driver, emit the above
>> error message and do not open the dataset, unless the user defines the
>> environment variable.
>> Similarly in the Create()/CreateCopy() methods.
>> If we ship this in 3.3, with a 3.5 milestone for removal, this would offer a
>> feedback period of one year / 2 feature versions.
>>
>> Here's my own list of candidates for retirement (probably over-conservative).
>> Mostly based on gut feeling. None of them are particularly bad citizens, but 
>> I
>> have no indication that they are still used, which doesn't mean they aren't.
>>
>> * Raster side:
>> BPG
>> DB2Raster
>> DOQ1
>> DOQ2
>> E00GRID
>> Epsilon
>> FujiBAS
>> GS7BG
>> GSAG
>> IDA
>> JDEM
>> JPEG2000 (Jasper): JP2OpenJPEG is a better replacement
>> JPEGLS
>> LAN
>> MFF
>> MG4Lidar ?
>> NDF
>> NTv1
>> SDTS Raster
>> SGI
>> XPM
>> ZMap
>>
>> * Vector side:
>> AERONAVFAA
>> ESRI ArcObjects
>> ARCGEN
>> BNA
>> Cloudant
>> CouchDB
>> DB2
>> DODS
>> FMEObjects Gateway
>> Geomedia MDB
>> GMT ASCII Vectors
>> GTM
>> HTF
>> INGRES
>> MongoDB (the old one, superseded by MongoDBv3)
>> OpenAIR
>> REC
>> SDTS
>> SUA
>> SVG
>> TIGER
>> WALK
>>
>>
>> Anything you'd add / remove ?
>>
>> What is not obvious is what would be the criterion for keeping a driver: 1,
>> 10, 100 users asking for the driver to be kept ?
>> If a GDAL developer contributing to the overall good of the project needs the
>> preservation of a driver to be able to justify its continued involvement, I'd
>> tend to think it to be enough to keep it.
>>
>>
>> Even
>>
>> --
>> Spatialys - Geospatial professional services
>> http://www.spatialys.com
>> ___
>> gdal-dev mailing list
>> gdal-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Considering drivers removal ?

[Jan Heckman]

A bit of googling sometimes gives an indication. I was a little confused
about SVG, not using it myself but getting quite a few references.
E.g. this one in stackexchange, with a mention how often it was looked up:
https://gis.stackexchange.com/questions/11476/importing-svg-into-gis
Just an idea. Obviously, it is not easy to decide exactly what to do with
this sort of information.
Jan

On Mon, Jan 11, 2021 at 12:02 AM Even Rouault 
wrote:

> Hi,
>
> It's not spring yet, but I'm in a mood lately of axing useless things, and
> we
> probably have tons of candidate for that in GDAL, especially in drivers.
> I was going to just axe the DB2 driver
> (https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.
>
> Any idea how we can know what is used and what isn't ? A "call-home"
> functionality where we would track driver usage would only be acceptable
> if
> people enable it and have network connectivity, so we won't probably get
> lots
> of feedback. Having a spreadsheet with the driver list and asking people
> to
> fill it would probably also receive little feedback. So the idea I had was
> to
> do something like the following in the Open() method of a candidate for
> removal:
>
> GDALDataset* FooDriver::Open(  )
> {
>if( !Identify(poOpenInfo) )
>   return nullptr;
>
>if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
>{
>CPLError(CE_Failure, CPLE_AppDefined,
> "Driver FOO is considered for removal in GDAL 3.5. You are invited "
> "to convert any dataset in that format to another more common one ."
> "If you need this driver in future GDAL versions, create a ticket at "
> "https://github.com/OSGeo/gdal (look first for an existing one first)
> to "
> "explain how critical it is for you (but the GDAL project may still "
> "remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
> "configuration option / environment variable to YES");
>return nullptr;
> }
> ...
> }
>
> That is, when we detect a file to be handled by the driver, emit the above
> error message and do not open the dataset, unless the user defines the
> environment variable.
> Similarly in the Create()/CreateCopy() methods.
> If we ship this in 3.3, with a 3.5 milestone for removal, this would offer
> a
> feedback period of one year / 2 feature versions.
>
> Here's my own list of candidates for retirement (probably
> over-conservative).
> Mostly based on gut feeling. None of them are particularly bad citizens,
> but I
> have no indication that they are still used, which doesn't mean they
> aren't.
>
> * Raster side:
> BPG
> DB2Raster
> DOQ1
> DOQ2
> E00GRID
> Epsilon
> FujiBAS
> GS7BG
> GSAG
> IDA
> JDEM
> JPEG2000 (Jasper): JP2OpenJPEG is a better replacement
> JPEGLS
> LAN
> MFF
> MG4Lidar ?
> NDF
> NTv1
> SDTS Raster
> SGI
> XPM
> ZMap
>
> * Vector side:
> AERONAVFAA
> ESRI ArcObjects
> ARCGEN
> BNA
> Cloudant
> CouchDB
> DB2
> DODS
> FMEObjects Gateway
> Geomedia MDB
> GMT ASCII Vectors
> GTM
> HTF
> INGRES
> MongoDB (the old one, superseded by MongoDBv3)
> OpenAIR
> REC
> SDTS
> SUA
> SVG
> TIGER
> WALK
>
>
> Anything you'd add / remove ?
>
> What is not obvious is what would be the criterion for keeping a driver:
> 1,
> 10, 100 users asking for the driver to be kept ?
> If a GDAL developer contributing to the overall good of the project needs
> the
> preservation of a driver to be able to justify its continued involvement,
> I'd
> tend to think it to be enough to keep it.
>
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Considering drivers removal ?

[Even Rouault]

Hi,

It's not spring yet, but I'm in a mood lately of axing useless things, and we 
probably have tons of candidate for that in GDAL, especially in drivers.
I was going to just axe the DB2 driver
(https://github.com/OSGeo/gdal/pull/3366) but the issue is more general.

Any idea how we can know what is used and what isn't ? A "call-home" 
functionality where we would track driver usage would only be acceptable if 
people enable it and have network connectivity, so we won't probably get lots 
of feedback. Having a spreadsheet with the driver list and asking people to 
fill it would probably also receive little feedback. So the idea I had was to 
do something like the following in the Open() method of a candidate for 
removal:

GDALDataset* FooDriver::Open(  )
{
   if( !Identify(poOpenInfo) )
  return nullptr;

   if( !CPLTestBool(CPLGetConfigOption("GDAL_ENABLE_DRIVER_FOO", "NO") )
   {
   CPLError(CE_Failure, CPLE_AppDefined,
"Driver FOO is considered for removal in GDAL 3.5. You are invited "
"to convert any dataset in that format to another more common one ."
"If you need this driver in future GDAL versions, create a ticket at "
"https://github.com/OSGeo/gdal (look first for an existing one first) to "
"explain how critical it is for you (but the GDAL project may still "
"remove it), and to enable it now, set the GDAL_ENABLE_DRIVER_FOO "
"configuration option / environment variable to YES");
   return nullptr;
}
...
}

That is, when we detect a file to be handled by the driver, emit the above 
error message and do not open the dataset, unless the user defines the 
environment variable.
Similarly in the Create()/CreateCopy() methods.
If we ship this in 3.3, with a 3.5 milestone for removal, this would offer a 
feedback period of one year / 2 feature versions.

Here's my own list of candidates for retirement (probably over-conservative). 
Mostly based on gut feeling. None of them are particularly bad citizens, but I 
have no indication that they are still used, which doesn't mean they aren't.

* Raster side:
BPG
DB2Raster
DOQ1
DOQ2
E00GRID
Epsilon
FujiBAS
GS7BG
GSAG
IDA
JDEM
JPEG2000 (Jasper): JP2OpenJPEG is a better replacement
JPEGLS
LAN
MFF
MG4Lidar ?
NDF
NTv1
SDTS Raster
SGI
XPM
ZMap

* Vector side:
AERONAVFAA
ESRI ArcObjects
ARCGEN
BNA
Cloudant
CouchDB
DB2
DODS
FMEObjects Gateway
Geomedia MDB
GMT ASCII Vectors
GTM
HTF
INGRES
MongoDB (the old one, superseded by MongoDBv3)
OpenAIR
REC
SDTS
SUA
SVG
TIGER
WALK


Anything you'd add / remove ?

What is not obvious is what would be the criterion for keeping a driver: 1, 
10, 100 users asking for the driver to be kept ?
If a GDAL developer contributing to the overall good of the project needs the 
preservation of a driver to be able to justify its continued involvement, I'd 
tend to think it to be enough to keep it.


Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev