Re: [gdal-dev] Under what (if any) conditions would GDALWriteBlock modify data buffer?
On Mon, 18 Dec 2023, Even Rouault wrote: Le 18/12/2023 à 21:15, Andrew C Aitchison a écrit : On Mon, 18 Dec 2023, Even Rouault via gdal-dev wrote: Le 18/12/2023 à 16:18, Andrew C Aitchison via gdal-dev a écrit : On Mon, 18 Dec 2023, Even Rouault via gdal-dev wrote: Hi, interesting question. No easy answer as it is highly driver dependent. I believe that all drivers make sure that the content of the buffer before and after the call is the same, but some drivers might temporarily modify it, to do byte swapping. For example the HFA driver does that when run on big-endian hosts for non-Byte data type. I wouldn't exclude that for formats with MSB-byte ordering, a similar situation would happen for little endian hosts. So it is definitely not safe to use WriteBlock() with a buffer that would come from a read-only section of the calling program. Doc updated to reflect that in https://github.com/OSGeo/gdal/commit/ea321723dfc69ef3a422b1e3fe4dc9ee0832861d Did you mean to say Note that even with eRWFlag==GF_Write, the content of the buffer might be temporarily modified during the execution of this method (and eventually restored back to its original content), so it is not safe to use a buffer stored in a read-only section of the calling program. Yes I meant that. I've less evidence in the RasterIO(GF_Write, ...) case than in the WriteBlock() case, but without checking all drivers, it is more prudent to assume that the buffer might be touched during RasterIO(GF_Write) by some drivers. In that case I don't understand Note that even with eRWFlag==GF_Write... It suggests to me that the buffer may be modified for eRWFlag==GF_Write *as well as* for eRWFlag==GF_Read (which implies that it is *more* likely to happen for GF_Read) ? Ah, perhaps non-idiomatic wording from mine. I struggle to make that clearer: if you've better wording to suggest, that's welcome Ah. I had misunderstood. I thought we were reading to or writing from *the buffer* not the raster. The text is fine. -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Under what (if any) conditions would GDALWriteBlock modify data buffer?
On Mon, 18 Dec 2023, Even Rouault via gdal-dev wrote: Le 18/12/2023 à 16:18, Andrew C Aitchison via gdal-dev a écrit : On Mon, 18 Dec 2023, Even Rouault via gdal-dev wrote: Hi, interesting question. No easy answer as it is highly driver dependent. I believe that all drivers make sure that the content of the buffer before and after the call is the same, but some drivers might temporarily modify it, to do byte swapping. For example the HFA driver does that when run on big-endian hosts for non-Byte data type. I wouldn't exclude that for formats with MSB-byte ordering, a similar situation would happen for little endian hosts. So it is definitely not safe to use WriteBlock() with a buffer that would come from a read-only section of the calling program. Doc updated to reflect that in https://github.com/OSGeo/gdal/commit/ea321723dfc69ef3a422b1e3fe4dc9ee0832861d Did you mean to say Note that even with eRWFlag==GF_Write, the content of the buffer might be temporarily modified during the execution of this method (and eventually restored back to its original content), so it is not safe to use a buffer stored in a read-only section of the calling program. Yes I meant that. I've less evidence in the RasterIO(GF_Write, ...) case than in the WriteBlock() case, but without checking all drivers, it is more prudent to assume that the buffer might be touched during RasterIO(GF_Write) by some drivers. In that case I don't understand Note that even with eRWFlag==GF_Write... It suggests to me that the buffer may be modified for eRWFlag==GF_Write *as well as* for eRWFlag==GF_Read (which implies that it is *more* likely to happen for GF_Read) ? Sorry to be confused. -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Under what (if any) conditions would GDALWriteBlock modify data buffer?
Le 18/12/2023 à 21:15, Andrew C Aitchison a écrit : On Mon, 18 Dec 2023, Even Rouault via gdal-dev wrote: Le 18/12/2023 à 16:18, Andrew C Aitchison via gdal-dev a écrit : On Mon, 18 Dec 2023, Even Rouault via gdal-dev wrote: Hi, interesting question. No easy answer as it is highly driver dependent. I believe that all drivers make sure that the content of the buffer before and after the call is the same, but some drivers might temporarily modify it, to do byte swapping. For example the HFA driver does that when run on big-endian hosts for non-Byte data type. I wouldn't exclude that for formats with MSB-byte ordering, a similar situation would happen for little endian hosts. So it is definitely not safe to use WriteBlock() with a buffer that would come from a read-only section of the calling program. Doc updated to reflect that in https://github.com/OSGeo/gdal/commit/ea321723dfc69ef3a422b1e3fe4dc9ee0832861d Did you mean to say Note that even with eRWFlag==GF_Write, the content of the buffer might be temporarily modified during the execution of this method (and eventually restored back to its original content), so it is not safe to use a buffer stored in a read-only section of the calling program. Yes I meant that. I've less evidence in the RasterIO(GF_Write, ...) case than in the WriteBlock() case, but without checking all drivers, it is more prudent to assume that the buffer might be touched during RasterIO(GF_Write) by some drivers. In that case I don't understand Note that even with eRWFlag==GF_Write... It suggests to me that the buffer may be modified for eRWFlag==GF_Write *as well as* for eRWFlag==GF_Read (which implies that it is *more* likely to happen for GF_Read) ? Ah, perhaps non-idiomatic wording from mine. I struggle to make that clearer: if you've better wording to suggest, that's welcome In the GF_Read case, modifying the buffer is obviously expected since it is then an output buffer, whereas in the GF_Write case it is a input buffer, and thus users could legitimately expect it to remain fully untouched, which isn't the case. -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Under what (if any) conditions would GDALWriteBlock modify data buffer?
Thanks very much for the help on this! Greatly appreciated! On Mon, Dec 18, 2023 at 10:38 AM Even Rouault via gdal-dev < gdal-dev@lists.osgeo.org> wrote: > > Le 18/12/2023 à 16:18, Andrew C Aitchison via gdal-dev a écrit : > > On Mon, 18 Dec 2023, Even Rouault via gdal-dev wrote: > > > >> Hi, > >> > >> interesting question. No easy answer as it is highly driver > >> dependent. I believe that all drivers make sure that the content of > >> the buffer before and after the call is the same, but some drivers > >> might temporarily modify it, to do byte swapping. For example the HFA > >> driver does that when run on big-endian hosts for non-Byte data type. > >> I wouldn't exclude that for formats with MSB-byte ordering, a similar > >> situation would happen for little endian hosts. So it is definitely > >> not safe to use WriteBlock() with a buffer that would come from a > >> read-only section of the calling program. Doc updated to reflect that > >> in > >> > https://github.com/OSGeo/gdal/commit/ea321723dfc69ef3a422b1e3fe4dc9ee0832861d > > > > Did you mean to say > > Note that even with eRWFlag==GF_Write, the content of the buffer > > might be temporarily modified during the execution of this method > > (and eventually restored back to its original content), so it is not > > safe to use a buffer stored in a read-only section of the calling > > program. > > Yes I meant that. I've less evidence in the RasterIO(GF_Write, ...) case > than in the WriteBlock() case, but without checking all drivers, it is > more prudent to assume that the buffer might be touched during > RasterIO(GF_Write) by some drivers. > > > -- > http://www.spatialys.com > My software is free, but my time generally not. > > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > -- The content of this email is intended for the person or entity to which it is addressed only. This email may contain confidential information. If you are not the person to whom this message is addressed, be aware that any use, reproduction, or distribution of this message is strictly prohibited. If you received this in error, please contact the sender and immediately delete this email and any attachments. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Under what (if any) conditions would GDALWriteBlock modify data buffer?
Le 18/12/2023 à 16:18, Andrew C Aitchison via gdal-dev a écrit : On Mon, 18 Dec 2023, Even Rouault via gdal-dev wrote: Hi, interesting question. No easy answer as it is highly driver dependent. I believe that all drivers make sure that the content of the buffer before and after the call is the same, but some drivers might temporarily modify it, to do byte swapping. For example the HFA driver does that when run on big-endian hosts for non-Byte data type. I wouldn't exclude that for formats with MSB-byte ordering, a similar situation would happen for little endian hosts. So it is definitely not safe to use WriteBlock() with a buffer that would come from a read-only section of the calling program. Doc updated to reflect that in https://github.com/OSGeo/gdal/commit/ea321723dfc69ef3a422b1e3fe4dc9ee0832861d Did you mean to say Note that even with eRWFlag==GF_Write, the content of the buffer might be temporarily modified during the execution of this method (and eventually restored back to its original content), so it is not safe to use a buffer stored in a read-only section of the calling program. Yes I meant that. I've less evidence in the RasterIO(GF_Write, ...) case than in the WriteBlock() case, but without checking all drivers, it is more prudent to assume that the buffer might be touched during RasterIO(GF_Write) by some drivers. -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Under what (if any) conditions would GDALWriteBlock modify data buffer?
On Mon, 18 Dec 2023, Even Rouault via gdal-dev wrote: Hi, interesting question. No easy answer as it is highly driver dependent. I believe that all drivers make sure that the content of the buffer before and after the call is the same, but some drivers might temporarily modify it, to do byte swapping. For example the HFA driver does that when run on big-endian hosts for non-Byte data type. I wouldn't exclude that for formats with MSB-byte ordering, a similar situation would happen for little endian hosts. So it is definitely not safe to use WriteBlock() with a buffer that would come from a read-only section of the calling program. Doc updated to reflect that in https://github.com/OSGeo/gdal/commit/ea321723dfc69ef3a422b1e3fe4dc9ee0832861d Did you mean to say Note that even with eRWFlag==GF_Write, the content of the buffer might be temporarily modified during the execution of this method (and eventually restored back to its original content), so it is not safe to use a buffer stored in a read-only section of the calling program. or Note that even with eRWFlag==GF_Read, ... -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Under what (if any) conditions would GDALWriteBlock modify data buffer?
Hi, interesting question. No easy answer as it is highly driver dependent. I believe that all drivers make sure that the content of the buffer before and after the call is the same, but some drivers might temporarily modify it, to do byte swapping. For example the HFA driver does that when run on big-endian hosts for non-Byte data type. I wouldn't exclude that for formats with MSB-byte ordering, a similar situation would happen for little endian hosts. So it is definitely not safe to use WriteBlock() with a buffer that would come from a read-only section of the calling program. Doc updated to reflect that in https://github.com/OSGeo/gdal/commit/ea321723dfc69ef3a422b1e3fe4dc9ee0832861d Even Le 17/12/2023 à 23:09, Fitch, Simeon via gdal-dev a écrit : The last argument to function `CPLErr GDALWriteBlock(GDALRasterBandH, int, int, void*)`, pointing to the block array, is not `const`. I can't find anything in the documentation discussing ownership, etc. but need to know what kind of invariants can be assumed here. Are there (driver dependent?) circumstances where `GDALWriteBlock` will mutate the given array, or can callers assume it's effectively `const void*`? Thanks, Simeon The content of this email is intended for the person or entity to which it is addressed only. This email may contain confidential information. If you are not the person to whom this message is addressed, be aware that any use, reproduction, or distribution of this message is strictly prohibited. If you received this in error, please contact the sender and immediately delete this email and any attachments. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Under what (if any) conditions would GDALWriteBlock modify data buffer?
I don't think it does, but some parts (the older ones) of the public GDAL API don't use const. I admit I didn't check the implementation this time though. Laurentiu On Mon, Dec 18, 2023, at 00:09, Fitch, Simeon via gdal-dev wrote: > The last argument to function `CPLErr GDALWriteBlock(GDALRasterBandH, int, > int, void*)`, pointing to the block array, is not `const`. I can't find > anything in the documentation discussing ownership, etc. but need to know > what kind of invariants can be assumed here. > > Are there (driver dependent?) circumstances where `GDALWriteBlock` will > mutate the given array, or can callers assume it's effectively `const void*`? > > Thanks, > > Simeon > > The content of this email is intended for the person or entity to which it is > addressed only. This email may contain confidential information. If you are > not the person to whom this message is addressed, be aware that any use, > reproduction, or distribution of this message is strictly prohibited. If you > received this in error, please contact the sender and immediately delete this > email and any attachments. > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
