Re: [Qgis-developer] Processing with raster files > 4GB

[Paolo Cavallini]

Hi all.

Il 22/11/2014 14:28, Giovanni Manghi ha scritto:
> yes I understand, but we have a clear issue. The qgis raster calc is
> broken and when it works it need a very cumbersome logic (especially
> for common tasks like reclassifying). GRASS r.reclass works fine, but
> it does not allow to enter interactively the reclass rules (that can
> be an advantage in some situations, but usually for quick
> reclassifications is not), so adding gdal raster calc seems anyway a
> good idea.

sorry I was unclear: I meant I do not see the point in maintaining a qgis raster
calc, given the problems you refer, and the availability of several alternatives
(also the saga one).
I agree that adding gdal raster calc is a plus, especially because it requires
minimal effort compared to fixing and maintaining qgis one.
All the best.

-- 
Paolo Cavallini - www.faunalia.eu
Corsi QGIS e PostGIS: http://www.faunalia.eu/training.html
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Giovanni Manghi]

> AFAIK Processing does not import the raster, but uses it as an external data 
> source
> (if not, it should).

yes you are right, it uses r.external.



>> I agree anyway that it too bad that the qgis raster calculator (and
>> georeferencer) are drawing so little attention. Maybe there isn't much
>> people that uses them?
>
> quite possible; however, maintaining multiple calcs seems more of a burden 
> than an
> advantage.


yes I understand, but we have a clear issue. The qgis raster calc is
broken and when it works it need a very cumbersome logic (especially
for common tasks like reclassifying). GRASS r.reclass works fine, but
it does not allow to enter interactively the reclass rules (that can
be an advantage in some situations, but usually for quick
reclassifications is not), so adding gdal raster calc seems anyway a
good idea.

cheers!
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Paolo Cavallini]

Il 21/11/2014 15:13, Giovanni Manghi ha scritto:

> grass of course does the job, especially when you use the grass data
> model from the cli, native grass gui or qgis plugin. Inside processing
> large raster suffer from the overhead that is needed to import/export
> the raster to, and out of, a temp location.

AFAIK Processing does not import the raster, but uses it as an external data 
source
(if not, it should).

> I agree anyway that it too bad that the qgis raster calculator (and
> georeferencer) are drawing so little attention. Maybe there isn't much
> people that uses them?

quite possible; however, maintaining multiple calcs seems more of a burden than 
an
advantage.

all the best.
-- 
Paolo Cavallini - www.faunalia.eu
Corsi QGIS e PostGIS: http://www.faunalia.eu/training.html
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Giovanni Manghi]

Hi Andreas,

> It may also be an issue with SAGA or GRASS directly.

I have also tested a very large raster (8gb) and SAGA seems to choke
also when used natively from the cli interface.

grass of course does the job, especially when you use the grass data
model from the cli, native grass gui or qgis plugin. Inside processing
large raster suffer from the overhead that is needed to import/export
the raster to, and out of, a temp location.

gdal does the reclass job very quickly and I should be able to add the
tool to processing.

I agree anyway that it too bad that the qgis raster calculator (and
georeferencer) are drawing so little attention. Maybe there isn't much
people that uses them?

cheers!

-- G --
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Andreas Neumann]

Hi,

I was able to do my color replacements with gdal_calc.py. It works fast 
and efficient.


Here is my syntax:

gdal_calc.py -A image.tif --outfile=image_output.tif 
--calc="(A==249)*255+(A!=249)*A"


Syntax looks similar to the QGIS raster calculator - but as opposed to 
the QGIS raster calculator it actually works!


I will still continue to find out about SAGA. Maybe directly converting 
from TIFF to SAGA before processing files in SAGA.


I believe that the QGIS raster calculator would need some developer love 
- to make it more robust and easier to use ;-)


I am also pretty sure that the Processing integration of GRASS/SAGA does 
not work well with big files - at least on Windows and when TIFF files 
are used.


Thanks to all who helped!
Andreas


Am 2014-11-19 11:01, schrieb Carlos López PSIG:

Hi Andreas,

Yes, you can use gdal_calc.py
Certainly you need to perform several processes, but are very fast!

Be careful with bands!
Perhaps you will want to separate them.

Ciao,

 [3]

CARLOS LÓPEZ QUINTANILLA

www.psig.es [3]
carlos.lo...@psig.es
+34 699.680.261

2014-11-19 9:26 GMT+01:00 Andreas Neumann :


Hi Carlos,

GDAL would be my first choice - because it easy to automate.
However, I don't know how can do my task with GDAL.

From my grayscale tiff images I want to replace one pixel value
(249) with white (255). All other pixels should stay at the value
they have.

I came across https://github.com/chiatt/gdal_reclassify [1] - but
this reclassifies all pixels, not only selected ones.

Maybe I could use http://www.gdal.org/gdal_calc.html [2] with a
conditional syntax?

Thanks,
Andreas

Am 2014-11-19 09:10, schrieb Carlos López PSIG:
Hi Andreas,

I also had problems to reclassify BIGTIFF with SAGA and GRASS!
Finally I used GDAL!

Best regards,

 [2]

CARLOS LÓPEZ QUINTANILLA

www.psig.es [3] [2]

carlos.lo...@psig.es
+34 699.680.261 [4]

2014-11-19 9:05 GMT+01:00 Andreas Neumann :

Hi,

I tried various tools, e.g.

* SAGA reclassify
* GRASS r.mapcalc

Both work fine on smaller data sets, but fail on big data sets.

This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM
and 40GB disk free on C:, more free space on other disks where the
data resides.

I am not totally sure if it is the Bigtiff issue or some other
issue. It is just a guess that maybe I am hitting a limit with
tiff.
It is not fully transparent to me what Processing is doing in the
background (what file formats with what options it creates).

It may also be an issue with SAGA or GRASS directly.

Thanks if you know anything,
Andreas

Am 2014-11-18 21:15, schrieb Giovanni Manghi:

Hi,

I am having a lot of troubles with large raster files and
processing.

The intermediate results can go >4GB. Processing stops with not so
helpful error messages.

I wonder if processing is hitting the 4GB limit. Can I force
processing
to use the BIGTIFF option or a raster format that can handle >4GB
files?

Hi Andreas, what tool specifically?

cheers

-- Giovanni --
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer [5] [1]
 ___
 Qgis-developer mailing list
 Qgis-developer@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/qgis-developer [5] [1]

Links:
--
[1] http://lists.osgeo.org/mailman/listinfo/qgis-developer [5]
[2] http://www.psig.es [3]




Links:
--
[1] https://github.com/chiatt/gdal_reclassify
[2] http://www.gdal.org/gdal_calc.html
[3] http://www.psig.es
[4] tel:%2B34%20699.680.261
[5] http://lists.osgeo.org/mailman/listinfo/qgis-developer

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Paolo Cavallini]

Il 19/11/2014 11:41, Andreas Neumann ha scritto:

> I will have to do further testing about GRASS. On Windows (OSGeo4W install) 
> it does
> not work too well. It probably works better on Linux.

Why so? It shouldn't, at lest not grass64.
IMHO dealing with very large rasters is one of the strengths of the good old 
grass.
Also OTB way of handling large files is very interesting.
All the best.

-- 
Paolo Cavallini - www.faunalia.eu
Corsi QGIS e PostGIS: http://www.faunalia.eu/training.html
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[G. Allegri]

Nice to know that gdal_calc works fine ;)
I work a lot with gdal + numpy, even for custom tools, and I admit they
work very well together!


2014-11-19 11:41 GMT+01:00 Andreas Neumann :

>  Hi,
>
> I will have to do further testing about GRASS. On Windows (OSGeo4W
> install) it does not work too well. It probably works better on Linux.
>
> The gdal_calc.py solution works very fast though - glad that we came to
> this solution!
>
> Andreas
>
>
> On 19.11.2014 11:25, G. Allegri wrote:
>
> And do you have problems with GRASS (directly, not through Processing)?
> In GRASS7 you can even drectly write to Geotiff without passing through
> the internal format (
> http://grass.osgeo.org/grass70/manuals/r.external.out.html).
> If GDAL is built with bigtiff support there souldn't be problems...
>
>  giovanni
>
> 2014-11-19 10:44 GMT+01:00 Andreas Neumann :
>
>> Hi Giovanni,
>>
>> I need to change all pixels that have the value of 249 to a value of 255.
>> All other pixels should stay at their value.
>>
>> Andreas
>>
>> Am 2014-11-19 10:21, schrieb G. Allegri:
>>
>>>  From my grayscale tiff images I want to replace one pixel value

>>> (249) with white (255). All other pixels should stay at the value they
>>> have.
>>>  >
>>>   > I came across https://github.com/chiatt/gdal_reclassify [1] - but
>>> this reclassifies all pixels, not only selected ones.
>>>  >
>>>   > Maybe I could use http://www.gdal.org/gdal_calc.html [2] with a
>>> conditional syntax?
>>>
>>> Hi Andreas, do you need to change the value of one specific pixel
>>> only? I mean the pixel at [x,y] index?
>>>
>>> giovanni
>>>
>>>
   > Thanks,
>>>  > Andreas
>>>  >
>>>  > Am 2014-11-19 09:10, schrieb Carlos López PSIG:
>>>  >>
>>>  >> Hi Andreas,
>>>  >>
>>>  >> I also had problems to reclassify BIGTIFF with SAGA and GRASS!
>>>  >> Finally I used GDAL!
>>>  >>
>>>  >> Best regards,
>>>  >>
>>>  >>  [2]
>>>  >>
>>>  >> CARLOS LÓPEZ QUINTANILLA
>>>  >>
>>>   >> www.psig.es [3] [2]
>>>
>>>  >>
>>>  >> carlos.lo...@psig.es
>>>  >> +34 699.680.261
>>>  >>
>>>  >> 2014-11-19 9:05 GMT+01:00 Andreas Neumann :
>>>  >>
>>>  >>> Hi,
>>>  >>>
>>>  >>> I tried various tools, e.g.
>>>  >>>
>>>  >>> * SAGA reclassify
>>>  >>> * GRASS r.mapcalc
>>>  >>>
>>>  >>> Both work fine on smaller data sets, but fail on big data sets.
>>>  >>>
>>>  >>> This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM
>>>  >>> and 40GB disk free on C:, more free space on other disks where
>>> the
>>>  >>> data resides.
>>>  >>>
>>>  >>> I am not totally sure if it is the Bigtiff issue or some other
>>>  >>> issue. It is just a guess that maybe I am hitting a limit with
>>> tiff.
>>>  >>> It is not fully transparent to me what Processing is doing in the
>>>  >>> background (what file formats with what options it creates).
>>>  >>>
>>>  >>> It may also be an issue with SAGA or GRASS directly.
>>>  >>>
>>>  >>> Thanks if you know anything,
>>>  >>> Andreas
>>>  >>>
>>>  >>> Am 2014-11-18 21:15, schrieb Giovanni Manghi:
>>>  >>>
>>>  >>> Hi,
>>>  >>>
>>>  >>> I am having a lot of troubles with large raster files and
>>>  >>> processing.
>>>  >>>
>>>  >>> The intermediate results can go >4GB. Processing stops with not
>>> so
>>>  >>> helpful error messages.
>>>  >>>
>>>  >>> I wonder if processing is hitting the 4GB limit. Can I force
>>>  >>> processing
>>>  >>> to use the BIGTIFF option or a raster format that can handle >4GB
>>>  >>> files?
>>>  >>>
>>>  >>> Hi Andreas, what tool specifically?
>>>  >>>
>>>  >>> cheers
>>>  >>>
>>>  >>> -- Giovanni --
>>>  >>> ___
>>>  >>> Qgis-developer mailing list
>>>  >>> Qgis-developer@lists.osgeo.org
>>>   >>> http://lists.osgeo.org/mailman/listinfo/qgis-developer [4] [1]
>>>  >>
>>>  >>  ___
>>>  >>  Qgis-developer mailing list
>>>  >>  Qgis-developer@lists.osgeo.org
>>>   >>  http://lists.osgeo.org/mailman/listinfo/qgis-developer [4] [1]
>>>  >>
>>>  >>
>>>  >>
>>>  >> Links:
>>>  >> --
>>>  >> [1] http://lists.osgeo.org/mailman/listinfo/qgis-developer [4]
>>>  >> [2] http://www.psig.es [3]
>>>  >
>>>  > ___
>>>  > Qgis-developer mailing list
>>>  > Qgis-developer@lists.osgeo.org
>>>   > http://lists.osgeo.org/mailman/listinfo/qgis-developer [4]
>>>
>>> Links:
>>> --
>>> [1] https://github.com/chiatt/gdal_reclassify
>>> [2] http://www.gdal.org/gdal_calc.html
>>> [3] http://www.psig.es
>>> [4] http://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>
>>
>
>
>  --
>  Giovanni Allegri
> http://about.me/giovanniallegri
> Twitter: https://twitter.com/_giohappy_
> blog: http://blog.spaziogis.it
> GEO+ geomatica in Italia http://bit.ly/GEOplus
>
>
>


-- 
Giovanni Allegri
http://about.me/giovanniallegri
Twitter: https://twitter.com/_giohappy_
blog: http://blog.spaziogis.it
GEO+ geomatica in Italia http://bit.ly/GEOplus
___
Qgis-developer mailing list
Qgis-develop

Re: [Qgis-developer] Processing with raster files > 4GB

[Andreas Neumann]

Hi,

I will have to do further testing about GRASS. On Windows (OSGeo4W 
install) it does not work too well. It probably works better on Linux.


The gdal_calc.py solution works very fast though - glad that we came to 
this solution!


Andreas

On 19.11.2014 11:25, G. Allegri wrote:

And do you have problems with GRASS (directly, not through Processing)?
In GRASS7 you can even drectly write to Geotiff without passing 
through the internal format 
(http://grass.osgeo.org/grass70/manuals/r.external.out.html).

If GDAL is built with bigtiff support there souldn't be problems...

giovanni

2014-11-19 10:44 GMT+01:00 Andreas Neumann >:


Hi Giovanni,

I need to change all pixels that have the value of 249 to a value
of 255. All other pixels should stay at their value.

Andreas

Am 2014-11-19 10:21, schrieb G. Allegri:

From my grayscale tiff images I want to replace one pixel
value

(249) with white (255). All other pixels should stay at the
value they
have.
 >
 > I came across https://github.com/chiatt/gdal_reclassify [1]
- but
this reclassifies all pixels, not only selected ones.
 >
 > Maybe I could use http://www.gdal.org/gdal_calc.html [2] with a
conditional syntax?

Hi Andreas, do you need to change the value of one specific pixel
only? I mean the pixel at [x,y] index?

giovanni


 > Thanks,
 > Andreas
 >
 > Am 2014-11-19 09:10, schrieb Carlos López PSIG:
 >>
 >> Hi Andreas,
 >>
 >> I also had problems to reclassify BIGTIFF with SAGA and GRASS!
 >> Finally I used GDAL!
 >>
 >> Best regards,
 >>
 >>  [2]
 >>
 >> CARLOS LÓPEZ QUINTANILLA
 >>
 >> www.psig.es  [3] [2]

 >>
 >> carlos.lo...@psig.es 
 >> +34 699.680.261 
 >>
 >> 2014-11-19 9:05 GMT+01:00 Andreas Neumann
mailto:a.neum...@carto.net>>:
 >>
 >>> Hi,
 >>>
 >>> I tried various tools, e.g.
 >>>
 >>> * SAGA reclassify
 >>> * GRASS r.mapcalc
 >>>
 >>> Both work fine on smaller data sets, but fail on big data
sets.
 >>>
 >>> This is on Win7 64bit, with the OSGeo4W installer. I have
8gb RAM
 >>> and 40GB disk free on C:, more free space on other disks
where
the
 >>> data resides.
 >>>
 >>> I am not totally sure if it is the Bigtiff issue or some
other
 >>> issue. It is just a guess that maybe I am hitting a limit
with
tiff.
 >>> It is not fully transparent to me what Processing is
doing in the
 >>> background (what file formats with what options it creates).
 >>>
 >>> It may also be an issue with SAGA or GRASS directly.
 >>>
 >>> Thanks if you know anything,
 >>> Andreas
 >>>
 >>> Am 2014-11-18 21:15, schrieb Giovanni Manghi:
 >>>
 >>> Hi,
 >>>
 >>> I am having a lot of troubles with large raster files and
 >>> processing.
 >>>
 >>> The intermediate results can go >4GB. Processing stops
with not
so
 >>> helpful error messages.
 >>>
 >>> I wonder if processing is hitting the 4GB limit. Can I force
 >>> processing
 >>> to use the BIGTIFF option or a raster format that can
handle >4GB
 >>> files?
 >>>
 >>> Hi Andreas, what tool specifically?
 >>>
 >>> cheers
 >>>
 >>> -- Giovanni --
 >>> ___
 >>> Qgis-developer mailing list
 >>> Qgis-developer@lists.osgeo.org

 >>> http://lists.osgeo.org/mailman/listinfo/qgis-developer
[4] [1]
 >>
 >>  ___
 >>  Qgis-developer mailing list
 >> Qgis-developer@lists.osgeo.org

 >> http://lists.osgeo.org/mailman/listinfo/qgis-developer [4] [1]
 >>
 >>
 >>
 >> Links:
 >> --
 >> [1] http://lists.osgeo.org/mailman/listinfo/qgis-developer [4]
 >> [2] http://www.psig.es [3]
 >
 > ___
 > Qgis-developer mailing list
 > Qgis-developer@lists.osgeo.org

 > http://lists.osgeo.org/mailman/listinfo/qgis-developer [4]

Links:
--
[1] https://github.com/chiatt/gdal_reclassify
[2] http://www.gdal.org/gdal_calc.html
[3] http:/

Re: [Qgis-developer] Processing with raster files > 4GB

[G. Allegri]

And do you have problems with GRASS (directly, not through Processing)?
In GRASS7 you can even drectly write to Geotiff without passing through the
internal format (http://grass.osgeo.org/grass70/manuals/r.external.out.html
).
If GDAL is built with bigtiff support there souldn't be problems...

giovanni

2014-11-19 10:44 GMT+01:00 Andreas Neumann :

> Hi Giovanni,
>
> I need to change all pixels that have the value of 249 to a value of 255.
> All other pixels should stay at their value.
>
> Andreas
>
> Am 2014-11-19 10:21, schrieb G. Allegri:
>
>> From my grayscale tiff images I want to replace one pixel value
>>>
>> (249) with white (255). All other pixels should stay at the value they
>> have.
>>  >
>>  > I came across https://github.com/chiatt/gdal_reclassify [1] - but
>> this reclassifies all pixels, not only selected ones.
>>  >
>>  > Maybe I could use http://www.gdal.org/gdal_calc.html [2] with a
>> conditional syntax?
>>
>> Hi Andreas, do you need to change the value of one specific pixel
>> only? I mean the pixel at [x,y] index?
>>
>> giovanni
>>
>>
>>>   > Thanks,
>>  > Andreas
>>  >
>>  > Am 2014-11-19 09:10, schrieb Carlos López PSIG:
>>  >>
>>  >> Hi Andreas,
>>  >>
>>  >> I also had problems to reclassify BIGTIFF with SAGA and GRASS!
>>  >> Finally I used GDAL!
>>  >>
>>  >> Best regards,
>>  >>
>>  >>  [2]
>>  >>
>>  >> CARLOS LÓPEZ QUINTANILLA
>>  >>
>>  >> www.psig.es [3] [2]
>>
>>  >>
>>  >> carlos.lo...@psig.es
>>  >> +34 699.680.261
>>  >>
>>  >> 2014-11-19 9:05 GMT+01:00 Andreas Neumann :
>>  >>
>>  >>> Hi,
>>  >>>
>>  >>> I tried various tools, e.g.
>>  >>>
>>  >>> * SAGA reclassify
>>  >>> * GRASS r.mapcalc
>>  >>>
>>  >>> Both work fine on smaller data sets, but fail on big data sets.
>>  >>>
>>  >>> This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM
>>  >>> and 40GB disk free on C:, more free space on other disks where
>> the
>>  >>> data resides.
>>  >>>
>>  >>> I am not totally sure if it is the Bigtiff issue or some other
>>  >>> issue. It is just a guess that maybe I am hitting a limit with
>> tiff.
>>  >>> It is not fully transparent to me what Processing is doing in the
>>  >>> background (what file formats with what options it creates).
>>  >>>
>>  >>> It may also be an issue with SAGA or GRASS directly.
>>  >>>
>>  >>> Thanks if you know anything,
>>  >>> Andreas
>>  >>>
>>  >>> Am 2014-11-18 21:15, schrieb Giovanni Manghi:
>>  >>>
>>  >>> Hi,
>>  >>>
>>  >>> I am having a lot of troubles with large raster files and
>>  >>> processing.
>>  >>>
>>  >>> The intermediate results can go >4GB. Processing stops with not
>> so
>>  >>> helpful error messages.
>>  >>>
>>  >>> I wonder if processing is hitting the 4GB limit. Can I force
>>  >>> processing
>>  >>> to use the BIGTIFF option or a raster format that can handle >4GB
>>  >>> files?
>>  >>>
>>  >>> Hi Andreas, what tool specifically?
>>  >>>
>>  >>> cheers
>>  >>>
>>  >>> -- Giovanni --
>>  >>> ___
>>  >>> Qgis-developer mailing list
>>  >>> Qgis-developer@lists.osgeo.org
>>  >>> http://lists.osgeo.org/mailman/listinfo/qgis-developer [4] [1]
>>  >>
>>  >>  ___
>>  >>  Qgis-developer mailing list
>>  >>  Qgis-developer@lists.osgeo.org
>>  >>  http://lists.osgeo.org/mailman/listinfo/qgis-developer [4] [1]
>>  >>
>>  >>
>>  >>
>>  >> Links:
>>  >> --
>>  >> [1] http://lists.osgeo.org/mailman/listinfo/qgis-developer [4]
>>  >> [2] http://www.psig.es [3]
>>  >
>>  > ___
>>  > Qgis-developer mailing list
>>  > Qgis-developer@lists.osgeo.org
>>  > http://lists.osgeo.org/mailman/listinfo/qgis-developer [4]
>>
>> Links:
>> --
>> [1] https://github.com/chiatt/gdal_reclassify
>> [2] http://www.gdal.org/gdal_calc.html
>> [3] http://www.psig.es
>> [4] http://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>


-- 
Giovanni Allegri
http://about.me/giovanniallegri
Twitter: https://twitter.com/_giohappy_
blog: http://blog.spaziogis.it
GEO+ geomatica in Italia http://bit.ly/GEOplus
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Carlos López PSIG]

Hi Andreas,

Yes, you can use gdal_calc.py
Certainly you need to perform several processes, but are very fast!

Be careful with bands!
Perhaps you will want to separate them.

Ciao,



* *

*Carlos López Quintanilla*

www.psig.es
carlos.lo...@psig.es
+34 699.680.261



2014-11-19 9:26 GMT+01:00 Andreas Neumann :

> Hi Carlos,
>
> GDAL would be my first choice - because it easy to automate. However, I
> don't know how can do my task with GDAL.
>
> From my grayscale tiff images I want to replace one pixel value (249) with
> white (255). All other pixels should stay at the value they have.
>
> I came across https://github.com/chiatt/gdal_reclassify - but this
> reclassifies all pixels, not only selected ones.
>
> Maybe I could use http://www.gdal.org/gdal_calc.html with a conditional
> syntax?
>
> Thanks,
> Andreas
>
> Am 2014-11-19 09:10, schrieb Carlos López PSIG:
>
>> Hi Andreas,
>>
>> I also had problems to reclassify BIGTIFF with SAGA and GRASS!
>> Finally I used GDAL!
>>
>> Best regards,
>>
>>  [2]
>>
>> CARLOS LÓPEZ QUINTANILLA
>>
>> www.psig.es [2]
>>
>> carlos.lo...@psig.es
>> +34 699.680.261
>>
>> 2014-11-19 9:05 GMT+01:00 Andreas Neumann :
>>
>>  Hi,
>>>
>>> I tried various tools, e.g.
>>>
>>> * SAGA reclassify
>>> * GRASS r.mapcalc
>>>
>>> Both work fine on smaller data sets, but fail on big data sets.
>>>
>>> This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM
>>> and 40GB disk free on C:, more free space on other disks where the
>>> data resides.
>>>
>>> I am not totally sure if it is the Bigtiff issue or some other
>>> issue. It is just a guess that maybe I am hitting a limit with tiff.
>>> It is not fully transparent to me what Processing is doing in the
>>> background (what file formats with what options it creates).
>>>
>>> It may also be an issue with SAGA or GRASS directly.
>>>
>>> Thanks if you know anything,
>>> Andreas
>>>
>>> Am 2014-11-18 21:15, schrieb Giovanni Manghi:
>>>
>>> Hi,
>>>
>>> I am having a lot of troubles with large raster files and
>>> processing.
>>>
>>> The intermediate results can go >4GB. Processing stops with not so
>>> helpful error messages.
>>>
>>> I wonder if processing is hitting the 4GB limit. Can I force
>>> processing
>>> to use the BIGTIFF option or a raster format that can handle >4GB
>>> files?
>>>
>>> Hi Andreas, what tool specifically?
>>>
>>> cheers
>>>
>>> -- Giovanni --
>>> ___
>>> Qgis-developer mailing list
>>> Qgis-developer@lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/qgis-developer [1]
>>>
>>  ___
>>  Qgis-developer mailing list
>>  Qgis-developer@lists.osgeo.org
>>  http://lists.osgeo.org/mailman/listinfo/qgis-developer [1]
>>
>>
>>
>> Links:
>> --
>> [1] http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> [2] http://www.psig.es
>>
>
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Andreas Neumann]

Hi Giovanni,

I need to change all pixels that have the value of 249 to a value of 
255. All other pixels should stay at their value.


Andreas

Am 2014-11-19 10:21, schrieb G. Allegri:

From my grayscale tiff images I want to replace one pixel value

(249) with white (255). All other pixels should stay at the value they
have.
 >
 > I came across https://github.com/chiatt/gdal_reclassify [1] - but
this reclassifies all pixels, not only selected ones.
 >
 > Maybe I could use http://www.gdal.org/gdal_calc.html [2] with a
conditional syntax?

Hi Andreas, do you need to change the value of one specific pixel
only? I mean the pixel at [x,y] index?

giovanni




 > Thanks,
 > Andreas
 >
 > Am 2014-11-19 09:10, schrieb Carlos López PSIG:
 >>
 >> Hi Andreas,
 >>
 >> I also had problems to reclassify BIGTIFF with SAGA and GRASS!
 >> Finally I used GDAL!
 >>
 >> Best regards,
 >>
 >>  [2]
 >>
 >> CARLOS LÓPEZ QUINTANILLA
 >>
 >> www.psig.es [3] [2]
 >>
 >> carlos.lo...@psig.es
 >> +34 699.680.261
 >>
 >> 2014-11-19 9:05 GMT+01:00 Andreas Neumann :
 >>
 >>> Hi,
 >>>
 >>> I tried various tools, e.g.
 >>>
 >>> * SAGA reclassify
 >>> * GRASS r.mapcalc
 >>>
 >>> Both work fine on smaller data sets, but fail on big data sets.
 >>>
 >>> This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM
 >>> and 40GB disk free on C:, more free space on other disks where
the
 >>> data resides.
 >>>
 >>> I am not totally sure if it is the Bigtiff issue or some other
 >>> issue. It is just a guess that maybe I am hitting a limit with
tiff.
 >>> It is not fully transparent to me what Processing is doing in the
 >>> background (what file formats with what options it creates).
 >>>
 >>> It may also be an issue with SAGA or GRASS directly.
 >>>
 >>> Thanks if you know anything,
 >>> Andreas
 >>>
 >>> Am 2014-11-18 21:15, schrieb Giovanni Manghi:
 >>>
 >>> Hi,
 >>>
 >>> I am having a lot of troubles with large raster files and
 >>> processing.
 >>>
 >>> The intermediate results can go >4GB. Processing stops with not
so
 >>> helpful error messages.
 >>>
 >>> I wonder if processing is hitting the 4GB limit. Can I force
 >>> processing
 >>> to use the BIGTIFF option or a raster format that can handle >4GB
 >>> files?
 >>>
 >>> Hi Andreas, what tool specifically?
 >>>
 >>> cheers
 >>>
 >>> -- Giovanni --
 >>> ___
 >>> Qgis-developer mailing list
 >>> Qgis-developer@lists.osgeo.org
 >>> http://lists.osgeo.org/mailman/listinfo/qgis-developer [4] [1]
 >>
 >>  ___
 >>  Qgis-developer mailing list
 >>  Qgis-developer@lists.osgeo.org
 >>  http://lists.osgeo.org/mailman/listinfo/qgis-developer [4] [1]
 >>
 >>
 >>
 >> Links:
 >> --
 >> [1] http://lists.osgeo.org/mailman/listinfo/qgis-developer [4]
 >> [2] http://www.psig.es [3]
 >
 > ___
 > Qgis-developer mailing list
 > Qgis-developer@lists.osgeo.org
 > http://lists.osgeo.org/mailman/listinfo/qgis-developer [4]

Links:
--
[1] https://github.com/chiatt/gdal_reclassify
[2] http://www.gdal.org/gdal_calc.html
[3] http://www.psig.es
[4] http://lists.osgeo.org/mailman/listinfo/qgis-developer

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Roy]

Hi Andreas,

you can surely try to import the raster directly into GRASS in a 64bit 
linux,

very often this overcome memory issues, and use mapcalc sintiax,
something like:

reclassed_raster = if(original_raster == 249,255,original_raster)

regards,

Roy.



GDAL would be my first choice - because it easy to automate. However, 
I don't know how can do my task with GDAL.


From my grayscale tiff images I want to replace one pixel value (249) 
with white (255). All other pixels should stay at the value they have.


I came across https://github.com/chiatt/gdal_reclassify - but this 
reclassifies all pixels, not only selected ones.


Maybe I could use http://www.gdal.org/gdal_calc.html with a 
conditional syntax?


Thanks,
Andreas

Am 2014-11-19 09:10, schrieb Carlos López PSIG:

Hi Andreas,

I also had problems to reclassify BIGTIFF with SAGA and GRASS!
Finally I used GDAL!

Best regards,

 [2]

CARLOS LÓPEZ QUINTANILLA

www.psig.es [2]
carlos.lo...@psig.es
+34 699.680.261

2014-11-19 9:05 GMT+01:00 Andreas Neumann :


Hi,

I tried various tools, e.g.

* SAGA reclassify
* GRASS r.mapcalc

Both work fine on smaller data sets, but fail on big data sets.

This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM
and 40GB disk free on C:, more free space on other disks where the
data resides.

I am not totally sure if it is the Bigtiff issue or some other
issue. It is just a guess that maybe I am hitting a limit with tiff.
It is not fully transparent to me what Processing is doing in the
background (what file formats with what options it creates).

It may also be an issue with SAGA or GRASS directly.

Thanks if you know anything,
Andreas

Am 2014-11-18 21:15, schrieb Giovanni Manghi:

Hi,

I am having a lot of troubles with large raster files and
processing.

The intermediate results can go >4GB. Processing stops with not so
helpful error messages.

I wonder if processing is hitting the 4GB limit. Can I force
processing
to use the BIGTIFF option or a raster format that can handle >4GB
files?

Hi Andreas, what tool specifically?

cheers

-- Giovanni --
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer [1]

 ___
 Qgis-developer mailing list
 Qgis-developer@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/qgis-developer [1]



Links:
--
[1] http://lists.osgeo.org/mailman/listinfo/qgis-developer
[2] http://www.psig.es

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer


___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[G. Allegri]

> From my grayscale tiff images I want to replace one pixel value (249)
with white (255). All other pixels should stay at the value they have.
>
> I came across https://github.com/chiatt/gdal_reclassify - but this
reclassifies all pixels, not only selected ones.
>
> Maybe I could use http://www.gdal.org/gdal_calc.html with a conditional
syntax?

Hi Andreas, do you need to change the value of one specific pixel only? I
mean the pixel at [x,y] index?

giovanni

>
> Thanks,
> Andreas
>
> Am 2014-11-19 09:10, schrieb Carlos López PSIG:
>>
>> Hi Andreas,
>>
>> I also had problems to reclassify BIGTIFF with SAGA and GRASS!
>> Finally I used GDAL!
>>
>> Best regards,
>>
>>  [2]
>>
>> CARLOS LÓPEZ QUINTANILLA
>>
>> www.psig.es [2]
>>
>> carlos.lo...@psig.es
>> +34 699.680.261
>>
>> 2014-11-19 9:05 GMT+01:00 Andreas Neumann :
>>
>>> Hi,
>>>
>>> I tried various tools, e.g.
>>>
>>> * SAGA reclassify
>>> * GRASS r.mapcalc
>>>
>>> Both work fine on smaller data sets, but fail on big data sets.
>>>
>>> This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM
>>> and 40GB disk free on C:, more free space on other disks where the
>>> data resides.
>>>
>>> I am not totally sure if it is the Bigtiff issue or some other
>>> issue. It is just a guess that maybe I am hitting a limit with tiff.
>>> It is not fully transparent to me what Processing is doing in the
>>> background (what file formats with what options it creates).
>>>
>>> It may also be an issue with SAGA or GRASS directly.
>>>
>>> Thanks if you know anything,
>>> Andreas
>>>
>>> Am 2014-11-18 21:15, schrieb Giovanni Manghi:
>>>
>>> Hi,
>>>
>>> I am having a lot of troubles with large raster files and
>>> processing.
>>>
>>> The intermediate results can go >4GB. Processing stops with not so
>>> helpful error messages.
>>>
>>> I wonder if processing is hitting the 4GB limit. Can I force
>>> processing
>>> to use the BIGTIFF option or a raster format that can handle >4GB
>>> files?
>>>
>>> Hi Andreas, what tool specifically?
>>>
>>> cheers
>>>
>>> -- Giovanni --
>>> ___
>>> Qgis-developer mailing list
>>> Qgis-developer@lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/qgis-developer [1]
>>
>>  ___
>>  Qgis-developer mailing list
>>  Qgis-developer@lists.osgeo.org
>>  http://lists.osgeo.org/mailman/listinfo/qgis-developer [1]
>>
>>
>>
>> Links:
>> --
>> [1] http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> [2] http://www.psig.es
>
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Andreas Neumann]

Hi Carlos,

GDAL would be my first choice - because it easy to automate. However, I 
don't know how can do my task with GDAL.


From my grayscale tiff images I want to replace one pixel value (249) 
with white (255). All other pixels should stay at the value they have.


I came across https://github.com/chiatt/gdal_reclassify - but this 
reclassifies all pixels, not only selected ones.


Maybe I could use http://www.gdal.org/gdal_calc.html with a conditional 
syntax?


Thanks,
Andreas

Am 2014-11-19 09:10, schrieb Carlos López PSIG:

Hi Andreas,

I also had problems to reclassify BIGTIFF with SAGA and GRASS!
Finally I used GDAL!

Best regards,

 [2]

CARLOS LÓPEZ QUINTANILLA

www.psig.es [2]
carlos.lo...@psig.es
+34 699.680.261

2014-11-19 9:05 GMT+01:00 Andreas Neumann :


Hi,

I tried various tools, e.g.

* SAGA reclassify
* GRASS r.mapcalc

Both work fine on smaller data sets, but fail on big data sets.

This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM
and 40GB disk free on C:, more free space on other disks where the
data resides.

I am not totally sure if it is the Bigtiff issue or some other
issue. It is just a guess that maybe I am hitting a limit with tiff.
It is not fully transparent to me what Processing is doing in the
background (what file formats with what options it creates).

It may also be an issue with SAGA or GRASS directly.

Thanks if you know anything,
Andreas

Am 2014-11-18 21:15, schrieb Giovanni Manghi:

Hi,

I am having a lot of troubles with large raster files and
processing.

The intermediate results can go >4GB. Processing stops with not so
helpful error messages.

I wonder if processing is hitting the 4GB limit. Can I force
processing
to use the BIGTIFF option or a raster format that can handle >4GB
files?

Hi Andreas, what tool specifically?

cheers

-- Giovanni --
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer [1]

 ___
 Qgis-developer mailing list
 Qgis-developer@lists.osgeo.org
 http://lists.osgeo.org/mailman/listinfo/qgis-developer [1]



Links:
--
[1] http://lists.osgeo.org/mailman/listinfo/qgis-developer
[2] http://www.psig.es

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Carlos López PSIG]

Hi Andreas,
I also had problems to reclassify BIGTIFF with SAGA and GRASS!
Finally I used GDAL!

Best regards,


* *

*Carlos López Quintanilla*

www.psig.es
carlos.lo...@psig.es
+34 699.680.261



2014-11-19 9:05 GMT+01:00 Andreas Neumann :

> Hi,
>
> I tried various tools, e.g.
>
> * SAGA reclassify
> * GRASS r.mapcalc
>
> Both work fine on smaller data sets, but fail on big data sets.
>
> This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM and 40GB
> disk free on C:, more free space on other disks where the data resides.
>
> I am not totally sure if it is the Bigtiff issue or some other issue. It
> is just a guess that maybe I am hitting a limit with tiff. It is not fully
> transparent to me what Processing is doing in the background (what file
> formats with what options it creates).
>
> It may also be an issue with SAGA or GRASS directly.
>
> Thanks if you know anything,
> Andreas
>
> Am 2014-11-18 21:15, schrieb Giovanni Manghi:
>
>  Hi,
>>>
>>> I am having a lot of troubles with large raster files and processing.
>>>
>>> The intermediate results can go >4GB. Processing stops with not so
>>> helpful error messages.
>>>
>>> I wonder if processing is hitting the 4GB limit. Can I force processing
>>> to use the BIGTIFF option or a raster format that can handle >4GB files?
>>>
>>>
>>
>>
>> Hi Andreas, what tool specifically?
>>
>> cheers
>>
>> -- Giovanni --
>> ___
>> Qgis-developer mailing list
>> Qgis-developer@lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Andreas Neumann]

Hi,

I tried various tools, e.g.

* SAGA reclassify
* GRASS r.mapcalc

Both work fine on smaller data sets, but fail on big data sets.

This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM and 
40GB disk free on C:, more free space on other disks where the data 
resides.


I am not totally sure if it is the Bigtiff issue or some other issue. It 
is just a guess that maybe I am hitting a limit with tiff. It is not 
fully transparent to me what Processing is doing in the background (what 
file formats with what options it creates).


It may also be an issue with SAGA or GRASS directly.

Thanks if you know anything,
Andreas

Am 2014-11-18 21:15, schrieb Giovanni Manghi:

Hi,

I am having a lot of troubles with large raster files and processing.

The intermediate results can go >4GB. Processing stops with not so
helpful error messages.

I wonder if processing is hitting the 4GB limit. Can I force 
processing
to use the BIGTIFF option or a raster format that can handle >4GB 
files?






Hi Andreas, what tool specifically?

cheers

-- Giovanni --
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing with raster files > 4GB

[Giovanni Manghi]

> Hi,
>
> I am having a lot of troubles with large raster files and processing.
>
> The intermediate results can go >4GB. Processing stops with not so
> helpful error messages.
>
> I wonder if processing is hitting the 4GB limit. Can I force processing
> to use the BIGTIFF option or a raster format that can handle >4GB files?
>



Hi Andreas, what tool specifically?

cheers

-- Giovanni --
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

[Qgis-developer] Processing with raster files > 4GB

[Andreas Neumann]

Hi,

I am having a lot of troubles with large raster files and processing.

The intermediate results can go >4GB. Processing stops with not so 
helpful error messages.


I wonder if processing is hitting the 4GB limit. Can I force processing 
to use the BIGTIFF option or a raster format that can handle >4GB files?


Thank you,
Andreas
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer