Re: [GRASS-user] Best format for exporting raster data

2010-09-03 Thread Rainer M Krug 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/09/10 21:21, Glynn Clements wrote:
> 
> Rainer M Krug wrote:
> 
>>> But GRASS doesn't generally read data from files per se, but from
>>> either the GRASS "database" or from GDAL (and the former might
>>> eventually go away if we can get "native" GRASS support into GDAL).
>>
>> I don't understand that statement - I am the database is ultimately a
>> file, so grass is requesting the data from the database which then reads
>> it from the mapset which consists of files.
> 
> Ultimately, it's files, but only at the lowest level of the core
> libraries. Modules get their data from the "GRASS database", but the
> nature of that database is an implementation detail.

OK - that's clear.

> 
>> And gdal is used when data is read which is not in the mapset -
>> right?
> 
> GDAL is used if you create a "linked" map using r.external or
> r.external.out. 

Makes sense.

> 
>>> Similarly, while there exist filesystems which can mount archives, tar
>>> files (and especially compressed tar files) are a poor choice, as they
>>> are designed for sequential access. ZIP/RAR are more suited to such
>>> tasks.
>>
>> but they can actually be mounted as was pointed out by Markus Neteler
>> (http://www.linux.com/archive/feature/132196).
> 
> They can, but the filesystem will have to decompress a substantial
> part of the archive just to access one file. If you use zip/rar, it
> only has to decompress files which are accessed.

That is definitely an interesting detail, I was not aware of. I will
definitely think about using one of those.

> 
>>> Ultimately, I don't think that this situation is common enough to be
>>> worth doing anything about. If you want to access archived data, you
>>> just unpack the archives first (or use an archive format which can be
>>> mounted).
>>
>> I'll folow the following approach: I'll leave the data uncompressed,
>> until the analysis are completed (I have a new 2TB hdd now) and will
>> compress them into tar.gz files as soon as the project is finished. If I
>> need them again, I will use archivemount to access the data.
> 
> If you're planning on using archivemount, use zip/rar rather than
> .tar.gz (and don't use "solid" rar archives, which have the same
> problems with random access as tar files).

Thanks for this tip - I will likely do as suggested.

Cheers,

Rainer

> 


- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Natural Sciences Building
Office Suite 2039
Stellenbosch University
Main Campus, Merriman Avenue
Stellenbosch
South Africa

Tel:+33 - (0)9 53 10 27 44
Cell:   +27 - (0)8 39 47 90 42
Fax (SA):   +27 - (0)8 65 16 27 82
Fax (D) :   +49 - (0)3 21 21 25 22 44
Fax (FR):   +33 - (0)9 58 10 27 44
email:  rai...@krugs.de

Skype:  RMkrug
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkyAqeoACgkQoYgNqgF2egry3QCfe2RwvE91BYHGpr1CsApwvZ+U
zuwAoIS6ywDmwxGIevvei40AxEXpmMh6
=1NJE
-END PGP SIGNATURE-
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-09-02 Thread Glynn Clements 

Rainer M Krug wrote:

> > But GRASS doesn't generally read data from files per se, but from
> > either the GRASS "database" or from GDAL (and the former might
> > eventually go away if we can get "native" GRASS support into GDAL).
> 
> I don't understand that statement - I am the database is ultimately a
> file, so grass is requesting the data from the database which then reads
> it from the mapset which consists of files.

Ultimately, it's files, but only at the lowest level of the core
libraries. Modules get their data from the "GRASS database", but the
nature of that database is an implementation detail.

> And gdal is used when data is read which is not in the mapset -
> right?

GDAL is used if you create a "linked" map using r.external or
r.external.out. 

> > Similarly, while there exist filesystems which can mount archives, tar
> > files (and especially compressed tar files) are a poor choice, as they
> > are designed for sequential access. ZIP/RAR are more suited to such
> > tasks.
> 
> but they can actually be mounted as was pointed out by Markus Neteler
> (http://www.linux.com/archive/feature/132196).

They can, but the filesystem will have to decompress a substantial
part of the archive just to access one file. If you use zip/rar, it
only has to decompress files which are accessed.

> > Ultimately, I don't think that this situation is common enough to be
> > worth doing anything about. If you want to access archived data, you
> > just unpack the archives first (or use an archive format which can be
> > mounted).
> 
> I'll folow the following approach: I'll leave the data uncompressed,
> until the analysis are completed (I have a new 2TB hdd now) and will
> compress them into tar.gz files as soon as the project is finished. If I
> need them again, I will use archivemount to access the data.

If you're planning on using archivemount, use zip/rar rather than
.tar.gz (and don't use "solid" rar archives, which have the same
problems with random access as tar files).

-- 
Glynn Clements 
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-09-02 Thread Rainer M Krug 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/10 19:45, Glynn Clements wrote:
> 
> Rainer M Krug wrote:
> 
>> It would be nice, if grass would be able to deal with on-the-fly
>> decompression - not from a .tar.gz file, but from gz compressed files.
> 
> GRASS rasters are already compressed by default, either using RLE or
> zlib compression (OTOH, the null bitmap isn't compressed; that will
> cease to be an issue if we embed nulls into the raster data).

That is good to know - as mentioned in an earlier email, I will then
leave as it is. I actually compared the size of the compresses grass
folder with the non-compressed one, and the difference did not justify
the effort.

> 
> But GRASS doesn't generally read data from files per se, but from
> either the GRASS "database" or from GDAL (and the former might
> eventually go away if we can get "native" GRASS support into GDAL).

I don't understand that statement - I am the database is ultimately a
file, so grass is requesting the data from the database which then reads
it from the mapset which consists of files. And gdal is used when data
is read which is not in the mapset - right?

> 
> The main issue with on-the-fly decompression using general-purpose
> formats is that rasters aren't guaranteed to be read sequentially,
> while compression algorithms require sequential access.

Makes sense - thanks for the clarification.

> 
> Similarly, while there exist filesystems which can mount archives, tar
> files (and especially compressed tar files) are a poor choice, as they
> are designed for sequential access. ZIP/RAR are more suited to such
> tasks.

but they can actually be mounted as was pointed out by Markus Neteler
(http://www.linux.com/archive/feature/132196).

> 
> Ultimately, I don't think that this situation is common enough to be
> worth doing anything about. If you want to access archived data, you
> just unpack the archives first (or use an archive format which can be
> mounted).

I'll folow the following approach: I'll leave the data uncompressed,
until the analysis are completed (I have a new 2TB hdd now) and will
compress them into tar.gz files as soon as the project is finished. If I
need them again, I will use archivemount to access the data.

Thanks everybody for your help,

Rainer

> 


- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Natural Sciences Building
Office Suite 2039
Stellenbosch University
Main Campus, Merriman Avenue
Stellenbosch
South Africa

Tel:+33 - (0)9 53 10 27 44
Cell:   +27 - (0)8 39 47 90 42
Fax (SA):   +27 - (0)8 65 16 27 82
Fax (D) :   +49 - (0)3 21 21 25 22 44
Fax (FR):   +33 - (0)9 58 10 27 44
email:  rai...@krugs.de

Skype:  RMkrug
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkx/XyEACgkQoYgNqgF2egqrBwCggpE9+rv+KlfDBU3goF9lK0JK
GYUAn0nq2oam3Y2yG2tbRiZoQDfbLXou
=+nHe
-END PGP SIGNATURE-
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-09-01 Thread Glynn Clements 

Rainer M Krug wrote:

> It would be nice, if grass would be able to deal with on-the-fly
> decompression - not from a .tar.gz file, but from gz compressed files.

GRASS rasters are already compressed by default, either using RLE or
zlib compression (OTOH, the null bitmap isn't compressed; that will
cease to be an issue if we embed nulls into the raster data).

But GRASS doesn't generally read data from files per se, but from
either the GRASS "database" or from GDAL (and the former might
eventually go away if we can get "native" GRASS support into GDAL).

The main issue with on-the-fly decompression using general-purpose
formats is that rasters aren't guaranteed to be read sequentially,
while compression algorithms require sequential access.

Similarly, while there exist filesystems which can mount archives, tar
files (and especially compressed tar files) are a poor choice, as they
are designed for sequential access. ZIP/RAR are more suited to such
tasks.

Ultimately, I don't think that this situation is common enough to be
worth doing anything about. If you want to access archived data, you
just unpack the archives first (or use an archive format which can be
mounted).

-- 
Glynn Clements 
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-09-01 Thread Markus Metz 
Rainer M Krug wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hi
>
> I am creating a huge amount of raster layers during my simulations, and
> I would like to archive then to enable further analysis. At the moment I
> am leaving them in the grass database and compress the whole mapset into
> a tar.gz file. But this is rather cumbersome, if I want to extract some
> selected layers and analyse them further (my analysis is done in R).
> Therefore I would like to export the created layers while the simulation
> is running and to delete them from the grass database.
> My question: what is the best format for this?
> It should :
> - - contain all the information contained in the raster layer in the grass
> mapset
> - - be readable by at least gdal
> - - be preferably compressed (but I can compress them after export)
>
> At the moment I am using for a similar purpose the esri asc grid, but I
> am somehow critical about the fact that it uses a text representation of
> my data with limited decimals, therefore probably loosing information
> compared with the grass file.
>
> Are Binary fiels a better option (in the manual it states "Exports", not
> "converts" as in the esri ascii grid) and can I read them from R or gdal?
>
Raster data (the actual data grids) are already compressed in GRASS,
nothing much to gain there to compress already compressed data. You
can try to set GRASS_INT_ZLIB [1] and check if this gives better
compression than the default RLE for CELL maps.

Generally, the recommended export format is GeoTIFF which supports
various internal compression methods. A high compression ratio is
achieved with LZW and DEFLATE.

Not all packages using gdal support all gdal features, DEFLATE in
particular is often not supported by packages using their own modified
gdal version (does not apply to packages using an external gdal
library, e.g. GRASS, R, QGIS).

Markus M

[1] http://grass.osgeo.org/grass64/manuals/html64_user/variables.html
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-09-01 Thread Rainer M Krug 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/10 10:15, Rainer M Krug wrote:
> On 31/08/10 16:56, Sylvain Maillard wrote:
>> Perhaps can you add an extra loop in your scripts for processing your data:
>> 1 - extract the tarball
>> 2 - import the raster in R
>> 3 - and then delete the temporary uncompressed mapset.
> 
> Yes - that would be the "crude force" approach. In my case, it would
> very likely take more time, as there are MANY more layers in the .tar.gz
> then I use.
> I am now thinking about extracting all files with "fire" in  them and to
> use this subset as a mapset - I will see if it works.
> 
> It would be nice, if grass would be able to deal with on-the-fly
> decompression - not from a .tar.gz file, but from gz compressed files.
> 
> Cheers,
> 
> Rainer
> 
> 
>> it will take a little bit more space but just for one mapset at a time,
>> and I don't thing the process will be much slower than to access the
>> files directly into the compressed tarball ...
> 
>> you can also buy more hard drive :D
> 
>> if you make some benchmark test between different solution I will be
>> interreted in the results, I'm also working on a huge amound of raster
>> data within GRASS and R ...
> 
> I'll do - although I don't think I will do benchmarks at that time.

I am trying archivemounter and it has some interesting benchmarks on the
website http://www.linux.com/archive/feature/132196

Cheers,

Rainer

> 
> Cheers,
> 
> Rainer
> 
> 
> 
>> regards,
>> Sylvain Maillard
> 
>> Doctorant en Sciences de l'Environnement
>> Laboratoire Chimie Provence - UMR 6264 / Université de Provence
>> la Tour du Valat - Centre de recherche pour la conservation des zones
>> humides méditerranéennes
>> Le Sambuc
>> 13200 Arles
>> France
>> tél:04.90.97.29.79
>> fax:04.90.97.20.19
>> www.tourduvalat.org 
> 
> 
> 
>> 2010/8/31 Rainer M Krug mailto:r.m.k...@gmail.com>>
> 
>> On 31/08/10 16:38, Markus Neteler wrote:
>>> On Tue, Aug 31, 2010 at 4:13 PM, Rainer M Krug > > wrote:
>>> ...
> I would leave it in GRASS and use the R-GRASS interface and/or the
> GDAL-GRASS plugin. See the Wiki for details.

 I am doing that already - but I don't think that works when I
>> have the
 grass mapset compressed as a .tar.gz?
> 
>>> I found "archivemount" which apparently lets you
>>> mount a possibly compressed tarball as a filesystem:
> 
>>> http://en.wikipedia.org/wiki/Archivemount
>>> http://www.linux.com/archive/feature/132196
> 
>> Sounds interesting - I'll look into that.
> 
>> Thanks,
> 
>> Rainer
> 
> 
>>> Cheers
>>> Markus
> 
> 
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org 
> http://lists.osgeo.org/mailman/listinfo/grass-user
> 

- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Natural Sciences Building
Office Suite 2039
Stellenbosch University
Main Campus, Merriman Avenue
Stellenbosch
South Africa

Tel:+33 - (0)9 53 10 27 44
Cell:   +27 - (0)8 39 47 90 42
Fax (SA):   +27 - (0)8 65 16 27 82
Fax (D) :   +49 - (0)3 21 21 25 22 44
Fax (FR):   +33 - (0)9 58 10 27 44
email:  rai...@krugs.de

Skype:  RMkrug
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkx+D7kACgkQoYgNqgF2egrYKwCfSRRC23jZdwv8S30jrtFFp2hu
0X4AoIIbHJ/ZccwilSmaTcM7YJCa6tDZ
=P1+t
-END PGP SIGNATURE-
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-09-01 Thread Rainer M Krug 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 31/08/10 16:56, Sylvain Maillard wrote:
> Perhaps can you add an extra loop in your scripts for processing your data:
> 1 - extract the tarball
> 2 - import the raster in R
> 3 - and then delete the temporary uncompressed mapset.

Yes - that would be the "crude force" approach. In my case, it would
very likely take more time, as there are MANY more layers in the .tar.gz
then I use.
I am now thinking about extracting all files with "fire" in  them and to
use this subset as a mapset - I will see if it works.

It would be nice, if grass would be able to deal with on-the-fly
decompression - not from a .tar.gz file, but from gz compressed files.

Cheers,

Rainer


> it will take a little bit more space but just for one mapset at a time,
> and I don't thing the process will be much slower than to access the
> files directly into the compressed tarball ...
> 
> you can also buy more hard drive :D
> 
> if you make some benchmark test between different solution I will be
> interreted in the results, I'm also working on a huge amound of raster
> data within GRASS and R ...

I'll do - although I don't think I will do benchmarks at that time.

Cheers,

Rainer


> 
> regards,
> Sylvain Maillard
> 
> Doctorant en Sciences de l'Environnement
> Laboratoire Chimie Provence - UMR 6264 / Université de Provence
> la Tour du Valat - Centre de recherche pour la conservation des zones
> humides méditerranéennes
> Le Sambuc
> 13200 Arles
> France
> tél:04.90.97.29.79
> fax:04.90.97.20.19
> www.tourduvalat.org 
> 
> 
> 
> 2010/8/31 Rainer M Krug mailto:r.m.k...@gmail.com>>
> 
> On 31/08/10 16:38, Markus Neteler wrote:
>> On Tue, Aug 31, 2010 at 4:13 PM, Rainer M Krug  > wrote:
>> ...
 I would leave it in GRASS and use the R-GRASS interface and/or the
 GDAL-GRASS plugin. See the Wiki for details.
>>>
>>> I am doing that already - but I don't think that works when I
> have the
>>> grass mapset compressed as a .tar.gz?
> 
>> I found "archivemount" which apparently lets you
>> mount a possibly compressed tarball as a filesystem:
> 
>> http://en.wikipedia.org/wiki/Archivemount
>> http://www.linux.com/archive/feature/132196
> 
> Sounds interesting - I'll look into that.
> 
> Thanks,
> 
> Rainer
> 
> 
>> Cheers
>> Markus
> 
> 
___
grass-user mailing list
grass-user@lists.osgeo.org 
http://lists.osgeo.org/mailman/listinfo/grass-user

- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Natural Sciences Building
Office Suite 2039
Stellenbosch University
Main Campus, Merriman Avenue
Stellenbosch
South Africa

Tel:+33 - (0)9 53 10 27 44
Cell:   +27 - (0)8 39 47 90 42
Fax (SA):   +27 - (0)8 65 16 27 82
Fax (D) :   +49 - (0)3 21 21 25 22 44
Fax (FR):   +33 - (0)9 58 10 27 44
email:  rai...@krugs.de

Skype:  RMkrug
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkx+C5QACgkQoYgNqgF2egoSOwCeLJPZ4WGaem4o8k+lvDpT5QiG
l4gAnRe9trZfFaD+STl7yByUXA3IYiXS
=+YiD
-END PGP SIGNATURE-
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-08-31 Thread Sylvain Maillard 
Perhaps can you add an extra loop in your scripts for processing your data:
1 - extract the tarball
2 - import the raster in R
3 - and then delete the temporary uncompressed mapset.
it will take a little bit more space but just for one mapset at a time, and
I don't thing the process will be much slower than to access the files
directly into the compressed tarball ...

you can also buy more hard drive :D

if you make some benchmark test between different solution I will be
interreted in the results, I'm also working on a huge amound of raster data
within GRASS and R ...

regards,
Sylvain Maillard

Doctorant en Sciences de l'Environnement
Laboratoire Chimie Provence - UMR 6264 / Université de Provence
la Tour du Valat - Centre de recherche pour la conservation des zones
humides méditerranéennes
Le Sambuc
13200 Arles
France
tél:04.90.97.29.79
fax:04.90.97.20.19
www.tourduvalat.org



2010/8/31 Rainer M Krug 

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 31/08/10 16:38, Markus Neteler wrote:
> > On Tue, Aug 31, 2010 at 4:13 PM, Rainer M Krug 
> wrote:
> > ...
> >>> I would leave it in GRASS and use the R-GRASS interface and/or the
> >>> GDAL-GRASS plugin. See the Wiki for details.
> >>
> >> I am doing that already - but I don't think that works when I have the
> >> grass mapset compressed as a .tar.gz?
> >
> > I found "archivemount" which apparently lets you
> > mount a possibly compressed tarball as a filesystem:
> >
> > http://en.wikipedia.org/wiki/Archivemount
> > http://www.linux.com/archive/feature/132196
>
> Sounds interesting - I'll look into that.
>
> Thanks,
>
> Rainer
>
> >
> > Cheers
> > Markus
>
>
> - --
> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
> Biology, UCT), Dipl. Phys. (Germany)
>
> Centre of Excellence for Invasion Biology
> Natural Sciences Building
> Office Suite 2039
> Stellenbosch University
> Main Campus, Merriman Avenue
> Stellenbosch
> South Africa
>
> Tel:+33 - (0)9 53 10 27 44
> Cell:   +27 - (0)8 39 47 90 42
> Fax (SA):   +27 - (0)8 65 16 27 82
> Fax (D) :   +49 - (0)3 21 21 25 22 44
> Fax (FR):   +33 - (0)9 58 10 27 44
> email:  rai...@krugs.de
>
> Skype:  RMkrug
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkx9Fo0ACgkQoYgNqgF2egot0gCaA4EJiyfYkRYjYXeJpKSBLaGT
> V5QAoITs1O5Npb81z0UvSYtY2tTtc7cy
> =rcm6
> -END PGP SIGNATURE-
> ___
> grass-user mailing list
> grass-user@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user
>
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-08-31 Thread Rainer M Krug 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 31/08/10 16:38, Markus Neteler wrote:
> On Tue, Aug 31, 2010 at 4:13 PM, Rainer M Krug  wrote:
> ...
>>> I would leave it in GRASS and use the R-GRASS interface and/or the
>>> GDAL-GRASS plugin. See the Wiki for details.
>>
>> I am doing that already - but I don't think that works when I have the
>> grass mapset compressed as a .tar.gz?
> 
> I found "archivemount" which apparently lets you
> mount a possibly compressed tarball as a filesystem:
> 
> http://en.wikipedia.org/wiki/Archivemount
> http://www.linux.com/archive/feature/132196

Sounds interesting - I'll look into that.

Thanks,

Rainer

> 
> Cheers
> Markus


- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Natural Sciences Building
Office Suite 2039
Stellenbosch University
Main Campus, Merriman Avenue
Stellenbosch
South Africa

Tel:+33 - (0)9 53 10 27 44
Cell:   +27 - (0)8 39 47 90 42
Fax (SA):   +27 - (0)8 65 16 27 82
Fax (D) :   +49 - (0)3 21 21 25 22 44
Fax (FR):   +33 - (0)9 58 10 27 44
email:  rai...@krugs.de

Skype:  RMkrug
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkx9Fo0ACgkQoYgNqgF2egot0gCaA4EJiyfYkRYjYXeJpKSBLaGT
V5QAoITs1O5Npb81z0UvSYtY2tTtc7cy
=rcm6
-END PGP SIGNATURE-
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-08-31 Thread Markus Neteler 
On Tue, Aug 31, 2010 at 4:13 PM, Rainer M Krug  wrote:
...
>> I would leave it in GRASS and use the R-GRASS interface and/or the
>> GDAL-GRASS plugin. See the Wiki for details.
>
> I am doing that already - but I don't think that works when I have the
> grass mapset compressed as a .tar.gz?

I found "archivemount" which apparently lets you
mount a possibly compressed tarball as a filesystem:

http://en.wikipedia.org/wiki/Archivemount
http://www.linux.com/archive/feature/132196

Cheers
Markus
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-08-31 Thread Rainer M Krug 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 31/08/10 15:38, Markus Neteler wrote:
> hi Rainer,

Hi Markus,

> 
> On Tue, Aug 31, 2010 at 2:49 PM, Rainer M Krug  wrote:
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>
>> Hi
>>
>> I am creating a huge amount of raster layers during my simulations, and
>> I would like to archive then to enable further analysis. At the moment I
>> am leaving them in the grass database and compress the whole mapset into
>> a tar.gz file. But this is rather cumbersome, if I want to extract some
>> selected layers and analyse them further (my analysis is done in R).
> 
> Wht don't you simple use the R-GRASS interface?

I do - but I haven't mentioned it, after completing the simulation I
want to compress the data to save space.

> 
>> Therefore I would like to export the created layers while the simulation
>> is running and to delete them from the grass database.
>> My question: what is the best format for this?
>> It should :
>> - - contain all the information contained in the raster layer in the grass
>> mapset
>> - - be readable by at least gdal
>> - - be preferably compressed (but I can compress them after export)
> 
> I would leave it in GRASS and use the R-GRASS interface and/or the
> GDAL-GRASS plugin. See the Wiki for details.

I am doing that already - but I don't think that works when I have the
grass mapset compressed as a .tar.gz?

Cheers,

Rainer


> 
> Cheers
> Markus


- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Natural Sciences Building
Office Suite 2039
Stellenbosch University
Main Campus, Merriman Avenue
Stellenbosch
South Africa

Tel:+33 - (0)9 53 10 27 44
Cell:   +27 - (0)8 39 47 90 42
Fax (SA):   +27 - (0)8 65 16 27 82
Fax (D) :   +49 - (0)3 21 21 25 22 44
Fax (FR):   +33 - (0)9 58 10 27 44
email:  rai...@krugs.de

Skype:  RMkrug
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkx9DgIACgkQoYgNqgF2egqIxwCfYAkZaF4aogCqf0xVKkoAUxzW
egcAnieVccH0L7RlO7dqa7vOndjT04uQ
=p69/
-END PGP SIGNATURE-
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

Re: [GRASS-user] Best format for exporting raster data

2010-08-31 Thread Markus Neteler 
hi Rainer,

On Tue, Aug 31, 2010 at 2:49 PM, Rainer M Krug  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hi
>
> I am creating a huge amount of raster layers during my simulations, and
> I would like to archive then to enable further analysis. At the moment I
> am leaving them in the grass database and compress the whole mapset into
> a tar.gz file. But this is rather cumbersome, if I want to extract some
> selected layers and analyse them further (my analysis is done in R).

Wht don't you simple use the R-GRASS interface?

> Therefore I would like to export the created layers while the simulation
> is running and to delete them from the grass database.
> My question: what is the best format for this?
> It should :
> - - contain all the information contained in the raster layer in the grass
> mapset
> - - be readable by at least gdal
> - - be preferably compressed (but I can compress them after export)

I would leave it in GRASS and use the R-GRASS interface and/or the
GDAL-GRASS plugin. See the Wiki for details.

Cheers
Markus
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user

[GRASS-user] Best format for exporting raster data

2010-08-31 Thread Rainer M Krug 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi

I am creating a huge amount of raster layers during my simulations, and
I would like to archive then to enable further analysis. At the moment I
am leaving them in the grass database and compress the whole mapset into
a tar.gz file. But this is rather cumbersome, if I want to extract some
selected layers and analyse them further (my analysis is done in R).
Therefore I would like to export the created layers while the simulation
is running and to delete them from the grass database.
My question: what is the best format for this?
It should :
- - contain all the information contained in the raster layer in the grass
mapset
- - be readable by at least gdal
- - be preferably compressed (but I can compress them after export)

At the moment I am using for a similar purpose the esri asc grid, but I
am somehow critical about the fact that it uses a text representation of
my data with limited decimals, therefore probably loosing information
compared with the grass file.

Are Binary fiels a better option (in the manual it states "Exports", not
"converts" as in the esri ascii grid) and can I read them from R or gdal?

Any ideas welcome,

Rainer


- -- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation
Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Natural Sciences Building
Office Suite 2039
Stellenbosch University
Main Campus, Merriman Avenue
Stellenbosch
South Africa

Tel:+33 - (0)9 53 10 27 44
Cell:   +27 - (0)8 39 47 90 42
Fax (SA):   +27 - (0)8 65 16 27 82
Fax (D) :   +49 - (0)3 21 21 25 22 44
Fax (FR):   +33 - (0)9 58 10 27 44
email:  rai...@krugs.de

Skype:  RMkrug
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkx8+lEACgkQoYgNqgF2egqgdwCeM2FB10D4qFSNLGM9hRW58h9M
7pcAn0RI0w2NGABN/YOdA+HCuSeywSbs
=MbeI
-END PGP SIGNATURE-
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user