Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
Tom Roche Tue, Aug 28, 2012 at 4:23 PM >> Can ncdf4 be made to fail more helpfully? E.g., to fail immediately >> on nc_open without assignment? David W. Pierce Tue, 28 Aug 2012 13:23:53 -0700 > I'll see if there exists some way to do a better check for this. TIA. > One of the many great things about users: they provide opportunities > to add in more explanatory text for errors you never would have > thought of. :) Oh, it's worse than that :-) Drawing on my career (and many colleagues') as a producer and consumer of software, I have come to formulate Roche's 3 Laws of Software Adoption: 1, A software project becomes alive when it has some community of users. 2. If the intelligence of individual users can always be quantified, there is some PDF characterizing the collective intelligence of the project's users at every point in its life. 3. At any time t when that project's user community is increasing, new users tend to add to the LHS of the intelligence PDF at t. (OK, 2 definitions and 1 law :-) FWIW, Tom Roche __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
De rien. You're welcome.
Pascal
Le 12/08/29 5:23, Tom Roche a écrit :
Pascal Oettli: MERCI BEAUCOUP! (though I would have thanked you
earlier if I hadn't had to dig through the r-help digest first :-)
Tom Roche Mon, 27 Aug 2012 23:31:23 -0400
summary: I can successfully ncvar_put(...) data to a file, but when
I try to ncvar_get(...) the same data I get
Error in if (nc$var[[li]]$hasAddOffset) addOffset = nc$var[[li]]$addOffset else
addOffset = 0 :
argument is of length zero
https://stat.ethz.ch/pipermail/r-help/2012-August/322576.html
The following works fine for me:
* > nc <- nc_open("~/GEIA_N2O_oceanic.nc")
emi_n2o <- ncvar_get(nc, 'emi_n2o', start=c(1,1,1), count=c(-1,-1,1))
And that appears to have been the problem, since when I
-nc_open(netcdf.fp,
- write=FALSE,
- readunlim=TRUE)
+netcdf.file <- nc_open(netcdf.fp,
+ write=FALSE,
+ readunlim=TRUE)
I don't get the error when I subsequently ncvar_get, and the code @
https://github.com/TomRoche/GEIA_to_NetCDF
now works.
Dave Pierce: I assert that the current error is waaay too subtle:
- I don't get an error when I nc_open without assigning.
- I don't get an error when I ncvar_put to that file's datavar.
- I only get an error when I ncvar_get from the datavar.
- Nothing about the error text (IMHO) would lead one to the fix.
(Note also that neither ncvar_put or nc_close appear to require
assignment, which is probably what made me think I could nc_open
without assignment.)
Can ncdf4 be made to fail more helpfully? E.g., to fail immediately on
nc_open without assignment?
thanks again! Tom Roche
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
Roy Mendelssohn Tue, 28 Aug 2012 09:07:32 -0700
> here is the relevant section from your code:
> > netcdf.file <- nc_create(
> > filename=netcdf.fn,
> > # vars=list(emis.var),
> > # verbose=TRUE)
> > vars=list(emis.var))
> > # Write data to data variable: gotta have file first.
> > # Gotta convert 2d global.emis.mx[lat,lon] to 3d
> > global.emis.arr[time,lat,lon]
> > # Do this before adding _FillValue to prevent:
> > # > Error in R_nc4_put_vara_double: NetCDF: Not a valid data type or =
> _FillValue type mismatch
> > ## global.emis.arr <- global.emis.mx
> > ## dim(global.emis.arr) <- c(1, dim(global.emis.mx))
> > ## global.emis.arr[1,,] <- global.emis.mx
> > # Note
> > # * global.emis.mx[lat,lon]
> > # * datavar needs [lon, lat, time] (with time *last*)
> > ncvar_put(
> > nc=netcdf.file,
> > varid=emis.var,
> > # vals=global.emis.arr,
> > vals=t(global.emis.mx),
> > # start=rep.int(1, length(dim(global.emis.arr))),
> > start=c(1, 1, 1),
> > # count=dim(global.emis.arr))
> > count=c(-1,-1, 1)) # -1 -> all data
> You can't write until all dimensions have been defined, and all
> variables defined.
But in fact, that's only a part of the code ... which omits the prior
dimension and variable definitions :-( See the current version @
https://github.com/TomRoche/GEIA_to_netCDF/blob/c380c0a28dc8c71dbf0c2ba18130a2439a4fe089/GEIA.to.netCDF.r
I've also attached that (quoted) following my .sig, with the top-most
constant and function declarations removed for brevity. The dimension
definitions are prefixed with '1', the variable definition is prefixed
with '2'.
HTH, Tom Roche
current GEIA.to.netCDF.r code block follows to end of post
# code
> # process input
> library(maps) # on tlrPanP5 as well as clusters
> # input file path
> GEIA.emis.txt.fp <- sprintf('%s/%s', GEIA.emis.txt.dir, GEIA.emis.txt.fn)
> # columns are grid#, mass
> GEIA.emis.mx <-
> as.matrix(read.table(GEIA.emis.txt.fp, skip=GEIA.emis.txt.n.header))
> # mask zeros? no, use NA for non-ocean areas
> # GEIA.emis.mx[GEIA.emis.mx == 0] <- NA
> #
> dim(GEIA.emis.mx) ## [1] 36143 2
> # start debug
> GEIA.emis.mx.rows <- dim(GEIA.emis.mx)[1]
> if (GEIA.emis.mx.rows > GEIA.emis.grids.dim) {
> cat(sprintf('ERROR: %s: GEIA.emis.mx.rows=%.0d >
> GEIA.emis.grids.dim=%.0d\n',
> this.fn, GEIA.emis.mx.rows, GEIA.emis.grids.dim))
> } else {
> cat(sprintf('debug: %s: GEIA.emis.mx.rows < GEIA.emis.grids.dim\n',
> this.fn))
> }
> # end debug
> #
> global.emis.vec <-
> create.global.emissions.vector(
> GEIA.emis.grids.dim, GEIA.emis.mx.rows, GEIA.emis.mx)
> #
> # Need sorted lat and lon vectors: we know what those are a priori
> # Add 0.5 since grid centers
> lon.vec <- 0.5 +
> seq(from=grid.lon.degree.start, by=grid.lon.degree.per.cell,
> length.out=GEIA.emis.lon.dim)
> lat.vec <- 0.5 +
> seq(from=grid.lat.degree.start, by=grid.lat.degree.per.cell,
> length.out=GEIA.emis.lat.dim)
> # Create emissions matrix corresponding to those dimensional vectors
> # (i.e., global.emis.mx is the "projection" of global.emis.vec)
> # First, create empty global.emis.mx? No, fill from global.emis.vec.
> # Fill using byrow=T? or "bycol" == byrow=FALSE? (row=lat)
> # I assigned (using lon.lat.vec.to.grid.index)
> # "grid indices" (global.emis.vec.index values)
> # "lon-majorly" (i.e., iterate over lats before incrementing lon),
> # so we want to fill byrow=FALSE ... BUT,
> # that will "fill from top" (i.e., starting @ 90N) and
> # we want to "fill from bottom" (i.e., starting @ 90S) ...
> # global.emis.mx <- matrix(
> # global.emis.vec, nrow=GEIA.emis.lat.dim, ncol=GEIA.emis.lon.dim,
> # # so flip/reverse rows/latitudes when done
> # byrow=FALSE)[GEIA.emis.lat.dim:1,]
> # NO: I cannot just fill global.emis.mx from global.emis.vec:
> # latter's/GEIA's grid numbering system ensures 1000 lons per lat!
> # Which overflows the "real-spatial" global.emis.mx :-(
> # So I need to fill global.emis.mx using a for loop to decode the grid
> indices :-(
> # (but at least I can fill in whatever direction I want :-)
> global.emis.mx <- matrix(
> rep(NA, GEIA.emis.grids.dim), nrow=GEIA.emis.lat.dim,
> ncol=GEIA.emis.lon.dim)
> # 1: works if subsequently transposed: TODO: FIXME
> for (i.lon in 1:GEIA.emis.lon.dim) {
> for (i.lat in 1:GEIA.emis.lat.dim) {
> # 2: fails with 'dimensions of z are not length(x)(-1) times length(y)(-1)'
> # for (i.lat in 1:GEIA.emis.lat.dim) {
> # for (i.lon in 1:GEIA.emis.lon.dim) {
> # 3: fails with 'dimensions of z are not length(x)(-1) times length(y)(-1)'
> # for (i.lon in GEIA.emis.lon.dim:1) {
> # for (i.lat in GEIA.emis.lat.dim:1) {
> # 4: fails with 'dimensions of z are not length(x)(-1) times length(y)(-1)'
> # for (i.lat in GEIA.emis.lat.dim:1) {
> # for (i.lon in GEIA.emis.lon.dim:1) {
> lon <- lon.vec[i.lon]
> lat <- lat.vec[i.lat]
> GEIA
Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
> > - Nothing about the error text (IMHO) would lead one to the fix. > > (Note also that neither ncvar_put or nc_close appear to require > assignment, which is probably what made me think I could nc_open > without assignment.) > > Can ncdf4 be made to fail more helpfully? E.g., to fail immediately on > nc_open without assignment? > > thanks again! Tom Roche > Yeah, it's clear that the failure mode is confusing and unhelpful in the extreme. I'll see if there exists some way to do a better check for this. One of the many great things about users: they provide opportunities to add in more explanatory text for errors you never would have thought of. :) --Dave -- David W. Pierce Division of Climate, Atmospheric Science, and Physical Oceanography Scripps Institution of Oceanography, La Jolla, California, USA (858) 534-8276 (voice) / (858) 534-8561 (fax)dpie...@ucsd.edu [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
Pascal Oettli: MERCI BEAUCOUP! (though I would have thanked you
earlier if I hadn't had to dig through the r-help digest first :-)
Tom Roche Mon, 27 Aug 2012 23:31:23 -0400
>> summary: I can successfully ncvar_put(...) data to a file, but when
>> I try to ncvar_get(...) the same data I get
>> > Error in if (nc$var[[li]]$hasAddOffset) addOffset = nc$var[[li]]$addOffset
>> > else addOffset = 0 :
>> > argument is of length zero
https://stat.ethz.ch/pipermail/r-help/2012-August/322576.html
> The following works fine for me:
* > nc <- nc_open("~/GEIA_N2O_oceanic.nc")
> > emi_n2o <- ncvar_get(nc, 'emi_n2o', start=c(1,1,1), count=c(-1,-1,1))
And that appears to have been the problem, since when I
-nc_open(netcdf.fp,
- write=FALSE,
- readunlim=TRUE)
+netcdf.file <- nc_open(netcdf.fp,
+ write=FALSE,
+ readunlim=TRUE)
I don't get the error when I subsequently ncvar_get, and the code @
https://github.com/TomRoche/GEIA_to_NetCDF
now works.
Dave Pierce: I assert that the current error is waaay too subtle:
- I don't get an error when I nc_open without assigning.
- I don't get an error when I ncvar_put to that file's datavar.
- I only get an error when I ncvar_get from the datavar.
- Nothing about the error text (IMHO) would lead one to the fix.
(Note also that neither ncvar_put or nc_close appear to require
assignment, which is probably what made me think I could nc_open
without assignment.)
Can ncdf4 be made to fail more helpfully? E.g., to fail immediately on
nc_open without assignment?
thanks again! Tom Roche
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
On Aug 28, 2012, at 12:24 PM, Tom Roche wrote:
>
> Roy Mendelssohn Tue, 28 Aug 2012 09:07:32 -0700
>> here is the relevant section from your code:
>
>>> netcdf.file <- nc_create(
>>> filename=netcdf.fn,
>>> # vars=list(emis.var),
>>> # verbose=TRUE)
>>> vars=list(emis.var))
>
>>> # Write data to data variable: gotta have file first.
>>> # Gotta convert 2d global.emis.mx[lat,lon] to 3d
>>> global.emis.arr[time,lat,lon]
>>> # Do this before adding _FillValue to prevent:
>>> # > Error in R_nc4_put_vara_double: NetCDF: Not a valid data type or =
>> _FillValue type mismatch
>>> ## global.emis.arr <- global.emis.mx
>>> ## dim(global.emis.arr) <- c(1, dim(global.emis.mx))
>>> ## global.emis.arr[1,,] <- global.emis.mx
>
>>> # Note
>>> # * global.emis.mx[lat,lon]
>>> # * datavar needs [lon, lat, time] (with time *last*)
>
>>> ncvar_put(
>>> nc=netcdf.file,
>>> varid=emis.var,
>>> # vals=global.emis.arr,
>>> vals=t(global.emis.mx),
>>> # start=rep.int(1, length(dim(global.emis.arr))),
>>> start=c(1, 1, 1),
>>> # count=dim(global.emis.arr))
>>> count=c(-1,-1, 1)) # -1 -> all data
>
>> You can't write until all dimensions have been defined, and all
>> variables defined.
>
> But in fact, that's only a part of the code ... which omits the prior
> dimension and variable definitions :-( See the current version @
>
> https://github.com/TomRoche/GEIA_to_netCDF/blob/c380c0a28dc8c71dbf0c2ba18130a2439a4fe089/GEIA.to.netCDF.r
>
> I've also attached that (quoted) following my .sig, with the top-most
> constant and function declarations removed for brevity. The dimension
> definitions are prefixed with '1', the variable definition is prefixed
> with '2'.
>
> HTH, Tom Roche
> current GEIA.to.netCDF.r code block follows to end of post
> # code
>
>> # process input
>> library(maps) # on tlrPanP5 as well as clusters
>
>> # input file path
>> GEIA.emis.txt.fp <- sprintf('%s/%s', GEIA.emis.txt.dir, GEIA.emis.txt.fn)
>> # columns are grid#, mass
>> GEIA.emis.mx <-
>> as.matrix(read.table(GEIA.emis.txt.fp, skip=GEIA.emis.txt.n.header))
>> # mask zeros? no, use NA for non-ocean areas
>> # GEIA.emis.mx[GEIA.emis.mx == 0] <- NA
>> #
>> dim(GEIA.emis.mx) ## [1] 36143 2
>> # start debug
>> GEIA.emis.mx.rows <- dim(GEIA.emis.mx)[1]
>> if (GEIA.emis.mx.rows > GEIA.emis.grids.dim) {
>> cat(sprintf('ERROR: %s: GEIA.emis.mx.rows=%.0d >
>> GEIA.emis.grids.dim=%.0d\n',
>>this.fn, GEIA.emis.mx.rows, GEIA.emis.grids.dim))
>> } else {
>> cat(sprintf('debug: %s: GEIA.emis.mx.rows < GEIA.emis.grids.dim\n',
>>this.fn))
>> }
>> # end debug
>> #
>
>> global.emis.vec <-
>> create.global.emissions.vector(
>>GEIA.emis.grids.dim, GEIA.emis.mx.rows, GEIA.emis.mx)
>
>> #
>> # Need sorted lat and lon vectors: we know what those are a priori
>> # Add 0.5 since grid centers
>> lon.vec <- 0.5 +
>> seq(from=grid.lon.degree.start, by=grid.lon.degree.per.cell,
>> length.out=GEIA.emis.lon.dim)
>> lat.vec <- 0.5 +
>> seq(from=grid.lat.degree.start, by=grid.lat.degree.per.cell,
>> length.out=GEIA.emis.lat.dim)
>
>> # Create emissions matrix corresponding to those dimensional vectors
>> # (i.e., global.emis.mx is the "projection" of global.emis.vec)
>> # First, create empty global.emis.mx? No, fill from global.emis.vec.
>> # Fill using byrow=T? or "bycol" == byrow=FALSE? (row=lat)
>> # I assigned (using lon.lat.vec.to.grid.index)
>> # "grid indices" (global.emis.vec.index values)
>> # "lon-majorly" (i.e., iterate over lats before incrementing lon),
>> # so we want to fill byrow=FALSE ... BUT,
>> # that will "fill from top" (i.e., starting @ 90N) and
>> # we want to "fill from bottom" (i.e., starting @ 90S) ...
>> # global.emis.mx <- matrix(
>> # global.emis.vec, nrow=GEIA.emis.lat.dim, ncol=GEIA.emis.lon.dim,
>> # # so flip/reverse rows/latitudes when done
>> # byrow=FALSE)[GEIA.emis.lat.dim:1,]
>
>> # NO: I cannot just fill global.emis.mx from global.emis.vec:
>> # latter's/GEIA's grid numbering system ensures 1000 lons per lat!
>> # Which overflows the "real-spatial" global.emis.mx :-(
>> # So I need to fill global.emis.mx using a for loop to decode the grid
>> indices :-(
>> # (but at least I can fill in whatever direction I want :-)
>> global.emis.mx <- matrix(
>> rep(NA, GEIA.emis.grids.dim), nrow=GEIA.emis.lat.dim,
>> ncol=GEIA.emis.lon.dim)
>
>> # 1: works if subsequently transposed: TODO: FIXME
>> for (i.lon in 1:GEIA.emis.lon.dim) {
>> for (i.lat in 1:GEIA.emis.lat.dim) {
>> # 2: fails with 'dimensions of z are not length(x)(-1) times length(y)(-1)'
>> # for (i.lat in 1:GEIA.emis.lat.dim) {
>> # for (i.lon in 1:GEIA.emis.lon.dim) {
>> # 3: fails with 'dimensions of z are not length(x)(-1) times length(y)(-1)'
>> # for (i.lon in GEIA.emis.lon.dim:1) {
>> # for (i.lat in GEIA.emis.lat.dim:1) {
>> # 4: fails with 'dimensions of z are not length(x)(-1) times length(y)(-1)'
>> # for (i.lat in GEIA.emis.lat.dim:1) {
>> # for (i.lon in G
Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
Hi Tom: Sorry for top-posting. My bad etiquette for the day. The order that you do things when you create nercdf files matters a lot. Here is a section from the help for ncdf4: > If you want to WRITE data to a new netCDF file, the procedure is to first > define the dimensions your data array has, then define the variable, then > create the file. So, first call ncdim_def to define the dimensions that your > data exists along (for example, latitude, longitude, and time). Then call > ncvar_def to define a variable that uses those dimensions, and will hold your > data. Then callnc_create to create the netCDF file. Finally, call ncvar_put > to write your data to the newly created netCDF file, and nc_closewhen you are > done. And here is the relevant section from your code: > netcdf.file <- nc_create( > filename=netcdf.fn, > # vars=list(emis.var), > # verbose=TRUE) > vars=list(emis.var)) > > # Write data to data variable: gotta have file first. > # Gotta convert 2d global.emis.mx[lat,lon] to 3d global.emis.arr[time,lat,lon] > # Do this before adding _FillValue to prevent: > # > Error in R_nc4_put_vara_double: NetCDF: Not a valid data type or > _FillValue type mismatch > ## global.emis.arr <- global.emis.mx > ## dim(global.emis.arr) <- c(1, dim(global.emis.mx)) > ## global.emis.arr[1,,] <- global.emis.mx > > # Note > # * global.emis.mx[lat,lon] > # * datavar needs [lon, lat, time] (with time *last*) > > ncvar_put( > nc=netcdf.file, > varid=emis.var, > # vals=global.emis.arr, > vals=t(global.emis.mx), > # start=rep.int(1, length(dim(global.emis.arr))), > start=c(1, 1, 1), > # count=dim(global.emis.arr)) > count=c(-1,-1, 1)) # -1 -> all data You can't write until all dimensions have been defined, and all variables defined. HTH, -Roy On Aug 28, 2012, at 4:29 AM, Tom Roche wrote: > > Tom Roche Mon, 27 Aug 2012 23:31:23 -0400 >>> summary: I can successfully ncvar_put(...) data to a file, but when >>> I try to ncvar_get(...) the same data > > in the one and only datavar > >>> I get > Error in if (nc$var[[li]]$hasAddOffset) addOffset = nc$var[[li]]$addOffset else addOffset = 0 : argument is of length zero > > David W. Pierce Mon, 27 Aug 2012 21:35:35 -0700 >> If you want to create a file and write to it, and then read back in from >> that same file, close it first and then reopen it. > > So nc_sync is not enough--good to know. But ... > > Unfortunately I do that (nc_close then nc_open) in the latest code @ > > https://github.com/TomRoche/GEIA_to_NetCDF > > (direct link to relevant file= > > https://github.com/TomRoche/GEIA_to_netCDF/blob/master/GEIA.to.netCDF.r > > ) but no fix--same error. Any other suggestions? > > your assistance is appreciated, Tom Roche > > ___ > netcdfgroup mailing list > netcdfgr...@unidata.ucar.edu > For list information or to unsubscribe, visit: > http://www.unidata.ucar.edu/mailing_lists/ ** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center 1352 Lighthouse Avenue Pacific Grove, CA 93950-2097 e-mail: roy.mendelss...@noaa.gov (Note new e-mail address) voice: (831)-648-9029 fax: (831)-648-8440 www: http://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
Tom Roche Mon, 27 Aug 2012 23:31:23 -0400 >> summary: I can successfully ncvar_put(...) data to a file, but when >> I try to ncvar_get(...) the same data in the one and only datavar >> I get >> > Error in if (nc$var[[li]]$hasAddOffset) addOffset = nc$var[[li]]$addOffset >> > else addOffset = 0 : >> > argument is of length zero David W. Pierce Mon, 27 Aug 2012 21:35:35 -0700 > If you want to create a file and write to it, and then read back in from that > same file, close it first and then reopen it. So nc_sync is not enough--good to know. But ... Unfortunately I do that (nc_close then nc_open) in the latest code @ https://github.com/TomRoche/GEIA_to_NetCDF (direct link to relevant file= https://github.com/TomRoche/GEIA_to_netCDF/blob/master/GEIA.to.netCDF.r ) but no fix--same error. Any other suggestions? your assistance is appreciated, Tom Roche __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
> On Mon, Aug 27, 2012 at 8:40 PM, Roy Mendelssohn wrote:
> Just looked at your code, and you do call nc_sync. I am wondering if it
works if you call nc_close, reopen and > then write. Perhaps the nc_sync is
sufficent when you have been in define mode in the nc_create call.
>
> -Roy
There's always some awkwardness because the netcdf_id object doesn't "know"
about changes to the file you've made after creating it. If you could pass
objects by reference you could modify the netcdf_id object on each call to
ncvar_put, ncatt_put, etc., and the netcdf_id object would always be
synchronized with the state of the file. But with R semantics the most
straightforward way to do this would be something like this:
ncid <- nc_create(...)
ncid <- ncvar_put( ncid, ... )
ncid <- ncatt_put( ncid, ... )
etc. which is awkward and ugly. So facing the choice between an awkward and
ugly solution that has to be done *every* *single* *time* (passing the ncid
in and passing the changed ncid back out on each call) and an awkward and
ugly solution that is rare (having to close a file first if you want to
create it, modify it, and then read it), I chose the latter.
Or, the short form: If you want to create a file and write to it, and then
read back in from that same file, close it first and then reopen it.
Regards,
--Dave
> On Aug 27, 2012, at 8:31 PM, Tom Roche wrote:
>
> >
> > summary: I can successfully ncvar_put(...) data to a file, but when I
> > try to ncvar_get(...) the same data, I get
> >
> >> Error in if (nc$var[[li]]$hasAddOffset) addOffset =
> nc$var[[li]]$addOffset else addOffset = 0 :
> >> argument is of length zero
> >
> > How to fix or debug?
> >
> > details:
> >
> > R code @
> >
> > https://github.com/TomRoche/GEIA_to_NetCDF
> >
> > successfully (if crudely) uses R packages={ncdf4, maps, fields} to
> >
> > + extract data from a GEIA-distributed datafile
> >
> > https://github.com/downloads/TomRoche/GEIA_to_netCDF/N2OOC90Y.1A
> >
> > + display the data (mostly successfully--the map's legend has problems
> > which I'll attack later)
> >
> > https://github.com/downloads/TomRoche/GEIA_to_netCDF/output.1.png
> >
> > + create a netCDF file using the data read from the GEIA file. (At
> > least, after nc_sync(netcdf.file), the file `ncdump -h`s properly.)
> >
> > However, I can only *put* the data to the netCDF file:
> >
> >> ncvar_put(
> >> + nc=netcdf.file,
> >> + varid=emis.var,
> >> + vals=t(global.emis.mx),
> >> + start=c(1, 1, 1),
> >> + count=c(-1,-1, 1)) # -1 -> all data
> >
> > When I try to *pull* the data *from* the netCDF I created,
> >
> >>> target.data <- ncvar_get(
> >> + nc=netcdf.file,
> >> + varid=emis.var,
> >> + # read all the data
> >> + start=rep(1, emis.var$ndims),
> >> + # count=rep(-1, emis.var$ndims))
> >
> > I get
> >
> >> Error in if (nc$var[[li]]$hasAddOffset) addOffset =
> nc$var[[li]]$addOffset else addOffset = 0 :
> >> argument is of length zero
> >
> > And I get the same error if I try the minor variation(s)
> >
> >> target.data <- ncvar_get(
> > + nc=netcdf.file,
> > + # varid=emis.var,
> > + varid=emis.var.name,
> > + # read all the data
> > + start=rep(1, emis.var$ndims),
> > + count=c(-1, -1, 1))
> >
> > But the data itself appears to be OK--at least, it virtualizes
> > properly (above). So I'm thinking I must just be missing something
> > simple, and hoping Someone Out There with fresh eyeballs can point to
> > my error(s).
> >
> > (And, in case you're wondering:
> >
> > - I'm not just ncvar_put'ing the data for the exercise: I want the
> > GEIA data in netCDF format for subsequent use.
> >
> > - I tried to find the GEIA data distributed in netCDF format, and
> > asked around, but have gotten no responses.
> >
> > ) TIA, Tom Roche
> >
> > ___
> > netcdfgroup mailing list
> > netcdfgr...@unidata.ucar.edu
> > For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
> **
> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> **
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> 1352 Lighthouse Avenue
> Pacific Grove, CA 93950-2097
>
> e-mail: roy.mendelss...@noaa.gov (Note new e-mail address)
> voice: (831)-648-9029
> fax: (831)-648-8440
> www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK
> Jr.
>
> ___
> netcdfgroup mailing list
> netcdfgr...@unidata.ucar.edu
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
--
David W. Pierce
Division of Climate, Atmospheric Science, and Physical Oceanography
Scripps Institution of Oceanography, La
Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
Just looked at your code, and you do call nc_sync. I am wondering if it works
if you call nc_close, reopen and then write. Perhaps the nc_sync is sufficent
when you have been in define mode in the nc_create call.
-Roy
On Aug 27, 2012, at 8:31 PM, Tom Roche wrote:
>
> summary: I can successfully ncvar_put(...) data to a file, but when I
> try to ncvar_get(...) the same data, I get
>
>> Error in if (nc$var[[li]]$hasAddOffset) addOffset = nc$var[[li]]$addOffset
>> else addOffset = 0 :
>> argument is of length zero
>
> How to fix or debug?
>
> details:
>
> R code @
>
> https://github.com/TomRoche/GEIA_to_NetCDF
>
> successfully (if crudely) uses R packages={ncdf4, maps, fields} to
>
> + extract data from a GEIA-distributed datafile
>
> https://github.com/downloads/TomRoche/GEIA_to_netCDF/N2OOC90Y.1A
>
> + display the data (mostly successfully--the map's legend has problems
> which I'll attack later)
>
> https://github.com/downloads/TomRoche/GEIA_to_netCDF/output.1.png
>
> + create a netCDF file using the data read from the GEIA file. (At
> least, after nc_sync(netcdf.file), the file `ncdump -h`s properly.)
>
> However, I can only *put* the data to the netCDF file:
>
>> ncvar_put(
>> + nc=netcdf.file,
>> + varid=emis.var,
>> + vals=t(global.emis.mx),
>> + start=c(1, 1, 1),
>> + count=c(-1,-1, 1)) # -1 -> all data
>
> When I try to *pull* the data *from* the netCDF I created,
>
>>> target.data <- ncvar_get(
>> + nc=netcdf.file,
>> + varid=emis.var,
>> + # read all the data
>> + start=rep(1, emis.var$ndims),
>> + # count=rep(-1, emis.var$ndims))
>
> I get
>
>> Error in if (nc$var[[li]]$hasAddOffset) addOffset = nc$var[[li]]$addOffset
>> else addOffset = 0 :
>> argument is of length zero
>
> And I get the same error if I try the minor variation(s)
>
>> target.data <- ncvar_get(
> + nc=netcdf.file,
> + # varid=emis.var,
> + varid=emis.var.name,
> + # read all the data
> + start=rep(1, emis.var$ndims),
> + count=c(-1, -1, 1))
>
> But the data itself appears to be OK--at least, it virtualizes
> properly (above). So I'm thinking I must just be missing something
> simple, and hoping Someone Out There with fresh eyeballs can point to
> my error(s).
>
> (And, in case you're wondering:
>
> - I'm not just ncvar_put'ing the data for the exercise: I want the
> GEIA data in netCDF format for subsequent use.
>
> - I tried to find the GEIA data distributed in netCDF format, and
> asked around, but have gotten no responses.
>
> ) TIA, Tom Roche
>
> ___
> netcdfgroup mailing list
> netcdfgr...@unidata.ucar.edu
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
**
"The contents of this message do not reflect any position of the U.S.
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097
e-mail: roy.mendelss...@noaa.gov (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/
"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] [netcdfgroup] [ncdf4] error converting GEIA data to netCDF
If I had to guess, you did not call nc_close or nc_sync. In netcdf, the data
is actually really written until that is called.
-Roy
On Aug 27, 2012, at 8:31 PM, Tom Roche wrote:
>
> summary: I can successfully ncvar_put(...) data to a file, but when I
> try to ncvar_get(...) the same data, I get
>
>> Error in if (nc$var[[li]]$hasAddOffset) addOffset = nc$var[[li]]$addOffset
>> else addOffset = 0 :
>> argument is of length zero
>
> How to fix or debug?
>
> details:
>
> R code @
>
> https://github.com/TomRoche/GEIA_to_NetCDF
>
> successfully (if crudely) uses R packages={ncdf4, maps, fields} to
>
> + extract data from a GEIA-distributed datafile
>
> https://github.com/downloads/TomRoche/GEIA_to_netCDF/N2OOC90Y.1A
>
> + display the data (mostly successfully--the map's legend has problems
> which I'll attack later)
>
> https://github.com/downloads/TomRoche/GEIA_to_netCDF/output.1.png
>
> + create a netCDF file using the data read from the GEIA file. (At
> least, after nc_sync(netcdf.file), the file `ncdump -h`s properly.)
>
> However, I can only *put* the data to the netCDF file:
>
>> ncvar_put(
>> + nc=netcdf.file,
>> + varid=emis.var,
>> + vals=t(global.emis.mx),
>> + start=c(1, 1, 1),
>> + count=c(-1,-1, 1)) # -1 -> all data
>
> When I try to *pull* the data *from* the netCDF I created,
>
>>> target.data <- ncvar_get(
>> + nc=netcdf.file,
>> + varid=emis.var,
>> + # read all the data
>> + start=rep(1, emis.var$ndims),
>> + # count=rep(-1, emis.var$ndims))
>
> I get
>
>> Error in if (nc$var[[li]]$hasAddOffset) addOffset = nc$var[[li]]$addOffset
>> else addOffset = 0 :
>> argument is of length zero
>
> And I get the same error if I try the minor variation(s)
>
>> target.data <- ncvar_get(
> + nc=netcdf.file,
> + # varid=emis.var,
> + varid=emis.var.name,
> + # read all the data
> + start=rep(1, emis.var$ndims),
> + count=c(-1, -1, 1))
>
> But the data itself appears to be OK--at least, it virtualizes
> properly (above). So I'm thinking I must just be missing something
> simple, and hoping Someone Out There with fresh eyeballs can point to
> my error(s).
>
> (And, in case you're wondering:
>
> - I'm not just ncvar_put'ing the data for the exercise: I want the
> GEIA data in netCDF format for subsequent use.
>
> - I tried to find the GEIA data distributed in netCDF format, and
> asked around, but have gotten no responses.
>
> ) TIA, Tom Roche
>
> ___
> netcdfgroup mailing list
> netcdfgr...@unidata.ucar.edu
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
**
"The contents of this message do not reflect any position of the U.S.
Government or NOAA."
**
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097
e-mail: roy.mendelss...@noaa.gov (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/
"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
