Re: problem with netCDF4 OpenDAP
On 08/13/2015 05:55 PM, Jason Swails wrote: On Thu, Aug 13, 2015 at 6:32 AM, Tom P werot...@freent.dd mailto:werot...@freent.dd wrote: I'm having a problem trying to access OpenDAP files using netCDF4. The netCDF4 is installed from the Anaconda package. According to their changelog, openDAP is supposed to be supported. netCDF4.__version__ Out[7]: '1.1.8' Here's some code: url = 'http://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc' nc = netCDF4.Dataset(url) I get the error - netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Dataset.__init__ (netCDF4/_netCDF4.c:9551)() RuntimeError: NetCDF: file not found However if I download the same file, it works - url = '/home/tom/Downloads/ersst.201507.nc http://ersst.201507.nc' nc = netCDF4.Dataset(url) print nc . . . . Is it something I'm doing wrong? Yes. URLs are not files and cannot be opened like normal files. netCDF4 *requires* a local file as far as I can tell. All the best, Jason -- Jason M. Swails BioMaPS, Rutgers University Postdoctoral Researcher Thanks for the reply but that is not what the documentation says. http://unidata.github.io/netcdf4-python/#section8 Remote OPeNDAP-hosted datasets can be accessed for reading over http if a URL is provided to the netCDF4.Dataset constructor instead of a filename. However, this requires that the netCDF library be built with OPenDAP support, via the --enable-dap configure option (added in version 4.0.1). and for the Anaconda package - http://docs.continuum.io/anaconda/changelog 2013-05-08: 1.5.0: Highlights: updates to all important packages: python, numpy, scipy, ipython, matplotlib, pandas, cython added netCDF4 (with OpenDAP support) on Linux and MacOSX -- https://mail.python.org/mailman/listinfo/python-list
Re: problem with netCDF4 OpenDAP
On 08/14/2015 03:15 PM, Jason Swails wrote:
On Aug 14, 2015, at 3:18 AM, Tom P werot...@freent.dd wrote:
Thanks for the reply but that is not what the documentation says.
http://unidata.github.io/netcdf4-python/#section8
Remote OPeNDAP-hosted datasets can be accessed for reading over http if a URL
is provided to the netCDF4.Dataset constructor instead of a filename. However, this
requires that the netCDF library be built with OPenDAP support, via the --enable-dap
configure option (added in version 4.0.1).”
Huh, so it does. Your error message says file not found, though, which
suggested to me that it's trying to interpret the NetCDF file as a local file instead of
a URL. Indeed, when I run that example, the traceback is more complete (the traceback
you printed had omitted some information):
netCDF4.Dataset('http://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc')
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or
SCAN_ERROR
context: !DOCTYPE^ HTML PUBLIC -//IETF//DTD HTML 2.0//ENhtmlheadtitle404 Not
Found/title/headbodyh1Not Found/h1pThe requested URL /pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc.dds was not found
on this server./p/body/html
Traceback (most recent call last):
File stdin, line 1, in module
File netCDF4/_netCDF4.pyx, line 1547, in netCDF4._netCDF4.Dataset.__init__
(netCDF4/_netCDF4.c:9551)
RuntimeError: NetCDF: file not found
So it’s clear that netCDF4 is at least *trying* to go online to look for the file,
but it simply can’t find it. Since the docs say it’s linking to libcurl, I tried
using curl to download the file (curl -#
http://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc
test.nc) and it worked fine. What’s more, it *seems* like the file
(/pub/.../ersst.201507.nc.dds) was decorated with the ‘.dds’ suffix for some
reason (not sure if the server redirected the request there or not). But this
looks like a netCDF4 issue. Perhaps you can go to their project page on Github
and file an issue there -- they will be more likely to have your answer than
people here.
HTH,
Jason
Hi,
yes the file does appear to be there, I can download it and I can
open and read the URL using urllib. Since there are a whole bunch of
files in the directory, I really need MFDataset, but according to the
documentation that doesn't work with URLs. Maybe the solution really is
to D/L them all into a temporary folder and use MFDataset.
and for the Anaconda package -
http://docs.continuum.io/anaconda/changelog
2013-05-08: 1.5.0:
Highlights:
updates to all important packages: python, numpy, scipy, ipython, matplotlib,
pandas, cython
added netCDF4 (with OpenDAP support) on Linux and MacOSX
--
https://mail.python.org/mailman/listinfo/python-list
--
Jason M. Swails
BioMaPS,
Rutgers University
Postdoctoral Researcher
--
https://mail.python.org/mailman/listinfo/python-list
Re: problem with netCDF4 OpenDAP
Tom P wrote: yes the file does appear to be there, I can download it and I can open and read the URL using urllib. Since there are a whole bunch of files in the directory, I really need MFDataset, but according to the documentation that doesn't work with URLs. Maybe the solution really is to D/L them all into a temporary folder and use MFDataset. Not sure about the size and other aspects of your deployment. But the safest way to backup an OpenLDAP database is to export it to a single LDIF file because this can be done while slapd is running and it's guaranteed that the LDIF contains only data of finished transactions. Ciao, Michael. -- https://mail.python.org/mailman/listinfo/python-list
Re: problem with netCDF4 OpenDAP
On Aug 14, 2015, at 3:18 AM, Tom P werot...@freent.dd wrote:
Thanks for the reply but that is not what the documentation says.
http://unidata.github.io/netcdf4-python/#section8
Remote OPeNDAP-hosted datasets can be accessed for reading over http if a
URL is provided to the netCDF4.Dataset constructor instead of a filename.
However, this requires that the netCDF library be built with OPenDAP support,
via the --enable-dap configure option (added in version 4.0.1).”
Huh, so it does. Your error message says file not found, though, which
suggested to me that it's trying to interpret the NetCDF file as a local file
instead of a URL. Indeed, when I run that example, the traceback is more
complete (the traceback you printed had omitted some information):
netCDF4.Dataset('http://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc')
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or
SCAN_ERROR
context: !DOCTYPE^ HTML PUBLIC -//IETF//DTD HTML
2.0//ENhtmlheadtitle404 Not Found/title/headbodyh1Not
Found/h1pThe requested URL
/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc.dds was not found on this
server./p/body/html
Traceback (most recent call last):
File stdin, line 1, in module
File netCDF4/_netCDF4.pyx, line 1547, in netCDF4._netCDF4.Dataset.__init__
(netCDF4/_netCDF4.c:9551)
RuntimeError: NetCDF: file not found
So it’s clear that netCDF4 is at least *trying* to go online to look for the
file, but it simply can’t find it. Since the docs say it’s linking to libcurl,
I tried using curl to download the file (curl -#
http://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc
test.nc) and it worked fine. What’s more, it *seems* like the file
(/pub/.../ersst.201507.nc.dds) was decorated with the ‘.dds’ suffix for some
reason (not sure if the server redirected the request there or not). But this
looks like a netCDF4 issue. Perhaps you can go to their project page on Github
and file an issue there -- they will be more likely to have your answer than
people here.
HTH,
Jason
and for the Anaconda package -
http://docs.continuum.io/anaconda/changelog
2013-05-08: 1.5.0:
Highlights:
updates to all important packages: python, numpy, scipy, ipython,
matplotlib, pandas, cython
added netCDF4 (with OpenDAP support) on Linux and MacOSX
--
https://mail.python.org/mailman/listinfo/python-list
--
Jason M. Swails
BioMaPS,
Rutgers University
Postdoctoral Researcher
--
https://mail.python.org/mailman/listinfo/python-list
Re: problem with netCDF4 OpenDAP
On Thu, Aug 13, 2015 at 6:32 AM, Tom P werot...@freent.dd wrote: I'm having a problem trying to access OpenDAP files using netCDF4. The netCDF4 is installed from the Anaconda package. According to their changelog, openDAP is supposed to be supported. netCDF4.__version__ Out[7]: '1.1.8' Here's some code: url = ' http://www1.ncdc.noaa.gov/pub/data/cmb/ersst/v3b/netcdf/ersst.201507.nc' nc = netCDF4.Dataset(url) I get the error - netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Dataset.__init__ (netCDF4/_netCDF4.c:9551)() RuntimeError: NetCDF: file not found However if I download the same file, it works - url = '/home/tom/Downloads/ersst.201507.nc' nc = netCDF4.Dataset(url) print nc . . . . Is it something I'm doing wrong? Yes. URLs are not files and cannot be opened like normal files. netCDF4 *requires* a local file as far as I can tell. All the best, Jason -- Jason M. Swails BioMaPS, Rutgers University Postdoctoral Researcher -- https://mail.python.org/mailman/listinfo/python-list
