Re: [CF-metadata] machine-accessible CF compliance checker?
Hi Stephen, I've been out of the office the last couple of days and not seen this till now. #3 is already on my to-do list :-) #2 I actually do this quite a bit and just supply the ncdump -c ouput passed through ncgen and ignore the couple of errors for the checks that can't be performed. If this would be useful for others then it should be relatively easy to officially add a headers-only option. Off the top of my head I think there are 2 checks that require the variable data (1) That variable data lies within cell boundaries and (2) The check that coord data is monotonic I also plan in the next 6months to completely re-write the CF checker, it's developed a lot since I first wrote it and is now becoming rather unwieldy. One thing I'd like to do is remove the dependency on CDAT. So if anyone has any further suggestions for improvements or additional functionality they'd like to see please let me know. Regards, Ros. On 29/02/12 15:31, stephen.pas...@stfc.ac.uk wrote: Hi Phil, So far my role has simply been to package and deploy Rosalyn's cfchecker code, therefore I really don't know to what extent it needs the real NetCDF. I think an alternative checker (i.e. codebase) is a bad idea because they might give contradictory results. I would suggest several options: 1. Work on making the cfchecker package easier to install so that users can run it on their real NetCDF. A user would have to know how to install python packages, compile UDUNITS and edit a config file. 2. Add a headers-only feature to Rosalyn's code so that user's can create dummy NetCDF with ncgen to upload to one of the online checkers. 3. As an addition to #2, add a feature to Rosalyn's code, or the CGI wrapper, to upload CDL. I am willing to do #1, if people think it would meet user's needs, but it would have to be added to my ever-expanding TODO list :-). Rosalyn, do you have any comments on #23? Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK -Original Message- From: Bentley, Philip [mailto:philip.bent...@metoffice.gov.uk] Sent: 29 February 2012 14:56 To: Pascoe, Stephen (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu Subject: RE: [CF-metadata] machine-accessible CF compliance checker? Hi Stephen, I may well be wrong, but I suspect that one of the reasons - perhaps the main reason - why folks would like access to a stand-alone cfchecker utility is the large size of their netcdf files. (It's certainly the reason we use a local version.) As you know only too well (from CMIP5), most of the netcdf files we're dealing with these days run into the high 100s of MB or low GBs. It's not really practical to upload such files in order to perform a CF compliance check. At least not on more than a few files. I wonder, therefore, if a possible solution is to develop an alternative checker utility that operates not on the netcdf file itself, but rather on the header info output by the ncdump -h command. Generating and uploading those would be trivial by comparison. Off the top of my head (and having given it almost zero thought!) the only test that I can think of which examines the actual data values is the check for coord vars being monotonic. But I suspect there are others I'm unaware of. If nothing else, such a utility might have some value as a 'first pass' checker. Any mileage in this? Apologies if it's already done the rounds! Regards, Phil -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of stephen.pas...@stfc.ac.uk Sent: 29 February 2012 11:39 To: christopher.s.lyn...@nasa.gov; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] machine-accessible CF compliance checker? Hi Christopher, Are you looking for a way to run the cfchecker locally or submit checking requests to one of the websites in a script? I think both would be possible with a little help. The cfchecker is now available on the Python Package Index at http://pypi.python.org/pypi/cfchecker. It needs a little help to work out of the box -- you need to configure UDUNITS and the location of several CF XML files. It's on my long-term TODO list to make this configuration easier. Also the CF-checker at BADC and NCAS-CMS has a very simple HTTP interface that could be scripted. Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Lynnes, Christopher S. (GSFC-6102) Sent: 28 February 2012 16:03 To: cf-metadata@cgd.ucar.edu Subject: [CF-metadata] machine-accessible CF compliance checker? I know about the web sites where you use a form to upload files to check CF compliance, but is there a service somewhere
Re: [CF-metadata] machine-accessible CF compliance checker?
Great, I had further thoughts last night and the logical way forward is to get the CGI script to generate a dummy netcdf file from CDL submitted in the web form. An official headers-only option would definitely help. Ros, is the CGM CGI script written in perl? Ours is but I have no idea whether we share the same code. I think it makes sense to share the same codebase if we are going to add features to the CGI -- my preference is obviously Python. Remove CDAT: +1 from me. I would recommend netcdf4-python as a better binding. I believe the various Python tooling efforts happening in the UK (cf-python, IRIS) are using netcdf4-python so it would consolidate our toolset. Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK From: Rosalyn Hatcher [mailto:r.s.hatc...@reading.ac.uk] Sent: 01 March 2012 08:32 To: Pascoe, Stephen (STFC,RAL,RALSP) Cc: philip.bent...@metoffice.gov.uk; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] machine-accessible CF compliance checker? Hi Stephen, I've been out of the office the last couple of days and not seen this till now. #3 is already on my to-do list :-) #2 I actually do this quite a bit and just supply the ncdump -c ouput passed through ncgen and ignore the couple of errors for the checks that can't be performed. If this would be useful for others then it should be relatively easy to officially add a headers-only option. Off the top of my head I think there are 2 checks that require the variable data (1) That variable data lies within cell boundaries and (2) The check that coord data is monotonic I also plan in the next 6months to completely re-write the CF checker, it's developed a lot since I first wrote it and is now becoming rather unwieldy. One thing I'd like to do is remove the dependency on CDAT. So if anyone has any further suggestions for improvements or additional functionality they'd like to see please let me know. Regards, Ros. On 29/02/12 15:31, stephen.pas...@stfc.ac.ukmailto:stephen.pas...@stfc.ac.uk wrote: Hi Phil, So far my role has simply been to package and deploy Rosalyn's cfchecker code, therefore I really don't know to what extent it needs the real NetCDF. I think an alternative checker (i.e. codebase) is a bad idea because they might give contradictory results. I would suggest several options: 1. Work on making the cfchecker package easier to install so that users can run it on their real NetCDF. A user would have to know how to install python packages, compile UDUNITS and edit a config file. 2. Add a headers-only feature to Rosalyn's code so that user's can create dummy NetCDF with ncgen to upload to one of the online checkers. 3. As an addition to #2, add a feature to Rosalyn's code, or the CGI wrapper, to upload CDL. I am willing to do #1, if people think it would meet user's needs, but it would have to be added to my ever-expanding TODO list :-). Rosalyn, do you have any comments on #23? Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK -Original Message- From: Bentley, Philip [mailto:philip.bent...@metoffice.gov.uk] Sent: 29 February 2012 14:56 To: Pascoe, Stephen (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edumailto:cf-metadata@cgd.ucar.edu Subject: RE: [CF-metadata] machine-accessible CF compliance checker? Hi Stephen, I may well be wrong, but I suspect that one of the reasons - perhaps the main reason - why folks would like access to a stand-alone cfchecker utility is the large size of their netcdf files. (It's certainly the reason we use a local version.) As you know only too well (from CMIP5), most of the netcdf files we're dealing with these days run into the high 100s of MB or low GBs. It's not really practical to upload such files in order to perform a CF compliance check. At least not on more than a few files. I wonder, therefore, if a possible solution is to develop an alternative checker utility that operates not on the netcdf file itself, but rather on the header info output by the ncdump -h command. Generating and uploading those would be trivial by comparison. Off the top of my head (and having given it almost zero thought!) the only test that I can think of which examines the actual data values is the check for coord vars being monotonic. But I suspect there are others I'm unaware of. If nothing else, such a utility might have some value as a 'first pass' checker. Any mileage in this? Apologies if it's already done the rounds! Regards, Phil -Original Message- From: cf-metadata-boun...@cgd.ucar.edumailto:cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of stephen.pas...@stfc.ac.ukmailto:stephen.pas...@stfc.ac.uk Sent: 29
Re: [CF-metadata] machine-accessible CF compliance checker?
Hi Stephen, Yes, I agree and I think it would be pretty easy to add in the conversion of a CDL to netcdf. I'm going to have to make one more release of the current incarnation of the checker for CF-1.6 so I'll have a look at this at the same time. The CGI script is indeed written in perl. Ag originally wrote the web interface IIRC. The interface on PUMA definitely started out life as the same as the BADC one. Again it was supposed to be making its way into the subversion repository, but never made it. netcdf4-python was what I was looking at for the replacement for CDAT. So it's good that you recommend it too. In terms of the web interface, that would also get redone and I have some ideas on this too, which you'll be glad to hear would be python. :-) Cheers, Ros. On 01/03/12 09:59, stephen.pas...@stfc.ac.uk wrote: Great, I had further thoughts last night and the logical way forward is to get the CGI script to generate a dummy netcdf file from CDL submitted in the web form. An official headers-only option would definitely help. Ros, is the CGM CGI script written in perl? Ours is but I have no idea whether we share the same code. I think it makes sense to share the same codebase if we are going to add features to the CGI -- my preference is obviously Python. Remove CDAT: +1 from me. I would recommend netcdf4-python as a better binding. I believe the various Python tooling efforts happening in the UK (cf-python, IRIS) are using netcdf4-python so it would consolidate our toolset. Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK *From:*Rosalyn Hatcher [mailto:r.s.hatc...@reading.ac.uk] *Sent:* 01 March 2012 08:32 *To:* Pascoe, Stephen (STFC,RAL,RALSP) *Cc:* philip.bent...@metoffice.gov.uk; cf-metadata@cgd.ucar.edu *Subject:* Re: [CF-metadata] machine-accessible CF compliance checker? Hi Stephen, I've been out of the office the last couple of days and not seen this till now. #3 is already on my to-do list :-) #2 I actually do this quite a bit and just supply the ncdump -c ouput passed through ncgen and ignore the couple of errors for the checks that can't be performed. If this would be useful for others then it should be relatively easy to officially add a headers-only option. Off the top of my head I think there are 2 checks that require the variable data (1) That variable data lies within cell boundaries and (2) The check that coord data is monotonic I also plan in the next 6months to completely re-write the CF checker, it's developed a lot since I first wrote it and is now becoming rather unwieldy. One thing I'd like to do is remove the dependency on CDAT. So if anyone has any further suggestions for improvements or additional functionality they'd like to see please let me know. Regards, Ros. On 29/02/12 15:31, stephen.pas...@stfc.ac.uk mailto:stephen.pas...@stfc.ac.uk wrote: Hi Phil, So far my role has simply been to package and deploy Rosalyn's cfchecker code, therefore I really don't know to what extent it needs the real NetCDF. I think an alternative checker (i.e. codebase) is a bad idea because they might give contradictory results. I would suggest several options: 1. Work on making the cfchecker package easier to install so that users can run it on their real NetCDF. A user would have to know how to install python packages, compile UDUNITS and edit a config file. 2. Add a headers-only feature to Rosalyn's code so that user's can create dummy NetCDF with ncgen to upload to one of the online checkers. 3. As an addition to #2, add a feature to Rosalyn's code, or the CGI wrapper, to upload CDL. I am willing to do #1, if people think it would meet user's needs, but it would have to be added to my ever-expanding TODO list :-). Rosalyn, do you have any comments on #23? Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK -Original Message- From: Bentley, Philip [mailto:philip.bent...@metoffice.gov.uk] Sent: 29 February 2012 14:56 To: Pascoe, Stephen (STFC,RAL,RALSP);cf-metadata@cgd.ucar.edu mailto:cf-metadata@cgd.ucar.edu Subject: RE: [CF-metadata] machine-accessible CF compliance checker? Hi Stephen, I may well be wrong, but I suspect that one of the reasons - perhaps the main reason - why folks would like access to a stand-alone cfchecker utility is the large size of their netcdf files. (It's certainly the reason we use a local version.) As you know only too well (from CMIP5), most of the netcdf files we're dealing with these days run into the high 100s of MB or low GBs. It's not really practical to upload such files in order to perform a CF compliance check. At least not on more than a few files. I wonder, therefore, if a possible solution
Re: [CF-metadata] machine-accessible CF compliance checker?
Hi all: If the checker can read from opendap, then putting the files on an opendap server might be helpful, in that only the metadata is read instead of the entire file. There are a few difference among servers (Hyrax, TDS, GDS, etc) but im pretty sure this would work for netcdf files well enough for the checker's needs. could be worth finding out. john On 2/29/2012 7:55 AM, Bentley, Philip wrote: Hi Stephen, I may well be wrong, but I suspect that one of the reasons - perhaps the main reason - why folks would like access to a stand-alone cfchecker utility is the large size of their netcdf files. (It's certainly the reason we use a local version.) As you know only too well (from CMIP5), most of the netcdf files we're dealing with these days run into the high 100s of MB or low GBs. It's not really practical to upload such files in order to perform a CF compliance check. At least not on more than a few files. I wonder, therefore, if a possible solution is to develop an alternative checker utility that operates not on the netcdf file itself, but rather on the header info output by the ncdump -h command. Generating and uploading those would be trivial by comparison. Off the top of my head (and having given it almost zero thought!) the only test that I can think of which examines the actual data values is the check for coord vars being monotonic. But I suspect there are others I'm unaware of. If nothing else, such a utility might have some value as a 'first pass' checker. Any mileage in this? Apologies if it's already done the rounds! Regards, Phil -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of stephen.pas...@stfc.ac.uk Sent: 29 February 2012 11:39 To: christopher.s.lyn...@nasa.gov; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] machine-accessible CF compliance checker? Hi Christopher, Are you looking for a way to run the cfchecker locally or submit checking requests to one of the websites in a script? I think both would be possible with a little help. The cfchecker is now available on the Python Package Index at http://pypi.python.org/pypi/cfchecker. It needs a little help to work out of the box -- you need to configure UDUNITS and the location of several CF XML files. It's on my long-term TODO list to make this configuration easier. Also the CF-checker at BADC and NCAS-CMS has a very simple HTTP interface that could be scripted. Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Lynnes, Christopher S. (GSFC-6102) Sent: 28 February 2012 16:03 To: cf-metadata@cgd.ucar.edu Subject: [CF-metadata] machine-accessible CF compliance checker? I know about the web sites where you use a form to upload files to check CF compliance, but is there a service somewhere that can be accessed by a script? -- Dr. Christopher Lynnes NASA/GSFC, Code 610.2phone: 301-614-5185 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- Scanned by iCritical. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] machine-accessible CF compliance checker?
On Mar 1, 2012, at 8:49 AM, John Caron wrote: Hi all: If the checker can read from opendap, then putting the files on an opendap server might be helpful, in that only the metadata is read instead of the entire file. There are a few difference among servers (Hyrax, TDS, GDS, etc) but im pretty sure this would work for netcdf files well enough for the checker's needs. could be worth finding out. Good idea! We have a server running on test.opendap.org that we would be willing to let others use. James john On 2/29/2012 7:55 AM, Bentley, Philip wrote: Hi Stephen, I may well be wrong, but I suspect that one of the reasons - perhaps the main reason - why folks would like access to a stand-alone cfchecker utility is the large size of their netcdf files. (It's certainly the reason we use a local version.) As you know only too well (from CMIP5), most of the netcdf files we're dealing with these days run into the high 100s of MB or low GBs. It's not really practical to upload such files in order to perform a CF compliance check. At least not on more than a few files. I wonder, therefore, if a possible solution is to develop an alternative checker utility that operates not on the netcdf file itself, but rather on the header info output by the ncdump -h command. Generating and uploading those would be trivial by comparison. Off the top of my head (and having given it almost zero thought!) the only test that I can think of which examines the actual data values is the check for coord vars being monotonic. But I suspect there are others I'm unaware of. If nothing else, such a utility might have some value as a 'first pass' checker. Any mileage in this? Apologies if it's already done the rounds! Regards, Phil -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of stephen.pas...@stfc.ac.uk Sent: 29 February 2012 11:39 To: christopher.s.lyn...@nasa.gov; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] machine-accessible CF compliance checker? Hi Christopher, Are you looking for a way to run the cfchecker locally or submit checking requests to one of the websites in a script? I think both would be possible with a little help. The cfchecker is now available on the Python Package Index at http://pypi.python.org/pypi/cfchecker. It needs a little help to work out of the box -- you need to configure UDUNITS and the location of several CF XML files. It's on my long-term TODO list to make this configuration easier. Also the CF-checker at BADC and NCAS-CMS has a very simple HTTP interface that could be scripted. Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Lynnes, Christopher S. (GSFC-6102) Sent: 28 February 2012 16:03 To: cf-metadata@cgd.ucar.edu Subject: [CF-metadata] machine-accessible CF compliance checker? I know about the web sites where you use a form to upload files to check CF compliance, but is there a service somewhere that can be accessed by a script? -- Dr. Christopher Lynnes NASA/GSFC, Code 610.2phone: 301-614-5185 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- Scanned by iCritical. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- James Gallagher jgallagher at opendap.org 406.723.8663 signature.asc Description: Message signed with OpenPGP using GPGMail ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] machine-accessible CF compliance checker?
Hi Christopher, Are you looking for a way to run the cfchecker locally or submit checking requests to one of the websites in a script? I think both would be possible with a little help. The cfchecker is now available on the Python Package Index at http://pypi.python.org/pypi/cfchecker. It needs a little help to work out of the box -- you need to configure UDUNITS and the location of several CF XML files. It's on my long-term TODO list to make this configuration easier. Also the CF-checker at BADC and NCAS-CMS has a very simple HTTP interface that could be scripted. Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Lynnes, Christopher S. (GSFC-6102) Sent: 28 February 2012 16:03 To: cf-metadata@cgd.ucar.edu Subject: [CF-metadata] machine-accessible CF compliance checker? I know about the web sites where you use a form to upload files to check CF compliance, but is there a service somewhere that can be accessed by a script? -- Dr. Christopher Lynnes NASA/GSFC, Code 610.2phone: 301-614-5185 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- Scanned by iCritical. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] machine-accessible CF compliance checker?
Hi Stephen, I may well be wrong, but I suspect that one of the reasons - perhaps the main reason - why folks would like access to a stand-alone cfchecker utility is the large size of their netcdf files. (It's certainly the reason we use a local version.) As you know only too well (from CMIP5), most of the netcdf files we're dealing with these days run into the high 100s of MB or low GBs. It's not really practical to upload such files in order to perform a CF compliance check. At least not on more than a few files. I wonder, therefore, if a possible solution is to develop an alternative checker utility that operates not on the netcdf file itself, but rather on the header info output by the ncdump -h command. Generating and uploading those would be trivial by comparison. Off the top of my head (and having given it almost zero thought!) the only test that I can think of which examines the actual data values is the check for coord vars being monotonic. But I suspect there are others I'm unaware of. If nothing else, such a utility might have some value as a 'first pass' checker. Any mileage in this? Apologies if it's already done the rounds! Regards, Phil -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of stephen.pas...@stfc.ac.uk Sent: 29 February 2012 11:39 To: christopher.s.lyn...@nasa.gov; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] machine-accessible CF compliance checker? Hi Christopher, Are you looking for a way to run the cfchecker locally or submit checking requests to one of the websites in a script? I think both would be possible with a little help. The cfchecker is now available on the Python Package Index at http://pypi.python.org/pypi/cfchecker. It needs a little help to work out of the box -- you need to configure UDUNITS and the location of several CF XML files. It's on my long-term TODO list to make this configuration easier. Also the CF-checker at BADC and NCAS-CMS has a very simple HTTP interface that could be scripted. Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Lynnes, Christopher S. (GSFC-6102) Sent: 28 February 2012 16:03 To: cf-metadata@cgd.ucar.edu Subject: [CF-metadata] machine-accessible CF compliance checker? I know about the web sites where you use a form to upload files to check CF compliance, but is there a service somewhere that can be accessed by a script? -- Dr. Christopher Lynnes NASA/GSFC, Code 610.2phone: 301-614-5185 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- Scanned by iCritical. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Re: [CF-metadata] machine-accessible CF compliance checker?
Hi Phil, So far my role has simply been to package and deploy Rosalyn's cfchecker code, therefore I really don't know to what extent it needs the real NetCDF. I think an alternative checker (i.e. codebase) is a bad idea because they might give contradictory results. I would suggest several options: 1. Work on making the cfchecker package easier to install so that users can run it on their real NetCDF. A user would have to know how to install python packages, compile UDUNITS and edit a config file. 2. Add a headers-only feature to Rosalyn's code so that user's can create dummy NetCDF with ncgen to upload to one of the online checkers. 3. As an addition to #2, add a feature to Rosalyn's code, or the CGI wrapper, to upload CDL. I am willing to do #1, if people think it would meet user's needs, but it would have to be added to my ever-expanding TODO list :-). Rosalyn, do you have any comments on #23? Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK -Original Message- From: Bentley, Philip [mailto:philip.bent...@metoffice.gov.uk] Sent: 29 February 2012 14:56 To: Pascoe, Stephen (STFC,RAL,RALSP); cf-metadata@cgd.ucar.edu Subject: RE: [CF-metadata] machine-accessible CF compliance checker? Hi Stephen, I may well be wrong, but I suspect that one of the reasons - perhaps the main reason - why folks would like access to a stand-alone cfchecker utility is the large size of their netcdf files. (It's certainly the reason we use a local version.) As you know only too well (from CMIP5), most of the netcdf files we're dealing with these days run into the high 100s of MB or low GBs. It's not really practical to upload such files in order to perform a CF compliance check. At least not on more than a few files. I wonder, therefore, if a possible solution is to develop an alternative checker utility that operates not on the netcdf file itself, but rather on the header info output by the ncdump -h command. Generating and uploading those would be trivial by comparison. Off the top of my head (and having given it almost zero thought!) the only test that I can think of which examines the actual data values is the check for coord vars being monotonic. But I suspect there are others I'm unaware of. If nothing else, such a utility might have some value as a 'first pass' checker. Any mileage in this? Apologies if it's already done the rounds! Regards, Phil -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of stephen.pas...@stfc.ac.uk Sent: 29 February 2012 11:39 To: christopher.s.lyn...@nasa.gov; cf-metadata@cgd.ucar.edu Subject: Re: [CF-metadata] machine-accessible CF compliance checker? Hi Christopher, Are you looking for a way to run the cfchecker locally or submit checking requests to one of the websites in a script? I think both would be possible with a little help. The cfchecker is now available on the Python Package Index at http://pypi.python.org/pypi/cfchecker. It needs a little help to work out of the box -- you need to configure UDUNITS and the location of several CF XML files. It's on my long-term TODO list to make this configuration easier. Also the CF-checker at BADC and NCAS-CMS has a very simple HTTP interface that could be scripted. Cheers, Stephen. --- Stephen Pascoe +44 (0)1235 445980 Centre of Environmental Data Archival STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK -Original Message- From: cf-metadata-boun...@cgd.ucar.edu [mailto:cf-metadata-boun...@cgd.ucar.edu] On Behalf Of Lynnes, Christopher S. (GSFC-6102) Sent: 28 February 2012 16:03 To: cf-metadata@cgd.ucar.edu Subject: [CF-metadata] machine-accessible CF compliance checker? I know about the web sites where you use a form to upload files to check CF compliance, but is there a service somewhere that can be accessed by a script? -- Dr. Christopher Lynnes NASA/GSFC, Code 610.2phone: 301-614-5185 ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- Scanned by iCritical. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata -- Scanned by iCritical. ___ CF-metadata mailing list CF-metadata@cgd.ucar.edu http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
