Re: [arts-users] arts run time error in z_fieldfrom HSE
Dear Ole martin, Thanks for your quick answer. I assume it comes from the fact that we combined surface measurements (temperature, pressure) with a vertical sounding profile for which the first measurement only starts around 10 meters. We noted some large gap between the surface measurements and the first radiosonde level that maybe caused a different altitude grid computed from the z_fieldfromhse method. If you say that the surface altitude does not have a lot of influence for an upward looking instrument, I will first try doing my simulation by changing the altitude grid to be lower than the surface measurement and see if the results look good. Thanks again, Best regards, Pauline - Météo-France - Dr. Pauline Martinet Chercheur CNRM/GMEI/MNPCA pauline.marti...@meteo.fr Fixe : +33 561079031 - Mail original - De: "Ole Martin Christensen" À: "Pauline Martinet" , "arts users mi" Envoyé: Jeudi 7 Décembre 2017 13:47:34 Objet: Re: arts run time error in z_fieldfrom HSE Hi, when you use z_fieldfromhse arts assumes an altitude at some pressure level and uses your specified temperature profile to extrapolate the altitudes at other pressure levels. If the lowest of these altitude levels is above your surface, arts gives the error you have posted. So my guess is that you changed the temperature profile you are using, or possibly the reference pressure level. Anyhow, In general if you instrument is placed above the surface, and looking Uppwards, where you define the surface should not have any influence. Ole martin From: arts_users.mi-boun...@lists.uni-hamburg.de on behalf of Pauline Martinet Sent: Thursday, December 7, 2017 11:44:51 AM To: arts users mi Subject: [arts-users] arts run time error in z_fieldfrom HSE Hi everybody, I am facing an error I had with ARTS several years ago that I though was resolved. For some profiles I get the error message: Run-time error in method: z_fieldFromHSE The surface altitude (*z_surface*) cannot be outside of the altitudes in *z_field*. Stopping ARTS execution. To avoid this problem, I define the first level of the z_field grid to be 1 meter lower than the z_surface. 1 meter difference was enough but this time for a new dataset, the z_field grid needs to start at least 3 to 5 meters below the z_surface so that ARTS does not crash. Can it cause large differences to the simulation if I define the altitude grid 5 meters lower ? Can this problem come from the pressure grid even though the error message only specifies the altitude ? I am simulating a ground-based microwave radiometer. Thank a lot for any help, Best regards, Pauline - Météo-France - Dr. Pauline Martinet Chercheur CNRM/GMEI/MNPCA pauline.marti...@meteo.fr Fixe : +33 561079031 ___ arts_users.mi mailing list arts_users.mi@lists.uni-hamburg.de https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
[arts-users] arts run time error in z_fieldfrom HSE
Hi everybody, I am facing an error I had with ARTS several years ago that I though was resolved. For some profiles I get the error message: Run-time error in method: z_fieldFromHSE The surface altitude (*z_surface*) cannot be outside of the altitudes in *z_field*. Stopping ARTS execution. To avoid this problem, I define the first level of the z_field grid to be 1 meter lower than the z_surface. 1 meter difference was enough but this time for a new dataset, the z_field grid needs to start at least 3 to 5 meters below the z_surface so that ARTS does not crash. Can it cause large differences to the simulation if I define the altitude grid 5 meters lower ? Can this problem come from the pressure grid even though the error message only specifies the altitude ? I am simulating a ground-based microwave radiometer. Thank a lot for any help, Best regards, Pauline - Météo-France - Dr. Pauline Martinet Chercheur CNRM/GMEI/MNPCA pauline.marti...@meteo.fr Fixe : +33 561079031 ___ arts_users.mi mailing list arts_users.mi@lists.uni-hamburg.de https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
Re: [arts-users] Advise concerning ARTS computation speed
Hi everybody again, It is just to tell you that even if I still have to figure out why I get the message when I create the lookup table from Qarts, I managed to create a look-up-table from an ARTS controlfile but I have trouble to use it in the ARTS control file. I am simulating frequencies grid 22 to 60 GHz and I get the error message: Cannot find new frequency 0 (2.224e+10Hz) in the lookup table frequency grid However, when I create my look-up table, I generate the frequency vector to be between 10e9 and 80e9 : VectorNLinSpace( f_grid, 100, 10e9, 80e9 ) Thanks for any help and sorry for the several emails these past few days, Best regards, Pauline - Météo-France - Dr. Pauline Martinet Chercheur CNRM/GMEI/LISA pauline.marti...@meteo.fr Fixe : +33 561079031 - Mail original - De: "Patrick Eriksson" À: "Pauline Martinet" , "Oliver Lemke" Cc: "francesco deangelis1" , "ARTS Users List" , "domenico cimini" Envoyé: Mercredi 13 Juillet 2016 11:01:00 Objet: Re: [arts-users] Advise concerning ARTS computation speed Hi again, > I will see what I can find in the users guide to prepare a look-up-table for > my instrument. How to practically generate a look-up table is best described by demo scripts. For pure arts see controlfiles/artscomponents/absorption/TestAbs.arts For Qarts see demos/qarts_abstable_demo.m /Patrick ___ arts_users.mi mailing list arts_users.mi@lists.uni-hamburg.de https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
Re: [arts-users] Advise concerning ARTS computation speed
Hi everybody, I tried to generate an absorption look-up-table from Qarts following the example in demos/qarts_abstable_demo.m When I call arts_y, I get the error message described later in the email. Does anyone know what I should modify to make it work ? I also generated an .xml look-up-table from the arts controfile directory with the good frequency grid and absorption species, but I am still investigating how to use it directly in a control file or in Qarts. Thanks a lot, Best regards Pauline Run-time error in controlfile: /tmp/atmlab-martinet-tped3b3242_9737_421a_8c1f_5eccc8a34610/cfile.arts Run-time error in method: propmat_clearsky_fieldCalc Run-time error in agenda: propmat_clearsky_agenda Run-time error in method: propmat_clearskyAddFromLookup Problem with gas absorption lookup table. Temperature T is outside the range covered by the lookup table. Your temperature was 215.506 K at a pressure of 2596.02 Pa. The temperature offset value is -61.4155. The allowed range is -17.1429 to 17.1429. The temperature perturbation grid range in the table is -15 to 15. We allow a bit of extrapolation, but NOT SO MUCH! Stopping ARTS execution. Goodbye. - Météo-France - Dr. Pauline Martinet Chercheur CNRM/GMEI/LISA pauline.marti...@meteo.fr Fixe : +33 561079031 - Mail original - De: "Patrick Eriksson" À: "Pauline Martinet" , "Oliver Lemke" Cc: "francesco deangelis1" , "ARTS Users List" , "domenico cimini" Envoyé: Mercredi 13 Juillet 2016 11:01:00 Objet: Re: [arts-users] Advise concerning ARTS computation speed Hi again, > I will see what I can find in the users guide to prepare a look-up-table for > my instrument. How to practically generate a look-up table is best described by demo scripts. For pure arts see controlfiles/artscomponents/absorption/TestAbs.arts For Qarts see demos/qarts_abstable_demo.m /Patrick ___ arts_users.mi mailing list arts_users.mi@lists.uni-hamburg.de https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
Re: [arts-users] Advise concerning ARTS computation speed
Hi everybody, In fact, I modified the cmake compilation yesterday to RELEASE mode and I did not see a real impact on the computation time. For the on-the-fly, Patrick is right we are working with ground-based instrument. The problem is I have never used a look-up-table with ARTS, I am not sure I can prepare this table before the review submission to evaluate how much gain we get in computation time. I will see what I can find in the users guide to prepare a look-up-table for my instrument. Cheers, Pauline - Météo-France - Dr. Pauline Martinet Chercheur CNRM/GMEI/LISA pauline.marti...@meteo.fr Fixe : +33 561079031 - Mail original - De: "Oliver Lemke" À: "Pauline Martinet" Cc: "Richard Larsson" , "francesco deangelis1" , "ARTS Users List" , "domenico cimini" Envoyé: Mercredi 13 Juillet 2016 07:06:27 Objet: Re: [arts-users] Advise concerning ARTS computation speed Hi Pauline, If you didn't pass any options to cmake at the time you compiled ARTS, then you have an executable with debug information because that's the default. To get a non-debug build you have to explicitly set it to release mode as Richard explained: cmake -DCMAKE_BUILD_TYPE=Release .. make clean make cheers, /oliver > On 12 Jul 2016, at 17:29, Pauline Martinet wrote: > > Hi Richard, > > Thanks for your advice. > > I am using the debug mode only for atmlab (the matlab interface to call ARTS). > > I do not think I compiled ARTS in any debug mode. Thus, I assume it should > not have an impact on the ARTS speed. > > Best regards, > > Pauline ___ arts_users.mi mailing list arts_users.mi@lists.uni-hamburg.de https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
Re: [arts-users] Advise concerning ARTS computation speed
Hi Richard, Thanks for your advice. I am using the debug mode only for atmlab (the matlab interface to call ARTS). I do not think I compiled ARTS in any debug mode. Thus, I assume it should not have an impact on the ARTS speed. Best regards, Pauline - Météo-France - Dr. Pauline Martinet Chercheur CNRM/GMEI/LISA pauline.marti...@meteo.fr Fixe : +33 561079031 - Mail original - De: "Richard Larsson" À: "Pauline Martinet" Cc: "Stefan Buehler" , "francesco deangelis1" , "arts users mi" , "domenico cimini" Envoyé: Mardi 12 Juillet 2016 14:09:17 Objet: Re: [arts-users] Advise concerning ARTS computation speed Just some input, I have not followed your discussion in detail but ARTS is slow in debug mode. I would suggest compiling with cmake -DCMAKE_BUILD_TYPE=Release .. make clean make if you intend to make it a bit faster. I have no real tests ready but the standard "make check" tests are like 20% faster that way. There is a lot of checks that are only important if you are a programmer running ARTS in the debug that are ignored otherwise. However, this 20% speed-gain is not universal. I suspect they are found specifically in the interpolation routines, where we do not want to break things. I am not sure if you will gain more or less time. //Richard 2016-07-12 13:54 GMT+02:00 Pauline Martinet < pauline.marti...@meteo.fr > : Dear Patrick and Stephan, Thanks a lot for your kind help. I managed to run ARTS with atmlab and a timer (Patrick's suggestion) as well as run ARTS directly from the cfile created by ATMLAB (with a Debug mode). I assume this way we have a fair evaluation of the computation time of the ARTS simulation. I already use the "on-the-fly" option to compute the absorption which should be less time consuming than the look-up table (from Patrick's advice). Patrick also mentioned that nothing should be critical if we use standard setting. Stephan, do you have any example of not standard setting (to check if I have an option which increases the computation time of ARTS). I also was successful to compute the Jacobians directly from an arts controlfile. But when I want to copy the output into a matrix to write the values in an ASCII file (following the example TestWfuns.arts: Copy( Ja, jacobian ) I get the error message: Method Copy needs input variable: jacobian I will have a closer look to the users guide and I let you know what is unclear. Thanks a lot again, Best regards, Pauline - Météo-France - Dr. Pauline Martinet Chercheur CNRM/GMEI/LISA pauline.marti...@meteo.fr Fixe : +33 561079031 - Mail original - De: "Stefan Buehler" < stefan.bueh...@uni-hamburg.de > À: "Pauline Martinet" < pauline.marti...@meteo.fr > Cc: "Patrick Eriksson" < patrick.eriks...@chalmers.se >, "domenico cimini" < domenico.cim...@imaa.cnr.it >, "francesco deangelis1" < francesco.deangel...@graduate.univaq.it >, "arts users mi" < arts_users.mi@lists.uni-hamburg.de > Envoyé: Lundi 11 Juillet 2016 17:58:37 Objet: Re: Advise concerning ARTS computation speed Dear Pauline, > Could anyone help in explaining what is the function to compute Jacobians > from ARTS alone with atmlab ? There are several example ARTS controlfiles with Jacobians included in the distribution, e.g.: controlfiles/artscomponents/wfuns/TestWfuns.arts controlfiles/classroom_exercises/exe3_jacobian/jacobian.arts You find documentation on each individual ARTS method on the docserver: http://arts.mi.uni-hamburg.de/docserver-trunk/ or if you start the docserver that comes with your ARTS distribution. > When I have the cfile from ARTS, what is the terminal command to execute it > without any matlab interface ? arts jacobian.arts If your arts executable is not in your shell path, you have to give the right path, e.g., ~/arts_build/src/arts jacobian.arts jacobian.arts here is the name of the controlfile. > (it is probably a silly question but I really had troubles to understand how > to use ARTS without > matlab this is why I never used). Have you tried the ARTS user guide (available at radiativetransfer.org )? How to run ARTS is described in Chapter 1. Please let us know if anything there is unclear, so that we can improve the documentation. Running ARTS without Matlab is really easier (and more thoroughly documented) than through the Matlab interface. (But so far lacking the capabilities to run retrievals, of course.) Best wishes, Stefan ___ arts_users.mi mailing list arts_users.mi@lists.uni-hamburg.de https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi ___ arts_users.mi mailing list arts_users.mi@lists.uni-hamburg.de https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
Re: [arts-users] Advise concerning ARTS computation speed
Hi Stefan, Thanks for your help I am now able to run everything (direct simulations and Jacobians) from a terminal :) Last step will be to learn how to produce xml format input data but it should be quite simple. Thanks again, Best regards, Pauline - Météo-France - Dr. Pauline Martinet Chercheur CNRM/GMEI/LISA pauline.marti...@meteo.fr Fixe : +33 561079031 - Mail original - De: "Stefan Buehler" À: "Pauline Martinet" Cc: "Patrick Eriksson" , "domenico cimini" , "francesco deangelis1" , "arts users mi" Envoyé: Mardi 12 Juillet 2016 14:23:57 Objet: Re: Advise concerning ARTS computation speed Dear Pauline, > I already use the "on-the-fly" option to compute the absorption which should > be less time consuming > than the look-up table (from Patrick's advice). Patrick also mentioned that > nothing should be critical if we use standard setting. > > Stephan, do you have any example of not standard setting (to check if I have > an option which increases the computation time of ARTS). If you do a whole batch of calculations in one go (something you can easily do with ARTS alone, but not through Matlab), then absorption from lookup table can be much faster than on the fly absorption. That was the reason for implementing it. You can read about this in my paper: https://arts.mi.uni-hamburg.de/publications/group.php?bibonly=buehler11:_absorption_jqsrt > I also was successful to compute the Jacobians directly from an arts > controlfile. But when I want to copy the output into a matrix > to write the values in an ASCII file (following the example TestWfuns.arts: > Copy( Ja, jacobian ) I get the error message: > Method Copy needs input variable: jacobin You can write the Jacobian to a file directly, no need to copy it first, I think. > I will have a closer look to the users guide and I let you know what is > unclear. Great. A general comment: ARTS is not meant to be very fast, but exact. There is no doubt that RTTOV is orders of magnitude faster. I don’t think the actual numbers are that interesting. I would reply to the reviewer of your paper along those lines, perhaps also make this clear in the text. The reviewer is right that there are lot’s of things you can do inside the ARTS job to speed it up. But I don’t think it should be the subject of your article to explore this. You don’t have the background to do this well. If you seriously feel that you need this aspect, then one of us can explore it for you. But then you should add him/her as coauthor. Best wishes, Stefan ___ arts_users.mi mailing list arts_users.mi@lists.uni-hamburg.de https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
Re: [arts-users] Advise concerning ARTS computation speed
Dear Patrick and Stephan, Thanks a lot for your kind help. I managed to run ARTS with atmlab and a timer (Patrick's suggestion) as well as run ARTS directly from the cfile created by ATMLAB (with a Debug mode). I assume this way we have a fair evaluation of the computation time of the ARTS simulation. I already use the "on-the-fly" option to compute the absorption which should be less time consuming than the look-up table (from Patrick's advice). Patrick also mentioned that nothing should be critical if we use standard setting. Stephan, do you have any example of not standard setting (to check if I have an option which increases the computation time of ARTS). I also was successful to compute the Jacobians directly from an arts controlfile. But when I want to copy the output into a matrix to write the values in an ASCII file (following the example TestWfuns.arts: Copy( Ja, jacobian ) I get the error message: Method Copy needs input variable: jacobian I will have a closer look to the users guide and I let you know what is unclear. Thanks a lot again, Best regards, Pauline - Météo-France - Dr. Pauline Martinet Chercheur CNRM/GMEI/LISA pauline.marti...@meteo.fr Fixe : +33 561079031 - Mail original - De: "Stefan Buehler" À: "Pauline Martinet" Cc: "Patrick Eriksson" , "domenico cimini" , "francesco deangelis1" , "arts users mi" Envoyé: Lundi 11 Juillet 2016 17:58:37 Objet: Re: Advise concerning ARTS computation speed Dear Pauline, > Could anyone help in explaining what is the function to compute Jacobians > from ARTS alone with atmlab ? There are several example ARTS controlfiles with Jacobians included in the distribution, e.g.: controlfiles/artscomponents/wfuns/TestWfuns.arts controlfiles/classroom_exercises/exe3_jacobian/jacobian.arts You find documentation on each individual ARTS method on the docserver: http://arts.mi.uni-hamburg.de/docserver-trunk/ or if you start the docserver that comes with your ARTS distribution. > When I have the cfile from ARTS, what is the terminal command to execute it > without any matlab interface ? arts jacobian.arts If your arts executable is not in your shell path, you have to give the right path, e.g., ~/arts_build/src/arts jacobian.arts jacobian.arts here is the name of the controlfile. > (it is probably a silly question but I really had troubles to understand how > to use ARTS without > matlab this is why I never used). Have you tried the ARTS user guide (available at radiativetransfer.org)? How to run ARTS is described in Chapter 1. Please let us know if anything there is unclear, so that we can improve the documentation. Running ARTS without Matlab is really easier (and more thoroughly documented) than through the Matlab interface. (But so far lacking the capabilities to run retrievals, of course.) Best wishes, Stefan ___ arts_users.mi mailing list arts_users.mi@lists.uni-hamburg.de https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
Re: [arts-users] Advise concerning ARTS computation speed
Hi Patrick (cc. arts users),
Thanks for your reply.
We use the on-the-fly option and not the lookup table to compute the absorption.
Could anyone help in explaining what is the function to compute Jacobians from
ARTS alone with atmlab ?
When I have the cfile from ARTS, what is the terminal command to execute it
without any matlab interface ?
(it is probably a silly question but I really had troubles to understand how
to use ARTS without
matlab this is why I never used).
Thanks a lot for your kind help,
Best regards,
Pauline
- Météo-France -
Dr. Pauline Martinet
Chercheur CNRM/GMEI/LISA
pauline.marti...@meteo.fr
Fixe : +33 561079031
- Mail original -
De: "Patrick Eriksson"
À: "domenico cimini"
Cc: "Stefan Buehler" , "pauline martinet"
, "francesco deangelis1"
, "arts users mi"
Envoyé: Lundi 11 Juillet 2016 17:13:10
Objet: Re: Advise concerning ARTS computation speed
Dear Nico,
A relatively quick answer as I am on vacation. Please send questions
like this to the arts users mailing list. See cc.
As I understand it, this deals with the relative speed of ARTS and
RTTOV. I personally don't have any practical experience of RTTOV.
Unfortunately, there is no single recommendation on how to use ARTS for
maximum speed. The most critical aspect is how the absorption is
calculated, on-the-fly or by lookup table. On-the-fly can be the fastest
option, particularly for ground-based measurement of the strato- and
mesosphere. I assume you should use the lookup table option. If you
allow some poorer accuracy, the polynomial order used to interpolate the
lookup table could be decreased to increase the speed. If you use
standard settings, other aspects should not be critical. It would be
lengthy discuss all possible bad settings, ask if you are unsure about
any setting.
You don't need Qpack to calculate Jacobians. This can be done by ARTS
alone. However, it should be possible to time the actual time used by
ARTS to calculate the spectrum + Jacobian, even from Qpack.
Try this
Q.YCALC_WSMS = { 'timerStart', 'yCalc', 'timerStop' };
You could need to set this
atmlab('FMODEL_VERBOSITY',1)
But all this could fail, we have experienced problems with how matlab
picks up the screen output of arts.
For more advanced qpack users, one option is to set
atmlab('DEBUG',1)
and locate the relavnt temporary folder, add the time commands to the
cfile and run the cfile from the terminal.
Regards,
Patrick
*Från:* "domenico.cimini"
*Skickat:* 8 juli 2016 10:54:18 CEST
*Till:* Stefan Buehler ,
patrick.eriks...@chalmers.se
*Cc:* Pauline Martinet , Francesco De Angelis
*Ämne:* Advise concerning ARTS computation speed
Dear Stefan and Patrick,
let us intrude in your mailbox for asking your advise concerning ARTS.
Some months ago we submitted a manuscript presenting the modification we
made to RTTOV to simulate ground-based microwave radiometer observations:
http://www.geosci-model-dev-discuss.net/gmd-2016-65/
As we compare results from RTTOV and ARTS, we also reported on the
different computing times.
Referee #2 commented that the speed of ARTS depends critically on
several settings.
For details, see the comment at the bottom of C3 in the following link:
http://editor.copernicus.org/index.php/gmd-2016-65-RC2.pdf?_mdl=msover_md&_jrl=365&_lcm=oc108lcm109w&_acm=get_comm_file&_ms=50514&c=108717&salt=1176682561237999184
Of course we want to be fair with ARTS, but we may miss the necessary
familiarity with ARTS settings;
thus we would like to ask your advise on the following issues.
1. Do you have any feeling of the computation speed of ARTS with respect
to RTTOV?
2. We used ARTS + ATMLAB for the direct simulations (through the
arts_y(Q) function) and ARTS + Qpack2 to calculate and evaluate the
computation time of the Jacobians. This probably slows down the
computation as it requires a matlab interface.
Is there any recommended settings to optimize the computation speed?
Is there another solution than using Qpack to compute the Jacobian
matrix with ARTS (without coding a rather slow "brute force" method)?
Thanks much for your attention and valuable time.
Kind regards,
Nico, Pauline and Francesco
___
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
