Hi Byungsuk,
We provide the TIPS as published by HITRAN via an XML file in:
arts-xml-data/spectroscopy/PartitionSums/TIPS/tips.xml. The README
contains some more information.
About the 300 K limit today, I do not know where it is from and the
differences between it and the TIPS data is not strongly defined. The
builtin partition_functions are historical, and tests show that it should
be mostly reliable up to 300 K but not much more than that.
If you need a comparison for the temperature ranges you are interested in,
you can run "nlte_fieldSetLteInternalPartitionFunction" and
"nlte_fieldSetLteExternalPartitionFunction"
after having set partition_functions to tips.xml. The NLTE field is a the
size of your atmosphere times the size of the energy levels you are
interested in.
With hope,
//Richard
2018-09-18 2:33 GMT+02:00 이병석 <cb...@aracnt.com>:
> Hello Richard,
>
> Thank you for your reply.
>
> May I ask which tips xml files you are referring to?
>
> Also a question: it seems that the bad partition function warning occurs
> for any input t_field temperature beyond the range between 150 K and 300 K.
> In reality the surface temperature of the Earth frequently exceeds 300 K
> (26.85 deg C). For input values such as 30 deg C or a little higher (e.g.
> 30-40 deg C, early 300's K), would the results still be bad if the warning
> is suppressed?
>
> Regards,
>
>
> Byungsuk Lee
>
> ARA Consulting & Technology
>
> 30 Songdomirae-ro D-1510, Yeonsu-gu, Incheon 21990, South Korea
>
> -----Original Message-----
> *From:* "Richard Larsson"<ric.lars...@gmail.com>
> *To:* "이병석"<cb...@aracnt.com>;
> *Cc:* "Simon Pfreundschuh"<simon.pfreundsc...@chalmers.se>; <
> arts_users...@mailman.rrz.uni-hamburg.de>;
> *Sent:* 2018-09-17 (월) 18:21:57
> *Subject:* Re: [arts-users] ARTS OEM
>
> Hi,
>
> Just as a warning, suppressing the bad partition function error is not
> good practice since it is known that they are, you know, bad. Standard
> tests in ARTS fail by several tenths of Kelvin because the partition
> functions in ARTS are that bad. Just run with the provided tips xml-files
> instead, because if your temperatures are lower than 70 K or higher than
> 3000 K you probably have another problem.
>
> With hope,
> //Richard
>
> 2018-09-17 10:31 GMT+02:00 이병석 <cb...@aracnt.com>:
>
> Hello Simon Pfreundschuh,
>
> Thank you for your reply.
>
> I have employed both of the suggestions and still get the irregular
> errors. From the same controlfile modified in the previous email, I have
> included and changed to the following lines:
>
> OEM(method="lm",
> max_iter=15,
> display_progress=1,
> lm_ga_settings=[100,5,2,1000,1,99])
>
> and
>
> AgendaSet( inversion_iterate_agenda ){
>
> xClip(ijq = 0, limit_low = 150.0, limit_high = 350)
>
> Ignore(inversion_iteration_counter)
> # Map x to ARTS' variables
> x2artsStandard
>
> # To be safe, rerun checks dealing with the atmosphere
> atmfields_checkedCalc
> atmgeom_checkedCalc
>
> # Calculate yf and Jacobian matching x.
> yCalc( y=yf )
>
> # Add baseline term
> VectorAddVector( yf, yf, y_baseline )
>
> # This method takes cares of some "fixes" that are needed to get the
> Jacobian
> # right for iterative solutions. No need to call this WSM for linear
> inversions.
> jacobianAdjustAndTransform
> }
>
> I included both of the suggestions, and from there
> increased the maximum number of iterations to 15
> and set the temperature limits to be between 150 K and 350 K.
> All other lines remain the same, including the input absorption line and
> atmospheric fields data.
>
> Now I irregularly get the errors during the OEM computation as before, of
> t_field having NaN values or of bad partition function warning (which
> happens for any t_field value outside between 150 K and 300 K). While the
> bad partition function warning could be suppressed in the code, it seems
> that the same problem as before persists.
>
> Could you please look into this issue a little more? I am not sure how to
> tackle the problem.
>
> Thank you for your service.
>
> Regards,
>
>
> Byungsuk Lee
>
> ARA Consulting & Technology
>
> 30 Songdomirae-ro D-1510, Yeonsu-gu, Incheon 21990, South Korea
>
>
>
> -----Original Message-----
> *From:* "Simon Pfreundschuh"<simon.pfreundsc...@chalmers.se>
> *To:* "이병석"<cb...@aracnt.com>; "arts_users...@mailman.rrz.uni-hamburg.de"<
> arts_users...@mailman.rrz.uni-hamburg.de>;
> *Cc:*
> *Sent:* 2018-09-14 (금) 15:13:55
> *Subject:* Re: [arts-users] ARTS OEM
>
> Hi Byungsuk Lee,
>
> It is a bit surprising that these errors occur irregularly, since there is
> no randomness
> in the test setup. However, if you are running retrievals with external
> observation data
> it may well happen that the iteration ends up in a state that produces
> invalid values.
>
> For your temperature retrieval I would recommend one or both of the
> following solutions:
>
> 1. Use the Levenberg-Marquardt (LM) iteration scheme: Using the LM scheme
> with a high start
> value for the gamma parameter can help avoid reaching invalid values
> when the forward
> model is non-linear:
>
> OEM(method="lm",
> max_iter=5,
> display_progress=1,
> lm_ga_settings=[100,5,2,1000,1,99])
>
> 2. Clip the x-vector: Using the xClip WSM inside your inversion iterate
> agenda you can force the
> x vector to contain only temperatures within a certain range. If you
> use this approach you
> should check that none of the retrieved temperatures in your final
> state are clipped otherwise your
> results may not be valid.
>
> AgendaSet( inversion_iterate_agenda ){
>
> xClip(ijq = 0, limit_low = 100.0)
> Ignore(inversion_iteration_counter)
> # Map x to ARTS' variables
> x2artsStandard
>
> # To be safe, rerun checks dealing with the atmosphere
> atmfields_checkedCalc
> atmgeom_checkedCalc
>
> # Calculate yf and Jacobian matching x.
> yCalc( y=yf )
>
> # Add baseline term
> VectorAddVector( yf, yf, y_baseline )
>
> # This method takes cares of some "fixes" that are needed to get the
> Jacobian
> # right for iterative solutions. No need to call this WSM for linear
> inversions.
> jacobianAdjustAndTransform
> }
>
>
> Regards,
>
> Simon
>
> ------------------------------
> *From:* arts_users.mi-boun...@mailman.rrz.uni-hamburg.de <
> arts_users.mi-boun...@mailman.rrz.uni-hamburg.de> on behalf of 이병석 <
> cb...@aracnt.com>
> *Sent:* Friday, September 14, 2018 2:31:25 AM
> *To:* arts_users...@mailman.rrz.uni-hamburg.de
> *Subject:* [arts-users] ARTS OEM
>
> Dear ARTS users,
>
> I am writing to address a question with the use of OEM in the ARTS
> development version.
>
> From the "TestOEM.arts" in "ARTS/controlfiles/artscomponents/oem", this
> time I changed the codes as follows.
>
> 1) From (line 115):
> retrievalAddAbsSpecies(
> species = "O3",
> unit = "vmr",
> g1 = p_ret_grid,
> g2 = lat_grid,
> g3 = lon_grid
> )
> To:
> retrievalAddTemperature (
> g1 = p_ret_grid,
> g2 = lat_grid,
> g3 = lon_grid,
> hse = "off" )
>
> 2) From (line 125):
> VectorSetConstant(vars, nelem, 1e-12)
> DiagonalMatrix(sparse_block, vars)
> covmat_sxAddBlock(block = sparse_block)
> To:
> VectorSetConstant(vars, nelem, 0.1)
> DiagonalMatrix(sparse_block, vars)
> covmat_sxAddBlock(block = sparse_block)
>
> 3) From (line 37):
> NumericSet( f_start, 110.436e9 )
> NumericSet( f_end, 111.236e9 )
> IndexSet( nf, 801 )
> IndexSet( np, 81 )
> To:
> NumericSet( f_start, 110.436e9 )
> NumericSet( f_end, 111.236e9 )
> IndexSet( nf, 18 )
> IndexSet( np, 18 )
>
> 4) From (line 42):
> VectorNLinSpace( f_grid, nf, f_start, f_end )
> VectorNLogSpace( p_grid, 361, 500e2, 0.1 )
> VectorNLogSpace( p_ret_grid, np, 500e2, 0.1 )
> To:
> VectorNLinSpace( f_grid, nf, f_start, f_end )
> VectorNLogSpace( p_grid, 18, 500e2, 0.1 )
> VectorNLogSpace( p_ret_grid, np, 500e2, 0.1 )
>
> Overall, I changed the ozone VMR retrieval to temperature retrieval and
> reduced the number of frequencies and pressure layers.
>
> When I run the controlfile multiple times, I irregularly get errors. The
> error messages are
>
>
> Run-time error in oem computation: Forward Model Evaluation Error
>
> Run-time error in agenda: inversion_iterate_agenda
>
> Run-time error in method: atmfields_checkedCalc
>
> The variable *t_field* contains one or several NaNs. This is not
> allowed!
>
>
>
> or
>
>
>
> All temperatures in *t_field* must be > 0.
>
> or seldom the bad partition function warning.
>
> Here are my questions:
> 1) Is this an anticipated error, simply coming from failing to find a
> converging solution? Is it expected that the OEM sometimes throws an error
> as above?
> 2) Are there mechanisms (e.g. workspace methods) within the ARTS that can
> prevent the t_field or vmr_field from becoming either negative or NaN
> during the OEM iterations?
>
> Thank you for your assistance.
>
> Regards,
>
>
> Byungsuk Lee
>
> ARA Consulting & Technology
>
> 30 Songdomirae-ro D-1510, Yeonsu-gu, Incheon 21990, South Korea
>
> *Email: * cb...@aracnt.com
>
> _______________________________________________
> arts_users.mi mailing list
> arts_users.mi@lists.uni-hamburg.de
> <https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi>
> https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
>
>
>
_______________________________________________
arts_users.mi mailing list
arts_users.mi@lists.uni-hamburg.de
https://mailman.rrz.uni-hamburg.de/mailman/listinfo/arts_users.mi
