Hi Patrick, Thanks a lot for the information provided. I am writing again because I did that I said in the previous mail and retrieved ozone profiles are not bad, but the residual (yf-y) is not around zero...It goes down to around 0.3 (the shape is good, but goes down). It happens when I introduce the error related to T and H2O, and also when I introduce only T or only H2O. Residual are very similar in this 3 cases (T and H2O, T or H2O). When I do not include these errors (only considering noise) it does not happen. I was looking for what is wrong and I could not find it. Do you have any suggestion about it? On the other hand, what is exactly the use of the variable 'xnorm'?
Thanks a lot Best Regards Facundo 2015-11-12 5:05 GMT-03:00 Patrick Eriksson <[email protected]>: > Hi Facundo, > > Yes, totally correct. This is exactly what should be done if you treat H2O > and temperature as forward model parameter uncertainties. That this is OK > (and better than setting Se=Sn) is not clearly expressed by Rodgers, and as > a reference for this I append one of my first articles: > > https://dl.dropboxusercontent.com/u/102809064/eriksson-2000-analy.pdf > > See Eq 8. Yes, you are right, the impact of H2O and temperature will > automatically be part of the retrieval error (e.g. in L2.speciesX_eo). > > > Let me just make another thing clear. As I have written, you obtain the > same result by instead retrieving H2O and temperature in parallel to ozone. > For a linear case, you will get identical results. And this is the way I > think most people are handling interfering effects today. However, it is > here important to note that with this later set-up, the impact of H2O and > temperature comes out as part of the smoothing error (l2.speciesX_es). > Accidentally, I am writing a document right now where I felt forced to dig > into this and give a detailed explanation. So maybe I will come back to > this point later, when I have something that can be circulated. > > Bye, > > Patrick > > > > > > On 11/11/15 22:01, Facundo Orte wrote: > >> Patrick, >> thank you very much for your reply. >> >> I will proceed to share what I did. I implemented the first of your >> suggestions. So, >> >> Se = Sn + Kt*St*Kt^T + Kh2o*Sh2o*Kh2o^T >> >> (I changed the nomenclature with respect to the previous email to not >> confuse with the nomenclature of Matlab functions) >> >> where: >> Sn: covariance related with noise (in Matlab function, >> Sn=Y.TNOISE^2.*Q.TNOISE_C) >> Kt: jacobian temperature >> St: covariance matrix of temperature >> Kh2o: jacobian of water vapour >> Sh2o: covariance matrix of water vapour >> >> I implemented it inside of a function called qp2_y2Q, which is inside of >> qpack2.m. qp2_y2Q is implemented as: >> >> [Q,Se] = qp2_y2Q( Q, Y, m); >> >> where the output Se is now the quantity in the equation of the >> covariance matrix as above (Se = Sn + Kt*St*Kt^T + Kh2o*Sh2o*Kh2o^T). >> Therefore, the retrieval error will consider the error due to noise, >> temperature and H2O without any other modification in Qpack because the >> calculation of other retrieval parameters (such as cost, G, etc.) in >> oem.m takes into account the covariance matrix Se. >> >> Is it correct? >> >> Thanks in advance >> >> Best regards >> >> Facundo >> >> >> >> >> 2015-11-01 18:17 GMT-03:00 Patrick Eriksson >> <[email protected] <mailto:[email protected]>>: >> >> >> Facundo, >> >> You were not unclear, I tried to answer exactly what you wrote in >> your last email. >> >> First of all, if you want to sum up uncertainties you should not use >> matrix inverses. You do it as: >> >> S = Se + Kb*Sb*Kb^T >> >> >> (What you want to do was better supported in Qpack1, but I had a >> hard time to find a general way to cover all possible combinations >> that could be of interest. For Qpack2 you need to do some tricks.) >> >> Let us use T as example. If you want to get out the Kb matching T, >> you can e.g. deactivate all present retrieval variables, and just >> set T as retrieval variable, and Q.T.L2 = true. Then make a linear >> inversion and Kb = L2.J. (You can do the same thing for H2O. The >> uncertainty associated with O2 should be negligible.) >> >> With this Kb, the retrieval error due to T is calculated as >> >> Sr = G*Kb*St*Kb^T*G^T >> >> where St is the temperature covariance matrix. >> >> The above should answer your direct question. However, if the error >> due to T is significant, you can do better. This simple by >> retrieving T in parallel to O3. The extra calculation cost is quite >> small, and I would recommend you to do this as the error related to >> T will then be smaller. If you don't care about the result for T, >> you just set Q.T.L2=false. >> >> If you still want to calculate the error due to T, you can do as >> above, or use L2.jq and L2.ji to extract the part of L2.J that >> matches T, as I tried to describe in my last answer. >> >> If H2O should also be retrieved depends on your exact set-up. >> >> Bye, >> >> Patrick >> >> >> >> >> >> >> On 10/29/15 14:19, Facundo Orte wrote: >> >> Dear Patrick, >> Thank you so much for the reply and I am sorry for not be clear. >> I set Qpack to retrieve only ozone. Other gases and T (H2O, O2 >> and T) I >> set as Q.ABS_SPECIES.RETRIEVE=false, because I am not interesting >> in >> water vapor, oxygen, and T, but I include NCEP profiles of >> these gases and temperature in Q.DEFINITIONS.m for ARTS >> calculation and >> to model the spectrum. O2 is included because the frequency range >> of >> measurement is 110.83GHz and this gas also absorb the signal. >> These >> profiles have some uncertainties and I want to introduce these >> uncertainties in the covariance matrix for the cost calculation >> and so >> on. What I need is to include these uncertainties in the >> covariance matrix. >> >> I think that I can include it in the covariance matrix doing: >> >> S^(-1)=Se^(-1) + Kb^T*Sb^(-1)*Kb >> >> where Kb is the jacobian of non-retrieval parameters (in my case >> H2O, T, >> and O2), Sb is the covariance matrix related with the >> non-retrieval >> parameters and Se is the covariance matrix related with noise. S >> is the >> covariance matrix used to calculate the cost as following. >> >> cost=[y-F(x,b)]^T * *S*^(-1) * [y-F(x,b)] + [x-xa]^T * Sx^(-1) * >> [x-xa] >> >> At this time, I introduce the covariance matrix only related >> with noise >> (S=Se) and I not include the uncertainties for water vapor, T >> and O2 >> profiles. >> >> I think that the Jacobians of non-retrieval parameters are >> similar for >> different measurements. So, my idea is to calculate these >> jacobians one >> time and use it then for the calculation of different >> measurements. It >> will avoid the calculation of these jacobians in each measurement. >> >> My problem is that I do not know how to include the >> uncertainties for >> non-retrieval parameters profiles in the covariance matrix S. >> >> Thanks in advance >> Best regards >> >> 2015-10-27 16:55 GMT-03:00 Patrick Eriksson >> <[email protected] >> <mailto:[email protected]> >> <mailto:[email protected] >> >> <mailto:[email protected]>>>: >> >> >> Dear Facundo, >> >> As I understand your question, the answer is no. Or at >> least, this >> can not be done automatically. >> >> However, if you think that H2O and temperature can give >> substantial >> errors (I can not see how O2 should give rise to an error), >> you >> should retrieve H2O and T in parallel to O3. That will >> decrease the >> impact of H2O and T. In short, OEM will then adjust as far as >> possible to H2O and T uncertainties. The impact of H2O and >> T will >> then be included in the standard error estimate. >> >> You have seen that Qpack can give you the error covariance >> matrix? >> >> If you retrieve H2O and T, what you call df/dH2O and df/dT >> are part >> of the (total) Jacobian matrix, that you can trigger Qpack to >> output, i.e. L2.J. The fields L2.jq and L2.ji give you >> information >> about what part of J that belong to H2O and T etc. That >> should give >> you a way to calculate separate errors. >> >> Bye, >> >> Patrick >> >> >> >> >> On 10/27/15 20:15, Facundo Orte wrote: >> >> Dear all, >> I am very new using Qpack. I am setting Qpack to >> retrieve ozone >> profiles >> using ARTS. At this moment, I am able to retrieve >> coherent ozone >> profiles, but to calculate covariance matrix I am >> trying to get the >> jacobian for water vapor (df/dH2O), oxygen(df/dO2) and >> temperature >> (df/dT) (to include the uncertainties of these gases and >> temperature), >> due to these gases absorb radiation in the frequency >> range of >> measurement. Is it possible to get these jacobians at >> the same >> time as >> the retrieval species (O3)? >> Thanks in advance >> >> Regards >> Facundo >> >> >> _______________________________________________ >> qpack mailing list >> [email protected] <mailto:[email protected]> >> <mailto:[email protected] <mailto:[email protected]>> >> https://www.sat.ltu.se/mailman/listinfo/qpack >> >> >> >> >
_______________________________________________ qpack mailing list [email protected] https://www.sat.ltu.se/mailman/listinfo/qpack
