Re: [ccp4bb] [ccp4bb] AW: [ccp4bb] uncertainites associated with intensities from twinned crystals; Sorry for HTML.
Fulvio, First, to your point 2): Iobs(h1) and Iobs(h2) as well as Itrue(h1) and Itrue(h2) are /not/ correlated! The Iobses are /related/ to the Itrues by alpha (and the twin law), but the Itrues are totally uncorrelated to each other, and so are the Iobses, in my opinion (even though those will become more and more equal as alpha approaches 0.5, but this is not a correlation! And at alpha = 0.5 this formalism breaks down, anyways). So I do think that the simple error propagation is valid here. Now for your point 1): The formula I gave is only valid if you have an analytical relationship between the magnitudes you measure and the magnitudes you extract (and no correlation between them). For non-merohedral twins, this is not true, as you'll have to make that decision on a reflection by reflection base, so this is definitely /not/ generally applicable in that situation. And yes, the uncertainties associated with /detwinned/ intensities are much larger than the uncertainties associated with your measured data. This is one (but not the most important) reason, to refine against intensities and make the twin law part of your model. Hope that makes sense, Jens On Thu, 2013-11-07 at 09:22 +0100, fulvio.saccoc...@uniroma1.it wrote: > Dear all, > thank you for your reply. I would summarize my concerns and opinions, > so > far: > > 1) for QTLS (non-merohedral twinning - non intersecting lattices) I think one > should consider the variables as independent and random and it is possible to > recover the true intensities of a unique lattice from the stronger > diffracting > one (see for example Jenni & Ban, 2009, Acta D65, 101-111). Hence, the > quadratic formula (reported fomr Jens Kaiser) can be applied; > > 2) for TLS (merohedral twinning - perfectly overlapping spots) I think one > should not consider the two variable independent since they are related by > alpha (see the formulas I reported in my first message). In this case, I > think > the right formula should be that reported by Tim Grune, that as far as I know > overestimates the true error but in this case the quadratic is not applicable. > > Therefore, one would be prone to conclude that the uncertainties associated > to > merohedral-twinned crystals are higher than regular crystals or > non-merohedral > crystals. What's your opinion about? > > > In data mercoledì 6 novembre 2013 23:29:01, Jens Kaiser ha scritto: > > Tassos, > > I'm no expert either, and there are caveats for using this formula on > > correlated magnitudes. But I would assume that the intensities of twin > > related reflections should be independent from each other (that's my > > understanding of the sigmoid cumulative intensity distribution of > > twins). Thus, I think the simple Gaussian error propagation should be > > applicable to uncertainty estimates in detwinned intensities. > > > > Cheers, > > > > Jens > > > > On Thu, 2013-11-07 at 08:09 +0100, Anastassis Perrakis wrote: > > > Dear Jens, > > > > > > > > > That formula for error propagation is correct for independent > > > measurements. > > > Does this assumption stand true for Intensities in twinning? I am no > > > expert, but I would think not. > > > > > > > > > Tassos > > > > > > On 7 Nov 2013, at 7:53, Jens Kaiser wrote: > > > > Fulvio, Tim, > > > > > > > > error propagation is correct, but wrongly applied in Tim's > > > > > > > > example. > > > > s_f= \sqrt{ \left(\frac{\partial f}{\partial {x} }\right)^2 s_x^2 + > > > > \left(\frac{\partial f}{\partial {y} }\right)^2 s_y^2 + > > > > \left(\frac{\partial f}{\partial {z} }\right)^2 s_z^2 + ...} (see > > > > http://en.wikipedia.org/wiki/Propagation_of_uncertainty#Simplification) > > > > The uncertainty in a derived magnitude is always larger than any > > > > individual uncertainty, so no subtraction, anytime. Furthermore, in > > > > Tim's example you could end up with negative sigmas.. > > > > > > > > HTH, > > > > > > > > Jens > > > > > > > > On Thu, 2013-11-07 at 04:44 +0100, Tim Gruene wrote: > > > > > Dear Fulvio, > > > > > > > > > > with simple error propagation, the error would be > > > > > sigma(I(h1)) = (1-α)sigma(Iobs(h1))-α*sigma(Iobs(h2))/(1-2α) > > > > > > > > > > would it not? > > > > > > > > > > Although especially for theoretical aspects you should be concerned > > > > > about division by zero. > > > > > > > > > > Best, > > > > > Tim > > > > > > > > > > On 11/06/2013 05:54 PM, Fulvio Saccoccia wrote: > > > > > > Thank you for reply. My question mostly concern a theoretical > > > > > > aspect rather than practical one. To be not misunderstood, what is > > > > > > the mathematical model that one should apply to be able to deal > > > > > > with twinned intensities with their errors? I mean, I+_what? I ask > > > > > > this In order to state some general consideration on the accuracy > > > > > > about the recovery the true intensities on varying of alpha. Thanks > > > > > > > > > > > > Fulvio > > > > > > > > >
Re: [ccp4bb] [ccp4bb] AW: [ccp4bb] uncertainites associated with intensities from twinned crystals; Sorry for HTML.
Dear all, thank you for your reply. I would summarize my concerns and opinions, so far: 1) for QTLS (non-merohedral twinning - non intersecting lattices) I think one should consider the variables as independent and random and it is possible to recover the true intensities of a unique lattice from the stronger diffracting one (see for example Jenni & Ban, 2009, Acta D65, 101-111). Hence, the quadratic formula (reported fomr Jens Kaiser) can be applied; 2) for TLS (merohedral twinning - perfectly overlapping spots) I think one should not consider the two variable independent since they are related by alpha (see the formulas I reported in my first message). In this case, I think the right formula should be that reported by Tim Grune, that as far as I know overestimates the true error but in this case the quadratic is not applicable. Therefore, one would be prone to conclude that the uncertainties associated to merohedral-twinned crystals are higher than regular crystals or non-merohedral crystals. What's your opinion about? In data mercoledì 6 novembre 2013 23:29:01, Jens Kaiser ha scritto: > Tassos, > I'm no expert either, and there are caveats for using this formula on > correlated magnitudes. But I would assume that the intensities of twin > related reflections should be independent from each other (that's my > understanding of the sigmoid cumulative intensity distribution of > twins). Thus, I think the simple Gaussian error propagation should be > applicable to uncertainty estimates in detwinned intensities. > > Cheers, > > Jens > > On Thu, 2013-11-07 at 08:09 +0100, Anastassis Perrakis wrote: > > Dear Jens, > > > > > > That formula for error propagation is correct for independent > > measurements. > > Does this assumption stand true for Intensities in twinning? I am no > > expert, but I would think not. > > > > > > Tassos > > > > On 7 Nov 2013, at 7:53, Jens Kaiser wrote: > > > Fulvio, Tim, > > > > > > error propagation is correct, but wrongly applied in Tim's > > > > > > example. > > > s_f= \sqrt{ \left(\frac{\partial f}{\partial {x} }\right)^2 s_x^2 + > > > \left(\frac{\partial f}{\partial {y} }\right)^2 s_y^2 + > > > \left(\frac{\partial f}{\partial {z} }\right)^2 s_z^2 + ...} (see > > > http://en.wikipedia.org/wiki/Propagation_of_uncertainty#Simplification) > > > The uncertainty in a derived magnitude is always larger than any > > > individual uncertainty, so no subtraction, anytime. Furthermore, in > > > Tim's example you could end up with negative sigmas.. > > > > > > HTH, > > > > > > Jens > > > > > > On Thu, 2013-11-07 at 04:44 +0100, Tim Gruene wrote: > > > > Dear Fulvio, > > > > > > > > with simple error propagation, the error would be > > > > sigma(I(h1)) = (1-α)sigma(Iobs(h1))-α*sigma(Iobs(h2))/(1-2α) > > > > > > > > would it not? > > > > > > > > Although especially for theoretical aspects you should be concerned > > > > about division by zero. > > > > > > > > Best, > > > > Tim > > > > > > > > On 11/06/2013 05:54 PM, Fulvio Saccoccia wrote: > > > > > Thank you for reply. My question mostly concern a theoretical > > > > > aspect rather than practical one. To be not misunderstood, what is > > > > > the mathematical model that one should apply to be able to deal > > > > > with twinned intensities with their errors? I mean, I+_what? I ask > > > > > this In order to state some general consideration on the accuracy > > > > > about the recovery the true intensities on varying of alpha. Thanks > > > > > > > > > > Fulvio > > > > > > > > > > Fulvio Saccoccia PhD Dept. of Biochemical Sciences Sapienza > > > > > University of Rome 5, Piazzale A. Moro 00185 phone +39 0649910556 > > > > > > > > > > Messaggio Originale Da: herman.schreu...@sanofi.com > > > > > Inviato: 06/11/2013, 17:25 A: CCP4BB@JISCMAIL.AC.UK Oggetto: > > > > > [ccp4bb] AW: [ccp4bb] uncertainites associated with intensities > > > > > from twinned crystals > > > > > > > > > > > > > > > Dear Fulvio, you cannot detwin perfectly twinned data with this > > > > > formula. The term (1-2α) becomes zero, so you are dividing by zero. > > > > > With good refinement programs (ShelX, Refmac), refinement is done > > > > > against twinned data, which is better than to detwin the data with > > > > > the formula you mention. > > > > > > > > > > As I understand it, to get map coefficients, the calculated > > > > > contribution of the twin domain (Fcalc’s) is substracted from Fobs > > > > > (with the appropriate weighting factors), so what you see in coot > > > > > is detwinned electron density. In practical terms, the only thing > > > > > you have to do is to specify the TWIN keyword in Refmac. > > > > > > > > > > Best regards, Herman > > > > > > > > > > > > > > > > > > > > Von: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] Im Auftrag > > > > > von Fulvio Saccoccia Gesendet: Mittwoch, 6. November 2013 16:58 An: > > > > > CCP4BB@JISCMAIL.AC.UK Betreff: [ccp4bb] uncertainites associated
[ccp4bb] AW: [ccp4bb] [ccp4bb] AW: [ccp4bb] uncertainites associated with intensities from twinned crystals; Sorry for HTML.
Dear Fulvio and others, I do not understand this whole discussion. In case of perfectly twinned crystals, it is impossible to derive a detwinned F1 and F2 from two independent, but otherwise identical measurements. In this case, the only signal is noise, and one could as well use a random generator to get the detwinned data. It makes perfectly sense to me that in this case the theoretical error would be infinite. In practical terms, since in case of twinning intensities and not structure factors are added, the error cannot be larger than twice the largest of the two measurements plus twice the error for that measurement. There might be a formula to properly calculate this error. My 2 cents, Herman -Ursprüngliche Nachricht- Von: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] Im Auftrag von Jens Kaiser Gesendet: Donnerstag, 7. November 2013 08:29 An: CCP4BB@JISCMAIL.AC.UK Betreff: Re: [ccp4bb] [ccp4bb] AW: [ccp4bb] uncertainites associated with intensities from twinned crystals; Sorry for HTML. Tassos, I'm no expert either, and there are caveats for using this formula on correlated magnitudes. But I would assume that the intensities of twin related reflections should be independent from each other (that's my understanding of the sigmoid cumulative intensity distribution of twins). Thus, I think the simple Gaussian error propagation should be applicable to uncertainty estimates in detwinned intensities. Cheers, Jens On Thu, 2013-11-07 at 08:09 +0100, Anastassis Perrakis wrote: > Dear Jens, > > > That formula for error propagation is correct for independent > measurements. > Does this assumption stand true for Intensities in twinning? I am no > expert, but I would think not. > > > Tassos > > On 7 Nov 2013, at 7:53, Jens Kaiser wrote: > > > Fulvio, Tim, > > error propagation is correct, but wrongly applied in Tim's > > example. > > s_f= \sqrt{ \left(\frac{\partial f}{\partial {x} }\right)^2 s_x^2 + > > \left(\frac{\partial f}{\partial {y} }\right)^2 s_y^2 + > > \left(\frac{\partial f}{\partial {z} }\right)^2 s_z^2 + ...} (see > > http://en.wikipedia.org/wiki/Propagation_of_uncertainty#Simplificati > > on) The uncertainty in a derived magnitude is always larger than any > > individual uncertainty, so no subtraction, anytime. Furthermore, in > > Tim's example you could end up with negative sigmas.. > > > > HTH, > > > > Jens > > > > > > On Thu, 2013-11-07 at 04:44 +0100, Tim Gruene wrote: > > > Dear Fulvio, > > > > > > with simple error propagation, the error would be > > > sigma(I(h1)) = (1-α)sigma(Iobs(h1))-α*sigma(Iobs(h2))/(1-2α) > > > > > > would it not? > > > > > > Although especially for theoretical aspects you should be > > > concerned about division by zero. > > > > > > Best, > > > Tim > > > > > > On 11/06/2013 05:54 PM, Fulvio Saccoccia wrote: > > > > Thank you for reply. My question mostly concern a theoretical > > > > aspect rather than practical one. To be not misunderstood, what > > > > is the mathematical model that one should apply to be able to > > > > deal with twinned intensities with their errors? I mean, > > > > I+_what? I ask this In order to state some general consideration > > > > on the accuracy about the recovery the true intensities on > > > > varying of alpha. Thanks Fulvio > > > > > > > > Fulvio Saccoccia PhD Dept. of Biochemical Sciences Sapienza > > > > University of Rome 5, Piazzale A. Moro 00185 phone +39 > > > > 0649910556 > > > > > > > > Messaggio Originale Da: herman.schreu...@sanofi.com > > > > Inviato: 06/11/2013, 17:25 A: CCP4BB@JISCMAIL.AC.UK Oggetto: > > > > [ccp4bb] AW: [ccp4bb] uncertainites associated with intensities > > > > from twinned crystals > > > > > > > > > > > > Dear Fulvio, you cannot detwin perfectly twinned data with this > > > > formula. The term (1-2α) becomes zero, so you are dividing by zero. > > > > With good refinement programs (ShelX, Refmac), refinement is > > > > done against twinned data, which is better than to detwin the > > > > data with the formula you mention. > > > > > > > > As I understand it, to get map coefficients, the calculated > > > > contribution of the twin domain (Fcalc’s) is substracted from > > > > Fobs (with the appropriate weighting factors), so what you see > > > > in coot is detwinned electron density. In pra
Re: [ccp4bb] [ccp4bb] AW: [ccp4bb] uncertainites associated with intensities from twinned crystals; Sorry for HTML.
Tassos, I'm no expert either, and there are caveats for using this formula on correlated magnitudes. But I would assume that the intensities of twin related reflections should be independent from each other (that's my understanding of the sigmoid cumulative intensity distribution of twins). Thus, I think the simple Gaussian error propagation should be applicable to uncertainty estimates in detwinned intensities. Cheers, Jens On Thu, 2013-11-07 at 08:09 +0100, Anastassis Perrakis wrote: > Dear Jens, > > > That formula for error propagation is correct for independent > measurements. > Does this assumption stand true for Intensities in twinning? I am no > expert, but I would think not. > > > Tassos > > On 7 Nov 2013, at 7:53, Jens Kaiser wrote: > > > Fulvio, Tim, > > error propagation is correct, but wrongly applied in Tim's > > example. > > s_f= \sqrt{ \left(\frac{\partial f}{\partial {x} }\right)^2 s_x^2 + > > \left(\frac{\partial f}{\partial {y} }\right)^2 s_y^2 + > > \left(\frac{\partial f}{\partial {z} }\right)^2 s_z^2 + ...} (see > > http://en.wikipedia.org/wiki/Propagation_of_uncertainty#Simplification) > > The uncertainty in a derived magnitude is always larger than any > > individual uncertainty, so no subtraction, anytime. Furthermore, in > > Tim's example you could end up with negative sigmas.. > > > > HTH, > > > > Jens > > > > > > On Thu, 2013-11-07 at 04:44 +0100, Tim Gruene wrote: > > > Dear Fulvio, > > > > > > with simple error propagation, the error would be > > > sigma(I(h1)) = (1-α)sigma(Iobs(h1))-α*sigma(Iobs(h2))/(1-2α) > > > > > > would it not? > > > > > > Although especially for theoretical aspects you should be concerned > > > about division by zero. > > > > > > Best, > > > Tim > > > > > > On 11/06/2013 05:54 PM, Fulvio Saccoccia wrote: > > > > Thank you for reply. My question mostly concern a theoretical > > > > aspect rather than practical one. To be not misunderstood, what is > > > > the mathematical model that one should apply to be able to deal > > > > with twinned intensities with their errors? I mean, I+_what? I ask > > > > this In order to state some general consideration on the accuracy > > > > about the recovery the true intensities on varying of alpha. Thanks > > > > Fulvio > > > > > > > > Fulvio Saccoccia PhD Dept. of Biochemical Sciences Sapienza > > > > University of Rome 5, Piazzale A. Moro 00185 phone +39 0649910556 > > > > > > > > Messaggio Originale Da: herman.schreu...@sanofi.com > > > > Inviato: 06/11/2013, 17:25 A: CCP4BB@JISCMAIL.AC.UK Oggetto: > > > > [ccp4bb] AW: [ccp4bb] uncertainites associated with intensities > > > > from twinned crystals > > > > > > > > > > > > Dear Fulvio, you cannot detwin perfectly twinned data with this > > > > formula. The term (1-2α) becomes zero, so you are dividing by zero. > > > > With good refinement programs (ShelX, Refmac), refinement is done > > > > against twinned data, which is better than to detwin the data with > > > > the formula you mention. > > > > > > > > As I understand it, to get map coefficients, the calculated > > > > contribution of the twin domain (Fcalc’s) is substracted from Fobs > > > > (with the appropriate weighting factors), so what you see in coot > > > > is detwinned electron density. In practical terms, the only thing > > > > you have to do is to specify the TWIN keyword in Refmac. > > > > > > > > Best regards, Herman > > > > > > > > > > > > > > > > Von: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] Im Auftrag > > > > von Fulvio Saccoccia Gesendet: Mittwoch, 6. November 2013 16:58 An: > > > > CCP4BB@JISCMAIL.AC.UK Betreff: [ccp4bb] uncertainites associated > > > > with intensities from twinned crystals > > > > > > > > > > > > Dear ccp4 users > > > > > > > > a question about the recovering of true intensities from merohedral > > > > twinned crystal. Providing alpha and the twin operator one should > > > > be able to recover the intensities from the formulas: > > > > > > > > > > > > > > > > I(h1) = (1-α)Iobs(h1)-αIobs(h2)/(1-2α) > > > > > > > > I(h2) = -αIobs(h1)+(1+α)Iobs(h2)/(1-2α) > > > > > > > > as stated in many papers and books*. > > > > > > > > However I was wondering about the uncertainties associated to these > > > > measurements, I mean: for all physical observable an uncertainty > > > > should be given. > > > > > > > > Hence, what is the uncertainty associated to a perfect merohedrally > > > > twinned crystal (alpha=0.5)? It is clear that in this case we drop > > > > in a singular value of the above formulas. > > > > > > > > Please, let me know your hints or your concerns on the matter. > > > > Probably there is something that it is not so clear to me. > > > > > > > > > > > > > > > > Thanks in advance > > > > > > > > > > > > > > > > Fulvio > > > > > > > > > > > > > > > > > > > > > > > > ref. **(C. Giacovazzo, H. L. Monaco, G. Artioli, D. Viterbo, M. > > > > Milaneso, G. Ferraris, G. Gilli, P. Gilli, G. Zanotti and M. Catti. > >
Re: [ccp4bb] [ccp4bb] AW: [ccp4bb] uncertainites associated with intensities from twinned crystals; Sorry for HTML.
Dear Jens, That formula for error propagation is correct for independent measurements. Does this assumption stand true for Intensities in twinning? I am no expert, but I would think not. Tassos On 7 Nov 2013, at 7:53, Jens Kaiser wrote: > Fulvio, Tim, > error propagation is correct, but wrongly applied in Tim's example. > (see http://en.wikipedia.org/wiki/Propagation_of_uncertainty#Simplification) > The uncertainty in a derived magnitude is always larger than any individual > uncertainty, so no subtraction, anytime. Furthermore, in Tim's example you > could end up with negative sigmas.. > > HTH, > > Jens > > > On Thu, 2013-11-07 at 04:44 +0100, Tim Gruene wrote: >> >> Dear Fulvio, >> >> with simple error propagation, the error would be >> sigma(I(h1)) = (1-α)sigma(Iobs(h1))-α*sigma(Iobs(h2))/(1-2α) >> >> would it not? >> >> Although especially for theoretical aspects you should be concerned >> about division by zero. >> >> Best, >> Tim >> >> On 11/06/2013 05:54 PM, Fulvio Saccoccia wrote: >> > Thank you for reply. My question mostly concern a theoretical >> > aspect rather than practical one. To be not misunderstood, what is >> > the mathematical model that one should apply to be able to deal >> > with twinned intensities with their errors? I mean, I+_what? I ask >> > this In order to state some general consideration on the accuracy >> > about the recovery the true intensities on varying of alpha. Thanks >> > Fulvio >> > >> > Fulvio Saccoccia PhD Dept. of Biochemical Sciences Sapienza >> > University of Rome 5, Piazzale A. Moro 00185 phone +39 0649910556 >> > >> > Messaggio Originale Da: herman.schreu...@sanofi.com >> > Inviato: 06/11/2013, 17:25 A: CCP4BB@JISCMAIL.AC.UK Oggetto: >> > [ccp4bb] AW: [ccp4bb] uncertainites associated with intensities >> > from twinned crystals >> > >> > >> > Dear Fulvio, you cannot detwin perfectly twinned data with this >> > formula. The term (1-2α) becomes zero, so you are dividing by zero. >> > With good refinement programs (ShelX, Refmac), refinement is done >> > against twinned data, which is better than to detwin the data with >> > the formula you mention. >> > >> > As I understand it, to get map coefficients, the calculated >> > contribution of the twin domain (Fcalc’s) is substracted from Fobs >> > (with the appropriate weighting factors), so what you see in coot >> > is detwinned electron density. In practical terms, the only thing >> > you have to do is to specify the TWIN keyword in Refmac. >> > >> > Best regards, Herman >> > >> > >> > >> > Von: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] Im Auftrag >> > von Fulvio Saccoccia Gesendet: Mittwoch, 6. November 2013 16:58 An: >> > CCP4BB@JISCMAIL.AC.UK Betreff: [ccp4bb] uncertainites associated >> > with intensities from twinned crystals >> > >> > >> > Dear ccp4 users >> > >> > a question about the recovering of true intensities from merohedral >> > twinned crystal. Providing alpha and the twin operator one should >> > be able to recover the intensities from the formulas: >> > >> > >> > >> > I(h1) = (1-α)Iobs(h1)-αIobs(h2)/(1-2α) >> > >> > I(h2) = -αIobs(h1)+(1+α)Iobs(h2)/(1-2α) >> > >> > as stated in many papers and books*. >> > >> > However I was wondering about the uncertainties associated to these >> > measurements, I mean: for all physical observable an uncertainty >> > should be given. >> > >> > Hence, what is the uncertainty associated to a perfect merohedrally >> > twinned crystal (alpha=0.5)? It is clear that in this case we drop >> > in a singular value of the above formulas. >> > >> > Please, let me know your hints or your concerns on the matter. >> > Probably there is something that it is not so clear to me. >> > >> > >> > >> > Thanks in advance >> > >> > >> > >> > Fulvio >> > >> > >> > >> > >> > >> > ref. **(C. Giacovazzo, H. L. Monaco, G. Artioli, D. Viterbo, M. >> > Milaneso, G. Ferraris, G. Gilli, P. Gilli, G. Zanotti and M. Catti. >> > Fundamentals of Crystallography, 3rd edition. IUCr Texts on >> > Crystallography No. 15, IUCr/Oxford University Press, 2011; >> > Chandra, N., Acharya, K. R., Moody, P. C. (1999). Acta Cryst. D55. >> > 1750-1758) >> > >> > -- >> > >> > Fulvio Saccoccia, PhD >> > >> > Dept. of Biochemical Sciences "A. Rossi Fanelli" >> > >> > Sapienza University of Rome >> > >> > Tel. +39 0649910556 >> > >> >