Dear Georg,
On Wed, 24 Jan 2018 14:38:09 +0100, Georg Mlynek <georg.mly...@univie.ac.at> wrote: >Dear Kay, thank you ver much for the (as always) detailed and nicely >explained answer. > >However this brings up some questions for me: > >1. Could you please tell me how the "correct high-resolution cutoff" >will effect the data processing in the INTEGRATE CORRECT step. > >In other words what will be the difference if I leave >INCLUDE_RESOLUTION_RANGE=50.0 0.0 a) it will take a bit longer to process the data b) the statistics, in particular the overall statistics, will not be meaningful since they will include shells that one is not interested in I'm not saying that overall statistics are necessarily useful, but to get an idea about the data quality, it is better to have statistics for 9 resolution shells to look at, than just 3 or 4 or 5! Concerning data quality, the results in the good resolution should not be different whether you also integrate higher shells without signal, or not. Personally, I always use INCLUDE_RESOLUTION_RANGE=X 0.0 in the first run, where X depends on beamstop shadow size and position. And I always mask the beamstop unless the shadow is circular and centered. In the optimization run, I adjust the high resolution cutoff, and FRIEDEL'S_LAW depending on what I find in the first run. > >2. Additionally if I leave FRIEDEL'S_LAW= TRUE in XDS.INP and choose >FRIEDEL'S_LAW= FALSE in XDSCONV.INP will the results be different? > if there actually _is_ anomalous signal but you have FRIEDEL'S_LAW= TRUE in XDS.INP, then you lie to the program, and you risk rejection of strong anomalous differences as outliers. Conversely, if you have FRIEDEL'S_LAW= FALSE in XDS but there is no anomalous signal, the statistics are less useful, and the outlier rejection is less efficient. >3. With all the optimizations done in the last years are these tips and >tricks still valid? > >FRIEDEL'S_LAW=FALSE ! This acts only on the CORRECT step ! If the anom >signal turns out to be, or is known to be, very low or absent, ! use >FRIEDEL'S_LAW=TRUE instead (or comment out the line); re-run CORRECT ! >remove the "!" in the following line: ! This should not be seen as a trick, rather as saying "just use what is adequate for the situation" >STRICT_ABSORPTION_CORRECTION=TRUE ! if the anomalous signal is strong: >in that case, in CORRECT.LP the three ! "CHI^2-VALUE OF FIT OF >CORRECTION FACTORS" values are significantly> 1, e.g. 1.5 (from >https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/2QVO.xds) this is what I do but it might not make a big difference in practice. But to see CHI^2 come down to 1 with STRICT_ABSORPTION_CORRECTION=TRUE when it is 2 with STRICT_ABSORPTION_CORRECTION=FALSE tells me that it is actually the anomalous signal that was responsible for the elevated differences between sym-related observations. I understand your questions in a context of streamlined data processing, and economizing efforts and time. On the other hand, questions such as - why not try make the most of your data? Sometimes this might tip the scale in your favor, and allow to solve a difficult structure - but you will never know unless you try - by _not_ adapting XDS.INP to the actual data, and running a second INTEGRATE CORRECT, can one really save time in the long run? After all, getting the crystals was worth your time, and that probably took much longer. must then also be answered. best wishes, Kay > >Many thanks in advance, best regards, Georg. > >Am 23.01.2018 um 21:16 schrieb Kay Diederichs: >> Dear Oliver, >> >> sorry for the trouble! >> A default should be correct in the majority of situations, but it's >> impossible to have it work for _all_ situations. The XDS default for >> REFINE(IDXREF) was changed (i.e. POSITION was removed) to improve the >> indexing for weak and lousy data, _and_ because the distance values from the >> header are nowadays almost always accurate. For data with very high >> resolution such as yours, and significantly wrong distance, this means that >> at high resolution in particular this default will lead to worse data. >> generate_XDS.INP (in XDSwiki) and (I think) autoPROC and xia2 explicitly >> include POSITION in the REFINE(IDXREF), i.e. override the default. Where did >> you get XDS.INP from? >> Some comments: >> a) this is why I recommend, for data sets that may be important, to always >> do one cycle of optimization, i.e. after CORRECT, "mv GXPARM.XDS XPARM.XDS", >> change XDS.INP to have the correct high-resolution cutoff (cutting after the >> last shell which still has a "*" in the CC1/2 column), adjust the >> FRIEDEL'S_LAW= setting, and run JOB=INTEGRATE CORRECT. In your case this >> would make the refined distance available to INTEGRATE, and should result in >> very good data. >> b) at which beamline did you collect the data? Such a high difference >> between refined distance and distance from header is unusual, and should be >> reported to (and fixed by) the beamline staff. >> c) for this beamline at least, one should use REFINE(IDXREF)=CELL BEAM >> ORIENTATION AXIS POSITION . I understand that you tried this and it didn't >> work? Maybe there is something else wrong then, e.g. another line later in >> XDS.INP resetting REFINE(IDXREF) to something else. >> >> If you cannot find the reason why including POSITiON into REFINE(IDXREF) >> does not help, pls send me enough data to reproduce the problem. >> >> best wishes, >> >> Kay >> >> On Tue, 23 Jan 2018 14:31:09 +0000, "Weiergräber, Oliver H." >> <o.h.weiergrae...@fz-juelich.de> wrote: >> >>> Dear all, >>> >>> After upgrading XDS from build date 20170601 to 20171218, I am experiencing >>> severe degradation of apparent data quality reported by CORRECT for certain >>> data sets. Following first indications of issues with a slightly >>> problematic candidate, I went back to a previously very well-behaved test >>> case with diffraction extending beyond 1.1 A. Using the same input with >>> both program versions, statistics are not too different out to approx. 1.6 >>> A, but become more and more divergent in outer shells. These are the values >>> for the highest-resolution shell (1.10--1.16 A): >>> >>> 20170601: I/sigI = 1.80; Rmeas = 59.9 %; CC(1/2) = 82.0 % >>> 20171218: I/sigI = 0.14; Rmeas = 506.4 %; CC(1/2) = 5.8 % >>> >>> The refined cell constants are unreasonably different as well. Obviously, >>> something is going awfully wrong here, presumably related to errors in >>> geometry parameters (which are expected to become increasingly detrimental >>> with decreasing dmin). As it turns out, geometry refinement behaves >>> differently in the latest version of XDS: most notably, IDXREF no longer >>> refines the detector distance by default. This is significant in this case, >>> as in version 20170601 the distance would shift by as much as 1.3 mm, >>> resulting in successful integration and scaling. Unfortunately, re-adding >>> POSITION refinement into REFINE(IDXREF) does not help, and even having it >>> in all refinement scopes (IDXREF, INTEGRATE, CORRECT) only yields a limited >>> improvement of CORRECT statistics. >>> It is worth noting that examination of a dataset from an unrelated crystal >>> (dmin = 1.4 A) did not reveal such enormous differences -- in this case, >>> however, the shift in detector distance applied in the old version of XDS >>> was quite small (0.2 mm), which, together with the lower resolution, >>> explains the absence of large discrepancies. >>> >>> To sum up, I suspect that, due to recent changes to the XDS code, the >>> refinement of geometry parameters is now overly restrained, resulting in >>> drastic problems in cases where the detector distance is not as precise as >>> desirable (and even more so at very high resolution). For such datasets, >>> the new version appears to be essentially unusable (at least within the >>> parameter space I have tested), and even in modest cases may often be >>> inferior to the previous one. Is there an option to revert to the former >>> behaviour? >>> >>> Best regards >>> Oliver >>> >>> ================================================ >>> PD Dr. Oliver H. Weiergr�ber >>> Institute of Complex Systems >>> ICS-6: Structural Biochemistry >>> Tel.: +49 2461 61-2028 >>> Fax: +49 2461 61-9540 >>> ================================================ >>> >>> >>> >>> >>> ------------------------------------------------------------------------------------------------ >>> ------------------------------------------------------------------------------------------------ >>> Forschungszentrum Juelich GmbH >>> 52425 Juelich >>> Sitz der Gesellschaft: Juelich >>> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 >>> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher >>> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), >>> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, >>> Prof. Dr. Sebastian M. Schmidt >>> ------------------------------------------------------------------------------------------------ >>> ------------------------------------------------------------------------------------------------
