The CCP4 routines for MTZ and map files are written in C and thus do not use a
Fortran unformatted OPEN statement, they use C-style block read & write.
Cheers
-- Ian
On Wed, 14 Nov 2018 at 08:59, Kay Diederichs <kay.diederi...@uni-konstanz.de
<mailto:kay.diederi...@uni-konstanz.de>> wrote:
It is not necessary to do error-prone conversions manually: the ifort
Compiler understands the convert='VAXD' Option in its OPEN statement - see
https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-open-convert-specifier
Thus one could just write a tiny read-write loop.
HTH
Kay
On Wed, 14 Nov 2018 00:51:02 +0000, Zhijie Li <zhijie...@utoronto.ca
<mailto:zhijie...@utoronto.ca>> wrote:
>It's also said here, at the end of file :
>
>https://www.cs.auckland.ac.nz/~patrice/210LN/DR4.pdf
>
>"add 1 to the left, with the binary point"
>
>0.10000.....
>
>
>
>________________________________
>From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK <mailto:CCP4BB@JISCMAIL.AC.UK>> on
behalf of Zhijie Li <zhijie...@utoronto.ca <mailto:zhijie...@utoronto.ca>>
>Sent: Tuesday, November 13, 2018 7:43 PM
>To: CCP4BB@JISCMAIL.AC.UK <mailto:CCP4BB@JISCMAIL.AC.UK>
>Subject: Re: [ccp4bb] VERY old mtz file..
>
>
>Hi all,
>
>
>I think I know why it is a division of 4 instead of 2 is involved in
conversion from VAX to IEEE now. Short answer: a 2 is in the exponent bits (bias
of 128 instead of 127, visible), another 2 is hidden in the scientific notation.
>
>
>I found this explanation+example on VAX F-float:
>
>http://courseweb.stthomas.edu/tpsturm/private/notes/qm300/FLOATPT.html
>
>
>So for IEEE754 float32, if we want to represent a same 12.75 (1100.11) in
the above example, we would first conceptually write it in scientific notation as
1.10011 x 1000 in binary. Then the mantissa part is the part after the dot filled
with zero to 23 bits: '10011000000000000000000', the exponent part is 3+127=130
(dec)=10000010(bin). Then the binary IEEE754 float32 number is
0[10000010][10011000000000000000000]. (You can check it here:
https://www.h-schmidt.net/FloatConverter/IEEE754.html)
>
>
>Now compare this with the VAX 12.75 in the linked example, we can find
that besides the bias becoming 128, the conceptual binary scientific notation is
actually 0.110011 x 10000, instead of 1.10011 x 1000. So the exponent needed is
4 instead of 3. Then the exponent bits are 4+128=132=10000100 and the VAX float32
becomes
>
>0[10000100][10011000000000000000000] ---if we write in a IEEE-style order.
Note that the mantissa appears to be same as the ieee mantissa, and the exponent
to be applied is 132-128=4. If this number is interpreted as IEEE754, then it will
be 1.10011 x 2exp(132-127)=1.10011 x 100000, four times of what it should be.
>
>
>So, for normalised values, rearranging the VAX F-float bytes, reading as
IEEE, then dividing by 4 gives the correct value. (The C[0]-1 treatment in the
ccp4 lib is neat.)
>
>
>In this link describing VAX floats, it is unfortunate that it only states
that the bias for F-float is 128, but not that the mantissa starts from 0.01
instead of 0.1. Therefore the confusion.
>
>https://nssdc.gsfc.nasa.gov/nssdc/formats/VAXFloatingPoint.htm
>
>
>
>Thanks to all responded!
>
>
>Zhijie
>
>
>________________________________
>From: Ian Tickle <ianj...@gmail.com <mailto:ianj...@gmail.com>>
>Sent: Tuesday, November 13, 2018 4:54 PM
>To: Zhijie Li
>Cc: CCP4BB@JISCMAIL.AC.UK <mailto:CCP4BB@JISCMAIL.AC.UK>
>Subject: Re: [ccp4bb] VERY old mtz file..
>
>
>Hi Zhijie
>
>It's definitely a factor 4. The code is in subroutine QTIEEE in the
Fortran source I mentioned previously at this line:
>
>See line:
>
> A(I)=((A(I)+SIGN(2,A(I)))/4.AND..NOT.MNAN).OR.MDN2
>
>If you prefer it in C code it's in function vaxF2ieeeF in:
>
>ccp4-7.0-src/checkout/libccp4/ccp4/vmslibrary.c
>
>See line:
>
>out.c[0] = buffer[i].c[1] - (uint8)1; /* subtracts 2 from exponent */
>
>i.e. subtract 2 from exponent -> division by 4.
>
>Cheers
>
>-- Ian
>
>
>On Tue, 13 Nov 2018 at 19:52, Zhijie Li <zhijie...@utoronto.ca
<mailto:zhijie...@utoronto.ca><mailto:zhijie...@utoronto.ca
<mailto:zhijie...@utoronto.ca>>> wrote:
>If somebody is going to send these files by email, please send one to me
too. Thanks in advance. I actually prefer to get a MTZ file because the miller
indices would serve as good clues for understanding the encodings. Even the first
1024 bytes of an MTZ would do (data array starts at byte 80 in MTZ).
>
>In my life I had only seen ieee754. According to what I can find, VAX has
an exponent bias of 128 (ieee754 uses 127). Then it seems to me that when
converting from vax to ieee a division of 2 is involved. However all procedures I
have seen use a division of 4, which is quite puzzling to me. A real data file
containing meaningful numbers (eg., HKL indices) would be very helpful. Thanks in
advance.
>
>Zhijie
>
>> On Nov 13, 2018, at 2:21 PM, Johan Hattne <hat...@ucla.edu
<mailto:hat...@ucla.edu><mailto:hat...@ucla.edu <mailto:hat...@ucla.edu>>> wrote:
>>
>> Related by not exactly on topic: would anybody on the list be able to
share old map files (not MTZ:s) with Convex, Cray, Fujitsu, or VAX reals/strings?
I�d be interested to see what those files actually look(ed) like.
>>
>> // Best wishes; Johan
>>
>>> On Nov 9, 2018, at 18:38, Zhijie Li <zhijie...@utoronto.ca
<mailto:zhijie...@utoronto.ca><mailto:zhijie...@utoronto.ca
<mailto:zhijie...@utoronto.ca>>> wrote:
>>>
>>> Hi all,
>>>
>>> On linux there are a few good GUI HEX editors. Here I�d like to
recommend BLESS, which conveniently displays all possible numerical interpretations of
the four bytes under cursor. It also allows the user to switch between big endian or
little endian through a checkbox. Unfortunately all floats are assumed to be IEEE754,
therefore VAX floats won�t be interpreted correctly. ( The simplest way to convert vax
to ieee float would be to write a little program to do some bit operations. I�d be happy
to take that as my weekend project)
>>>
>>>
>>> BTW, along the line of space efficiency, I can�t help noticing that the
miller indices are saved as float32 in mtz, as all other numbers in mtz. This certainly
have made mtz format a beautiful homogeneous data format ;). In this particular case,
if we have doubts about the reliability of the machine stamp, trying to restore the
miller indices would be a good way to test hypotheses.
>>>
>>> Zhijie
>>>
>>>> On Nov 9, 2018, at 9:04 PM, James Holton <0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk
<mailto:0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk><mailto:0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk
<mailto:0000270165b9f4cf-dmarc-requ...@jiscmail.ac.uk>>> wrote:
>>>>
>>>> As a beamline scientist I must say I am glad that diffraction image
data is not usually stored as ASCII text. In fact, I am slowly warming to the idea of
storing it as not just binary, but compressed formats. Problem, I'm sure will be that it
won't be long before we forget how to decompress them, as most of the algorithms we are
using aren't all that widespread. Probably around the same time future generations will
curse us for using ASCII instead of unicode, which is a 16-bit standard. I'm sure we will
be reviled for limiting ourselves so, just to save a factor of two in disk space.
>>>> In situations like this I always use the unix "od" command. It makes everything
"human readable" by converting the bytes into strings you can read. Then it is just a matter of
figuring out what the bytes are.
>>>> Unfortunately, "od" only decodes floats on the native platform, so if
the mtz is from another platform (Windows vs Linux, for example), then you might need to do some
swapping. Thus far, I have encountered files that require one of a few swapping strategies in order
to make them work:
>>>>
>>>> 1 2 3 4 - no swapping
>>>>
>>>> 4 3 2 1 - reverse all bytes
>>>>
>>>> 3 4 1 2 - swap words and swap bytes within the words
>>>> 2 1 4 3 - reverse of previous
>>>>
>>>> 2-1 1 4 3 - same as last, but if not all zero, decrement byte #2
before swapping
>>>> 3 4 1 2+1 - same as 3412, but if not all zero increment byte #2 before
swapping
>>>> I'm sure there are other combinations, but the oldest MTZ I have is
only from 1996.
>>>>
>>>> -James Holton
>>>> MAD Scientist
>>>>
>>>>
>>>>> On 11/9/2018 4:47 AM, Eleanor Dodson wrote:
>>>>> Anyone any idea what to do about this?? Created in 1992!!
>>>>> Seems unreadable..
>>>>>
>>>>> No CTYP lines input for file: 1
>>>>> Indices output even if all data items flagged "missing"
>>>>> Warning, NOT all LABOUT data lines given
>>>>> Warning: Machine stamp corrupted? Assuming native format.
>>>>>>>>>>> CCP4 library signal library_file:End of File (Error)
>>>>>
>>>>>
>>>>> To unsubscribe from the CCP4BB list, click the following link:
>>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>>>>
>>>>
>>>>
>>>> To unsubscribe from the CCP4BB list, click the following link:
>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>>>
>>>
>>> To unsubscribe from the CCP4BB list, click the following link:
>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>>>
>>
>> Research Specialist @ Gonen Lab
>> ____________________________________________________
>> UCLA * 615 Charles E. Young Drive South
>> BSRB #347 * Los Angeles, CA 90095
>>
>> ########################################################################
>>
>> To unsubscribe from the CCP4BB list, click the following link:
>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>
>########################################################################
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>
>________________________________
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>
>########################################################################
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>
########################################################################
To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
