On Monday, 28 September 2020 at 21:58:31 UTC, Steven
Schveighoffer wrote:
On 9/28/20 3:28 PM, Bastiaan Veelo wrote:
I’m leaning towards ditching the memory mapped I/O on the D
end, and replace it by regular serialisation/deserialisation.
That will be a manual rewrite though, which is a bit of bummer
as memory mapped files are widely used in our Pascal code. But
this will probably give the best end result.
2 things:
1. I agree this is the answer. If you ever ditch the old Pascal
code, then you can reactivate the memory-mapped code.
2. You can possibly do the translation outside of your
programs. That is, it wouldn't be entirely impossible to simply
have a process running that ensures the "D view" and the
"Pascal view" of the same file is kept in sync. Then you can
keep the memory mapped code the same, and just define sane
structures in your D code.
If you aren't required to have both Pascal and D programs
reading and writing the file at the same time, this shouldn't
be a problem.
There is no need to run both versions concurrently. The issue is
that design offices typically maintain a library of past designs
for as long as they are in existence, to build new designs off
of. So being able to read or import the files that were written
with an ancient version of our software is very valuable. Our old
compiler offered two alternatives for file i/o: one where all
elements are of the same type, the other one (memory mapped
files) being the "only" option for files of mixed type. Ideally,
the structs that are used for i/o do not have any pointers in
them, and certainly in the more recent file versions that would
be the case. In older versions that might not be the case; then
the pointers obviously would be given meaningful values after the
structs would have been read back in. These cases we would be
able to work around, though, by converting the old structs to new
ones upon import.
BTW, one further thing I don't understand -- if this is memory
mapped data, how come it has issues with the GC? And what do
the "pointers" mean in the memory mapped data? I'm sure there's
good answers, and your actual code is more complex than the
simple example, but I'm just curious.
The main problem is that the transpiler doesn't know which
structs are used for i/o and would need 1-byte alignment, and
which structs have pointers into GC memory and must not be 1-byte
aligned. The alternative to switching to
serialisation/deserialisation is to stay with the automated
translation of the memory mapped file implementation, not
automatically 1-byte align every struct but manually align the
ones that are used in i/o. This is however sensitive to mistakes,
and the translated mmfile implementation has a bit of a smell to
it. It is also not portable, as it uses the WinAPI directly.
Still, it may be the quickest route to get us back on track.
I am very glad to have identified the problem, and there being
ways to deal with it. I just hope this will be the last big
hurdle :-)
-Bastiaan.
