On 10/09/2020 13.25, sebb wrote:
Migration to Foal will be a huge job for some installations.
Whilst hopefully all snags will have been ironed out of any conversion
tool before it is deployed in earnest, it's possible that some edge
cases will cause issues, and will need subsequent adjustment.
Short of ironing out a standard for DKIM_ID, the migration tests I've
done have gone relatively well. There were IIRC a few snags, most
related to the ES 7.8.1 lib, but once I got migration started, it worked
as intended and everything on the new ES server was compatible. If we
could somehow get a migration test running on travis or such, that would
be ideal - but that is quite tricky - we'd have to maybe dockerize two
containers - one with old pony, one with foal, and then test migrating
across and checking that each document is obtainable.
To this end, I think it will be essential to know which records have
been migrated, and which version of the software was used to do so (as
well as the date).
It may be worth including version and timestamp info in the direct
archive and imports as well.
Do you mean adding a key/value to the migrated doc with a migration
note? That wouldn't be a bad idea, if nothing else, to keep score of
what was migrated and what's new.
One possible application would be to back-fill attachments which were
originally ignored.
This could be run as a background re-indexer perhaps? That grabs the
source document, re-parses attachments, and if it contained more than
originally thought, add them and update the email document.
S.
