On Dienstag, 3. Oktober 2023 21:33:07 CEST Jack via KMyMoney-devel wrote: > On 2023.10.03 15:06, Jack via KMyMoney-devel wrote: > > First, it seems that the most recent good file is my explicit backup > > from 30 Sep and even the oldest autobackup from 02 Oct is bad. The > > main action between them was OFX direct connect download of all > > mapped accounts. Now I'm wondering if there was something funny in > > the import. Actually I know there was something funny, related to > > telling KMM not to create the unknown equity, and apparently having > > it stop the import (at least for that account) at that point. > > However, I don't see how that could create this particular problem. > > > > Comparing XML files is difficult, because (as has been discussed on > > the mailing list in the past,) the order of attributes is not > > consistent. > > > > xmllint shows as bad only the oldest four of the autobackup files (02 > > Oct from 16:57 to 17:13.) I tried using xmllint -c11n, to possibly > > be able to diff the various versions, but it won't process the four > > "bad" files. > > > > These four files seems to have </KMYMONEY-FILE> inserted immediately > > before <PRICEPAIR from="E000525" to="USD"> but otherwise show changes > > to the ACCOUNTS and TRANSACTIONS related to my edits as part of > > reconciliation. These all seem to have all the pricepairs intact. > > The ends of those files show all my reports, followed by <BUDGETS/> > > and <ONLINEJOBS/> but no </KMYMONEY-FILE>. > I just noticed a second problem, which I should have seen earlier. Not > only has </KMYMONEY-FILE> moved from the end of the file about 400 > lines up, but </PRICES> has also moved up, from immediately before > <REPORTS> to one line above </KMYMONEY-FILE>. > > > > The remaining files have </KMYMONEY-FILE> back to the end of the file > > but have the pricepairs after E000525 with only one (wrong) price, > > then empty reports, budgets, and online jobs, with the final > > </KMYMONEY-FILE> > > > > So, at some point after a big OFX import (and possibly other > > (relatively minor?) changes, the last line of the file got moved just > > over 400 lines up. It's likely that the next four files were > > sequential saves, without re-reading the file. I'm guessing that > > after saving those four (and probably some earlier ones that got > > rolled over with the limit of 10 autobackups) I restarted KMM, and > > although it read the bad file, without apparent complaint and saved > > valid files after that, it lost all custom reports, and dropped some > > pricepars totally and all except one price for the others (after the > > pricepair prior to the stray </KMYMONEY-FILE>. > > > > I've just captured console output from a clean start of KMM, reading > > first the most recent good file, and then the earliest bad file. > > Those logs are identical through loading all the plugins then differe > > in the file name (URL) being opened. There are differences in the > > number of items loaded for "P", accounts, "T", "E", all increasing. > > There is a decrease in number of prices, even though there are 151 > > more "<PRICE " entries in the file. Increase in number of splits. > > Differences in time to process home view sections. > > > > The message ' 0 instead of 1 arguments to message "The size of the > > file..." supplied before conversion' shows up for both files. > > > > My first attempt at a cut/paste fix didn't work, so I'll try once > > more again, otherwise I'll just resign myself to starting over from > > the file from the 30th. > I'll guess now that it didn't work because I didn't correct the > misplaced </PRICES> line.
I re-visited this issue and noticed a possibility that could cause the problem. https://invent.kde.org/office/kmymoney/-/commit/edb22bd2be7dd7c29bd94e14f6a4632866c44ccc should fix it. The reason is when erroneous entries are detected during writing (see the code following the above change which contains continue statements that cause to reach the m_writer->writeEndElement() line again. Loop twice and you get </PRICES> in the first iteration and </KMYMONEY-FILE> during the second one. Only in that case another <PRICEPAIR> can be issued. Resetting the flag immediately prevents issuing the </PRICES> and </KMYMONEY-FILE> markers in this situation. @Jack: I expect those weird price entries to happen during OFX import. So maybe you can check in that area (OFX prices imported into the last good data file). -- Regards Thomas Baumgart ------------------------------------------------------------- As soon as there is an optimist who discovers a light that does not exist there is a pessimist who blows it out. (Giovanni Guareschi) -------------------------------------------------------------
signature.asc
Description: This is a digitally signed message part.