On 10/3/23 03:25, Thomas Baumgart wrote:
On Dienstag, 3. Oktober 2023 01:37:32 CEST Jack via KMyMoney-devel wrote:
TL;DR at some point in the past day or two (during my monthly
reconciliation of my Merrill Lynch investment accounts) a stray
</KMYMONEY-FILE> ended up in the middle of the <PRICES> section of my
xml data file. I noticed this because I started getting the
consistency-check message about no price on or before the first use of
a security. That led to noticing that many prices were missing, and I
ended up actually looking in the xml file.
That looks weird. As usual, the first question is: which version of KMM
(stable or master)? The logic to write the XML file differs significantly
between the two.
Master, compiled within the past few days. Although I do on occasion
use 5.1, I don't recall doing so in the past few weeks. It might be
interesting if there were some hint in the file about which version
saved it.
A few other questions come to my mind:
- do you switch between those versions?
- do you have auto saving enabled?
Only autosave on close if file is changed, 10 versions.
I'm in process (after making several copies of all saved backups from
the past two days) of splicing the missing <PRICEPAIRS> sections the
the whole <REPORTS> section from the last good file into the current
one (I have no budgets or onlinejobs, so those sections are empty.)
I'd love to know how this might have happened, as well as any things to
look for that this type of data surgery might introduce.
I have done this in the past when my ONLINEJOBS were gone. Not that I
needed them, but I'd like to keep them around. For me this was caused
by a faulty implementation of the XML writer for onlinejobs. That was
when I re-wrote the XML logic a few months ago.
I don't expect any problems if you proceed your surgery. It could be,
that a couple of prices are missing in case you have added some in the
meantime. Any modification on the reports would also have to be made
again. That's all.
Good. That matches what I thought.
-------
I'm doing my monthly reconciliation of all my Merrill Lynch investment
accounts, and I started getting the "no price for first use date" for
one security. The equity exists, and there are security accounts for
it in two different investmant accounts. In both cases, the initial
presence was from an Add Shares transaction (it was a dividend from a
related stock) and was all sold over the past few months. However,
there are currently NO prices for this equity in my file. Searching
through explicit backups and my autosave backups, the prices are in all
files prior to 17:15 this evening, but missing from all files after
that time.
Further looking shows that in the earliest bad file, within the
<PRICES>......</PRICES> section there is clearly a problems:
<PRICEPAIR from="E000520" to="USD">
<PRICE date="2023-09-01" price="5159/50" source="Yahoo Finance"/>
<PRICE date="2023-09-30" price="10287/100" source="Yahoo
Finance"/>
</PRICEPAIR>
</PRICES>
</KMYMONEY-FILE>
<PRICEPAIR from="E000525" to="USD">
<PRICE date="2022-06-22" price="2367/50" source="User"/>
<PRICE date="2022-09-11" price="2597/50" source="Yahoo Finance"/>
....
Where did that </KMYMONEY-FILE> come from? I have not been editing the
file manually (at least not today :-).
It's not only the </KMYMONEY-FILE> but also the </PRICES> entry. And the
<REPORTS> sections is missing completely. Does that file have a REPORTS
section later on or is it missing completely?
What I quoted was not all the way to the end of the file. I'm not at
home right now, but I believe all the files saved after the event
included <REPORTS /> and <ONLINEJOBS /> which I read as KMM aborting the
reading on hitting </KMYMONEY-FILE> and writing a complete file
including empty sections where it had no data. However, that does not
account for the those <PRICEPAIR ...> sections with correct Ennnnn and
date but wrong amount. I need to review the autosave files more
carefully to see what is actually after the </KMYMONEY-FILE>
Stranger, the current file at that point is
<PRICEPAIR from="E000520" to="USD">
<PRICE date="2022-05-24" price="25080049/380000"
source="Transaction"/>
<PRICE date="2022-06-03" price="3653/50" source="Yahoo Finance"/>
... all these PRICEs are correct
<PRICE date="2023-09-30" price="10287/100" source="Yahoo
Finance"/>
</PRICEPAIR>
<PRICEPAIR from="E000525" to="USD">
<PRICE date="2022-06-22" price="89971/1940" source="User"/>
</PRICEPAIR>
<PRICEPAIR from="E000526" to="USD">
<PRICE date="2022-06-24" price="20697/380" source="User"/>
</PRICEPAIR>
So it seems to have kept one pricepair for most but not all the
following equities, and the prices kept are for the first date of the
existing pairs, but the price itself seems totally unrelated.
Of course since everything past that point is gone, I now have no
custom reports either.
Do I just chalk this up to haveing a bad day, or perhaps sunspots, or
is there any hope of figuring out what might have ac tually happened.
<REPORTS/>
One odd thing is that I've got several files with </KMYMONEY-FILE> in
the middle of a <PRICEPAIRS> section, so KMM persisted in writing bad
files. All the accounts and transactions seem fine, it's just the
prices which are messed up (and the reports missing.)
Any thoughts or suggestions?
(The reason I don't want to simply revert to the last known good file
is that I'd have to redo over an hour of importing and reconciliation
work, although I know I may yet need to resort to that.
The reconciliation data is part of the ACCOUNT elements, so it seems
to be in good condition.
It's not so much the actual reconciliation record but my having cleared
all the transaction, after editing some of them, and actually having had
to create a few do to having had to remove them manually from the qfx
file because of wrong security name.
Can you reproduce the problem by using the last good file and repeat what
you have done to get to the first faulty one (in case you remember/can find
out what that was)?
I wish I knew what I had done. I suppose I can try using the last good
file and jsut redoing all the reconciliation work since that point.
Doesn't KMyMoney complain in case you load a faulty file?
I don't recall any complaints, but then when I'm doing that monthly
reconciliation, I might do over a dozen saves during the process - not
only one after finishing each account (separate for investment and
brokerage) but often after clearing all of each different activity
type. I'll try again opening a bad file when I get home.
