Re: Revisiting problems with price updates
On Dienstag, 22. Februar 2022 01:47:21 CET Jack via KMyMoney-devel wrote:
> The latest is https://bugs.kde.org/show_bug.cgi?id=450426. I think
> this is just another example of an issue that has been mentioned before
> on the mailing list, forum, and bug reports. The common complaint is
> that the price updater fails - but if you look carefully, it seems to
> be trying to retrieve a pair that doesn't make sense - it is reversed
> from what you would expect - the value of a currency in terms of a
> stock, instead of the price of the stock in the currency.
Indeed, that does not make sense and it is what is causing the failure during
online download. As of now, I have no idea how these entries get created.
In terms of the reported "I was able to delete them but once I ran
the update they appeared again" I think the downloader checks which
combinations are on file and creates them as it goes. I have to investigate
this.
> When i started looking into my own kmy file, I've found things I can't
> explain, that I think are related. Hopefully something here will
> trigger some thought and give a hint where we might look for the
> underlying problem..
>
> As one example, I have an equity E416 for symbol CARR (Carrier
> Global Corp.)
> If I look in the Price Editor for CARR, I see two entries - one is from
> KMyMOney Currency and the other from Yahoo Finance. It is not clear
> why there are two en tries for the same ID and Name.
See below. What are the dates of these entries?
> If I show all
> stored prices, the sources include Transaction, User, Yahoo Finance,
> Finance::Quote yahoo_json, Statement Importer, and KMyMoney Currency.
> All make sense to me except for the last. In addition, if I look in my
> kmy file, I have five entries for
> including one User and four KMyMoney Currency.
Those are the ones causing the trouble (I think). The User one seems to
be entered by the user (either directly as price or by a transaction).
Do you see a relationship between this price entry and a transaction? The
ones showing KMyMoney Currency are a cause of this if they have a later
date. Can you confirm?
> from="E000416" to="USD"> includes all the rest. I had almost expected
> all the KMyMoney Currency entries to be in one group. Can anyone
> explain why a stock price would ever get looked up through KMyMoney
> Currency? If I sort those price entries by date, all the KMyMoney
> Currency entries are in the latter half of the list. In a few cases
> where there is also another entry on the same date, they differ by a
> few cents - in only one case are they identical.
>
> Then, in the Price Editor, the Stock name for CARR is "Carrier Global
> Corp,Carrier Global Corp,Carrier Global Corp,Carrier Global
> Corp,Carrier Global Corp". How did the name get repeated five times?
Good question. I don't have an answer (yet).
> It only appears once in the Investment Views (once under Securities,
> and once under Equities tab for each account which held that stock (all
> not closed except one.) If I scroll through the complete Price Editor,
> there are several other entries with the Stock Name repeated multiple
> times, but I see no pattern.
>
> Then, if I look in the Price Updater, I see an entry for ID: CARR with
> Internal ID: E000416 and Source: Yahoo Finance. However, if I scroll
> down, I see ID: USD > E000416 with internal ID: "USE E000416" a price
> of "CARR0.0218 and Source KMyMOney Currency.
The internal ID should probably be "USD E000416" and is a typo on your end.
The price of "CARR0.0218" is certainly weird.
Regarding the names Yahoo Finance and KMyMoney Currency I can point to
https://invent.kde.org/office/kmymoney/-/commit/8559f9ec which we changed
when Yahoo was not available a few years ago. Still, I don't understand
why the currency exchange rate source is used for an investment price
update.
> Searching my kmy file for E000416 shows only the Security definition,
> the five Accounts holding that stock, and the two PRICEPAIR entries
> already mentioned. In particular, I don't see what generates that
> USD > E000416 entry in the price updater. My currency list includes
> only USD and EUR.
>
> Any thoughts or suggestions for further troubleshooting?
I scanned the term "KMyMoney Currency" and only found it in a few spots:
in WebPriceQuote::launchCSV it is only used as reference:
// for historical exchange rates we switch to Stooq
if (sourcename == QLatin1String("KMyMoney Currency"))
sourcename = QLatin1String("Stooq Currency");
and for the generation of the source information to select from:
result["KMyMoney Currency"] = WebPriceQuoteSource("KMyMoney Currency",
"https://fx-rate.net/%1/%2";,
QString(),
"https://fx-rate.net/([^/]+/[^/]+)",
the third location is the one introduced by above mentioned
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 --- Comment #14 from lp.allar...@gmail.com --- Is there a way to run KMM in debug mode? I tried running it from CLI to see if anything useful comes up, but other than a lot of verbose, nothing relevant ... I'm not of much use in term of development but I can try a lot of twists and bends to try to see what can come up... -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 Jack changed: What|Removed |Added CC|ostroffjh@users.sourceforge | |.net| --- Comment #13 from Jack --- Thanks, that sample kmy file is what I wanted. I also found some more evidence of strange things in my own kmy file. Yes, not updating a price if there is no source is fine - but inventing a price-pair that doesn't work and isn't what you entered is not ok. We need to figure out why that happens, and why we can't remove some of those bogus price entries. I suspect this may take a while. -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 --- Comment #12 from lp.allar...@gmail.com --- Created attachment 147028 --> https://bugs.kde.org/attachment.cgi?id=147028&action=edit kmy test file -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 --- Comment #11 from lp.allar...@gmail.com --- The last investment that occurred on 2022-12-13 for "CIG2323" was a "Add shares" transaction (year-end dividend reinvestment). I looked in the Price Editor for this investment, and indeed, the last price update is from 2022-01-21 (which was the previously reported update for the zombie pair commodity > CAD) and the other date before that is from 2021-09-01 which is now the reported update date for that zombie "pair" I am not sure to understand: do you want me to create a brand new Kmy file and create the same investment and add shares from one of those 3 investments that fail to update and try to update prices? If this is the case I already did, see attached. Just to clarify, these 3 investment (mutual funds) that do not update are not caused by a bug in KMM or my data file but because I dont have an online source at all to update them. Unless I misunderstood you, this is 100% normal I believe. -- You are receiving this mail because: You are the assignee for the bug.
Revisiting problems with price updates
The latest is https://bugs.kde.org/show_bug.cgi?id=450426. I think this is just another example of an issue that has been mentioned before on the mailing list, forum, and bug reports. The common complaint is that the price updater fails - but if you look carefully, it seems to be trying to retrieve a pair that doesn't make sense - it is reversed from what you would expect - the value of a currency in terms of a stock, instead of the price of the stock in the currency. When i started looking into my own kmy file, I've found things I can't explain, that I think are related. Hopefully something here will trigger some thought and give a hint where we might look for the underlying problem.. As one example, I have an equity E416 for symbol CARR (Carrier Global Corp.) If I look in the Price Editor for CARR, I see two entries - one is from KMyMOney Currency and the other from Yahoo Finance. It is not clear why there are two en tries for the same ID and Name. If I show all stored prices, the sources include Transaction, User, Yahoo Finance, Finance::Quote yahoo_json, Statement Importer, and KMyMoney Currency. All make sense to me except for the last. In addition, if I look in my kmy file, I have five entries for including one User and four KMyMoney Currency. from="E000416" to="USD"> includes all the rest. I had almost expected all the KMyMoney Currency entries to be in one group. Can anyone explain why a stock price would ever get looked up through KMyMoney Currency? If I sort those price entries by date, all the KMyMoney Currency entries are in the latter half of the list. In a few cases where there is also another entry on the same date, they differ by a few cents - in only one case are they identical. Then, in the Price Editor, the Stock name for CARR is "Carrier Global Corp,Carrier Global Corp,Carrier Global Corp,Carrier Global Corp,Carrier Global Corp". How did the name get repeated five times? It only appears once in the Investment Views (once under Securities, and once under Equities tab for each account which held that stock (all not closed except one.) If I scroll through the complete Price Editor, there are several other entries with the Stock Name repeated multiple times, but I see no pattern. Then, if I look in the Price Updater, I see an entry for ID: CARR with Internal ID: E000416 and Source: Yahoo Finance. However, if I scroll down, I see ID: USD > E000416 with internal ID: "USE E000416" a price of "CARR0.0218 and Source KMyMOney Currency. Searching my kmy file for E000416 shows only the Security definition, the five Accounts holding that stock, and the two PRICEPAIR entries already mentioned. In particular, I don't see what generates that USD > E000416 entry in the price updater. My currency list includes only USD and EUR. Any thoughts or suggestions for further troubleshooting? Jack
[kmymoney] [Bug 450653] Credit card balances are wrong in net worth report
https://bugs.kde.org/show_bug.cgi?id=450653 --- Comment #3 from lp.allar...@gmail.com --- Created attachment 147026 --> https://bugs.kde.org/attachment.cgi?id=147026&action=edit Ledger -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450653] Credit card balances are wrong in net worth report
https://bugs.kde.org/show_bug.cgi?id=450653 --- Comment #2 from lp.allar...@gmail.com --- Hello Thomas, No I have not reversd the transactions on the card. Each time I use the card to buy something, I create a CHARGE on the account, and when I pay I create a PAYMENT to the card. Charges increase the balance of the card (which is shown in RED in the ledger), and payments decrease the balance. Works as intended AFAIK... See screenshot. All transactions are in CAD on this credit card (since opening the account), and I have only one card. (In reply to Thomas Baumgart from comment #1) > I suppose you have reversed some/most transactions in the credit card > account. The payment column should contain deposits on the credit card > account. The charge column should contain the purchases you made. Could that > be? The balance is calculated by adding the amount of all (not only the > current year but all of them) splits referencing an account, so the Dec 31st > problem cannot occur. A credit card with a negative balance means an > available asset. > > Are you dealing with multiple currencies in the affected accounts or are > they all denominated in CAD? -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 Jack changed: What|Removed |Added CC||ostroffjh@users.sourceforge ||.net --- Comment #10 from Jack --- I think those date differences make sense. On the last one, what type of transaction was it on 12/13? That transaction would not have included a price update unless it was a buy or sell. In the price tool, you can type the symbol in the search box, and then click the box to show all prices. That will show you where each of those price entries came from. -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 --- Comment #9 from Jack --- Is it possible for you to create a new kmy file, with one investment account which owns a few shares of one of those funds? If the price update there fails in the same way, you can post the file, and maybe we can find what causes the problem. It doesn't make sense for me for KMyMoney Currency to be the source of a price of a stock (I just found some examples in my own data file) or a mutual fund. -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 --- Comment #8 from lp.allar...@gmail.com --- Other observations: The date of the last price update changes randomly (probably more a symptom than a root-cause...). Before I delete these prices they all had a last update date of 2022-01-29, now after deleting them and running the price updater, they have dates matching the last transaction dates of each commodities. -> CIG7710's last REAL transaction date is 2021-12-17 and the last price update listed by the Price editor is 2021-12-17 -> CIG2321's last REAL transaction date is 2021-12-20 and the last price update listed by the Price editor is 2021-12-20 -> CIG2323's last REAL transaction date is 2021-12-13 and the last price update listed by the Price editor is 2021-09-01 (why different???) (In reply to lp.allard.1 from comment #7) > Jack, > > I've tried deleting them in tools > prices, and it worked, but when I ran > the online price updater , KMM re-added them! > > -> Out of many mutual funds, I have 3 of them that I dont have online > sources to update prices for (yahoo used to list them but stopped a while > ago and Globe and Mail changed their website so the REGEXes are NO longer > working) > > -> These 3 funds (CIG2321, CIG2323 & CIG7710) are those causing this > currency issue > > See screenshots. -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 --- Comment #7 from lp.allar...@gmail.com --- Jack, I've tried deleting them in tools > prices, and it worked, but when I ran the online price updater , KMM re-added them! -> Out of many mutual funds, I have 3 of them that I dont have online sources to update prices for (yahoo used to list them but stopped a while ago and Globe and Mail changed their website so the REGEXes are NO longer working) -> These 3 funds (CIG2321, CIG2323 & CIG7710) are those causing this currency issue See screenshots. -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 --- Comment #6 from lp.allar...@gmail.com --- Created attachment 147024 --> https://bugs.kde.org/attachment.cgi?id=147024&action=edit Update Stocks and Currency Prices -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 --- Comment #5 from lp.allar...@gmail.com --- Created attachment 147023 --> https://bugs.kde.org/attachment.cgi?id=147023&action=edit Price Editor -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450152] "Dividend income" box appears in wrong places
https://bugs.kde.org/show_bug.cgi?id=450152 Jack changed: What|Removed |Added Status|NEEDSINFO |REPORTED Resolution|WAITINGFORINFO |--- --- Comment #3 from Jack --- Just updating status. -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450426] OFX import fails: Unable to find the DTD named opensp.dcl
https://bugs.kde.org/show_bug.cgi?id=450426 Jack changed: What|Removed |Added CC||ostroffjh@users.sourceforge ||.net Resolution|--- |WAITINGFORINFO Status|REPORTED|NEEDSINFO --- Comment #8 from Jack --- First, as a hint, when uploading multiple attachments, please provide a description for each of them, as otherwise it is hard to be sure which si which. My current understanding is that KMyMoney 5.0.6 from kmymoney5-mingw64-5.0.6-8.1-setup.exe works, and has worked for a long time. You tried to upgrade to KMyMoney 5.1.2 from kmymoney-5.1-1305-windows-msvc2019_64-cl.exe (from last week) but it fails. I see the paths of the dtd folder are different for the two instances - are they both installed at the same time? If they are truly in different locations it might work, but I'm not sure. Also, your screenshot of Help/About shows version 5.1.2-0af86e098, which is from last September based on finding that commit in git. Might there be any confusion about which versions you are actually using? Then, 5.1.2 (you say "another version" but not exactly which version) in a Windows VM does work. Have you tried that exact same version in your main system? If it works in the VM but not on the host system, I would first try again, being sure you uninstall all other versions first. -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450593] Unable to update price for CAD with fx-rate.net
https://bugs.kde.org/show_bug.cgi?id=450593 --- Comment #4 from Jack --- Yes, E07 is the underlying equity for some mutual fund. Internally, it is stored the same was as currencies, but KMM does know the difference. Normally, you would have a price pair E07 -> CAD, or the number of CAD (i.e., price) per share of the fund. For some reason (and we have seen this before, but still do not have an explanation) the reverse price pair gets created. If you go to Tools/Prices, you can sometimes delete such entries, but more likely not. We do not yet understand why. My suspicion (and it is no more than that yet) is that somehow, in entering a transaction for that mutual fund, the price got entered in some way that KMM thought it needed the number of E07 per CAD, instead of the other way around. However, as far as I can tell, the fact that it fails to retrieve that reverse price pair doesn't really matter, as long as the correct pair is retrieved successfully. -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 386186] Edit account for a loan account only allows editing of loan/schedule not online settings
https://bugs.kde.org/show_bug.cgi?id=386186 Jack changed: What|Removed |Added CC||dj_...@yahoo.com --- Comment #4 from Jack --- *** Bug 448933 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 448933] Cannot adjust names of accounts
https://bugs.kde.org/show_bug.cgi?id=448933 Jack changed: What|Removed |Added Resolution|--- |DUPLICATE CC||ostroffjh@users.sourceforge ||.net Status|CONFIRMED |RESOLVED --- Comment #3 from Jack --- Since you are set for now, I'm closing this as a duplicate of the older bug, and I've expanded that one to reflect the initial problem brought up here. *** This bug has been marked as a duplicate of bug 386186 *** -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 386186] Edit account for a loan account only allows editing of loan/schedule not online settings
https://bugs.kde.org/show_bug.cgi?id=386186 Jack changed: What|Removed |Added Version|4.8.1 |5.1.2 --- Comment #3 from Jack --- I'm updating the version, as it is still a problem. Note that it is not just the online settings which cannot be changed, or at least not easily. For example (see bug closed as duplicate of this one) just to change the name of a liability account created when creating a loan, you have to go through the entire wizard, even though no other loan settings are being changed. In addition, I do not believe it is possible to change the name of a scheduled transaction created as part of setting up a loan. -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450653] Credit card balances are wrong in net worth report
https://bugs.kde.org/show_bug.cgi?id=450653 Thomas Baumgart changed: What|Removed |Added Resolution|--- |WAITINGFORINFO Status|REPORTED|NEEDSINFO --- Comment #1 from Thomas Baumgart --- I suppose you have reversed some/most transactions in the credit card account. The payment column should contain deposits on the credit card account. The charge column should contain the purchases you made. Could that be? The balance is calculated by adding the amount of all (not only the current year but all of them) splits referencing an account, so the Dec 31st problem cannot occur. A credit card with a negative balance means an available asset. Are you dealing with multiple currencies in the affected accounts or are they all denominated in CAD? -- You are receiving this mail because: You are the assignee for the bug.
[kmymoney] [Bug 450653] New: Credit card balances are wrong in net worth report
https://bugs.kde.org/show_bug.cgi?id=450653 Bug ID: 450653 Summary: Credit card balances are wrong in net worth report Product: kmymoney Version: 5.0.8 Platform: Mint (Ubuntu based) OS: Linux Status: REPORTED Severity: major Priority: NOR Component: reports Assignee: kmymoney-devel@kde.org Reporter: lp.allar...@gmail.com Target Milestone: --- Initially posted to the maillist but I realize KDE bugtracker is the right place to document this and get resolution. This is a 2 part bug ticket as I think they are closely inter-related (hence why I created a single bug ticket). *Part 1/2 of this ticket* I have noticed that using a yearly report of net worth, the balance of my credit card under the Liability section is negative (and in red), and therefore gets ***substracted*** from the total of the liability. I do not understand this. All other liabilities are positive (and black) and by doing so, its like if the total of the liability was artificially decreased by the balance of the credit card Should be the opposite. Example: Assets Chequing account: 10,000 Car: 8,000 Investments: 350,000 TOTAL ASSETS: 368,000 Liabilities Mortgage: 250,000 Line of credit: 50,000 Credit card: -5,000 TOTAL LIABILITY: 295,000 (while it should be 305,000) !!! NET WORTH: 73,000 (while it should be 63,000) !!! Is this normal? Thomas Beaumgart said in bug https://bugs.kde.org/show_bug.cgi?id=369608: "Liability accounts are shown with reverse sign to better match the statements sent out by financial institutions. You could also take a look at reports which will show your funds on the credit card account as negative liabilities (which are in fact assets)." However, in my case the other liabilities (mortgage, line of credit, etc) are all positive numbers and are counting towards the total of liabilities in the intended way (see my example above). *Part 2/2 of this ticket* Additionally, the balances displayed seem way too high (they are all over 5000$ CAD) which is impossible, the credit card has a limit of 3000$ CAD. How is this amount calculated? Is it calculated by summing up all transactions during the year, or is it the balance of the card on Dec 31 of each year? If balance on Dec31 then there is a major issue... I thought the amount presented in the report was the sum of all payments done on the card during the year, but I confirm this is NOT the case as for example, in 2021, I paid 15,022$ on the card. The report says 4,763$. LEDGER transactions are all OK. This ticket only relates to reports and not the ledgers or other functions of KMM (at least I could not see this bug impact other areas of KMM). KMM 5.0.8 KDE Frameworks 5.68.0 Qt 5.12.8 (built against 5.12.8) The xcb windowing system MySQL database backend -- You are receiving this mail because: You are the assignee for the bug.
