Re: Scope of GNUCash

[Wm via gnucash-devel]

On 13/02/2018 01:42, Matt Graham wrote:



IIRC the discussion at the time was about whether gnc should or could be  used for 
trading.  the result was "no"


I said that.


A couple of times I have noticed that people have said "That's not what GNUCash is 
for". It begs the question - where is it defined what GNUCash is and isn't for? The 
charter for GNUCash doesn't seem to ever been formalised.


There is a long term plan, we never write it down because people ask us 
about it :)



Is it intended that people should establish a complementary but separate 
project for functionality they want, but is not included in GNUCash scope?


I don't see why not if that is what they want to do.

===

From *my* POV I don't see the point in trying to make gnc something it 
isn't or can't be.


Matt, the seniors definitely have direction, they have a plan, they are 
going somewhere.  I'm hoping they get their work done before I get so 
old I can't contribute :)


--
Wm

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: price.date, transaction.post_date and neutral time

[Wm via gnucash-devel]

On 12/02/2018 18:57, Alen Siljak wrote:




Sent: Monday, February 12, 2018 at 7:34 PM
From: "Wm via gnucash-devel" 
To: gnucash-de...@lists.gnucash.org
Subject: Re: price.date, transaction.post_date and neutral time

On 12/02/2018 14:27, Alen Siljak wrote:

 I am currently separating the Asset Allocation logic into its own
 Python package, for direct use (cli) and potential library reuse
 between GnuCash Portfolio and an Android app (perhaps an add-on for
 MoneyManagerEx or who knows where it leads).


MoneyManagerEx is single entry and will never get approval in the double
entry community.  I like they way they do reports, but their
understanding of accounting is broken.


True but, as a maintainer of MMEX for Android app, I have already implemented 
Asset Allocation module there. The mobile app served me well for years by 
providing transactions in .qif files, which GnuCash also handles well. So, it 
remains my mobile app of choice for the moment, in combination with GnuCash.



That is interesting.  There has been quite a lot of enthusiasm for 
mobile / gnc interaction.  Have you written about this before and I 
missed it ?  If MMEX for Android is working well with gnc I think you 
should make a wiki entry and ignore my view on MMEX as an accounting app.


Aside: I am completely confused how MMEX for Android could be used for 
Asset Allocation, MMEX doesn't understand Assets and Liabilities and 
Equity!  Looking at the forums, the dev's of MMEX haven't realised that 
prices aren't available from Y! any more, etc.  Crazy.  If the MMEX for 
Android app is moving beyond MMEX itself and may be working better with 
gnc then please let us know,



What I would like to do, is extract the AA logic from the app, implement in 
some platform-neutral way (python?) and use on desktop and mobile. Since it is 
not complex, there is no reason for GnuCash not to implement the same concept. 
After all, it simply takes user-set allocation and compares to the quantity of 
the various commodities owned, calculating their current value by using their 
current price. Hence the need to have Allocation and Price elements.


Asset Allocation is indeed simple, the problem is that no two people 
agree on how to do it :)


Further, most AA models are USA biased, I don't think gnc needs any more 
USA bias and any model that includes a stock allocation along the lines 
of "domestic / foreign" is rubbish if you live outside of the USA so 
let's not go there! [1]


The ledger-cli AA approach works for me because I say what my AA is and 
the sums just work. I can copy and paste the output into a spreadsheet 
and see a pie chart, I can make my AA as detailed or top down or bottom 
up as I like.  It is flexible not prescriptive.  As soon as you start 
coding any of that into an app like gnc you lose the flexibility and 
start telling people "do AA like this or that".  In essence you are 
saying "I'm right, you are wrong".  I disagree with that.


The problem is that AA is like Budgeting, we agree it should be done but 
not how or why or what the point is.  For that reason I ask that the gnc 
db is more accessible and that AA and Budgeting are ripped out of the 
main application.


Am I making sense yet?

I am arguing *strongly* that any personal view AA or Budget should 
*never* be included in the main gnc app.



We spoke about this before.  gnc is a bad platform for trading, end of
discussion.


I was not in any way referring to trading but to reusing the price database 
file/schema. But it seems that it would be too much trouble for such a small 
component so nevermind.


gnc is a bad place to start if you want a comprehensive price db

gnc uses the price db for valuing stuff.  it is *not* a good way of 
recording historic prices of commodities in general.  the purpose of the 
gnc price db is for *you* to know about *your* stuff rather than finding 
out about stuff you have never owned.


if you really want to know and keep the prices of everything (which is a 
dumb thing to desire) then get a broker account <-- they aren't cheap.


for a sanity check, I am certain the good people that maintain the code 
that allows us to get prices do *not* try to maintain a complete db of 
prices themselves.  there is, quite simply, too much information.



 I'm wondering what the opinion here in the project would be about such
 an idea in terms of reuse and/or tools for populating and reading the
 database.


My advice is don't go there.


I still need to read the latest available prices from somewhere. I could use 
GnuCash book, of course, but I hate to pollute it with a bunch of prices that 
don't have real use inside GnuCash anyway. Actually, I'd prefer not to download 
any prices into my GC book apart from the ones that are generated automatically 
on conversion. And even those I'd prefer to clean up from time to time.
I believe I'll do all the price-sensitive calculations outside GC by 

Re: price.date, transaction.post_date and neutral time

[Wm via gnucash-devel]

On 12/02/2018 21:00, Sébastien de Menten wrote:

When I enter a new price for a given day for a security on the NASDAQ via
the price editor, it is stored in the date column the UTC time for that day
at 00:00:00 local time (CET=Europe, not EST=New-York). Which is weird
because across timezone, the day of the price will be interpreted
differently.


Are you entering the prices by hand ?


But John R says "that's an absolute time anchored in the market's time
zone, not the user's. " which leaves me puzzled as for the example above on
the NASDAQ it uses European time (i.e.my local time) not NASDAQ time. But
maybe when using the Perl finance quote program, there is a more complete
time information (incl the correct market timezone).


If you are entering the prices yourself then it seems sensible to me the 
time is when you made the entry rather than the market's time.


Also, unless you have a special arrangement with a broker, most people 
don't get a market price, it will be a bit under or over so the market 
price is just for reference once you include tx costs, etc.



Anyway, my question was to understand the reasons of the encoding of a
day/date as an instant/datetime, reasons that are still a bit obscure
(except legacy issue)


For most purposes it shouldn't matter.  The prices db is independent of 
the actual transactions, it is used for working out the value of stuff 
and, as I am sure you are aware, the value of commodities changes every 
minute and second ... gnc is *never* going to keep up with that sort of 
price change.


In retrospect it may be thought that the design mistake was including a 
time in the price db at all :)


--
Wm

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Scope of GNUCash

[Matt Graham]

>IIRC the discussion at the time was about whether gnc should or could be  used 
>for trading.  the result was "no"

A couple of times I have noticed that people have said "That's not what GNUCash 
is for". It begs the question - where is it defined what GNUCash is and isn't 
for? The charter for GNUCash doesn't seem to ever been formalised.

Is it intended that people should establish a complementary but separate 
project for functionality they want, but is not included in GNUCash scope?

Thanks and regards,
Matt
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: price.date, transaction.post_date and neutral time

[Sébastien de Menten]

When I enter a new price for a given day for a security on the NASDAQ via
the price editor, it is stored in the date column the UTC time for that day
at 00:00:00 local time (CET=Europe, not EST=New-York). Which is weird
because across timezone, the day of the price will be interpreted
differently.

But John R says "that's an absolute time anchored in the market's time
zone, not the user's. " which leaves me puzzled as for the example above on
the NASDAQ it uses European time (i.e.my local time) not NASDAQ time. But
maybe when using the Perl finance quote program, there is a more complete
time information (incl the correct market timezone).


Anyway, my question was to understand the reasons of the encoding of a
day/date as an instant/datetime, reasons that are still a bit obscure
(except legacy issue)


On Feb 12, 2018 15:18, "Wm via gnucash-devel" 
wrote:

On 11/02/2018 12:33, Sébastien de Menten wrote:

> When exporting data from SQL backends, I see some inconsistencies in the
> handling of some date & datetime columns.
>
> In the prices table, when adding price via the price editor, I see in the
> date column a datetime with the UTC of the /MM/DD 00:00:00 of my local
> timezone (CET).
> For instance, for a price on 11/02/2018, I see  2018021023, which is
> the UTC value for 11/02/2018 00:00:00+01:00.
> What is the reason of having the prices.date as a datetime type (vs a
> simple date type) ?
> Shouldn't it also be stored as  20180211105900, i.e. in neutral time as the
> field transaction.post_date ?
>
> In the transactions table, the post_date is handled as a date in gnucash
> but stored also in a datetime type with the neutral time (10:59:00).
> So for a transaction on 11/02/2018, I see 20180211105900.
> What is the reason of having the transactions.post_date as a datetime type
> (vs a simple date type) ?
>
> If the reason are mostly legacy, are there some plans to change that in 3.0
> ?
>
> IIRC the discussion at the time was about whether gnc should or could be
used for trading.  the result was "no" and it was decided that the price db
should only have one entry per day (some people wanted multiple entries per
day in an attempt to use gnc for trading, that was never going to work as
gnc is a crap model for all the other stuff short term traders need, like
quick graphs, moving averages, etc).

given that, are you unhappy with the way gnc records date / price
combinations ?

-- 
Wm

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: price.date, transaction.post_date and neutral time

[Alen Siljak]

> Sent: Monday, February 12, 2018 at 7:34 PM
> From: "Wm via gnucash-devel" 
> To: gnucash-de...@lists.gnucash.org
> Subject: Re: price.date, transaction.post_date and neutral time
>
> On 12/02/2018 14:27, Alen Siljak wrote:
> > I am currently separating the Asset Allocation logic into its own
> > Python package, for direct use (cli) and potential library reuse
> > between GnuCash Portfolio and an Android app (perhaps an add-on for
> > MoneyManagerEx or who knows where it leads).
> 
> MoneyManagerEx is single entry and will never get approval in the double 
> entry community.  I like they way they do reports, but their 
> understanding of accounting is broken.

True but, as a maintainer of MMEX for Android app, I have already implemented 
Asset Allocation module there. The mobile app served me well for years by 
providing transactions in .qif files, which GnuCash also handles well. So, it 
remains my mobile app of choice for the moment, in combination with GnuCash.

What I would like to do, is extract the AA logic from the app, implement in 
some platform-neutral way (python?) and use on desktop and mobile. Since it is 
not complex, there is no reason for GnuCash not to implement the same concept. 
After all, it simply takes user-set allocation and compares to the quantity of 
the various commodities owned, calculating their current value by using their 
current price. Hence the need to have Allocation and Price elements.

> We spoke about this before.  gnc is a bad platform for trading, end of 
> discussion.

I was not in any way referring to trading but to reusing the price database 
file/schema. But it seems that it would be too much trouble for such a small 
component so nevermind.

> > I'm wondering what the opinion here in the project would be about such
> > an idea in terms of reuse and/or tools for populating and reading the
> > database.
> 
> My advice is don't go there.

I still need to read the latest available prices from somewhere. I could use 
GnuCash book, of course, but I hate to pollute it with a bunch of prices that 
don't have real use inside GnuCash anyway. Actually, I'd prefer not to download 
any prices into my GC book apart from the ones that are generated automatically 
on conversion. And even those I'd prefer to clean up from time to time.
I believe I'll do all the price-sensitive calculations outside GC by reading 
from a separate database (or another source).

> > I will definitely be creating a separate project for this, just hoping
> > someone else is interested.
> 
> There is lots of scope in other projects, I don't like people forcing 
> one to be another

Hm, that gives me a few ideas already. One thing I keep forgetting is to check 
if perhaps someone has a similar project already. Or I could simply try 
fetching and caching the latest prices for all symbols on startup. And to 
support offline calculations, I'd still need to save data to a db. Something to 
be worked-out.

Anyway, it would be nice to see Asset Allocation module in GnuCash. The current 
implementation is just two tables. But I'm not sure if it would satisfy 
everybody right now. 
The only other AA implementation I found is in ledger but I have not tested it 
properly because my exported book doesn't balance in ledger due to mismatch in 
handling capital gains transactions between GC and ledger. :(

Cheers
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: price.date, transaction.post_date and neutral time

[Wm via gnucash-devel]

On 12/02/2018 14:27, Alen Siljak wrote:

This is interesting information.
I am currently separating the Asset Allocation logic into its own
Python package, for direct use (cli) and potential library reuse
between GnuCash Portfolio and an Android app (perhaps an add-on for
MoneyManagerEx or who knows where it leads).


MoneyManagerEx is single entry and will never get approval in the double 
entry community.  I like they way they do reports, but their 
understanding of accounting is broken.



In any case, it seems that Prices seems a likely candidate for
separation, as well, as that can definitely be reused by multiple
projects, contains simple data, and can get huge hence it seems better
to have it separate to the main database.


All of the main text accounting projects can get prices in near real 
time.  They deliberately don't do minute by minute.



Now, this raises an interesting question of whether GnuCash would
benefit from the same concept and perhaps reuse the price db. This
might even be invisible to an average user but someone more advanced
could take advantage. The internal GnuCash logic could remain
more-or-less the same, in getting the latest price for the day only.
I think a single table might be enough for this purpose, pretty much
what's in GnuCash now. I have not yet looked into details for this.


We spoke about this before.  gnc is a bad platform for trading, end of 
discussion.



I'm wondering what the opinion here in the project would be about such
an idea in terms of reuse and/or tools for populating and reading the
database.


My advice is don't go there.


I will definitely be creating a separate project for this, just hoping
someone else is interested.


There is lots of scope in other projects, I don't like people forcing 
one to be another




Sent: Monday, February 12, 2018 at 3:17 PM
From: "Wm via gnucash-devel" 
To: gnucash-de...@lists.gnucash.org
Subject: Re: price.date, transaction.post_date and neutral time
IIRC the discussion at the time was about whether gnc should or could
be
used for trading. the result was "no" and it was decided that the price
db should only have one entry per day (some people wanted multiple
entries per day in an attempt to use gnc for trading, that was never
going to work as gnc is a crap model for all the other stuff short term
traders need, like quick graphs, moving averages, etc).




___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: how exactly to do unit testing in scheme...

[Christopher Lam]

Hi Robert and devel,

Thank you for pointers, this is useful. The examples below illustrate 
how to test multiple items together. But I think the complexity of the 
52(and counting) options (37 binary, the rest are multiples/multichoice) 
means, I think it'll be better to handwrite tests. I'll need to reuse 
existing test frameworks to attempt maximise coverage.


C

On 31/01/18 09:01, Robert Merkel wrote:

Sorry for the tardy response - I went away for the long weekend and was
busy yesterday!

As Phil says, you can test both leaf functions and controller code with
unit tests. With controller code, you may need to write mocks for some or
all of the called code.  I don't know whether there are prewritten mocking
libraries for guile, but given the extreme flexibility of scheme it
shouldn't be difficult to write code to replace one function call with
another where necessary.

As long as xaccTransGetDate and xaccSplitGetParent are adequately tested,
there is no particular reason from a testing perspective to throw in an
intermediate function.

As far as which options to test, if testing all the combinations
exhaustively results in too many tests (and it sounds like it does) try
pairwise combinatoric testing, which works as follows:

Say you've got options {o_1, o_2, o_3, ... o_n}, each of which can take a
limited set of values {o_i^1, o_i_2, o_i^k} (for instance, if the option
can either be "on" or "off" it has two values).

For any pair of options o_x and o_y, for all the possible combinations of
option values there should be at least one test that has that combination.

To give a very simple case, for three options {a, b, c}  each of which can
take two values {off, on}, you could have the following tests:

  Option   a b c
Test
1 off  off   off
2 off   on  off
3 on   off   on
4 onon  off
5 off   offon

For the pair (a,b) tests 1 through 4 give all the possible combinations,
for the pair (a, c) tests {1, 3, 4, 5} give all the possible combinations,
and for the pair (b, c) tests {1, 2, 3, 4} give all the possible
combinations.

5 is the minimum number of tests that can meet the pairwise combinatoric
criterion, as compared to 8 if you tried all the possible combinations.
However, it becomes an even bigger win if you're trying lots of options:
you only need 9 tests for 8 options with two values, whereas you would need
256 tests to try every possible option combination!

To generate optimal pairwise test sets, you can use "jenny" (this is a
really useful but unmaintained tool, I really should take over the
maintenance):

http://burtleburtle.net/bob/math/jenny.html

Hope this helps, and feel free to ask more questions.  I will try to
respond more promptly!


On Thu, Jan 25, 2018 at 3:03 AM, Christopher Lam 
wrote:


Dear Devel



To rgmerk: Welcome back, and it was a nice to meet irl!



While simplifying transaction.scm and thinking of unit testing, I now have
a conundrum worthy of an expert view.



The reports require 2 main functions – the options generator and the
renderer; the options generator generates a options.scm controller object,
and the renderer takes options and outputs html.



I understand unit testing to handle testing of ‘leaf’ functions e.g.
(split->date), rather than the controller code (e.g. renderer takes options
and outputs html) – but to me this is rather silly because split->date only
tests xaccTransGetDate and xaccSplitGetParent, whereas the controller tests
actual functionality.



With regards to unit testing I can see several issues



1. The refactored report has inlined most single-use functions into
lambda expressions – I figured that directly stating (xaccTransGetDate
(xaccSplitGetParent split)) is much more descriptive to a programmer than
to create a testable leaf function (split->date split). I can see the
benefits of both – leave as lambda expressions which will can be
understandable by anyone who is familiar with the API, or break them out
into 100s of single use functions which can be tested, but introduces a
whole layer of cognitive load to anyone hacking code – (what does
split->date actually do? Where is its definition). Also, breaking the
lambda functions into testable functions means the implementation is frozen
and the next hacker will have lesser scope to rework/optimise the report.



1. The refactored report is now flexible enough to accommodate derived
reports with a different multicolumn data function – eg
income-gst-statement.scm has been reworked into a transaction.scm
derivative which passes its own calculated-cells to report on GST sales and
purchases. This is not yet committed.



1. I think the most useful testing approach for a complex
transaction.scm will be to test functions of various combinations of
options values, and test the resulting html for 

Re: price.date, transaction.post_date and neutral time

[Alen Siljak]

   This is interesting information.
   I am currently separating the Asset Allocation logic into its own
   Python package, for direct use (cli) and potential library reuse
   between GnuCash Portfolio and an Android app (perhaps an add-on for
   MoneyManagerEx or who knows where it leads).
   In any case, it seems that Prices seems a likely candidate for
   separation, as well, as that can definitely be reused by multiple
   projects, contains simple data, and can get huge hence it seems better
   to have it separate to the main database.

   Now, this raises an interesting question of whether GnuCash would
   benefit from the same concept and perhaps reuse the price db. This
   might even be invisible to an average user but someone more advanced
   could take advantage. The internal GnuCash logic could remain
   more-or-less the same, in getting the latest price for the day only.
   I think a single table might be enough for this purpose, pretty much
   what's in GnuCash now. I have not yet looked into details for this.

   I'm wondering what the opinion here in the project would be about such
   an idea in terms of reuse and/or tools for populating and reading the
   database.
   I will definitely be creating a separate project for this, just hoping
   someone else is interested.

   Sent: Monday, February 12, 2018 at 3:17 PM
   From: "Wm via gnucash-devel" 
   To: gnucash-de...@lists.gnucash.org
   Subject: Re: price.date, transaction.post_date and neutral time
   IIRC the discussion at the time was about whether gnc should or could
   be
   used for trading. the result was "no" and it was decided that the price
   db should only have one entry per day (some people wanted multiple
   entries per day in an attempt to use gnc for trading, that was never
   going to work as gnc is a crap model for all the other stuff short term
   traders need, like quick graphs, moving averages, etc).
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: price.date, transaction.post_date and neutral time

[Wm via gnucash-devel]

On 11/02/2018 12:33, Sébastien de Menten wrote:

When exporting data from SQL backends, I see some inconsistencies in the
handling of some date & datetime columns.

In the prices table, when adding price via the price editor, I see in the
date column a datetime with the UTC of the /MM/DD 00:00:00 of my local
timezone (CET).
For instance, for a price on 11/02/2018, I see  2018021023, which is
the UTC value for 11/02/2018 00:00:00+01:00.
What is the reason of having the prices.date as a datetime type (vs a
simple date type) ?
Shouldn't it also be stored as  20180211105900, i.e. in neutral time as the
field transaction.post_date ?

In the transactions table, the post_date is handled as a date in gnucash
but stored also in a datetime type with the neutral time (10:59:00).
So for a transaction on 11/02/2018, I see 20180211105900.
What is the reason of having the transactions.post_date as a datetime type
(vs a simple date type) ?

If the reason are mostly legacy, are there some plans to change that in 3.0
?

IIRC the discussion at the time was about whether gnc should or could be 
used for trading.  the result was "no" and it was decided that the price 
db should only have one entry per day (some people wanted multiple 
entries per day in an attempt to use gnc for trading, that was never 
going to work as gnc is a crap model for all the other stuff short term 
traders need, like quick graphs, moving averages, etc).


given that, are you unhappy with the way gnc records date / price 
combinations ?


--
Wm

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel