Re: [GNC-dev] [GNC] Fwd: Failed - import of QIF files from Quicken

2020-02-19 Thread Derek Atkins
Hi,

Adrien Monteleone  writes:

> I seem to recall a development goal being to eliminate Scheme code
> except for reports.
>
> This might be a good candidate for just re-writing this in C/C++ with
> proper tests and verbose error output and/or logging.
>
> If someone with the skillset is interested of course.

I started a re-write of the QIF importer in C a very very long time
ago.  That (failed) re-write has long-since been removed from the repo.
It turns out that C is not very good at tokenizing and parsing strings.
Funny that.

I am sure that using std::string in C++ would make it a lot easier to
reprogram, but honestly the existing QIF importer is fairly robust and
fairly well tested (via time).

The real downside of the current implementation is that it doesn't have
a test suite where you can throw test-case QIF files at it and ensure
you get a reasonable answer.

Having said that, whether or not you can actually throw an error with
the line number will depend deeply on where it fails.  The way the
process works is that it reads in the file (line by line) and tokenizes
the data.  At this point it would be relatively easy to output the line
number.  Then, after the file is tokenized, it actually goes through and
parses the tokens for QIF correctness; most likely THIS is where the
failure occurs, but at this point we don't have line numbers any more.

So we COULD add a "see line number X" into the qif-file.scm (IIRC), but
I doubt that's the stage of the process where the error happens.

> Regards,
> Adrien

-derek
-- 
   Derek Atkins 617-623-3745
   de...@ihtfp.com www.ihtfp.com
   Computer and Internet Security Consultant
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel


Re: [GNC-dev] [GNC] Fwd: Failed - import of QIF files from Quicken

2020-02-18 Thread Adrien Monteleone
I seem to recall a development goal being to eliminate Scheme code except for 
reports.

This might be a good candidate for just re-writing this in C/C++ with proper 
tests and verbose error output and/or logging.

If someone with the skillset is interested of course.

Regards,
Adrien

> On Feb 18, 2020 w8d49, at 6:35 PM, James Peterson  wrote:
> 
> Well, with no hint of what the problem is, even smaller
> test sets won't necessarily find anything, so it could be 
> a lot of work for no benefit.  For example, I could export
> each account separately (although there are about 80 of them
> so that's a lot of work), and each account could load 
> okay separately, because maybe the problem is an interaction 
> between two different accounts. So then I have to do 80*79
> possible pairs of accounts, and so on.
> 
> My poking around in the 
> source code to try to find the problem suggests that the
> problem is in the Scheme part that seems to be working with
> all the accounts at the same time:
> 
> ;; Build a local account tree to hold converted transactions.
> 
> ;; Sort the account list on the depth of the account path.
> 
> ;; Make all the accounts.
> 
> ;; Run through the markable transactions marking any
>  ;; duplicates.  marked transactions/splits won't get imported.
> 
>   ;; Convert into a GnuCash transaction.
>   ;; rebalance and commit everything
> 
> So this looks like multiple runs thru the data, and even printing
> the line number won't narrow it down -- you have to know what
> step of the process we are doing, and where we are in the data
> at that point.
> 
> As things stand, we don't even know what the immediate problem
> is -- a divide by zero?  Looking for an account that should exist,
> but doesn't?, an account type that it doesn't know about? 
> 
> The code is not written to allow it to even explain what it is
> doing. If we are going to use this code base, someone who actually
> understands, and is comfortable with the code will need to add
> a "verbose input processing" level or something that will create
> a log file explaining what it is going to try to do, then do it,
> then explain what it got and how it interprets that. I've done 
> that with code before, but only C code, where, if need be, there is
> the equivalent of a printf to a log file between every major step,
> or each iteration of a loop, explaining what it thinks it is doing
> and with what input and with what result.  I just don't know how to
> add that to scheme.
> 
> jim

___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel