Re: [GNC] Third party OFX/CVS providers

[flywire]

That never used to be the case but it's still pretty useless for a tax
return:
Please note that if you wish to download transactions that is more than the
12 month period, you must choose the 'View Statement' option. However,
please be advised that it does not have the facility to transfer them in
the abovementioned formats, it will on a PDF format per month.

On Sun, Aug 7, 2022 at 12:14 PM Gyle McCollam  wrote:

> I have a Citibank credit card and I use the online banking tool to
> download a QFX file, but I went to their website and to the activity (not
> Statements) and selected the activity/statement month.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Tom Browder]

On Sun, Aug 7, 2022 at 09:32 Tom Browder  wrote:

> On Sun, Aug 7, 2022 at 09:09 Christopher Lam 
> wrote:
>
>> You may try this service which is free for personal use. (Not mine)
>>
>> https://bankstatementconverter.com/
>>
>
That does a good job, at least on my test pdf statements from
Hancock-Whitney. It's a good resource to have in my toolbox!

Thanks so much.

-Tom
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Fred Bone]

On 08 August 2022 at 10:30, Adrien Monteleone said:

> Kalpesh,
> 
> Thanks for the links. I have a need for a PDF to CSV solution for other
> software and this looks like it is a good fit for me. (folks may be
> shocked how many large businesses in 2022 can't generate decent business
> documents/data in a proper electronic format for using in various
> software.)

EDI standards have only been around for about forty years. It takes some 
people a long time to catch up.

___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Glenn Fowler]

I couldn't agree more.  I took all of my automations and scripts and
roughly calculated that I am saving at least a week every year in time.
That's a free week of vacation!
Now multiply that every single year and with the scripts and automations
continually being added it has a compound effect.

All from single scripts and automations that may *only* save a minute or
two each time.

On Mon, Aug 8, 2022 at 11:53 AM Adrien Monteleone <
adrien.montele...@lusfiber.net> wrote:

> Here's a use case:
>
> Vendor invoices containing 10+ line items that need to be entered into
> both an accounting package and an inventory/POS system.
>
> Multiply that case by a dozen or more a week.
>
> And compound it by said 'PDF' invoice being a *scan*. (which of course
> means really good OCR needs to be in the workflow) Despite computers
> being near ubiquitous in business today, I'd hazard a guess of 93.46% of
> them are clueless as to how to use computers efficiently. It seems as if
> their approach is akin to someone with only a hammer, Duck Tape, & WD-40
> in their toolbox no matter what the job requires.
>
> Now, let's multiply that again: you are the sole person responsible for
> this data entry for 2 or more businesses.
>
> I'd rather have a script take me time to craft so I can one-click the
> data 'conversion' from print to CSV and then spend my time reviewing the
> result rather than a high chance of data entry taking even more of my time.
>
> Regards,
> Adrien
>
> On 8/7/22 8:12 AM, David T. via gnucash-user wrote:
> > I can't imagine having so many transactions that the time it took to
> program the process would in the end save me time in doing the accounting.
> >
> > My method for processing pdf statements is to open the pdf statement in
> one window and enter my transactions in GnuCash in another-- the old
> fashioned way: by keying them in. This is remarkably quick in most cases,
> due to autofill-- and it gives me a sanity check on the data that's getting
> input (does that transaction look right?). It works pretty well for me.
>
>
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Adrien Monteleone]

Here's a use case:

Vendor invoices containing 10+ line items that need to be entered into 
both an accounting package and an inventory/POS system.


Multiply that case by a dozen or more a week.

And compound it by said 'PDF' invoice being a *scan*. (which of course 
means really good OCR needs to be in the workflow) Despite computers 
being near ubiquitous in business today, I'd hazard a guess of 93.46% of 
them are clueless as to how to use computers efficiently. It seems as if 
their approach is akin to someone with only a hammer, Duck Tape, & WD-40 
in their toolbox no matter what the job requires.


Now, let's multiply that again: you are the sole person responsible for 
this data entry for 2 or more businesses.


I'd rather have a script take me time to craft so I can one-click the 
data 'conversion' from print to CSV and then spend my time reviewing the 
result rather than a high chance of data entry taking even more of my time.


Regards,
Adrien

On 8/7/22 8:12 AM, David T. via gnucash-user wrote:

I can't imagine having so many transactions that the time it took to program 
the process would in the end save me time in doing the accounting.

My method for processing pdf statements is to open the pdf statement in one 
window and enter my transactions in GnuCash in another-- the old fashioned way: 
by keying them in. This is remarkably quick in most cases, due to autofill-- 
and it gives me a sanity check on the data that's getting input (does that 
transaction look right?). It works pretty well for me.



___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Adrien Monteleone]

Kalpesh,

Thanks for the links. I have a need for a PDF to CSV solution for other 
software and this looks like it is a good fit for me. (folks may be 
shocked how many large businesses in 2022 can't generate decent business 
documents/data in a proper electronic format for using in various software.)


Regards,
Adrien

On 8/6/22 8:57 PM, Kalpesh Patel wrote:

Not sure if Python is on the table or not but it looks simple to go from PDF
to CSV format programmatically in Python (I un-learned Perl in 2010's and
learned Python in place). This will do away with needing very fine external
programs likes of Ghostscript and/or or iText7.

Take a look at this link https://www.pythonpool.com/python-pdf-parser/ and
https://medium.com/analytics-vidhya/python-packages-for-pdf-data-extraction-
d14ec30f0ad0 if you want to be brave and venture into Python realm.



___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[flywire]

Oh, security isn't a thing anymore? Maybe supply your DOB and mother's
maiden name too so they don't have to get it from social media.

On Sun, Aug 7, 2022 at 09:09 Christopher Lam https://lists.gnucash.org/mailman/listinfo/gnucash-user>>

wrote:

>* You may try this service which is free for personal use. (Not mine)
*>>* https://bankstatementconverter.com/ *
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Phyllis Bruce]

So true about the need to dig, Greg.  At a couple of my banks I must use
the "I want to" option which provides different options depending on what
screen I'm visiting.  When we are getting QFX or whatever file to IMPORT
into GC, we are not looking for a statement, we are looking for a
download.  The statement makes a great backup but is always a pdf in my
experience.  At other banks, there is a down arrow at the right side of the
transactions screen which will allow you to download by choosing among
several options. including specific to and from dates.

On Sun, Aug 7, 2022 at 5:01 PM Greg Feneis  wrote:

> Same.
>
> Making FI data available to the user to download without Quicken to hold
> the user's hand seems like at best an afterthought in the design of most FI
> websites.  I find I really have to dig for it at many sites.
>
> Kind regards,
>
> Greg Feneis
>
>
>
>
> On Sat, Aug 6, 2022 at 7:16 PM Gyle McCollam  wrote:
>
> > I have a Citibank credit card and I use the online banking tool to
> > download a QFX file, but I went to their website and to the activity (not
> > Statements) and selected the activity/statement month.  Then I selected
> the
> > download icon and it gave me the following:
> > [cid:dc3e8429-3ab4-4639-9150-ff3c3a03ffd5]
> > As you can see there are many formats to choose from.  I don't know what
> > type of statement you are downloading, but I would think if they give you
> > these options on a credit card, they may give you these options on
> another
> > type of account.  However, it wouldn't be the first time the left hand
> > didn't know what the right hand was doing.
> >
> > I've added an attachment in case the picture doesn't get through.
> >
> >
> > Thank You,
> >
> > Gyle McCollam
> >
> > Gyle McCollam
> >
> > gmccol...@live.com<mailto:gmccol...@gyleshomes.com>   email
> >
> > 
> > From: gnucash-user 
> on
> > behalf of flywire 
> > Sent: Saturday, August 6, 2022 9:36 PM
> > To: Gnucash Users 
> > Subject: [GNC] Third party OFX/CVS providers
> >
> > My Citibank statements are only available as pdf and I use
> > https://github.com/flywire/pdf_statement_reader . It is based on tabula
> > and
> > forks another project to make it functional. More config info at
> > https://github.com/marlanperumal/pdf_statement_reader/issues/34 .
> > ___
> > gnucash-user mailing list
> > gnucash-user@gnucash.org
> > To update your subscription preferences or to unsubscribe:
> > https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > -
> > Please remember to CC this list on all your replies.
> > You can do this by using Reply-To-List or Reply-All.
> > ___
> > gnucash-user mailing list
> > gnucash-user@gnucash.org
> > To update your subscription preferences or to unsubscribe:
> > https://lists.gnucash.org/mailman/listinfo/gnucash-user
> > -
> > Please remember to CC this list on all your replies.
> > You can do this by using Reply-To-List or Reply-All.
> >
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Greg Feneis]

Same.

Making FI data available to the user to download without Quicken to hold
the user's hand seems like at best an afterthought in the design of most FI
websites.  I find I really have to dig for it at many sites.

Kind regards,

Greg Feneis




On Sat, Aug 6, 2022 at 7:16 PM Gyle McCollam  wrote:

> I have a Citibank credit card and I use the online banking tool to
> download a QFX file, but I went to their website and to the activity (not
> Statements) and selected the activity/statement month.  Then I selected the
> download icon and it gave me the following:
> [cid:dc3e8429-3ab4-4639-9150-ff3c3a03ffd5]
> As you can see there are many formats to choose from.  I don't know what
> type of statement you are downloading, but I would think if they give you
> these options on a credit card, they may give you these options on another
> type of account.  However, it wouldn't be the first time the left hand
> didn't know what the right hand was doing.
>
> I've added an attachment in case the picture doesn't get through.
>
>
> Thank You,
>
> Gyle McCollam
>
> Gyle McCollam
>
> gmccol...@live.com<mailto:gmccol...@gyleshomes.com>   email
>
> 
> From: gnucash-user  on
> behalf of flywire 
> Sent: Saturday, August 6, 2022 9:36 PM
> To: Gnucash Users 
> Subject: [GNC] Third party OFX/CVS providers
>
> My Citibank statements are only available as pdf and I use
> https://github.com/flywire/pdf_statement_reader . It is based on tabula
> and
> forks another project to make it functional. More config info at
> https://github.com/marlanperumal/pdf_statement_reader/issues/34 .
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Tom Browder]

On Sun, Aug 7, 2022 at 09:09 Christopher Lam 
wrote:

> You may try this service which is free for personal use. (Not mine)
>
> https://bankstatementconverter.com/
>

Thanks, Christopher, I’ll give it a shot.

-Tom
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Christopher Lam]

You may try this service which is free for personal use. (Not mine)

https://bankstatementconverter.com/

On Thu, 4 Aug 2022, 6:40 am Tom Browder,  wrote:

> My current bank (US, Hancock Whitney) unfortunately does not provide
> account data in a digital format other than monthly PDF statements for
> personal accounts. I know third-party programs, such as YNAB (ynab.com),
> can get access my bank while I can't, but I don't want the baggage that
> comes with it.
>
> Does anyone know of a third-party entity I could use just to get access to
> the raw data for my personal accounts?
>
> Thanks.
>
> -Tom
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Glenn Fowler]

On Windows I would recommend Ocular which is a PDF viewer and light editor
that you can find in the Windows Store. Ocular allows copying text.

Ocular is open source, ad free, and make by KDE so you won't have to worry
about them taking features away like mentioned with Adobe.

On Sun, Aug 7, 2022, 7:28 AM Tom Browder  wrote:

> On Sat, Aug 6, 2022 at 21:25 Chris Good  wrote:
> > From: Tom Browder 
> > Sent: Sunday, 7 August 2022 12:37 AM
> > To: Chris Good 
> > Cc: gnucash-user@gnucash.org
> > Subject: Re: [GNC] Third party OFX/CVS providers
> >
> >
> >
> > On Sat, Aug 6, 2022 at 08:51 Chris Good  wrote:
> >
> > ..,
> >
> > Can you copy the text out of the pdf statement?
> >
> >
> >
> > Getting "organized" text programmatically out of PDF is a giant pain. My
> current approach is almost completely programmatic. BTW, Raku is the
> "sister" language of Perl, and it is a "scripting" language.
> >
> >
> >
> > Raku can also use Perl modules so I can use Finance::Quote with it if
> need be. But, until I can get the bank data working, I won't be needing it.
> >
> >
> >
> > Thanks, Chris.
> >
> >
> >
> > -Tom
> >
> >
> >
> > Hi Tom,
> >
> >
> >
> > I was actually suggesting you just highlight the transactions in the pdf
> statement and copy/paste them into a
> >
> > spreadsheet or text file before using a scripting language to reformat
> them but if you can programmatically
> > extract the text that would be better long term.
>
> But I did see Peter West's warning about checking results
> carefully--it's not always the expected result. Although I hope the
> products from the same source are consistent enough to make it
> reasonably fool proof. (See my comments below.)
>
> > I notice with my PDF statement that (free) Adobe Acrobat DC no longer
> allows me to select text (as they want me to pay for a fancy tool - BOO)
> but if you send the pdf to Edge or Chrome you can select just the
> transaction text.
>
> Yes, I now see my "PDF COmplete" on Windows can do that--I never
> thought of doing that until now.
>
> > Thanks for the info re Raku.
>
> You're welcome! I'm always happy for programmers to learn about Raku
>
> > Finance::Quote is just for getting stock prices from websites (usually)
> so not useful for this task.
>
> Right, but good to know for later.
>
> > Are you sure you cannot export transactions from your bank? All mine do
> although sometimes it is hard to find.
>
> Yes, and I agree it's sometimes hard to find. Yesterday I did find one
> bank (the bank I'm leaving) does have two choices of download for the
> credit card: CSV and OFX. That is very helpful for working on a
> transformation algorithm. I did look at the PDF to text transformation
> on one of the new bank's statements (definitely nothing but PDFs) and
> it looks surprisingly usable, so that's exciting.
>
> I was too hasty in my outright condemnation of PDF to text because I
> had worked on a project with a PDF expert to generate PDF as a native
> file and saw how easy it is to get things out of place. I usually
> create beautiful, and accurate, PDFs by writing the PostScript code
> and relying on Ghostscript's ps2pdf converter. Thus going backwards in
> the workflow is a little different for my longtime mindset.
>
> Best regards,
>
> -Tom
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[David T. via gnucash-user]

I can't imagine having so many transactions that the time it took to program 
the process would in the end save me time in doing the accounting. 

My method for processing pdf statements is to open the pdf statement in one 
window and enter my transactions in GnuCash in another-- the old fashioned way: 
by keying them in. This is remarkably quick in most cases, due to autofill-- 
and it gives me a sanity check on the data that's getting input (does that 
transaction look right?). It works pretty well for me. 

David T.



On August 7, 2022 2:28:50 PM GMT+03:00, Tom Browder  
wrote:
>On Sat, Aug 6, 2022 at 21:25 Chris Good  wrote:
>> From: Tom Browder 
>> Sent: Sunday, 7 August 2022 12:37 AM
>> To: Chris Good 
>> Cc: gnucash-user@gnucash.org
>> Subject: Re: [GNC] Third party OFX/CVS providers
>>
>>
>>
>> On Sat, Aug 6, 2022 at 08:51 Chris Good  wrote:
>>
>> ..,
>>
>> Can you copy the text out of the pdf statement?
>>
>>
>>
>> Getting "organized" text programmatically out of PDF is a giant pain. My 
>> current approach is almost completely programmatic. BTW, Raku is the 
>> "sister" language of Perl, and it is a "scripting" language.
>>
>>
>>
>> Raku can also use Perl modules so I can use Finance::Quote with it if need 
>> be. But, until I can get the bank data working, I won't be needing it.
>>
>>
>>
>> Thanks, Chris.
>>
>>
>>
>> -Tom
>>
>>
>>
>> Hi Tom,
>>
>>
>>
>> I was actually suggesting you just highlight the transactions in the pdf 
>> statement and copy/paste them into a
>>
>> spreadsheet or text file before using a scripting language to reformat them 
>> but if you can programmatically
>> extract the text that would be better long term.
>
>But I did see Peter West's warning about checking results
>carefully--it's not always the expected result. Although I hope the
>products from the same source are consistent enough to make it
>reasonably fool proof. (See my comments below.)
>
>> I notice with my PDF statement that (free) Adobe Acrobat DC no longer allows 
>> me to select text (as they want me to pay for a fancy tool - BOO) but if you 
>> send the pdf to Edge or Chrome you can select just the transaction text.
>
>Yes, I now see my "PDF COmplete" on Windows can do that--I never
>thought of doing that until now.
>
>> Thanks for the info re Raku.
>
>You're welcome! I'm always happy for programmers to learn about Raku
>
>> Finance::Quote is just for getting stock prices from websites (usually) so 
>> not useful for this task.
>
>Right, but good to know for later.
>
>> Are you sure you cannot export transactions from your bank? All mine do 
>> although sometimes it is hard to find.
>
>Yes, and I agree it's sometimes hard to find. Yesterday I did find one
>bank (the bank I'm leaving) does have two choices of download for the
>credit card: CSV and OFX. That is very helpful for working on a
>transformation algorithm. I did look at the PDF to text transformation
>on one of the new bank's statements (definitely nothing but PDFs) and
>it looks surprisingly usable, so that's exciting.
>
>I was too hasty in my outright condemnation of PDF to text because I
>had worked on a project with a PDF expert to generate PDF as a native
>file and saw how easy it is to get things out of place. I usually
>create beautiful, and accurate, PDFs by writing the PostScript code
>and relying on Ghostscript's ps2pdf converter. Thus going backwards in
>the workflow is a little different for my longtime mindset.
>
>Best regards,
>
>-Tom
>___
>gnucash-user mailing list
>gnucash-user@gnucash.org
>To update your subscription preferences or to unsubscribe:
>https://lists.gnucash.org/mailman/listinfo/gnucash-user
>-
>Please remember to CC this list on all your replies.
>You can do this by using Reply-To-List or Reply-All.
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Tom Browder]

On Sat, Aug 6, 2022 at 21:25 Chris Good  wrote:
> From: Tom Browder 
> Sent: Sunday, 7 August 2022 12:37 AM
> To: Chris Good 
> Cc: gnucash-user@gnucash.org
> Subject: Re: [GNC] Third party OFX/CVS providers
>
>
>
> On Sat, Aug 6, 2022 at 08:51 Chris Good  wrote:
>
> ..,
>
> Can you copy the text out of the pdf statement?
>
>
>
> Getting "organized" text programmatically out of PDF is a giant pain. My 
> current approach is almost completely programmatic. BTW, Raku is the "sister" 
> language of Perl, and it is a "scripting" language.
>
>
>
> Raku can also use Perl modules so I can use Finance::Quote with it if need 
> be. But, until I can get the bank data working, I won't be needing it.
>
>
>
> Thanks, Chris.
>
>
>
> -Tom
>
>
>
> Hi Tom,
>
>
>
> I was actually suggesting you just highlight the transactions in the pdf 
> statement and copy/paste them into a
>
> spreadsheet or text file before using a scripting language to reformat them 
> but if you can programmatically
> extract the text that would be better long term.

But I did see Peter West's warning about checking results
carefully--it's not always the expected result. Although I hope the
products from the same source are consistent enough to make it
reasonably fool proof. (See my comments below.)

> I notice with my PDF statement that (free) Adobe Acrobat DC no longer allows 
> me to select text (as they want me to pay for a fancy tool - BOO) but if you 
> send the pdf to Edge or Chrome you can select just the transaction text.

Yes, I now see my "PDF COmplete" on Windows can do that--I never
thought of doing that until now.

> Thanks for the info re Raku.

You're welcome! I'm always happy for programmers to learn about Raku

> Finance::Quote is just for getting stock prices from websites (usually) so 
> not useful for this task.

Right, but good to know for later.

> Are you sure you cannot export transactions from your bank? All mine do 
> although sometimes it is hard to find.

Yes, and I agree it's sometimes hard to find. Yesterday I did find one
bank (the bank I'm leaving) does have two choices of download for the
credit card: CSV and OFX. That is very helpful for working on a
transformation algorithm. I did look at the PDF to text transformation
on one of the new bank's statements (definitely nothing but PDFs) and
it looks surprisingly usable, so that's exciting.

I was too hasty in my outright condemnation of PDF to text because I
had worked on a project with a PDF expert to generate PDF as a native
file and saw how easy it is to get things out of place. I usually
create beautiful, and accurate, PDFs by writing the PostScript code
and relying on Ghostscript's ps2pdf converter. Thus going backwards in
the workflow is a little different for my longtime mindset.

Best regards,

-Tom
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Chris Good]

From: Tom Browder  
Sent: Sunday, 7 August 2022 12:37 AM
To: Chris Good 
Cc: gnucash-user@gnucash.org
Subject: Re: [GNC] Third party OFX/CVS providers

 

On Sat, Aug 6, 2022 at 08:51 Chris Good mailto:goodchri...@gmail.com> > wrote:

..,

Can you copy the text out of the pdf statement?

 

Getting "organized" text programmatically out of PDF is a giant pain. My 
current approach is almost completely programmatic. BTW, Raku is the "sister" 
language of Perl, and it is a "scripting" language. 

 

Raku can also use Perl modules so I can use Finance::Quote with it if need be. 
But, until I can get the bank data working, I won't be needing it.

 

Thanks, Chris.

 

-Tom

 

Hi Tom,

 

I was actually suggesting you just highlight the transactions in the pdf 
statement and copy/paste them into a

spreadsheet or text file before using a scripting language to reformat them but 
if you can programmatically

extract the text that would be better long term.

 

I notice with my PDF statement that (free) Adobe Acrobat DC no longer allows me 
to select text (as they want me to pay for a fancy tool - BOO) but if you send 
the pdf to Edge or Chrome you can select just the transaction text.

 

Thanks for the info re Raku.

 

Finance::Quote is just for getting stock prices from websites (usually) so not 
useful for this task.

 

Are you sure you cannot export transactions from your bank? All mine do 
although sometimes it is hard to find.

 

Regards,

Chris Good

 

___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Kalpesh Patel]

Tom,

 

Not sure if Python is on the table or not but it looks simple to go from PDF
to CSV format programmatically in Python (I un-learned Perl in 2010's and
learned Python in place). This will do away with needing very fine external
programs likes of Ghostscript and/or or iText7. 

 

Take a look at this link https://www.pythonpool.com/python-pdf-parser/ and
https://medium.com/analytics-vidhya/python-packages-for-pdf-data-extraction-
d14ec30f0ad0 if you want to be brave and venture into Python realm. Then you
can use yahoo_fin library in python to pull in prices for security as well.
I wrote a script that does this latter part to create a Quicken 2017
importable CSV file for prices. It is available (to anyone) for asking. 

 

Kalpesh.

 

--

 

Message: 5

Date: Fri, 5 Aug 2022 19:47:45 -0500

From: Tom Browder 

To: Gnucash Users 

Subject: Re: [GNC] Third party OFX/CVS providers

Message-ID:

 


Content-Type: text/plain; charset="UTF-8"

 

On Wed, Aug 3, 2022 at 17:39 Tom Browder  wrote:

 

> My current bank (US, Hancock Whitney) unfortunately does not provide 

> account data in a digital format other than monthly PDF statements for 

> personal accounts. I know third-party programs, such as YNAB 

> (ynab.com), can get access my bank while I can't, but I don't want the 

> baggage that comes with it.

> 

 

I changed my mind. I now have an YNAB account and found that is has a
well-documented API. Not only that, I was able to link it successfully to my
two banks and access my checking, savings, and credit card transactions!

It might be possible to use the API to periodically download bank and credit
card transactions into YNAB's standard JSON format, then convert that into
the OFX format, and then upload the OFX into GnuCash.

 

I'm proceeding on the project with the hope of success. If anyone is
interested in following it, the Raku (formerly Perl 6, see https://raku.org)
module skeleton is on my Github repo now at
https://github.com/tbrowder/API-YNAB.git. (Note I may soon change the title
to something like YNAB2OFX or YNAB-OFX--suggestions are very welcome.)

 

The downside, assuming I have success, is that YNAB has a subscription fee
of about $15 per month plus tax (cancel any time) or $100 plus tax a year,
sums well worth it if I can get a periodic good OFX-to-GnuCash import.

 

Best regards,

 

-Tom

 

--

 

 

 

___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

[GNC] Third party OFX/CVS providers

[flywire]

My Citibank statements are only available as pdf and I use
https://github.com/flywire/pdf_statement_reader . It is based on tabula and
forks another project to make it functional. More config info at
https://github.com/marlanperumal/pdf_statement_reader/issues/34 .
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Tom Browder]

On Sat, Aug 6, 2022 at 18:40 Peter West  wrote:

> The other great tool for processing pdfs is PDFBox. It’s a java jar file,
> so you need to have a reasonably recent JVM installed.
>
...

Thanks, Peter.

And God bless you and yours!

-Tom
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Peter West]

The other great tool for processing pdfs is PDFBox. It’s a java jar file, so 
you need to have a reasonably recent JVM installed.

https://pdfbox.apache.org/ 

I’m still using version2.
Here are the commandline tools.
https://pdfbox.apache.org/2.0/commandline.html 



I have a series of scripts for various types of manipulation and text 
extraction.

It is worth noting that people who naïvely believe that text extraction from a 
PDF is simple will get burnt if they are not checking the results. PDF files 
are not obliged to store their text in any particular order. Mostly they do, 
until they don’t.

I can post my scripts if anyone is interested.

There are also a number of other tools for linux that i access on  my Mac 
through MacPorts.

God bless you.

—
Peter West
p...@pbw.id.au
“The kingdom of heaven is like treasure hidden in a field, which a man found 
and covered up. Then in his joy he goes and sells all that he has and buys that 
field.”

> On 7 Aug 2022, at 2:52 am, Tom Browder  wrote:
> 
> On Sat, Aug 6, 2022 at 11:43 AM Glenn Fowler  wrote:
>> 
>> My scripts are in PowerShell. For GhostScript I'm just using CLI:
> 
> Thanks, Glenn, that's close to what I've found for Linux:
> 
>$ gs -sDEVICE=txtwrite -o output.txt input.pdf
> 
> It just needs some tweaking and post-conversion parsing (very bank
> specific). I'll see how my current PDF statements look after text
> conversion.
> 
> But I'll also keep looking at YNAB for a more general solution.
> 
> Cheers!
> 
> -Tom
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.

___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Tom Browder]

On Sat, Aug 6, 2022 at 11:43 AM Glenn Fowler  wrote:
>
> My scripts are in PowerShell. For GhostScript I'm just using CLI:

Thanks, Glenn, that's close to what I've found for Linux:

$ gs -sDEVICE=txtwrite -o output.txt input.pdf

It just needs some tweaking and post-conversion parsing (very bank
specific). I'll see how my current PDF statements look after text
conversion.

But I'll also keep looking at YNAB for a more general solution.

Cheers!

-Tom
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Glenn Fowler]

My scripts are in PowerShell. For GhostScript I'm just using CLI:

"C:\Program Files\gs\gs9.56.1\bin\gswin64c.exe" -dBATCH -dNOPAUSE -dQUIET
-dNoCancel -sDEVICE=txtwrite -sOutputFile=%stdout "InputFile.pdf"

This give you the whole PDF and now just parse the text.

For iText7 I am using PSWritePDF module:

https://github.com/EvotecIT/PSWritePDF
https://www.powershellgallery.com/packages/PSWritePDF/0.0.19

You just use it's Cmdlet ConvertPDFToText:

Convert-PDFToText -FilePath "InputFile.pdf"

Now just parse the text.

On Sat, Aug 6, 2022 at 11:46 AM Tom Browder  wrote:

> On Sat, Aug 6, 2022 at 10:43 AM Glenn Fowler  wrote:
> >
> > Look into GhostScript and iText7. I have used both to pull text out of
> PDF's easily in scripts.
>
> What program language? Can you show me an example of your code to do that?
>
> Thanks.
>
> -Tom
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Tom Browder]

On Sat, Aug 6, 2022 at 10:43 AM Glenn Fowler  wrote:
>
> Look into GhostScript and iText7. I have used both to pull text out of PDF's 
> easily in scripts.

What program language? Can you show me an example of your code to do that?

Thanks.

-Tom
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Glenn Fowler]

Look into GhostScript and iText7. I have used both to pull text out of
PDF's easily in scripts.

On Sat, Aug 6, 2022, 10:36 AM Tom Browder  wrote:

> On Sat, Aug 6, 2022 at 08:51 Chris Good  wrote:
> ..,
>
> > Can you copy the text out of the pdf statement?
>
>
> Getting "organized" text programmatically out of PDF is a giant pain. My
> current approach is almost completely programmatic. BTW, Raku is the
> "sister" language of Perl, and it is a "scripting" language.
>
> Raku can also use Perl modules so I can use Finance::Quote with it if need
> be. But, until I can get the bank data working, I won't be needing it.
>
> Thanks, Chris.
>
> -Tom
> ___
> gnucash-user mailing list
> gnucash-user@gnucash.org
> To update your subscription preferences or to unsubscribe:
> https://lists.gnucash.org/mailman/listinfo/gnucash-user
> -
> Please remember to CC this list on all your replies.
> You can do this by using Reply-To-List or Reply-All.
>
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Tom Browder]

On Sat, Aug 6, 2022 at 08:51 Chris Good  wrote:
..,

> Can you copy the text out of the pdf statement?


Getting "organized" text programmatically out of PDF is a giant pain. My
current approach is almost completely programmatic. BTW, Raku is the
"sister" language of Perl, and it is a "scripting" language.

Raku can also use Perl modules so I can use Finance::Quote with it if need
be. But, until I can get the bank data working, I won't be needing it.

Thanks, Chris.

-Tom
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Chris Good]

> Message: 5
> Date: Fri, 5 Aug 2022 19:47:45 -0500
> From: Tom Browder 
> To: Gnucash Users 
> Subject: Re: [GNC] Third party OFX/CVS providers
> Message-ID:
>
> Content-Type: text/plain; charset="UTF-8"
> 
>> On Wed, Aug 3, 2022 at 17:39 Tom Browder  wrote:
>> 
>> My current bank (US, Hancock Whitney) unfortunately does not provide
>> account data in a digital format other than monthly PDF statements for
>> personal accounts. I know third-party programs, such as YNAB (ynab.com),
>> can get access my bank while I can't, but I don't want the baggage that
>> comes with it.
>> 
> 
> I changed my mind. I now have an YNAB account and found that is has a
> well-documented API. Not only that, I was able to link it successfully to
> my two banks and access my checking, savings, and credit card transactions!
> It might be possible to use the API to periodically download bank and
> credit card transactions into YNAB's standard JSON format, then convert
> that into the OFX format, and then upload the OFX into GnuCash.
> 
> I'm proceeding on the project with the hope of success. If anyone is
> interested in following it, the Raku (formerly Perl 6, see https://raku.org)
> module skeleton is on my Github repo now at
> https://github.com/tbrowder/API-YNAB.git. (Note I may soon change the title
> to something like YNAB2OFX or YNAB-OFX--suggestions are very welcome.)
> 
> The downside, assuming I have success, is that YNAB has a subscription fee
> of about $15 per month plus tax (cancel any time) or $100 plus tax a year,
> sums well worth it if I can get a periodic good OFX-to-GnuCash import.
> 
> Best regards,
> 
> -Tom
> 
Hi Tom,

Can you copy the text out of the pdf statement?
If so, you could probably write a spreadsheet macro to create a csv, ofx or qif 
file to import. Alternatively, if you are on Linux, use some scripting language 
like Perl or awk.
If you use Finance::Quote to import market prices, you already have Perl 
installed.

Regards, Chris Good
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

Re: [GNC] Third party OFX/CVS providers

[Tom Browder]

On Wed, Aug 3, 2022 at 17:39 Tom Browder  wrote:

> My current bank (US, Hancock Whitney) unfortunately does not provide
> account data in a digital format other than monthly PDF statements for
> personal accounts. I know third-party programs, such as YNAB (ynab.com),
> can get access my bank while I can't, but I don't want the baggage that
> comes with it.
>

I changed my mind. I now have an YNAB account and found that is has a
well-documented API. Not only that, I was able to link it successfully to
my two banks and access my checking, savings, and credit card transactions!
It might be possible to use the API to periodically download bank and
credit card transactions into YNAB's standard JSON format, then convert
that into the OFX format, and then upload the OFX into GnuCash.

I'm proceeding on the project with the hope of success. If anyone is
interested in following it, the Raku (formerly Perl 6, see https://raku.org)
module skeleton is on my Github repo now at
https://github.com/tbrowder/API-YNAB.git. (Note I may soon change the title
to something like YNAB2OFX or YNAB-OFX--suggestions are very welcome.)

The downside, assuming I have success, is that YNAB has a subscription fee
of about $15 per month plus tax (cancel any time) or $100 plus tax a year,
sums well worth it if I can get a periodic good OFX-to-GnuCash import.

Best regards,

-Tom
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.

[GNC] Third party OFX/CVS providers

[Tom Browder]

My current bank (US, Hancock Whitney) unfortunately does not provide
account data in a digital format other than monthly PDF statements for
personal accounts. I know third-party programs, such as YNAB (ynab.com),
can get access my bank while I can't, but I don't want the baggage that
comes with it.

Does anyone know of a third-party entity I could use just to get access to
the raw data for my personal accounts?

Thanks.

-Tom
___
gnucash-user mailing list
gnucash-user@gnucash.org
To update your subscription preferences or to unsubscribe:
https://lists.gnucash.org/mailman/listinfo/gnucash-user
-
Please remember to CC this list on all your replies.
You can do this by using Reply-To-List or Reply-All.