In case anyone is interested, I think I found where the slowness of my Python script occurs. Piecash uses something called SQLAlchemy, and it seems the slowness is in this package when committing new data to the database. I'm going to put up with the slowness. I only import new transactions to the database every month or two, and I'll just go and do something else while it does its work.
Cheers, Tim. Sent with ProtonMail secure email. ------- Original Message ------- On Monday, May 9th, 2022 at 07:18, Tim Hume via gnucash-user <gnucash-user@gnucash.org> wrote: > Hi people, > > I guess the underlying issue is probably Piecash - or possibly the way it > interacts with SQLite. Though GnuCash itself is pretty slow at exit time too > (maybe 30 seconds to a minute) - but not in the same league of slowness as my > Python program using Piecash. > > I was simply hoping that maybe someone else had experienced this and might > have some tips - I realise this mailing list is for GnuCash, but given > Piecash is used for manipulating GnuCash files, the thought occurred that > others on the list may have some tips. The MySQL/PostgreSQL suggestion may be > a good one. But before I go down this track I'll a browse though the Piecash > code. > > I'll report back if I find anything or have success in speeding things up. > > Cheers, > > Tim. > > Sent with ProtonMail secure email. > > ------- Original Message ------- > On Sunday, May 8th, 2022 at 15:30, john jra...@ceridwen.us wrote: > > > Is this about your python program being slow or about GnuCash being slow? > > Note that Piecash isn't GnuCash and we don't provide any support for it, > > nor obviously can we provide any support for your program. > > > > If it's a GnuCash problem, please provide more details about exactly what > > you're doing and what step is taking 20 minutes. > > > > Regards, > > John Ralls > > > > > On May 8, 2022, at 4:50 AM, Tim Hume via gnucash-user > > > gnucash-user@gnucash.org wrote: > > > > > > It is an SSD drive (I'm using a Mac Mini about one year old - with the M1 > > > chip). > > > > > > The slow part seems to be saving the database - inserting the > > > transactions is fast enough, the save operation is what takes so long. It > > > may be 20 minutes - I haven't timed it. Even closing GnuCash is slow - > > > perhaps 30 seconds to a minute. > > > > > > Sent with ProtonMail secure email. > > > ------- Original Message ------- > > > On Sunday, May 8th, 2022 at 11:40, Kenneth Schneider > > > kschnei...@bout-tyme.net wrote: > > > > > > > I don’t believe the file is getting too large. If you haven’t looked > > > > into using an SSD drive at least for the database I highly recommend > > > > it. Not only will you amazed at how fast data will be written to the > > > > file but also how fast you will be able to generate reports. > > > > > > > > Ken Schneider > > > > > > > > > On May 8, 2022, at 7:24 AM, Tim Hume via gnucash-user > > > > > gnucash-user@gnucash.org wrote: > > > > > > > > > > Hi, > > > > > > > > > > My GnuCash file (SQLite format) has become rather large - 110 MBytes. > > > > > Every month or two I import a lot more transactions in CSV format. I > > > > > use piecash in a Python script I wrote, because the importing of CSV > > > > > data built into GnuCash crashes when I import too many transactions. > > > > > > > > > > I might be importing perhaps 4000 or 5000 transactions. Saving them > > > > > into my GnuCash file takes many many minutes. I know my file is > > > > > getting too large, but it surely shouldn't take 10+ minutes to do > > > > > this operation? Whilst I do it, I can see Python is taking 100% of a > > > > > CPU core. > > > > > > > > > > Come the end of the financial year (30 June) I'm going to start a new > > > > > file. But this will quickly grow in size, and no doubt I'll run into > > > > > this problem again. Is there anything I can do to speed this up, or > > > > > do I just have to live with it? > > > > > > > > > > Cheers, > > > > > > > > > > Tim. > > > > > > > > > > Sent with ProtonMail secure email. > > > > > _______________________________________________ > > > > > gnucash-user mailing list > > > > > gnucash-user@gnucash.org > > > > > To update your subscription preferences or to unsubscribe: > > > > > https://lists.gnucash.org/mailman/listinfo/gnucash-user > > > > > If you are using Nabble or Gmane, please see > > > > > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > > > > > ----- > > > > > Please remember to CC this list on all your replies. > > > > > You can do this by using Reply-To-List or Reply-All. > > > > > > > > _______________________________________________ > > > > gnucash-user mailing list > > > > gnucash-user@gnucash.org > > > > To update your subscription preferences or to unsubscribe: > > > > https://lists.gnucash.org/mailman/listinfo/gnucash-user > > > > If you are using Nabble or Gmane, please see > > > > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > > > > ----- > > > > Please remember to CC this list on all your replies. > > > > You can do this by using Reply-To-List or Reply-All. > > > > > > _______________________________________________ > > > gnucash-user mailing list > > > gnucash-user@gnucash.org > > > To update your subscription preferences or to unsubscribe: > > > https://lists.gnucash.org/mailman/listinfo/gnucash-user > > > If you are using Nabble or Gmane, please > > > seehttps://wiki.gnucash.org/wiki/Mailing_Listsfor more information. > > > ----- > > > Please remember to CC this list on all your replies. > > > You can do this by using Reply-To-List or Reply-All. > > _______________________________________________ > gnucash-user mailing list > gnucash-user@gnucash.org > To update your subscription preferences or to unsubscribe: > https://lists.gnucash.org/mailman/listinfo/gnucash-user > If you are using Nabble or Gmane, please see > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > ----- > Please remember to CC this list on all your replies. > You can do this by using Reply-To-List or Reply-All. _______________________________________________ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. ----- Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
