Re: [GNC] Archiving transactions
Hi, I should clarify that when I mean transactions I am referring to double entries. Each business transaction, from the e-commerce stuff I do, outputs on average five double entries: one each for sales, commissions, shipping, coupons and so on. So 500k double entries would yield around 100k actual business transactions over 3 or so years. The software is not slow to the point of being non-functional or a even frustration. It is only that I prefer a tad bit more sprightliness. Also it seems to run perfectly well on my relatively slower laptop which just has 6GB of RAM. Noted about the loading of transactions to memory. I will keep a closer eye on the memory footprint. Unfortunately my programming experience is limited to VBA on MS Excel. I have sadly not worked with apps, and with databases only very little. I am not sure how I could contribute to the development over someone with actual database experience. Thanks, Rizwan On Mon, Nov 18, 2019, 11:21 AM D wrote: > Rizwan, > > I am intrigued. You say you're a new user, but you have 500,000 > transactions. Wow! > > You've been getting the usual kinds of responses to your question: how > fast is your computer, how many reports are you running, be patient, don't > close Gnucash, etc. > > But it sounds as if you're pushing boundaries that most of us > personal/small business users aren't. For example, my data file loads and > runs pretty well. It covers over ten years of transactions, and the last > time I checked, it was around 200,000 transactions all told. It may be that > you are an outlier user who tests the functional limits of Gnucash. > > One major issue that has long been known is that the software is not built > to think like a database app. It loads the entire data file into memory on > start up. It does this even with the SQL back ends, which is a roundabout > way to tell you that changing to a database back end won't help at this > time, sadly. > > I see that you had another query about transaction imports that suggests > that you have some programming experience. Many of us would cheer if you > were to put that experience to work helping the small development team. One > really big area they are looking at is the migration of Gnucash to a true > db app--a long term goal, to be sure, since there are so many immediate > issues that need to be addressed. > > Cheers, > David T. > > On November 18, 2019, at 8:17 AM, David Carlson < > david.carlson@gmail.com> wrote: > > There is a setting under Edit > Preferences > General to compress the data > file. I think it is selected by default. In Windows 10 compression > happens very fast so that is not likely to be an issue with speed. > > However, because the entire database is [theoretically] in RAM, at some > point that will cause RAM to spill over into swap space on the local > drive. Then things really slow down. This will happen more quickly if a > web browser is running concurrently. In Windows open the Task Manager and > look under performance to see how your machine is doing. > > David Carlson > > On Sun, Nov 17, 2019 at 8:14 PM M. Rizwan Muzzammil > wrote: > > > I will check. Thanks for the suggestion. > > > > On Mon, Nov 18, 2019, 10:06 AM Greg Feneis wrote: > > > > > It seems like there used to be an option to encrypt, or zip the working > > > file when it's the default xml type. If that's still an option, and is > > > enabled, it could cause a delay relative to the file size > > > > > > Kind regards, Greg Feneis > > > (Pixel 3) > > > > > > > > > On Sun, Nov 17, 2019, 17:49 M. Rizwan Muzzammil > > > wrote: > > > > > > > Thanks for your replyI am using the default file types. Does it > > make > > > a > > > > difference if I change? > > > > If so to what other type and how can it be done? > > > > > > > > > > > > > > > > > > > ___ > > > gnucash-user mailing list > > > gnucash-user@gnucash.org > > > To update your subscription preferences or to unsubscribe: > > > https://lists.gnucash.org/mailman/listinfo/gnucash-user > > > If you are using Nabble or Gmane, please see > > > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > > > - > > > Please remember to CC this list on all your replies. > > > You can do this by using Reply-To-List or Reply-All. > > > > > ___ > > gnucash-user mailing list > > gnucash-user@gnucash.org > > To update your subscription preferences or to unsubscribe: > > https://lists.gnucash.org/mailman/listinfo/gnucash-user > > If you are using Nabble or Gmane, please see > > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > > - > > Please remember to CC this list on all your replies. > > You can do this by using Reply-To-List or Reply-All. > > > > > -- > David Carlson > ___ > gnucash-user mailing list > gnucash-user@gnucash.org > To update your subscription preferences or to unsubscribe: >
Re: [GNC] Archiving transactions
Rizwan, I am intrigued. You say you're a new user, but you have 500,000 transactions. Wow! You've been getting the usual kinds of responses to your question: how fast is your computer, how many reports are you running, be patient, don't close Gnucash, etc. But it sounds as if you're pushing boundaries that most of us personal/small business users aren't. For example, my data file loads and runs pretty well. It covers over ten years of transactions, and the last time I checked, it was around 200,000 transactions all told. It may be that you are an outlier user who tests the functional limits of Gnucash. One major issue that has long been known is that the software is not built to think like a database app. It loads the entire data file into memory on start up. It does this even with the SQL back ends, which is a roundabout way to tell you that changing to a database back end won't help at this time, sadly. I see that you had another query about transaction imports that suggests that you have some programming experience. Many of us would cheer if you were to put that experience to work helping the small development team. One really big area they are looking at is the migration of Gnucash to a true db app--a long term goal, to be sure, since there are so many immediate issues that need to be addressed. Cheers, David T. On November 18, 2019, at 8:17 AM, David Carlson wrote: There is a setting under Edit > Preferences > General to compress the data file. I think it is selected by default. In Windows 10 compression happens very fast so that is not likely to be an issue with speed. However, because the entire database is [theoretically] in RAM, at some point that will cause RAM to spill over into swap space on the local drive. Then things really slow down. This will happen more quickly if a web browser is running concurrently. In Windows open the Task Manager and look under performance to see how your machine is doing. David Carlson On Sun, Nov 17, 2019 at 8:14 PM M. Rizwan Muzzammil wrote: > I will check. Thanks for the suggestion. > > On Mon, Nov 18, 2019, 10:06 AM Greg Feneis wrote: > > > It seems like there used to be an option to encrypt, or zip the working > > file when it's the default xml type. If that's still an option, and is > > enabled, it could cause a delay relative to the file size > > > > Kind regards, Greg Feneis > > (Pixel 3) > > > > > > On Sun, Nov 17, 2019, 17:49 M. Rizwan Muzzammil > > wrote: > > > > > Thanks for your replyI am using the default file types. Does it > make > > a > > > difference if I change? > > > If so to what other type and how can it be done? > > > > > > > > > > > > > > ___ > > gnucash-user mailing list > > gnucash-user@gnucash.org > > To update your subscription preferences or to unsubscribe: > > https://lists.gnucash.org/mailman/listinfo/gnucash-user > > If you are using Nabble or Gmane, please see > > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > > - > > Please remember to CC this list on all your replies. > > You can do this by using Reply-To-List or Reply-All. > > > ___ > gnucash-user mailing list > gnucash-user@gnucash.org > To update your subscription preferences or to unsubscribe: > https://lists.gnucash.org/mailman/listinfo/gnucash-user > If you are using Nabble or Gmane, please see > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > - > Please remember to CC this list on all your replies. > You can do this by using Reply-To-List or Reply-All. > -- David Carlson ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All. ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
Re: [GNC] Archiving transactions
There is a setting under Edit > Preferences > General to compress the data file. I think it is selected by default. In Windows 10 compression happens very fast so that is not likely to be an issue with speed. However, because the entire database is [theoretically] in RAM, at some point that will cause RAM to spill over into swap space on the local drive. Then things really slow down. This will happen more quickly if a web browser is running concurrently. In Windows open the Task Manager and look under performance to see how your machine is doing. David Carlson On Sun, Nov 17, 2019 at 8:14 PM M. Rizwan Muzzammil wrote: > I will check. Thanks for the suggestion. > > On Mon, Nov 18, 2019, 10:06 AM Greg Feneis wrote: > > > It seems like there used to be an option to encrypt, or zip the working > > file when it's the default xml type. If that's still an option, and is > > enabled, it could cause a delay relative to the file size > > > > Kind regards, Greg Feneis > > (Pixel 3) > > > > > > On Sun, Nov 17, 2019, 17:49 M. Rizwan Muzzammil > > wrote: > > > > > Thanks for your replyI am using the default file types. Does it > make > > a > > > difference if I change? > > > If so to what other type and how can it be done? > > > > > > > > > > > > > > ___ > > gnucash-user mailing list > > gnucash-user@gnucash.org > > To update your subscription preferences or to unsubscribe: > > https://lists.gnucash.org/mailman/listinfo/gnucash-user > > If you are using Nabble or Gmane, please see > > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > > - > > Please remember to CC this list on all your replies. > > You can do this by using Reply-To-List or Reply-All. > > > ___ > gnucash-user mailing list > gnucash-user@gnucash.org > To update your subscription preferences or to unsubscribe: > https://lists.gnucash.org/mailman/listinfo/gnucash-user > If you are using Nabble or Gmane, please see > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > - > Please remember to CC this list on all your replies. > You can do this by using Reply-To-List or Reply-All. > -- David Carlson ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
Re: [GNC] Archiving transactions
I will check. Thanks for the suggestion. On Mon, Nov 18, 2019, 10:06 AM Greg Feneis wrote: > It seems like there used to be an option to encrypt, or zip the working > file when it's the default xml type. If that's still an option, and is > enabled, it could cause a delay relative to the file size > > Kind regards, Greg Feneis > (Pixel 3) > > > On Sun, Nov 17, 2019, 17:49 M. Rizwan Muzzammil > wrote: > > > Thanks for your replyI am using the default file types. Does it make > a > > difference if I change? > > If so to what other type and how can it be done? > > > > > > > > > ___ > gnucash-user mailing list > gnucash-user@gnucash.org > To update your subscription preferences or to unsubscribe: > https://lists.gnucash.org/mailman/listinfo/gnucash-user > If you are using Nabble or Gmane, please see > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > - > Please remember to CC this list on all your replies. > You can do this by using Reply-To-List or Reply-All. > ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
Re: [GNC] Archiving transactions
It seems like there used to be an option to encrypt, or zip the working file when it's the default xml type. If that's still an option, and is enabled, it could cause a delay relative to the file size Kind regards, Greg Feneis (Pixel 3) On Sun, Nov 17, 2019, 17:49 M. Rizwan Muzzammil wrote: > Thanks for your replyI am using the default file types. Does it make a > difference if I change? > If so to what other type and how can it be done? > > > > ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
Re: [GNC] Archiving transactions
Thanks for your reply. I do not believe it is a hardware issue as this is a new PC. I am running gnucash on Win10 and reading from a, relatively slowish but still very quick, local NVMe SSD. I am using the default file types. Does it make a difference if I change? If so to what other type and how can it be done? On Sun, Nov 17, 2019, 5:11 AM Colin Law wrote: > On Sat, 16 Nov 2019 at 18:14, M. Rizwan Muzzammil > wrote: > > ... > > My accounts have a large number of transactions in them. > > > > This seems to slow the program down, and it take a minute or so to open > it > > each time. > > How large is your accounts file (I assume that you are using the > default xml file storage rather than sqlite or mysql)? > What processor are you running on, if you know that. Otherwise what > make/model PC is it? Also what operating system? > Is the file stored on a disc on the PC or on a network drive? > > Colin > ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
Re: [GNC] Archiving transactions
Thanks for your reply. I don't usually keep reports open so I do not believe it is a reason. Also I recall on an occasion where a report tab, that was kept open, took time to reload on activating the tab. So i assume that time was saved on opening. Will check later today. Slowness is perhaps due to transaction volume. I don't have an exact count but I guess it is near to the 500,000 mark. I was mainly searching for an automatic method to truncate. If there is none I guess I will have to do it manually as you suggest. On Sun, Nov 17, 2019, 5:17 AM David Carlson wrote: > Slowness to open and to save and close, and even to commit any transaction > edit has been a known problem for some time now. Releases after about 3.4 > or so are supposed to be much improved, but there may still be slowness if > there are a lot of account tabs to open or reports to recreate. There was > also an issue if the account windows were rather large, if I recall > correctly. GnuCash may still seem slow in release 3.7, but I have not > tried it personally yet, so I have no personal experience. > > That said, it is possible to truncate history out of a file, but the > process is not automated and requires a combination of opening a new file > with the same account structure then manually setting all the opening > balances and importing the data that you want to keep from exports from the > old file. > > If you only have a few thousand transactions it is easier to fine tune > your work method to minimize the lags. > > On Sat, Nov 16, 2019 at 12:14 PM M. Rizwan Muzzammil > wrote: > >> Hi all, >> >> New user of GNU cash here. >> >> My accounts have a large number of transactions in them. >> >> This seems to slow the program down, and it take a minute or so to open it >> each time. >> >> Would there be a way to archive older previous year transactions so that >> the program runs faster? >> >> Thanks very much in advance for any help. >> >> Rizwan >> ___ >> gnucash-user mailing list >> gnucash-user@gnucash.org >> To update your subscription preferences or to unsubscribe: >> https://lists.gnucash.org/mailman/listinfo/gnucash-user >> If you are using Nabble or Gmane, please see >> https://wiki.gnucash.org/wiki/Mailing_Lists for more information. >> - >> Please remember to CC this list on all your replies. >> You can do this by using Reply-To-List or Reply-All. >> > > > -- > David Carlson > ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
Re: [GNC] Archiving transactions
Also note that leaving report tabs open when closing GnuCash will slow it down opening back up as each of those reports is loaded once again. (same goes for all tabs) If your workflow can handle it, either close all tabs except the Accounts tab before exiting, or, don’t close GnuCash unless you really need to. (I leave it open 24/7 and only restart when updating) Regards, Adrien > On Nov 16, 2019 w46d320, at 12:03 PM, M. Rizwan Muzzammil > wrote: > > Hi all, > > New user of GNU cash here. > > My accounts have a large number of transactions in them. > > This seems to slow the program down, and it take a minute or so to open it > each time. > > Would there be a way to archive older previous year transactions so that > the program runs faster? > > Thanks very much in advance for any help. > > Rizwan ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
Re: [GNC] Archiving transactions
Slowness to open and to save and close, and even to commit any transaction edit has been a known problem for some time now. Releases after about 3.4 or so are supposed to be much improved, but there may still be slowness if there are a lot of account tabs to open or reports to recreate. There was also an issue if the account windows were rather large, if I recall correctly. GnuCash may still seem slow in release 3.7, but I have not tried it personally yet, so I have no personal experience. That said, it is possible to truncate history out of a file, but the process is not automated and requires a combination of opening a new file with the same account structure then manually setting all the opening balances and importing the data that you want to keep from exports from the old file. If you only have a few thousand transactions it is easier to fine tune your work method to minimize the lags. On Sat, Nov 16, 2019 at 12:14 PM M. Rizwan Muzzammil wrote: > Hi all, > > New user of GNU cash here. > > My accounts have a large number of transactions in them. > > This seems to slow the program down, and it take a minute or so to open it > each time. > > Would there be a way to archive older previous year transactions so that > the program runs faster? > > Thanks very much in advance for any help. > > Rizwan > ___ > gnucash-user mailing list > gnucash-user@gnucash.org > To update your subscription preferences or to unsubscribe: > https://lists.gnucash.org/mailman/listinfo/gnucash-user > If you are using Nabble or Gmane, please see > https://wiki.gnucash.org/wiki/Mailing_Lists for more information. > - > Please remember to CC this list on all your replies. > You can do this by using Reply-To-List or Reply-All. > -- David Carlson ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
Re: [GNC] Archiving transactions
On Sat, 16 Nov 2019 at 18:14, M. Rizwan Muzzammil wrote: > ... > My accounts have a large number of transactions in them. > > This seems to slow the program down, and it take a minute or so to open it > each time. How large is your accounts file (I assume that you are using the default xml file storage rather than sqlite or mysql)? What processor are you running on, if you know that. Otherwise what make/model PC is it? Also what operating system? Is the file stored on a disc on the PC or on a network drive? Colin ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
[GNC] Archiving transactions
Hi all, New user of GNU cash here. My accounts have a large number of transactions in them. This seems to slow the program down, and it take a minute or so to open it each time. Would there be a way to archive older previous year transactions so that the program runs faster? Thanks very much in advance for any help. Rizwan ___ gnucash-user mailing list gnucash-user@gnucash.org To update your subscription preferences or to unsubscribe: https://lists.gnucash.org/mailman/listinfo/gnucash-user If you are using Nabble or Gmane, please see https://wiki.gnucash.org/wiki/Mailing_Lists for more information. - Please remember to CC this list on all your replies. You can do this by using Reply-To-List or Reply-All.
