Is this why the native disk storage just keeps growing and does not reduce after we delete from ignite using SQL? We are up to 80GB on disk now on some instances. We implemented a custom archiving feature to move older data out of ignite cache to a PostgresSQL database but when we delete that data from ignite instance, the disk data size ignite is using stays the same, and then keeps growing, and growing....
On Thu, Jun 24, 2021 at 7:10 PM Denis Magda <dma...@apache.org> wrote: > Ignite fellows, > > I remember some of us worked on the persistence defragmentation features. > Has it been merged? > > @Valentin Kulichenko <valentin.kuliche...@gmail.com> probably you know > the latest state. > > - > Denis > > On Thu, Jun 24, 2021 at 11:59 AM Ilya Kasnacheev < > ilya.kasnach...@gmail.com> wrote: > >> Hello! >> >> You can probably drop the entire cache and then re-populate it via >> loadCache(), etc. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> ср, 23 июн. 2021 г. в 21:47, Ryan Trollip <ryanonthebe...@gmail.com>: >> >>> Thanks, Ilya, we may have to consider moving back to non-native storage >>> and caching more selectively as the performance degrades when there is a >>> lot of write/delete activity or tables with large amounts of rows. This is >>> with SQL with indexes and the use of query plans etc. >>> >>> Is there any easy way to rebuild the entire native database after hours? >>> e.g. with a batch run on the weeknds? >>> >>> On Wed, Jun 23, 2021 at 7:39 AM Ilya Kasnacheev < >>> ilya.kasnach...@gmail.com> wrote: >>> >>>> Hello! >>>> >>>> I don't think there's anything ready to use, but "killing performance" >>>> from fragmentation is also not something reported too often. >>>> >>>> Regards, >>>> -- >>>> Ilya Kasnacheev >>>> >>>> >>>> ср, 16 июн. 2021 г. в 04:39, Ryan Trollip <ryanonthebe...@gmail.com>: >>>> >>>>> We see continual very large growth to data with ignite native. We have >>>>> a very chatty use case that's creating and deleting stuff often. The data >>>>> on disk just keeps growing at an explosive rate. So much so we ported this >>>>> to a DB to see the difference and the DB is much smaller. I was searching >>>>> to see if someone has the same issue. This is also killing performance. >>>>> >>>>> Founds this: >>>>> >>>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-47%3A+Native+persistence+defragmentation >>>>> >>>>> Apparently, there is no auto-rebalancing of pages? or cleanup of >>>>> pages? >>>>> >>>>> Has anyone implemented a workaround to rebuild the cache and indexes >>>>> say on a weekly basis to get it to behave reasonably? >>>>> >>>>> Thanks >>>>> >>>>
