Alexey, >> The solution which Anton suggested does not look easy because it will most likely significantly hurt performance Mostly agree here, but what drop do we expect? What price do we ready to pay? Not sure, but seems some vendors ready to pay, for example, 5% drop for this.
>> it is hard to maintain a data structure to choose "page from free-list with enough space closest to the beginning of the file". We can just split each free-list bucket to the couple and use first for pages in the first half of the file and the second for the last. Only two buckets required here since, during the file shrink, first bucket's window will be shrank too. Seems, this give us the same price on put, just use the first bucket in case it's not empty. Remove price (with merge) will be increased, of course. The compromise solution is to have priority put (to the first path of the file), with keeping removal as is, and schedulable per-page migration for the rest of the data during the low activity period. On Wed, Sep 18, 2019 at 8:03 PM Alexey Goncharuk <alexey.goncha...@gmail.com> wrote: > Denis, > > It's not fundamental, but quite complex. In postgres, for example, this is > not maintained automatically and store compaction is performed using the > full vacuum command, which acquires exclusive table lock, so no concurrent > activities on the table are possible. > > The solution which Anton suggested does not look easy because it will most > likely significantly hurt performance: it is hard to maintain a data > structure to choose "page from free-list with enough space closest to the > beginning of the file". Overall, we can think of something similar to > postgres, when a space can be freed in some maintenance mode. > > Online space cleanup sounds tricky for me, or at least I cannot think about > a plausible solution right away. > > --AG > > пт, 13 сент. 2019 г. в 19:43, Denis Magda <dma...@apache.org>: > > > The issue starts hitting others who deploy Ignite persistence in > > production: > > https://issues.apache.org/jira/browse/IGNITE-12152 > > > > Alex, I'm curious is this a fundamental problem. Asked the same question > in > > JIRA but, probably, this discussion is a better place to get to the > bottom > > first: > > https://issues.apache.org/jira/browse/IGNITE-10862 > > > > - > > Denis > > > > > > On Thu, Jan 10, 2019 at 6:01 AM Anton Vinogradov <a...@apache.org> wrote: > > > > > Dmitriy, > > > > > > This does not look like a production-ready case :) > > > > > > How about > > > 1) Once you need to write an entry - you have to chose not random "page > > > from free-list with enough space" > > > but "page from free-list with enough space closest to the beginning of > > the > > > file". > > > > > > 2) Once you remove entry you have to merge the rest of the entries at > > this > > > page to the > > > "page from free-list with enough space closest to the beginning of the > > > file" > > > if possible. (optional) > > > > > > 3) Partition file tail with empty pages can bу removed at any time. > > > > > > 4) In case you have cold data inside the tail, just lock the page and > > > perform migration to > > > "page from free-list with enough space closest to the beginning of the > > > file". > > > This operation can be scheduled. > > > > > > On Wed, Jan 9, 2019 at 4:43 PM Dmitriy Pavlov <dpav...@apache.org> > > wrote: > > > > > > > In the TC Bot, I used to create the second cache with CacheV2 name > and > > > > migrate needed data from Cache V1 to V2. > > > > > > > > After CacheV1 destroy(), files are removed and disk space is freed. > > > > > > > > ср, 9 янв. 2019 г. в 12:04, Павлухин Иван <vololo...@gmail.com>: > > > > > > > > > Vyacheslav, > > > > > > > > > > Have you investigated how other vendors (Oracle, Postgres) tackle > > this > > > > > problem? > > > > > > > > > > I have one wild idea. Could the problem be solved by stopping a > node > > > > > which need to be defragmented, clearing persistence files and > > > > > restarting the node? After rebalance the node will receive all data > > > > > back without fragmentation. I see a big downside -- sending data > > > > > across the network. But perhaps we can play with affinity and start > > > > > new node on the same host which will receive the same data, after > > that > > > > > old node can be stopped. It looks more as kind of workaround but > > > > > perhaps it can be turned into workable solution. > > > > > > > > > > ср, 9 янв. 2019 г. в 10:49, Vyacheslav Daradur < > daradu...@gmail.com > > >: > > > > > > > > > > > > Yes, it's about Page Memory defragmentation. > > > > > > > > > > > > Pages in partitions files are stored sequentially, possible, it > > makes > > > > > > sense to defragment pages first to avoid interpages gaps since we > > use > > > > > > pages offset to manage them. > > > > > > > > > > > > I filled an issue [1], I hope we will be able to find resources > to > > > > > > solve the issue before 2.8 release. > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-10862 > > > > > > > > > > > > On Sat, Dec 29, 2018 at 10:47 AM Павлухин Иван < > > vololo...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > I suppose it is about Ignite Page Memory pages defragmentation. > > > > > > > > > > > > > > We can get 100 allocated pages each of which becomes only e.g. > > 50% > > > > > > > filled after removal some entries. But they will occupy a space > > for > > > > > > > 100 pages on a hard drive. > > > > > > > > > > > > > > пт, 28 дек. 2018 г. в 20:45, Denis Magda <dma...@apache.org>: > > > > > > > > > > > > > > > > Shouldn't the OS care of defragmentation? What we need to do > is > > > to > > > > > give a > > > > > > > > way to remove stale data and "release" the allocated space > > > somehow > > > > > through > > > > > > > > the tools, MBeans or API methods. > > > > > > > > > > > > > > > > -- > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Dec 28, 2018 at 6:24 AM Vladimir Ozerov < > > > > > voze...@gridgain.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi Vyacheslav, > > > > > > > > > > > > > > > > > > AFAIK this is not implemented. Shrinking/defragmentation is > > > > > important > > > > > > > > > optimization. Not only because it releases free space, but > > also > > > > > because it > > > > > > > > > decreases total number of pages. But is it not very easy to > > > > > implement, as > > > > > > > > > you have to both reshuffle data entries and index entries, > > > > > maintaining > > > > > > > > > consistency for concurrent reads and updates at the same > > time. > > > Or > > > > > > > > > alternatively we can think of offline defragmentation. It > > will > > > be > > > > > easier to > > > > > > > > > implement and faster, but concurrent operations will be > > > > prohibited. > > > > > > > > > > > > > > > > > > On Fri, Dec 28, 2018 at 4:08 PM Vyacheslav Daradur < > > > > > daradu...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Igniters, we have faced with the following problem on one > > of > > > > our > > > > > > > > > > deployments. > > > > > > > > > > > > > > > > > > > > Let's imagine that we have used IgniteCache with enabled > > PDS > > > > > during the > > > > > > > > > > time: > > > > > > > > > > - hardware disc space has been occupied during growing up > > of > > > an > > > > > amount > > > > > > > > > > of data, e.g. 100Gb; > > > > > > > > > > - then, we removed non-actual data, e.g 50Gb, which > became > > > > > useless for > > > > > > > > > us; > > > > > > > > > > - disc space stopped growing up with new data, but it was > > not > > > > > > > > > > released, and still took 100Gb, instead of expected 50Gb; > > > > > > > > > > > > > > > > > > > > Another use case: > > > > > > > > > > - a user extracts data from IgniteCache to store it in > > > separate > > > > > > > > > > IgniteCache or another store; > > > > > > > > > > - disc still is occupied and the user is not able to > store > > > data > > > > > in the > > > > > > > > > > different cache at the same cluster because of disc > > > limitation; > > > > > > > > > > > > > > > > > > > > How can we help the user to free up the disc space, if an > > > > amount > > > > > of > > > > > > > > > > data in IgniteCache has been reduced many times and will > > not > > > be > > > > > > > > > > increased in the nearest future? > > > > > > > > > > > > > > > > > > > > AFAIK, we have mechanics of reusing memory pages, that > > allows > > > > us > > > > > to > > > > > > > > > > use pages which have been allocated and stored removed > data > > > for > > > > > > > > > > storing new data. > > > > > > > > > > Are there any chances to shrink data and free up space on > > > disc > > > > > (with > > > > > > > > > > defragmentation if possible)? > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Best Regards, Vyacheslav D. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best regards, > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Best Regards, Vyacheslav D. > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > >
