Shiva, Hopefully, someone from the dev community will pick this ticket up soon and solve the task. In the meantime, Artem, would you mind documenting this limitation referring to ticket 10862?
- Denis On Tue, Sep 10, 2019 at 12:50 AM Shiva Kumar <shivakumar....@gmail.com> wrote: > I have filed a bug https://issues.apache.org/jira/browse/IGNITE-12152 but > this is same as https://issues.apache.org/jira/browse/IGNITE-10862 > Any idea on the timeline of these tickets? > In the documentation > https://apacheignite.readme.io/v2.7/docs/expiry-policies > it says when native persistence is enabled "*expired entries are removed > from both memory and disk tiers*" but in the disk it just mark the pages > as unwanted pages and same disk space used by these unwanted pages will be > used to store new pages but it will not remove unwanted pages from disk and > so it will not release disk space used by these unwanted pages. > > here is the developer's discussion link > > http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html > > > On Mon, Sep 9, 2019 at 11:53 PM Shiva Kumar <shivakumar....@gmail.com> > wrote: > >> Hi >> I have deployed ignite on kubernetes and configured two seperate >> persistent volume for WAL and persistence. >> The issue Iam facing is same as >> https://issues.apache.org/jira/browse/IGNITE-10862 >> >> Thanks >> Shiva >> >> On Mon, 9 Sep, 2019, 10:47 PM Andrei Aleksandrov, < >> aealexsand...@gmail.com> wrote: >> >>> Hello, >>> >>> I guess that generated WAL will take this disk space. Please read about >>> WAL here: >>> >>> https://apacheignite.readme.io/docs/write-ahead-log >>> >>> Please provide the size of every folder under /opt/ignite/persistence. >>> >>> BR, >>> Andrei >>> 9/6/2019 9:45 PM, Shiva Kumar пишет: >>> >>> Hi all, >>> I have set cache expiry policy like this >>> >>> >>> </property> >>> <property name="cacheConfiguration"> >>> <list> >>> <bean id="cache-template-bean" abstract="true" >>> class="org.apache.ignite.configuration.CacheConfiguration"> >>> <property name="name" value="templateEternal*"/> >>> <property name="cacheMode" value="PARTITIONED"/> >>> <property name="backups" value="1"/> >>> <property name="groupName" value="groupEternal"/> >>> <property name="expiryPolicyFactory"> >>> <bean class="javax.cache.expiry.CreatedExpiryPolicy" >>> factory-method="factoryOf"> >>> <constructor-arg> >>> <bean class="javax.cache.expiry.Duration"> >>> <constructor-arg value="MINUTES"/> >>> <constructor-arg value="10"/> >>> </bean> >>> </constructor-arg> >>> </bean> >>> </property> >>> >>> </bean> >>> </list> >>> </property> >>> >>> >>> And batch inserting records to one of the table which is created with >>> above cache template. >>> Around 10 minutes, I ingested ~1.5GB of data and after 10 minutes >>> records started reducing(expiring) when I monitored from sqlline. >>> >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 248896 >>> -------------------------------- >>> 1 row selected (0.86 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 222174 >>> -------------------------------- >>> 1 row selected (0.313 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 118154 >>> -------------------------------- >>> 1 row selected (0.15 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 76061 >>> -------------------------------- >>> 1 row selected (0.106 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 41671 >>> -------------------------------- >>> 1 row selected (0.063 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 18455 >>> -------------------------------- >>> 1 row selected (0.037 seconds) >>> 0: jdbc:ignite:thin://192.168.*.*:10800> select count(ID) from >>> DIMENSIONS; >>> -------------------------------- >>> >>> COUNT(ID) >>> -------------------------------- >>> >>> 0 >>> -------------------------------- >>> 1 row selected (0.014 seconds) >>> >>> >>> But in the meantime, the disk space used by the persistence store was in >>> the same usage level instead of decreasing. >>> >>> >>> [ignite@ignite-cluster-ign-shiv-0 ignite]$ while true ; do df -h >>> /opt/ignite/persistence/; sleep 1s; done >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> Filesystem Size Used Avail Use% Mounted on >>> /dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence >>> >>> >>> >>> This means that expiry policy not deleting records from the disk, but >>> ignite document says when expiry policy is set and native persistence is >>> enabled then it deletes records from disk as well. >>> Am I missing some configuration? >>> Any help is appreciated. >>> >>> Shiva >>> >>>
