Hi Denis,
That's on the expiry policies page:
https://apacheignite.readme.io/docs/expiry-policies
On 13.09.2019 19:46, Denis Magda wrote:
Artem, thanks, could you please share a reference to the updated page?
Can't find anything here:
https://apacheignite.readme.io/docs/evictions
Shiva, I've restarted the discussion on the dev list, to get to the
bottom of this gap and how it can be addressed:
http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html
-
Denis
On Fri, Sep 13, 2019 at 7:05 AM Artem Budnikov
<a.budnikov.ign...@gmail.com <mailto:a.budnikov.ign...@gmail.com>> wrote:
Hi Denis,
I updated the page about eviction policies. Not freeing up space
on disk was not implemented for reasons explained in the dev-list
thread. I'll update the page once more if/when a solution is
implemented.
Artem
On 13.09.2019 00:34, Denis Magda wrote:
Shiva,
Hopefully, someone from the dev community will pick this ticket
up soon and solve the task. In the meantime, Artem, would you
mind documenting this limitation referring to ticket 10862?
-
Denis
On Tue, Sep 10, 2019 at 12:50 AM Shiva Kumar
<shivakumar....@gmail.com <mailto:shivakumar....@gmail.com>> wrote:
I have filed a bug
https://issues.apache.org/jira/browse/IGNITE-12152 but this
is same as https://issues.apache.org/jira/browse/IGNITE-10862
Any idea on the timeline of these tickets?
In the documentation
https://apacheignite.readme.io/v2.7/docs/expiry-policies
it says when native persistence is enabled "*expired entries
are removed from both memory and disk tiers*" but in the disk
it just mark the pages as unwanted pages and same disk space
used by these unwanted pages will be used to store new pages
but it will not remove unwanted pages from disk and so it
will not release disk space used by these unwanted pages.
here is the developer's discussion link
http://apache-ignite-developers.2346864.n4.nabble.com/How-to-free-up-space-on-disc-after-removing-entries-from-IgniteCache-with-enabled-PDS-td39839.html
On Mon, Sep 9, 2019 at 11:53 PM Shiva Kumar
<shivakumar....@gmail.com <mailto:shivakumar....@gmail.com>>
wrote:
Hi
I have deployed ignite on kubernetes and configured two
seperate persistent volume for WAL and persistence.
The issue Iam facing is same as
https://issues.apache.org/jira/browse/IGNITE-10862
Thanks
Shiva
On Mon, 9 Sep, 2019, 10:47 PM Andrei Aleksandrov,
<aealexsand...@gmail.com
<mailto:aealexsand...@gmail.com>> wrote:
Hello,
I guess that generated WAL will take this disk space.
Please read about WAL here:
https://apacheignite.readme.io/docs/write-ahead-log
Please provide the size of every folder under
/opt/ignite/persistence.
BR,
Andrei
9/6/2019 9:45 PM, Shiva Kumar пишет:
Hi all,
I have set cache expiry policy like this
</property>
<property name="cacheConfiguration">
<list>
<bean id="cache-template-bean"
abstract="true"
class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name"
value="templateEternal*"/>
<property name="cacheMode"
value="PARTITIONED"/>
<property name="backups" value="1"/>
<property name="groupName"
value="groupEternal"/>
<property name="expiryPolicyFactory">
<bean
class="javax.cache.expiry.CreatedExpiryPolicy"
factory-method="factoryOf">
<constructor-arg>
<bean
class="javax.cache.expiry.Duration">
<constructor-arg value="MINUTES"/>
<constructor-arg value="10"/>
</bean>
</constructor-arg>
</bean>
</property>
</bean>
</list>
</property>
And batch inserting records to one of the table
which is created with above cache template.
Around 10 minutes, I ingested ~1.5GB of data and
after 10 minutes records started reducing(expiring)
when I monitored from sqlline.
0: jdbc:ignite:thin://192.168.*.*:10800> select
count(ID) from DIMENSIONS;
--------------------------------
COUNT(ID)
--------------------------------
248896
--------------------------------
1 row selected (0.86 seconds)
0: jdbc:ignite:thin://192.168.*.*:10800> select
count(ID) from DIMENSIONS;
--------------------------------
COUNT(ID)
--------------------------------
222174
--------------------------------
1 row selected (0.313 seconds)
0: jdbc:ignite:thin://192.168.*.*:10800> select
count(ID) from DIMENSIONS;
--------------------------------
COUNT(ID)
--------------------------------
118154
--------------------------------
1 row selected (0.15 seconds)
0: jdbc:ignite:thin://192.168.*.*:10800>
0: jdbc:ignite:thin://192.168.*.*:10800> select
count(ID) from DIMENSIONS;
--------------------------------
COUNT(ID)
--------------------------------
76061
--------------------------------
1 row selected (0.106 seconds)
0: jdbc:ignite:thin://192.168.*.*:10800>
0: jdbc:ignite:thin://192.168.*.*:10800> select
count(ID) from DIMENSIONS;
--------------------------------
COUNT(ID)
--------------------------------
41671
--------------------------------
1 row selected (0.063 seconds)
0: jdbc:ignite:thin://192.168.*.*:10800> select
count(ID) from DIMENSIONS;
--------------------------------
COUNT(ID)
--------------------------------
18455
--------------------------------
1 row selected (0.037 seconds)
0: jdbc:ignite:thin://192.168.*.*:10800> select
count(ID) from DIMENSIONS;
--------------------------------
COUNT(ID)
--------------------------------
0
--------------------------------
1 row selected (0.014 seconds)
But in the meantime, the disk space used by the
persistence store was in the same usage level
instead of decreasing.
[ignite@ignite-cluster-ign-shiv-0 ignite]$ while
true ; do df -h /opt/ignite/persistence/; sleep 1s; done
Filesystem Size Used Avail Use% Mounted on
/dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence
Filesystem Size Used Avail Use% Mounted on
/dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence
Filesystem Size Used Avail Use% Mounted on
/dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence
Filesystem Size Used Avail Use% Mounted on
/dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence
Filesystem Size Used Avail Use% Mounted on
/dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence
Filesystem Size Used Avail Use% Mounted on
/dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence
Filesystem Size Used Avail Use% Mounted on
/dev/vdj 15G 1.6G 14G 11% /opt/ignite/persistence
This means that expiry policy not deleting records
from the disk, but ignite document says when expiry
policy is set and native persistence is enabled then
it deletes records from disk as well.
Am I missing some configuration?
Any help is appreciated.
Shiva
