Hello! I don't think that anything will get evicted with the configuration that you have provided.
I think you should check whether keys are really unique (yes I remember that you include currentTimeMillis in them, still it makes sense to double-check) and also that all values are of type Data. If some of them are not of type Data, SQL will not see them. Can you split your data into more batches (e.g. 10 batches, 20k records each), provide counts after every batch is ingested? Regards, -- Ilya Kasnacheev 2018-01-30 17:28 GMT+03:00 Raghav <[email protected]>: > Hi, > > I would like to add below points : > 1) Ignite YARN is started once [Server] and it will not be stopped between > iterations. This means that only once the Ignite nodes are negotiated > between YARN and Ignite. Once finalized this should be the same. > > Please find below the server logs. > [12:30:46] Topology snapshot [ver=1, servers=1, clients=0, CPUs=48, > heap=9.9GB] > [12:30:46] Topology snapshot [ver=2, servers=2, clients=0, CPUs=96, > heap=20.0GB] > [12:30:47] Topology snapshot [ver=3, servers=3, clients=0, CPUs=144, > heap=30.0GB] > [12:30:47] Topology snapshot [ver=4, servers=4, clients=0, CPUs=192, > heap=39.0GB] > [12:30:47] Topology snapshot [ver=5, servers=5, clients=0, CPUs=240, > heap=49.0GB] > [12:30:47] Topology snapshot [ver=6, servers=6, clients=0, CPUs=240, > heap=59.0GB] > [12:30:47] Topology snapshot [ver=7, servers=7, clients=0, CPUs=240, > heap=69.0GB] > [12:30:48] Topology snapshot [ver=8, servers=8, clients=0, CPUs=240, > heap=79.0GB] > [12:30:48] Topology snapshot [ver=9, servers=9, clients=0, CPUs=240, > heap=89.0GB] > [12:30:48] Topology snapshot [ver=10, servers=10, clients=0, CPUs=240, > heap=99.0GB] > [12:50:26] Topology snapshot [ver=11, servers=10, clients=1, CPUs=240, > heap=120.0GB] > [12:54:18] Topology snapshot [ver=12, servers=10, clients=0, CPUs=240, > heap=99.0GB] > [12:56:07] Topology snapshot [ver=13, servers=10, clients=1, CPUs=240, > heap=120.0GB] > [13:00:49] Topology snapshot [ver=14, servers=10, clients=0, CPUs=240, > heap=99.0GB] > [13:06:28] Topology snapshot [ver=15, servers=10, clients=1, CPUs=240, > heap=120.0GB] > [13:07:17] Topology snapshot [ver=16, servers=10, clients=0, CPUs=240, > heap=99.0GB] > > > 2) Only ignite clients are started and stopped in different Iterations. As > we could see the client count becomes 0 and 1 whereas the servers count > remain the same as 10 > > 3) /tmp is a HDFS path where we have configured for Ignite and provided in > cluster.properties. We could change this to any path. > > It would be helpful if there is a way to enable persistence in YARN > deployment. > > Thank you. > > Best Regards, > Raghav > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
