Hi,
I've loaded 50 million BinaryObjects into TEST cache using Apache Spark
They look like this:
o.a.i.i.binary.BinaryObjectImpl | DATASET1 [hash=86282065, F01=-206809353,
F00=A1782096681-B2022047863-C554782990, F03=Must be timestamp,
F02=2.6983596317719E8, F05=182918247,
F04=A1997114384-B293944165-C1698394711, F07=Some string,
F06=-9.9171693326279E8, F09=47148799, F08=A1582165880-B1796903902-C784688220
BinaryObject name is DATASET1
I'm trying to get count of those objects using SQL:
String sqlStr = "SELECT COUNT(*) FROM TEST.DATASET1";
Ignite ignite = Ignition.start();
IgniteCache<String, BinaryObject> cache =
ignite.cache(cacheName);
logger.info("Starting SQL query : " + sqlStr);
logger.info("ALL Cache size is " +
cache.sizeLong(CachePeekMode.ALL));
logger.info("PRIMARY Cache size is " +
cache.sizeLong(CachePeekMode.PRIMARY));
logger.info("NEAR Cache size is " +
cache.sizeLong(CachePeekMode.NEAR));
logger.info("OFFHEAP Cache size is " +
cache.sizeLong(CachePeekMode.OFFHEAP));
logger.info("ONHEAP Cache size is " +
cache.sizeLong(CachePeekMode.ONHEAP));
SqlFieldsQuery sql = new SqlFieldsQuery(sqlStr).setLocal(false);
FieldsQueryCursor<List<?>> result = cache.query(sql);
printResults(result);
17/11/27 19:34:18 INFO IgnLoader: Starting SQL query : SELECT COUNT(*) FROM
TEST.DATASET1
17/11/27 19:34:23 INFO IgnLoader: ALL Cache size is 50000000
17/11/27 19:34:23 INFO IgnLoader: PRIMARY Cache size is 50000000
17/11/27 19:34:23 INFO IgnLoader: NEAR Cache size is 0
17/11/27 19:34:23 INFO IgnLoader: OFFHEAP Cache size is 50000000
17/11/27 19:34:23 INFO IgnLoader: ONHEAP Cache size is 5120
17/11/27 19:34:45 INFO IgnLoader: Result size : 1
row 0:[COUNT(*)(Long) :*9859833*; ]
17/11/27 19:34:45 INFO IgnLoader: Done
My function printResults reports only 9,859,833 records even if I use
setLocal(false)
What should I do to run Query against full data set?
Thank you for help.
Alexey
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/