Not sure how to check the GC log, but here's a minimal complete example using
two java classes:
Try these two queries.
SELECT COUNT(DISTINCT((idnumber,value))) FROM athing
SELECT COUNT (*) FROM (SELECT COUNT(*) FROM athing GROUP BY idnumber,value)
Both queries do the same thing, you'll find the latter takes about 10 times
more than the former, and it gets worse as you increase the number of records.
Maybe something O(x) vs O(x^2) is happening? What do you think? The real query
is way gnarlier, but I think I'm capturing the main thrust of it.
Mike
Startup Class:
package IgniteStartup;
import DataObjects.*;
import org.apache.ignite.*;
import org.apache.ignite.cache.CacheMode;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Random;
public class FireUp {
public static <T> boolean CreateCache(Ignite ignite, String cacheName,
CacheMode cm,Class<T> vclass, List<T> vals)
{
CacheConfiguration myCache = new CacheConfiguration<>(cacheName);
myCache.setSqlSchema("PUBLIC");
myCache.setCacheMode(cm);
if(cm==CacheMode.PARTITIONED)
{
myCache.setQueryParallelism(Runtime.getRuntime().availableProcessors()*2);
}
myCache.setIndexedTypes(Long.class,vclass);
try (IgniteCache<Long,T> cache = ignite.getOrCreateCache(myCache)) {
try (IgniteDataStreamer<Long, T> stmr =
ignite.dataStreamer(cacheName)) {
long i = 0;
for (T val : vals)
{
stmr.addData(i++, val);
}
stmr.flush();
}
}
return true;
}
public static void main(String[] args)
{
List<aThing> lThings = new ArrayList<>();
Random z = new Random(0);
Iterator<Integer> iStream= z.ints(0,10).iterator();
for(int i = 0; i < 1_000_000;++i)
{
lThings.add(new aThing(i,iStream.next()));
}
System.out.println("read in");
IgniteConfiguration cfg = new IgniteConfiguration();
Ignite ignite = Ignition.start(cfg);
CreateCache(ignite,"mapCache",CacheMode.PARTITIONED,
aThing.class,lThings);
System.out.println("launched");
}
}
aThing data object class:
package DataObjects;
import org.apache.ignite.cache.affinity.AffinityKeyMapped;
import org.apache.ignite.cache.query.annotations.QuerySqlField;
import java.io.Serializable;
public class aThing implements Serializable {
@AffinityKeyMapped
@QuerySqlField(index=true)
int IdNumber;
@QuerySqlField
double value;
public aThing(int id,double val)
{
IdNumber=id;
value = val;
}
}
-----Original Message-----
From: slava.koptilin [mailto:[email protected]]
Sent: Monday, February 26, 2018 12:07 PM
To: [email protected]
Subject: RE: Slow Group-By
Hi Mike,
Have you checked GC log? Have you seen long pauses?
Is it possible to share SQL query and corresponding execution plan [1]?
Also, please share cache configurations.
[1]
https://urldefense.proofpoint.com/v2/url?u=https-3A__apacheignite-2Dsql.readme.io_docs_performance-2Dand-2Ddebugging-23using-2Dexplain-2Dstatement&d=DwICAg&c=9g4MJkl2VjLjS6R4ei18BA&r=ipRRuqPnuP3BWnXGSOR_sLoARpltax56uFYU6n57c3GFvMdyEV-dz2ez2lZZpYl0&m=k6XxWa56JYcjlQAzPcbU2SWWL75Rg1Dtn4Z8pwFJaUQ&s=Xq6WB1XIW2MH6QETwNEeGJb4TAbjNbll6Fke9Auwvtg&e=
Thanks!
--
Sent from:
https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dignite-2Dusers.70518.x6.nabble.com_&d=DwICAg&c=9g4MJkl2VjLjS6R4ei18BA&r=ipRRuqPnuP3BWnXGSOR_sLoARpltax56uFYU6n57c3GFvMdyEV-dz2ez2lZZpYl0&m=k6XxWa56JYcjlQAzPcbU2SWWL75Rg1Dtn4Z8pwFJaUQ&s=Tp0ZzfXbunNjEVy2zziCsOJlLCOlx1_GB2fVRVU7Zo8&e=
