Re: Scaling with SQL query

[Tom M]

Hi Pavel,

 

Thank you for the reply. The cache is partitioned (with 3 copies).

 

[[SELECT
    __Z0.ID AS __C0_0,
    __Z0.CS AS __C0_1,
    __Z0.TIME AS __C0_2,
    __Z0.SID AS __C0_3,
    __Z0.SCITY AS __C0_4,
    __Z0.SADDRESS AS __C0_5,
    __Z0.IID AS __C0_6,
    __Z0.IURL AS __C0_7
FROM "Logs".LOGS __Z0
    /* "Logs".TIME_IDX: TIME > 0 */
WHERE (__Z0.TIME > 0)
ORDER BY 3 DESC
LIMIT ?1
/* index sorted */], [SELECT
    __C0_0 AS ID,
    __C0_1 AS CS,
    __C0_2 AS TIME,
    __C0_3 AS SID,
    __C0_4 AS SCITY,
    __C0_5 AS SADDRESS,
    __C0_6 AS IID,
    __C0_7 AS IURL
FROM PUBLIC.__T0
    /* "Logs"."merge_sorted" */
ORDER BY 3 DESC
LIMIT ?1
/* index sorted */]]

 

This is what I have (I have added "WHERE time > unix_epoch_time" to the original query here, changed to 0 above).

 

I thought that adding more nodes shouldn't introduce much overhead. How will this query be processed? Sending requests to each node, looking up for data at each node's index and returning the data to the "reducer" node? Any inter-node data exchange involved?

 

Sent: Wednesday, June 27, 2018 at 1:36 PM
From: "Pavel Vinokurov" <vinokurov.pa...@gmail.com>
To: user@ignite.apache.org
Subject: Re: Scaling with SQL query

Hi Tom,

 

In case of a replicated cache the Ignite plans the execution of the sql query across whole cluster by splitting into multiple map queries and a single reduce query.

Thus it is possible communication overheads caused by  that the "reduce" node collects data from multiple nodes.

Please show metrics for this query for you configuration.

 

Thanks,

Pavel

 

 

2018-06-26 6:24 GMT+03:00 Tom M <tar...@mail.com>:

Hi,

 

I have a cluster of 10 nodes, and a cache with replication factor 3 and no persistency enabled.

The SQL query is pretty simple -- "SELECT * FROM Logs ORDER by time DESC LIMIT 100".

I have checked the index for "time" attribute is applied.

 

When I increase the number of nodes, throughput drops and latency increases.

Can you please explain why and how Ignite processes this SQL request?

 

 

--

Regards

Pavel Vinokurov