Hi Christo,
I have tried the following:
- When app starts, I stream into PARTITIONED CACHE one (1) table needed for
the distributed jobs
- Now when a Task is submitted, each Job takes over some CATEGORYIDs.
Performs an Ignite cache-query (I paste it below), gathers the results, runs
calculations on them, transforms them to other objects and saves them to
some other smaller table in the Database. Finally it returns a success flag
true/false.
[13:29:09,520][WARN ][pub-#50%ppsp-cluster-IV%][IgniteH2Indexing] Query
execution is too long [time=5007 ms, sql='SELECT
"PPSP-IMDG-CACHE".SimulationInitialValues._KEY,
"PPSP-IMDG-CACHE".SimulationInitialValues._VAL FROM
"PPSP-IMDG-CACHE".SimulationInitialValues WHERE categoryId in ( 1, 2, 3, 4,
5, 6, 8 ) and geoChannelId in ( 3, 4, 5, 6 ) and type= 3 and week between
1888 and 1939', plan=
SELECT
"PPSP-IMDG-CACHE".SIMULATIONINITIALVALUES._KEY,
"PPSP-IMDG-CACHE".SIMULATIONINITIALVALUES._VAL
FROM "PPSP-IMDG-CACHE".SIMULATIONINITIALVALUES
/* "PPSP-IMDG-CACHE"."type_idx": TYPE = 3 */
WHERE ((WEEK >= 1888)
AND (WEEK <= 1939))
AND ((TYPE = 3)
AND ((CATEGORYID IN(1, 2, 3, 4, 5, 6, 8))
AND (GEOCHANNELID IN(3, 4, 5, 6))))
, parameters=[]]
Now, if I can align the CATEGORYIDs that each node has on its local cache
with the CATEGORYIDs that it receives along with a Job execution, then the
SQL query could be local-only. I have seen examples of affinityRun and
affinityCompute buf these are for distributed closures only. I would like to
do similar with ComputeTask approach because I need it in order to control
other things as well.
My *ComputeTask*is:
public class SimulationComputeTask extends
ComputeTaskSplitAdapter<SimulationMessage[], Boolean> {
...
}
*SimulationMessage *is:
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public class SimulationMessage implements Serializable {
private long projectId;
private long scenarioId;
private long[] categoryIds;
private long[] geoChannels;
private int channelType;
private int weekStart;
private int weekEnd;
}
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Distributed-Closures-VS-Executor-Service-tp11192p11742.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
