Guys,
Currently I'm looking into the problem how to properly deliver computation
to data in most efficient way.
Basically I need to iterate over all cache partitions on all grid nodes,
compute some function on each key-value pair and return aggregated result
to a caller.
This is a job for map-re
Alexey,
Have you taken a look at the Affinity API in Ignite? It seems that it has
all the functionality you may need to map partitions to nodes. You can take
that info, and use it to route your computations.
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/affinity/Affini
Dmitriy,
I know I could, but it requires too much work and messing with various APIs
for correct routing/failover.
What about automatic partition reservation for consitency? How could I
achieve what using only affinity API ?
I think this should be available out of the box, as for affinityRun/Cal
Alexei,
I think AffinityKeyMapped supposed to work for ComputeJobs and any closures
executed by IgniteCompute, but it seems there are no tests for this and
this functionalty is broken now, and I think this should be fixed.
Thanks,
Semyon
On Wed, Apr 19, 2017 at 9:59 PM, Alexei Scherbakov <
alexe
In this case, you should be using this API on IgniteCompute:
affinityRun(Collection cacheNames, int partId, IgniteRunnable job)
This will ensure that the partition is not migrated while the computation
is in progress. Why is this method insufficient?
As far as IgniteCompute deprecating the "with
Semyon,
This is exactly what I want.
But currently I don't undestand how affinity would be calculated without
reference to a cache (or caches).
I think something must be added to annotate such things.
I'll create a ticket for this improvement.
2017-04-19 22:04 GMT+03:00 Semyon Boikov :
> Ale
affinityRun will work only for single closure.
But I need to have the same functionality for all jobs in my compute task,
with support for task sessions and other benefits from map-reduce API.
2017-04-19 22:07 GMT+03:00 Dmitriy Setrakyan :
> In this case, you should be using this API on Ignite
I think the following should be done:
- AffinityKeyMapped annotation should be fixed if it's broken
(obviously).
- There should be a way to specify cache name in addition to key
(another annotation?).
- Semantics of a job or closure executed with this annotation should be
the sam