[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2020-07-03 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-640:
-
Fix Version/s: (was: 2.9)

> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>Priority: Major
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets all values for a collection of keys at a certain index.
> Map getAll(Collection, index);
> // Gets values for specified indexes for a key.
> List get(K, Iterable indexes);
> // Gets all values for a collection of keys at specified indexes.
> Map> getAll(Collection, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> List get(K, int min, int max);
> // Gets all values for a collection of keys for a specified index range, 
> between min and max.
> Map> getAll(Collection, int min, int max);
> // Gets all values for a specific key.
> List get(K);
> // Gets all values for a collection of keys.
> Map> getAll(Collection);
> // Iterate through all elements with a certain index.
> Iterator> iterate(int idx);
> // Do we need this?
> Collection> get(K, IgniteBiPredicate)
> {code}
> Multimap should also support colocated and non-colocated modes, similar to 
> [IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
>  and its implementation, 
> [GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
> h2. Design Details
> The most natural way to implement such map, would be to store every value 
> under a separate key in an Ignite cache. For example, let's say that we have 
> a key {{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should 
> end up with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. 
> This means that we need to wrap user key into our own, internal key, which 
> will also have {{index}} field. 
> Also note that we need to collocate all the values for the same key on the 
> same node, which means that we need to define user key K as the affinity key, 
> like so:
> {code}
> class MultiKey {
> @CacheAffinityMapped
> private K key;
> int index;
> }
> {code}
> Look ups of values at specific indexes becomes very simple. Just attach a 
> specific index to a key and do a cache lookup. Look ups for all values for a 
> key should work as following:
> {code}
> MultiKey key;
> V v = null;
> int index = 0;
> List res = new LinkedList<>();
> do {
> v = cache.get(MultiKey(K, index));
> if (v != null)
> res.add(v);
> index++;
> }
> while (v != null);
> return res;
> {code}
> We could also use batching for performance reason. In this case the batch 
> size should be configurable.
> {code}
> int index = 0;
> List res = new LinkedList<>();
> while (true) {
> List batch = new ArrayList<>(batchSize);
> // Populate batch.
> for (; index < batchSize; index++)
> batch.add(new MultiKey(K, index % batchSize);
> Map batchRes = cache.getAll(batch);
> // Potentially need to properly sort values, based on the key order,
> // if the returning map does not do it automatically.
> res.addAll(batchRes.values());
> if (res.size() < batch.size())
> break;
> }
> return res;
> {code}
> h2. Evictions
> Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
> keys, and maximum number of values for a key. The maximum number of keys 
> should be controlled by Ignite standard eviction policy. The maximum number 
> of values for a key should be controlled by the implementation of the 
> multi-map. Either eviction parameter should be configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2019-10-08 Thread Maxim Muzafarov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maxim Muzafarov updated IGNITE-640:
---
Fix Version/s: (was: 2.8)
   2.9

> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>Assignee: Amir Akhmedov
>Priority: Major
> Fix For: 2.9
>
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets all values for a collection of keys at a certain index.
> Map getAll(Collection, index);
> // Gets values for specified indexes for a key.
> List get(K, Iterable indexes);
> // Gets all values for a collection of keys at specified indexes.
> Map> getAll(Collection, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> List get(K, int min, int max);
> // Gets all values for a collection of keys for a specified index range, 
> between min and max.
> Map> getAll(Collection, int min, int max);
> // Gets all values for a specific key.
> List get(K);
> // Gets all values for a collection of keys.
> Map> getAll(Collection);
> // Iterate through all elements with a certain index.
> Iterator> iterate(int idx);
> // Do we need this?
> Collection> get(K, IgniteBiPredicate)
> {code}
> Multimap should also support colocated and non-colocated modes, similar to 
> [IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
>  and its implementation, 
> [GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
> h2. Design Details
> The most natural way to implement such map, would be to store every value 
> under a separate key in an Ignite cache. For example, let's say that we have 
> a key {{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should 
> end up with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. 
> This means that we need to wrap user key into our own, internal key, which 
> will also have {{index}} field. 
> Also note that we need to collocate all the values for the same key on the 
> same node, which means that we need to define user key K as the affinity key, 
> like so:
> {code}
> class MultiKey {
> @CacheAffinityMapped
> private K key;
> int index;
> }
> {code}
> Look ups of values at specific indexes becomes very simple. Just attach a 
> specific index to a key and do a cache lookup. Look ups for all values for a 
> key should work as following:
> {code}
> MultiKey key;
> V v = null;
> int index = 0;
> List res = new LinkedList<>();
> do {
> v = cache.get(MultiKey(K, index));
> if (v != null)
> res.add(v);
> index++;
> }
> while (v != null);
> return res;
> {code}
> We could also use batching for performance reason. In this case the batch 
> size should be configurable.
> {code}
> int index = 0;
> List res = new LinkedList<>();
> while (true) {
> List batch = new ArrayList<>(batchSize);
> // Populate batch.
> for (; index < batchSize; index++)
> batch.add(new MultiKey(K, index % batchSize);
> Map batchRes = cache.getAll(batch);
> // Potentially need to properly sort values, based on the key order,
> // if the returning map does not do it automatically.
> res.addAll(batchRes.values());
> if (res.size() < batch.size())
> break;
> }
> return res;
> {code}
> h2. Evictions
> Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
> keys, and maximum number of values for a key. The maximum number of keys 
> should be controlled by Ignite standard eviction policy. The maximum number 
> of values for a key should be controlled by the implementation of the 
> multi-map. Either eviction parameter should be configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2018-09-24 Thread Amir Akhmedov (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amir Akhmedov updated IGNITE-640:
-
Fix Version/s: (was: 2.7)
   2.8

> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>Assignee: Amir Akhmedov
>Priority: Major
> Fix For: 2.8
>
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets all values for a collection of keys at a certain index.
> Map getAll(Collection, index);
> // Gets values for specified indexes for a key.
> List get(K, Iterable indexes);
> // Gets all values for a collection of keys at specified indexes.
> Map> getAll(Collection, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> List get(K, int min, int max);
> // Gets all values for a collection of keys for a specified index range, 
> between min and max.
> Map> getAll(Collection, int min, int max);
> // Gets all values for a specific key.
> List get(K);
> // Gets all values for a collection of keys.
> Map> getAll(Collection);
> // Iterate through all elements with a certain index.
> Iterator> iterate(int idx);
> // Do we need this?
> Collection> get(K, IgniteBiPredicate)
> {code}
> Multimap should also support colocated and non-colocated modes, similar to 
> [IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
>  and its implementation, 
> [GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
> h2. Design Details
> The most natural way to implement such map, would be to store every value 
> under a separate key in an Ignite cache. For example, let's say that we have 
> a key {{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should 
> end up with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. 
> This means that we need to wrap user key into our own, internal key, which 
> will also have {{index}} field. 
> Also note that we need to collocate all the values for the same key on the 
> same node, which means that we need to define user key K as the affinity key, 
> like so:
> {code}
> class MultiKey {
> @CacheAffinityMapped
> private K key;
> int index;
> }
> {code}
> Look ups of values at specific indexes becomes very simple. Just attach a 
> specific index to a key and do a cache lookup. Look ups for all values for a 
> key should work as following:
> {code}
> MultiKey key;
> V v = null;
> int index = 0;
> List res = new LinkedList<>();
> do {
> v = cache.get(MultiKey(K, index));
> if (v != null)
> res.add(v);
> index++;
> }
> while (v != null);
> return res;
> {code}
> We could also use batching for performance reason. In this case the batch 
> size should be configurable.
> {code}
> int index = 0;
> List res = new LinkedList<>();
> while (true) {
> List batch = new ArrayList<>(batchSize);
> // Populate batch.
> for (; index < batchSize; index++)
> batch.add(new MultiKey(K, index % batchSize);
> Map batchRes = cache.getAll(batch);
> // Potentially need to properly sort values, based on the key order,
> // if the returning map does not do it automatically.
> res.addAll(batchRes.values());
> if (res.size() < batch.size())
> break;
> }
> return res;
> {code}
> h2. Evictions
> Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
> keys, and maximum number of values for a key. The maximum number of keys 
> should be controlled by Ignite standard eviction policy. The maximum number 
> of values for a key should be controlled by the implementation of the 
> multi-map. Either eviction parameter should be configurable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2018-06-26 Thread Dmitriy Pavlov (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Pavlov updated IGNITE-640:
--
Fix Version/s: (was: 2.6)
   2.7

> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>Assignee: Amir Akhmedov
>Priority: Major
> Fix For: 2.7
>
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets all values for a collection of keys at a certain index.
> Map getAll(Collection, index);
> // Gets values for specified indexes for a key.
> List get(K, Iterable indexes);
> // Gets all values for a collection of keys at specified indexes.
> Map> getAll(Collection, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> List get(K, int min, int max);
> // Gets all values for a collection of keys for a specified index range, 
> between min and max.
> Map> getAll(Collection, int min, int max);
> // Gets all values for a specific key.
> List get(K);
> // Gets all values for a collection of keys.
> Map> getAll(Collection);
> // Iterate through all elements with a certain index.
> Iterator> iterate(int idx);
> // Do we need this?
> Collection> get(K, IgniteBiPredicate)
> {code}
> Multimap should also support colocated and non-colocated modes, similar to 
> [IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
>  and its implementation, 
> [GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
> h2. Design Details
> The most natural way to implement such map, would be to store every value 
> under a separate key in an Ignite cache. For example, let's say that we have 
> a key {{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should 
> end up with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. 
> This means that we need to wrap user key into our own, internal key, which 
> will also have {{index}} field. 
> Also note that we need to collocate all the values for the same key on the 
> same node, which means that we need to define user key K as the affinity key, 
> like so:
> {code}
> class MultiKey {
> @CacheAffinityMapped
> private K key;
> int index;
> }
> {code}
> Look ups of values at specific indexes becomes very simple. Just attach a 
> specific index to a key and do a cache lookup. Look ups for all values for a 
> key should work as following:
> {code}
> MultiKey key;
> V v = null;
> int index = 0;
> List res = new LinkedList<>();
> do {
> v = cache.get(MultiKey(K, index));
> if (v != null)
> res.add(v);
> index++;
> }
> while (v != null);
> return res;
> {code}
> We could also use batching for performance reason. In this case the batch 
> size should be configurable.
> {code}
> int index = 0;
> List res = new LinkedList<>();
> while (true) {
> List batch = new ArrayList<>(batchSize);
> // Populate batch.
> for (; index < batchSize; index++)
> batch.add(new MultiKey(K, index % batchSize);
> Map batchRes = cache.getAll(batch);
> // Potentially need to properly sort values, based on the key order,
> // if the returning map does not do it automatically.
> res.addAll(batchRes.values());
> if (res.size() < batch.size())
> break;
> }
> return res;
> {code}
> h2. Evictions
> Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
> keys, and maximum number of values for a key. The maximum number of keys 
> should be controlled by Ignite standard eviction policy. The maximum number 
> of values for a key should be controlled by the implementation of the 
> multi-map. Either eviction parameter should be configurable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2018-04-18 Thread Amir Akhmedov (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amir Akhmedov updated IGNITE-640:
-
Fix Version/s: 2.6

> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>Assignee: Amir Akhmedov
>Priority: Major
> Fix For: 2.6
>
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets all values for a collection of keys at a certain index.
> Map getAll(Collection, index);
> // Gets values for specified indexes for a key.
> List get(K, Iterable indexes);
> // Gets all values for a collection of keys at specified indexes.
> Map getAll(Collection, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> List get(K, int min, int max);
> // Gets all values for a collection of keys for a specified index range, 
> between min and max.
> Map getAll(Collection, int min, int max);
> // Gets all values for a specific key.
> List get(K);
> // Gets all values for a collection of keys.
> Map getAll(Collection);
> // Iterate through all elements with a certain index.
> Iterator> iterate(int idx);
> // Do we need this?
> Collection get(K, IgniteBiPredicate)
> {code}
> Multimap should also support colocated and non-colocated modes, similar to 
> [IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
>  and its implementation, 
> [GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
> h2. Design Details
> The most natural way to implement such map, would be to store every value 
> under a separate key in an Ignite cache. For example, let's say that we have 
> a key {{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should 
> end up with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. 
> This means that we need to wrap user key into our own, internal key, which 
> will also have {{index}} field. 
> Also note that we need to collocate all the values for the same key on the 
> same node, which means that we need to define user key K as the affinity key, 
> like so:
> {code}
> class MultiKey {
> @CacheAffinityMapped
> private K key;
> int index;
> }
> {code}
> Look ups of values at specific indexes becomes very simple. Just attach a 
> specific index to a key and do a cache lookup. Look ups for all values for a 
> key should work as following:
> {code}
> MultiKey key;
> V v = null;
> int index = 0;
> List res = new LinkedList<>();
> do {
> v = cache.get(MultiKey(K, index));
> if (v != null)
> res.add(v);
> index++;
> }
> while (v != null);
> return res;
> {code}
> We could also use batching for performance reason. In this case the batch 
> size should be configurable.
> {code}
> int index = 0;
> List res = new LinkedList<>();
> while (true) {
> List batch = new ArrayList<>(batchSize);
> // Populate batch.
> for (; index < batchSize; index++)
> batch.add(new MultiKey(K, index % batchSize);
> Map batchRes = cache.getAll(batch);
> // Potentially need to properly sort values, based on the key order,
> // if the returning map does not do it automatically.
> res.addAll(batchRes.values());
> if (res.size() < batch.size())
> break;
> }
> return res;
> {code}
> h2. Evictions
> Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
> keys, and maximum number of values for a key. The maximum number of keys 
> should be controlled by Ignite standard eviction policy. The maximum number 
> of values for a key should be controlled by the implementation of the 
> multi-map. Either eviction parameter should be configurable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2016-03-31 Thread Valentin Kulichenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Valentin Kulichenko updated IGNITE-640:
---
Assignee: (was: Konstantin Margorin)

> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets all values for a collection of keys at a certain index.
> Map getAll(Collection, index);
> // Gets values for specified indexes for a key.
> List get(K, Iterable indexes);
> // Gets all values for a collection of keys at specified indexes.
> Map getAll(Collection, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> List get(K, int min, int max);
> // Gets all values for a collection of keys for a specified index range, 
> between min and max.
> Map getAll(Collection, int min, int max);
> // Gets all values for a specific key.
> List get(K);
> // Gets all values for a collection of keys.
> Map getAll(Collection);
> // Iterate through all elements with a certain index.
> Iterator> iterate(int idx);
> // Do we need this?
> Collection get(K, IgniteBiPredicate)
> {code}
> Multimap should also support colocated and non-colocated modes, similar to 
> [IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
>  and its implementation, 
> [GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
> h2. Design Details
> The most natural way to implement such map, would be to store every value 
> under a separate key in an Ignite cache. For example, let's say that we have 
> a key {{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should 
> end up with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. 
> This means that we need to wrap user key into our own, internal key, which 
> will also have {{index}} field. 
> Also note that we need to collocate all the values for the same key on the 
> same node, which means that we need to define user key K as the affinity key, 
> like so:
> {code}
> class MultiKey {
> @CacheAffinityMapped
> private K key;
> int index;
> }
> {code}
> Look ups of values at specific indexes becomes very simple. Just attach a 
> specific index to a key and do a cache lookup. Look ups for all values for a 
> key should work as following:
> {code}
> MultiKey key;
> V v = null;
> int index = 0;
> List res = new LinkedList<>();
> do {
> v = cache.get(MultiKey(K, index));
> if (v != null)
> res.add(v);
> index++;
> }
> while (v != null);
> return res;
> {code}
> We could also use batching for performance reason. In this case the batch 
> size should be configurable.
> {code}
> int index = 0;
> List res = new LinkedList<>();
> while (true) {
> List batch = new ArrayList<>(batchSize);
> // Populate batch.
> for (; index < batchSize; index++)
> batch.add(new MultiKey(K, index % batchSize);
> Map batchRes = cache.getAll(batch);
> // Potentially need to properly sort values, based on the key order,
> // if the returning map does not do it automatically.
> res.addAll(batchRes.values());
> if (res.size() < batch.size())
> break;
> }
> return res;
> {code}
> h2. Evictions
> Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
> keys, and maximum number of values for a key. The maximum number of keys 
> should be controlled by Ignite standard eviction policy. The maximum number 
> of values for a key should be controlled by the implementation of the 
> multi-map. Either eviction parameter should be configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2016-02-23 Thread Dmitriy Setrakyan (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Setrakyan updated IGNITE-640:
-
Description: 
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets all values for a collection of keys at a certain index.
Map getAll(Collection, index);

// Gets values for specified indexes for a key.
List get(K, Iterable indexes);

// Gets all values for a collection of keys at specified indexes.
Map getAll(Collection, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
List get(K, int min, int max);

// Gets all values for a collection of keys for a specified index range, 
between min and max.
Map getAll(Collection, int min, int max);

// Gets all values for a specific key.
List get(K);

// Gets all values for a collection of keys.
Map getAll(Collection);

// Iterate through all elements with a certain index.
Iterator> iterate(int idx);

// Do we need this?
Collection get(K, IgniteBiPredicate)
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
h2. Design Details
The most natural way to implement such map, would be to store every value under 
a separate key in an Ignite cache. For example, let's say that we have a key 
{{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should end up 
with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. This means 
that we need to wrap user key into our own, internal key, which will also have 
{{index}} field. 

Also note that we need to collocate all the values for the same key on the same 
node, which means that we need to define user key K as the affinity key, like 
so:
{code}
class MultiKey {
@CacheAffinityMapped
private K key;

int index;
}
{code}
Look ups of values at specific indexes becomes very simple. Just attach a 
specific index to a key and do a cache lookup. Look ups for all values for a 
key should work as following:
{code}
MultiKey key;
V v = null;
int index = 0;

List res = new LinkedList<>();

do {
v = cache.get(MultiKey(K, index));

if (v != null)
res.add(v);

index++;
}
while (v != null);

return res;
{code}
We could also use batching for performance reason. In this case the batch size 
should be configurable.
{code}
int index = 0;

List res = new LinkedList<>();

while (true) {
List batch = new ArrayList<>(batchSize);

// Populate batch.
for (; index < batchSize; index++)
batch.add(new MultiKey(K, index % batchSize);

Map batchRes = cache.getAll(batch);

// Potentially need to properly sort values, based on the key order,
// if the returning map does not do it automatically.
res.addAll(batchRes.values());

if (res.size() < batch.size())
break;
}

return res;
{code}
h2. Evictions
Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
keys, and maximum number of values for a key. The maximum number of keys should 
be controlled by Ignite standard eviction policy. The maximum number of values 
for a key should be controlled by the implementation of the multi-map. Either 
eviction parameter should be configurable.

  was:
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets all values for a collection of keys at a certain index.
Map getAll(Collection, index);

// Gets values for specified indexes for a key.
List get(K, Iterable indexes);

// Gets all values 

[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2015-11-02 Thread Dmitriy Setrakyan (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Setrakyan updated IGNITE-640:
-
Description: 
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets values for specified indexes for a key.
Iterable get(K, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Iterable get(K, int min, int max);
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].


  was:
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html]

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].



> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>Assignee: Vladisav Jelisavcic
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets values for specified indexes for a key.
> Iterable get(K, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> Iterable get(K, int min, int max);
> {code}
> Multimap should also support colocated and non-colocated modes, similar to 
> [IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
>  and its implementation, 
> [GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2015-11-02 Thread Dmitriy Setrakyan (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Setrakyan updated IGNITE-640:
-
Description: 
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets values for specified indexes for a key.
Iterable get(K, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Iterable get(K, int min, int max);

// Gets all values for a specific key.
List get(K);
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].

h2. Design Details
The most natural way to implement such map, would be to store every value under 
a separate key in an Ignite cache. For example, let's say that we have a key 
{{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should end up 
with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. This means 
that we need to wrap user key into our own, internal key, which will also have 
{{index}} field.

Look ups of values at specific indexes becomes very simple. Just attach a 
specific index to a key and do a cache lookup. Look ups for all values for a 
key should work as following:
{code}
Object key;
V v = null;
int index = 0;

List res = new ArrayList<>();

do {
v = cache.get(Key(K, index));

if (v != null)
res.add(v);
}
while (v != null);

return res;
{code}

  was:
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets values for specified indexes for a key.
Iterable get(K, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Iterable get(K, int min, int max);
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].

h2. Design Details
The most natural way to implement such map, would be to store every value under 
a separate key in an Ignite cache. For example, let's say that we have a key 
{{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should end up 
with the following values {{K.0, V0}}



> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>Assignee: Vladisav Jelisavcic
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets values for specified indexes for a key.
> Iterable get(K, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> Iterable get(K, int min, int max);
> // Gets 

[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2015-11-02 Thread Dmitriy Setrakyan (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Setrakyan updated IGNITE-640:
-
Description: 
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets values for specified indexes for a key.
Iterable get(K, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Iterable get(K, int min, int max);
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].

h2. Design Details
The most natural way to implement such map, would be to store every value under 
a separate key in an Ignite cache. For example, let's say that we have a key 
{{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should end up 
with the following values {{K.0, V0}}


  was:
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets values for specified indexes for a key.
Iterable get(K, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Iterable get(K, int min, int max);
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].



> Implement IgniteMultimap data structures
> 
>
> Key: IGNITE-640
> URL: https://issues.apache.org/jira/browse/IGNITE-640
> Project: Ignite
>  Issue Type: Sub-task
>  Components: data structures
>Reporter: Dmitriy Setrakyan
>Assignee: Vladisav Jelisavcic
>
> We need to add {{IgniteMultimap}} data structure in addition to other data 
> structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
> {{java.util.Map}} class in JDK, but support the semantics of multiple values 
> per key, similar to [Guava 
> Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
>  
> However, unlike in Guava, our multi-map should work with Lists, not 
> Collections. Lists should make it possible to support the following methods:
> {code}
> // Gets value at a certain index for a key.
> V get(K, index);
> // Gets values for specified indexes for a key.
> Iterable get(K, Iterable indexes);
> // Gets values for specified range of indexes, between min and max.
> Iterable get(K, int min, int max);
> {code}
> Multimap should also support colocated and non-colocated modes, similar to 
> [IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
>  and its implementation, 
> [GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
> h2. Design Details
> The most natural way to implement such map, would be to store every value 
> under a separate key in an Ignite cache. For example, let's say that we have 
> a key {{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should 
> end up with the following values {{K.0, V0}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2015-11-02 Thread Dmitriy Setrakyan (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Setrakyan updated IGNITE-640:
-
Description: 
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets values for specified indexes for a key.
Collection get(K, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Collection get(K, int min, int max);

// Gets all values for a specific key.
Collection get(K);
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
h2. Design Details
The most natural way to implement such map, would be to store every value under 
a separate key in an Ignite cache. For example, let's say that we have a key 
{{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should end up 
with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. This means 
that we need to wrap user key into our own, internal key, which will also have 
{{index}} field. 

Also note that we need to collocate all the values for the same key on the same 
node, which means that we need to define user key K as the affinity key, like 
so:
{code}
class MultiKey {
@CacheAffinityMapped
private K key;

int index;
}
{code}
Look ups of values at specific indexes becomes very simple. Just attach a 
specific index to a key and do a cache lookup. Look ups for all values for a 
key should work as following:
{code}
MultiKey key;
V v = null;
int index = 0;

List res = new LinkedList<>();

do {
v = cache.get(MultiKey(K, index));

if (v != null)
res.add(v);

index++;
}
while (v != null);

return res;
{code}
We could also use batching for performance reason. In this case the batch size 
should be configurable.
{code}
int index = 0;

List res = new LinkedList<>();

while (true) {
List batch = new ArrayList<>(batchSize);

// Populate batch.
for (; index < batchSize; index++)
batch.add(new MultiKey(K, index % batchSize);

Map batchRes = cache.getAll(batch);

// Potentially need to properly sort values, based on the key order,
// if the returning map does not do it automatically.
res.addAll(batchRes.values());

if (res.size() < batch.size())
break;
}

return res;
{code}
h2. Evictions
Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
keys, and maximum number of values for a key. The maximum number of keys should 
be controlled by Ignite standard eviction policy. The maximum number of values 
for a key should be controlled by the implementation of the multi-map. Either 
eviction parameter should be configurable.

  was:
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets values for specified indexes for a key.
Collection get(K, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Collection get(K, int min, int max);

// Gets all values for a specific key.
Collection get(K);
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
h2. Design Details
The most natural way to implement such map, would be to store every value under 
a separate key in an Ignite cache. For 

[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2015-11-02 Thread Dmitriy Setrakyan (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Setrakyan updated IGNITE-640:
-
Description: 
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets all values for a collection of keys at a certain index.
Map getAll(Collection, index);

// Gets values for specified indexes for a key.
Collection get(K, Iterable indexes);

// Gets all values for a collection of keys at specified indexes.
Map getAll(Collection, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Collection get(K, int min, int max);

// Gets all values for a collection of keys for a specified index range, 
between min and max.
Map getAll(Collection, int min, int max);

// Gets all values for a specific key.
Collection get(K);

// Gets all values for a collection of keys.
Map getAll(Collection);
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
h2. Design Details
The most natural way to implement such map, would be to store every value under 
a separate key in an Ignite cache. For example, let's say that we have a key 
{{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should end up 
with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. This means 
that we need to wrap user key into our own, internal key, which will also have 
{{index}} field. 

Also note that we need to collocate all the values for the same key on the same 
node, which means that we need to define user key K as the affinity key, like 
so:
{code}
class MultiKey {
@CacheAffinityMapped
private K key;

int index;
}
{code}
Look ups of values at specific indexes becomes very simple. Just attach a 
specific index to a key and do a cache lookup. Look ups for all values for a 
key should work as following:
{code}
MultiKey key;
V v = null;
int index = 0;

List res = new LinkedList<>();

do {
v = cache.get(MultiKey(K, index));

if (v != null)
res.add(v);

index++;
}
while (v != null);

return res;
{code}
We could also use batching for performance reason. In this case the batch size 
should be configurable.
{code}
int index = 0;

List res = new LinkedList<>();

while (true) {
List batch = new ArrayList<>(batchSize);

// Populate batch.
for (; index < batchSize; index++)
batch.add(new MultiKey(K, index % batchSize);

Map batchRes = cache.getAll(batch);

// Potentially need to properly sort values, based on the key order,
// if the returning map does not do it automatically.
res.addAll(batchRes.values());

if (res.size() < batch.size())
break;
}

return res;
{code}
h2. Evictions
Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
keys, and maximum number of values for a key. The maximum number of keys should 
be controlled by Ignite standard eviction policy. The maximum number of values 
for a key should be controlled by the implementation of the multi-map. Either 
eviction parameter should be configurable.

  was:
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets values for specified indexes for a key.
Collection get(K, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Collection get(K, int min, int max);

// Gets all values for a specific key.
Collection get(K);
{code}

Multimap should also support colocated and non-colocated modes, similar to 

[jira] [Updated] (IGNITE-640) Implement IgniteMultimap data structures

2015-11-02 Thread Dmitriy Setrakyan (JIRA)

 [ 
https://issues.apache.org/jira/browse/IGNITE-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Setrakyan updated IGNITE-640:
-
Description: 
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets values for specified indexes for a key.
Collection get(K, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Collection get(K, int min, int max);

// Gets all values for a specific key.
Collection get(K);
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
h2. Design Details
The most natural way to implement such map, would be to store every value under 
a separate key in an Ignite cache. For example, let's say that we have a key 
{{K}} with multiple values: {{V0, V1, V2, ...}}. Then the cache should end up 
with the following values {{K0, V0}}, {{K1, V1}}, {{K2, V2}}, etc. This means 
that we need to wrap user key into our own, internal key, which will also have 
{{index}} field. 

Also note that we need to collocate all the values for the same key on the same 
node, which means that we need to define user key K as the affinity key, like 
so:
{code}
class MultiKey {
@CacheAffinityMapped
private K key;

int index;
}
{code}
Look ups of values at specific indexes becomes very simple. Just attach a 
specific index to a key and do a cache lookup. Look ups for all values for a 
key should work as following:
{code}
MultiKey key;
V v = null;
int index = 0;

List res = new LinkedList<>();

do {
v = cache.get(MultiKey(K, index));

if (v != null)
res.add(v);

index++;
}
while (v != null);

return res;
{code}
We could also use batching for performance reason. In this case the batch size 
should be configurable.
{code}
int index = 0;

List res = new LinkedList<>();

while (true) {
List batch = new ArrayList<>(batchSize);

// Populate batch.
for (; index < batchSize; index++)
batch.add(new MultiKey(K, index % batchSize);

Map batchRes = cache.getAll(batch);

// Potentially need to properly sort values, based on the key order,
// if the returning map does not do it automatically.
res.addAll(batchRes.values());

if (res.size() < batch.size())
break;
}

return res;
{code}
h2. Evictions
Evictions in the {{IgniteMultiMap}} should have 2 levels: maximum number of 
keys, and maximum number of values for a key. The maximum number of keys should 
be controlled by Ignite standard eviction policy. The maximum number of values 
for a key should be controlled by the implementation of the multi-map.

  was:
We need to add {{IgniteMultimap}} data structure in addition to other data 
structures provided by Ignite. {{IgniteMultiMap}} should have similar API to 
{{java.util.Map}} class in JDK, but support the semantics of multiple values 
per key, similar to [Guava 
Multimap|http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Multimap.html].
 

However, unlike in Guava, our multi-map should work with Lists, not 
Collections. Lists should make it possible to support the following methods:
{code}
// Gets value at a certain index for a key.
V get(K, index);

// Gets values for specified indexes for a key.
Collection get(K, Iterable indexes);

// Gets values for specified range of indexes, between min and max.
Collection get(K, int min, int max);

// Gets all values for a specific key.
Collection get(K);
{code}

Multimap should also support colocated and non-colocated modes, similar to 
[IgniteQueue|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/IgniteQueue.java]
 and its implementation, 
[GridAtomicCacheQueueImpl|https://github.com/apache/incubator-ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/datastructures/GridAtomicCacheQueueImpl.java].
h2. Design Details
The most natural way to implement such map, would be to store every value under 
a separate key in an Ignite cache. For example, let's say that we have a key 
{{K}} with