[jira] [Comment Edited] (IGNITE-11704) Write tombstones during rebalance to get rid of deferred delete buffer
[
https://issues.apache.org/jira/browse/IGNITE-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946165#comment-16946165
]
Alexei Scherbakov edited comment on IGNITE-11704 at 10/8/19 7:04 AM:
-
[~jokser] [~sboikov]
I've reviewed changes. Overall looks good, but still I have some questions.
1. My main concern is regarding the necessity of tombstoneBytes 5-bytes object.
Seems it's possible to implement tombstone by treating absence of value as a
tombstone.
For example, valLen=0 could be treated as tombstone presense. Doing so we can
get rid of 5 bytes comparison, and instead do null check:
{noformat}
private Boolean isTombstone(ByteBuffer buf, int offset) {
int valLen = buf.getInt(buf.position() + offset);
if (valLen != tombstoneBytes.length)
return Boolean.FALSE;
...
}
{noformat}
Instead we can do something like {{if (valLen == 0) return true}}
2. With new changes in PartitionsEvictManager it's possible to have two tasks
of different types for the same partition.
Consider a scenario:
* node finished rebalancing and starts to clear thombstones
* another node joins topology and become an owner for clearing partition.
* eviction is started for already clearing partition.
Probably this should not be allowed.
3. I see changes having no obvious relation to contribution, for example:
static String cacheGroupMetricsRegistryName(String cacheGrp)
DropCacheContextDuringEvictionTest.java
GridCommandHandlerIndexingTest.java
What's the purpose of these ?
4. Could you clarify the change in
org.apache.ignite.internal.processors.cache.GridCacheMapEntry#initialValue:
update0 |= (!preload && val == null); ?
was (Author: ascherbakov):
[~jokser] [~sboikov]
I've reviewed changes. Overall looks good, but still I have some questions.
1. My main concern is regarding the necessity of tombstoneBytes 5-bytes object.
Seems it's possible to implement tombstone by treating absence of value as a
tombstone.
For example, valLen=0 could be treated as tombstone presense. Doing so we can
get rid of 5 bytes comparison, and instead do null check:
{noformat}
private Boolean isTombstone(ByteBuffer buf, int offset) {
int valLen = buf.getInt(buf.position() + offset);
if (valLen != tombstoneBytes.length)
return Boolean.FALSE;
...
}
{noformat}
Instead we can do something like {{if (valLen == 0) return true}}
2. With new changes in PartitionsEvictManager it's possible to have two tasks
of different types for the same partition.
Consider a scenario:
* node finished rebalancing and starts to clear thombstones
* another node joins topology and become an owner for clearing partition.
* eviction is started for already clearing partition.
Probably this should not be allowed.
3. I see changes having no obvious relation to contribution, for example:
static String cacheGroupMetricsRegistryName(String cacheGrp)
DropCacheContextDuringEvictionTest.java
GridCommandHandlerIndexingTest.java
What's the purpose of these ?
4. Could you explain the modification in
org.apache.ignite.internal.processors.cache.GridCacheMapEntry#initialValue:
update0 |= (!preload && val == null); ?
> Write tombstones during rebalance to get rid of deferred delete buffer
> --
>
> Key: IGNITE-11704
> URL: https://issues.apache.org/jira/browse/IGNITE-11704
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Pavel Kovalenko
>Priority: Major
> Labels: rebalance
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently Ignite relies on deferred delete buffer in order to handle
> write-remove conflicts during rebalance. Given the limit size of the buffer,
> this approach is fundamentally flawed, especially in case when persistence is
> enabled.
> I suggest to extend the logic of data storage to be able to store key
> tombstones - to keep version for deleted entries. The tombstones will be
> stored when rebalance is in progress and should be cleaned up when rebalance
> is completed.
> Later this approach may be used to implement fast partition rebalance based
> on merkle trees (in this case, tombstones should be written on an incomplete
> baseline).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-11704) Write tombstones during rebalance to get rid of deferred delete buffer
[
https://issues.apache.org/jira/browse/IGNITE-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946165#comment-16946165
]
Alexei Scherbakov edited comment on IGNITE-11704 at 10/8/19 7:10 AM:
-
[~jokser] [~sboikov]
I've reviewed changes. Overall looks good, but still I have some questions.
1. My main concern is regarding the necessity of tombstoneBytes 5-bytes object.
Seems it's possible to implement tombstone by treating absence of value as a
tombstone.
For example, valLen=0 could be treated as tombstone presense. Doing so we can
get rid of 5 bytes comparison, and instead do null check:
{noformat}
private Boolean isTombstone(ByteBuffer buf, int offset) {
int valLen = buf.getInt(buf.position() + offset);
if (valLen != tombstoneBytes.length)
return Boolean.FALSE;
...
}
{noformat}
Instead we can do something like {{if (valLen == 0) return Boolean.TRUE}}
2. With new changes in PartitionsEvictManager it's possible to have two tasks
of different types for the same partition.
Consider a scenario:
* node finished rebalancing and starts to clear thombstones
* another node joins topology and become an owner for clearing partition.
* eviction is started for already clearing partition.
Probably this should not be allowed.
3. I see changes having no obvious relation to contribution, for example:
static String cacheGroupMetricsRegistryName(String cacheGrp)
DropCacheContextDuringEvictionTest.java
GridCommandHandlerIndexingTest.java
What's the purpose of these ?
4. Could you clarify the change in
org.apache.ignite.internal.processors.cache.GridCacheMapEntry#initialValue:
update0 |= (!preload && val == null); ?
was (Author: ascherbakov):
[~jokser] [~sboikov]
I've reviewed changes. Overall looks good, but still I have some questions.
1. My main concern is regarding the necessity of tombstoneBytes 5-bytes object.
Seems it's possible to implement tombstone by treating absence of value as a
tombstone.
For example, valLen=0 could be treated as tombstone presense. Doing so we can
get rid of 5 bytes comparison, and instead do null check:
{noformat}
private Boolean isTombstone(ByteBuffer buf, int offset) {
int valLen = buf.getInt(buf.position() + offset);
if (valLen != tombstoneBytes.length)
return Boolean.FALSE;
...
}
{noformat}
Instead we can do something like {{if (valLen == 0) return true}}
2. With new changes in PartitionsEvictManager it's possible to have two tasks
of different types for the same partition.
Consider a scenario:
* node finished rebalancing and starts to clear thombstones
* another node joins topology and become an owner for clearing partition.
* eviction is started for already clearing partition.
Probably this should not be allowed.
3. I see changes having no obvious relation to contribution, for example:
static String cacheGroupMetricsRegistryName(String cacheGrp)
DropCacheContextDuringEvictionTest.java
GridCommandHandlerIndexingTest.java
What's the purpose of these ?
4. Could you clarify the change in
org.apache.ignite.internal.processors.cache.GridCacheMapEntry#initialValue:
update0 |= (!preload && val == null); ?
> Write tombstones during rebalance to get rid of deferred delete buffer
> --
>
> Key: IGNITE-11704
> URL: https://issues.apache.org/jira/browse/IGNITE-11704
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Pavel Kovalenko
>Priority: Major
> Labels: rebalance
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently Ignite relies on deferred delete buffer in order to handle
> write-remove conflicts during rebalance. Given the limit size of the buffer,
> this approach is fundamentally flawed, especially in case when persistence is
> enabled.
> I suggest to extend the logic of data storage to be able to store key
> tombstones - to keep version for deleted entries. The tombstones will be
> stored when rebalance is in progress and should be cleaned up when rebalance
> is completed.
> Later this approach may be used to implement fast partition rebalance based
> on merkle trees (in this case, tombstones should be written on an incomplete
> baseline).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-11704) Write tombstones during rebalance to get rid of deferred delete buffer
[ https://issues.apache.org/jira/browse/IGNITE-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946592#comment-16946592 ] Alexei Scherbakov commented on IGNITE-11704: 5. I would add one more load test scenario: Start a node, backups=1. Load many keys (like 100k). Join another node triggering rebalance. Delay each batch. Remove keys supplied in the batch. Release batch. Validate cache is empty and tombstones are cleared. > Write tombstones during rebalance to get rid of deferred delete buffer > -- > > Key: IGNITE-11704 > URL: https://issues.apache.org/jira/browse/IGNITE-11704 > Project: Ignite > Issue Type: Improvement >Reporter: Alexey Goncharuk >Assignee: Pavel Kovalenko >Priority: Major > Labels: rebalance > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > Currently Ignite relies on deferred delete buffer in order to handle > write-remove conflicts during rebalance. Given the limit size of the buffer, > this approach is fundamentally flawed, especially in case when persistence is > enabled. > I suggest to extend the logic of data storage to be able to store key > tombstones - to keep version for deleted entries. The tombstones will be > stored when rebalance is in progress and should be cleaned up when rebalance > is completed. > Later this approach may be used to implement fast partition rebalance based > on merkle trees (in this case, tombstones should be written on an incomplete > baseline). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12267) ClassCastException after change column type (drop, add)
[
https://issues.apache.org/jira/browse/IGNITE-12267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946645#comment-16946645
]
Ignite TC Bot commented on IGNITE-12267:
{panel:title=Branch: [pull/6945/head] Base: [master] : Possible Blockers
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cache 5{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=4670720]]
* IgniteCacheWithIndexingTestSuite:
CacheRegisterMetadataLocallyTest.testClientFindsValueByAffinityKeyStaticCacheWithoutExtraRequest
- Test has low fail rate in base branch 0,0% and is not flaky
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4669667&buildTypeId=IgniteTests24Java8_RunAll]
> ClassCastException after change column type (drop, add)
> ---
>
> Key: IGNITE-12267
> URL: https://issues.apache.org/jira/browse/IGNITE-12267
> Project: Ignite
> Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> SQL column type change is not present, but it is possible to delete and
> create with a new type.
> The application of the migration script passes without errors.
> The error occurs whenever the column is accessed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12267) ClassCastException after change column type (drop, add)
[ https://issues.apache.org/jira/browse/IGNITE-12267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946646#comment-16946646 ] Kirill Tkalenko commented on IGNITE-12267: -- My changes do not break the tests above. [~Denis Chudov], please review the code. > ClassCastException after change column type (drop, add) > --- > > Key: IGNITE-12267 > URL: https://issues.apache.org/jira/browse/IGNITE-12267 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > SQL column type change is not present, but it is possible to delete and > create with a new type. > The application of the migration script passes without errors. > The error occurs whenever the column is accessed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12268) Adds possibility to set up custom REST processor.
PetrovMikhail created IGNITE-12268: -- Summary: Adds possibility to set up custom REST processor. Key: IGNITE-12268 URL: https://issues.apache.org/jira/browse/IGNITE-12268 Project: Ignite Issue Type: Improvement Reporter: PetrovMikhail Assignee: PetrovMikhail It's needed to add ability to configure your own REST processor via functionality of Ignite plugins. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-10479) [ML] Umbrella: Ensemble training and inference
[ https://issues.apache.org/jira/browse/IGNITE-10479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev reassigned IGNITE-10479: Assignee: Alexey Zinoviev (was: Yury Babak) > [ML] Umbrella: Ensemble training and inference > -- > > Key: IGNITE-10479 > URL: https://issues.apache.org/jira/browse/IGNITE-10479 > Project: Ignite > Issue Type: New Feature > Components: ml >Reporter: Yury Babak >Assignee: Alexey Zinoviev >Priority: Major > > We want to unify API/usage of any ensembles of models. > Currently we already have only boosting and bagging and we want to implement > stacking. > Stacking is an ensemble learning technique that combines multiple > classification or regression models via a meta-classifier or a > meta-regressor. The base level models are trained based on a complete > training set, then the meta-model is trained on the outputs of the base level > model as features. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12189) Implement correct limit for TextQuery
[
https://issues.apache.org/jira/browse/IGNITE-12189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946684#comment-16946684
]
Ignite TC Bot commented on IGNITE-12189:
{panel:title=Branch: [pull/6917/head] Base: [master] : Possible Blockers
(5)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Queries 1{color} [[tests
2|https://ci.ignite.apache.org/viewLog.html?buildId=4661690]]
* IgniteBinaryCacheQueryTestSuite:
BasicIndexMultinodeTest.testDynamicIndexesWithPersistenceIndexRebuild - Test
has low fail rate in base branch 0,0% and is not flaky
* IgniteBinaryCacheQueryTestSuite:
BasicIndexMultinodeTest.testAllIndexesWithPersistence - Test has low fail rate
in base branch 0,0% and is not flaky
{color:#d04437}MVCC PDS 1{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=4661707]]
* IgnitePdsMvccTestSuite:
IgniteClusterActivateDeactivateTestWithPersistence.testDeactivateSimple_5_Servers_5_Clients_FromClient
- Test has low fail rate in base branch 0,0% and is not flaky
{color:#d04437}Thin client: PHP{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=4661649]]
{color:#d04437}[Check Code Style]{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=4661712]]
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4661714&buildTypeId=IgniteTests24Java8_RunAll]
> Implement correct limit for TextQuery
> -
>
> Key: IGNITE-12189
> URL: https://issues.apache.org/jira/browse/IGNITE-12189
> Project: Ignite
> Issue Type: Improvement
> Components: general
>Reporter: Yuriy Shuliha
>Assignee: Yuriy Shuliha
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> PROBLEM
> For now each server-node returns all response records to the client-node and
> it may contain ~thousands, ~hundred thousands records.
> Event if we need only first 10-100. Again, all the results are added to
> queue in _*GridCacheQueryFutureAdapter*_ in arbitrary order by pages.
> There are no any means to deliver deterministic result.
> SOLUTION
> Implement _*limit*_ as parameter for _*TextQuery*_ and
> _*GridCacheQueryRequest*_
> It should be passed as limit parameter in Lucene's
> _*IndexSearcher.search()*_ in _*GridLuceneIndex*_.
> For distributed queries _*limit*_ will also trim response queue when merging
> results.
> Type: long
> Special value: : 0 -> No limit (Integer.MAX_VALUE);
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-896) Configuration inconsistency
[ https://issues.apache.org/jira/browse/IGNITE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946709#comment-16946709 ] Maxim Muzafarov commented on IGNITE-896: [~zzzadruga] Are you still working on this issue? Can we move it to the next release? > Configuration inconsistency > --- > > Key: IGNITE-896 > URL: https://issues.apache.org/jira/browse/IGNITE-896 > Project: Ignite > Issue Type: Bug > Components: general >Affects Versions: sprint-5 >Reporter: Valentin Kulichenko >Assignee: Nikolai Kulagin >Priority: Minor > Labels: Usability > Fix For: 2.8 > > > I noticed that some entities on cache configuration are configured via > factories, while others are set directly. For example, we use factory for > ExpiryPolicy, but not for EvictionPolicy, which looks inconsistent. Since > factory-based approach comes from JCache, I think we should use it wherever > possible. > Here is the list of settings that need to be fixed: > * Affinity > * AffinityMapper > * EvictionFilter > * EvictionPolicy > * CacheInterceptor > * TopologyValidator > Need to add new configuration properties that use factories and deprecate old > ones (do not remove for compatibility). > Also need to check other configuration beans (list above is for cache config > only). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-1025) Need to print out warning if IP finder has a lot of addresses on Windows
[ https://issues.apache.org/jira/browse/IGNITE-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-1025: Fix Version/s: (was: 2.8) > Need to print out warning if IP finder has a lot of addresses on Windows > > > Key: IGNITE-1025 > URL: https://issues.apache.org/jira/browse/IGNITE-1025 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: sprint-4 >Reporter: Valentin Kulichenko >Assignee: Ivan Fedotov >Priority: Minor > Labels: Usability, newbie > > Windows OS has a known issue: when we try to connect to unavailable address, > we have to wait for socket timeout (Linux, for example, will throw an > exception immediately in this case). > Because of this issue node startup process can take significant amount of > time if there is a long list of addresses in IP finder. And now it looks like > a node simply hangs for a while (sometimes several minutes). > We should print the warning in this case to tell the user that startup can > take time and make suggestions on how to avoid it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-6504) Very quick checkpoint can cause AssertionError on next start from LFS
[
https://issues.apache.org/jira/browse/IGNITE-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946713#comment-16946713
]
Maxim Muzafarov commented on IGNITE-6504:
-
[~ivan.glukos]
Hello, can we move this issue to the next release?
> Very quick checkpoint can cause AssertionError on next start from LFS
> -
>
> Key: IGNITE-6504
> URL: https://issues.apache.org/jira/browse/IGNITE-6504
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Affects Versions: 2.1
>Reporter: Ivan Rakov
>Assignee: Ivan Rakov
>Priority: Major
> Labels: newbie
> Fix For: 2.8
>
>
> Checkpoint markers are compared using their timestamps. If checkpoint took
> less than 1 millisecond, two subsequent markers will have same timestamp,
> which will lead to error:
> {noformat}
> java.lang.AssertionError:
> o1=/data/teamcity/tmpfs/work/db/127_0_0_1_47503/cp/1506338145591-c4f23411-e1b1-4468-856a-4419003bba93-END.bin,
>
> o2=/data/teamcity/tmpfs/work/db/127_0_0_1_47503/cp/1506338145591-f76c023b-9982-40d7-a1eb-855a33b710f2-END.bin
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$4.compare(GridCacheDatabaseSharedManager.java:216)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$4.compare(GridCacheDatabaseSharedManager.java:195)
> at java.util.TimSort.binarySort(TimSort.java:265)
> at java.util.TimSort.sort(TimSort.java:208)
> at java.util.TimSort.sort(TimSort.java:173)
> at java.util.Arrays.sort(Arrays.java:659)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$CheckpointHistory.loadHistory(GridCacheDatabaseSharedManager.java:2704)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$CheckpointHistory.access$2600(GridCacheDatabaseSharedManager.java:2685)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1468)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:562)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:722)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:613)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2289)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-1025) Need to print out warning if IP finder has a lot of addresses on Windows
[ https://issues.apache.org/jira/browse/IGNITE-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov reassigned IGNITE-1025: --- Assignee: (was: Ivan Fedotov) > Need to print out warning if IP finder has a lot of addresses on Windows > > > Key: IGNITE-1025 > URL: https://issues.apache.org/jira/browse/IGNITE-1025 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: sprint-4 >Reporter: Valentin Kulichenko >Priority: Minor > Labels: Usability, newbie > > Windows OS has a known issue: when we try to connect to unavailable address, > we have to wait for socket timeout (Linux, for example, will throw an > exception immediately in this case). > Because of this issue node startup process can take significant amount of > time if there is a long list of addresses in IP finder. And now it looks like > a node simply hangs for a while (sometimes several minutes). > We should print the warning in this case to tell the user that startup can > take time and make suggestions on how to avoid it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-7593) Improve data used in DecisionTreesExample
[
https://issues.apache.org/jira/browse/IGNITE-7593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946715#comment-16946715
]
Maxim Muzafarov commented on IGNITE-7593:
-
[~amalykh][~oignatenko]
Hello folks! Can we move this issue to the next release?
> Improve data used in DecisionTreesExample
> -
>
> Key: IGNITE-7593
> URL: https://issues.apache.org/jira/browse/IGNITE-7593
> Project: Ignite
> Issue Type: Task
> Components: ml
>Affects Versions: 2.4
>Reporter: Oleg Ignatenko
>Assignee: Artem Malykh
>Priority: Major
> Fix For: 2.8
>
>
> Data currently used in {{DecisionTreesExample}} looks not quite optimal:
> # It is large, as evidenced in the warning in javadocs: "It is recommended to
> start at least one node prior to launching this example if you intend to run
> it with default memory settings."
> # It makes example run for quite a long time.
> # It doesn't have license (likely meaning "all rights reserved" by default)
> which makes it troublesome to include in project sources so that current
> approach is to prompt user to download it, additionally complicated by making
> example skip when run unattended from {{IgniteExamplesMLTestSuite}}.
> Suggest to find or construct a smaller data for this example which would
> still make sense to demonstrate how algorithm works and in the same time
> would be 1) easier on memory usage, 2) quicker to run and 3) would allow
> carrying it within project instead of prompting user to download it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-7882) Atomic update requests should always use topology mappings instead of affinity
[ https://issues.apache.org/jira/browse/IGNITE-7882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946716#comment-16946716 ] Maxim Muzafarov commented on IGNITE-7882: - [~jokser] Hello, can we move this issue to the next release? > Atomic update requests should always use topology mappings instead of affinity > -- > > Key: IGNITE-7882 > URL: https://issues.apache.org/jira/browse/IGNITE-7882 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.4 >Reporter: Pavel Kovalenko >Assignee: Pavel Kovalenko >Priority: Major > Fix For: 2.8 > > > Currently for mapping cache atomic updates we can use two ways: > 1) Use nodes reporting status OWNING for partition where we send the update. > 2) Use only affinity nodes mapping if rebalance is finished. > Using the second way we may route update request only to affinity node, while > there is also node which is still owner and can process read requests. > It can lead to reading null values for some key, while update for such key > was successful a moment ago. > - > Problem with using topology mapping: > 1) We send update request with key K to near node N > 2) N performs mapping for K to nodes P, B1, B2, B3 (Primary and backups) and > starts waiting for succesful update responses for all of these nodes. > 3) N sends update request to P. During this time B3 change status to RENTING > (Eviction). > 4) P also performs mapping for K to backup nodes B1, B2. > 5) All updates are succesful, but N is still waiting for response from B3. > Update request will be not finished and hangs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8063) Transaction rollback is unmanaged in case when commit produced Runtime exception
[ https://issues.apache.org/jira/browse/IGNITE-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946717#comment-16946717 ] Maxim Muzafarov commented on IGNITE-8063: - [~jokser] The issue sounds like a bug. Can we convert it and move to the next release? > Transaction rollback is unmanaged in case when commit produced Runtime > exception > > > Key: IGNITE-8063 > URL: https://issues.apache.org/jira/browse/IGNITE-8063 > Project: Ignite > Issue Type: Improvement > Components: cache >Affects Versions: 2.4 >Reporter: Pavel Kovalenko >Assignee: Pavel Kovalenko >Priority: Minor > Fix For: 2.8 > > > When 'userCommit' produces an runtime exception transaction state is moved to > UNKNOWN, and tx.finishFuture() completes, after that rollback process runs > asynchronously and there is no simple way to await rollback completion on > such transactions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-8120) Improve test coverage of rebalance failing
[ https://issues.apache.org/jira/browse/IGNITE-8120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-8120: Fix Version/s: (was: 2.8) 2.9 > Improve test coverage of rebalance failing > -- > > Key: IGNITE-8120 > URL: https://issues.apache.org/jira/browse/IGNITE-8120 > Project: Ignite > Issue Type: Test > Components: general >Affects Versions: 2.4 >Reporter: Ivan Daschinskiy >Assignee: Ivan Daschinskiy >Priority: Minor > Labels: test > Fix For: 2.9 > > > Need to cover situation, when some archived wal segments, which are not > reserved by IgniteWriteAheadLogManager, are deleted during rebalancing or > were deleted before. However, rebalancing from WAL is broken. When fix > [IGNITE-8116|https://issues.apache.org/jira/browse/IGNITE-8116] is available, > it will be implemented. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-896) Configuration inconsistency
[ https://issues.apache.org/jira/browse/IGNITE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946718#comment-16946718 ] Nikolai Kulagin commented on IGNITE-896: [~mmuzaf], I'm waiting for ticket IGNITE-9005 to continue working. I suppose we can move it to the next release. > Configuration inconsistency > --- > > Key: IGNITE-896 > URL: https://issues.apache.org/jira/browse/IGNITE-896 > Project: Ignite > Issue Type: Bug > Components: general >Affects Versions: sprint-5 >Reporter: Valentin Kulichenko >Assignee: Nikolai Kulagin >Priority: Minor > Labels: Usability > Fix For: 2.8 > > > I noticed that some entities on cache configuration are configured via > factories, while others are set directly. For example, we use factory for > ExpiryPolicy, but not for EvictionPolicy, which looks inconsistent. Since > factory-based approach comes from JCache, I think we should use it wherever > possible. > Here is the list of settings that need to be fixed: > * Affinity > * AffinityMapper > * EvictionFilter > * EvictionPolicy > * CacheInterceptor > * TopologyValidator > Need to add new configuration properties that use factories and deprecate old > ones (do not remove for compatibility). > Also need to check other configuration beans (list above is for cache config > only). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8279) Clients can't operate on services after deactivation
[
https://issues.apache.org/jira/browse/IGNITE-8279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946719#comment-16946719
]
Maxim Muzafarov commented on IGNITE-8279:
-
[~pvinokurov]
Hello! Any updates on this issue?
> Clients can't operate on services after deactivation
>
>
> Key: IGNITE-8279
> URL: https://issues.apache.org/jira/browse/IGNITE-8279
> Project: Ignite
> Issue Type: Bug
> Components: managed services
>Affects Versions: 2.4
>Reporter: Denis Mekhanikov
>Assignee: Pavel Vinokurov
>Priority: Major
> Fix For: 2.8
>
> Attachments: ServiceDeploymentAfterDeactivationTest.java
>
>
> When a cluster gets deactivated and activated back again, clients become
> incapable of service deployment. Calls to {{IgniteService.deploy*}} methods
> hang indefinitely, and no services are getting deployed to these clients.
> After deactivation, {{ServiceEntriesListener}} stops being invoked on service
> cache changes.
> Find attached test, reproducing this problem.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-8293) BinaryUtils#isCustomJavaSerialization fails when only readObject is declared in a class
[
https://issues.apache.org/jira/browse/IGNITE-8293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-8293:
Fix Version/s: (was: 2.8)
2.9
> BinaryUtils#isCustomJavaSerialization fails when only readObject is declared
> in a class
> ---
>
> Key: IGNITE-8293
> URL: https://issues.apache.org/jira/browse/IGNITE-8293
> Project: Ignite
> Issue Type: Bug
> Components: binary
>Affects Versions: 2.4
>Reporter: MihkelJ
>Assignee: MihkelJ
>Priority: Minor
> Fix For: 2.9
>
>
> Consider this class:
>
> {code:java}
> public class Test implements Serializable {
> private transient AtomicBoolean dirty = new AtomicBoolean(false);
> private void readObject(java.io.ObjectInputStream in) throws IOException,
> ClassNotFoundException {
> dirty = new AtomicBoolean(false);
> }
> //methods to check and mark class as dirty
> }{code}
> {{isCustomJavaSerialization}} will get a {{NoSuchMethodException}} when
> trying to grab the {{writeObject}} method and falsely conclude that Test
> doesn't use custom serialization.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8293) BinaryUtils#isCustomJavaSerialization fails when only readObject is declared in a class
[
https://issues.apache.org/jira/browse/IGNITE-8293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946720#comment-16946720
]
Maxim Muzafarov commented on IGNITE-8293:
-
Moved to 2.9 due to inactivity. Please feel free to move back if you will be
able to complete the ticket by AI 2.8 code freeze date, December 2, 2019.
> BinaryUtils#isCustomJavaSerialization fails when only readObject is declared
> in a class
> ---
>
> Key: IGNITE-8293
> URL: https://issues.apache.org/jira/browse/IGNITE-8293
> Project: Ignite
> Issue Type: Bug
> Components: binary
>Affects Versions: 2.4
>Reporter: MihkelJ
>Assignee: MihkelJ
>Priority: Minor
> Fix For: 2.8
>
>
> Consider this class:
>
> {code:java}
> public class Test implements Serializable {
> private transient AtomicBoolean dirty = new AtomicBoolean(false);
> private void readObject(java.io.ObjectInputStream in) throws IOException,
> ClassNotFoundException {
> dirty = new AtomicBoolean(false);
> }
> //methods to check and mark class as dirty
> }{code}
> {{isCustomJavaSerialization}} will get a {{NoSuchMethodException}} when
> trying to grab the {{writeObject}} method and falsely conclude that Test
> doesn't use custom serialization.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-8391) Removing some WAL history segments leads to WAL rebalance hanging
[
https://issues.apache.org/jira/browse/IGNITE-8391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-8391:
Fix Version/s: (was: 2.8)
2.9
> Removing some WAL history segments leads to WAL rebalance hanging
> -
>
> Key: IGNITE-8391
> URL: https://issues.apache.org/jira/browse/IGNITE-8391
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.4
>Reporter: Pavel Kovalenko
>Assignee: Vladislav Pyatkov
>Priority: Major
> Fix For: 2.9
>
>
> Problem:
> 1) Start 2 nodes, load some data to it.
> 2) Stop node 2, load some data to cache.
> 3) Remove WAL archived segment which doesn't contain Checkpoint record needed
> to find start point for WAL rebalance, but contains necessary data for
> rebalancing.
> 4) Start node 2, this node will start rebalance data from node 1 using WAL.
> Rebalance will be hanged with following assertion:
> {noformat}
> java.lang.AssertionError: Partitions after rebalance should be either done or
> missing: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
> 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplier.handleDemandMessage(GridDhtPartitionSupplier.java:417)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleDemandMessage(GridDhtPreloader.java:364)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:379)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:364)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:99)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1603)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2752)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1516)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1485)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
>
> This happened because we never reached necessary data and updateCounters
> contained in removed WAL segment.
> To resolve such problems we should introduce some fallback strategy if
> rebalance by WAL has been failed. Example of fallback strategy is - re-run
> full rebalance for partitions that were not able properly rebalanced using
> WAL.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8391) Removing some WAL history segments leads to WAL rebalance hanging
[
https://issues.apache.org/jira/browse/IGNITE-8391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946721#comment-16946721
]
Maxim Muzafarov commented on IGNITE-8391:
-
Moved to 2.9 due to inactivity. Please feel free to move back if you will be
able to complete the ticket by AI 2.8 code freeze date, December 2, 2019.
> Removing some WAL history segments leads to WAL rebalance hanging
> -
>
> Key: IGNITE-8391
> URL: https://issues.apache.org/jira/browse/IGNITE-8391
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.4
>Reporter: Pavel Kovalenko
>Assignee: Vladislav Pyatkov
>Priority: Major
> Fix For: 2.8
>
>
> Problem:
> 1) Start 2 nodes, load some data to it.
> 2) Stop node 2, load some data to cache.
> 3) Remove WAL archived segment which doesn't contain Checkpoint record needed
> to find start point for WAL rebalance, but contains necessary data for
> rebalancing.
> 4) Start node 2, this node will start rebalance data from node 1 using WAL.
> Rebalance will be hanged with following assertion:
> {noformat}
> java.lang.AssertionError: Partitions after rebalance should be either done or
> missing: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
> 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplier.handleDemandMessage(GridDhtPartitionSupplier.java:417)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleDemandMessage(GridDhtPreloader.java:364)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:379)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:364)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:99)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1603)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2752)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1516)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1485)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
>
> This happened because we never reached necessary data and updateCounters
> contained in removed WAL segment.
> To resolve such problems we should introduce some fallback strategy if
> rebalance by WAL has been failed. Example of fallback strategy is - re-run
> full rebalance for partitions that were not able properly rebalanced using
> WAL.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8784) Deadlock during simultaneous client reconnect and node stop
[
https://issues.apache.org/jira/browse/IGNITE-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946725#comment-16946725
]
Maxim Muzafarov commented on IGNITE-8784:
-
Moved to 2.9 due to inactivity.
> Deadlock during simultaneous client reconnect and node stop
> ---
>
> Key: IGNITE-8784
> URL: https://issues.apache.org/jira/browse/IGNITE-8784
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.5
>Reporter: Pavel Kovalenko
>Priority: Critical
> Fix For: 2.8
>
>
> {noformat}
> [18:48:22,665][ERROR][tcp-client-disco-msg-worker-#467%client%][IgniteKernal%client]
> Failed to reconnect, will stop node
> class org.apache.ignite.IgniteException: Failed to wait for local node joined
> event (grid is stopping).
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.localJoin(GridDiscoveryManager.java:2193)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.onKernalStart(GridCachePartitionExchangeManager.java:583)
> at
> org.apache.ignite.internal.processors.cache.GridCacheSharedContext.onReconnected(GridCacheSharedContext.java:396)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReconnected(GridCacheProcessor.java:1159)
> at
> org.apache.ignite.internal.IgniteKernal.onReconnected(IgniteKernal.java:3915)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery0(GridDiscoveryManager.java:830)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery(GridDiscoveryManager.java:589)
> at
> org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.notifyDiscovery(ClientImpl.java:2423)
> at
> org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.notifyDiscovery(ClientImpl.java:2402)
> at
> org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.processNodeAddFinishedMessage(ClientImpl.java:2047)
> at
> org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.processDiscoveryMessage(ClientImpl.java:1896)
> at
> org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1788)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to wait for
> local node joined event (grid is stopping).
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.onKernalStop0(GridDiscoveryManager.java:1657)
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.onKernalStop(GridManagerAdapter.java:652)
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2218)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2166)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2588)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2551)
> at org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:372)
> at org.apache.ignite.Ignition.stop(Ignition.java:229)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.stopGrid(GridAbstractTest.java:1088)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.stopAllGrids(GridAbstractTest.java:1128)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.stopAllGrids(GridAbstractTest.java:1109)
> at
> org.gridgain.grid.internal.processors.cache.database.IgniteDbSnapshotNotStableTopologiesTest.afterTest(IgniteDbSnapshotNotStableTopologiesTest.java:250)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.tearDown(GridAbstractTest.java:1694)
> at
> org.apache.ignite.testframework.junits.common.GridCommonAbstractTest.tearDown(GridCommonAbstractTest.java:492)
> at junit.framework.TestCase.runBare(TestCase.java:146)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:369)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:275)
> at
> org.apache.m
[jira] [Updated] (IGNITE-8784) Deadlock during simultaneous client reconnect and node stop
[
https://issues.apache.org/jira/browse/IGNITE-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-8784:
Fix Version/s: (was: 2.8)
2.9
> Deadlock during simultaneous client reconnect and node stop
> ---
>
> Key: IGNITE-8784
> URL: https://issues.apache.org/jira/browse/IGNITE-8784
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.5
>Reporter: Pavel Kovalenko
>Priority: Critical
> Fix For: 2.9
>
>
> {noformat}
> [18:48:22,665][ERROR][tcp-client-disco-msg-worker-#467%client%][IgniteKernal%client]
> Failed to reconnect, will stop node
> class org.apache.ignite.IgniteException: Failed to wait for local node joined
> event (grid is stopping).
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.localJoin(GridDiscoveryManager.java:2193)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.onKernalStart(GridCachePartitionExchangeManager.java:583)
> at
> org.apache.ignite.internal.processors.cache.GridCacheSharedContext.onReconnected(GridCacheSharedContext.java:396)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.onReconnected(GridCacheProcessor.java:1159)
> at
> org.apache.ignite.internal.IgniteKernal.onReconnected(IgniteKernal.java:3915)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery0(GridDiscoveryManager.java:830)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$4.onDiscovery(GridDiscoveryManager.java:589)
> at
> org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.notifyDiscovery(ClientImpl.java:2423)
> at
> org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.notifyDiscovery(ClientImpl.java:2402)
> at
> org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.processNodeAddFinishedMessage(ClientImpl.java:2047)
> at
> org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.processDiscoveryMessage(ClientImpl.java:1896)
> at
> org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1788)
> at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to wait for
> local node joined event (grid is stopping).
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.onKernalStop0(GridDiscoveryManager.java:1657)
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.onKernalStop(GridManagerAdapter.java:652)
> at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2218)
> at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2166)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2588)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2551)
> at org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:372)
> at org.apache.ignite.Ignition.stop(Ignition.java:229)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.stopGrid(GridAbstractTest.java:1088)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.stopAllGrids(GridAbstractTest.java:1128)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.stopAllGrids(GridAbstractTest.java:1109)
> at
> org.gridgain.grid.internal.processors.cache.database.IgniteDbSnapshotNotStableTopologiesTest.afterTest(IgniteDbSnapshotNotStableTopologiesTest.java:250)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.tearDown(GridAbstractTest.java:1694)
> at
> org.apache.ignite.testframework.junits.common.GridCommonAbstractTest.tearDown(GridCommonAbstractTest.java:492)
> at junit.framework.TestCase.runBare(TestCase.java:146)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:369)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:275)
> at
> org.apache.maven.surefire.junit4.JUnit4Provide
[jira] [Updated] (IGNITE-8632) Exception in Ignite shutdown routine can prevent JVM shutdown
[ https://issues.apache.org/jira/browse/IGNITE-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-8632: Fix Version/s: (was: 2.8) > Exception in Ignite shutdown routine can prevent JVM shutdown > - > > Key: IGNITE-8632 > URL: https://issues.apache.org/jira/browse/IGNITE-8632 > Project: Ignite > Issue Type: Bug > Components: general >Reporter: Anton Kurbanov >Assignee: Anton Kurbanov >Priority: Major > > The exception like below left node in stopping state fo forever: > java.lang.NullPointerException: null > at > java.util.concurrent.ConcurrentLinkedQueue.checkNotNull(ConcurrentLinkedQueue.java:914) > at > java.util.concurrent.ConcurrentLinkedQueue.offer(ConcurrentLinkedQueue.java:327) > at > java.util.concurrent.ConcurrentLinkedQueue.add(ConcurrentLinkedQueue.java:297) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateResponse.addFailedKey(GridNearAtomicUpdateResponse.java:347) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicSingleUpdateFuture.addFailedKeys(GridDhtAtomicSingleUpdateFuture.java:166) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFuture.java:446) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFuture.java:56) > at > org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:345) > at > org.apache.ignite.internal.processors.cache.GridCacheMvccManager.cancelClientFutures(GridCacheMvccManager.java:388) > at > org.apache.ignite.internal.processors.cache.GridCacheMvccManager.onStop(GridCacheMvccManager.java:370) > at > org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:956) > at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2095) > at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2041) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2397) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2360) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:1871) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8632) Exception in Ignite shutdown routine can prevent JVM shutdown
[ https://issues.apache.org/jira/browse/IGNITE-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946723#comment-16946723 ] Maxim Muzafarov commented on IGNITE-8632: - Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > Exception in Ignite shutdown routine can prevent JVM shutdown > - > > Key: IGNITE-8632 > URL: https://issues.apache.org/jira/browse/IGNITE-8632 > Project: Ignite > Issue Type: Bug > Components: general >Reporter: Anton Kurbanov >Assignee: Anton Kurbanov >Priority: Major > Fix For: 2.8 > > > The exception like below left node in stopping state fo forever: > java.lang.NullPointerException: null > at > java.util.concurrent.ConcurrentLinkedQueue.checkNotNull(ConcurrentLinkedQueue.java:914) > at > java.util.concurrent.ConcurrentLinkedQueue.offer(ConcurrentLinkedQueue.java:327) > at > java.util.concurrent.ConcurrentLinkedQueue.add(ConcurrentLinkedQueue.java:297) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateResponse.addFailedKey(GridNearAtomicUpdateResponse.java:347) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicSingleUpdateFuture.addFailedKeys(GridDhtAtomicSingleUpdateFuture.java:166) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFuture.java:446) > at > org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture.onDone(GridDhtAtomicAbstractUpdateFuture.java:56) > at > org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:345) > at > org.apache.ignite.internal.processors.cache.GridCacheMvccManager.cancelClientFutures(GridCacheMvccManager.java:388) > at > org.apache.ignite.internal.processors.cache.GridCacheMvccManager.onStop(GridCacheMvccManager.java:370) > at > org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:956) > at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2095) > at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2041) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2397) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2360) > at > org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.run(IgnitionEx.java:1871) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8850) Add a GA example that solves 'Traveling Salesman Problem'
[ https://issues.apache.org/jira/browse/IGNITE-8850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946726#comment-16946726 ] Maxim Muzafarov commented on IGNITE-8850: - Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > Add a GA example that solves 'Traveling Salesman Problem' > - > > Key: IGNITE-8850 > URL: https://issues.apache.org/jira/browse/IGNITE-8850 > Project: Ignite > Issue Type: New Feature > Components: ml >Reporter: Turik Campbell >Assignee: Turik Campbell >Priority: Minor > Fix For: 2.8 > > > The Travelling Salesman Problem (TSP) asks the following question: "Given a > list of cities and the distances between each pair of cities, what is the > shortest possible route that visits each city and returns to the origin city? > Additional Information: > https://en.wikipedia.org/wiki/Travelling_salesman_problem -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-8850) Add a GA example that solves 'Traveling Salesman Problem'
[ https://issues.apache.org/jira/browse/IGNITE-8850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-8850: Fix Version/s: (was: 2.8) 2.9 > Add a GA example that solves 'Traveling Salesman Problem' > - > > Key: IGNITE-8850 > URL: https://issues.apache.org/jira/browse/IGNITE-8850 > Project: Ignite > Issue Type: New Feature > Components: ml >Reporter: Turik Campbell >Assignee: Turik Campbell >Priority: Minor > Fix For: 2.9 > > > The Travelling Salesman Problem (TSP) asks the following question: "Given a > list of cities and the distances between each pair of cities, what is the > shortest possible route that visits each city and returns to the origin city? > Additional Information: > https://en.wikipedia.org/wiki/Travelling_salesman_problem -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-9437) [ML] Add performance benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946730#comment-16946730 ] Maxim Muzafarov commented on IGNITE-9437: - Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > [ML] Add performance benchmarks > --- > > Key: IGNITE-9437 > URL: https://issues.apache.org/jira/browse/IGNITE-9437 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Yury Babak >Priority: Major > Fix For: 2.8 > > > We want to have some performance benchmarks for ML algorithms -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-9414) [ML] Using sparce vectors in Tree-based algorithms.
[ https://issues.apache.org/jira/browse/IGNITE-9414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946728#comment-16946728 ] Maxim Muzafarov commented on IGNITE-9414: - Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > [ML] Using sparce vectors in Tree-based algorithms. > --- > > Key: IGNITE-9414 > URL: https://issues.apache.org/jira/browse/IGNITE-9414 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Alexey Platonov >Priority: Minor > Fix For: 2.8 > > > We need to support sparce vectors in DecisionTrees, RF, GDB -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-9415) [ML] Using sparce vectors in LSQR and MLP
[ https://issues.apache.org/jira/browse/IGNITE-9415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946729#comment-16946729 ] Maxim Muzafarov commented on IGNITE-9415: - Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > [ML] Using sparce vectors in LSQR and MLP > - > > Key: IGNITE-9415 > URL: https://issues.apache.org/jira/browse/IGNITE-9415 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Alexey Platonov >Priority: Minor > Fix For: 2.8 > > > We need to investigate and apply sparce vectors support in BLAS for LSQR and > MLP (or implement own version) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-9415) [ML] Using sparce vectors in LSQR and MLP
[ https://issues.apache.org/jira/browse/IGNITE-9415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-9415: Fix Version/s: (was: 2.8) 2.9 > [ML] Using sparce vectors in LSQR and MLP > - > > Key: IGNITE-9415 > URL: https://issues.apache.org/jira/browse/IGNITE-9415 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Alexey Platonov >Priority: Minor > Fix For: 2.9 > > > We need to investigate and apply sparce vectors support in BLAS for LSQR and > MLP (or implement own version) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-9414) [ML] Using sparce vectors in Tree-based algorithms.
[ https://issues.apache.org/jira/browse/IGNITE-9414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-9414: Fix Version/s: (was: 2.8) 2.9 > [ML] Using sparce vectors in Tree-based algorithms. > --- > > Key: IGNITE-9414 > URL: https://issues.apache.org/jira/browse/IGNITE-9414 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Alexey Platonov >Priority: Minor > Fix For: 2.9 > > > We need to support sparce vectors in DecisionTrees, RF, GDB -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-10035) Fix tests IgniteWalFormatFileFailoverTest
[ https://issues.apache.org/jira/browse/IGNITE-10035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-10035: - Fix Version/s: (was: 2.8) 2.9 > Fix tests IgniteWalFormatFileFailoverTest > - > > Key: IGNITE-10035 > URL: https://issues.apache.org/jira/browse/IGNITE-10035 > Project: Ignite > Issue Type: Test > Components: cache >Affects Versions: 2.8 >Reporter: Pavel Kovalenko >Assignee: Pavel Kovalenko >Priority: Minor > Fix For: 2.9 > > > After IGNITE-9420 introduce, WAL Archiver component is started together with > WAL manager. Tests suppose that WAL Archiver will be started after first > activation, and proper file io factory will be injected to it. Need to find > out how to inject file io factory before file archiver is started. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-10035) Fix tests IgniteWalFormatFileFailoverTest
[ https://issues.apache.org/jira/browse/IGNITE-10035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946731#comment-16946731 ] Maxim Muzafarov commented on IGNITE-10035: -- Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > Fix tests IgniteWalFormatFileFailoverTest > - > > Key: IGNITE-10035 > URL: https://issues.apache.org/jira/browse/IGNITE-10035 > Project: Ignite > Issue Type: Test > Components: cache >Affects Versions: 2.8 >Reporter: Pavel Kovalenko >Assignee: Pavel Kovalenko >Priority: Minor > Fix For: 2.8 > > > After IGNITE-9420 introduce, WAL Archiver component is started together with > WAL manager. Tests suppose that WAL Archiver will be started after first > activation, and proper file io factory will be injected to it. Need to find > out how to inject file io factory before file archiver is started. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-9437) [ML] Add performance benchmarks
[ https://issues.apache.org/jira/browse/IGNITE-9437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-9437: Fix Version/s: (was: 2.8) 2.9 > [ML] Add performance benchmarks > --- > > Key: IGNITE-9437 > URL: https://issues.apache.org/jira/browse/IGNITE-9437 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Yury Babak >Priority: Major > Fix For: 2.9 > > > We want to have some performance benchmarks for ML algorithms -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-10035) Fix tests IgniteWalFormatFileFailoverTest
[ https://issues.apache.org/jira/browse/IGNITE-10035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-10035: - Issue Type: Test (was: New Feature) > Fix tests IgniteWalFormatFileFailoverTest > - > > Key: IGNITE-10035 > URL: https://issues.apache.org/jira/browse/IGNITE-10035 > Project: Ignite > Issue Type: Test > Components: cache >Affects Versions: 2.8 >Reporter: Pavel Kovalenko >Assignee: Pavel Kovalenko >Priority: Minor > Fix For: 2.8 > > > After IGNITE-9420 introduce, WAL Archiver component is started together with > WAL manager. Tests suppose that WAL Archiver will be started after first > activation, and proper file io factory will be injected to it. Need to find > out how to inject file io factory before file archiver is started. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-10144) Optimize bagging upstream transformer
[ https://issues.apache.org/jira/browse/IGNITE-10144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946733#comment-16946733 ] Maxim Muzafarov commented on IGNITE-10144: -- Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > Optimize bagging upstream transformer > - > > Key: IGNITE-10144 > URL: https://issues.apache.org/jira/browse/IGNITE-10144 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Artem Malykh >Assignee: Artem Malykh >Priority: Minor > Fix For: 2.8 > > > For now BaggingUpstreamTransformer makes upstream sequential to make > transformation deterministic. Maybe we should do it other way, for example > use mapping of the form (entryIdx, en) -> Stream.generate(() -> en).limit(new > PoissonDistribution(Well19937c(entryIdx + seed), ...).sample()) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-9676) Ignite as storage in Spring Session
[ https://issues.apache.org/jira/browse/IGNITE-9676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946732#comment-16946732 ] Maxim Muzafarov commented on IGNITE-9676: - Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > Ignite as storage in Spring Session > --- > > Key: IGNITE-9676 > URL: https://issues.apache.org/jira/browse/IGNITE-9676 > Project: Ignite > Issue Type: New Feature > Components: spring >Reporter: Anton Kurbanov >Assignee: Anton Kurbanov >Priority: Minor > Fix For: 2.8 > > > Implement repository backed with Ignite for sessions clustering with Spring > Session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-10144) Optimize bagging upstream transformer
[ https://issues.apache.org/jira/browse/IGNITE-10144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-10144: - Fix Version/s: (was: 2.8) 2.9 > Optimize bagging upstream transformer > - > > Key: IGNITE-10144 > URL: https://issues.apache.org/jira/browse/IGNITE-10144 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Artem Malykh >Assignee: Artem Malykh >Priority: Minor > Fix For: 2.9 > > > For now BaggingUpstreamTransformer makes upstream sequential to make > transformation deterministic. Maybe we should do it other way, for example > use mapping of the form (entryIdx, en) -> Stream.generate(() -> en).limit(new > PoissonDistribution(Well19937c(entryIdx + seed), ...).sample()) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-10541) [ML] Umbrella: EM (GMM) with adding and removal of components
[ https://issues.apache.org/jira/browse/IGNITE-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946734#comment-16946734 ] Maxim Muzafarov commented on IGNITE-10541: -- Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > [ML] Umbrella: EM (GMM) with adding and removal of components > - > > Key: IGNITE-10541 > URL: https://issues.apache.org/jira/browse/IGNITE-10541 > Project: Ignite > Issue Type: New Feature > Components: ml >Reporter: Yury Babak >Assignee: Yury Babak >Priority: Major > Fix For: 2.8 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-9676) Ignite as storage in Spring Session
[ https://issues.apache.org/jira/browse/IGNITE-9676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-9676: Fix Version/s: (was: 2.8) 2.9 > Ignite as storage in Spring Session > --- > > Key: IGNITE-9676 > URL: https://issues.apache.org/jira/browse/IGNITE-9676 > Project: Ignite > Issue Type: New Feature > Components: spring >Reporter: Anton Kurbanov >Assignee: Anton Kurbanov >Priority: Minor > Fix For: 2.9 > > > Implement repository backed with Ignite for sessions clustering with Spring > Session. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-10541) [ML] Umbrella: EM (GMM) with adding and removal of components
[ https://issues.apache.org/jira/browse/IGNITE-10541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-10541: - Fix Version/s: (was: 2.8) 2.9 > [ML] Umbrella: EM (GMM) with adding and removal of components > - > > Key: IGNITE-10541 > URL: https://issues.apache.org/jira/browse/IGNITE-10541 > Project: Ignite > Issue Type: New Feature > Components: ml >Reporter: Yury Babak >Assignee: Yury Babak >Priority: Major > Fix For: 2.9 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-10655) .NET client should have peerClassLoading property in IgniteConfiguration class
[ https://issues.apache.org/jira/browse/IGNITE-10655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946735#comment-16946735 ] Maxim Muzafarov commented on IGNITE-10655: -- Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > .NET client should have peerClassLoading property in IgniteConfiguration class > -- > > Key: IGNITE-10655 > URL: https://issues.apache.org/jira/browse/IGNITE-10655 > Project: Ignite > Issue Type: Improvement > Components: platforms >Affects Versions: 2.7 >Reporter: Andrey Aleksandrov >Assignee: Vladimir Pligin >Priority: Major > Fix For: 2.8 > > > In case if java server nodes in a cluster were set up with peerClassLoading > set to true then there is no way to start node via .NET client using > IgniteConfiguration (because the default value is false). > The only way to do it is using of the XML configuration. > Possibility to set this property from .Net client programmatically should be > added. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-10655) .NET client should have peerClassLoading property in IgniteConfiguration class
[ https://issues.apache.org/jira/browse/IGNITE-10655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-10655: - Fix Version/s: (was: 2.8) 2.9 > .NET client should have peerClassLoading property in IgniteConfiguration class > -- > > Key: IGNITE-10655 > URL: https://issues.apache.org/jira/browse/IGNITE-10655 > Project: Ignite > Issue Type: Improvement > Components: platforms >Affects Versions: 2.7 >Reporter: Andrey Aleksandrov >Assignee: Vladimir Pligin >Priority: Major > Fix For: 2.9 > > > In case if java server nodes in a cluster were set up with peerClassLoading > set to true then there is no way to start node via .NET client using > IgniteConfiguration (because the default value is false). > The only way to do it is using of the XML configuration. > Possibility to set this property from .Net client programmatically should be > added. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-7062) Ignite page with video resources and recording
[ https://issues.apache.org/jira/browse/IGNITE-7062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946737#comment-16946737 ] Maxim Muzafarov commented on IGNITE-7062: - [~pgarg] [~dmagda] Can we move this issue to the next release? Should we wait for any updates here? > Ignite page with video resources and recording > -- > > Key: IGNITE-7062 > URL: https://issues.apache.org/jira/browse/IGNITE-7062 > Project: Ignite > Issue Type: Task > Components: site >Reporter: Denis A. Magda >Assignee: Prachi Garg >Priority: Major > Fix For: 2.8 > > > There is a plenty of recordings of Ignite meetups, webinars and conference > talks available on the Internet. Some of them introduce basic components and > capabilities, some share best practices and pitfalls while the other share > use cases. > Generally, it's beneficial for both Ignite community and users to gather and > expose the most useful ones under a special video recording section. For > instance, we might consider these talks to be added right away: > * Ignite use case: https://youtu.be/1D8hyLWMtfM > * Ignite essentials: https://www.youtube.com/watch?v=G22L2KW9gEQ > * Kubernetes: https://www.youtube.com/watch?v=igDB0wyodr8 > Instead of creating a new page for this purpose I would rework the > screencasts' page combining all the media content there: > https://ignite.apache.org/screencasts.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11019) SQL: explain plan of a simple query contains merge table
[
https://issues.apache.org/jira/browse/IGNITE-11019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11019:
-
Fix Version/s: (was: 2.8)
2.9
> SQL: explain plan of a simple query contains merge table
>
>
> Key: IGNITE-11019
> URL: https://issues.apache.org/jira/browse/IGNITE-11019
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Alexander Lapin
>Priority: Major
> Fix For: 2.9
>
>
> In case of simple* query like following "select * from Organization org where
> org._KEY = 1 or org._KEY = 2" explain plan will contain merge table despite
> the fact that it's skipped within regular query flow.
> {code:java|title=GridReduceQueryExecutor#query}
> final boolean skipMergeTbl = !qry.explain() && qry.skipMergeTable()
> {code}
> Explain plan output:
> : [SELECT
> ORG__Z0.NAME AS __C0_0,
> ORG__Z0.DEBTCAPITAL AS __C0_1,
> ORG__Z0.ID AS __C0_2
> FROM "orgBetweenTest".ORGANIZATION ORG__Z0
> /* "orgBetweenTest"."_key_PK": _KEY IN(1, 2) */
> WHERE ORG__Z0._KEY IN(1, 2)]
> : [SELECT
> __C0_0 AS NAME,
> __C0_1 AS DEBTCAPITAL,
> __C0_2 AS ID
> FROM PUBLIC.__T0
> /* "orgBetweenTest"."merge_scan" */]
> *simple query by means of GridSqlQuery#simpleQuery
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11019) SQL: explain plan of a simple query contains merge table
[
https://issues.apache.org/jira/browse/IGNITE-11019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11019:
-
Ignite Flags: Docs Required,Release Notes Required (was: Docs Required)
> SQL: explain plan of a simple query contains merge table
>
>
> Key: IGNITE-11019
> URL: https://issues.apache.org/jira/browse/IGNITE-11019
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Alexander Lapin
>Priority: Major
> Fix For: 2.9
>
>
> In case of simple* query like following "select * from Organization org where
> org._KEY = 1 or org._KEY = 2" explain plan will contain merge table despite
> the fact that it's skipped within regular query flow.
> {code:java|title=GridReduceQueryExecutor#query}
> final boolean skipMergeTbl = !qry.explain() && qry.skipMergeTable()
> {code}
> Explain plan output:
> : [SELECT
> ORG__Z0.NAME AS __C0_0,
> ORG__Z0.DEBTCAPITAL AS __C0_1,
> ORG__Z0.ID AS __C0_2
> FROM "orgBetweenTest".ORGANIZATION ORG__Z0
> /* "orgBetweenTest"."_key_PK": _KEY IN(1, 2) */
> WHERE ORG__Z0._KEY IN(1, 2)]
> : [SELECT
> __C0_0 AS NAME,
> __C0_1 AS DEBTCAPITAL,
> __C0_2 AS ID
> FROM PUBLIC.__T0
> /* "orgBetweenTest"."merge_scan" */]
> *simple query by means of GridSqlQuery#simpleQuery
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-10971) SQL: Support partition pruning for distributed joins
[
https://issues.apache.org/jira/browse/IGNITE-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-10971:
-
Ignite Flags: Docs Required,Release Notes Required (was: Docs Required)
> SQL: Support partition pruning for distributed joins
>
>
> Key: IGNITE-10971
> URL: https://issues.apache.org/jira/browse/IGNITE-10971
> Project: Ignite
> Issue Type: Task
> Components: sql
>Reporter: Vladimir Ozerov
>Priority: Major
> Fix For: 2.9
>
>
> During IGNITE-10307 implementation it was revealed that distributed joins do
> not work with partition pruning. We never observed it before because it was
> impossible to derive partitions from joins.
> The problem appears as timeout exception from reducer due to some
> timeouts/retries inside distributed joins logic. Failures could be reproduced
> as follows:
> 1) Remove {{GridSqlQuerySplitter.distributedJoins}} usage which prevents
> partition to be derived for map query.
> 2) Run any of the following tests and observe that some of tests cases fails
> with reducer timeout:
> {{IgniteSqlSplitterSelfTest}}
> {{IgniteCacheJoinQueryWithAffinityKeyTest}}
> {{IgniteCacheDistributedJoinQueryConditionsTest}}
> {{IgniteCacheCrossCacheJoinRandomTest}}
> Root cause is unknown, but most likely this is due some missing messages,
> because some parts of distributed join engine is not aware of extracted
> partitions and await for replies from not involved nodes.
> Note that most likely the same problem will appear for queries with
> distributed joins and explicit partitions ({{SqlFieldsQuery.partitions}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11019) SQL: explain plan of a simple query contains merge table
[
https://issues.apache.org/jira/browse/IGNITE-11019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11019:
-
Issue Type: Bug (was: Improvement)
> SQL: explain plan of a simple query contains merge table
>
>
> Key: IGNITE-11019
> URL: https://issues.apache.org/jira/browse/IGNITE-11019
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Alexander Lapin
>Priority: Major
> Fix For: 2.9
>
>
> In case of simple* query like following "select * from Organization org where
> org._KEY = 1 or org._KEY = 2" explain plan will contain merge table despite
> the fact that it's skipped within regular query flow.
> {code:java|title=GridReduceQueryExecutor#query}
> final boolean skipMergeTbl = !qry.explain() && qry.skipMergeTable()
> {code}
> Explain plan output:
> : [SELECT
> ORG__Z0.NAME AS __C0_0,
> ORG__Z0.DEBTCAPITAL AS __C0_1,
> ORG__Z0.ID AS __C0_2
> FROM "orgBetweenTest".ORGANIZATION ORG__Z0
> /* "orgBetweenTest"."_key_PK": _KEY IN(1, 2) */
> WHERE ORG__Z0._KEY IN(1, 2)]
> : [SELECT
> __C0_0 AS NAME,
> __C0_1 AS DEBTCAPITAL,
> __C0_2 AS ID
> FROM PUBLIC.__T0
> /* "orgBetweenTest"."merge_scan" */]
> *simple query by means of GridSqlQuery#simpleQuery
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11019) SQL: explain plan of a simple query contains merge table
[
https://issues.apache.org/jira/browse/IGNITE-11019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11019:
-
Ignite Flags: (was: Release Notes Required)
> SQL: explain plan of a simple query contains merge table
>
>
> Key: IGNITE-11019
> URL: https://issues.apache.org/jira/browse/IGNITE-11019
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Alexander Lapin
>Priority: Major
> Fix For: 2.9
>
>
> In case of simple* query like following "select * from Organization org where
> org._KEY = 1 or org._KEY = 2" explain plan will contain merge table despite
> the fact that it's skipped within regular query flow.
> {code:java|title=GridReduceQueryExecutor#query}
> final boolean skipMergeTbl = !qry.explain() && qry.skipMergeTable()
> {code}
> Explain plan output:
> : [SELECT
> ORG__Z0.NAME AS __C0_0,
> ORG__Z0.DEBTCAPITAL AS __C0_1,
> ORG__Z0.ID AS __C0_2
> FROM "orgBetweenTest".ORGANIZATION ORG__Z0
> /* "orgBetweenTest"."_key_PK": _KEY IN(1, 2) */
> WHERE ORG__Z0._KEY IN(1, 2)]
> : [SELECT
> __C0_0 AS NAME,
> __C0_1 AS DEBTCAPITAL,
> __C0_2 AS ID
> FROM PUBLIC.__T0
> /* "orgBetweenTest"."merge_scan" */]
> *simple query by means of GridSqlQuery#simpleQuery
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11019) SQL: explain plan of a simple query contains merge table
[
https://issues.apache.org/jira/browse/IGNITE-11019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11019:
-
Ignite Flags: Release Notes Required (was: Docs Required,Release Notes
Required)
> SQL: explain plan of a simple query contains merge table
>
>
> Key: IGNITE-11019
> URL: https://issues.apache.org/jira/browse/IGNITE-11019
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Alexander Lapin
>Priority: Major
> Fix For: 2.9
>
>
> In case of simple* query like following "select * from Organization org where
> org._KEY = 1 or org._KEY = 2" explain plan will contain merge table despite
> the fact that it's skipped within regular query flow.
> {code:java|title=GridReduceQueryExecutor#query}
> final boolean skipMergeTbl = !qry.explain() && qry.skipMergeTable()
> {code}
> Explain plan output:
> : [SELECT
> ORG__Z0.NAME AS __C0_0,
> ORG__Z0.DEBTCAPITAL AS __C0_1,
> ORG__Z0.ID AS __C0_2
> FROM "orgBetweenTest".ORGANIZATION ORG__Z0
> /* "orgBetweenTest"."_key_PK": _KEY IN(1, 2) */
> WHERE ORG__Z0._KEY IN(1, 2)]
> : [SELECT
> __C0_0 AS NAME,
> __C0_1 AS DEBTCAPITAL,
> __C0_2 AS ID
> FROM PUBLIC.__T0
> /* "orgBetweenTest"."merge_scan" */]
> *simple query by means of GridSqlQuery#simpleQuery
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-10971) SQL: Support partition pruning for distributed joins
[
https://issues.apache.org/jira/browse/IGNITE-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-10971:
-
Fix Version/s: (was: 2.8)
2.9
> SQL: Support partition pruning for distributed joins
>
>
> Key: IGNITE-10971
> URL: https://issues.apache.org/jira/browse/IGNITE-10971
> Project: Ignite
> Issue Type: Task
> Components: sql
>Reporter: Vladimir Ozerov
>Priority: Major
> Fix For: 2.9
>
>
> During IGNITE-10307 implementation it was revealed that distributed joins do
> not work with partition pruning. We never observed it before because it was
> impossible to derive partitions from joins.
> The problem appears as timeout exception from reducer due to some
> timeouts/retries inside distributed joins logic. Failures could be reproduced
> as follows:
> 1) Remove {{GridSqlQuerySplitter.distributedJoins}} usage which prevents
> partition to be derived for map query.
> 2) Run any of the following tests and observe that some of tests cases fails
> with reducer timeout:
> {{IgniteSqlSplitterSelfTest}}
> {{IgniteCacheJoinQueryWithAffinityKeyTest}}
> {{IgniteCacheDistributedJoinQueryConditionsTest}}
> {{IgniteCacheCrossCacheJoinRandomTest}}
> Root cause is unknown, but most likely this is due some missing messages,
> because some parts of distributed join engine is not aware of extracted
> partitions and await for replies from not involved nodes.
> Note that most likely the same problem will appear for queries with
> distributed joins and explicit partitions ({{SqlFieldsQuery.partitions}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11207) SQL: Remove MapNodeResults class
[
https://issues.apache.org/jira/browse/IGNITE-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11207:
-
Fix Version/s: (was: 2.8)
2.9
> SQL: Remove MapNodeResults class
>
>
> Key: IGNITE-11207
> URL: https://issues.apache.org/jira/browse/IGNITE-11207
> Project: Ignite
> Issue Type: Task
> Components: sql
>Reporter: Vladimir Ozerov
>Priority: Major
> Fix For: 2.9
>
>
> This class holds results for a specific node. Let's remove it and refactor
> associated code with the following goals in mind:
> # Performance: one CHM lookup instead of two
> # Uniformity: move both SELECT and DML under the same {{MapQueryResult}}
> umbrella
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11207) SQL: Remove MapNodeResults class
[
https://issues.apache.org/jira/browse/IGNITE-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11207:
-
Ignite Flags: (was: Docs Required)
> SQL: Remove MapNodeResults class
>
>
> Key: IGNITE-11207
> URL: https://issues.apache.org/jira/browse/IGNITE-11207
> Project: Ignite
> Issue Type: Task
> Components: sql
>Reporter: Vladimir Ozerov
>Priority: Major
> Fix For: 2.9
>
>
> This class holds results for a specific node. Let's remove it and refactor
> associated code with the following goals in mind:
> # Performance: one CHM lookup instead of two
> # Uniformity: move both SELECT and DML under the same {{MapQueryResult}}
> umbrella
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11320) JDBC Thin: add support for individual reconnect in case of best effort affinity mode.
[ https://issues.apache.org/jira/browse/IGNITE-11320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11320: - Fix Version/s: (was: 2.8) 2.9 > JDBC Thin: add support for individual reconnect in case of best effort > affinity mode. > - > > Key: IGNITE-11320 > URL: https://issues.apache.org/jira/browse/IGNITE-11320 > Project: Ignite > Issue Type: Task > Components: jdbc >Reporter: Alexander Lapin >Priority: Major > Labels: iep-23 > Fix For: 2.9 > > > Currently in case of best effort affinity mode we either connect to all nodes > specified by user or throw SQLException. Given logic needs to be improved. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11321) JDBC Thin: implement nodes multi version support in case of best effort affinity mode
[ https://issues.apache.org/jira/browse/IGNITE-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11321: - Fix Version/s: (was: 2.8) 2.9 > JDBC Thin: implement nodes multi version support in case of best effort > affinity mode > - > > Key: IGNITE-11321 > URL: https://issues.apache.org/jira/browse/IGNITE-11321 > Project: Ignite > Issue Type: Task > Components: jdbc >Reporter: Alexander Lapin >Priority: Major > Labels: iep-23 > Fix For: 2.9 > > > Currently in case of best effort affinity mode we throw SQLException if the > version of any of the nodes to which we connect is different from the version > of all other nodes.Given logic needs to be improved. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11325) SQL: Single place to start missing caches (H2Utils.checkAndStartNotStartedCache)
[
https://issues.apache.org/jira/browse/IGNITE-11325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11325:
-
Fix Version/s: (was: 2.8)
2.9
> SQL: Single place to start missing caches
> (H2Utils.checkAndStartNotStartedCache)
>
>
> Key: IGNITE-11325
> URL: https://issues.apache.org/jira/browse/IGNITE-11325
> Project: Ignite
> Issue Type: Task
> Components: sql
>Reporter: Vladimir Ozerov
>Priority: Major
> Fix For: 2.9
>
>
> We need to start missing caches for the given SELECT/DML statement because we
> need affinity info during query planning which is only available for started
> caches.
> We need to do the following:
> # Move the method {{H2Utils.checkAndStartNotStartedCache}} to some common
> place, e.g. parser, so that it has one and only one usage all over the code
> base
> # Make sure that this method doesn't produce multiple network hops: missing
> caches should be started in a single request if possible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12097) Code cleanup: useless GridDhtInvalidPartitionException handling that cannot be thrown on a demander (preloadEntry).
[
https://issues.apache.org/jira/browse/IGNITE-12097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-12097:
-
Fix Version/s: (was: 2.8)
2.9
> Code cleanup: useless GridDhtInvalidPartitionException handling that cannot
> be thrown on a demander (preloadEntry).
> ---
>
> Key: IGNITE-12097
> URL: https://issues.apache.org/jira/browse/IGNITE-12097
> Project: Ignite
> Issue Type: Improvement
> Components: general
>Reporter: Pavel Pereslegin
>Assignee: Pavel Pereslegin
>Priority: Minor
> Fix For: 2.9
>
>
> {{GridDhtPartitionDemander#preloadEntry}} (and {{mvccPreloadEntry}}) handles
> a *{{GridDhtInvalidPartitionException}}* that cannot be thrown.
> {{preloadEntry}} and {{mvccPreloadEntry}} always execute on a *reserved*
> partition, which cannot be evicted until it is released.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-11852) Assertion errors when changing PME coordinator to locally joining node
[
https://issues.apache.org/jira/browse/IGNITE-11852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946740#comment-16946740
]
Maxim Muzafarov commented on IGNITE-11852:
--
[~jokser] [~agoncharuk]
Hello, any updates here?
Who will be able to take a review?
> Assertion errors when changing PME coordinator to locally joining node
> --
>
> Key: IGNITE-11852
> URL: https://issues.apache.org/jira/browse/IGNITE-11852
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.5, 2.7
>Reporter: Pavel Kovalenko
>Assignee: Pavel Kovalenko
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When PME coordinator changed to locally joining node several assertion errors
> may occur:
> 1. When some other joining nodes finished PME:
> {noformat}
> [13:49:58] (err) Failed to notify listener:
> o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$8$1$1...@27296181java.lang.AssertionError:
> AffinityTopologyVersion [topVer=2, minorTopVer=0]
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager$11.applyx(CacheAffinitySharedManager.java:1546)
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager$11.applyx(CacheAffinitySharedManager.java:1535)
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.lambda$forAllRegisteredCacheGroups$e0a6939d$1(CacheAffinitySharedManager.java:1281)
> at
> org.apache.ignite.internal.util.IgniteUtils.doInParallel(IgniteUtils.java:10929)
> at
> org.apache.ignite.internal.util.IgniteUtils.doInParallel(IgniteUtils.java:10831)
> at
> org.apache.ignite.internal.util.IgniteUtils.doInParallel(IgniteUtils.java:10811)
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.forAllRegisteredCacheGroups(CacheAffinitySharedManager.java:1280)
> at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onLocalJoin(CacheAffinitySharedManager.java:1535)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processFullMessage(GridDhtPartitionsExchangeFuture.java:4189)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onBecomeCoordinator(GridDhtPartitionsExchangeFuture.java:4731)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$3400(GridDhtPartitionsExchangeFuture.java:145)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$8$1$1.apply(GridDhtPartitionsExchangeFuture.java:4622)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$8$1$1.apply(GridDhtPartitionsExchangeFuture.java:4611)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:398)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:510)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:489)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:466)
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture.checkComplete(GridCompoundFuture.java:281)
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:143)
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture.apply(GridCompoundFuture.java:44)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:398)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:510)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:489)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:455)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.InitNewCoordinatorFuture.onMessage(InitNewCoordinatorFuture.java:253)
> at
> org.apache.ignite.internal.
[jira] [Updated] (IGNITE-11839) SQL: table join order changes may lead to incorrect result
[
https://issues.apache.org/jira/browse/IGNITE-11839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11839:
-
Fix Version/s: (was: 2.8)
2.9
> SQL: table join order changes may lead to incorrect result
> --
>
> Key: IGNITE-11839
> URL: https://issues.apache.org/jira/browse/IGNITE-11839
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.7
>Reporter: Roman Kondakov
>Priority: Major
> Fix For: 2.9
>
>
> Under some circumstances table join order changes may lead to incorrect
> result. For example if one of joining tables is {{REPLICATED}} and another
> has {{queryparallelism > 1}}.
> This problem can be reproduced in test
> {{IgniteSqlSegmentedIndexSelfTest#testSegmentedPartitionedWithReplicated}} if
> swap tables {{Person}} and {{Organization}} in the method
> {{IgniteSqlSegmentedIndexSelfTest#checkLocalQueryWithSegmentedIndex}} and set
> {{enforceJoinOrder}} flag to {{true}}:
> {code:java}
> String select0 = "select o.name n1, p.name n2 from \"org\".Organization o,
> \"pers\".Person p where p.orgId = o._key";
> List> res = c1.query(new
> SqlFieldsQuery(select0).setLocal(true).setEnforceJoinOrder(true)).getAll();
> {code}
> Result is:
> {noformat}
> java.lang.AssertionError:
> Expected :956
> Actual :8
>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.ignite.testframework.junits.JUnitAssertAware.assertEquals(JUnitAssertAware.java:89)
> at
> org.apache.ignite.internal.processors.query.IgniteSqlSegmentedIndexSelfTest.checkLocalQueryWithSegmentedIndex(IgniteSqlSegmentedIndexSelfTest.java:280)
> at
> org.apache.ignite.internal.processors.query.IgniteSqlSegmentedIndexSelfTest.testSegmentedPartitionedWithReplicated(IgniteSqlSegmentedIndexSelfTest.java:222)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2145)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11795) JDBC thin datastreamer don't throws exception is case of problems on closing streamer.
[
https://issues.apache.org/jira/browse/IGNITE-11795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11795:
-
Fix Version/s: (was: 2.8)
2.9
> JDBC thin datastreamer don't throws exception is case of problems on closing
> streamer.
> --
>
> Key: IGNITE-11795
> URL: https://issues.apache.org/jira/browse/IGNITE-11795
> Project: Ignite
> Issue Type: Bug
> Components: jdbc, sql, thin client
>Reporter: Sergey Antonov
>Priority: Major
> Fix For: 2.9
>
>
> Now from code we can't detect problems, If some errors occurs in closing jdbc
> thin datastreamer. Now we could detect it in logs only. The main reason of
> this is using {{U.close()}} for streamers in
> {{SqlClientContext#disableStreaming()}}.
> You could add test to {{JdbcThinStreamingAbstractSelfTest}} and reproduce
> problem.
> {code:java}
> /**
> * @throws Exception if failed.
> */
> @Test
> public void testStreamedInsertFailsOnReadOnlyMode() throws Exception {
> for (Ignite grid : G.allGrids())
> ((IgniteEx) grid).context().cache().context().readOnlyMode(true);
> try {
> boolean failed = false;
> try (Connection ordinalCon = createOrdinaryConnection();
> Statement selectStmt = ordinalCon.createStatement()
> ) {
> try (ResultSet rs = selectStmt.executeQuery("select count(*)
> from PUBLIC.Person")) {
> assertTrue(rs.next());
> assertEquals(0, rs.getLong(1));
> }
> try (Connection conn = createStreamedConnection(true)) {
> try (PreparedStatement stmt =
> conn.prepareStatement("insert into
> PUBLIC.Person(\"id\", \"name\") values (?, ?)")
> ) {
> for (int i = 1; i <= 2; i++) {
> stmt.setInt(1, i);
> stmt.setString(2, nameForId(i));
> stmt.executeUpdate();
> }
> }
> }
> catch (Exception e) {
> log.error("Insert failed", e);
> failed = true;
> }
> try (ResultSet rs = selectStmt.executeQuery("select count(*)
> from PUBLIC.Person")) {
> assertTrue(rs.next());
> assertEquals("Insert should be failed!", 0,
> rs.getLong(1));
> }
> }
> assertTrue("Insert should be failed!", failed);
> }
> finally {
> for (Ignite grid : G.allGrids())
> ((IgniteEx)
> grid).context().cache().context().readOnlyMode(false);
> }
> }
> {code}
> Don't forget remove {{@Ignore}} annotation from
> {{JdbcThinStreamingAbstractSelfTest#testStreamedInsertFailsOnReadOnlyMode()}}
> test!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11709) .NET thin client: introduce Cluster API
[
https://issues.apache.org/jira/browse/IGNITE-11709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11709:
-
Fix Version/s: (was: 2.8)
2.9
> .NET thin client: introduce Cluster API
> ---
>
> Key: IGNITE-11709
> URL: https://issues.apache.org/jira/browse/IGNITE-11709
> Project: Ignite
> Issue Type: New Feature
> Components: thin client
>Reporter: Alexandr Shapkin
>Assignee: Alexandr Shapkin
>Priority: Major
> Fix For: 2.9
>
>
> Let's start implementing {{ICluster}} and {{IClusterGroup}} for thin client.
> Desired functionality, according to the regular client
> * {{ICluster.SetActive()}}
> * {{ICluster.IsActive()}}
> * {{ICluster.DisableWal()}}
> * {{ICluster.EnableWal()}}
> * {{ICluster.IsWalEnabled()}}
> * {{IClusterGroup.ForAttribute()}}
> * {{IClusterGroup.ForDataNodes()}}
> * {{IClusterGroup.ForDotNet()}}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11722) Batch BPlustTree updates for cache batch operations
[ https://issues.apache.org/jira/browse/IGNITE-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11722: - Ignite Flags: (was: Docs Required) > Batch BPlustTree updates for cache batch operations > --- > > Key: IGNITE-11722 > URL: https://issues.apache.org/jira/browse/IGNITE-11722 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Semen Boikov >Priority: Major > > Now cache batch operations (putAll, invokeAll, removeAll) update BPlustTree > entries one by one. It makes sense to support batch updates in BPlustTree > and use it for cache batch operations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11722) Batch BPlustTree updates for cache batch operations
[ https://issues.apache.org/jira/browse/IGNITE-11722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11722: - Fix Version/s: (was: 2.8) > Batch BPlustTree updates for cache batch operations > --- > > Key: IGNITE-11722 > URL: https://issues.apache.org/jira/browse/IGNITE-11722 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Semen Boikov >Priority: Major > > Now cache batch operations (putAll, invokeAll, removeAll) update BPlustTree > entries one by one. It makes sense to support batch updates in BPlustTree > and use it for cache batch operations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11731) CPP: Implement minimal Cluster API
[
https://issues.apache.org/jira/browse/IGNITE-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946741#comment-16946741
]
Maxim Muzafarov commented on IGNITE-11731:
--
Moved to 2.9 due to inactivity. Please, feel free to move it back if you will
be able to complete the ticket by 2.8 code freeze date, December 2, 2019.
> CPP: Implement minimal Cluster API
> --
>
> Key: IGNITE-11731
> URL: https://issues.apache.org/jira/browse/IGNITE-11731
> Project: Ignite
> Issue Type: New Feature
> Components: platforms
>Affects Versions: 2.7
>Reporter: Igor Sapego
>Priority: Major
> Labels: cpp
> Fix For: 2.8
>
>
> Let's start implementing Cluster API for C++.
> Desired functionality from Java:
> * {{IgniteCluster.active()}}
> * {{IgniteClusterGroup.forAttribute()}}
> * {{IgniteClusterGroup.forDataNodes()}}
> * {{IgniteClusterGroup.forServers()}}
> * {{ClusterNode.id()}}
> * {{ClusterNode.attribute()}}
> * {{IgniteCompute.compute(ClusterGroup)}}
> Also, we need to have one platform-specific method:
> * {{IgniteClusterGroup.forCpp()}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-11709) .NET thin client: introduce Cluster API
[
https://issues.apache.org/jira/browse/IGNITE-11709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946742#comment-16946742
]
Maxim Muzafarov commented on IGNITE-11709:
--
Moved to 2.9 due to inactivity. Please, feel free to move it back if you will
be able to complete the ticket by 2.8 code freeze date, December 2, 2019.
> .NET thin client: introduce Cluster API
> ---
>
> Key: IGNITE-11709
> URL: https://issues.apache.org/jira/browse/IGNITE-11709
> Project: Ignite
> Issue Type: New Feature
> Components: thin client
>Reporter: Alexandr Shapkin
>Assignee: Alexandr Shapkin
>Priority: Major
> Fix For: 2.8
>
>
> Let's start implementing {{ICluster}} and {{IClusterGroup}} for thin client.
> Desired functionality, according to the regular client
> * {{ICluster.SetActive()}}
> * {{ICluster.IsActive()}}
> * {{ICluster.DisableWal()}}
> * {{ICluster.EnableWal()}}
> * {{ICluster.IsWalEnabled()}}
> * {{IClusterGroup.ForAttribute()}}
> * {{IClusterGroup.ForDataNodes()}}
> * {{IClusterGroup.ForDotNet()}}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11731) CPP: Implement minimal Cluster API
[
https://issues.apache.org/jira/browse/IGNITE-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11731:
-
Fix Version/s: (was: 2.8)
2.9
> CPP: Implement minimal Cluster API
> --
>
> Key: IGNITE-11731
> URL: https://issues.apache.org/jira/browse/IGNITE-11731
> Project: Ignite
> Issue Type: New Feature
> Components: platforms
>Affects Versions: 2.7
>Reporter: Igor Sapego
>Priority: Major
> Labels: cpp
> Fix For: 2.9
>
>
> Let's start implementing Cluster API for C++.
> Desired functionality from Java:
> * {{IgniteCluster.active()}}
> * {{IgniteClusterGroup.forAttribute()}}
> * {{IgniteClusterGroup.forDataNodes()}}
> * {{IgniteClusterGroup.forServers()}}
> * {{ClusterNode.id()}}
> * {{ClusterNode.attribute()}}
> * {{IgniteCompute.compute(ClusterGroup)}}
> Also, we need to have one platform-specific method:
> * {{IgniteClusterGroup.forCpp()}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11630) Document changes to SQL views
[
https://issues.apache.org/jira/browse/IGNITE-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11630:
-
Component/s: documentation
> Document changes to SQL views
> -
>
> Key: IGNITE-11630
> URL: https://issues.apache.org/jira/browse/IGNITE-11630
> Project: Ignite
> Issue Type: Task
> Components: documentation, sql
>Reporter: Vladimir Ozerov
>Assignee: Artem Budnikov
>Priority: Major
> Fix For: 2.8
>
>
> The following changes were made to our views.
> {{CACHE_GROUPS}}
> # {{ID}} -> {{CACHE_GROUP_ID}}
> # {{GROUP_NAME}} -> {{CACHE_GROUP_NAME}}
> {{LOCAL_CACHE_GROUPS_IO}}
> # {{GROUP_ID}} -> {{CACHE_GROUP_ID}}
> # {{GROUP_NAME}} -> {{CACHE_GROUP_NAME}}
> {{CACHES}}
> # {{NAME}} -> {{CACHE_NAME}}
> # {{GROUP_ID}} -> {{CACHE_GROUP_ID}}
> # {{GROUP_NAME}} -> {{CACHE_GROUP_NAME}}
> {{INDEXES}}
> # {{GROUP_ID}} -> {{CACHE_GROUP_ID}}
> # {{GROUP_NAME}} -> {{CACHE_GROUP_NAME}}
> {{NODES}}
> # {{ID}} -> {{NODE_ID}}
> {{TABLES}}
> # Added {{CACHE_GROUP_ID}}
> # Added {{CACHE_GROUP_NAME}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11693) Add support SqlFieldsQuery.updateBatchSize for ODBC client.
[
https://issues.apache.org/jira/browse/IGNITE-11693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11693:
-
Fix Version/s: (was: 2.8)
2.9
> Add support SqlFieldsQuery.updateBatchSize for ODBC client.
> ---
>
> Key: IGNITE-11693
> URL: https://issues.apache.org/jira/browse/IGNITE-11693
> Project: Ignite
> Issue Type: Task
> Components: odbc
>Affects Versions: 2.7
>Reporter: Taras Ledkov
>Priority: Major
> Fix For: 2.9
>
>
> Add support {{SqlFieldsQuery.updateBatchSize}} for ODBC client.
> The property {{SqlFieldsQuery.updateBatchSize}} is introduced by the patch
> IGNITE-11499.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11705) Jdbc Thin: add ability to control affinity cache size.
[ https://issues.apache.org/jira/browse/IGNITE-11705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11705: - Fix Version/s: (was: 2.8) 2.9 > Jdbc Thin: add ability to control affinity cache size. > -- > > Key: IGNITE-11705 > URL: https://issues.apache.org/jira/browse/IGNITE-11705 > Project: Ignite > Issue Type: Task > Components: jdbc >Reporter: Alexander Lapin >Priority: Major > Labels: iep-24 > Fix For: 2.9 > > > Within AffinityCache there are two properties DISTRIBUTIONS_CACHE_LIMIT and > SQL_CACHE_LIMIT that are hard coded. We should add an ability to control > given parameters within some sort of configuration. IgniteSystemProperties is > not an option however. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11674) Cancellation of SQL query request should await cancellation on all nodes
[ https://issues.apache.org/jira/browse/IGNITE-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11674: - Fix Version/s: (was: 2.8) 2.9 > Cancellation of SQL query request should await cancellation on all nodes > > > Key: IGNITE-11674 > URL: https://issues.apache.org/jira/browse/IGNITE-11674 > Project: Ignite > Issue Type: Task > Components: sql >Affects Versions: 2.8 >Reporter: Yury Gerzhedovich >Priority: Major > Labels: iep-29 > Fix For: 2.9 > > > As of now during cancellation process reduce's node send cancellation > requests to map nodes and don't wait answer from them. Need to await answer > from all of map nodes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11596) PHP thin client UUID marshalling is wrong
[
https://issues.apache.org/jira/browse/IGNITE-11596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11596:
-
Fix Version/s: (was: 2.8)
2.9
> PHP thin client UUID marshalling is wrong
> -
>
> Key: IGNITE-11596
> URL: https://issues.apache.org/jira/browse/IGNITE-11596
> Project: Ignite
> Issue Type: Bug
> Components: thin client
>Affects Versions: 2.7
>Reporter: Igor Sapego
>Priority: Major
> Fix For: 2.9
>
>
> Trying to put uuid with PHP client and get from Java or C++ client have
> different results
> PHP put
> [238,15,47,237,224,122,66,220,170,89,127,143,199,56,10,205] =
> "ee0f2fed-e07a-42dc-aa59-7f8fc7380acd"
> {code}
> $cache =
> $client->getOrCreateCache("UUID_PH")->setKeyType(ObjectType::INTEGER)->setValueType(ObjectType::UUID);
> $cache->put(1,[238,15,47,237,224,122,66,220,170,89,127,143,199,56,10,205]);
> {code}
> Java get
> {code}
> ClientCache cache = igniteClient.getOrCreateCache("UUID_PH");
> UUID id = cache.get(1);
> System.out.println(id);
> {code}
> Java output
> {code}
> dc427ae0-ed2f-0fee-cd0a-38c78f7f59aa
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11530) SQL Index performance monitoring
[ https://issues.apache.org/jira/browse/IGNITE-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11530: - Fix Version/s: (was: 2.8) 2.9 > SQL Index performance monitoring > > > Key: IGNITE-11530 > URL: https://issues.apache.org/jira/browse/IGNITE-11530 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Yury Gerzhedovich >Priority: Major > Labels: iep-29 > Fix For: 2.9 > > > We need to gather performance statistics for all types SQL indexes. > It should be at least number of look up, scans, inline miss. > Also need to investigate which statistics can be added here. > > After the ticket will be done we should expose it through JMX and SQL system > view. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11530) SQL Index performance monitoring
[ https://issues.apache.org/jira/browse/IGNITE-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946745#comment-16946745 ] Maxim Muzafarov commented on IGNITE-11530: -- [~nizhikov] Should we include this thread to IEP-35 activities? > SQL Index performance monitoring > > > Key: IGNITE-11530 > URL: https://issues.apache.org/jira/browse/IGNITE-11530 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Yury Gerzhedovich >Priority: Major > Labels: iep-29 > Fix For: 2.9 > > > We need to gather performance statistics for all types SQL indexes. > It should be at least number of look up, scans, inline miss. > Also need to investigate which statistics can be added here. > > After the ticket will be done we should expose it through JMX and SQL system > view. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11530) SQL Index performance monitoring
[ https://issues.apache.org/jira/browse/IGNITE-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-11530: - Labels: IEP-35 iep-29 (was: iep-29) > SQL Index performance monitoring > > > Key: IGNITE-11530 > URL: https://issues.apache.org/jira/browse/IGNITE-11530 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Yury Gerzhedovich >Priority: Major > Labels: IEP-35, iep-29 > Fix For: 2.9 > > > We need to gather performance statistics for all types SQL indexes. > It should be at least number of look up, scans, inline miss. > Also need to investigate which statistics can be added here. > > After the ticket will be done we should expose it through JMX and SQL system > view. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11529) SQL: Deprecate SqlQuery for NodeJS thin client
[ https://issues.apache.org/jira/browse/IGNITE-11529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11529: - Fix Version/s: (was: 2.8) 2.9 > SQL: Deprecate SqlQuery for NodeJS thin client > -- > > Key: IGNITE-11529 > URL: https://issues.apache.org/jira/browse/IGNITE-11529 > Project: Ignite > Issue Type: Task > Components: thin client >Affects Versions: 2.7 >Reporter: Taras Ledkov >Priority: Major > Fix For: 2.9 > > > This API is very limited comparing to SqlFieldsQuery. Let's deprecate it with > proper links to SqlFieldsQuery. This should be not only deprecation on public > API, but removal from examples as well. > Parent ticket: IGNITE-11334 > Ticket for documentation: IGNITE-11370 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11455) Introduce free lists rebuild mechanism
[ https://issues.apache.org/jira/browse/IGNITE-11455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11455: - Fix Version/s: (was: 2.8) 2.9 > Introduce free lists rebuild mechanism > -- > > Key: IGNITE-11455 > URL: https://issues.apache.org/jira/browse/IGNITE-11455 > Project: Ignite > Issue Type: Improvement > Components: cache >Affects Versions: 2.0 >Reporter: Pavel Kovalenko >Priority: Major > Fix For: 2.9 > > > Sometimes the state of free lists become invalid like in > https://issues.apache.org/jira/browse/IGNITE-10669 . It leads the node to an > unrecoverable state. At the same time, free lists don't hold any critical or > data information and can be built from scratch using existing data pages. It > may be useful to introduce a mechanism to rebuild free lists using an optimal > algorithm of partition data pages scanning. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11455) Introduce free lists rebuild mechanism
[ https://issues.apache.org/jira/browse/IGNITE-11455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11455: - Ignite Flags: (was: Docs Required) > Introduce free lists rebuild mechanism > -- > > Key: IGNITE-11455 > URL: https://issues.apache.org/jira/browse/IGNITE-11455 > Project: Ignite > Issue Type: Improvement > Components: cache >Affects Versions: 2.0 >Reporter: Pavel Kovalenko >Priority: Major > Fix For: 2.8 > > > Sometimes the state of free lists become invalid like in > https://issues.apache.org/jira/browse/IGNITE-10669 . It leads the node to an > unrecoverable state. At the same time, free lists don't hold any critical or > data information and can be built from scratch using existing data pages. It > may be useful to introduce a mechanism to rebuild free lists using an optimal > algorithm of partition data pages scanning. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11527) SQL: Deprecate SqlQuery for PHP thin client
[ https://issues.apache.org/jira/browse/IGNITE-11527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11527: - Fix Version/s: (was: 2.8) 2.9 > SQL: Deprecate SqlQuery for PHP thin client > --- > > Key: IGNITE-11527 > URL: https://issues.apache.org/jira/browse/IGNITE-11527 > Project: Ignite > Issue Type: Task > Components: thin client >Affects Versions: 2.7 >Reporter: Taras Ledkov >Priority: Major > Fix For: 2.9 > > > This API is very limited comparing to SqlFieldsQuery. Let's deprecate it with > proper links to SqlFieldsQuery. This should be not only deprecation on public > API, but removal from examples as well. > Parent ticket: IGNITE-11334 > Ticket for documentation: IGNITE-11370 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11528) SQL: Deprecate SqlQuery for Python thin client
[ https://issues.apache.org/jira/browse/IGNITE-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11528: - Fix Version/s: (was: 2.8) 2.9 > SQL: Deprecate SqlQuery for Python thin client > -- > > Key: IGNITE-11528 > URL: https://issues.apache.org/jira/browse/IGNITE-11528 > Project: Ignite > Issue Type: Task > Components: thin client >Affects Versions: 2.7 >Reporter: Taras Ledkov >Priority: Major > Fix For: 2.9 > > > This API is very limited comparing to SqlFieldsQuery. Let's deprecate it with > proper links to SqlFieldsQuery. This should be not only deprecation on public > API, but removal from examples as well. > Parent ticket: IGNITE-11334 > Ticket for documentation: IGNITE-11370 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11498) SQL: Rework DML data distribution logic
[
https://issues.apache.org/jira/browse/IGNITE-11498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-11498:
-
Fix Version/s: (was: 2.8)
2.9
> SQL: Rework DML data distribution logic
> ---
>
> Key: IGNITE-11498
> URL: https://issues.apache.org/jira/browse/IGNITE-11498
> Project: Ignite
> Issue Type: Task
> Components: sql
>Reporter: Vladimir Ozerov
>Priority: Major
> Fix For: 2.9
>
>
> Current DML implementation has a number of problems:
> 1) We fetch the whole data set to originator's node. There is
> "skipDmlOnReducer" flag to avoid this in some cases, but it is still in
> experimental state, and is not enabled by default
> 2) Updates are deadlock-prone: we update entries in batches equal to
> {{SqlFieldsQuery.pageSize}}. So we can deadlock easily with concurrent cache
> operations
> 3) We have very strange re-try logic. It is not clear why it is needed in the
> first place provided that DML is not transactional and no guarantees are
> needed.
> Proposal:
> # Implement proper routing logic: if a request could be executed on data
> nodes bypassing skipping reducer, do this. Otherwise fetch all data to
> reducer. This decision should be made in absolutely the same way as for MVCC
> (see {{GridNearTxQueryEnlistFuture}} as a starting point)
> # Distribute updates to primary data node in batches, but apply them one by
> one, similar to data streamer with {{allowOverwrite=false}}. Do not do any
> partition state or {{AffinityTopologyVersion}} checks, since DML is not
> transactional. Return and aggregate update counts back.
> # Remove or at least rethink retry logic. Why do we need it in the first
> place?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11507) SQL: Ensure that affinity topology version doesn't change during PartitionResult construction/application.
[ https://issues.apache.org/jira/browse/IGNITE-11507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11507: - Fix Version/s: (was: 2.8) 2.9 > SQL: Ensure that affinity topology version doesn't change during > PartitionResult construction/application. > -- > > Key: IGNITE-11507 > URL: https://issues.apache.org/jira/browse/IGNITE-11507 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Alexander Lapin >Priority: Major > Fix For: 2.9 > > > Currently some actions might be performed (for example cache removal) during > PartitionResult construction, so it might become invalid. Besides that it's > not possible to associate PartitionResult with affinity topology version, so > it is impossible to guarantee that the partition result is used on the same > version on which it was built. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11402) SQL: Add ability to specify inline size of PK and affinity key indexes from CREATE TABLE and QueryEntity
[ https://issues.apache.org/jira/browse/IGNITE-11402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11402: - Fix Version/s: (was: 2.8) 2.9 > SQL: Add ability to specify inline size of PK and affinity key indexes from > CREATE TABLE and QueryEntity > > > Key: IGNITE-11402 > URL: https://issues.apache.org/jira/browse/IGNITE-11402 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Vladimir Ozerov >Priority: Major > Fix For: 2.9 > > > Currently it is not possible to set inline size for automatically created > indexes easily. We need to make sure that user has a convenient way to set > them both programmatically and from DDL. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11434) SQL: Create a view with list of existing COLUMNS
[
https://issues.apache.org/jira/browse/IGNITE-11434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946754#comment-16946754
]
Maxim Muzafarov commented on IGNITE-11434:
--
[~tledkov-gridgain]
Any updates here? Can we move the issue to the next release?
> SQL: Create a view with list of existing COLUMNS
>
>
> Key: IGNITE-11434
> URL: https://issues.apache.org/jira/browse/IGNITE-11434
> Project: Ignite
> Issue Type: Task
> Components: sql
>Reporter: Yury Gerzhedovich
>Assignee: Taras Ledkov
>Priority: Major
> Labels: iep-29
> Fix For: 2.8
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Need to expose SQL system view with COLUMNS information.
> Need to investigate more deeper which of information should be there.
>
> As start point we can take
> [https://dev.mysql.com/doc/refman/8.0/en/columns-table.html]
> Columns description:
> || Name || Type || Description||
> | SCHEMA_NAME | string | Schema name |
> | TABLE_NAME | string | Table name |
> | COLUMN_NAME | string | Column name |
> | ORDINAL_POSITION | int | Column ordinal. Starts with 1 |
> | DEFAULT VALUE | string | Defaut column's value |
> | IS_NULLABLE | boolean | Nullable flag corresponds to
> {{QueryEntity#setNotNullFields}} |
> | DATA_TYPE | string | SQL data type |
> | CHARACTER_LENGTH | int | Size for char CAHR and VARCHAR types |
> | NUMERIC_PRECISION | int | Precision for numeric types |
> | NUMERIC_SCALE | int | Scale for numeric types |
> | IS_AFFINITY_KEY | boolean | {{true}} whan the column is affinity key |
> | IS_HIDDEN | boolean | {{true}} for hidden _ley nad _val columns. {{false}}
> for all columns available by asterisk mask |
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-12265) JavaDoc doesn't have documentation for the org.apache.ignite.client package
[
https://issues.apache.org/jira/browse/IGNITE-12265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk reassigned IGNITE-12265:
-
Assignee: Alexey Goncharuk
> JavaDoc doesn't have documentation for the org.apache.ignite.client package
> ---
>
> Key: IGNITE-12265
> URL: https://issues.apache.org/jira/browse/IGNITE-12265
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Mekhanikov
>Assignee: Alexey Goncharuk
>Priority: Major
> Labels: newbie
>
> JavaDoc published on the website doesn't have documentation for the
> {{org.apache.ignite.client}} package. Link to the website:
> [https://ignite.apache.org/releases/2.7.6/javadoc/]
> A lack of {{package-info.java}} file or exclusion from the
> {{maven-javadoc-plugin}} in the root {{pom.xml}} may be the reason.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-11397) Binary mode for Ignite Set
[ https://issues.apache.org/jira/browse/IGNITE-11397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11397: - Fix Version/s: (was: 2.8) 2.9 > Binary mode for Ignite Set > -- > > Key: IGNITE-11397 > URL: https://issues.apache.org/jira/browse/IGNITE-11397 > Project: Ignite > Issue Type: New Feature > Components: binary, data structures >Reporter: Uday Kale >Assignee: Uday Kale >Priority: Major > Fix For: 2.9 > > Time Spent: 1h > Remaining Estimate: 0h > > Add binary mode (withKeepBinary) to Data Structures Set. > This will allow retrieving values in binary format when needed or when class > is not available, and will allow implementing the structures in other > platforms (.NET, C++). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11368) use the same information about indexes for ODBC JDBC drivers as for system view INDEXES
[ https://issues.apache.org/jira/browse/IGNITE-11368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11368: - Fix Version/s: (was: 2.8) 2.9 > use the same information about indexes for ODBC JDBC drivers as for system > view INDEXES > --- > > Key: IGNITE-11368 > URL: https://issues.apache.org/jira/browse/IGNITE-11368 > Project: Ignite > Issue Type: Task > Components: jdbc, odbc, sql >Reporter: Yury Gerzhedovich >Priority: Major > Fix For: 2.9 > > > As of now indexes information for ODBC/JDBC drivers get by another way then > system SQL view INDEXES. Need to use single source of the information to have > consistent picture. > So, ODBC/JDBC drivers should use the same source as SQL view INDEXES > (org.apache.ignite.internal.processors.query.h2.sys.view.SqlSystemViewIndexes) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11397) Binary mode for Ignite Set
[ https://issues.apache.org/jira/browse/IGNITE-11397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946756#comment-16946756 ] Maxim Muzafarov commented on IGNITE-11397: -- Moved to 2.9 due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019. > Binary mode for Ignite Set > -- > > Key: IGNITE-11397 > URL: https://issues.apache.org/jira/browse/IGNITE-11397 > Project: Ignite > Issue Type: New Feature > Components: binary, data structures >Reporter: Uday Kale >Assignee: Uday Kale >Priority: Major > Fix For: 2.8 > > Time Spent: 1h > Remaining Estimate: 0h > > Add binary mode (withKeepBinary) to Data Structures Set. > This will allow retrieving values in binary format when needed or when class > is not available, and will allow implementing the structures in other > platforms (.NET, C++). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12238) RobinHoodBackwardShiftHashMap works incorrectly on big endian architectures
[
https://issues.apache.org/jira/browse/IGNITE-12238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey N. Gura updated IGNITE-12238:
Description:
{{RobinHoodBackwardShiftHashMap}} has bug that can be reproduced only on big
endinan architectures. In order to reproduce the problem run the following
tests:
* {{RobinHoodBackwardShiftHashMapTest.testCollisionOnRemove}}
* {{RobinHoodBackwardShiftHashMapTest.testRandomOpsPutRemove}}
The problem is {{setIdealBucket()}} method writes {{long}} value to the offheap
memory, while {{getIdealBucket()}} reads {{int}} value. For little endian
architectures it works because meaningful 4 bytes will written first to the
memory and leading zero bytes will be rewriteen by the next operation. On big
endian architecture always 4 zero bytes will be written to the memory.
was:
{{RobinHoodBackwardShiftHashMap}} has bug that can be reproduced only on big
endinan architectures. In order to reproduce the problem run the following
tests:
* {{RobinHoodBackwardShiftHashMapTest.testCollisionOnRemove}}
* {{testRandomOpsPutRemove}}
The problem is {{setIdealBucket()}} method writes {{long}} value to the offheap
memory, while {{getIdealBucket()}} reads {{int}} value. For little endian
architectures it works because meaningful 4 bytes will written first to the
memory and leading zero bytes will be rewriteen by the next operation. On big
endian architecture always 4 zero bytes will be written to the memory.
> RobinHoodBackwardShiftHashMap works incorrectly on big endian architectures
> ---
>
> Key: IGNITE-12238
> URL: https://issues.apache.org/jira/browse/IGNITE-12238
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey N. Gura
>Assignee: Andrey N. Gura
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{RobinHoodBackwardShiftHashMap}} has bug that can be reproduced only on big
> endinan architectures. In order to reproduce the problem run the following
> tests:
> * {{RobinHoodBackwardShiftHashMapTest.testCollisionOnRemove}}
> * {{RobinHoodBackwardShiftHashMapTest.testRandomOpsPutRemove}}
> The problem is {{setIdealBucket()}} method writes {{long}} value to the
> offheap memory, while {{getIdealBucket()}} reads {{int}} value. For little
> endian architectures it works because meaningful 4 bytes will written first
> to the memory and leading zero bytes will be rewriteen by the next operation.
> On big endian architecture always 4 zero bytes will be written to the memory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-10683) Prepare process of packaging and delivering thin clients
[
https://issues.apache.org/jira/browse/IGNITE-10683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946789#comment-16946789
]
Igor Sapego commented on IGNITE-10683:
--
[~dmagda] [~mmuzaf] [~vveider] I really think this will be a good feature to
include.
> Prepare process of packaging and delivering thin clients
>
>
> Key: IGNITE-10683
> URL: https://issues.apache.org/jira/browse/IGNITE-10683
> Project: Ignite
> Issue Type: Task
>Reporter: Peter Ivanov
>Assignee: Peter Ivanov
>Priority: Major
> Fix For: 2.8
>
>
> # **NodeJs client**
> #* +Instruction+:
> https://github.com/nobitlost/ignite/blob/ignite--docs/modules/platforms/nodejs/README.md#publish-ignite-nodejs-client-on-npmjscom-instruction
> #* +Uploaded+: https://www.npmjs.com/package/apache-ignite-client
> # **PHP client**
> #* +Instruction+:
> https://github.com/nobitlost/ignite/blob/ignite-7783-docs/modules/platforms/php/README.md#release-the-client-in-the-php-package-repository-instruction
> {panel}
> Cannot be uploaded on Packagist as the client should be in a dedicated
> repository for that -
> https://issues.apache.org/jira/browse/IGNITE-7783?focusedCommentId=16595476&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16595476
> Installation from the sources works.
> {panel}
> # **Python client**
> I have already registered the package `pyignite` on PyPI[1]. The person who
> is going to take the responsibility of maintaining it should create an
> account on PyPI and mail me in private, so that I can grant them the
> necessary rights. They also must install twine[3].
> The process of packaging is well described in the packaging tutorial[2]. In
> the nutshell, the maintainer must do the following:
> ## Clone/pull the sources from the git repository,
> ## Enter the directory in which the `setup.py` is resides (“the setup
> directory”), in our case it is `modules/platforms/python`.
> ## Create the packages with the command `python3 setup.py sdist bdist_wheel`.
> The packages will be created in `modules/platforms/python/dist` folder.
> ## Upload packages with twine: `twine upload dist/*`.
> It is very useful to have a dedicated Python virtual environment prepared to
> perform steps 3-4. Just do an editable install of `pyignite` into that
> environment from the setup directory: `pip3 install -e .` You can also
> install twine (`pip install twine`) in it.
> Consider also making a `.pypirc` file to save time on logging in to PyPI.
> Newest version of `twine` is said to support keyrings on Linux and Mac, but I
> have not tried this yet.
> [1] https://pypi.org/project/pyignite/
> [2] https://packaging.python.org/tutorials/packaging-projects/
> [3] https://twine.readthedocs.io/en/latest/
> Some other notes on PyPI and versioning.
> - The package version is located in the `setup.py`, it is a `version`
> argument of the `setuptools.setup()` function. Editing the `setup.py` is the
> only way to set the package version.
> - You absolutely can not replace a package in PyPI (hijacking prevention). If
> you have published the package by mistake, all you can do is delete the
> unwanted package, increment the version counter in `setup.py`, and try again.
> - If you upload the package through the web interface of PyPI (without
> twine), the package description will be garbled. Web interface does not
> support markdown.
> Anyway, I would like to join in the congratulations on successful release.
> Kudos to the team.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-10683) Prepare process of packaging and delivering thin clients
[
https://issues.apache.org/jira/browse/IGNITE-10683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946793#comment-16946793
]
Maxim Muzafarov commented on IGNITE-10683:
--
Folks,
Are we able to complete this feature by 2.8 code freeze date, December 2, 2019?
The feature is cool, no doubts! :-)
> Prepare process of packaging and delivering thin clients
>
>
> Key: IGNITE-10683
> URL: https://issues.apache.org/jira/browse/IGNITE-10683
> Project: Ignite
> Issue Type: Task
>Reporter: Peter Ivanov
>Assignee: Peter Ivanov
>Priority: Major
> Fix For: 2.8
>
>
> # **NodeJs client**
> #* +Instruction+:
> https://github.com/nobitlost/ignite/blob/ignite--docs/modules/platforms/nodejs/README.md#publish-ignite-nodejs-client-on-npmjscom-instruction
> #* +Uploaded+: https://www.npmjs.com/package/apache-ignite-client
> # **PHP client**
> #* +Instruction+:
> https://github.com/nobitlost/ignite/blob/ignite-7783-docs/modules/platforms/php/README.md#release-the-client-in-the-php-package-repository-instruction
> {panel}
> Cannot be uploaded on Packagist as the client should be in a dedicated
> repository for that -
> https://issues.apache.org/jira/browse/IGNITE-7783?focusedCommentId=16595476&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16595476
> Installation from the sources works.
> {panel}
> # **Python client**
> I have already registered the package `pyignite` on PyPI[1]. The person who
> is going to take the responsibility of maintaining it should create an
> account on PyPI and mail me in private, so that I can grant them the
> necessary rights. They also must install twine[3].
> The process of packaging is well described in the packaging tutorial[2]. In
> the nutshell, the maintainer must do the following:
> ## Clone/pull the sources from the git repository,
> ## Enter the directory in which the `setup.py` is resides (“the setup
> directory”), in our case it is `modules/platforms/python`.
> ## Create the packages with the command `python3 setup.py sdist bdist_wheel`.
> The packages will be created in `modules/platforms/python/dist` folder.
> ## Upload packages with twine: `twine upload dist/*`.
> It is very useful to have a dedicated Python virtual environment prepared to
> perform steps 3-4. Just do an editable install of `pyignite` into that
> environment from the setup directory: `pip3 install -e .` You can also
> install twine (`pip install twine`) in it.
> Consider also making a `.pypirc` file to save time on logging in to PyPI.
> Newest version of `twine` is said to support keyrings on Linux and Mac, but I
> have not tried this yet.
> [1] https://pypi.org/project/pyignite/
> [2] https://packaging.python.org/tutorials/packaging-projects/
> [3] https://twine.readthedocs.io/en/latest/
> Some other notes on PyPI and versioning.
> - The package version is located in the `setup.py`, it is a `version`
> argument of the `setuptools.setup()` function. Editing the `setup.py` is the
> only way to set the package version.
> - You absolutely can not replace a package in PyPI (hijacking prevention). If
> you have published the package by mistake, all you can do is delete the
> unwanted package, increment the version counter in `setup.py`, and try again.
> - If you upload the package through the web interface of PyPI (without
> twine), the package description will be garbled. Web interface does not
> support markdown.
> Anyway, I would like to join in the congratulations on successful release.
> Kudos to the team.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12265) JavaDoc doesn't have documentation for the org.apache.ignite.client package
[
https://issues.apache.org/jira/browse/IGNITE-12265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12265:
--
Fix Version/s: 2.8
> JavaDoc doesn't have documentation for the org.apache.ignite.client package
> ---
>
> Key: IGNITE-12265
> URL: https://issues.apache.org/jira/browse/IGNITE-12265
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Mekhanikov
>Assignee: Alexey Goncharuk
>Priority: Major
> Labels: newbie
> Fix For: 2.8
>
>
> JavaDoc published on the website doesn't have documentation for the
> {{org.apache.ignite.client}} package. Link to the website:
> [https://ignite.apache.org/releases/2.7.6/javadoc/]
> A lack of {{package-info.java}} file or exclusion from the
> {{maven-javadoc-plugin}} in the root {{pom.xml}} may be the reason.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12265) JavaDoc doesn't have documentation for the org.apache.ignite.client package
[
https://issues.apache.org/jira/browse/IGNITE-12265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946801#comment-16946801
]
Alexey Goncharuk commented on IGNITE-12265:
---
Fixed both the exclude and {{package-info.java}} issues, verified the javadoc
generation locally.
> JavaDoc doesn't have documentation for the org.apache.ignite.client package
> ---
>
> Key: IGNITE-12265
> URL: https://issues.apache.org/jira/browse/IGNITE-12265
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Mekhanikov
>Assignee: Alexey Goncharuk
>Priority: Major
> Labels: newbie
> Fix For: 2.8
>
>
> JavaDoc published on the website doesn't have documentation for the
> {{org.apache.ignite.client}} package. Link to the website:
> [https://ignite.apache.org/releases/2.7.6/javadoc/]
> A lack of {{package-info.java}} file or exclusion from the
> {{maven-javadoc-plugin}} in the root {{pom.xml}} may be the reason.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12269) [ML] The method printTree is corrupted on the example
Alexey Zinoviev created IGNITE-12269: Summary: [ML] The method printTree is corrupted on the example Key: IGNITE-12269 URL: https://issues.apache.org/jira/browse/IGNITE-12269 Project: Ignite Issue Type: Bug Components: ml Affects Versions: 2.8 Reporter: Alexey Zinoviev Assignee: Alexey Zinoviev Fix For: 2.8 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12269) [ML] The method printTree is corrupted on the example
[
https://issues.apache.org/jira/browse/IGNITE-12269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Zinoviev updated IGNITE-12269:
-
Description:
This example
{code:java}
package org.apache.ignite.examples.ml.tutorial;
import java.io.FileNotFoundException;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.Ignition;
import org.apache.ignite.ml.composition.ModelsComposition;
import
org.apache.ignite.ml.composition.boosting.convergence.mean.MeanAbsValueConvergenceCheckerFactory;
import
org.apache.ignite.ml.composition.boosting.convergence.median.MedianOfMedianConvergenceCheckerFactory;
import org.apache.ignite.ml.dataset.feature.extractor.Vectorizer;
import org.apache.ignite.ml.dataset.feature.extractor.impl.DummyVectorizer;
import org.apache.ignite.ml.math.primitives.vector.Vector;
import org.apache.ignite.ml.preprocessing.Preprocessor;
import org.apache.ignite.ml.preprocessing.encoding.EncoderTrainer;
import org.apache.ignite.ml.preprocessing.encoding.EncoderType;
import org.apache.ignite.ml.preprocessing.imputing.ImputerTrainer;
import org.apache.ignite.ml.preprocessing.minmaxscaling.MinMaxScalerTrainer;
import org.apache.ignite.ml.preprocessing.normalization.NormalizationTrainer;
import org.apache.ignite.ml.selection.scoring.evaluator.Evaluator;
import org.apache.ignite.ml.selection.scoring.metric.MetricName;
import org.apache.ignite.ml.selection.split.TrainTestDatasetSplitter;
import org.apache.ignite.ml.selection.split.TrainTestSplit;
import org.apache.ignite.ml.trainers.DatasetTrainer;
import org.apache.ignite.ml.tree.boosting.GDBBinaryClassifierOnTreesTrainer;
/**
* {@link MinMaxScalerTrainer} and {@link NormalizationTrainer} are used in
this example due to different values
* distribution in columns and rows.
*
* Code in this example launches Ignite grid and fills the cache with test data
(based on Titanic passengers data).
*
* After that it defines preprocessors that extract features from an upstream
data and perform other desired changes
* over the extracted data, including the scaling.
*
* Then, it trains the model based on the processed data using decision tree
classification.
*
* Finally, this example uses {@link Evaluator} functionality to compute
metrics from predictions.
*/
public class Step_11_Boosting {
/**
* Run example.
*/
public static void main(String[] args) {
System.out.println();
System.out.println(">>> Tutorial step 11 (Boosting) example started.");
try (Ignite ignite =
Ignition.start("examples/config/example-ignite.xml")) {
try {
IgniteCache dataCache =
TitanicUtils.readPassengers(ignite);
// Extracts "pclass", "sibsp", "parch", "sex", "embarked",
"age", "fare".
final Vectorizer vectorizer
= new DummyVectorizer(0, 3, 4, 5, 6, 8,
10).labeled(1);
TrainTestSplit split = new
TrainTestDatasetSplitter()
.split(0.75);
Preprocessor strEncoderPreprocessor = new
EncoderTrainer()
.withEncoderType(EncoderType.STRING_ENCODER)
.withEncodedFeature(1)
.withEncodedFeature(6) // <--- Changed index here.
.fit(ignite,
dataCache,
vectorizer
);
Preprocessor imputingPreprocessor = new
ImputerTrainer()
.fit(ignite,
dataCache,
strEncoderPreprocessor
);
Preprocessor minMaxScalerPreprocessor = new
MinMaxScalerTrainer()
.fit(
ignite,
dataCache,
imputingPreprocessor
);
Preprocessor normalizationPreprocessor = new
NormalizationTrainer()
.withP(1)
.fit(
ignite,
dataCache,
minMaxScalerPreprocessor
);
// Create classification trainer.
DatasetTrainer trainer = new
GDBBinaryClassifierOnTreesTrainer(0.01, 1000, 10, 0.)
.withCheckConvergenceStgyFactory(new
MedianOfMedianConvergenceCheckerFactory(0.01));
// Train decision tree model.
ModelsComposition mdl = trainer.fit(
ignite,
dataCache,
split.getTrainFilter(),
normalizationPreprocessor
);
System.out.println("\n>>> Trained model: " + mdl);
double accuracy = Evaluator.evaluate(
dataCache,
split.getTestFilter(),
mdl,
normalizat
[jira] [Updated] (IGNITE-12269) [ML] The method printTree is corrupted on the example
[
https://issues.apache.org/jira/browse/IGNITE-12269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Zinoviev updated IGNITE-12269:
-
Priority: Critical (was: Major)
> [ML] The method printTree is corrupted on the example
> -
>
> Key: IGNITE-12269
> URL: https://issues.apache.org/jira/browse/IGNITE-12269
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.8
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Critical
> Labels: await
> Fix For: 2.8
>
>
> This example
> {code:java}
> package org.apache.ignite.examples.ml.tutorial;
> import java.io.FileNotFoundException;
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.ml.composition.ModelsComposition;
> import
> org.apache.ignite.ml.composition.boosting.convergence.mean.MeanAbsValueConvergenceCheckerFactory;
> import
> org.apache.ignite.ml.composition.boosting.convergence.median.MedianOfMedianConvergenceCheckerFactory;
> import org.apache.ignite.ml.dataset.feature.extractor.Vectorizer;
> import org.apache.ignite.ml.dataset.feature.extractor.impl.DummyVectorizer;
> import org.apache.ignite.ml.math.primitives.vector.Vector;
> import org.apache.ignite.ml.preprocessing.Preprocessor;
> import org.apache.ignite.ml.preprocessing.encoding.EncoderTrainer;
> import org.apache.ignite.ml.preprocessing.encoding.EncoderType;
> import org.apache.ignite.ml.preprocessing.imputing.ImputerTrainer;
> import org.apache.ignite.ml.preprocessing.minmaxscaling.MinMaxScalerTrainer;
> import org.apache.ignite.ml.preprocessing.normalization.NormalizationTrainer;
> import org.apache.ignite.ml.selection.scoring.evaluator.Evaluator;
> import org.apache.ignite.ml.selection.scoring.metric.MetricName;
> import org.apache.ignite.ml.selection.split.TrainTestDatasetSplitter;
> import org.apache.ignite.ml.selection.split.TrainTestSplit;
> import org.apache.ignite.ml.trainers.DatasetTrainer;
> import org.apache.ignite.ml.tree.boosting.GDBBinaryClassifierOnTreesTrainer;
> /**
> * {@link MinMaxScalerTrainer} and {@link NormalizationTrainer} are used in
> this example due to different values
> * distribution in columns and rows.
> *
> * Code in this example launches Ignite grid and fills the cache with test
> data (based on Titanic passengers data).
> *
> * After that it defines preprocessors that extract features from an upstream
> data and perform other desired changes
> * over the extracted data, including the scaling.
> *
> * Then, it trains the model based on the processed data using decision tree
> classification.
> *
> * Finally, this example uses {@link Evaluator} functionality to compute
> metrics from predictions.
> */
> public class Step_11_Boosting {
> /**
> * Run example.
> */
> public static void main(String[] args) {
> System.out.println();
> System.out.println(">>> Tutorial step 11 (Boosting) example
> started.");
> try (Ignite ignite =
> Ignition.start("examples/config/example-ignite.xml")) {
> try {
> IgniteCache dataCache =
> TitanicUtils.readPassengers(ignite);
> // Extracts "pclass", "sibsp", "parch", "sex", "embarked",
> "age", "fare".
> final Vectorizer vectorizer
> = new DummyVectorizer(0, 3, 4, 5, 6, 8,
> 10).labeled(1);
> TrainTestSplit split = new
> TrainTestDatasetSplitter()
> .split(0.75);
> Preprocessor strEncoderPreprocessor = new
> EncoderTrainer()
> .withEncoderType(EncoderType.STRING_ENCODER)
> .withEncodedFeature(1)
> .withEncodedFeature(6) // <--- Changed index here.
> .fit(ignite,
> dataCache,
> vectorizer
> );
> Preprocessor imputingPreprocessor = new
> ImputerTrainer()
> .fit(ignite,
> dataCache,
> strEncoderPreprocessor
> );
> Preprocessor minMaxScalerPreprocessor = new
> MinMaxScalerTrainer()
> .fit(
> ignite,
> dataCache,
> imputingPreprocessor
> );
> Preprocessor normalizationPreprocessor = new
> NormalizationTrainer()
> .withP(1)
> .fit(
> ignite,
> dataCache,
> minMaxScalerPreprocessor
> );
> // Create classification trainer.
>
[jira] [Updated] (IGNITE-896) Configuration inconsistency
[ https://issues.apache.org/jira/browse/IGNITE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-896: --- Fix Version/s: (was: 2.8) 2.9 > Configuration inconsistency > --- > > Key: IGNITE-896 > URL: https://issues.apache.org/jira/browse/IGNITE-896 > Project: Ignite > Issue Type: Bug > Components: general >Affects Versions: sprint-5 >Reporter: Valentin Kulichenko >Assignee: Nikolai Kulagin >Priority: Minor > Labels: Usability > Fix For: 2.9 > > > I noticed that some entities on cache configuration are configured via > factories, while others are set directly. For example, we use factory for > ExpiryPolicy, but not for EvictionPolicy, which looks inconsistent. Since > factory-based approach comes from JCache, I think we should use it wherever > possible. > Here is the list of settings that need to be fixed: > * Affinity > * AffinityMapper > * EvictionFilter > * EvictionPolicy > * CacheInterceptor > * TopologyValidator > Need to add new configuration properties that use factories and deprecate old > ones (do not remove for compatibility). > Also need to check other configuration beans (list above is for cache config > only). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11330) Partition demander should skip entries for invalid partition.
[ https://issues.apache.org/jira/browse/IGNITE-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11330: - Fix Version/s: (was: 2.8) > Partition demander should skip entries for invalid partition. > - > > Key: IGNITE-11330 > URL: https://issues.apache.org/jira/browse/IGNITE-11330 > Project: Ignite > Issue Type: Bug > Components: cache >Reporter: Andrey Mashenkov >Priority: Major > Labels: performance, rebalance > > For now Partition demander tries to process all entries for invalid partition > instead of switching to the next partition. > Looks like, this was introduced with a fix IGNITE-8955 to make checkpoint > locks fine-grained. > A "for" loop under checkpoint readlock in > GridDhtPartitionDemander.handleSupplyMessage() causes the issue. A flow can > breaks inner "for" loop in case of invalid partition, but seems an outer > "while" loop expected. > > To resolve this we can e.g. move CacheEntryInfo collection processing into > separate method and just exit from it if invalid partition is detected. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11303) Python thin client: best effort affinity
[ https://issues.apache.org/jira/browse/IGNITE-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11303: - Fix Version/s: (was: 2.8) 2.9 > Python thin client: best effort affinity > > > Key: IGNITE-11303 > URL: https://issues.apache.org/jira/browse/IGNITE-11303 > Project: Ignite > Issue Type: New Feature > Components: thin client >Affects Versions: 2.7 >Reporter: Dmitry Melnichuk >Assignee: Dmitry Melnichuk >Priority: Major > Labels: Python, Python3, iep-23, thin > Fix For: 2.9 > > > The goal is to implement > [IEP-23|https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients] > using background thread (`threading` module). -- This message was sent by Atlassian Jira (v8.3.4#803005)
