[jira] [Commented] (IGNITE-12548) Possible tx desync during recovery on near node left.
[ https://issues.apache.org/jira/browse/IGNITE-12548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018300#comment-17018300 ] Alexey Scherbakov commented on IGNITE-12548: [~ivan.glukos] Can you take a look ? > Possible tx desync during recovery on near node left. > - > > Key: IGNITE-12548 > URL: https://issues.apache.org/jira/browse/IGNITE-12548 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.7.6 >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Blocker > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > The problem appears if a transaction is starting to rollback in PREPARED > state for some reason and concurrently near node is left triggering tx > recovery protocol. > Consider having two enlisted keys from different partitions mapped to > different nodes N1 and N2. > Due to race N1 local tx can be rolled back while N2 local tx is committed > breaking tx atomicity guarantee. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12546) Prevent partitions owned by other nodes switch their state to MOVING due to counter difference on node join.
[ https://issues.apache.org/jira/browse/IGNITE-12546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018301#comment-17018301 ] Alexey Scherbakov commented on IGNITE-12546: [~ivan.glukos] Can you take a look ? > Prevent partitions owned by other nodes switch their state to MOVING due to > counter difference on node join. > > > Key: IGNITE-12546 > URL: https://issues.apache.org/jira/browse/IGNITE-12546 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.7.6 >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > When node joins it's expected that MOVING partitions can only belong to > joining node. > But if somehow counters are desynced other nodes can switch state of owning > partitions to MOVING too, causing spurious rebalancing and assertions on > mapping to moving primary. > Possible solution: exclude other nodes partitions from switching state to > MOVING on node join. > This only affects persistent groups. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12548) Possible tx desync during recovery on near node left.
[
https://issues.apache.org/jira/browse/IGNITE-12548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018298#comment-17018298
]
Ignite TC Bot commented on IGNITE-12548:
{panel:title=Branch: [pull/7274/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4938854buildTypeId=IgniteTests24Java8_RunAll]
> Possible tx desync during recovery on near node left.
> -
>
> Key: IGNITE-12548
> URL: https://issues.apache.org/jira/browse/IGNITE-12548
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7.6
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Blocker
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The problem appears if a transaction is starting to rollback in PREPARED
> state for some reason and concurrently near node is left triggering tx
> recovery protocol.
> Consider having two enlisted keys from different partitions mapped to
> different nodes N1 and N2.
> Due to race N1 local tx can be rolled back while N2 local tx is committed
> breaking tx atomicity guarantee.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12546) Prevent partitions owned by other nodes switch their state to MOVING due to counter difference on node join.
[
https://issues.apache.org/jira/browse/IGNITE-12546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018299#comment-17018299
]
Ignite TC Bot commented on IGNITE-12546:
{panel:title=Branch: [pull/7273/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4938725buildTypeId=IgniteTests24Java8_RunAll]
> Prevent partitions owned by other nodes switch their state to MOVING due to
> counter difference on node join.
>
>
> Key: IGNITE-12546
> URL: https://issues.apache.org/jira/browse/IGNITE-12546
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.7.6
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When node joins it's expected that MOVING partitions can only belong to
> joining node.
> But if somehow counters are desynced other nodes can switch state of owning
> partitions to MOVING too, causing spurious rebalancing and assertions on
> mapping to moving primary.
> Possible solution: exclude other nodes partitions from switching state to
> MOVING on node join.
> This only affects persistent groups.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8532) [ML] GA Grid: Implement Roulette Wheel Selection
[ https://issues.apache.org/jira/browse/IGNITE-8532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018273#comment-17018273 ] Turik Campbell commented on IGNITE-8532: [~azinoviev], I will need to make an update to Genetic Algorithms section of Ignite ReadMe documentation in support of this ticket. Please advise on when https://apacheignite.readme.io/v2.8/docs will be available so I may update accordingly. > [ML] GA Grid: Implement Roulette Wheel Selection > > > Key: IGNITE-8532 > URL: https://issues.apache.org/jira/browse/IGNITE-8532 > Project: Ignite > Issue Type: New Feature > Components: ml >Reporter: Turik Campbell >Assignee: Yury Babak >Priority: Minor > Labels: genetics > Fix For: 2.8 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Roulette Wheel Selection Method is a type of selection used for selecting > potentially useful solutions for recombination. This selection method gives > more chances of selection to the better performing chromosomes. > IE: > https://en.wikipedia.org/wiki/Fitness_proportionate_selection -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12014) Getting affinity for topology version earlier than affinity is calculated for system cache
[
https://issues.apache.org/jira/browse/IGNITE-12014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018226#comment-17018226
]
Alexey Scherbakov commented on IGNITE-12014:
The root cause seems to be a race between cache start and accessing cache while
topology version is not changed yet.
Reproducible on any async tx op.
> Getting affinity for topology version earlier than affinity is calculated
> for system cache
> ---
>
> Key: IGNITE-12014
> URL: https://issues.apache.org/jira/browse/IGNITE-12014
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7.5
>Reporter: PetrovMikhail
>Assignee: Ilya Kasnacheev
>Priority: Major
> Attachments: JcacheExchangeAwaitTest.java
>
>
> The following exception was being caught occasionally on a big cluster (128
> nodes) after it's activation and concurrent Ignite#reentrantLock() method
> call from different nodes. (On 16 nodes cluster this exception was never
> detected with no code change).
> It's attached a presumptive reproducer of that problem which stable fails
> with the specified exception.
> {code:java}
> java.lang.IllegalStateException: Getting affinity for topology version
> earlier than affinity is calculated [locNode=TcpDiscoveryNode
> [id=cf397493-7528-46dc-bc5a-444f9d51, consistentId=127.0.0.1:47501,
> addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47501],
> discPort=47501, order=2, intOrder=2, lastExchangeTime=1564050248387,
> loc=true, ver=2.8.0#20190725-sha1:, isClient=false],
> grp=default-volatile-ds-group, topVer=AffinityTopologyVersion [topVer=2,
> minorTopVer=1], lastAffChangeTopVer=AffinityTopologyVersion [topVer=2,
> minorTopVer=1], head=AffinityTopologyVersion [topVer=2, minorTopVer=2],
> history=[AffinityTopologyVersion [topVer=2, minorTopVer=2]]]
> at
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.cachedAffinity(GridAffinityAssignmentCache.java:802)
> at
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.cachedAffinity(GridAffinityAssignmentCache.java:749)
> at
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.nodes(GridAffinityAssignmentCache.java:657)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.nodesByPartition(GridCacheAffinityManager.java:227)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primaryByPartition(GridCacheAffinityManager.java:273)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primaryByKey(GridCacheAffinityManager.java:264)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primaryByKey(GridCacheAffinityManager.java:288)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedCache.entryExx(GridDhtColocatedCache.java:161)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.entryEx(GridNearTxLocal.java:4470)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistRead(GridNearTxLocal.java:2709)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.getAllAsync(GridNearTxLocal.java:2188)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedCache$4.op(GridDhtColocatedCache.java:204)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter$AsyncOp.op(GridCacheAdapter.java:5644)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.asyncOp(GridCacheAdapter.java:4561)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.colocated.GridDhtColocatedCache.getAsync(GridDhtColocatedCache.java:202)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:4842)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.repairableGet(GridCacheAdapter.java:4808)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.get(GridCacheAdapter.java:1480)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.get(GridCacheProxyImpl.java:396)
> at
> org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:561)
> at
> org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor$4.applyx(DataStructuresProcessor.java:556)
> at
> org.apache.ignite.internal.processors.datastructures.DataStructuresProcessor.retryTopologySafe(DataStructuresProcessor.java:1664)
> at
>
[jira] [Commented] (IGNITE-12227) Default auto-adjust baseline enabled flag calculated incorrectly in some cases
[
https://issues.apache.org/jira/browse/IGNITE-12227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018135#comment-17018135
]
Ignite TC Bot commented on IGNITE-12227:
{panel:title=Branch: [pull/7150/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4938318buildTypeId=IgniteTests24Java8_RunAll]
> Default auto-adjust baseline enabled flag calculated incorrectly in some cases
> --
>
> Key: IGNITE-12227
> URL: https://issues.apache.org/jira/browse/IGNITE-12227
> Project: Ignite
> Issue Type: Bug
>Reporter: Anton Kalashnikov
>Assignee: Anton Kalashnikov
>Priority: Blocker
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> baselineAutoAdjustEnabled can be been different on different nodes because of
> the calculation of default value happening locally on each node and including
> only local configuration. It issue can happen by the following reasons:
> * If IGNITE_BASELINE_AUTO_ADJUST_ENABLED flag set to a different value on
> different nodes it leads to cluster hanging due to baseline calculation
> finishing with the unpredictable state on each node.
> * if cluster in mixed mode(included in-memory and persistent nodes) sometimes
> flag is set to a different value due to calculation doesn't consider remote
> nodes configuration.
> Possible solution(both points required):
> * Get rid of IGNITE_BASELINE_AUTO_ADJUST_ENABLED and replace it by the
> explicit call of IgniteCluster#baselineAutoAdjustEnabled where it
> required(test only).
> * Calculating default value on the first started node as early as
> possible(instead of activation) and this value always should be set to
> distributed metastorage(unlike it happening now). It means that instead of
> awaiting activation, the default value would be calculated by the first
> started node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12551) Partition desync if a partition is evicted then owned again and historically rebalanced
[ https://issues.apache.org/jira/browse/IGNITE-12551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018134#comment-17018134 ] Alexey Scherbakov commented on IGNITE-12551: Merged to master 5e95f65bbc1d3c24a0d094745ab5790be712feac > Partition desync if a partition is evicted then owned again and historically > rebalanced > --- > > Key: IGNITE-12551 > URL: https://issues.apache.org/jira/browse/IGNITE-12551 > Project: Ignite > Issue Type: Bug > Components: persistence >Affects Versions: 2.7.6 >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Fix For: 2.9 > > Time Spent: 20m > Remaining Estimate: 0h > > Where is a possibility of partition desync in the following scenario: > 1. Some partition is evicted with non zero counters. > 2. It is owned again and are going to be rebalanced. > 3. Some node in a grid has history for the partition defined by it's > (initial, current) counters pair. > In this scenario the partition will be historically rebalanced having only > partial data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12101) IgniteQueue.removeAll throws NPE
[
https://issues.apache.org/jira/browse/IGNITE-12101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Rakov updated IGNITE-12101:
Release Note: Fixed NullPointerException when IgniteQueue.removeAll is
called
> IgniteQueue.removeAll throws NPE
>
>
> Key: IGNITE-12101
> URL: https://issues.apache.org/jira/browse/IGNITE-12101
> Project: Ignite
> Issue Type: Bug
> Components: data structures
>Affects Versions: 2.5
>Reporter: Denis A. Magda
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> See more details here:
> https://stackoverflow.com/questions/57473783/ignite-2-5-ignitequeue-removeall-throwing-npe
> {noformat}
> 2019-08-09 18:18:39,241 ERROR [Inbound-Main-Pool-13] [TransactionId:
> e5b5bfe3-5246-4d54-a4d6-acd550240e13 Request ID - 27845] [ APP=Server,
> ACTION=APP_PROCESS, USER=tsgops ] ProcessWorkflowProcessor - Error while
> processing CLIENT process
> class org.apache.ignite.IgniteException: Failed to serialize object
> [typeName=LinkedList]
>at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:990)
>
>at
> org.apache.ignite.internal.processors.datastructures.GridCacheQueueAdapter$QueueIterator.remove(GridCacheQueueAdapter.java:687)
>
>at
> java.util.AbstractCollection.removeAll(AbstractCollection.java:376)
>at
> org.apache.ignite.internal.processors.datastructures.GridCacheQueueProxy.removeAll(GridCacheQueueProxy.java:180)
>
>at
> com.me.app.service.support.APPOrderProcessIgniteQueueService.removeAll(APPOrderProcessIgniteQueueService.java:63)
>
>at
> com.me.app.service.support.APPOrderContextProcessInputManager.removeAllFromCurrentProcessing(APPOrderContextProcessInputManager.java:201)
>
>at
> com.me.app.service.support.APPOrderContextProcessInputManager.lambda$removeAll$3(APPOrderContextProcessInputManager.java:100)
>
>at java.lang.Iterable.forEach(Iterable.java:75)
>at
> com.me.app.service.support.APPOrderContextProcessInputManager.removeAll(APPOrderContextProcessInputManager.java:100)
>
>at
> com.me.app.service.support.APPOrderContextProcessInputManager.removeAll(APPOrderContextProcessInputManager.java:90)
>
>at
> com.me.app.processor.support.ProcessWorkflowProcessor.processOrders(ProcessWorkflowProcessor.java:602)
>
>at
> com.me.app.processor.support.ProcessWorkflowProcessor.lambda$null$13(ProcessWorkflowProcessor.java:405)
>
>at java.util.HashMap.forEach(HashMap.java:1289)
>at
> com.me.app.processor.support.ProcessWorkflowProcessor.lambda$null$14(ProcessWorkflowProcessor.java:368)
>
>at java.util.HashMap.forEach(HashMap.java:1289)
>at
> com.me.app.processor.support.ProcessWorkflowProcessor.lambda$null$15(ProcessWorkflowProcessor.java:354)
>
>at java.util.HashMap.forEach(HashMap.java:1289)
>at
> com.me.app.processor.support.ProcessWorkflowProcessor.lambda$null$16(ProcessWorkflowProcessor.java:345)
>
>at java.util.HashMap.forEach(HashMap.java:1289)
>at
> com.me.app.processor.support.ProcessWorkflowProcessor.lambda$executeProcess$17(ProcessWorkflowProcessor.java:337)
>
>at java.util.HashMap.forEach(HashMap.java:1289)
>at
> com.me.app.processor.support.ProcessWorkflowProcessor.executeProcess(ProcessWorkflowProcessor.java:330)
>
>at
> com.me.app.processor.support.ProcessWorkflowProcessor.executeProcess(ProcessWorkflowProcessor.java:302)
>
>at
> com.me.app.processor.support.ProcessWorkflowProcessor.lambda$processProcessFromQueue$6(ProcessWorkflowProcessor.java:282)
>
>at
> com.me.app.locking.support.IgniteLockingService.execute(IgniteLockingService.java:39)
>
>at
> com.me.app.locking.support.IgniteLockingService.execute(IgniteLockingService.java:68)
>
>at
> com.me.app.processor.support.ProcessWorkflowProcessor.processProcessFromQueue(ProcessWorkflowProcessor.java:281)
>
>at
> com.me.app.facade.listener.support.APPProcessEventListener.listen(APPProcessEventListener.java:49)
>
>at
> com.me.app.facade.listener.support.APPProcessEventListener.listen(APPProcessEventListener.java:19)
>
>at
> com.me.app.common.listener.support.AbstractEventListener.onMessage(AbstractEventListener.java:44)
>
>at
> com.me.app.common.listener.support.AbstractEventListener$$FastClassBySpringCGLIB$$f1379f74.invoke()
>
>at
>
[jira] [Updated] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Kasnacheev updated IGNITE-6804: Reviewer: Nikolay Izhikov > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 50m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018091#comment-17018091 ] Ilya Kasnacheev commented on IGNITE-6804: - [~nizhikov] please review proposed fix. I have also fixed a few tests to not produce warnings, but far more exist. > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 50m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[
https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018089#comment-17018089
]
Ignite TC Bot commented on IGNITE-6804:
---
{panel:title=Branch: [pull/6976/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4936570buildTypeId=IgniteTests24Java8_RunAll]
> Print a warning if HashMap is passed into bulk update operations
>
>
> Key: IGNITE-6804
> URL: https://issues.apache.org/jira/browse/IGNITE-6804
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Reporter: Denis A. Magda
>Assignee: Ilya Kasnacheev
>Priority: Critical
> Labels: usability
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Ignite newcomers tend to stumble on deadlocks simply because the keys are
> passed in an unordered HashMap. Propose to do the following:
> * update bulk operations Java docs.
> * print out a warning if not SortedMap (e.g. HashMap,
> Weak/Identity/Concurrent/Linked HashMap etc) is passed into
> a bulk method (instead of SortedMap) and contains more than 1 element.
> However, we should make sure that we only print that warning once and not
> every time the API is called.
> * do not produce warning for explicit optimistic transactions
> More details are here:
> http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12545) Introduce listener interface for components to react to partition map exchange events
[ https://issues.apache.org/jira/browse/IGNITE-12545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-12545: - Fix Version/s: 2.9 > Introduce listener interface for components to react to partition map > exchange events > - > > Key: IGNITE-12545 > URL: https://issues.apache.org/jira/browse/IGNITE-12545 > Project: Ignite > Issue Type: Bug >Reporter: Ivan Rakov >Assignee: Ivan Rakov >Priority: Major > Fix For: 2.9 > > Time Spent: 20m > Remaining Estimate: 0h > > It would be handly to have listener interface for components that should > react to PME instead of just adding more and more calls to > GridDhtPartitionsExchangeFuture. > In general, there are four possible moments when a compnent can be notified: > on exchnage init (before and after topologies are updates and exchange latch > is acquired) and on exchange done (before and after readyTopVer is > incremented and user operations are unlocked). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12545) Introduce listener interface for components to react to partition map exchange events
[ https://issues.apache.org/jira/browse/IGNITE-12545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018059#comment-17018059 ] Maxim Muzafarov commented on IGNITE-12545: -- [~ivan.glukos] LGTM, let's proceed with merge. > Introduce listener interface for components to react to partition map > exchange events > - > > Key: IGNITE-12545 > URL: https://issues.apache.org/jira/browse/IGNITE-12545 > Project: Ignite > Issue Type: Bug >Reporter: Ivan Rakov >Assignee: Ivan Rakov >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > It would be handly to have listener interface for components that should > react to PME instead of just adding more and more calls to > GridDhtPartitionsExchangeFuture. > In general, there are four possible moments when a compnent can be notified: > on exchnage init (before and after topologies are updates and exchange latch > is acquired) and on exchange done (before and after readyTopVer is > incremented and user operations are unlocked). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12530) Pages list caching can cause IgniteOOME when checkpoint is triggered by "too many dirty pages" reason
[
https://issues.apache.org/jira/browse/IGNITE-12530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018062#comment-17018062
]
Aleksey Plekhanov commented on IGNITE-12530:
I've moved pages list cache limit to the database shared manager and added
second cache group to test. [~ivan.glukos], [~zstan] please have a look again.
> Pages list caching can cause IgniteOOME when checkpoint is triggered by "too
> many dirty pages" reason
> -
>
> Key: IGNITE-12530
> URL: https://issues.apache.org/jira/browse/IGNITE-12530
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Attachments: screenshot-1.png
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When a checkpoint is triggered, we need some amount of page memory to store
> pages list on-heap cache.
> If data region is too small, a checkpoint is triggered by "too many dirty
> pages" reason and pages list cache is rather big, we can get
> IgniteOutOfMemoryException.
> Reproducer:
> {code:java}
> @Override protected IgniteConfiguration getConfiguration(String name) throws
> Exception {
> IgniteConfiguration cfg = super.getConfiguration(name);
> cfg.setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new DataRegionConfiguration()
> .setPersistenceEnabled(true)
> .setMaxSize(50 * 1024 * 1024)
> ));
> return cfg;
> }
> @Test
> public void testUpdatesNotFittingIntoMemoryRegion() throws Exception {
> IgniteEx ignite = startGrid(0);
> ignite.cluster().active(true);
> ignite.getOrCreateCache(DEFAULT_CACHE_NAME);
> try (IgniteDataStreamer streamer =
> ignite.dataStreamer(DEFAULT_CACHE_NAME)) {
> for (int i = 0; i < 100_000; i++)
> streamer.addData(i, new byte[i % 2048]);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12535) Jdbc Thin: Add SSL CipherSuites support to JDBC thin client.
[ https://issues.apache.org/jira/browse/IGNITE-12535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018033#comment-17018033 ] Yury Gerzhedovich commented on IGNITE-12535: [~amashenkov] LGTM. Let's merge it. > Jdbc Thin: Add SSL CipherSuites support to JDBC thin client. > - > > Key: IGNITE-12535 > URL: https://issues.apache.org/jira/browse/IGNITE-12535 > Project: Ignite > Issue Type: Task > Components: thin client >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > We widely own Ignite SSL Factory implementation and allow (e.g. control.sh > tool) to pass cipher suites in most cases, but ThinClient. > Let's allow user to restrict cipher suites for ThinClient as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12049) Add user attributes to thin clients
[ https://issues.apache.org/jira/browse/IGNITE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17018017#comment-17018017 ] Alexey Scherbakov commented on IGNITE-12049: [~SomeFire] Right now I have no solution for this case. Maybe user classes should be serialized on thin client to byte[] using JDK marshaller and later deserialized on server side. I think it's time to go on dev list. Please describe the problem in details for each thin client type and the community will help to solve it. > Add user attributes to thin clients > --- > > Key: IGNITE-12049 > URL: https://issues.apache.org/jira/browse/IGNITE-12049 > Project: Ignite > Issue Type: Improvement >Reporter: Ryabov Dmitrii >Assignee: Ryabov Dmitrii >Priority: Minor > Time Spent: 1.5h > Remaining Estimate: 0h > > Add user attributes to thin clients (like node attributes for server nodes). > Make sure that custom authenticators can use these attributes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12049) Add user attributes to thin clients
[
https://issues.apache.org/jira/browse/IGNITE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017989#comment-17017989
]
Ryabov Dmitrii commented on IGNITE-12049:
-
[~ascherbakov],
You are right, {{GridClientConfiguration().setMarshaller(new
GridClientJdkMarshaller())}} works for GridClients, but other thin clients have
hardcoded binary optimized marshaller and I'm not sure that replacing
marshaller is good idea.
> Add user attributes to thin clients
> ---
>
> Key: IGNITE-12049
> URL: https://issues.apache.org/jira/browse/IGNITE-12049
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ryabov Dmitrii
>Assignee: Ryabov Dmitrii
>Priority: Minor
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Add user attributes to thin clients (like node attributes for server nodes).
> Make sure that custom authenticators can use these attributes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-12504) Auto-adjust breaks existing code, should be disabled by default
[
https://issues.apache.org/jira/browse/IGNITE-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Kalashnikov reassigned IGNITE-12504:
--
Assignee: Anton Kalashnikov
> Auto-adjust breaks existing code, should be disabled by default
> ---
>
> Key: IGNITE-12504
> URL: https://issues.apache.org/jira/browse/IGNITE-12504
> Project: Ignite
> Issue Type: Bug
> Components: cache, persistence
>Affects Versions: 2.8
>Reporter: Ilya Kasnacheev
>Assignee: Anton Kalashnikov
>Priority: Critical
> Fix For: 2.8
>
>
> We have automatic baseline adjustment now. However, it is 'on' by default,
> which means it breaks existing code. I see new exceptions when starting an
> existing project after bumping Ignite dependency version:
> {code}
> Caused by:
> org.apache.ignite.internal.processors.cluster.BaselineAdjustForbiddenException:
> Baseline auto-adjust is enabled, please turn-off it before try to adjust
> baseline manually
> {code}
> (Please see reproducer from attached UL discussion)
> I think we should disable auto-adjust by default, let people enable it when
> they see it fit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12535) Jdbc Thin: Add SSL CipherSuites support to JDBC thin client.
[ https://issues.apache.org/jira/browse/IGNITE-12535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017969#comment-17017969 ] Andrey Mashenkov commented on IGNITE-12535: --- [~jooger], please review. > Jdbc Thin: Add SSL CipherSuites support to JDBC thin client. > - > > Key: IGNITE-12535 > URL: https://issues.apache.org/jira/browse/IGNITE-12535 > Project: Ignite > Issue Type: Task > Components: thin client >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > We widely own Ignite SSL Factory implementation and allow (e.g. control.sh > tool) to pass cipher suites in most cases, but ThinClient. > Let's allow user to restrict cipher suites for ThinClient as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12545) Introduce listener interface for components to react to partition map exchange events
[
https://issues.apache.org/jira/browse/IGNITE-12545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017959#comment-17017959
]
Ignite TC Bot commented on IGNITE-12545:
{panel:title=Branch: [pull/7263/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4936336buildTypeId=IgniteTests24Java8_RunAll]
> Introduce listener interface for components to react to partition map
> exchange events
> -
>
> Key: IGNITE-12545
> URL: https://issues.apache.org/jira/browse/IGNITE-12545
> Project: Ignite
> Issue Type: Bug
>Reporter: Ivan Rakov
>Assignee: Ivan Rakov
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> It would be handly to have listener interface for components that should
> react to PME instead of just adding more and more calls to
> GridDhtPartitionsExchangeFuture.
> In general, there are four possible moments when a compnent can be notified:
> on exchnage init (before and after topologies are updates and exchange latch
> is acquired) and on exchange done (before and after readyTopVer is
> incremented and user operations are unlocked).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12551) Partition desync if a partition is evicted then owned again and historically rebalanced
[ https://issues.apache.org/jira/browse/IGNITE-12551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017952#comment-17017952 ] Ivan Rakov commented on IGNITE-12551: - [~ascherbakov] Please merge. > Partition desync if a partition is evicted then owned again and historically > rebalanced > --- > > Key: IGNITE-12551 > URL: https://issues.apache.org/jira/browse/IGNITE-12551 > Project: Ignite > Issue Type: Bug > Components: persistence >Affects Versions: 2.7.6 >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > Where is a possibility of partition desync in the following scenario: > 1. Some partition is evicted with non zero counters. > 2. It is owned again and are going to be rebalanced. > 3. Some node in a grid has history for the partition defined by it's > (initial, current) counters pair. > In this scenario the partition will be historically rebalanced having only > partial data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12547) Excessive AtomicLong instantiations leads to GC pressure.
[ https://issues.apache.org/jira/browse/IGNITE-12547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-12547: - Priority: Critical (was: Major) > Excessive AtomicLong instantiations leads to GC pressure. > -- > > Key: IGNITE-12547 > URL: https://issues.apache.org/jira/browse/IGNITE-12547 > Project: Ignite > Issue Type: Bug > Components: persistence >Affects Versions: 2.7.6 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Critical > Fix For: 2.9 > > Time Spent: 20m > Remaining Estimate: 0h > > Excessive AtomicLong usage (in FreeList), i.e. AtomicLong[] leads to GC > pressure and further starvation log messages, it would be more effective to > replace AtomicLong[] with AtomicLongArray. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12547) Excessive AtomicLong instantiations leads to GC pressure.
[ https://issues.apache.org/jira/browse/IGNITE-12547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-12547: - Ignite Flags: Release Notes Required > Excessive AtomicLong instantiations leads to GC pressure. > -- > > Key: IGNITE-12547 > URL: https://issues.apache.org/jira/browse/IGNITE-12547 > Project: Ignite > Issue Type: Bug > Components: persistence >Affects Versions: 2.7.6 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Critical > Fix For: 2.9 > > Time Spent: 20m > Remaining Estimate: 0h > > Excessive AtomicLong usage (in FreeList), i.e. AtomicLong[] leads to GC > pressure and further starvation log messages, it would be more effective to > replace AtomicLong[] with AtomicLongArray. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12547) Excessive AtomicLong instantiations leads to GC pressure.
[ https://issues.apache.org/jira/browse/IGNITE-12547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-12547: - Issue Type: Bug (was: Improvement) > Excessive AtomicLong instantiations leads to GC pressure. > -- > > Key: IGNITE-12547 > URL: https://issues.apache.org/jira/browse/IGNITE-12547 > Project: Ignite > Issue Type: Bug > Components: persistence >Affects Versions: 2.7.6 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Fix For: 2.9 > > Time Spent: 20m > Remaining Estimate: 0h > > Excessive AtomicLong usage (in FreeList), i.e. AtomicLong[] leads to GC > pressure and further starvation log messages, it would be more effective to > replace AtomicLong[] with AtomicLongArray. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12539) JAVADOC broken for hibernate 5.3 module
[
https://issues.apache.org/jira/browse/IGNITE-12539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017946#comment-17017946
]
Maxim Muzafarov commented on IGNITE-12539:
--
Merged to the master branch, cherry-picked to 2.8.
> JAVADOC broken for hibernate 5.3 module
> ---
>
> Key: IGNITE-12539
> URL: https://issues.apache.org/jira/browse/IGNITE-12539
> Project: Ignite
> Issue Type: Task
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Blocker
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code}
> [17:02:15]W: [Step 2/2] [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.7:run
> (javadoc-postprocessing-new) on project apache-ignite: An Ant BuildException
> has occured: Execution failed due to: Class doesn't have description in file:
> /opt/buildagent/work/7bc1c54bc719b67c/target/javadoc/core/org/apache/ignite/cache/hibernate/IgniteCollectionDataAccess.html
> [17:02:15]W: [Step 2/2] [ERROR] Class doesn't have description in file:
> /opt/buildagent/work/7bc1c54bc719b67c/target/javadoc/core/org/apache/ignite/cache/hibernate/IgniteDomainDataRegion.html
> [17:02:15]W: [Step 2/2] [ERROR] Class doesn't have description in file:
> /opt/buildagent/work/7bc1c54bc719b67c/target/javadoc/core/org/apache/ignite/cache/hibernate/IgniteEntityDataAccess.html
> [17:02:15]W: [Step 2/2] [ERROR] Class doesn't have description in file:
> /opt/buildagent/work/7bc1c54bc719b67c/target/javadoc/core/org/apache/ignite/cache/hibernate/IgniteNaturalIdDataAccess.html
> [17:02:15]W: [Step 2/2] [ERROR]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (IGNITE-12539) JAVADOC broken for hibernate 5.3 module
[
https://issues.apache.org/jira/browse/IGNITE-12539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov resolved IGNITE-12539.
--
Resolution: Fixed
> JAVADOC broken for hibernate 5.3 module
> ---
>
> Key: IGNITE-12539
> URL: https://issues.apache.org/jira/browse/IGNITE-12539
> Project: Ignite
> Issue Type: Task
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Blocker
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code}
> [17:02:15]W: [Step 2/2] [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-antrun-plugin:1.7:run
> (javadoc-postprocessing-new) on project apache-ignite: An Ant BuildException
> has occured: Execution failed due to: Class doesn't have description in file:
> /opt/buildagent/work/7bc1c54bc719b67c/target/javadoc/core/org/apache/ignite/cache/hibernate/IgniteCollectionDataAccess.html
> [17:02:15]W: [Step 2/2] [ERROR] Class doesn't have description in file:
> /opt/buildagent/work/7bc1c54bc719b67c/target/javadoc/core/org/apache/ignite/cache/hibernate/IgniteDomainDataRegion.html
> [17:02:15]W: [Step 2/2] [ERROR] Class doesn't have description in file:
> /opt/buildagent/work/7bc1c54bc719b67c/target/javadoc/core/org/apache/ignite/cache/hibernate/IgniteEntityDataAccess.html
> [17:02:15]W: [Step 2/2] [ERROR] Class doesn't have description in file:
> /opt/buildagent/work/7bc1c54bc719b67c/target/javadoc/core/org/apache/ignite/cache/hibernate/IgniteNaturalIdDataAccess.html
> [17:02:15]W: [Step 2/2] [ERROR]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12552) [IEP-35] Expose MetricRegistry to the public API
[ https://issues.apache.org/jira/browse/IGNITE-12552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017944#comment-17017944 ] Andrey N. Gura commented on IGNITE-12552: - [~nizhikov] [~agoncharuk] This fix doesn't solve issue because this interface itself exposes internal entities. Se also my reply on devlist where I describe other issues of this interface ( http://apache-ignite-developers.2346864.n4.nabble.com/Internal-classes-are-exposed-in-public-API-td45146.html#a45178 ) > [IEP-35] Expose MetricRegistry to the public API > > > Key: IGNITE-12552 > URL: https://issues.apache.org/jira/browse/IGNITE-12552 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.8 >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Blocker > Labels: IEP-35 > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > MetricRegistry is not a part of public API, but used in MetricExporter which > is the part of public API. > We should export MetricRegistry to the public API. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12551) Partition desync if a partition is evicted then owned again and historically rebalanced
[ https://issues.apache.org/jira/browse/IGNITE-12551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017940#comment-17017940 ] Alexey Scherbakov commented on IGNITE-12551: [~ivan.glukos] Can you take a look ? > Partition desync if a partition is evicted then owned again and historically > rebalanced > --- > > Key: IGNITE-12551 > URL: https://issues.apache.org/jira/browse/IGNITE-12551 > Project: Ignite > Issue Type: Bug > Components: persistence >Affects Versions: 2.7.6 >Reporter: Alexey Scherbakov >Assignee: Alexey Scherbakov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > Where is a possibility of partition desync in the following scenario: > 1. Some partition is evicted with non zero counters. > 2. It is owned again and are going to be rebalanced. > 3. Some node in a grid has history for the partition defined by it's > (initial, current) counters pair. > In this scenario the partition will be historically rebalanced having only > partial data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12551) Partition desync if a partition is evicted then owned again and historically rebalanced
[
https://issues.apache.org/jira/browse/IGNITE-12551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017939#comment-17017939
]
Ignite TC Bot commented on IGNITE-12551:
{panel:title=Branch: [pull/7270/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4938009buildTypeId=IgniteTests24Java8_RunAll]
> Partition desync if a partition is evicted then owned again and historically
> rebalanced
> ---
>
> Key: IGNITE-12551
> URL: https://issues.apache.org/jira/browse/IGNITE-12551
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Affects Versions: 2.7.6
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Where is a possibility of partition desync in the following scenario:
> 1. Some partition is evicted with non zero counters.
> 2. It is owned again and are going to be rebalanced.
> 3. Some node in a grid has history for the partition defined by it's
> (initial, current) counters pair.
> In this scenario the partition will be historically rebalanced having only
> partial data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12547) Excessive AtomicLong instantiations leads to GC pressure.
[ https://issues.apache.org/jira/browse/IGNITE-12547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Goncharuk updated IGNITE-12547: -- Release Note: Reduced GC pressure caused by excessive use of arrays of AtomicLong > Excessive AtomicLong instantiations leads to GC pressure. > -- > > Key: IGNITE-12547 > URL: https://issues.apache.org/jira/browse/IGNITE-12547 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.7.6 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Fix For: 2.9 > > Time Spent: 20m > Remaining Estimate: 0h > > Excessive AtomicLong usage (in FreeList), i.e. AtomicLong[] leads to GC > pressure and further starvation log messages, it would be more effective to > replace AtomicLong[] with AtomicLongArray. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12547) Excessive AtomicLong instantiations leads to GC pressure.
[
https://issues.apache.org/jira/browse/IGNITE-12547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017931#comment-17017931
]
Ignite TC Bot commented on IGNITE-12547:
{panel:title=Branch: [pull/7267/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4935131buildTypeId=IgniteTests24Java8_RunAll]
> Excessive AtomicLong instantiations leads to GC pressure.
> --
>
> Key: IGNITE-12547
> URL: https://issues.apache.org/jira/browse/IGNITE-12547
> Project: Ignite
> Issue Type: Improvement
> Components: persistence
>Affects Versions: 2.7.6
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Excessive AtomicLong usage (in FreeList), i.e. AtomicLong[] leads to GC
> pressure and further starvation log messages, it would be more effective to
> replace AtomicLong[] with AtomicLongArray.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12553) [IEP-35] public Java metric API
[ https://issues.apache.org/jira/browse/IGNITE-12553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-12553: - Priority: Blocker (was: Major) > [IEP-35] public Java metric API > --- > > Key: IGNITE-12553 > URL: https://issues.apache.org/jira/browse/IGNITE-12553 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Blocker > Labels: IEP-35 > > Right now, there is no simple way to get metrics values from the java code. > We need to create a java API to access metric values. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12553) [IEP-35] public Java metric API
[ https://issues.apache.org/jira/browse/IGNITE-12553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-12553: - Fix Version/s: 2.8 > [IEP-35] public Java metric API > --- > > Key: IGNITE-12553 > URL: https://issues.apache.org/jira/browse/IGNITE-12553 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Blocker > Labels: IEP-35 > Fix For: 2.8 > > > Right now, there is no simple way to get metrics values from the java code. > We need to create a java API to access metric values. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12530) Pages list caching can cause IgniteOOME when checkpoint is triggered by "too many dirty pages" reason
[
https://issues.apache.org/jira/browse/IGNITE-12530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017925#comment-17017925
]
Aleksey Plekhanov commented on IGNITE-12530:
I've found one problem, the offheap manager created per cache group, so the
limit will affect only one cache group, but not the entire data region. I think
the database shared manager is a better place to store limits. I will fix it
shortly.
> Pages list caching can cause IgniteOOME when checkpoint is triggered by "too
> many dirty pages" reason
> -
>
> Key: IGNITE-12530
> URL: https://issues.apache.org/jira/browse/IGNITE-12530
> Project: Ignite
> Issue Type: Bug
> Components: persistence
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Attachments: screenshot-1.png
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When a checkpoint is triggered, we need some amount of page memory to store
> pages list on-heap cache.
> If data region is too small, a checkpoint is triggered by "too many dirty
> pages" reason and pages list cache is rather big, we can get
> IgniteOutOfMemoryException.
> Reproducer:
> {code:java}
> @Override protected IgniteConfiguration getConfiguration(String name) throws
> Exception {
> IgniteConfiguration cfg = super.getConfiguration(name);
> cfg.setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new DataRegionConfiguration()
> .setPersistenceEnabled(true)
> .setMaxSize(50 * 1024 * 1024)
> ));
> return cfg;
> }
> @Test
> public void testUpdatesNotFittingIntoMemoryRegion() throws Exception {
> IgniteEx ignite = startGrid(0);
> ignite.cluster().active(true);
> ignite.getOrCreateCache(DEFAULT_CACHE_NAME);
> try (IgniteDataStreamer streamer =
> ignite.dataStreamer(DEFAULT_CACHE_NAME)) {
> for (int i = 0; i < 100_000; i++)
> streamer.addData(i, new byte[i % 2048]);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12448) Calcite integration. Communication protocol.
[ https://issues.apache.org/jira/browse/IGNITE-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Seliverstov updated IGNITE-12448: -- Reviewer: Roman Kondakov > Calcite integration. Communication protocol. > > > Key: IGNITE-12448 > URL: https://issues.apache.org/jira/browse/IGNITE-12448 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Igor Seliverstov >Assignee: Igor Seliverstov >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > We need to introduce a communication protocol between query fragments in > scope of query execution. > * Protocol should have Push semantics with back pressure ability > * Protocol have to provide ordering guaranties - the data batches processed > in same order they sent to remote node. > * Protocol have to provide delivery guaranties. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-12553) [IEP-35] public Java metric API
[ https://issues.apache.org/jira/browse/IGNITE-12553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov reassigned IGNITE-12553: Assignee: Nikolay Izhikov > [IEP-35] public Java metric API > --- > > Key: IGNITE-12553 > URL: https://issues.apache.org/jira/browse/IGNITE-12553 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > > Right now, there is no simple way to get metrics values from the java code. > We need to create a java API to access metric values. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12553) [IEP-35] public Java metric API
[ https://issues.apache.org/jira/browse/IGNITE-12553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-12553: - Labels: IEP-35 (was: ) > [IEP-35] public Java metric API > --- > > Key: IGNITE-12553 > URL: https://issues.apache.org/jira/browse/IGNITE-12553 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Priority: Major > Labels: IEP-35 > > Right now, there is no simple way to get metrics values from the java code. > We need to create a java API to access metric values. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12049) Add user attributes to thin clients
[ https://issues.apache.org/jira/browse/IGNITE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017880#comment-17017880 ] Alexey Scherbakov edited comment on IGNITE-12049 at 1/17/20 10:20 AM: -- [~SomeFire] I still do not understand the relation to peer class loading. Can we get rid of marshallercontext in this scenario? For example, using JdkMarshaller or BinaryMarshaller with compactFooter=false ? was (Author: ascherbakov): [~SomeFire] I still do not understand the relation with peer class loading. Can we get rid of marshallercontext in this scenario? For example, using JdkMarshaller or BinaryMarshaller with compactFooter=false ? > Add user attributes to thin clients > --- > > Key: IGNITE-12049 > URL: https://issues.apache.org/jira/browse/IGNITE-12049 > Project: Ignite > Issue Type: Improvement >Reporter: Ryabov Dmitrii >Assignee: Ryabov Dmitrii >Priority: Minor > Time Spent: 1.5h > Remaining Estimate: 0h > > Add user attributes to thin clients (like node attributes for server nodes). > Make sure that custom authenticators can use these attributes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12049) Add user attributes to thin clients
[ https://issues.apache.org/jira/browse/IGNITE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017880#comment-17017880 ] Alexey Scherbakov commented on IGNITE-12049: [~SomeFire] I still do not understand the relation with peer class loading. Can we get rid of marshallercontext in this scenario? For example, using JdkMarshaller or BinaryMarshaller with compactFooter=false ? > Add user attributes to thin clients > --- > > Key: IGNITE-12049 > URL: https://issues.apache.org/jira/browse/IGNITE-12049 > Project: Ignite > Issue Type: Improvement >Reporter: Ryabov Dmitrii >Assignee: Ryabov Dmitrii >Priority: Minor > Time Spent: 1.5h > Remaining Estimate: 0h > > Add user attributes to thin clients (like node attributes for server nodes). > Make sure that custom authenticators can use these attributes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12553) [IEP-35] public Java metric API
Nikolay Izhikov created IGNITE-12553: Summary: [IEP-35] public Java metric API Key: IGNITE-12553 URL: https://issues.apache.org/jira/browse/IGNITE-12553 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Right now, there is no simple way to get metrics values from the java code. We need to create a java API to access metric values. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12049) Add user attributes to thin clients
[
https://issues.apache.org/jira/browse/IGNITE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017834#comment-17017834
]
Ryabov Dmitrii commented on IGNITE-12049:
-
[~ascherbakov]
Before marshaller will write object, it will check class in
{{MarshallerContext}} to make sure that receiving node have this class too. But
{{MarshallerContext}} contains classes from {{META-INF/classnames.properties}}
only (and no classes from classpath). So, marshaller tries to send class
descriptor to node. But thin clients have no such mechanism.
> Add user attributes to thin clients
> ---
>
> Key: IGNITE-12049
> URL: https://issues.apache.org/jira/browse/IGNITE-12049
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ryabov Dmitrii
>Assignee: Ryabov Dmitrii
>Priority: Minor
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Add user attributes to thin clients (like node attributes for server nodes).
> Make sure that custom authenticators can use these attributes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12049) Add user attributes to thin clients
[ https://issues.apache.org/jira/browse/IGNITE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017830#comment-17017830 ] Alexey Scherbakov commented on IGNITE-12049: [~SomeFire] I do not understand how classloading is related to the issue. All classes are available in the same JVM in my test. For me it looks like marshaller issue. Can you explain in details ? > Add user attributes to thin clients > --- > > Key: IGNITE-12049 > URL: https://issues.apache.org/jira/browse/IGNITE-12049 > Project: Ignite > Issue Type: Improvement >Reporter: Ryabov Dmitrii >Assignee: Ryabov Dmitrii >Priority: Minor > Time Spent: 1.5h > Remaining Estimate: 0h > > Add user attributes to thin clients (like node attributes for server nodes). > Make sure that custom authenticators can use these attributes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12049) Add user attributes to thin clients
[ https://issues.apache.org/jira/browse/IGNITE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016979#comment-17016979 ] Ryabov Dmitrii edited comment on IGNITE-12049 at 1/17/20 9:12 AM: -- [~ascherbakov], may be we allow primitive types only? And add user classes later. UPD: I investigated the issue. Server nodes use discovery for loading classes, so, to enable classloading we need to add discovery manager into thin clients or make another classloading mechanism. I think these solutions are too heavy for lightweight thin clients. was (Author: somefire): [~ascherbakov], may be we allow primitive types only? And add user classes later. > Add user attributes to thin clients > --- > > Key: IGNITE-12049 > URL: https://issues.apache.org/jira/browse/IGNITE-12049 > Project: Ignite > Issue Type: Improvement >Reporter: Ryabov Dmitrii >Assignee: Ryabov Dmitrii >Priority: Minor > Time Spent: 1.5h > Remaining Estimate: 0h > > Add user attributes to thin clients (like node attributes for server nodes). > Make sure that custom authenticators can use these attributes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[
https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989749#comment-16989749
]
Vyacheslav Koptilin edited comment on IGNITE-6804 at 1/17/20 9:06 AM:
--
Hello [~nizhikov], [~avinogradov],
> Maybe we should deprecate putAll(Map) and introduce putAll(SortedMap) instead?
I don't think that {{putAll(Map)}} can be deprecated just because it is a part
of JSR-107.
> We should just perform instanceOf check inside the putAll() method if
> pessimistic.
-The user can use any kind of implementation of Map interface. It does not seem
a reliable solution.- Looks like this comment does not make sense (parameter
must extend/implement {{SortedMap}})
> This will affect atomics and optimistic transactions, which are resistant to
> this issue.
It must be done for the atomic cache as well. Please take a look at
{{GridDhtAtomicCache.updateAllAsyncInternal0}}
{code:java}
// If batch store update is enabled, we need to lock all entries.
// First, need to acquire locks on cache entries, then check filter.
List locked = lockEntries(req,
req.topologyVersion());;
{code}
was (Author: slava.koptilin):
Hello [~nizhikov], [~avinogradov],
> Maybe we should deprecate putAll(Map) and introduce putAll(SortedMap) instead?
I don't think that {{putAll(Map)}} can be deprecated just because it is a part
of JSR-109.
> We should just perform instanceOf check inside the putAll() method if
> pessimistic.
-The user can use any kind of implementation of Map interface. It does not seem
a reliable solution.- Looks like this comment does not make sense (parameter
must extend/implement {{SortedMap}})
> This will affect atomics and optimistic transactions, which are resistant to
> this issue.
It must be done for the atomic cache as well. Please take a look at
{{GridDhtAtomicCache.updateAllAsyncInternal0}}
{code:java}
// If batch store update is enabled, we need to lock all entries.
// First, need to acquire locks on cache entries, then check filter.
List locked = lockEntries(req,
req.topologyVersion());;
{code}
> Print a warning if HashMap is passed into bulk update operations
>
>
> Key: IGNITE-6804
> URL: https://issues.apache.org/jira/browse/IGNITE-6804
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Reporter: Denis A. Magda
>Assignee: Ilya Kasnacheev
>Priority: Critical
> Labels: usability
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Ignite newcomers tend to stumble on deadlocks simply because the keys are
> passed in an unordered HashMap. Propose to do the following:
> * update bulk operations Java docs.
> * print out a warning if not SortedMap (e.g. HashMap,
> Weak/Identity/Concurrent/Linked HashMap etc) is passed into
> a bulk method (instead of SortedMap) and contains more than 1 element.
> However, we should make sure that we only print that warning once and not
> every time the API is called.
> * do not produce warning for explicit optimistic transactions
> More details are here:
> http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12531) Cluster is unable to change BLT on 2.8 if storage was initially created on 2.7 or less
[ https://issues.apache.org/jira/browse/IGNITE-12531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-12531: Reviewer: Ivan Rakov > Cluster is unable to change BLT on 2.8 if storage was initially created on > 2.7 or less > -- > > Key: IGNITE-12531 > URL: https://issues.apache.org/jira/browse/IGNITE-12531 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.8 >Reporter: Ivan Rakov >Assignee: Vyacheslav Koptilin >Priority: Blocker > Fix For: 2.8 > > Attachments: TestBltChangeFail.java > > Time Spent: 10m > Remaining Estimate: 0h > > Due to bug in https://issues.apache.org/jira/browse/IGNITE-10348, after > storage migration from 2.7- to 2.8 any updates of metastorage are not > persisted. > S2R: > (on 2.7) > - Activate persistent cluster with 2 nodes > - Shutdown the cluster > (on 2.8) > - Start cluster with 2 nodes based on persistent storage from 2.7 > - Start 3rd node > - Change baseline > - Shutdown the cluster > - Start initial two nodes > - Start 3rd node (join is rejected: first two nodes has old BLT of two nodes, > 3rd node has new BLT of three nodes) > Reproducer is attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12531) Cluster is unable to change BLT on 2.8 if storage was initially created on 2.7 or less
[ https://issues.apache.org/jira/browse/IGNITE-12531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017822#comment-17017822 ] Ivan Rakov commented on IGNITE-12531: - [~slava.koptilin] looks good. > Cluster is unable to change BLT on 2.8 if storage was initially created on > 2.7 or less > -- > > Key: IGNITE-12531 > URL: https://issues.apache.org/jira/browse/IGNITE-12531 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.8 >Reporter: Ivan Rakov >Assignee: Vyacheslav Koptilin >Priority: Blocker > Fix For: 2.8 > > Attachments: TestBltChangeFail.java > > Time Spent: 10m > Remaining Estimate: 0h > > Due to bug in https://issues.apache.org/jira/browse/IGNITE-10348, after > storage migration from 2.7- to 2.8 any updates of metastorage are not > persisted. > S2R: > (on 2.7) > - Activate persistent cluster with 2 nodes > - Shutdown the cluster > (on 2.8) > - Start cluster with 2 nodes based on persistent storage from 2.7 > - Start 3rd node > - Change baseline > - Shutdown the cluster > - Start initial two nodes > - Start 3rd node (join is rejected: first two nodes has old BLT of two nodes, > 3rd node has new BLT of three nodes) > Reproducer is attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-12533) Remote security context tests refactoring.
[ https://issues.apache.org/jira/browse/IGNITE-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Garus reassigned IGNITE-12533: Assignee: Denis Garus > Remote security context tests refactoring. > -- > > Key: IGNITE-12533 > URL: https://issues.apache.org/jira/browse/IGNITE-12533 > Project: Ignite > Issue Type: Improvement >Reporter: Denis Garus >Assignee: Denis Garus >Priority: Major > > To make tests more readable and robust, we should use the > _AbstractRemoteSecurityContextCheckTest.Verifier#register(String)_ method in > all related tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
