[jira] [Assigned] (IGNITE-12365) Concurrent removeAll() on the same cache leads to deadlock
[ https://issues.apache.org/jira/browse/IGNITE-12365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mirza Aliev reassigned IGNITE-12365: Assignee: Mirza Aliev > Concurrent removeAll() on the same cache leads to deadlock > -- > > Key: IGNITE-12365 > URL: https://issues.apache.org/jira/browse/IGNITE-12365 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Mirza Aliev >Priority: Critical > Labels: stupid > Attachments: RemoveAllDeadlockTest.java, removeAll-deadlock.txt > > > That's because removeAll is implemented as invokeAll with HashSet parameter. > invokeAll + HashSet = 💀 > I will attach simple reproducer (please note that multiple invocations of > test is needed, typically 5-10) as well as thread dumps. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12121) Double checkpoint triggering due to incorrect place of update current checkpoint
[ https://issues.apache.org/jira/browse/IGNITE-12121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989952#comment-16989952 ] Maxim Muzafarov commented on IGNITE-12121: -- [~akalashnikov], [~DmitriyGovorukhin] Do we have a reproducer for this issue? > Double checkpoint triggering due to incorrect place of update current > checkpoint > > > Key: IGNITE-12121 > URL: https://issues.apache.org/jira/browse/IGNITE-12121 > Project: Ignite > Issue Type: Bug >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Major > Fix For: 2.8 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Double checkpoint triggering due to incorrect place of update current > checkpoint. This can lead to two ckeckpoints one by one if checkpoint trigger > was 'too many dirty pages'. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11857) Investigate performance drop after IGNITE-10078
[ https://issues.apache.org/jira/browse/IGNITE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989896#comment-16989896 ] Alexei Scherbakov commented on IGNITE-11857: [~alex_pl] Ok, let's proceed with Map version. > Investigate performance drop after IGNITE-10078 > --- > > Key: IGNITE-11857 > URL: https://issues.apache.org/jira/browse/IGNITE-11857 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Aleksey Plekhanov >Priority: Major > Attachments: ignite-config.xml, > run.properties.tx-optimistic-put-b-backup > > Time Spent: 20m > Remaining Estimate: 0h > > After IGNITE-10078 yardstick tests show performance drop up to 8% in some > scenarios: > * tx-optim-repRead-put-get > * tx-optimistic-put > * tx-putAll > Partially this is due new update counter implementation, but not only. > Investigation is required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12200) More informative assertion message at constructor of CachedDeploymentInfo (GridCacheDeploymentManager class)
[
https://issues.apache.org/jira/browse/IGNITE-12200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989882#comment-16989882
]
Maxim Muzafarov commented on IGNITE-12200:
--
[~cyberdemon] I've left a comment. Please, take a look.
> More informative assertion message at constructor of CachedDeploymentInfo
> (GridCacheDeploymentManager class)
>
>
> Key: IGNITE-12200
> URL: https://issues.apache.org/jira/browse/IGNITE-12200
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.5, 2.7.5
>Reporter: Dmitriy Sorokin
>Assignee: Dmitriy Sorokin
>Priority: Minor
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code:java}
> /**
> * @param sndId Sender.
> * @param ldrId Loader ID.
> * @param userVer User version.
> * @param depMode Deployment mode.
> * @param participants Participants.
> */
> private CachedDeploymentInfo(UUID sndId, IgniteUuid ldrId, String userVer,
> DeploymentMode depMode,
> Map participants) {
> assert sndId.equals(ldrId.globalId()) || participants != null;
> this.sndId = sndId;
> this.ldrId = ldrId;
> this.userVer = userVer;
> this.depMode = depMode;
> this.participants = participants == null || participants.isEmpty() ? null
> :
> new ConcurrentLinkedHashMap<>(participants);
> }
> {code}
> The code above may produce the following stacktrace, where AssertionError
> should contain more informative message for better root cause analysis:
> {noformat}
> 2019-09-17
> 18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
> Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
> msg=GridCacheQueryRequest [id=4922,
> cacheName=com.sbt.limits.data.entity.LimitTemplateV1Entity_DPL_union-module,
> type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=null,
> rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false,
> incMeta=false, all=false, keepBinary=true,
> subjId=4c071d12-325a-4bb1-a68d-cc910f636562, taskHash=0, part=-1,
> topVer=AffinityTopologyVersion [topVer=191, minorTopVer=0],
> super=GridCacheIdMessage [cacheId=-724666788]]]2019-09-17
> 18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
> Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
> msg=GridCacheQueryRequest [id=4922,
> cacheName=com.sbt.limits.data.entity.LimitTemplateV1Entity_DPL_union-module,
> type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=null,
> rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false,
> incMeta=false, all=false, keepBinary=true,
> subjId=4c071d12-325a-4bb1-a68d-cc910f636562, taskHash=0, part=-1,
> topVer=AffinityTopologyVersion [topVer=191, minorTopVer=0],
> super=GridCacheIdMessage [cacheId=-724666788]]]
> java.lang.AssertionError: null
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:918)
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:889)
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.p2pContext(GridCacheDeploymentManager.java:422)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1547)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:102)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:301)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-09-17
> 18:29:29.912[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][org.apache.ignite.Ignite]
[jira] [Resolved] (IGNITE-12421) Update pom depencencies to 2.9.0-SNAPSHOT version
[
https://issues.apache.org/jira/browse/IGNITE-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov resolved IGNITE-12421.
--
Resolution: Fixed
> Update pom depencencies to 2.9.0-SNAPSHOT version
> -
>
> Key: IGNITE-12421
> URL: https://issues.apache.org/jira/browse/IGNITE-12421
> Project: Ignite
> Issue Type: Task
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Since the {{ignite-2.8}} the release branch created all version dependencies
> must be updated to {{2.9.0-SNAPSHOT}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12421) Update pom depencencies to 2.9.0-SNAPSHOT version
[
https://issues.apache.org/jira/browse/IGNITE-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-12421:
-
Issue Type: Task (was: Bug)
> Update pom depencencies to 2.9.0-SNAPSHOT version
> -
>
> Key: IGNITE-12421
> URL: https://issues.apache.org/jira/browse/IGNITE-12421
> Project: Ignite
> Issue Type: Task
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Since the {{ignite-2.8}} the release branch created all version dependencies
> must be updated to {{2.9.0-SNAPSHOT}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12421) Update pom depencencies to 2.9.0-SNAPSHOT version
[
https://issues.apache.org/jira/browse/IGNITE-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-12421:
-
Ignite Flags: (was: Docs Required,Release Notes Required)
> Update pom depencencies to 2.9.0-SNAPSHOT version
> -
>
> Key: IGNITE-12421
> URL: https://issues.apache.org/jira/browse/IGNITE-12421
> Project: Ignite
> Issue Type: Task
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Since the {{ignite-2.8}} the release branch created all version dependencies
> must be updated to {{2.9.0-SNAPSHOT}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12421) Update pom depencencies to 2.9.0-SNAPSHOT version
[
https://issues.apache.org/jira/browse/IGNITE-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989877#comment-16989877
]
Ignite TC Bot commented on IGNITE-12421:
{panel:title=Branch: [pull/7106/head] Base: [master] : Possible Blockers
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}JCache TCK 1.1{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=4817643]]
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4817051&buildTypeId=IgniteTests24Java8_RunAll]
> Update pom depencencies to 2.9.0-SNAPSHOT version
> -
>
> Key: IGNITE-12421
> URL: https://issues.apache.org/jira/browse/IGNITE-12421
> Project: Ignite
> Issue Type: Bug
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Since the {{ignite-2.8}} the release branch created all version dependencies
> must be updated to {{2.9.0-SNAPSHOT}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-11857) Investigate performance drop after IGNITE-10078
[
https://issues.apache.org/jira/browse/IGNITE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989854#comment-16989854
]
Aleksey Plekhanov edited comment on IGNITE-11857 at 12/6/19 3:03 PM:
-
[~ascherbakov]
We can only change the type from {{long}} to {{int}} for delta values. There is
no boxing for delta values in the code.
Resident size for both structures is also relatively low.
{code:java}
System.out.println("Size: " +
GraphLayout.parseInstance(benchmark.partCntr).totalSize())
{code}
Old implementation: 1752 bytes
TreeMap (long delta): 2248 bytes
TreeMap (int delta): 2072 bytes
TreeMap: 1816 bytes
Even if we have 100 cache groups with 1000 partitions on each node it's about
30-50 extra megabytes of heap.
This test is synthetic, in real production cases I think there will be much
lower values.
TreeMap with int delta has a little drop compared to long deltas and map of
has a little drop compared to int deltas.
Overall comparison table:
||Implementation||Throughput||GC pressure||Footprint||
|Old implementation|4,338 ± 0,208 ops/us|25 182 056|1752|
|TreeMap (long delta)|6,089 ± 0,459 ops/us|11 828 272|2248|
|TreeMap (long delta)|5,905 ± 0,338 ops/us|11 436 760|2072|
|TreeMap|4,857 ± 0,318 ops/us|10 733 888|1816|
I think we should get "TreeMap (long delta)" version. What do you
think?
was (Author: alex_pl):
[~ascherbakov]
We can only change the type from {{long}} to {{int}} for delta values. There is
no boxing for delta values in the code.
Resident size for both structures is also relatively low.
{code:java}
System.out.println("Size: " +
GraphLayout.parseInstance(benchmark.partCntr).totalSize())
{code}
Old implementation: 1752 bytes
TreeMap (long delta): 2248 bytes
TreeMap (int delta): 2072 bytes
TreeMap: 1816 bytes
Even if we have 100 cache groups with 1000 partitions on each node it's about
30-50 extra megabytes of heap.
This test is synthetic, in real production cases I think there will be much
lower values.
TreeMap with int delta has a little drop compared to long deltas and map of
has a little drop compared to int deltas.
Overall comparison table:
||Implementation||Throughput||GC pressure||Footprint||
|Old implementation|4,338 ± 0,208 ops/us|25 182 056|1752|
|TreeMap (long delta)|6,089 ± 0,459 ops/us|11 828 272|2248|
|TreeMap (long delta)|5,905 ± 0,338 ops/us|11 436 760|2072|
|TreeMap|4,857 ± 0,318 ops/us|10 733 888|1816|
I think we should get "TreeMap (long delta)" version. What do you
think?
> Investigate performance drop after IGNITE-10078
> ---
>
> Key: IGNITE-11857
> URL: https://issues.apache.org/jira/browse/IGNITE-11857
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexei Scherbakov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Attachments: ignite-config.xml,
> run.properties.tx-optimistic-put-b-backup
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After IGNITE-10078 yardstick tests show performance drop up to 8% in some
> scenarios:
> * tx-optim-repRead-put-get
> * tx-optimistic-put
> * tx-putAll
> Partially this is due new update counter implementation, but not only.
> Investigation is required.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-11857) Investigate performance drop after IGNITE-10078
[
https://issues.apache.org/jira/browse/IGNITE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989854#comment-16989854
]
Aleksey Plekhanov commented on IGNITE-11857:
[~ascherbakov]
We can only change the type from {{long}} to {{int}} for delta values. There is
no boxing for delta values in the code.
Resident size for both structures is also relatively low.
{code:java}
System.out.println("Size: " +
GraphLayout.parseInstance(benchmark.partCntr).totalSize())
{code}
Old implementation: 1752 bytes
TreeMap (long delta): 2248 bytes
TreeMap (int delta): 2072 bytes
TreeMap: 1816 bytes
Even if we have 100 cache groups with 1000 partitions on each node it's about
30-50 extra megabytes of heap.
This test is synthetic, in real production cases I think there will be much
lower values.
TreeMap with int delta has a little drop compared to long deltas and map of
has a little drop compared to int deltas.
Overall comparison table:
||Implementation||Throughput||GC pressure||Footprint||
|Old implementation|4,338 ± 0,208 ops/us|25 182 056|1752|
|TreeMap (long delta)|6,089 ± 0,459 ops/us|11 828 272|2248|
|TreeMap (long delta)|5,905 ± 0,338 ops/us|11 436 760|2072|
|TreeMap|4,857 ± 0,318 ops/us|10 733 888|1816|
I think we should get "TreeMap (long delta)" version. What do you
think?
> Investigate performance drop after IGNITE-10078
> ---
>
> Key: IGNITE-11857
> URL: https://issues.apache.org/jira/browse/IGNITE-11857
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexei Scherbakov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Attachments: ignite-config.xml,
> run.properties.tx-optimistic-put-b-backup
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After IGNITE-10078 yardstick tests show performance drop up to 8% in some
> scenarios:
> * tx-optim-repRead-put-get
> * tx-optimistic-put
> * tx-putAll
> Partially this is due new update counter implementation, but not only.
> Investigation is required.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12422) Clean up GG-XXX internal ticket references from the code base.
[ https://issues.apache.org/jira/browse/IGNITE-12422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexei Scherbakov updated IGNITE-12422: --- Description: Replace with Apache Ignite equivalent if possible. Also it's desirable to implement checkstyle rule to prevent foreigh links in TODOs [1] [1] https://checkstyle.sourceforge.io/config_misc.html#TodoComment was:Replace with Apache Ignite equivalent if possible. > Clean up GG-XXX internal ticket references from the code base. > -- > > Key: IGNITE-12422 > URL: https://issues.apache.org/jira/browse/IGNITE-12422 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Alexei Scherbakov >Priority: Major > Fix For: 2.9 > > > Replace with Apache Ignite equivalent if possible. > Also it's desirable to implement checkstyle rule to prevent foreigh links in > TODOs [1] > [1] https://checkstyle.sourceforge.io/config_misc.html#TodoComment -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[
https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989749#comment-16989749
]
Vyacheslav Koptilin edited comment on IGNITE-6804 at 12/6/19 2:45 PM:
--
Hello [~nizhikov], [~avinogradov],
> Maybe we should deprecate putAll(Map) and introduce putAll(SortedMap) instead?
I don't think that {{putAll(Map)}} can be deprecated just because it is a part
of JSR-109.
> We should just perform instanceOf check inside the putAll() method if
> pessimistic.
-The user can use any kind of implementation of Map interface. It does not seem
a reliable solution.- Looks like this comment does not make sense (parameter
must extend/implement {{SortedMap}})
> This will affect atomics and optimistic transactions, which are resistant to
> this issue.
It must be done for the atomic cache as well. Please take a look at
{{GridDhtAtomicCache.updateAllAsyncInternal0}}
{code:java}
// If batch store update is enabled, we need to lock all entries.
// First, need to acquire locks on cache entries, then check filter.
List locked = lockEntries(req,
req.topologyVersion());;
{code}
was (Author: slava.koptilin):
Hello [~nizhikov], [~avinogradov],
> Maybe we should deprecate putAll(Map) and introduce putAll(SortedMap) instead?
I don't think that {{putAll(Map)}} can be deprecated just because it is a part
of JSR-109.
> We should just perform instanceOf check inside the putAll() method if
> pessimistic.
-The user can use any kind of implementation of Map interface. It does not seem
a reliable solution.- Look like this comment does not make sense (parameter
must extend/implement {{SortedMap}})
> This will affect atomics and optimistic transactions, which are resistant to
> this issue.
It must be done for the atomic cache as well. Please take a look at
{{GridDhtAtomicCache.updateAllAsyncInternal0}}
{code:java}
// If batch store update is enabled, we need to lock all entries.
// First, need to acquire locks on cache entries, then check filter.
List locked = lockEntries(req,
req.topologyVersion());;
{code}
> Print a warning if HashMap is passed into bulk update operations
>
>
> Key: IGNITE-6804
> URL: https://issues.apache.org/jira/browse/IGNITE-6804
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Reporter: Denis A. Magda
>Assignee: Ilya Kasnacheev
>Priority: Critical
> Labels: usability
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Ignite newcomers tend to stumble on deadlocks simply because the keys are
> passed in an unordered HashMap. Propose to do the following:
> * update bulk operations Java docs.
> * print out a warning if not SortedMap (e.g. HashMap,
> Weak/Identity/Concurrent/Linked HashMap etc) is passed into
> a bulk method (instead of SortedMap) and contains more than 1 element.
> However, we should make sure that we only print that warning once and not
> every time the API is called.
> * do not produce warning for explicit optimistic transactions
> More details are here:
> http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[
https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989749#comment-16989749
]
Vyacheslav Koptilin edited comment on IGNITE-6804 at 12/6/19 1:34 PM:
--
Hello [~nizhikov], [~avinogradov],
> Maybe we should deprecate putAll(Map) and introduce putAll(SortedMap) instead?
I don't think that {{putAll(Map)}} can be deprecated just because it is a part
of JSR-109.
> We should just perform instanceOf check inside the putAll() method if
> pessimistic.
-The user can use any kind of implementation of Map interface. It does not seem
a reliable solution.- Look like this comment does not make sense (parameter
must extend/implement {{SortedMap}})
> This will affect atomics and optimistic transactions, which are resistant to
> this issue.
It must be done for the atomic cache as well. Please take a look at
{{GridDhtAtomicCache.updateAllAsyncInternal0}}
{code:java}
// If batch store update is enabled, we need to lock all entries.
// First, need to acquire locks on cache entries, then check filter.
List locked = lockEntries(req,
req.topologyVersion());;
{code}
was (Author: slava.koptilin):
Hello [~nizhikov], [~avinogradov],
> Maybe we should deprecate putAll(Map) and introduce putAll(SortedMap) instead?
I don't think that {{putAll(Map)}} can be deprecated just because it is a part
of JSR-109.
> We should just perform instanceOf check inside the putAll() method if
> pessimistic.
The user can use any kind of implementation of Map interface. It does not seem
a reliable solution.
> This will affect atomics and optimistic transactions, which are resistant to
> this issue.
It must be done for the atomic cache as well. Please take a look at
{{GridDhtAtomicCache.updateAllAsyncInternal0}}
{code:java}
// If batch store update is enabled, we need to lock all entries.
// First, need to acquire locks on cache entries, then check filter.
List locked = lockEntries(req,
req.topologyVersion());;
{code}
> Print a warning if HashMap is passed into bulk update operations
>
>
> Key: IGNITE-6804
> URL: https://issues.apache.org/jira/browse/IGNITE-6804
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Reporter: Denis A. Magda
>Assignee: Ilya Kasnacheev
>Priority: Critical
> Labels: usability
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Ignite newcomers tend to stumble on deadlocks simply because the keys are
> passed in an unordered HashMap. Propose to do the following:
> * update bulk operations Java docs.
> * print out a warning if not SortedMap (e.g. HashMap,
> Weak/Identity/Concurrent/Linked HashMap etc) is passed into
> a bulk method (instead of SortedMap) and contains more than 1 element.
> However, we should make sure that we only print that warning once and not
> every time the API is called.
> * do not produce warning for explicit optimistic transactions
> More details are here:
> http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[
https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989749#comment-16989749
]
Vyacheslav Koptilin commented on IGNITE-6804:
-
Hello [~nizhikov], [~avinogradov],
> Maybe we should deprecate putAll(Map) and introduce putAll(SortedMap) instead?
I don't think that {{putAll(Map)}} can be deprecated just because it is a part
of JSR-109.
> We should just perform instanceOf check inside the putAll() method if
> pessimistic.
The user can use any kind of implementation of Map interface. It does not seem
a reliable solution.
> This will affect atomics and optimistic transactions, which are resistant to
> this issue.
It must be done for the atomic cache as well. Please take a look at
{{GridDhtAtomicCache.updateAllAsyncInternal0}}
{code:java}
// If batch store update is enabled, we need to lock all entries.
// First, need to acquire locks on cache entries, then check filter.
List locked = lockEntries(req,
req.topologyVersion());;
{code}
> Print a warning if HashMap is passed into bulk update operations
>
>
> Key: IGNITE-6804
> URL: https://issues.apache.org/jira/browse/IGNITE-6804
> Project: Ignite
> Issue Type: Improvement
> Components: cache
>Reporter: Denis A. Magda
>Assignee: Ilya Kasnacheev
>Priority: Critical
> Labels: usability
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Ignite newcomers tend to stumble on deadlocks simply because the keys are
> passed in an unordered HashMap. Propose to do the following:
> * update bulk operations Java docs.
> * print out a warning if not SortedMap (e.g. HashMap,
> Weak/Identity/Concurrent/Linked HashMap etc) is passed into
> a bulk method (instead of SortedMap) and contains more than 1 element.
> However, we should make sure that we only print that warning once and not
> every time the API is called.
> * do not produce warning for explicit optimistic transactions
> More details are here:
> http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-11857) Investigate performance drop after IGNITE-10078
[ https://issues.apache.org/jira/browse/IGNITE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989717#comment-16989717 ] Alexei Scherbakov edited comment on IGNITE-11857 at 12/6/19 1:10 PM: - [~alex_pl] You have only measured heap allocation (GC pressure) which seems to be very low for both implementation. You also should measure resident size of both structures. Long for value can be replaced with Integer because no tx with batch of size close to or larger than Integer.MAX_VALUE is viable. For most frequent use cases object creation will be handled by Integer boxing cache. I think I'm ok with proposed improvement, just make sure we couldn't do better. was (Author: ascherbakov): [~alex_pl] You have only measured heap allocation (GC pressure) which seems to be very low for both implementation. You also should measure resident size of both structures. Long for value can be replaced with Integer because no tx with batch of size close to or larger than Integer.MAX_VALUE is viable. For most frequent use cases object creation will be handled by Integer boxing cache. I think I'm ok this proposed improvement, just make sure we couldn't do better. > Investigate performance drop after IGNITE-10078 > --- > > Key: IGNITE-11857 > URL: https://issues.apache.org/jira/browse/IGNITE-11857 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Aleksey Plekhanov >Priority: Major > Attachments: ignite-config.xml, > run.properties.tx-optimistic-put-b-backup > > Time Spent: 20m > Remaining Estimate: 0h > > After IGNITE-10078 yardstick tests show performance drop up to 8% in some > scenarios: > * tx-optim-repRead-put-get > * tx-optimistic-put > * tx-putAll > Partially this is due new update counter implementation, but not only. > Investigation is required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-11857) Investigate performance drop after IGNITE-10078
[ https://issues.apache.org/jira/browse/IGNITE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989717#comment-16989717 ] Alexei Scherbakov edited comment on IGNITE-11857 at 12/6/19 12:39 PM: -- [~alex_pl] You have only measured heap allocation (GC pressure) which seems to be very low for both implementation. You also should measure resident size of both structures. Long for value can be replaced with Integer because no tx with batch of size close to or larger than Integer.MAX_VALUE is viable. For most frequent use cases object creation will be handled by Integer boxing cache. I think I'm ok this proposed improvement, just make sure we couldn't do better. was (Author: ascherbakov): [~alex_pl] You have only measured heap allocation (GC pressure) which seems to be very low for both implementation. You also should measure resident size of both structures. Long for value can be replaced with Integer because no tx with batch of size Integer.MAX_VALUE is viable. For most frequent use cases object creation will be handled by Integer boxing cache. I think I'm ok this proposed improvement, just make sure we couldn't do better. > Investigate performance drop after IGNITE-10078 > --- > > Key: IGNITE-11857 > URL: https://issues.apache.org/jira/browse/IGNITE-11857 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Aleksey Plekhanov >Priority: Major > Attachments: ignite-config.xml, > run.properties.tx-optimistic-put-b-backup > > Time Spent: 20m > Remaining Estimate: 0h > > After IGNITE-10078 yardstick tests show performance drop up to 8% in some > scenarios: > * tx-optim-repRead-put-get > * tx-optimistic-put > * tx-putAll > Partially this is due new update counter implementation, but not only. > Investigation is required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11857) Investigate performance drop after IGNITE-10078
[ https://issues.apache.org/jira/browse/IGNITE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989717#comment-16989717 ] Alexei Scherbakov commented on IGNITE-11857: [~alex_pl] You have only measured heap allocation (GC pressure) which seems to be very low for both implementation. You also should measure resident size of both structures. Long for value can be replaced with Integer because no tx with batch of size Integer.MAX_VALUE is viable. For most frequent use cases object creation will be handled by Integer boxing cache. I think I'm ok this proposed improvement, just make sure we couldn't do better. > Investigate performance drop after IGNITE-10078 > --- > > Key: IGNITE-11857 > URL: https://issues.apache.org/jira/browse/IGNITE-11857 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Aleksey Plekhanov >Priority: Major > Attachments: ignite-config.xml, > run.properties.tx-optimistic-put-b-backup > > Time Spent: 20m > Remaining Estimate: 0h > > After IGNITE-10078 yardstick tests show performance drop up to 8% in some > scenarios: > * tx-optim-repRead-put-get > * tx-optimistic-put > * tx-putAll > Partially this is due new update counter implementation, but not only. > Investigation is required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989716#comment-16989716 ] Anton Vinogradov commented on IGNITE-6804: -- [~nizhikov] This will affect atomics and optimistic transactions, which are resistant to this issue. > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 50m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989706#comment-16989706 ] Anton Vinogradov edited comment on IGNITE-6804 at 12/6/19 12:34 PM: Folks, A warning will be ignored. Javadoс will be ignored too. We should just perform instanceOf check inside the putAll() method if pessimistic. Such check will not drop overall performance since it will be evaluated on the client node. BTW, Can we perform same check by some annotation-based contract using static analysis? was (Author: avinogradov): Folks, A warning will be ignored. Javadoс will be ignored too. We should just perform instanceOf check inside the putAll() method if pessimistic. Such check will not drop overall performance since it will be evaluated on the client node. BTW, Can we perform same check by some annotation contract using static analysis? > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 50m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989707#comment-16989707 ] Nikolay Izhikov commented on IGNITE-6804: - [~avinogradov] Maybe we should deprecate putAll(Map) and introduce putAll(SortedMap) instead? > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 50m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989706#comment-16989706 ] Anton Vinogradov commented on IGNITE-6804: -- Folks, A warning will be ignored. Javadoс will be ignored too. We should just perform instanceOf check inside the putAll() method if pessimistic. Such check will not drop overall performance since it will be evaluated on the client node. BTW, Can we perform same check by some annotation contract using static analysis? > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 50m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[
https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989701#comment-16989701
]
Stanilovsky Evgeny edited comment on IGNITE-12374 at 12/6/19 12:29 PM:
---
[~yow] look, you can got some speed boost if :
1. replicated cache appropriate for your case :WITH
\"template=REPLICATED,CACHE_NAME=cdrtest_cacheR,WRITE_SYNCHRONIZATION_MODE=PRIMARY_SYNC\"
2. call run.sh with default 50 2000 with clear persistent base can give flaky
results cause some time need for inner structures initialization, partitions
etc, run first warmup run and second one will give you clear results.
3. can you change your key for example for Long ?
4. looks like 10k per one thread is impossible now.
looks like that`s all i can give you.
was (Author: zstan):
[~yow] look, you can got some speed boost if :
1. replicated cache appropriate for your case :WITH
\"template=REPLICATED,CACHE_NAME=cdrtest_cacheR,WRITE_SYNCHRONIZATION_MODE=PRIMARY_SYNC\"
2. call run.sh with default 50 2000 witch clear persistent base can give flaky
results cause some time need for inner structures initialization, partitions
etc, run first warmup run and second one will give you clear results.
3. can you change your key for example for Long ?
4. looks like 10k per one thread is impossible now.
looks like that`s all i can give you.
> Too low performance ~200TPS for single ODBC client
> --
>
> Key: IGNITE-12374
> URL: https://issues.apache.org/jira/browse/IGNITE-12374
> Project: Ignite
> Issue Type: Improvement
> Components: clients, odbc
>Affects Versions: 2.7.5
> Environment: Ignite server run on top of Kubernetes, with 2 server
> nodes, persistence enabled. Both CPU and RAM at server/client server is
> sufficient according to system reports.
>Reporter: swy
>Priority: Major
> Attachments: ignite-9bb53f34.0.log,
> ignite-logs-config-source-20191128.zip,
> ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample,
> odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png,
> profiling03.png, profling02.png, screenshot-1.png,
> snapshot-1574845275597.nps, threaddump-1573207804944.tdump,
> threaddump-1574845414243.tdump, values.yaml
>
>
> Hi, in our test ignite performance with ODBC connection is too bad to proceed
> with product integration. It is about ~200 TPS, each transaction with
> select+update operation.
> Please refer to attach sample program. It is just a simple test case.
> Based on the profiling most of the time consumed by sql execution. Please
> advice if the application did not do the right thing.
> Thank you.
> local ignite server, but the result same to remote container Ignite
> deployment too.
> {{cat /etc/apache-ignite/my-config.xml}}
> {noformat}
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd";>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration"/>
>
>
>
>
>
> class="org.apache.ignite.binary.BinaryBasicIdMapper">
>
>
>
>
>
>
>
> {noformat}
> {{g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include
> -I./ignite/common/os/linux/include -I./ignite/common/os/win/include
> -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample
> -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[
https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989701#comment-16989701
]
Stanilovsky Evgeny commented on IGNITE-12374:
-
[~yow] look, you can got some speed boost if :
1. replicated cache appropriate for your case :WITH
\"template=REPLICATED,CACHE_NAME=cdrtest_cacheR,WRITE_SYNCHRONIZATION_MODE=PRIMARY_SYNC\"
2. call run.sh with default 50 2000 witch clear persistent base can give flaky
results cause some time need for inner structures initialization, partitions
etc, run first warmup run and second one will give you clear results.
3. can you change your key for example for Long ?
4. looks like 10k per one thread is impossible now.
looks like that`s all i can give you.
> Too low performance ~200TPS for single ODBC client
> --
>
> Key: IGNITE-12374
> URL: https://issues.apache.org/jira/browse/IGNITE-12374
> Project: Ignite
> Issue Type: Improvement
> Components: clients, odbc
>Affects Versions: 2.7.5
> Environment: Ignite server run on top of Kubernetes, with 2 server
> nodes, persistence enabled. Both CPU and RAM at server/client server is
> sufficient according to system reports.
>Reporter: swy
>Priority: Major
> Attachments: ignite-9bb53f34.0.log,
> ignite-logs-config-source-20191128.zip,
> ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample,
> odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png,
> profiling03.png, profling02.png, screenshot-1.png,
> snapshot-1574845275597.nps, threaddump-1573207804944.tdump,
> threaddump-1574845414243.tdump, values.yaml
>
>
> Hi, in our test ignite performance with ODBC connection is too bad to proceed
> with product integration. It is about ~200 TPS, each transaction with
> select+update operation.
> Please refer to attach sample program. It is just a simple test case.
> Based on the profiling most of the time consumed by sql execution. Please
> advice if the application did not do the right thing.
> Thank you.
> local ignite server, but the result same to remote container Ignite
> deployment too.
> {{cat /etc/apache-ignite/my-config.xml}}
> {noformat}
>
> http://www.springframework.org/schema/beans";
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd";>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration"/>
>
>
>
>
>
> class="org.apache.ignite.binary.BinaryBasicIdMapper">
>
>
>
>
>
>
>
> {noformat}
> {{g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include
> -I./ignite/common/os/linux/include -I./ignite/common/os/win/include
> -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample
> -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-6804) Print a warning if HashMap is passed into bulk update operations
[ https://issues.apache.org/jira/browse/IGNITE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989688#comment-16989688 ] Nikolay Izhikov commented on IGNITE-6804: - Hello, [~ilyak]. Any progress on that issue? As far as I know, users still stumble with this issue. It a very confusing Ignite behaviour, let's fix it! > Print a warning if HashMap is passed into bulk update operations > > > Key: IGNITE-6804 > URL: https://issues.apache.org/jira/browse/IGNITE-6804 > Project: Ignite > Issue Type: Improvement > Components: cache >Reporter: Denis A. Magda >Assignee: Ilya Kasnacheev >Priority: Critical > Labels: usability > Time Spent: 50m > Remaining Estimate: 0h > > Ignite newcomers tend to stumble on deadlocks simply because the keys are > passed in an unordered HashMap. Propose to do the following: > * update bulk operations Java docs. > * print out a warning if not SortedMap (e.g. HashMap, > Weak/Identity/Concurrent/Linked HashMap etc) is passed into > a bulk method (instead of SortedMap) and contains more than 1 element. > However, we should make sure that we only print that warning once and not > every time the API is called. > * do not produce warning for explicit optimistic transactions > More details are here: > http://apache-ignite-developers.2346864.n4.nabble.com/Re-Ignite-2-0-0-GridUnsafe-unmonitor-td23706.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12419) JCache TCK fails with IgniteCheckedException and NullPointerException
[
https://issues.apache.org/jira/browse/IGNITE-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989679#comment-16989679
]
Ignite TC Bot commented on IGNITE-12419:
{panel:title=Branch: [pull/7103/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4815294&buildTypeId=IgniteTests24Java8_RunAll]
> JCache TCK fails with IgniteCheckedException and NullPointerException
> -
>
> Key: IGNITE-12419
> URL: https://issues.apache.org/jira/browse/IGNITE-12419
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.7.6
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Blocker
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems a long period of time, JCache suite did not run TCK tests (it seems
> that TeamCity uses the wrong maven goal - {{surefire:test}} instead of
> {{test}}) and for now, more than 90 tests are broken.
> For example:
> {noformat}
> javax.cache.CacheException: class org.apache.ignite.IgniteCheckedException:
> Failed to complete exchange process.
> at
> org.jsr107.tck.integration.CacheWriterTest.onBeforeEachTest(CacheWriterTest.java:109)
> Caused by: org.apache.ignite.IgniteCheckedException: Failed to complete
> exchange process.
> shouldWriteThroughUsingGetAndPut_SingleEntryMultipleTimes(org.jsr107.tck.integration.CacheWriterTest)
> Time elapsed: 0.004 s <<< ERROR!
> java.lang.NullPointerException
> at
> org.jsr107.tck.integration.CacheWriterTest.cleanup(CacheWriterTest.java:117)
> {noformat}
> {code:java}
> SEVERE: Failed to initialize cache(s) (will try to rollback)
> [exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1,
> minorTopVer=161], discoEvt=DiscoveryCustomEvent
> [customMsg=DynamicCacheChangeBatch
> [id=fe5df36de61-0dd20864-f118-4e11-bcf2-12a651e87cb0, reqs=ArrayList
> [DynamicCacheChangeRequest [cacheName=cache-loader-test, hasCfg=true,
> nodeId=4bed4ec0-7d64-473d-bc55-e5f79b9a02ab, clientStartOnly=false,
> stop=false, destroy=false, disabledAfterStartfalse]],
> exchangeActions=ExchangeActions [startCaches=[cache-loader-test],
> stopCaches=null, startGrps=[cache-loader-test], stopGrps=[], resetParts=null,
> stateChangeRequest=null], startCaches=false],
> affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=161],
> super=DiscoveryEvent [evtNode=TcpDiscoveryNode
> [id=4bed4ec0-7d64-473d-bc55-e5f79b9a02ab,
> consistentId=0:0:0:0:0:0:0:1,127.0.0.1,172.17.251.129,172.25.4.114:47500,
> addrs=ArrayList [0:0:0:0:0:0:0:1, 127.0.0.1, 172.17.251.129, 172.25.4.114],
> sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
> DESKTOP-IMRCA1M.mshome.net/172.17.251.129:47500, /172.25.4.114:47500],
> discPort=47500, order=1, intOrder=1, lastExchangeTime=1575552670125,
> loc=true, ver=2.8.0#20191205-sha1:, isClient=false], topVer=1,
> nodeId8=4bed4ec0, msg=null, type=DISCOVERY_CUSTOM_EVT,
> tstamp=1575552674811]], nodeId=4bed4ec0, evt=DISCOVERY_CUSTOM_EVT],
> caches=[o.a.i.i.processors.cache.ExchangeActions$CacheGroupActionData@24671b0e]]SEVERE:
> Failed to initialize cache(s) (will try to rollback)
> [exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1,
> minorTopVer=161], discoEvt=DiscoveryCustomEvent
> [customMsg=DynamicCacheChangeBatch
> [id=fe5df36de61-0dd20864-f118-4e11-bcf2-12a651e87cb0, reqs=ArrayList
> [DynamicCacheChangeRequest [cacheName=cache-loader-test, hasCfg=true,
> nodeId=4bed4ec0-7d64-473d-bc55-e5f79b9a02ab, clientStartOnly=false,
> stop=false, destroy=false, disabledAfterStartfalse]],
> exchangeActions=ExchangeActions [startCaches=[cache-loader-test],
> stopCaches=null, startGrps=[cache-loader-test], stopGrps=[], resetParts=null,
> stateChangeRequest=null], startCaches=false],
> affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=161],
> super=DiscoveryEvent [evtNode=TcpDiscoveryNode
> [id=4bed4ec0-7d64-473d-bc55-e5f79b9a02ab,
> consistentId=0:0:0:0:0:0:0:1,127.0.0.1,172.17.251.129,172.25.4.114:47500,
> addrs=ArrayList [0:0:0:0:0:0:0:1, 127.0.0.1, 172.17.251.129, 172.25.4.114],
> sockAddrs=HashSet [/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500,
> DESKTOP-IMRCA1M.mshome.net/172.17.251.129:47500, /172.25.4.114:47500],
> discPort=47500, order=1, intOrder=1, lastExchangeTime=1575552670125,
> loc=true, ver=2.8.0#20191205-sha1:, isClient=false], topVer=1,
> nodeId8=4bed4ec0, msg=null, type=DISCOVERY_CUSTOM_EVT,
> tstamp=1575552674811]], nodeId=4bed4ec0, evt=DISCOVERY_CUSTOM_EVT],
> caches=[o.a.i.i.processors.cache.ExchangeActions$CacheGroupActionData@24671b0e]]class
> o
[jira] [Resolved] (IGNITE-6191) Integration for Unix system
[
https://issues.apache.org/jira/browse/IGNITE-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov resolved IGNITE-6191.
-
Resolution: Fixed
> Integration for Unix system
> ---
>
> Key: IGNITE-6191
> URL: https://issues.apache.org/jira/browse/IGNITE-6191
> Project: Ignite
> Issue Type: Task
> Components: build
>Affects Versions: 2.1
>Reporter: Sergey Kozlov
>Priority: Major
>
> Current installation procedure of Apache Ignite is poor: unzip and run.
> We need to deep integration for Unix operation system, namely:
> # Provide a script to *nix for start/stop/restart/reload Apache Ignite as
> system service (a deamon). At the beginning focus on Ubuntu and RedHat
> (CentOS)
> # Provide the start confugration for Apache Ignite where provide non-spring
> options like number of Apache Ignite instances, JVM options, work/log
> directories etc (at might be an ini file)
> # Provide platform-adopted packages (*.pkg, *deb) in the own repository that
> will install (upgrade) Apache Ignite in proper directories, for instance:
> configuration in /etc/ignite/config, visorcmd in /usr/sbin, start scripts in
> /etc/init.d, work directory in /var/lib/ignite, logs in /var/log/ignite/. It
> will significally reduce the efforts to install Apache Ignite for newbies
> ("just add the reporsitory and run package manager")
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (IGNITE-8296) Move Spark Scala DataFrames code examples to correct directory and prefix with "Scalar" to follow convention used with other Scala examples
[ https://issues.apache.org/jira/browse/IGNITE-8296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov resolved IGNITE-8296. - Resolution: Not A Problem > Move Spark Scala DataFrames code examples to correct directory and prefix > with "Scalar" to follow convention used with other Scala examples > > > Key: IGNITE-8296 > URL: https://issues.apache.org/jira/browse/IGNITE-8296 > Project: Ignite > Issue Type: Improvement > Components: spark >Affects Versions: 3.0 >Reporter: Akmal Chaudhri >Assignee: Alexey Zinoviev >Priority: Minor > > # The Spark Scala DataFrames code examples are in the wrong directory. They > should be moved to the correct directory structure. > # The Spark Scala DataFrames code examples should follow the naming > convention used for other Scala code examples and be prefixed with "Scalar". > or move SparkScalar example and its logic to the appropriate folder in spark > folder -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8296) Move Spark Scala DataFrames code examples to correct directory and prefix with "Scalar" to follow convention used with other Scala examples
[ https://issues.apache.org/jira/browse/IGNITE-8296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989654#comment-16989654 ] Nikolay Izhikov commented on IGNITE-8296: - Hello [~zaleslaw], [~abchaudhri] The examples in the right place. Why do you think it's not? scalar directory is for scalar module examples. It differs from data frame examples. > Move Spark Scala DataFrames code examples to correct directory and prefix > with "Scalar" to follow convention used with other Scala examples > > > Key: IGNITE-8296 > URL: https://issues.apache.org/jira/browse/IGNITE-8296 > Project: Ignite > Issue Type: Improvement > Components: spark >Affects Versions: 3.0 >Reporter: Akmal Chaudhri >Assignee: Alexey Zinoviev >Priority: Minor > > # The Spark Scala DataFrames code examples are in the wrong directory. They > should be moved to the correct directory structure. > # The Spark Scala DataFrames code examples should follow the naming > convention used for other Scala code examples and be prefixed with "Scalar". > or move SparkScalar example and its logic to the appropriate folder in spark > folder -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11857) Investigate performance drop after IGNITE-10078
[ https://issues.apache.org/jira/browse/IGNITE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989651#comment-16989651 ] Aleksey Plekhanov commented on IGNITE-11857: About 2.8, agreed with you, should be merged to master (2.8 already has own branch). > Investigate performance drop after IGNITE-10078 > --- > > Key: IGNITE-11857 > URL: https://issues.apache.org/jira/browse/IGNITE-11857 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Aleksey Plekhanov >Priority: Major > Attachments: ignite-config.xml, > run.properties.tx-optimistic-put-b-backup > > Time Spent: 20m > Remaining Estimate: 0h > > After IGNITE-10078 yardstick tests show performance drop up to 8% in some > scenarios: > * tx-optim-repRead-put-get > * tx-optimistic-put > * tx-putAll > Partially this is due new update counter implementation, but not only. > Investigation is required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12247) [Spark] Add initial support of Spark 2.4
[ https://issues.apache.org/jira/browse/IGNITE-12247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989643#comment-16989643 ] Nikolay Izhikov commented on IGNITE-12247: -- Hello [~zaleslaw] I had a very minor comment for your patch - let's ignore tests instead of commenting. Can you, please, resolve it, so we can include 2.4 initial support to Ignite 2.8 release. > [Spark] Add initial support of Spark 2.4 > > > Key: IGNITE-12247 > URL: https://issues.apache.org/jira/browse/IGNITE-12247 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.8 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await > Fix For: 2.8 > > Time Spent: 0.5h > Remaining Estimate: 0h > > This solution provides the initial support of spark 2.4 with copied codebase > from spark and with initial support of ExternalCatalog refactoring -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11857) Investigate performance drop after IGNITE-10078
[
https://issues.apache.org/jira/browse/IGNITE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989606#comment-16989606
]
Aleksey Plekhanov commented on IGNITE-11857:
[~ascherbakov],
If we get rid of {{Item}} class, there will be a new {{Long}} object created
each time we merge sequence with previous. With {{Item}} class we can just add
delta to the old object. Sorted array of primitive tuples will get a drop on
remove/add items to the middle of the array. Bitmaps in some cases will consume
too much memory, the amount of memory depends on the difference between LWM and
HWM (instead of a count of gaps). Also, I'm not sure about the performance
boost in case of bitmaps usage. The current implementation is less risky, we
can't get drop on some unpredictable place.
New solution use less heap. There is no {{tailSet}} and {{headSet}} methods
usage. These methods allocate new objects for new {{Set}}s.
I've test heap allocation using this code:
{code:java}
JmhPartitionUpdateCounterBenchmark benchmark = new
JmhPartitionUpdateCounterBenchmark();
benchmark.setup();
ThreadMXBean bean = (ThreadMXBean)ManagementFactory.getThreadMXBean();
long allocated0 = bean.getThreadAllocatedBytes(Thread.currentThread().getId());
for (int i = 0; i < 100_000; i++)
benchmark.updateWithGap();
long allocated1 = bean.getThreadAllocatedBytes(Thread.currentThread().getId());
System.out.println("Memory allocated: " + (allocated1 - allocated0));
{code}
Results:
Old implementation: 25 181 680 bytes
New implementation: 11 828 272 bytes
> Investigate performance drop after IGNITE-10078
> ---
>
> Key: IGNITE-11857
> URL: https://issues.apache.org/jira/browse/IGNITE-11857
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexei Scherbakov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Attachments: ignite-config.xml,
> run.properties.tx-optimistic-put-b-backup
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After IGNITE-10078 yardstick tests show performance drop up to 8% in some
> scenarios:
> * tx-optim-repRead-put-get
> * tx-optimistic-put
> * tx-putAll
> Partially this is due new update counter implementation, but not only.
> Investigation is required.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12273) Slow TX recovery
[ https://issues.apache.org/jira/browse/IGNITE-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-12273: -- Description: TX recovery cause B*B*N*2 GridCacheTxRecoveryRequest messages sending (B - backups, N - prepared txs amount). Seems, we able to perform recovery more efficiently. For example, we may send only B*B*2 messages, accumulates txs together. was: TX recovery cause B*N*2 GridCacheTxRecoveryRequest messages sending (B - backups, N - prepared txs amount). Seems, we able to perform recovery more efficiently. For example, we may send only B*B*2 messages, accumulates txs together. > Slow TX recovery > > > Key: IGNITE-12273 > URL: https://issues.apache.org/jira/browse/IGNITE-12273 > Project: Ignite > Issue Type: Task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > > TX recovery cause B*B*N*2 GridCacheTxRecoveryRequest messages sending (B - > backups, N - prepared txs amount). > Seems, we able to perform recovery more efficiently. > For example, we may send only B*B*2 messages, accumulates txs together. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12423) PME duration histogram updates only if log info enabled
Amelchev Nikita created IGNITE-12423:
Summary: PME duration histogram updates only if log info enabled
Key: IGNITE-12423
URL: https://issues.apache.org/jira/browse/IGNITE-12423
Project: Ignite
Issue Type: Bug
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita
Fix For: 2.8
The updates histogram method is placed at the log info block:
{noformat}
if (log.isInfoEnabled()) {
...
updateDurationHistogram(System.currentTimeMillis() - initTime);
...
}
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-11857) Investigate performance drop after IGNITE-10078
[ https://issues.apache.org/jira/browse/IGNITE-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16989525#comment-16989525 ] Alexei Scherbakov commented on IGNITE-11857: [~alex_pl] I've looked at your contribution. Changing TreeSet to TreeMap looks like a very minor change. I think you can go further and get rid of Item class. Out of order updates can be kept in SortedMap where key is a start and value is a range (or even in sorted array of primitive tuples). Another possibility is storing missing updates in a bitmap. You should also check a new solution for heap usage in comparison to the old. For many partitions configurations less heap usage could be more significant advantage other the minor performance boost. Also I have a little concern about the robustness of a fix. It might be risky to merge it to 2.8 without extensive testing. So, I would postpone the change and improved the patch first. > Investigate performance drop after IGNITE-10078 > --- > > Key: IGNITE-11857 > URL: https://issues.apache.org/jira/browse/IGNITE-11857 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Aleksey Plekhanov >Priority: Major > Attachments: ignite-config.xml, > run.properties.tx-optimistic-put-b-backup > > Time Spent: 20m > Remaining Estimate: 0h > > After IGNITE-10078 yardstick tests show performance drop up to 8% in some > scenarios: > * tx-optim-repRead-put-get > * tx-optimistic-put > * tx-putAll > Partially this is due new update counter implementation, but not only. > Investigation is required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12422) Clean up GG-XXX internal ticket references from the code base.
[ https://issues.apache.org/jira/browse/IGNITE-12422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexei Scherbakov updated IGNITE-12422: --- Summary: Clean up GG-XXX internal ticket references from the code base. (was: Clean up GG-XXX internal ticket references from code base.) > Clean up GG-XXX internal ticket references from the code base. > -- > > Key: IGNITE-12422 > URL: https://issues.apache.org/jira/browse/IGNITE-12422 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Alexei Scherbakov >Priority: Major > Fix For: 2.9 > > > Replace with Apache Ignite equivalent if possible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
