[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16626999#comment-16626999 ] Dmitriy Govorukhin commented on IGNITE-1678: [~NIzhikov] I think no, moved to 2.8. > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.7 > > Attachments: Screenshot from 2016-12-15 10-50-22.png > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623497#comment-16623497 ] Nikolay Izhikov commented on IGNITE-1678: - [~DmitriyGovorukhin] Do we have a chance to resolve this ticket until the code freeze of 2.7? > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.7 > > Attachments: Screenshot from 2016-12-15 10-50-22.png > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16595480#comment-16595480 ] ASF GitHub Bot commented on IGNITE-1678: GitHub user DmitriyGovorukhin opened a pull request: https://github.com/apache/ignite/pull/4637 IGNITE-1678 IgniteAtomicSequence: make following reservations in advance You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-1678 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4637.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4637 commit ed9c2d27cdac5e65fe1689a4583678b9cbd1b3d1 Author: Dmitriy Govorukhin Date: 2018-08-26T07:35:01Z IGNITE-1678 add percentage for atomic seq reservation in config + interfaces commit cbfe5932a4b1230bb1aac8075004eea73479e7b2 Author: Dmitriy Govorukhin Date: 2018-08-26T07:53:47Z IGNITE-1678 add percentage as parameter for atomic seq commit 4a1d0c244fe015c7aaee7861857e84f05b985fc2 Author: Dmitriy Govorukhin Date: 2018-08-26T09:31:07Z IGNITE-1678 add new test + small test updates commit 839f43adf80a04e4a2778983560031592a20a89e Author: Dmitriy Govorukhin Date: 2018-08-26T11:45:49Z IGNITE-1678 first implementation (fully not working) commit bd46569fc1e1f65709ac7a838f9b76d4df5c39d9 Author: Dmitriy Govorukhin Date: 2018-08-26T17:36:22Z IGNITE-1678 minor test fix commit 62bc2c94601d15a3eb3c939313ae88fbb81ca0fb Author: Dmitriy Govorukhin Date: 2018-08-26T18:29:52Z IGNITE-1678 fix config for chaining commit 2d2627056ca288d95c68934286c0e80e66b4ca4e Author: Dmitriy Govorukhin Date: 2018-08-26T18:48:07Z IGNITE-1678 set default reservation percentage as 100 commit cf9a98f938f740c226b71ad73e734ac1834d97fe Author: Dmitriy Govorukhin Date: 2018-08-27T10:44:39Z IGNITE-1678 minor assert added + fix NPE commit 0415064f4dbbfb3bea6d359467af1525af235b68 Author: Dmitriy Govorukhin Date: 2018-08-28T08:36:57Z IGNITE-1678 rework reservation, add reservation context commit 589297a406421a72254f949a1b0cb19cf14b9a26 Author: Dmitriy Govorukhin Date: 2018-08-28T09:42:21Z IGNITE-1678 test refactoring commit 4cdaf252de5526c19ba8672ac10819e9a983e1a1 Author: Dmitriy Govorukhin Date: 2018-08-28T16:50:00Z Merge branch 'master' into ignite-1678 commit 7d1cf05c37867aab1a6496a97cf6ffc4727c8501 Author: Dmitriy Govorukhin Date: 2018-08-28T19:14:35Z IGNITE-1678 add benchmarks > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.7 > > Attachments: Screenshot from 2016-12-15 10-50-22.png > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503143#comment-16503143 ] Alexey Goncharuk commented on IGNITE-1678: -- [~DmitriyGovorukhin] can you please pull master to your PR so we can re-run benchmarks? > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.6 > > Attachments: Screenshot from 2016-12-15 10-50-22.png > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448220#comment-16448220 ] Dmitriy Govorukhin commented on IGNITE-1678: [~dpavlov] Issue still in patch state, I guess we must review again it and re-run benchmarks. > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.6 > > Attachments: Screenshot from 2016-12-15 10-50-22.png > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448109#comment-16448109 ] Dmitriy Pavlov commented on IGNITE-1678: [~DmitriyGovorukhin] [~agoncharuk], is this issue still actual? Coulld we move it to resolved or to open? > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin >Priority: Major > Fix For: 2.6 > > Attachments: Screenshot from 2016-12-15 10-50-22.png > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184137#comment-16184137 ] Vladimir Ozerov commented on IGNITE-1678: - Moved to 2.4 due to inactivity. > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin > Fix For: 2.4 > > Attachments: Screenshot from 2016-12-15 10-50-22.png > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15750704#comment-15750704 ] Dmitriy Govorukhin commented on IGNITE-1678: I got the following results, see attach. > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin > Fix For: 2.0 > > Attachments: Screenshot from 2016-12-15 10-50-22.png > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490389#comment-15490389 ] Dmitriy Govorukhin commented on IGNITE-1678: Unfortunately i get worse performance time. Master branch 1,038,823.05 op/sec, my branch 893 212.52 op/sec. I try to find problem and fix it. > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin > Fix For: 1.8 > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15489733#comment-15489733 ] Semen Boikov commented on IGNITE-1678: -- Dmitry, New tests added in this branch fail for me locally. Do they work for you? Do you have link to TeamCity run (if you did not check TeamCity please do it)? Thanks > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin > Fix For: 1.8 > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15487453#comment-15487453 ] Dmitriy Govorukhin commented on IGNITE-1678: Hi [~sboikov], I corrected comments, move all code in one branch (ignite-1678-2). Also added benchmark. > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin > Fix For: 1.8 > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15486737#comment-15486737 ] Dmitriy Govorukhin commented on IGNITE-1678: I got it, fix it soon. > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin > Fix For: 1.8 > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15473807#comment-15473807 ] Dmitriy Govorukhin commented on IGNITE-1678: [~sboikov]] I commit the changes, added small fixes, please look it. > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Dmitriy Govorukhin > Fix For: 1.8 > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15000979#comment-15000979 ] Artem Shutak commented on IGNITE-1678: -- Fixed some bugs in implementation. Now all tests pass. Benchmarking status - I benchmarked old and new implementation only on local machine (3 clients, 2 servers) - Current benchmark results show direct dependency between atomicSequenceReserveSize and op/sec. Look like only number of transactions cache op-s have meaning. Need to check it with many phisicale machines - Need to solve with default percentage value. Left: - Add tests on new reserevePercetage paramether - Create runners of GridCachePartitionedAtomicSequenceMultiThreadedTest in another modes: replicated, transactional - Benchmark with many phisicale machines > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Artem Shutak > Fix For: 1.6 > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999277#comment-14999277 ] Artem Shutak commented on IGNITE-1678: -- Upd. The issue (using of internal transactions) was in running background task on system-poll. But system pool is used for cache operations. Use public pool instead of system pool now. Added tests. Start benchmarking: benchmarked with different percentages and with different atomicSequenceReserveSize. > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Artem Shutak > Fix For: 1.6 > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (IGNITE-1678) IgniteAtomicSequence: make following reservations in advance
[ https://issues.apache.org/jira/browse/IGNITE-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997222#comment-14997222 ] Artem Shutak commented on IGNITE-1678: -- Finished with an implementation. Working on tests: improve old tests and add new tests. In process of testing faced with an issue related to using of internal transactions and dead-lock after. Will investigate. > IgniteAtomicSequence: make following reservations in advance > > > Key: IGNITE-1678 > URL: https://issues.apache.org/jira/browse/IGNITE-1678 > Project: Ignite > Issue Type: Improvement > Components: data structures >Affects Versions: ignite-1.4 >Reporter: Denis Magda >Assignee: Artem Shutak > Fix For: 1.6 > > > In current implementation a new reservation is made when the current local > sequence boundary is exceeded. > In cases when there are many nodes that use the same atomic sequence there > can be a situation when all the nodes start doing a new reservation at the > same time. This can lead to performance drops. > As a performance optimization it makes sense to start reserving new sequence > slot in advance (in background), like when around 80% of current reservation > has already been used. Probably we should add a special parameter to > {{AtomicConfiguration}} that will manage such behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)