[jira] [Commented] (IGNITE-17274) Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor
[ https://issues.apache.org/jira/browse/IGNITE-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17562815#comment-17562815 ] Arthur Naseef commented on IGNITE-17274: Thank you for walking me through the update and the quick turn-around! > Startup of large numbers of servers slowed by linear lookup in > IgniteServiceProcessor > - > > Key: IGNITE-17274 > URL: https://issues.apache.org/jira/browse/IGNITE-17274 > Project: Ignite > Issue Type: Improvement > Components: managed services >Affects Versions: 2.13 >Reporter: Arthur Naseef >Assignee: Arthur Naseef >Priority: Major > Fix For: 2.14 > > Time Spent: 2h > Remaining Estimate: 0h > > Using a small POC, spinning up many servers is slow. In addition, the > startup time appears to be exponential. > Using timing measurements, found a linear lookup inside the > IgniteServiceProcessor that is taking most of the time. > Replacing that linear lookup with a Map lookup, and maintaining the map, > significantly speeds up the process, and startup time is now linear with the > number of services started. > Note this was tested with 20K and 50K services on a 1-node ignite cluster. > Timings against the stock 2.13.0 code come in at 30s for 20K and 250s for 50K > services. Modifying the linear lookup to use a Map, the timing come in at 8s > for 20K and 14s for 50K services. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17274) Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor
[ https://issues.apache.org/jira/browse/IGNITE-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561728#comment-17561728 ] Pavel Tupitsyn commented on IGNITE-17274: - Merged to master: 3e5cd22187814a1cf23d719fca37d709c0c15311 [~artnaseef] than you for the contribution! > Startup of large numbers of servers slowed by linear lookup in > IgniteServiceProcessor > - > > Key: IGNITE-17274 > URL: https://issues.apache.org/jira/browse/IGNITE-17274 > Project: Ignite > Issue Type: Improvement > Components: managed services >Affects Versions: 2.13 >Reporter: Arthur Naseef >Assignee: Arthur Naseef >Priority: Major > Fix For: 2.14 > > Time Spent: 2h > Remaining Estimate: 0h > > Using a small POC, spinning up many servers is slow. In addition, the > startup time appears to be exponential. > Using timing measurements, found a linear lookup inside the > IgniteServiceProcessor that is taking most of the time. > Replacing that linear lookup with a Map lookup, and maintaining the map, > significantly speeds up the process, and startup time is now linear with the > number of services started. > Note this was tested with 20K and 50K services on a 1-node ignite cluster. > Timings against the stock 2.13.0 code come in at 30s for 20K and 250s for 50K > services. Modifying the linear lookup to use a Map, the timing come in at 8s > for 20K and 14s for 50K services. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17274) Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor
[
https://issues.apache.org/jira/browse/IGNITE-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561720#comment-17561720
]
Ignite TC Bot commented on IGNITE-17274:
{panel:title=Branch: [pull/10123/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10123/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6660650&buildTypeId=IgniteTests24Java8_RunAll]
> Startup of large numbers of servers slowed by linear lookup in
> IgniteServiceProcessor
> -
>
> Key: IGNITE-17274
> URL: https://issues.apache.org/jira/browse/IGNITE-17274
> Project: Ignite
> Issue Type: Improvement
> Components: managed services
>Affects Versions: 2.13
>Reporter: Arthur Naseef
>Assignee: Arthur Naseef
>Priority: Major
> Fix For: 2.14
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Using a small POC, spinning up many servers is slow. In addition, the
> startup time appears to be exponential.
> Using timing measurements, found a linear lookup inside the
> IgniteServiceProcessor that is taking most of the time.
> Replacing that linear lookup with a Map lookup, and maintaining the map,
> significantly speeds up the process, and startup time is now linear with the
> number of services started.
> Note this was tested with 20K and 50K services on a 1-node ignite cluster.
> Timings against the stock 2.13.0 code come in at 30s for 20K and 250s for 50K
> services. Modifying the linear lookup to use a Map, the timing come in at 8s
> for 20K and 14s for 50K services.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17274) Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor
[
https://issues.apache.org/jira/browse/IGNITE-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561631#comment-17561631
]
Ignite TC Bot commented on IGNITE-17274:
{panel:title=Branch: [pull/10123/head] Base: [master] : Possible Blockers
(10)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}PDS (Compatibility){color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=6660612]]
{color:#d04437}PDS (Indexing){color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=6660613]]
{color:#d04437}Snapshots{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6660632]]
* IgniteSnapshotTestSuite:
EncryptedSnapshotTest.testSnapshotRestoringAfterMultipleReencryption[Encryption
is enabled.] - Test has low fail rate in base branch 0,0% and is not flaky
{color:#d04437}JDBC Driver{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6660596]]
* IgniteJdbcDriverTestSuite:
JdbcThinJdbcToCacheDataTypesCoverageTest.testIntDataType[atomicityMode=ATOMIC,
cacheMode=PARTITIONED, ttlFactory=null, backups=2, evictionFactory=null,
onheapCacheEnabled=false, writeSyncMode=FULL_SYNC, persistenceEnabled=false,
useBinaryArrays=false] - Test has low fail rate in base branch 0,0% and is not
flaky
{color:#d04437}Cache 1{color} [[tests 0 TIMEOUT , Out Of Memory Error , Exit
Code |https://ci.ignite.apache.org/viewLog.html?buildId=6660546]]
{color:#d04437}Queries 2{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6660622]]
* org.apache.ignite.testsuites.IgniteBinaryCacheQueryTestSuite2:
org.apache.ignite.internal.processors.cache.distributed.near.IgniteCacheQueryNodeRestartDistributedJoinSelfTest.
- History for base branch is absent.
{color:#d04437}Queries 1 (lazy=true){color} [[tests 1 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=6660621]]
* IgniteBinaryCacheQueryLazyTestSuite:
BasicIndexMultinodeTest.testEqualFieldsDynamicIndexesWithPersistence - Test has
low fail rate in base branch 0,0% and is not flaky
{color:#d04437}Platform C++ CMake (Win x64 / Release){color} [[tests 0
BuildFailureOnMessage
|https://ci.ignite.apache.org/viewLog.html?buildId=6660615]]
{color:#d04437}Continuous Query 1{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6660578]]
* IgniteCacheQuerySelfTestSuite3:
CacheContinuousQueryCounterReplicatedTxTest.testTwoQueryListener - Test has low
fail rate in base branch 0,0% and is not flaky
{panel}
{panel:title=Branch: [pull/10123/head] Base: [master] : New Tests
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#8b}Queries 2{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6660622]]
* {color:#8b}org.apache.ignite.testsuites.IgniteBinaryCacheQueryTestSuite2:
org.apache.ignite.internal.processors.cache.distributed.near.IgniteCacheQueryNodeRestartDistributedJoinSelfTest.
- FAILED{color}
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6660650&buildTypeId=IgniteTests24Java8_RunAll]
> Startup of large numbers of servers slowed by linear lookup in
> IgniteServiceProcessor
> -
>
> Key: IGNITE-17274
> URL: https://issues.apache.org/jira/browse/IGNITE-17274
> Project: Ignite
> Issue Type: Improvement
> Components: managed services
>Affects Versions: 2.13
>Reporter: Arthur Naseef
>Assignee: Arthur Naseef
>Priority: Major
> Fix For: 2.14
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Using a small POC, spinning up many servers is slow. In addition, the
> startup time appears to be exponential.
> Using timing measurements, found a linear lookup inside the
> IgniteServiceProcessor that is taking most of the time.
> Replacing that linear lookup with a Map lookup, and maintaining the map,
> significantly speeds up the process, and startup time is now linear with the
> number of services started.
> Note this was tested with 20K and 50K services on a 1-node ignite cluster.
> Timings against the stock 2.13.0 code come in at 30s for 20K and 250s for 50K
> services. Modifying the linear lookup to use a Map, the timing come in at 8s
> for 20K and 14s for 50K services.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17274) Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor
[ https://issues.apache.org/jira/browse/IGNITE-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561530#comment-17561530 ] Pavel Tupitsyn commented on IGNITE-17274: - Changes look good to me. TC build started: https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_RunAll/6660650 > Startup of large numbers of servers slowed by linear lookup in > IgniteServiceProcessor > - > > Key: IGNITE-17274 > URL: https://issues.apache.org/jira/browse/IGNITE-17274 > Project: Ignite > Issue Type: Improvement > Components: managed services >Affects Versions: 2.13 >Reporter: Arthur Naseef >Assignee: Arthur Naseef >Priority: Major > Fix For: 2.14 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Using a small POC, spinning up many servers is slow. In addition, the > startup time appears to be exponential. > Using timing measurements, found a linear lookup inside the > IgniteServiceProcessor that is taking most of the time. > Replacing that linear lookup with a Map lookup, and maintaining the map, > significantly speeds up the process, and startup time is now linear with the > number of services started. > Note this was tested with 20K and 50K services on a 1-node ignite cluster. > Timings against the stock 2.13.0 code come in at 30s for 20K and 250s for 50K > services. Modifying the linear lookup to use a Map, the timing come in at 8s > for 20K and 14s for 50K services. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17274) Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor
[ https://issues.apache.org/jira/browse/IGNITE-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17560836#comment-17560836 ] Pavel Tupitsyn commented on IGNITE-17274: - [~artnaseef] please see some comments on GitHub. > Startup of large numbers of servers slowed by linear lookup in > IgniteServiceProcessor > - > > Key: IGNITE-17274 > URL: https://issues.apache.org/jira/browse/IGNITE-17274 > Project: Ignite > Issue Type: Improvement > Components: managed services >Affects Versions: 2.13 >Reporter: Arthur Naseef >Assignee: Arthur Naseef >Priority: Major > Fix For: 2.14 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Using a small POC, spinning up many servers is slow. In addition, the > startup time appears to be exponential. > Using timing measurements, found a linear lookup inside the > IgniteServiceProcessor that is taking most of the time. > Replacing that linear lookup with a Map lookup, and maintaining the map, > significantly speeds up the process, and startup time is now linear with the > number of services started. > Note this was tested with 20K and 50K services on a 1-node ignite cluster. > Timings against the stock 2.13.0 code come in at 30s for 20K and 250s for 50K > services. Modifying the linear lookup to use a Map, the timing come in at 8s > for 20K and 14s for 50K services. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17274) Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor
[ https://issues.apache.org/jira/browse/IGNITE-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17560616#comment-17560616 ] Arthur Naseef commented on IGNITE-17274: The patch adds a new Map to track services by name, in addition to the original map that tracks the services by service-id. Changes to the original map (get/put/...) updated to also update the new map. Lookup by name now uses the map instead of a linear search of the original map's value set. > Startup of large numbers of servers slowed by linear lookup in > IgniteServiceProcessor > - > > Key: IGNITE-17274 > URL: https://issues.apache.org/jira/browse/IGNITE-17274 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.13 >Reporter: Arthur Naseef >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Using a small POC, spinning up many servers is slow. In addition, the > startup time appears to be exponential. > Using timing measurements, found a linear lookup inside the > IgniteServiceProcessor that is taking most of the time. > Replacing that linear lookup with a Map lookup, and maintaining the map, > significantly speeds up the process, and startup time is now linear with the > number of services started. > Note this was tested with 20K and 50K services on a 1-node ignite cluster. > Timings against the stock 2.13.0 code come in at 30s for 20K and 250s for 50K > services. Modifying the linear lookup to use a Map, the timing come in at 8s > for 20K and 14s for 50K services. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17274) Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor
[ https://issues.apache.org/jira/browse/IGNITE-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17560611#comment-17560611 ] Arthur Naseef commented on IGNITE-17274: Pull request: [https://github.com/apache/ignite/pull/10123] > Startup of large numbers of servers slowed by linear lookup in > IgniteServiceProcessor > - > > Key: IGNITE-17274 > URL: https://issues.apache.org/jira/browse/IGNITE-17274 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.13 >Reporter: Arthur Naseef >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Using a small POC, spinning up many servers is slow. In addition, the > startup time appears to be exponential. > Using timing measurements, found a linear lookup inside the > IgniteServiceProcessor that is taking most of the time. > Replacing that linear lookup with a Map lookup, and maintaining the map, > significantly speeds up the process, and startup time is now linear with the > number of services started. > Note this was tested with 20K and 50K services on a 1-node ignite cluster. > Timings against the stock 2.13.0 code come in at 30s for 20K and 250s for 50K > services. Modifying the linear lookup to use a Map, the timing come in at 8s > for 20K and 14s for 50K services. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17274) Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor
[ https://issues.apache.org/jira/browse/IGNITE-17274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17560608#comment-17560608 ] Arthur Naseef commented on IGNITE-17274: POC code used to test: [https://github.com/opennms-forge/distributed-scheduling-poc/] Performance Notes: [https://github.com/opennms-forge/distributed-scheduling-poc/wiki/Ignite-Startup-Performance] Pull request is imminent. > Startup of large numbers of servers slowed by linear lookup in > IgniteServiceProcessor > - > > Key: IGNITE-17274 > URL: https://issues.apache.org/jira/browse/IGNITE-17274 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.13 >Reporter: Arthur Naseef >Priority: Major > > Using a small POC, spinning up many servers is slow. In addition, the > startup time appears to be exponential. > Using timing measurements, found a linear lookup inside the > IgniteServiceProcessor that is taking most of the time. > Replacing that linear lookup with a Map lookup, and maintaining the map, > significantly speeds up the process, and startup time is now linear with the > number of services started. > Note this was tested with 20K and 50K services on a 1-node ignite cluster. > Timings against the stock 2.13.0 code come in at 30s for 20K and 250s for 50K > services. Modifying the linear lookup to use a Map, the timing come in at 8s > for 20K and 14s for 50K services. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
