[jira] [Created] (IGNITE-15132) REST API must reuse general netty infrastructure from network module
Kirill Gusakov created IGNITE-15132: --- Summary: REST API must reuse general netty infrastructure from network module Key: IGNITE-15132 URL: https://issues.apache.org/jira/browse/IGNITE-15132 Project: Ignite Issue Type: Task Reporter: Kirill Gusakov At the moment - RestModule prepares its own Netty infra for handling requests. But it will be great to have one root Netty pools and etc. configs and just separate handlers. Also, maybe we should use the "one port for all operations" concept for REST API too. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15131) Provide any method to receive node configs from public Java API
[ https://issues.apache.org/jira/browse/IGNITE-15131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-15131: Labels: ignite-3 (was: ) > Provide any method to receive node configs from public Java API > --- > > Key: IGNITE-15131 > URL: https://issues.apache.org/jira/browse/IGNITE-15131 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Assignee: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > We need to implement a way to receive configs of started ignite node from the > Java API. > It can be used for getting any auto-discovered ports and etc. in integration > tests and client code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-15131) Provide any method to receive node configs from public Java API
Kirill Gusakov created IGNITE-15131: --- Summary: Provide any method to receive node configs from public Java API Key: IGNITE-15131 URL: https://issues.apache.org/jira/browse/IGNITE-15131 Project: Ignite Issue Type: Task Reporter: Kirill Gusakov Assignee: Kirill Gusakov We need to implement a way to receive configs of started ignite node from the Java API. It can be used for getting any auto-discovered ports and etc. in integration tests and client code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-14999) Support snapshot encryption with same master-key.
[
https://issues.apache.org/jira/browse/IGNITE-14999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381621#comment-17381621
]
Ignite TC Bot commented on IGNITE-14999:
{panel:title=Branch: [pull/9260/head] Base: [master] : Possible Blockers
(3)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Basic 3{color} [[tests
2|https://ci.ignite.apache.org/viewLog.html?buildId=6088663]]
* IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotRestoreSelfTest.testNodeFailDuringRestore[Encryption=true]
- History for base branch is absent.
* IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotRestoreSelfTest.testIncompatibleMetasUpdate[Encryption=true]
- New test duration 62s is more that 1 minute
{color:#d04437}[Javadoc]{color} [[tests 0 BuildFailureOnMessage
|https://ci.ignite.apache.org/viewLog.html?buildId=6088665]]
{panel}
{panel:title=Branch: [pull/9260/head] Base: [master] : New Tests
(169)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}PDS (Indexing){color} [[tests
16|https://ci.ignite.apache.org/viewLog.html?buildId=6088691]]
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotWithIndexesTest.testClusterSnapshotConsistentConfig[Encryption=true]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotWithIndexesTest.testClusterSnapshotWithIndexes[Encryption=true]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotWithIndexesTest.testClusterSnapshotConsistentConfig[Encryption=false]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotWithIndexesTest.testClusterSnapshotWithIndexes[Encryption=false]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotCheckWithIndexesTest.testClusterSnapshotCheckEmptyCache[Encryption=true]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotCheckWithIndexesTest.testClusterSnapshotCheckWithIndexes[Encryption=false]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotCheckWithIndexesTest.testClusterSnapshotCheckWithNodeFilter[Encryption=false]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotCheckWithIndexesTest.testClusterSnapshotCheckEmptyCache[Encryption=false]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotRestoreWithIndexingTest.testBasicClusterSnapshotRestoreWithMetadata[Encryption=true]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotRestoreWithIndexingTest.testClusterSnapshotRestoreOnBiggerTopology[Encryption=true]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
IgniteClusterSnapshotRestoreWithIndexingTest.testBasicClusterSnapshotRestore[Encryption=true]
- PASSED{color}
... and 5 new tests
{color:#8b}Basic 3{color} [[tests
153|https://ci.ignite.apache.org/viewLog.html?buildId=6088663]]
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotCheckTest.testClusterSnapshotCheckCRCFail[Encryption=false]
- PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotCheckTest.testClusterSnapshotCheckWithTwoCachesCheckNullInput[Encryption=false]
- PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotCheckTest.testClusterSnapshotCheckMissedPart[Encryption=false]
- PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotCheckTest.testClusterSnapshotCheckMissedMeta[Encryption=false]
- PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotCheckTest.testClusterSnapshotCheckHashesSameAsIdleVerifyHashes[Encryption=true]
- PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotCheckTest.testClusterSnapshotCheckOtherCluster[Encryption=false]
- PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotCheckTest.testClusterSnapshotCheckWithTwoCachesCheckNotCorrupted[Encryption=false]
- PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotCheckTest.testClusterSnapshotCheckPartitionCounters[Encryption=false]
- PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotCheckTest.testClusterSnapshotCheckMissedGroup[Encryption=false]
- PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotCheckTest.testClusterSnapshotCheckHashesSameAsIdleVerifyHashes[Encryption=false]
- PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
IgniteClusterSnapshotSelfTest.testClusterSnapshotIncorrectNameFails[Encryption=true]
- PASSED{color}
... and 142 new tests
{panel}
[TeamCity *-- Run :: All*
[jira] [Updated] (IGNITE-14840) Get rid of slf4j in jraft package and for the rest modules of the project
[ https://issues.apache.org/jira/browse/IGNITE-14840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mirza Aliev updated IGNITE-14840: - Summary: Get rid of slf4j in jraft package and for the rest modules of the project (was: Get rid of slf4j in jraft package and for the rest modules of project) > Get rid of slf4j in jraft package and for the rest modules of the project > - > > Key: IGNITE-14840 > URL: https://issues.apache.org/jira/browse/IGNITE-14840 > Project: Ignite > Issue Type: Task >Reporter: Alexey Scherbakov >Assignee: Mirza Aliev >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha3 > > Time Spent: 10m > Remaining Estimate: 0h > > We should replace slf4j logging with IgniteLogger. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-14840) Get rid of slf4j in jraft package and for the rest modules of project
[ https://issues.apache.org/jira/browse/IGNITE-14840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mirza Aliev updated IGNITE-14840: - Summary: Get rid of slf4j in jraft package and for the rest modules of project (was: Get rid of slf4j in jraft package) > Get rid of slf4j in jraft package and for the rest modules of project > - > > Key: IGNITE-14840 > URL: https://issues.apache.org/jira/browse/IGNITE-14840 > Project: Ignite > Issue Type: Task >Reporter: Alexey Scherbakov >Assignee: Mirza Aliev >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha3 > > Time Spent: 10m > Remaining Estimate: 0h > > We should replace slf4j logging with IgniteLogger. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15130) Network message generator must generate "equals" and "hashCode"
[
https://issues.apache.org/jira/browse/IGNITE-15130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-15130:
-
Description: Network annotation processor must automatically generate
equals and hashCode implementations, because some code (especially in tests)
needs to compare message contents. (was: Network annotation processor must
automatically generate {{equals and hashCode }}implementations, because some
code (especially in tests) needs to compare message contents.)
> Network message generator must generate "equals" and "hashCode"
> ---
>
> Key: IGNITE-15130
> URL: https://issues.apache.org/jira/browse/IGNITE-15130
> Project: Ignite
> Issue Type: Task
> Components: networking
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
> Labels: ignite-3
>
> Network annotation processor must automatically generate equals and hashCode
> implementations, because some code (especially in tests) needs to compare
> message contents.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-15130) Network message generator must generate "equals" and "hashCode"
[
https://issues.apache.org/jira/browse/IGNITE-15130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-15130:
-
Description: Network annotation processor must automatically generate
{{equals}} and {{hashCode}} implementations, because some code (especially in
tests) needs to compare message contents. (was: Network annotation processor
must automatically generate equals and hashCode implementations, because some
code (especially in tests) needs to compare message contents.)
> Network message generator must generate "equals" and "hashCode"
> ---
>
> Key: IGNITE-15130
> URL: https://issues.apache.org/jira/browse/IGNITE-15130
> Project: Ignite
> Issue Type: Task
> Components: networking
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
> Labels: ignite-3
>
> Network annotation processor must automatically generate {{equals}} and
> {{hashCode}} implementations, because some code (especially in tests) needs
> to compare message contents.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-15130) Network message generator must generate "equals" and "hashCode"
[
https://issues.apache.org/jira/browse/IGNITE-15130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-15130:
-
Description: Network annotation processor must automatically generate
{{equals and hashCode }}implementations, because some code (especially in
tests) needs to compare message contents. (was: Network annotation processor
must automatically generate {{equals }}and{{ hashCode implementations}},
because some code (especially in tests) needs to compare message contents.)
> Network message generator must generate "equals" and "hashCode"
> ---
>
> Key: IGNITE-15130
> URL: https://issues.apache.org/jira/browse/IGNITE-15130
> Project: Ignite
> Issue Type: Task
> Components: networking
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
> Labels: ignite-3
>
> Network annotation processor must automatically generate {{equals and
> hashCode }}implementations, because some code (especially in tests) needs to
> compare message contents.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-15130) Network message generator must generate "equals" and "hashCode"
Aleksandr Polovtcev created IGNITE-15130:
Summary: Network message generator must generate "equals" and
"hashCode"
Key: IGNITE-15130
URL: https://issues.apache.org/jira/browse/IGNITE-15130
Project: Ignite
Issue Type: Task
Components: networking
Reporter: Aleksandr Polovtcev
Assignee: Aleksandr Polovtcev
Network annotation processor must automatically generate {{equals }}and{{
hashCode implementations}}, because some code (especially in tests) needs to
compare message contents.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-14743) Support Row with large values.
[
https://issues.apache.org/jira/browse/IGNITE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Mashenkov updated IGNITE-14743:
--
Description:
h3. Motivation.
For now, TupleAssembler writes offsets for varlen columns as 2-byte \{{short
}}type.
This implicitly restricts key/value sizes down to 64 kB in total.
On another side, for small rows that columns can be addressed with \{{byte}}
type, we will waste few bytes.
h3. Description.
Let's
# allow 4 byte types (byte, short, int) for offsets.
# implement and benchmark different approaches that allow us to write rows in
the most compact way.
# then choose and merge the best one.
We can introduce several formats for writing Vartable (using byte/short/int
offsets).
Additional information about Vartable format can be coded into chunk flags.
The first approach is to precalculate chunk total size, then choose the most
compact format and write a chunk.
The second approach is to write a chunk with the widest format then convert the
chunk into the most compact format in place.
was:
For now, TupleAssembler writes offsets for varlen columns as 2-byte \{{short
}}type.
This implicitly restricts key/value sizes down to 64 kB in total.
Let's use 4 byte types (byte, short, int) for offsets.
> Support Row with large values.
> --
>
> Key: IGNITE-14743
> URL: https://issues.apache.org/jira/browse/IGNITE-14743
> Project: Ignite
> Issue Type: Improvement
>Reporter: Andrey Mashenkov
>Assignee: Andrey Mashenkov
>Priority: Major
> Labels: iep-54, ignite-3
> Fix For: 3.0.0-alpha3
>
> Attachments: Byte array columns only benchmark.txt, Fixlen cols only
> benchmark.txt, Latin1 string columns benchmark.txt, Non-latin string columns
> benchmark.txt, String marshalling comparison
>
> Original Estimate: 168h
> Time Spent: 2h 10m
> Remaining Estimate: 165h 50m
>
> h3. Motivation.
> For now, TupleAssembler writes offsets for varlen columns as 2-byte \{{short
> }}type.
> This implicitly restricts key/value sizes down to 64 kB in total.
> On another side, for small rows that columns can be addressed with \{{byte}}
> type, we will waste few bytes.
> h3. Description.
> Let's
> # allow 4 byte types (byte, short, int) for offsets.
> # implement and benchmark different approaches that allow us to write rows in
> the most compact way.
> # then choose and merge the best one.
> We can introduce several formats for writing Vartable (using byte/short/int
> offsets).
> Additional information about Vartable format can be coded into chunk flags.
> The first approach is to precalculate chunk total size, then choose the most
> compact format and write a chunk.
> The second approach is to write a chunk with the widest format then convert
> the chunk into the most compact format in place.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-14728) Change IGNITE_PDS_WAL_REBALANCE_THRESHOLD from System property to Distributed property
[ https://issues.apache.org/jira/browse/IGNITE-14728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381399#comment-17381399 ] Alexander Lapin commented on IGNITE-14728: -- [~erixon] LGTM > Change IGNITE_PDS_WAL_REBALANCE_THRESHOLD from System property to Distributed > property > -- > > Key: IGNITE-14728 > URL: https://issues.apache.org/jira/browse/IGNITE-14728 > Project: Ignite > Issue Type: Improvement >Reporter: Eduard Rakhmankulov >Assignee: Eduard Rakhmankulov >Priority: Minor > Time Spent: 1h 20m > Remaining Estimate: 0h > > We have a system property named "IGNITE_PDS_WAL_REBALANCE_THRESHOLD", that > determines a count of entries of partition starting from which cluster will > try applying historical rebalance for that partition. > But the system property is not convenient to use and is hidden from most part > of users. > I propose the creation of a new DMS property *history.rebalance.threshold* > instead of a system property. > The next algorithm will be used: > # On node start *history.rebalance.threshold* property will be checked. > # If there is no value, then the old system property > *IGNITE_PDS_WAL_REBALANCE_THRESHOLD* value will be written to DMS. > # If a system property is not defined then the default value from java will > be written to DMS (*500*). > # On property check print to log value and source of value. > Mark the system property *IGNITE_PDS_WAL_REBALANCE_THRESHOLD* as deprecated. > Dev list discussion > [http://apache-ignite-developers.2346864.n4.nabble.com/Change-IGNITE-PDS-WAL-REBALANCE-THRESHOLD-from-System-property-to-Configuraton-tp52560.html] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-15099) Wrong heartbeat update while waiting for a checkpoint by timeout
[
https://issues.apache.org/jira/browse/IGNITE-15099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381379#comment-17381379
]
Ilya Shishkov commented on IGNITE-15099:
[~alex_pl], LGTM.
> Wrong heartbeat update while waiting for a checkpoint by timeout
>
>
> Key: IGNITE-15099
> URL: https://issues.apache.org/jira/browse/IGNITE-15099
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.11, 2.12
>Reporter: Ilya Shishkov
>Assignee: Aleksey Plekhanov
>Priority: Minor
> Labels: ise
> Attachments: Checkpointer_fake_block_master.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This problem occurs under these conditions:
> * native persistence is turned on
> * failureDetectionTimeout < checkpointFrequency
> * checkpoints are sometimes skipped by timeout (more often, more probable
> the problem occurrence)
> There is a race condition between a listener execution and finishing of a
> pending future (see CheckpointContextImpl#executor body [1]). In some cases
> future can finish before listener closure, therefore updating of a heartbeat
> in listener can occur after call of _blockingSectionBegin_ in
> Checkpointer#waitCheckpointEvent, i.e. after Checkpointer started to wait for
> next checkpoint (see [2]).
> {code:java|title=CheckpointContextImpl#executor}
> @Override public Executor executor() {
> return asyncRunner == null ? null : cmd -> {
> try {
> GridFutureAdapter res = new GridFutureAdapter<>();
> res.listen(fut -> heartbeatUpdater.updateHeartbeat()); //
> Listener is invoked concurrently with pending future finish
> asyncRunner.execute(U.wrapIgniteFuture(cmd, res));
> pendingTaskFuture.add(res);
> }
> catch (RejectedExecutionException e) {
> assert false : "A task should never be rejected by async
> runner";
> }
> };
> }
> {code}
> {code:java|title=Checkpointer#waitCheckpointEvent}
> try {
> synchronized (this) {
> long remaining = U.nanosToMillis(scheduledCp.nextCpNanos -
> System.nanoTime());
> while (remaining > 0 && !isCancelled()) {
> blockingSectionBegin();
> try {
> wait(remaining);
> // At this point and till blockingSectionEnd call heartbeat
> should be equal to Long.MAX_VALUE
> remaining = U.nanosToMillis(scheduledCp.nextCpNanos -
> System.nanoTime());
> }
> finally {
> blockingSectionEnd();
> }
> }
> }
> }
> {code}
>
> If interval between checkpoints (_checkpointFrequency_) is greater than the
> _failureDetectionTimeout_, then update of heartbeat in _blockingSectionEnd_
> may cause an error message in log, because a checkpoint thread is treated as
> blocked (but in fact it was not).
> *Reproducer of problem:* [3]. Even if test was not failed you can see log
> message with incorrect heartbeat after waiting for checkpoint.
>
> Links:
> #
> [CheckpointContextImpl#executor|https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/checkpoint/CheckpointContextImpl.java#L104]
> #
> [Checkpointer#waitCheckpointEvent|https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/checkpoint/Checkpointer.java#L816]
> # Reproducer: [^Checkpointer_fake_block_master.patch].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-15069) Fix flaky test: GridCommandHandlerTest.testSetState
[
https://issues.apache.org/jira/browse/IGNITE-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376044#comment-17376044
]
Sergei Ryzhov edited comment on IGNITE-15069 at 7/15/21, 2:18 PM:
--
problem with DiscoveryDataClusterState#state()
the cache.put occurred before the cluster changed its DiscoveryDataClusterState
was (Author: ryzhovsv):
problem with DiscoveryDataClusterState#state()
> Fix flaky test: GridCommandHandlerTest.testSetState
> ---
>
> Key: IGNITE-15069
> URL: https://issues.apache.org/jira/browse/IGNITE-15069
> Project: Ignite
> Issue Type: Test
>Reporter: Sergei Ryzhov
>Assignee: Sergei Ryzhov
>Priority: Minor
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_ControlUtilityZookeeper/6073175?expandedTest=build%3A%28id%3A6073175%29%2Cid%3A22184=6073175_22184_1623.22131.22184=debug
> {code:java}
> [2021-07-05 06:25:53,131][ERROR][main][root] Test failed
> [test=GridCommandHandlerTest#testSetState, duration=8469]
> javax.cache.CacheException: class
> org.apache.ignite.transactions.TransactionRollbackException: Transaction has
> been rolled back: 0ed43b47a71--0e1f-597b--0001
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1277)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2084)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1320)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:868)
> at
> org.apache.ignite.internal.processors.cache.ClusterStateTestUtils.putSomeDataAndCheck(ClusterStateTestUtils.java:101)
> at
> org.apache.ignite.util.GridCommandHandlerTest.setState(GridCommandHandlerTest.java:979)
> at
> org.apache.ignite.util.GridCommandHandlerTest.testSetState(GridCommandHandlerTest.java:947)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2432)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class
> org.apache.ignite.transactions.TransactionRollbackException: Transaction has
> been rolled back: 0ed43b47a71--0e1f-597b--0001
> at
> org.apache.ignite.internal.util.IgniteUtils$11.apply(IgniteUtils.java:977)
> at
> org.apache.ignite.internal.util.IgniteUtils$11.apply(IgniteUtils.java:975)
> ... 17 more
> Caused by: class
> org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
> Transaction has been rolled back:
> 0ed43b47a71--0e1f-597b--0001
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4429)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2624)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2602)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2581)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1317)
> ... 14 more
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> perform cache operation (cache topology is not valid): part_cache
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxImplicitSingleStateImpl.validateTopology(IgniteTxImplicitSingleStateImpl.java:141)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:1105)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.prepareAsync(GridDhtTxLocal.java:400)
> at
>
[jira] [Commented] (IGNITE-15086) Design a public tx API
[
https://issues.apache.org/jira/browse/IGNITE-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381348#comment-17381348
]
Vyacheslav Koptilin commented on IGNITE-15086:
--
In general, the proposed API looks good to me.
> Design a public tx API
> --
>
> Key: IGNITE-15086
> URL: https://issues.apache.org/jira/browse/IGNITE-15086
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Labels: iep-61, ignite-3
> Fix For: 3.0.0-alpha3
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Design a public tx API.
> The proposed design includes async and sync APIs to execute a transaction.
> The transaction facade looks like this:
> {code:java}
> /**
> * Ignite Transactions facade.
> */
> public interface IgniteTransactions {
> /**
> * Begins the transaction.
> *
> * @return The future.
> */
> CompletableFuture beginAsync();
> /**
> * Synchronously executes a closure within a transaction.
> *
> * If the closure is executed normally (no exceptions), the transaction
> is automatically committed.
> *
> * @param clo The closure.
> */
> void runInTransaction(Consumer clo);
> }
> {code}
> Transacton interface:
> {code:java}
> /**
> * The transaction.
> */
> public interface Transaction {
> /**
> * Synchronously commits a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> */
> void commit();
> /**
> * Asynchronously commits a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> *
> * @return The future.
> */
> CompletableFuture commitAsync();
> /**
> * Synchronously rolls back a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> */
> void rollback();
> /**
> * Asynchronously rolls back a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> *
> * @return The future.
> */
> CompletableFuture rollbackAsync();
> }
> {code}
> Example of synchronous transaction:
> {code:java}
> igniteTransactions.runInTransaction(tx -> {
> Table txTable = table.withTransaction(tx); // table view enlisted
> in the transaction.
> txTable.upsertAsync(...);
> tx.commit();
> });
> {code}
> Example of asynchronous transaction:
> {code:java}
> igniteTransactions.beginAsync()
> .thenApply(tx -> accounts.withTransaction(tx))
> .thenCompose(txTbl -> txTbl.upsertAsync(...).thenApply(i ->
> txTbl.transaction()))
> .thenCompose(Transaction::commitAsync);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-15086) Design a public tx API
[
https://issues.apache.org/jira/browse/IGNITE-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-15086:
-
Description:
Design a public tx API.
The proposed design includes async and sync APIs to execute a transaction.
The transaction facade looks like this:
{code:java}
/**
* Ignite Transactions facade.
*/
public interface IgniteTransactions {
/**
* Begins the transaction.
*
* @return The future.
*/
CompletableFuture beginAsync();
/**
* Synchronously executes a closure within a transaction.
*
* If the closure is executed normally (no exceptions), the transaction is
automatically committed.
*
* @param clo The closure.
*/
void runInTransaction(Consumer clo);
}
{code}
Transacton interface:
{code:java}
/**
* The transaction.
*/
public interface Transaction {
/**
* Synchronously commits a transaction.
* Does nothing if it's already finished by committing or rolling back.
*/
void commit();
/**
* Asynchronously commits a transaction.
* Does nothing if it's already finished by committing or rolling back.
*
* @return The future.
*/
CompletableFuture commitAsync();
/**
* Synchronously rolls back a transaction.
* Does nothing if it's already finished by committing or rolling back.
*/
void rollback();
/**
* Asynchronously rolls back a transaction.
* Does nothing if it's already finished by committing or rolling back.
*
* @return The future.
*/
CompletableFuture rollbackAsync();
}
{code}
Example of synchronous transaction:
{code:java}
igniteTransactions.runInTransaction(tx -> {
Table txTable = table.withTransaction(tx); // table view enlisted
in the transaction.
txTable.upsertAsync(...);
tx.commit();
});
{code}
Example of asynchronous transaction:
{code:java}
igniteTransactions.beginAsync()
.thenApply(tx -> accounts.withTransaction(tx))
.thenCompose(txTbl -> txTbl.upsertAsync(...).thenApply(i ->
txTbl.transaction()))
.thenCompose(Transaction::commitAsync);
{code}
was:
Design a public tx API.
The proposed design includes async and sync APIs to execute a transaction.
The transaction facade looks like this:
{code}
/**
* Ignite Transactions facade.
*/
public interface IgniteTransactions {
/**
* Begins the transaction.
*
* @return The future.
*/
CompletableFuture beginAsync();
/**
* Synchronously executes a closure within a transaction.
*
* If the closure is executed normally (no exceptions), the transaction is
automatically committed.
*
* @param clo The closure.
*/
void runInTransaction(Consumer clo);
}
{code}
> Design a public tx API
> --
>
> Key: IGNITE-15086
> URL: https://issues.apache.org/jira/browse/IGNITE-15086
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Labels: iep-61, ignite-3
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Design a public tx API.
> The proposed design includes async and sync APIs to execute a transaction.
> The transaction facade looks like this:
> {code:java}
> /**
> * Ignite Transactions facade.
> */
> public interface IgniteTransactions {
> /**
> * Begins the transaction.
> *
> * @return The future.
> */
> CompletableFuture beginAsync();
> /**
> * Synchronously executes a closure within a transaction.
> *
> * If the closure is executed normally (no exceptions), the transaction
> is automatically committed.
> *
> * @param clo The closure.
> */
> void runInTransaction(Consumer clo);
> }
> {code}
> Transacton interface:
> {code:java}
> /**
> * The transaction.
> */
> public interface Transaction {
> /**
> * Synchronously commits a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> */
> void commit();
> /**
> * Asynchronously commits a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> *
> * @return The future.
> */
> CompletableFuture commitAsync();
> /**
> * Synchronously rolls back a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> */
> void rollback();
> /**
> * Asynchronously rolls back a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> *
> * @return The future.
> */
> CompletableFuture rollbackAsync();
> }
> {code}
> Example of synchronous transaction:
> {code:java}
>
[jira] [Updated] (IGNITE-15086) Design a public tx API
[
https://issues.apache.org/jira/browse/IGNITE-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-15086:
-
Fix Version/s: 3.0.0-alpha3
> Design a public tx API
> --
>
> Key: IGNITE-15086
> URL: https://issues.apache.org/jira/browse/IGNITE-15086
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Labels: iep-61, ignite-3
> Fix For: 3.0.0-alpha3
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Design a public tx API.
> The proposed design includes async and sync APIs to execute a transaction.
> The transaction facade looks like this:
> {code:java}
> /**
> * Ignite Transactions facade.
> */
> public interface IgniteTransactions {
> /**
> * Begins the transaction.
> *
> * @return The future.
> */
> CompletableFuture beginAsync();
> /**
> * Synchronously executes a closure within a transaction.
> *
> * If the closure is executed normally (no exceptions), the transaction
> is automatically committed.
> *
> * @param clo The closure.
> */
> void runInTransaction(Consumer clo);
> }
> {code}
> Transacton interface:
> {code:java}
> /**
> * The transaction.
> */
> public interface Transaction {
> /**
> * Synchronously commits a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> */
> void commit();
> /**
> * Asynchronously commits a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> *
> * @return The future.
> */
> CompletableFuture commitAsync();
> /**
> * Synchronously rolls back a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> */
> void rollback();
> /**
> * Asynchronously rolls back a transaction.
> * Does nothing if it's already finished by committing or rolling back.
> *
> * @return The future.
> */
> CompletableFuture rollbackAsync();
> }
> {code}
> Example of synchronous transaction:
> {code:java}
> igniteTransactions.runInTransaction(tx -> {
> Table txTable = table.withTransaction(tx); // table view enlisted
> in the transaction.
> txTable.upsertAsync(...);
> tx.commit();
> });
> {code}
> Example of asynchronous transaction:
> {code:java}
> igniteTransactions.beginAsync()
> .thenApply(tx -> accounts.withTransaction(tx))
> .thenCompose(txTbl -> txTbl.upsertAsync(...).thenApply(i ->
> txTbl.transaction()))
> .thenCompose(Transaction::commitAsync);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-15129) Improve timestamp generation for transactions
[ https://issues.apache.org/jira/browse/IGNITE-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Scherbakov updated IGNITE-15129: --- Description: Current timestamp implementation doesn't include globalId part, so is useful only then all transactions ids are generated on single node. It should be improved by adding global unique jvm id to the timestamp. Additionally, current timestamp uses nanoSeconds, which is far away from real time. Instead, NTP timestamp can be used as a localTime part. IgniteUuid can be used for this purpose as well - it's the simplest solution. Also, the performance of current implementation may not be optimal - probably it should be CAS powered or/and use optimistic locking (StampedLock). was: Current timestamp implementation doesn't include globalId part, so is useful only then all transactions ids are generated on single node. It should be improved by adding global unique jvm id to the timestamp. Additionally, current timestamp uses nanoSeconds, which is far away from real time. Instead, NTP timestamp can be used as a localTime part. IgniteUuid can be used for this purpose as well - it's the simplest solution. Also, the perfornace of current implementation may not be optimal - probably it should be CAS powered or/and use optimistic locking (StampedLock). > Improve timestamp generation for transactions > - > > Key: IGNITE-15129 > URL: https://issues.apache.org/jira/browse/IGNITE-15129 > Project: Ignite > Issue Type: Improvement >Reporter: Alexey Scherbakov >Priority: Major > Labels: iep-61, ignite-3 > Fix For: 3.0.0-alpha3 > > > Current timestamp implementation doesn't include globalId part, so is useful > only then all transactions ids are generated on single node. > It should be improved by adding global unique jvm id to the timestamp. > Additionally, current timestamp uses nanoSeconds, which is far away from real > time. > Instead, NTP timestamp can be used as a localTime part. > IgniteUuid can be used for this purpose as well - it's the simplest solution. > Also, the performance of current implementation may not be optimal - probably > it should be CAS powered or/and use optimistic locking (StampedLock). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15086) Design a public tx API
[
https://issues.apache.org/jira/browse/IGNITE-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-15086:
-
Description:
Design a public tx API.
The proposed design includes async and sync APIs to execute a transaction.
The transaction facade looks like this:
{code}
/**
* Ignite Transactions facade.
*/
public interface IgniteTransactions {
/**
* Begins the transaction.
*
* @return The future.
*/
CompletableFuture beginAsync();
/**
* Synchronously executes a closure within a transaction.
*
* If the closure is executed normally (no exceptions), the transaction is
automatically committed.
*
* @param clo The closure.
*/
void runInTransaction(Consumer clo);
}
{code}
was:
Design a public tx API.
The proposed design includes async and sync APIs to execute a transaction.
The transaction facade looks like this:
{code}
/**
* Ignite Transactions facade.
*/
public interface IgniteTransactions {
/**
* Begins the transaction.
*
* @return The future.
*/
CompletableFuture beginAsync();
/**
* Synchronously executes a closure within a transaction.
*
* @param clo The closure.
*/
void runInTransaction(Consumer clo);
}
{code}
> Design a public tx API
> --
>
> Key: IGNITE-15086
> URL: https://issues.apache.org/jira/browse/IGNITE-15086
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Labels: iep-61, ignite-3
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Design a public tx API.
> The proposed design includes async and sync APIs to execute a transaction.
> The transaction facade looks like this:
> {code}
> /**
> * Ignite Transactions facade.
> */
> public interface IgniteTransactions {
> /**
> * Begins the transaction.
> *
> * @return The future.
> */
> CompletableFuture beginAsync();
> /**
> * Synchronously executes a closure within a transaction.
> *
> * If the closure is executed normally (no exceptions), the transaction
> is automatically committed.
> *
> * @param clo The closure.
> */
> void runInTransaction(Consumer clo);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-15069) Fix flaky test: GridCommandHandlerTest.testSetState
[
https://issues.apache.org/jira/browse/IGNITE-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381319#comment-17381319
]
Ignite TC Bot commented on IGNITE-15069:
{panel:title=Branch: [pull/9252/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9252/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6084633buildTypeId=IgniteTests24Java8_RunAll]
> Fix flaky test: GridCommandHandlerTest.testSetState
> ---
>
> Key: IGNITE-15069
> URL: https://issues.apache.org/jira/browse/IGNITE-15069
> Project: Ignite
> Issue Type: Test
>Reporter: Sergei Ryzhov
>Assignee: Sergei Ryzhov
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_ControlUtilityZookeeper/6073175?expandedTest=build%3A%28id%3A6073175%29%2Cid%3A22184=6073175_22184_1623.22131.22184=debug
> {code:java}
> [2021-07-05 06:25:53,131][ERROR][main][root] Test failed
> [test=GridCommandHandlerTest#testSetState, duration=8469]
> javax.cache.CacheException: class
> org.apache.ignite.transactions.TransactionRollbackException: Transaction has
> been rolled back: 0ed43b47a71--0e1f-597b--0001
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1277)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2084)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1320)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:868)
> at
> org.apache.ignite.internal.processors.cache.ClusterStateTestUtils.putSomeDataAndCheck(ClusterStateTestUtils.java:101)
> at
> org.apache.ignite.util.GridCommandHandlerTest.setState(GridCommandHandlerTest.java:979)
> at
> org.apache.ignite.util.GridCommandHandlerTest.testSetState(GridCommandHandlerTest.java:947)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2432)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class
> org.apache.ignite.transactions.TransactionRollbackException: Transaction has
> been rolled back: 0ed43b47a71--0e1f-597b--0001
> at
> org.apache.ignite.internal.util.IgniteUtils$11.apply(IgniteUtils.java:977)
> at
> org.apache.ignite.internal.util.IgniteUtils$11.apply(IgniteUtils.java:975)
> ... 17 more
> Caused by: class
> org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
> Transaction has been rolled back:
> 0ed43b47a71--0e1f-597b--0001
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4429)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2624)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2602)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2581)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1317)
> ... 14 more
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> perform cache operation (cache topology is not valid): part_cache
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxImplicitSingleStateImpl.validateTopology(IgniteTxImplicitSingleStateImpl.java:141)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:1105)
> at
>
[jira] [Commented] (IGNITE-14864) Schema update. Merge multiple converters stages.
[ https://issues.apache.org/jira/browse/IGNITE-14864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381313#comment-17381313 ] Konstantin Orlov commented on IGNITE-14864: --- [~amashenkov], I've left few comments. Please see PR. > Schema update. Merge multiple converters stages. > > > Key: IGNITE-14864 > URL: https://issues.apache.org/jira/browse/IGNITE-14864 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Fix For: 3.0.0-alpha3 > > Time Spent: 50m > Remaining Estimate: 0h > > Assumed schema was updated a number of times and there are n >1 converters to > apply. > Let's chain converters to apply all at once rather then one-by-one having a > row of intermediate version in between. > Also, opposite changes like "adding column A" then "remove column A" or > "Changing column default" then "remove column A" can be effectively omitted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-15129) Improve timestamp generation for transactions
Alexey Scherbakov created IGNITE-15129: -- Summary: Improve timestamp generation for transactions Key: IGNITE-15129 URL: https://issues.apache.org/jira/browse/IGNITE-15129 Project: Ignite Issue Type: Improvement Reporter: Alexey Scherbakov Fix For: 3.0.0-alpha3 Current timestamp implementation doesn't include globalId part, so is useful only then all transactions ids are generated on single node. It should be improved by adding global unique jvm id to the timestamp. Additionally, current timestamp uses nanoSeconds, which is far away from real time. Instead, NTP timestamp can be used as a localTime part. IgniteUuid can be used for this purpose as well - it's the simplest solution. Also, the perfornace of current implementation may not be optimal - probably it should be CAS powered or/and use optimistic locking (StampedLock). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-15099) Wrong heartbeat update while waiting for a checkpoint by timeout
[
https://issues.apache.org/jira/browse/IGNITE-15099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381307#comment-17381307
]
Aleksey Plekhanov commented on IGNITE-15099:
[~shishkovilja], can you please have a look at the patch?
> Wrong heartbeat update while waiting for a checkpoint by timeout
>
>
> Key: IGNITE-15099
> URL: https://issues.apache.org/jira/browse/IGNITE-15099
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.11, 2.12
>Reporter: Ilya Shishkov
>Assignee: Aleksey Plekhanov
>Priority: Minor
> Labels: ise
> Attachments: Checkpointer_fake_block_master.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This problem occurs under these conditions:
> * native persistence is turned on
> * failureDetectionTimeout < checkpointFrequency
> * checkpoints are sometimes skipped by timeout (more often, more probable
> the problem occurrence)
> There is a race condition between a listener execution and finishing of a
> pending future (see CheckpointContextImpl#executor body [1]). In some cases
> future can finish before listener closure, therefore updating of a heartbeat
> in listener can occur after call of _blockingSectionBegin_ in
> Checkpointer#waitCheckpointEvent, i.e. after Checkpointer started to wait for
> next checkpoint (see [2]).
> {code:java|title=CheckpointContextImpl#executor}
> @Override public Executor executor() {
> return asyncRunner == null ? null : cmd -> {
> try {
> GridFutureAdapter res = new GridFutureAdapter<>();
> res.listen(fut -> heartbeatUpdater.updateHeartbeat()); //
> Listener is invoked concurrently with pending future finish
> asyncRunner.execute(U.wrapIgniteFuture(cmd, res));
> pendingTaskFuture.add(res);
> }
> catch (RejectedExecutionException e) {
> assert false : "A task should never be rejected by async
> runner";
> }
> };
> }
> {code}
> {code:java|title=Checkpointer#waitCheckpointEvent}
> try {
> synchronized (this) {
> long remaining = U.nanosToMillis(scheduledCp.nextCpNanos -
> System.nanoTime());
> while (remaining > 0 && !isCancelled()) {
> blockingSectionBegin();
> try {
> wait(remaining);
> // At this point and till blockingSectionEnd call heartbeat
> should be equal to Long.MAX_VALUE
> remaining = U.nanosToMillis(scheduledCp.nextCpNanos -
> System.nanoTime());
> }
> finally {
> blockingSectionEnd();
> }
> }
> }
> }
> {code}
>
> If interval between checkpoints (_checkpointFrequency_) is greater than the
> _failureDetectionTimeout_, then update of heartbeat in _blockingSectionEnd_
> may cause an error message in log, because a checkpoint thread is treated as
> blocked (but in fact it was not).
> *Reproducer of problem:* [3]. Even if test was not failed you can see log
> message with incorrect heartbeat after waiting for checkpoint.
>
> Links:
> #
> [CheckpointContextImpl#executor|https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/checkpoint/CheckpointContextImpl.java#L104]
> #
> [Checkpointer#waitCheckpointEvent|https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/checkpoint/Checkpointer.java#L816]
> # Reproducer: [^Checkpointer_fake_block_master.patch].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-15086) Design a public tx API
[
https://issues.apache.org/jira/browse/IGNITE-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381249#comment-17381249
]
Alexey Scherbakov commented on IGNITE-15086:
It seems the proposed API is fine at this point.
Dev-list discussion [1]
Waiting for a formal review.
[1]
[https://lists.apache.org/thread.html/r0d66b3dc788e3caa2afc75aa8792e82a25abe3cda213500cb6238b2c%40%3Cdev.ignite.apache.org%3E]
> Design a public tx API
> --
>
> Key: IGNITE-15086
> URL: https://issues.apache.org/jira/browse/IGNITE-15086
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
> Labels: iep-61, ignite-3
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Design a public tx API.
> The proposed design includes async and sync APIs to execute a transaction.
> The transaction facade looks like this:
> {code}
> /**
> * Ignite Transactions facade.
> */
> public interface IgniteTransactions {
> /**
> * Begins the transaction.
> *
> * @return The future.
> */
> CompletableFuture beginAsync();
> /**
> * Synchronously executes a closure within a transaction.
> *
> * @param clo The closure.
> */
> void runInTransaction(Consumer clo);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-14866) Calcite. Check SQL function works
[ https://issues.apache.org/jira/browse/IGNITE-14866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381239#comment-17381239 ] Yury Gerzhedovich commented on IGNITE-14866: List of function which works: JSON: JSON_EXISTS; JSON_VALUE; JSON_QUERY; JSON_OBJECT; JSON_ARRAY; JSON_PRETTY STRING: CHAR_LENGTH; CHARACTER_LENGTH; UPPER; LOWER; POSITION; TRIM; OVERLAY; SUBSTRING; INITCAP; STRCMP REVERSE REGEXP_REPLACE SHA1 MD5 LTRIM TO_BASE64 FROM_BASE64 COMPRESS CONCAT TRANSLATE; ASCII; LEFT RIGHT REPEAT SOUNDEX NUMERIC: POWER; ABS; MOD; SQRT; LN; LOG10; EXP; CEIL; FLOOR; RAND ACOS; ASIN; ATAN; ATAN2; CBRT; COS; COT; DEGREES; PI(); RADIANS; ROUND; SIGN; SIN; TAN; TRUNCATE; CHR COSH SINH TANH GENERAL: NULLIF; COALESCE; CAST; NVL GREATEST TIMESTAMP: TIMESTAMP_ADD; TIMESTAMP_DIFF; EXTRACT; LAST_DAY > Calcite. Check SQL function works > - > > Key: IGNITE-14866 > URL: https://issues.apache.org/jira/browse/IGNITE-14866 > Project: Ignite > Issue Type: New Feature > Components: sql >Reporter: Yury Gerzhedovich >Assignee: Yury Gerzhedovich >Priority: Major > Labels: calcite2-required, calcite3-required > Time Spent: 10m > Remaining Estimate: 0h > > We need to write tests on existing simple SQL functions (nor aggregation or > window functions) and have a list of supported functions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-15128) Take own control of SQL functions
Yury Gerzhedovich created IGNITE-15128: -- Summary: Take own control of SQL functions Key: IGNITE-15128 URL: https://issues.apache.org/jira/browse/IGNITE-15128 Project: Ignite Issue Type: Improvement Components: sql Reporter: Yury Gerzhedovich As of now, we use a set of 4 database function dialects: SqlLibrary.STANDARD, SqlLibrary.POSTGRESQL, SqlLibrary.ORACLE, SqlLibrary.MYSQL Seems we should have owned our dialect with a subset of the aforementioned functions and have the ability to modify already exists functions and add a new one. During implementation need to sort out similar functions and choose just one of them to avoid duplication, See : org.apache.calcite.util.BuiltInMethod org.apache.calcite.sql.fun.SqlLibraryOperators org.apache.calcite.runtime.SqlFunctions org.apache.ignite.internal.processors.query.calcite.exec.exp.RexImpTable -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-15127) Calcite. Investigate CoreRules.JOIN_COMMUTE[_OUTER] usage.
Stanilovsky Evgeny created IGNITE-15127: --- Summary: Calcite. Investigate CoreRules.JOIN_COMMUTE[_OUTER] usage. Key: IGNITE-15127 URL: https://issues.apache.org/jira/browse/IGNITE-15127 Project: Ignite Issue Type: Improvement Components: sql Reporter: Stanilovsky Evgeny Assignee: Stanilovsky Evgeny Looks like join commute rule can dramatically reduce some join costs, investigation and tests are required. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-15126) Cluster hangs when error is thrown on activation
Aleksey Plekhanov created IGNITE-15126:
--
Summary: Cluster hangs when error is thrown on activation
Key: IGNITE-15126
URL: https://issues.apache.org/jira/browse/IGNITE-15126
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
When an error occurs on cluster activation (In methods
{{IgniteChangeGlobalStateSupport.onActivate()}}) state-change process hangs
(stay in "in transition" state) and switches the cluster to an inoperable state
even if the problematic node is stopped by the failure handler.
Reproducer:
{code:java}
public class ErrorOnActivationTest extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
return super.getConfiguration(igniteInstanceName).setFailureHandler(new
StopNodeFailureHandler())
.setClusterStateOnStart(ClusterState.INACTIVE);
}
@Test
public void testErrorOnActivation() throws Exception {
Ignite ignite = startGrid(getConfiguration("test1"));
startGrid(getConfiguration("test2"))
.context().internalSubscriptionProcessor().registerDatabaseListener(
new DatabaseLifecycleListener() {
@Override public void
afterInitialise(IgniteCacheDatabaseSharedManager mgr) throws
IgniteCheckedException {
throw new IgniteCheckedException("Test");
}
}
);
ignite.cluster().state(ClusterState.ACTIVE);
startClientGrid(); // Hangs here.
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-15069) Fix flaky test: GridCommandHandlerTest.testSetState
[
https://issues.apache.org/jira/browse/IGNITE-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381144#comment-17381144
]
Ignite TC Bot commented on IGNITE-15069:
{panel:title=Branch: [pull/9252/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9252/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6084633buildTypeId=IgniteTests24Java8_RunAll]
> Fix flaky test: GridCommandHandlerTest.testSetState
> ---
>
> Key: IGNITE-15069
> URL: https://issues.apache.org/jira/browse/IGNITE-15069
> Project: Ignite
> Issue Type: Test
>Reporter: Sergei Ryzhov
>Assignee: Sergei Ryzhov
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_ControlUtilityZookeeper/6073175?expandedTest=build%3A%28id%3A6073175%29%2Cid%3A22184=6073175_22184_1623.22131.22184=debug
> {code:java}
> [2021-07-05 06:25:53,131][ERROR][main][root] Test failed
> [test=GridCommandHandlerTest#testSetState, duration=8469]
> javax.cache.CacheException: class
> org.apache.ignite.transactions.TransactionRollbackException: Transaction has
> been rolled back: 0ed43b47a71--0e1f-597b--0001
> at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1277)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2084)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1320)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:868)
> at
> org.apache.ignite.internal.processors.cache.ClusterStateTestUtils.putSomeDataAndCheck(ClusterStateTestUtils.java:101)
> at
> org.apache.ignite.util.GridCommandHandlerTest.setState(GridCommandHandlerTest.java:979)
> at
> org.apache.ignite.util.GridCommandHandlerTest.testSetState(GridCommandHandlerTest.java:947)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2432)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class
> org.apache.ignite.transactions.TransactionRollbackException: Transaction has
> been rolled back: 0ed43b47a71--0e1f-597b--0001
> at
> org.apache.ignite.internal.util.IgniteUtils$11.apply(IgniteUtils.java:977)
> at
> org.apache.ignite.internal.util.IgniteUtils$11.apply(IgniteUtils.java:975)
> ... 17 more
> Caused by: class
> org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
> Transaction has been rolled back:
> 0ed43b47a71--0e1f-597b--0001
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4429)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2624)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2602)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2581)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1317)
> ... 14 more
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> perform cache operation (cache topology is not valid): part_cache
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxImplicitSingleStateImpl.validateTopology(IgniteTxImplicitSingleStateImpl.java:141)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:1105)
> at
>
[jira] [Updated] (IGNITE-14579) Start rest manager
[ https://issues.apache.org/jira/browse/IGNITE-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-14579: Remaining Estimate: 72h Original Estimate: 72h > Start rest manager > -- > > Key: IGNITE-14579 > URL: https://issues.apache.org/jira/browse/IGNITE-14579 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Kirill Gusakov >Priority: Major > Labels: iep-73 > Original Estimate: 72h > Remaining Estimate: 72h > > Update IgntionImpl with code that will start rest manager. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-15125) Implement naive baseline
Alexander Lapin created IGNITE-15125: Summary: Implement naive baseline Key: IGNITE-15125 URL: https://issues.apache.org/jira/browse/IGNITE-15125 Project: Ignite Issue Type: Bug Reporter: Alexander Lapin Assignee: Alexander Lapin In order to satisfy the needs on assignments recalculation on topology changes it's required to implement baseline concept. It might be a naive one that will contain all topology nodes and will change on every node add/left event. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-15122) Calcite. Remove sync mode of RootNode#closeInternal
[ https://issues.apache.org/jira/browse/IGNITE-15122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17381063#comment-17381063 ] Stanilovsky Evgeny commented on IGNITE-15122: - looks good. > Calcite. Remove sync mode of RootNode#closeInternal > --- > > Key: IGNITE-15122 > URL: https://issues.apache.org/jira/browse/IGNITE-15122 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Konstantin Orlov >Assignee: Konstantin Orlov >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Currently we are waiting on future to complete inside RootNode#closeInternal. > It potentially could causes a query to stale in case someone will invoke this > method inside a taskExecutor. -- This message was sent by Atlassian Jira (v8.3.4#803005)
