[jira] [Assigned] (IGNITE-17883) Remove the invoke method from KeyValurView and RecordView interfaces
[
https://issues.apache.org/jira/browse/IGNITE-17883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin reassigned IGNITE-17883:
Assignee: Vyacheslav Koptilin
> Remove the invoke method from KeyValurView and RecordView interfaces
>
>
> Key: IGNITE-17883
> URL: https://issues.apache.org/jira/browse/IGNITE-17883
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> _KeyValueView_ and _RecordView_ provide the following methods:
> {code:java}
> /**
> * Executes invoke processor code against the value associated with the
> provided key.
> *
> * @param tx The transaction or {@code null} to auto commit.
> * @param key A key associated with the value that invoke processor will be
> applied to. The key cannot be {@code null}.
> * @param proc An invocation processor.
> * @param args Optional invoke processor arguments.
> * @param Invoke processor result type.
> * @return Result of the processing.
> * @see InvokeProcessor
> */
> R invoke(@Nullable Transaction tx, @NotNull K key,
> InvokeProcessor proc, Serializable... args);
> /**
> * Asynchronously executes invoke processor code against the value associated
> with the provided key.
> *
> * @param tx The transaction or {@code null} to auto commit.
> * @param key A key associated with the value that invoke processor will be
> applied to. The key cannot be {@code null}.
> * @param proc An invocation processor.
> * @param args Optional invoke processor arguments.
> * @param Invoke processor result type.
> * @return Future representing pending completion of the operation.
> * @see InvokeProcessor
> */
> @NotNull CompletableFuture invokeAsync(
> @Nullable Transaction tx,
> @NotNull K key,
> InvokeProcessor proc,
> Serializable... args);
> /**
> * Executes invoke processor code against values associated with the provided
> keys.
> *
> * @param tx The transaction or {@code null} to auto commit.
> * @param Invoke processor result type.
> * @param keys Ordered collection of keys which values associated with should
> be processed. The keys cannot be {@code null}.
> * @param proc An invocation processor.
> * @param args Optional invoke processor arguments.
> * @return Results of the processing.
> * @see InvokeProcessor
> */
> Map invokeAll(
> @Nullable Transaction tx,
> @NotNull Collection keys,
> InvokeProcessor proc,
> Serializable... args);
> /**
> * Asynchronously executes invoke processor code against values associated
> with the provided keys.
> *
> * @param tx The transaction or {@code null} to auto commit.
> * @param Invoke processor result type.
> * @param keys Ordered collection of keys which values associated with should
> be processed. The keys cannot be {@code null}.
> * @param proc An invocation processor.
> * @param args Optional invoke processor arguments.
> * @return Future representing pending completion of the operation.
> * @see InvokeProcessor
> */
> @NotNull CompletableFuture> invokeAllAsync(
> @Nullable Transaction tx,
> @NotNull Collection keys,
> InvokeProcessor proc,
> Serializable... args);
> {code}
> The main problem with these methods assume that the user defined closure -
> _InvokeProcessor_ must be idempotent. This is a consequence of the
> limitations of the replication protocol. However, the implementation does not
> have the right way to check this requirement. For example, a simple closure
> that increments a value for a key could lead to data inconsistency under some
> circumstances.
> For now, _IgniteCompute.executeColocated_ can be used in order to handle
> these needs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19410) Node failure in case multiple nodes join and leave a cluster simultaneously with security is enabled.
[
https://issues.apache.org/jira/browse/IGNITE-19410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Pavlov updated IGNITE-19410:
---
Labels: ise (was: )
> Node failure in case multiple nodes join and leave a cluster simultaneously
> with security is enabled.
> --
>
> Key: IGNITE-19410
> URL: https://issues.apache.org/jira/browse/IGNITE-19410
> Project: Ignite
> Issue Type: Bug
>Reporter: Mikhail Petrov
>Priority: Major
> Labels: ise
> Attachments: NodeSecurityContextTest.java
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The case when nodes with security enabled join and leave the cluster
> simultaneously can cause the joining nodes to fail with the following
> exception:
> {code:java}
> [2023-05-03T14:54:31,208][ERROR][disco-notifier-worker-#332%ignite.NodeSecurityContextTest2%][IgniteTestResources]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
> failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION,
> err=java.lang.IllegalStateException: Failed to find security context for
> subject with given ID : 4725544a-f144-4486-a705-46b2ac200011]]
> java.lang.IllegalStateException: Failed to find security context for subject
> with given ID : 4725544a-f144-4486-a705-46b2ac200011
> at
> org.apache.ignite.internal.processors.security.IgniteSecurityProcessor.withContext(IgniteSecurityProcessor.java:164)
> ~[classes/:?]
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$3$SecurityAwareNotificationTask.run(GridDiscoveryManager.java:949)
> ~[classes/:?]
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.java:2822)
> ~[classes/:?]
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java:2860)
> [classes/:?]
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
> [classes/:?]
> at java.lang.Thread.run(Thread.java:750) [?:1.8.0_351] {code}
> Reproducer is attached.
> Simplified steps that leads to the failure:
> 1. The client node sends an arbitrary discovery message which produces an
> acknowledgement message when it processed by the all cluster nodes .
> 2. The client node gracefully leaves the cluster.
> 3. The new node joins the cluster and receives a topology snapshot that does
> not include the left client node.
> 4. The new node receives an acknowledgment for the message from the step 1
> and fails during its processing because message originator node is not listed
> in the current discovery cache or discovery cache history (see
> IgniteSecurityProcessor#withContext(java.util.UUID)) . This is because
> currently the GridDiscoveryManager#historicalNode method only aware of the
> topology history that occurs after a node has joined the cluster. The
> complete cluster topology history that exists at the time a new node joined
> the cluster is stored in GridDiscoveryManager#topHist and is not taken into
> account by the GridDiscoveryManager#historicalNode method.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-18047) High failure rate of test suite 'Cache 2' suite on TC1
[ https://issues.apache.org/jira/browse/IGNITE-18047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Shishkov updated IGNITE-18047: --- Description: Failure rate is about 30% on master branch. Most of time test suite fails by timeout [1], sometimes with an error: "Out of memory: GC overhead limit exceeded" [2]. But, on TC2 there are only a few failures [3] of the suite. # https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds # https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2/6821056 # https://ci2.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds was: Failure rate is abot 30% on master branch. Most of time test suite fails by timeout [1], sometimes with an error: "Out of memory: GC overhead limit exceeded" [2]. But, on TC2 there are only a few failures [3] of the suite. # https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds # https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2/6821056 # https://ci2.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds > High failure rate of test suite 'Cache 2' suite on TC1 > -- > > Key: IGNITE-18047 > URL: https://issues.apache.org/jira/browse/IGNITE-18047 > Project: Ignite > Issue Type: Task >Reporter: Ilya Shishkov >Priority: Major > Labels: ise, teamcity > > Failure rate is about 30% on master branch. Most of time test suite fails by > timeout [1], sometimes with an error: "Out of memory: GC overhead limit > exceeded" [2]. But, on TC2 there are only a few failures [3] of the suite. > # > https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds > # > https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2/6821056 > # > https://ci2.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-18047) High failure rate of test suite 'Cache 2' suite on TC1
[ https://issues.apache.org/jira/browse/IGNITE-18047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724190#comment-17724190 ] Ilya Shishkov edited comment on IGNITE-18047 at 5/19/23 8:28 AM: - The problem is not relevant now, see: https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds&page=2 was (Author: shishkovilja): The problem is not relevant now. > High failure rate of test suite 'Cache 2' suite on TC1 > -- > > Key: IGNITE-18047 > URL: https://issues.apache.org/jira/browse/IGNITE-18047 > Project: Ignite > Issue Type: Task >Reporter: Ilya Shishkov >Priority: Major > Labels: ise, teamcity > > Failure rate is about 30% on master branch. Most of time test suite fails by > timeout [1], sometimes with an error: "Out of memory: GC overhead limit > exceeded" [2]. But, on TC2 there are only a few failures [3] of the suite. > # > https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds > # > https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2/6821056 > # > https://ci2.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-18047) High failure rate of test suite 'Cache 2' suite on TC1
[ https://issues.apache.org/jira/browse/IGNITE-18047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Shishkov resolved IGNITE-18047. Resolution: Cannot Reproduce The problem is not relevant now. > High failure rate of test suite 'Cache 2' suite on TC1 > -- > > Key: IGNITE-18047 > URL: https://issues.apache.org/jira/browse/IGNITE-18047 > Project: Ignite > Issue Type: Task >Reporter: Ilya Shishkov >Priority: Major > Labels: ise, teamcity > > Failure rate is about 30% on master branch. Most of time test suite fails by > timeout [1], sometimes with an error: "Out of memory: GC overhead limit > exceeded" [2]. But, on TC2 there are only a few failures [3] of the suite. > # > https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds > # > https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2/6821056 > # > https://ci2.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Cache2?branch=%3Cdefault%3E&mode=builds -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18512) Add ACL to CdcKafkaReplicationTest
[ https://issues.apache.org/jira/browse/IGNITE-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Shishkov updated IGNITE-18512: --- Description: We should check, that CDC through Kafka works with ACL and without extra admin permisssions. (was: We should check, that CDC through Kafka works with ACL and without extra admin permisssions. Here is an example patch:) > Add ACL to CdcKafkaReplicationTest > -- > > Key: IGNITE-18512 > URL: https://issues.apache.org/jira/browse/IGNITE-18512 > Project: Ignite > Issue Type: Test > Components: extensions >Reporter: Ilya Shishkov >Priority: Minor > Labels: IEP-59, ise > > We should check, that CDC through Kafka works with ACL and without extra > admin permisssions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-18512) Add ACL to CdcKafkaReplicationTest
[ https://issues.apache.org/jira/browse/IGNITE-18512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Shishkov resolved IGNITE-18512. Resolution: Won't Fix Issue is not relevant, because CDC through Ignite does not use any admin API. > Add ACL to CdcKafkaReplicationTest > -- > > Key: IGNITE-18512 > URL: https://issues.apache.org/jira/browse/IGNITE-18512 > Project: Ignite > Issue Type: Test > Components: extensions >Reporter: Ilya Shishkov >Priority: Minor > Labels: IEP-59, ise > > We should check, that CDC through Kafka works with ACL and without extra > admin permisssions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-18992) CDC: Use single partition topic for metadata updates
[
https://issues.apache.org/jira/browse/IGNITE-18992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov resolved IGNITE-18992.
Release Note: Fixed in IGNITE-19369
Resolution: Fixed
> CDC: Use single partition topic for metadata updates
> -
>
> Key: IGNITE-18992
> URL: https://issues.apache.org/jira/browse/IGNITE-18992
> Project: Ignite
> Issue Type: Task
> Components: extensions
>Reporter: Ilya Shishkov
>Priority: Major
> Labels: IEP-59, ise
>
> In order to read data with guaranteed order, metadata topic must have only
> one partition. To achieve this we should:
> * Write binary types and marshaller mappings only to a single partition in
> {{IgniteToKafkaCdcStreamer#onMappings}} and {{#onTypes}}.
> * Print warnings into logs, when topic has more than one partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-18992) CDC: Use single partition topic for metadata updates
[
https://issues.apache.org/jira/browse/IGNITE-18992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-18992:
---
Ignite Flags: (was: Docs Required,Release Notes Required)
> CDC: Use single partition topic for metadata updates
> -
>
> Key: IGNITE-18992
> URL: https://issues.apache.org/jira/browse/IGNITE-18992
> Project: Ignite
> Issue Type: Task
> Components: extensions
>Reporter: Ilya Shishkov
>Priority: Major
> Labels: IEP-59, ise
>
> In order to read data with guaranteed order, metadata topic must have only
> one partition. To achieve this we should:
> * Write binary types and marshaller mappings only to a single partition in
> {{IgniteToKafkaCdcStreamer#onMappings}} and {{#onTypes}}.
> * Print warnings into logs, when topic has more than one partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-18992) CDC: Use single partition topic for metadata updates
[
https://issues.apache.org/jira/browse/IGNITE-18992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-18992:
---
Release Note: (was: Fixed in IGNITE-19369)
> CDC: Use single partition topic for metadata updates
> -
>
> Key: IGNITE-18992
> URL: https://issues.apache.org/jira/browse/IGNITE-18992
> Project: Ignite
> Issue Type: Task
> Components: extensions
>Reporter: Ilya Shishkov
>Priority: Major
> Labels: IEP-59, ise
>
> In order to read data with guaranteed order, metadata topic must have only
> one partition. To achieve this we should:
> * Write binary types and marshaller mappings only to a single partition in
> {{IgniteToKafkaCdcStreamer#onMappings}} and {{#onTypes}}.
> * Print warnings into logs, when topic has more than one partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-18992) CDC: Use single partition topic for metadata updates
[
https://issues.apache.org/jira/browse/IGNITE-18992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724198#comment-17724198
]
Ilya Shishkov commented on IGNITE-18992:
Fixed in IGNITE-19369
> CDC: Use single partition topic for metadata updates
> -
>
> Key: IGNITE-18992
> URL: https://issues.apache.org/jira/browse/IGNITE-18992
> Project: Ignite
> Issue Type: Task
> Components: extensions
>Reporter: Ilya Shishkov
>Priority: Major
> Labels: IEP-59, ise
>
> In order to read data with guaranteed order, metadata topic must have only
> one partition. To achieve this we should:
> * Write binary types and marshaller mappings only to a single partition in
> {{IgniteToKafkaCdcStreamer#onMappings}} and {{#onTypes}}.
> * Print warnings into logs, when topic has more than one partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19527) .NET: Thin 3.0: Add nunit.analyzers
Pavel Tupitsyn created IGNITE-19527: --- Summary: .NET: Thin 3.0: Add nunit.analyzers Key: IGNITE-19527 URL: https://issues.apache.org/jira/browse/IGNITE-19527 Project: Ignite Issue Type: Improvement Components: platforms, thin client Affects Versions: 3.0.0-beta1 Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 3.0.0-beta2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19527) .NET: Thin 3.0: Add nunit.analyzers
[ https://issues.apache.org/jira/browse/IGNITE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-19527: Description: Add https://github.com/nunit/nunit.analyzers to the project for improved static analysis of the test code. > .NET: Thin 3.0: Add nunit.analyzers > --- > > Key: IGNITE-19527 > URL: https://issues.apache.org/jira/browse/IGNITE-19527 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET, ignite-3 > Fix For: 3.0.0-beta2 > > > Add https://github.com/nunit/nunit.analyzers to the project for improved > static analysis of the test code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19527) .NET: Thin 3.0: Add nunit.analyzers
[ https://issues.apache.org/jira/browse/IGNITE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-19527: Ignite Flags: (was: Docs Required,Release Notes Required) > .NET: Thin 3.0: Add nunit.analyzers > --- > > Key: IGNITE-19527 > URL: https://issues.apache.org/jira/browse/IGNITE-19527 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET, ignite-3 > Fix For: 3.0.0-beta2 > > > Add https://github.com/nunit/nunit.analyzers to the project for improved > static analysis of the test code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19063) Fix incorrect behaviour of MemorizingAppender with MutableLogEvent
[
https://issues.apache.org/jira/browse/IGNITE-19063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Pavlov updated IGNITE-19063:
---
Labels: (was: ise)
> Fix incorrect behaviour of MemorizingAppender with MutableLogEvent
> --
>
> Key: IGNITE-19063
> URL: https://issues.apache.org/jira/browse/IGNITE-19063
> Project: Ignite
> Issue Type: Test
>Reporter: Ilya Shishkov
>Priority: Minor
> Attachments: MemorizingAppenderTest.patch
>
>
> Since 2.6 version by default Log4j2 utilizes {{MutableLogEvent}} [1], which
> is a reusable implementation of {{LogEvent}}.
> Test Log4j2 appender {{MemorizingAppener}} in Ignite captures such events "as
> is", which can lead to mutation of captured events.
> Currently, it is not happened, because Log4j2 implicitly considers, that it
> is initialized in web application environment, as we can see in static
> initialization of Log4j2 {{ReusableLogEventFactory}} and configuration
> constants and here: [2-5].
> As we can see in [5], availability of {{javax.servlet.Servlet}} or
> {{jakarta.servlet.Servlet}} (the first one is available in Ignite) classes is
> a sufficient condition _to eliminate utilization of_
> {{ReusableLogEventFactory}}.
> If we turn off "web app mode" by setting up system property
> "log4j2.is.webapp" to "false", than events in {{MemorizingAppener}} will be
> mutated after some invocations of logger methods.
> Here is a reproducer patch: [^MemorizingAppenderTest.patch]
> Problem can be fixed by if we will store immutable version of events
> ({{LogEvent#toImmutable}}) in {{MemorizingAppender#append}}.
> Links:
> # https://issues.apache.org/jira/browse/LOG4J2-1334
> #
> [https://github.com/apache/logging-log4j2/blob/e8fba8caf1ffab6dc0161f8f48611dadedb86b2c/log4j-core/src/main/java/org/apache/logging/log4j/core/config/LoggerConfig.java#L101]
> #
> [https://github.com/apache/logging-log4j2/blob/b69b7b802539d87aab6b51aca0a0df8a669ce6ee/log4j-core/src/main/java/org/apache/logging/log4j/core/util/Constants.java#L90]
> #
> [https://github.com/apache/logging-log4j2/blob/c6fd104ebc14d53ea2adce9f9e6fde9a167529b4/log4j-api/src/main/java/org/apache/logging/log4j/util/Constants.java#L41]
> #
> [https://github.com/apache/logging-log4j2/blob/c6fd104ebc14d53ea2adce9f9e6fde9a167529b4/log4j-api/src/main/java/org/apache/logging/log4j/util/Constants.java#L30]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17509) [Extensions] Spring Data pageable request result contains incorrect total value.
[
https://issues.apache.org/jira/browse/IGNITE-17509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Pavlov updated IGNITE-17509:
---
Labels: (was: ise)
> [Extensions] Spring Data pageable request result contains incorrect total
> value.
>
>
> Key: IGNITE-17509
> URL: https://issues.apache.org/jira/browse/IGNITE-17509
> Project: Ignite
> Issue Type: Bug
>Reporter: Mikhail Petrov
>Priority: Major
> Attachments: Reproduces_incorrect_pageable_request_total_value_.patch
>
>
> Assume that Spring Data repository contains the following method
> {code:java}
> public Page findByFirstNameContaining(String val, Pageable
> pageable);
> {code}
> In this case the following checks will fail
> {code:java}
> Page res = repo.findByFirstNameContaining("person",
> PageRequest.of(2, 100));
> assertEquals(CACHE_SIZE, res.getTotalElements());
> {code}
> where 'repo' is the instance of the previously mention repository.
> The full reproduccer is attached.
> The main reason of the such behaviour is that IgniteRepositoryQuery.java:614
> does not make a separate request of the total rows count and just sets Page
> 'total' value to 0.
> See also org.springframework.data.domain.PageImpl#PageImpl(java.util.List,
> org.springframework.data.domain.Pageable, long) logic to understand the how
> the final result of 'getTotalElements()' is calculated. In bref the result
> will contain the summ of the offset and cout of values in the last page.
> It seems that as a workaround, you can explicitly request the total number of
> rows with a separate query.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19082) Describe Catalog operation flow in README and cleanup dead code.
[ https://issues.apache.org/jira/browse/IGNITE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19082: -- Description: Let's * ensure Catalog is used by default as a single schema management point by TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * Describe CatalogService operations in README.md * drop schema related code from configuration. * drop outdated code from TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * make a PR for merging feature branch to main (if applicable). * ensure there are end-to-end tests for the cases (if applicable) described in CatalogServiceSelfTest. Also drop InternalSchemaTest. was: Let's * ensure Catalog is used by default as a single schema management point by TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * Describe CatalogService operations in README.md * drop schema related code from configuration. * drop outdated code from TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * make a PR for merging feature branch to main (if applicable). > Describe Catalog operation flow in README and cleanup dead code. > > > Key: IGNITE-19082 > URL: https://issues.apache.org/jira/browse/IGNITE-19082 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Let's > * ensure Catalog is used by default as a single schema management point by > TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. > * Describe CatalogService operations in README.md > * drop schema related code from configuration. > * drop outdated code from TableManager, IndexManager, SqlSchemaManager, > SchemaRegistry. > * make a PR for merging feature branch to main (if applicable). > * ensure there are end-to-end tests for the cases (if applicable) described > in CatalogServiceSelfTest. Also drop InternalSchemaTest. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19488) RemoteFragmentExecutionException when inserting more than 30 000 rows into one table
[
https://issues.apache.org/jira/browse/IGNITE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724226#comment-17724226
]
Pavel Pereslegin commented on IGNITE-19488:
---
[~lunigorn],
I checked the latest server logs you provided (logs_with_ignored_erorr.zip),
there is only the "Queue full" exception (and no NPE exception). That's another
issue, and it's already fixed in IGNITE-19278, I think you should take a new
version, so that this error does not bother you when testing.
> RemoteFragmentExecutionException when inserting more than 30 000 rows into
> one table
>
>
> Key: IGNITE-19488
> URL: https://issues.apache.org/jira/browse/IGNITE-19488
> Project: Ignite
> Issue Type: Bug
> Components: jdbc, sql
>Reporter: Igor
>Priority: Critical
> Labels: ignite-3
> Attachments: logs.zip, logs_with_ignored_erorr.zip
>
>
> h1. Steps to reproduce
> Ignite 3 main branch commit 45380a6c802203dab0d72bd1eb9fb202b2a345b0
> # Create table with 5 columns
> # Insert into table rows by batches 1000 rows each batch.
> # Repeat previous step untill exception is thrown.
> h1. Expected behaviour
> Created more than 30 000 rows.
> h1. Actual behaviour
> An exception after 29 000 rows are inserted:
> {code:java}
> Exception while executing query [query=SELECT COUNT(*) FROM
> rows_capacity_table]. Error message:IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 Query remote fragment execution
> failed: nodeName=TablesAmountCapacityTest_cluster_0,
> queryId=ecd14026-5366-4ee2-b73a-f38757d3ba4f, fragmentId=1561,
> originalMessage=IGN-CMN-1 TraceId:24c93463-f078-410a-8831-36b5c549a907
> java.sql.SQLException: Exception while executing query [query=SELECT COUNT(*)
> FROM rows_capacity_table]. Error message:IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 Query remote fragment execution
> failed: nodeName=TablesAmountCapacityTest_cluster_0,
> queryId=ecd14026-5366-4ee2-b73a-f38757d3ba4f, fragmentId=1561,
> originalMessage=IGN-CMN-1 TraceId:24c93463-f078-410a-8831-36b5c549a907
> at
> org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
> at
> org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:149)
> at
> org.apache.ignite.internal.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:108)
> {code}
> Logs are in the attachment.
> [^logs.zip]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19528) Add examples for CDC
Ilya Shishkov created IGNITE-19528:
--
Summary: Add examples for CDC
Key: IGNITE-19528
URL: https://issues.apache.org/jira/browse/IGNITE-19528
Project: Ignite
Issue Type: Improvement
Components: extensions
Reporter: Ilya Shishkov
Assignee: Ilya Shishkov
Currently, when CDC through Kafka is used there are possible delays in
replication between clusters when {{KafkaToIgniteCdcStreamerApplier}} tries to
update binary metadata and marshaller mappings.
Delays caused by calls of {{KafkaConsumer#poll}} in
{{KafkaToIgniteMetadataUpdater#updateMetadata}} , when meta topic is empty:
# When first {{KafkaToIgniteCdcStreamerApplier}} meets {{META_UPDATE_MARKER}}
it calls {{KafkaToIgniteMetadataUpdater#updateMetadata}} which in turn calls
{{KafkaConsumer#poll}}, which returns immediately [1] when data is present in
metadata topic. If there are few binary types and mappings to update, first
{{KafkaToIgniteCdcStreamerApplier}} will consume all entries from metadata
topic.
# {{KafkaToIgniteCdcStreamerApplier}} consequently call
{{KafkaToIgniteMetadataUpdater#updateMetadata}} for each partition with meta
update marker. All further consequent calls will wait for {{kafkaReqTimeout}}.
# Also there is a bottleneck, when multiple applier threads tries to update
metadata and call synchronized method
{{KafkaToIgniteMetadataUpdater#updateMetadata}}, because
{{KafkaToIgniteMetadataUpdater}} is shared between applier threads.
# Because {{META_UPDATE_MARKER}} is sent twice to each Kafka partition of event
topic from every node: firstly, in case of type mappings updates, secondly, in
case of binary types update there are possible delays up to {{clusterSize x
(topicPartitions x 2 - 1) x kafkaReqTimeout}}.
# Data updates are blocked for Kafka partitions with unprocessed update markers.
# For example for default timeout and 16 Kafka partitions _last partition will
be consumed after 1.5 minutes_ in case of two one-node clusters.
Links:
#
[https://kafka.apache.org/27/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#poll-java.time.Duration-]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19477) Custom operation timeout for sendWithRetry
[ https://issues.apache.org/jira/browse/IGNITE-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Chudov updated IGNITE-19477: -- Description: *Motivation* Default operation timeout for sendWithRetry is 3000 ms and may be acceptable for such things as transactional requests. But for some other things, for example, DDL operations, it can be too small. When user creates a table, the least likely thing they expect is timeout exception after 3 sec. We should add argument with custom timeout for this operation. *Definition of done* sendWithRetry is able to be executed with custom timeout. *Implementation notes* In fact, we need raft commands to be executed with custom timeout, so an overloaded method should be added to RaftGroupService, and there is already sendWithRetry with custom timeout. Also, the timeouts for meta storage operations can be custimized at once. was: *Motivation* Default operation timeout for sendWithRetry is 3000 ms and may be acceptable for such things as transactional requests. But for some other things, for example, DDL operations, it can be too small. When user creates a table, the least likely thing they expect is timeout exception after 3 sec. We should add argument with custom timeout for this operation. *Definition of done* sendWithRetry is able to be executed with custom timeout. > Custom operation timeout for sendWithRetry > -- > > Key: IGNITE-19477 > URL: https://issues.apache.org/jira/browse/IGNITE-19477 > Project: Ignite > Issue Type: Bug >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > *Motivation* > Default operation timeout for sendWithRetry is 3000 ms and may be acceptable > for such things as transactional requests. But for some other things, for > example, DDL operations, it can be too small. When user creates a table, the > least likely thing they expect is timeout exception after 3 sec. We should > add argument with custom timeout for this operation. > *Definition of done* > sendWithRetry is able to be executed with custom timeout. > *Implementation notes* > In fact, we need raft commands to be executed with custom timeout, so an > overloaded method should be added to RaftGroupService, and there is already > sendWithRetry with custom timeout. > Also, the timeouts for meta storage operations can be custimized at once. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19528) Add examples for CDC
[
https://issues.apache.org/jira/browse/IGNITE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-19528:
---
Description: Currently, there are no examples for different CDC
configurations and corresponding documentation. (was: Currently, when CDC
through Kafka is used there are possible delays in replication between clusters
when {{KafkaToIgniteCdcStreamerApplier}} tries to update binary metadata and
marshaller mappings.
Delays caused by calls of {{KafkaConsumer#poll}} in
{{KafkaToIgniteMetadataUpdater#updateMetadata}} , when meta topic is empty:
# When first {{KafkaToIgniteCdcStreamerApplier}} meets {{META_UPDATE_MARKER}}
it calls {{KafkaToIgniteMetadataUpdater#updateMetadata}} which in turn calls
{{KafkaConsumer#poll}}, which returns immediately [1] when data is present in
metadata topic. If there are few binary types and mappings to update, first
{{KafkaToIgniteCdcStreamerApplier}} will consume all entries from metadata
topic.
# {{KafkaToIgniteCdcStreamerApplier}} consequently call
{{KafkaToIgniteMetadataUpdater#updateMetadata}} for each partition with meta
update marker. All further consequent calls will wait for {{kafkaReqTimeout}}.
# Also there is a bottleneck, when multiple applier threads tries to update
metadata and call synchronized method
{{KafkaToIgniteMetadataUpdater#updateMetadata}}, because
{{KafkaToIgniteMetadataUpdater}} is shared between applier threads.
# Because {{META_UPDATE_MARKER}} is sent twice to each Kafka partition of event
topic from every node: firstly, in case of type mappings updates, secondly, in
case of binary types update there are possible delays up to {{clusterSize x
(topicPartitions x 2 - 1) x kafkaReqTimeout}}.
# Data updates are blocked for Kafka partitions with unprocessed update markers.
# For example for default timeout and 16 Kafka partitions _last partition will
be consumed after 1.5 minutes_ in case of two one-node clusters.
Links:
#
[https://kafka.apache.org/27/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#poll-java.time.Duration-])
> Add examples for CDC
>
>
> Key: IGNITE-19528
> URL: https://issues.apache.org/jira/browse/IGNITE-19528
> Project: Ignite
> Issue Type: Improvement
> Components: extensions
>Reporter: Ilya Shishkov
>Assignee: Ilya Shishkov
>Priority: Minor
> Labels: IEP-59, ise
>
> Currently, there are no examples for different CDC configurations and
> corresponding documentation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19419) Corrupted LogStorage under intensive SQL load
[
https://issues.apache.org/jira/browse/IGNITE-19419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denis Chudov updated IGNITE-19419:
--
Description:
I tried to run a SQL benchmark (slightly modified [Benchbase TPC-C
scenario|https://github.com/cmu-db/benchbase/wiki/TPC-C]) against Ignite3
(commit hash b48ddcba7cd2bd3b9a053ae131c25b44a0400e27).
Ignite3 was running in a single-node mode with -Xmx8G -Xms8G memory limit
(increased memory size comparing to IGNITE-19412). After some time it has
become unavailable to the client. Log is filled with error messages like the
following:
{code:java}
2023-05-03 17:51:57:705 +0300
[WARNING][%node0%JRaft-FSMCaller-Disruptor-_stripe_10-0][NodeImpl] Node
got error: Error
[type=ERROR_TYPE_LOG, status=Status[EBUSY<1009>: LogManager is busy, disk queue
overload.]].
2023-05-03 17:51:57:705 +0300
[WARNING][CompletableFutureDelayScheduler][RaftGroupServiceImpl] Recoverable
error during the request type=ActionRequestImpl occurred (will be retried on
the randomly selected node):
java.util.concurrent.CompletionException: java.util.concurrent.TimeoutException
at
java.base/java.util.concurrent.CompletableFuture.encodeRelay(CompletableFuture.java:367)
at
java.base/java.util.concurrent.CompletableFuture.completeRelay(CompletableFuture.java:376)
at
java.base/java.util.concurrent.CompletableFuture$UniRelay.tryFire(CompletableFuture.java:1019)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at
java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
at
java.base/java.util.concurrent.CompletableFuture$Timeout.run(CompletableFuture.java:2792)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.util.concurrent.TimeoutException
... 7 more
2023-05-03 17:51:57:705 +0300
[ERROR][%node0%JRaft-LogManager-Disruptor-_stripe_10-0][NodeImpl] Node
append [2268, 2268] failed,
status=Status[EIO<1014>: Corrupted LogStorage].
2023-05-03 17:51:57:705 +0300
[WARNING][%node0%JRaft-FSMCaller-Disruptor-_stripe_10-0][FSMCallerImpl]
FSMCaller already in error status, ignore new error
Error [type=ERROR_TYPE_LOG, status=Status[EBUSY<1009>: LogManager is busy, disk
queue overload.]]
at
org.apache.ignite.raft.jraft.storage.impl.LogManagerImpl.reportError(LogManagerImpl.java:587)
at
org.apache.ignite.raft.jraft.storage.impl.LogManagerImpl.appendEntries(LogManagerImpl.java:337)
at
org.apache.ignite.raft.jraft.core.NodeImpl.executeApplyingTasks(NodeImpl.java:1621)
at
org.apache.ignite.raft.jraft.core.NodeImpl$LogEntryAndClosureHandler.onEvent(NodeImpl.java:305)
at
org.apache.ignite.raft.jraft.core.NodeImpl$LogEntryAndClosureHandler.onEvent(NodeImpl.java:288)
at
org.apache.ignite.raft.jraft.disruptor.StripedDisruptor$StripeEntryHandler.onEvent(StripedDisruptor.java:217)
at
org.apache.ignite.raft.jraft.disruptor.StripedDisruptor$StripeEntryHandler.onEvent(StripedDisruptor.java:181)
at
com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:137)
at java.base/java.lang.Thread.run(Thread.java:829)
2023-05-03 17:51:57:706 +0300
[INFO][%node0%JRaft-FSMCaller-Disruptor-_stripe_10-0][ReplicatorGroupImpl] Fail
to find the next candidate.
2023-05-03 17:51:57:706 +0300
[ERROR][%node0%JRaft-LogManager-Disruptor-_stripe_10-0][NodeImpl] Node
append [2271, 2271]
failed, status=Status[EIO<1014>: Corrupted LogStorage].
2023-05-03 17:51:57:706 +0300
[INFO][%node0%JRaft-FSMCaller-Disruptor-_stripe_10-0][StateMachineAdapter]
onLeaderStop: status=Status[EBADNODE<10009>: Raft node(leader or candidate) is
in error.].
2023-05-03 17:51:57:706 +0300
[ERROR][%node0%JRaft-LogManager-Disruptor-_stripe_10-0][NodeImpl] Node
<30181391-2f04-40c0-8c35-a77592b0731f_part_5/node0> append [2285, 2285] failed,
status=Status[EIO<1014>: Corrupted LogStorage].
{code}
More errors could be observed in the attached log archive.
Actually the log storage is not corrupted but the queue is overloaded. There
are some performance improvements in main JRaft repository (
[https://github.com/sofastack/sofa-jraft] ) that may fix that, more exactly it
is commit 5de2fbbcabb70ddbefb06a1d3737821781c3e85c in sofa-jraft repo.
was:
I tried to run a SQL benchmark (slightly modified [Benchbase TPC-C
scenario|http
[jira] [Updated] (IGNITE-19528) Add examples for CDC
[ https://issues.apache.org/jira/browse/IGNITE-19528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Shishkov updated IGNITE-19528: --- Description: Currently, there are no examples for different CDC configurations and corresponding documentation. We need step-by-step documentation for active-passive, active-active cases and for different clients and transports (Ignite2Ignite, Ignite through Kafka). (was: Currently, there are no examples for different CDC configurations and corresponding documentation.) > Add examples for CDC > > > Key: IGNITE-19528 > URL: https://issues.apache.org/jira/browse/IGNITE-19528 > Project: Ignite > Issue Type: Improvement > Components: extensions >Reporter: Ilya Shishkov >Assignee: Ilya Shishkov >Priority: Minor > Labels: IEP-59, ise > > Currently, there are no examples for different CDC configurations and > corresponding documentation. We need step-by-step documentation for > active-passive, active-active cases and for different clients and transports > (Ignite2Ignite, Ignite through Kafka). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18537) Implement create/alter/drop zones operations in Catalog
[ https://issues.apache.org/jira/browse/IGNITE-18537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-18537: -- Summary: Implement create/alter/drop zones operations in Catalog (was: Implement missed create/alter/drop Catalog operations) > Implement create/alter/drop zones operations in Catalog > --- > > Key: IGNITE-18537 > URL: https://issues.apache.org/jira/browse/IGNITE-18537 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > Every CRETE/ALTER/DROP operation should trigger a new Catalog version. > We already have CREATE TABLE DDL operation implemented. Let's implement > others in similar way. > Please keep in mind that Distribution zones is out of scope now. -the > affinity is not the part of the schema definition. Data storage parameters > are not the part of it as well.- -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18537) Implement create/alter/drop zones operations in Catalog
[ https://issues.apache.org/jira/browse/IGNITE-18537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-18537: -- Fix Version/s: 3.0.0-beta2 > Implement create/alter/drop zones operations in Catalog > --- > > Key: IGNITE-18537 > URL: https://issues.apache.org/jira/browse/IGNITE-18537 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Every CRETE/ALTER/DROP ZONE operation should trigger a new Catalog version. > We already have CREATE TABLE DDL operation implemented. Let's implement > others in similar way. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-18537) Implement create/alter/drop zones operations in Catalog
[ https://issues.apache.org/jira/browse/IGNITE-18537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov reassigned IGNITE-18537: - Assignee: Andrey Mashenkov > Implement create/alter/drop zones operations in Catalog > --- > > Key: IGNITE-18537 > URL: https://issues.apache.org/jira/browse/IGNITE-18537 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Assignee: Andrey Mashenkov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Every CRETE/ALTER/DROP ZONE operation should trigger a new Catalog version. > We already have CREATE TABLE DDL operation implemented. Let's implement > others in similar way. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18537) Implement create/alter/drop zones operations in Catalog
[ https://issues.apache.org/jira/browse/IGNITE-18537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-18537: -- Description: Every CRETE/ALTER/DROP ZONE operation should trigger a new Catalog version. We already have CREATE TABLE DDL operation implemented. Let's implement others in similar way. was: Every CRETE/ALTER/DROP operation should trigger a new Catalog version. We already have CREATE TABLE DDL operation implemented. Let's implement others in similar way. Please keep in mind that Distribution zones is out of scope now. -the affinity is not the part of the schema definition. Data storage parameters are not the part of it as well.- > Implement create/alter/drop zones operations in Catalog > --- > > Key: IGNITE-18537 > URL: https://issues.apache.org/jira/browse/IGNITE-18537 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > Every CRETE/ALTER/DROP ZONE operation should trigger a new Catalog version. > We already have CREATE TABLE DDL operation implemented. Let's implement > others in similar way. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18537) Implement create/alter/drop zones operations in Catalog
[ https://issues.apache.org/jira/browse/IGNITE-18537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-18537: -- Priority: Minor (was: Major) > Implement create/alter/drop zones operations in Catalog > --- > > Key: IGNITE-18537 > URL: https://issues.apache.org/jira/browse/IGNITE-18537 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Priority: Minor > Labels: ignite-3 > > Every CRETE/ALTER/DROP ZONE operation should trigger a new Catalog version. > We already have CREATE TABLE DDL operation implemented. Let's implement > others in similar way. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19529) IgniteTxStateImpl non-thread safe fields may cause incorrect node behavior
Anton Vinogradov created IGNITE-19529: - Summary: IgniteTxStateImpl non-thread safe fields may cause incorrect node behavior Key: IGNITE-19529 URL: https://issues.apache.org/jira/browse/IGNITE-19529 Project: Ignite Issue Type: Improvement Reporter: Anton Vinogradov Assignee: Anton Vinogradov IgniteTxStateImpl's fields - activeCacheIds - txMap are not thread safe, but are widely used from different threads without proper sync. This may (and caused: IGNITE-19445) node crashes and other issues. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19529) IgniteTxStateImpl's non-thread safe fields may cause incorrect node behavior
[ https://issues.apache.org/jira/browse/IGNITE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-19529: -- Summary: IgniteTxStateImpl's non-thread safe fields may cause incorrect node behavior (was: IgniteTxStateImpl non-thread safe fields may cause incorrect node behavior ) > IgniteTxStateImpl's non-thread safe fields may cause incorrect node behavior > - > > Key: IGNITE-19529 > URL: https://issues.apache.org/jira/browse/IGNITE-19529 > Project: Ignite > Issue Type: Improvement >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > > IgniteTxStateImpl's fields > - activeCacheIds > - txMap > are not thread safe, but are widely used from different threads without > proper sync. > This may (and caused: IGNITE-19445) node crashes and other issues. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19529) IgniteTxStateImpl's non-thread safe fields may cause incorrect node behavior
[ https://issues.apache.org/jira/browse/IGNITE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-19529: -- Description: IgniteTxStateImpl's fields - activeCacheIds - txMap are not thread safe, but are widely used from different threads without proper sync. This may (and caused: IGNITE-19445) node crashes and other issues. was: IgniteTxStateImpl's fields - activeCacheIds - txMap are not thread safe, but are widely used from different threads without proper sync. This may (and caused: IGNITE-19445) node crashes and other issues. > IgniteTxStateImpl's non-thread safe fields may cause incorrect node behavior > - > > Key: IGNITE-19529 > URL: https://issues.apache.org/jira/browse/IGNITE-19529 > Project: Ignite > Issue Type: Improvement >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > > IgniteTxStateImpl's fields > - activeCacheIds > - txMap > are not thread safe, but are widely used from different threads without > proper sync. > This may (and caused: IGNITE-19445) node crashes and other issues. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19529) IgniteTxStateImpl's non-thread safe fields may cause incorrect node behavior
[ https://issues.apache.org/jira/browse/IGNITE-19529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-19529: -- Description: IgniteTxStateImpl's fields - activeCacheIds - txMap are not thread safe, but are widely used from different threads without the proper sync. This may (and caused: IGNITE-19445) node crashes and other issues. was: IgniteTxStateImpl's fields - activeCacheIds - txMap are not thread safe, but are widely used from different threads without proper sync. This may (and caused: IGNITE-19445) node crashes and other issues. > IgniteTxStateImpl's non-thread safe fields may cause incorrect node behavior > - > > Key: IGNITE-19529 > URL: https://issues.apache.org/jira/browse/IGNITE-19529 > Project: Ignite > Issue Type: Improvement >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > > IgniteTxStateImpl's fields > - activeCacheIds > - txMap > are not thread safe, but are widely used from different threads without the > proper sync. > This may (and caused: IGNITE-19445) node crashes and other issues. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19445) NPE during transaction recovery
[
https://issues.apache.org/jira/browse/IGNITE-19445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-19445:
--
Fix Version/s: 2.16
> NPE during transaction recovery
> ---
>
> Key: IGNITE-19445
> URL: https://issues.apache.org/jira/browse/IGNITE-19445
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikita Amelchev
>Assignee: Anton Vinogradov
>Priority: Major
> Labels: ise
> Fix For: 2.16
>
> Attachments: reproducer-19445.patch
>
>
> The transaction can be enlisted during recovery(two different threads). The
> {{GridIntList}} used in the {{IgniteTxStateImpl}} is not thread safe - it
> produces NPE:
> {noformat}
> [2023-05-10T12:23:32,438][ERROR][sys-#129%dht.TxRecoveryStoreEnabledTest1%][IgniteTestResources]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=StopNodeFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=class o.a.i.IgniteCheckedException: null]]
> org.apache.ignite.IgniteCheckedException: null
> at
> org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7929)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:659)
> [classes/:?]
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
> [classes/:?]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_352]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_352]
> at java.lang.Thread.run(Thread.java:750) [?:1.8.0_352]
> Caused by: java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxStateImpl.storeWriteThrough(IgniteTxStateImpl.java:342)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.storeWriteThrough(IgniteTxAdapter.java:517)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager$TxRecoveryInitRunnable.run(IgniteTxManager.java:3341)
> ~[classes/:?]
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7487)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:649)
> ~[classes/:?]
> ... 4 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-19445) NPE during transaction recovery
[
https://issues.apache.org/jira/browse/IGNITE-19445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724238#comment-17724238
]
Anton Vinogradov commented on IGNITE-19445:
---
Ad-hoc fix here.
Proper fix is expected at
!https://issues.apache.org/jira/secure/viewavatar?size=xsmall&avatarId=21140&avatarType=issuetype|width=16,height=16!
IGNITE-19529
> NPE during transaction recovery
> ---
>
> Key: IGNITE-19445
> URL: https://issues.apache.org/jira/browse/IGNITE-19445
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikita Amelchev
>Assignee: Anton Vinogradov
>Priority: Major
> Labels: ise
> Fix For: 2.16
>
> Attachments: reproducer-19445.patch
>
>
> The transaction can be enlisted during recovery(two different threads). The
> {{GridIntList}} used in the {{IgniteTxStateImpl}} is not thread safe - it
> produces NPE:
> {noformat}
> [2023-05-10T12:23:32,438][ERROR][sys-#129%dht.TxRecoveryStoreEnabledTest1%][IgniteTestResources]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=StopNodeFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=class o.a.i.IgniteCheckedException: null]]
> org.apache.ignite.IgniteCheckedException: null
> at
> org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7929)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:659)
> [classes/:?]
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
> [classes/:?]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_352]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_352]
> at java.lang.Thread.run(Thread.java:750) [?:1.8.0_352]
> Caused by: java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxStateImpl.storeWriteThrough(IgniteTxStateImpl.java:342)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.storeWriteThrough(IgniteTxAdapter.java:517)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager$TxRecoveryInitRunnable.run(IgniteTxManager.java:3341)
> ~[classes/:?]
> at
> org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7487)
> ~[classes/:?]
> at
> org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:649)
> ~[classes/:?]
> ... 4 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-19488) RemoteFragmentExecutionException when inserting more than 30 000 rows into one table
[
https://issues.apache.org/jira/browse/IGNITE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky reassigned IGNITE-19488:
---
Assignee: Evgeny Stanilovsky
> RemoteFragmentExecutionException when inserting more than 30 000 rows into
> one table
>
>
> Key: IGNITE-19488
> URL: https://issues.apache.org/jira/browse/IGNITE-19488
> Project: Ignite
> Issue Type: Bug
> Components: jdbc, sql
>Reporter: Igor
>Assignee: Evgeny Stanilovsky
>Priority: Critical
> Labels: ignite-3
> Attachments: logs.zip, logs_with_ignored_erorr.zip
>
>
> h1. Steps to reproduce
> Ignite 3 main branch commit 45380a6c802203dab0d72bd1eb9fb202b2a345b0
> # Create table with 5 columns
> # Insert into table rows by batches 1000 rows each batch.
> # Repeat previous step untill exception is thrown.
> h1. Expected behaviour
> Created more than 30 000 rows.
> h1. Actual behaviour
> An exception after 29 000 rows are inserted:
> {code:java}
> Exception while executing query [query=SELECT COUNT(*) FROM
> rows_capacity_table]. Error message:IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 Query remote fragment execution
> failed: nodeName=TablesAmountCapacityTest_cluster_0,
> queryId=ecd14026-5366-4ee2-b73a-f38757d3ba4f, fragmentId=1561,
> originalMessage=IGN-CMN-1 TraceId:24c93463-f078-410a-8831-36b5c549a907
> java.sql.SQLException: Exception while executing query [query=SELECT COUNT(*)
> FROM rows_capacity_table]. Error message:IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 Query remote fragment execution
> failed: nodeName=TablesAmountCapacityTest_cluster_0,
> queryId=ecd14026-5366-4ee2-b73a-f38757d3ba4f, fragmentId=1561,
> originalMessage=IGN-CMN-1 TraceId:24c93463-f078-410a-8831-36b5c549a907
> at
> org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
> at
> org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:149)
> at
> org.apache.ignite.internal.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:108)
> {code}
> Logs are in the attachment.
> [^logs.zip]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-19454) Validate backwards schema compatibility
[ https://issues.apache.org/jira/browse/IGNITE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724247#comment-17724247 ] Roman Puchkovskiy commented on IGNITE-19454: Thanks! > Validate backwards schema compatibility > --- > > Key: IGNITE-19454 > URL: https://issues.apache.org/jira/browse/IGNITE-19454 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > See description in IGNITE-19228 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19527) .NET: Thin 3.0: Add nunit.analyzers
[ https://issues.apache.org/jira/browse/IGNITE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724250#comment-17724250 ] Igor Sapego commented on IGNITE-19527: -- Looks good to me > .NET: Thin 3.0: Add nunit.analyzers > --- > > Key: IGNITE-19527 > URL: https://issues.apache.org/jira/browse/IGNITE-19527 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET, ignite-3 > Fix For: 3.0.0-beta2 > > > Add https://github.com/nunit/nunit.analyzers to the project for improved > static analysis of the test code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19459) Kafka Connect IgniteSinkConnector drops messages in case of error
[
https://issues.apache.org/jira/browse/IGNITE-19459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vedran Ljubovic updated IGNITE-19459:
-
Attachment: event_dropping.patch
> Kafka Connect IgniteSinkConnector drops messages in case of error
> -
>
> Key: IGNITE-19459
> URL: https://issues.apache.org/jira/browse/IGNITE-19459
> Project: Ignite
> Issue Type: Bug
> Components: extensions
>Affects Versions: 2.15
>Reporter: Vedran Ljubovic
>Priority: Major
> Attachments: event_dropping.patch
>
>
> We are using Kafka Connect (KC) to stream messages from Kafka to Ignite.
> Since the Kafka topic is using null key, we have created a custom
> SingleTupleExtractor to generate keys from payload. This works very well when
> everything is ok. However, if there are any kind of issues with starting a
> cache on Ignite (such as if cluster state is inactive or if cache has
> lostParts), we expect KC to fail to start. Instead, KC will start and appear
> to be running, and the messages will be dropped - which means that once the
> problems are removed, KC will not attempt to resend the messages even after
> restart! This for us is unacceptable, we believe that the system should be
> reliable and fault-tolerant.
> In logs we notice errors such as:
> {code:java}
> Failed to stream a record with null key! {code}
> which is useless since we do have a SingleTupleExtractor for this purpose and
> we can see that it isn't being called at all!
> When KC REST API [1] is used, we find the state is RUNNING which means that
> we have no way to detect this error other than parsing the logs which is
> unreliable.
> Upon investigating this issue, we found the following:
> * Ignite connection and IgniteDataStreamer are declared as private static
> final fields of an inner class, they will be initialized when calling the
> start() method of IgniteSinkConnector. From KC docs [2], we conclude that
> method initialize() should be overloaded and the connections created there,
> also that appropriate exception types should be thrown so that KC knows that
> connection has failed and terminate the task/connector.
> * When start() method is called, StreamerContext.getStreamer() in line 72
> will fail with exception. This exception is not handled by KC so it doesn't
> know that task failed to start. In addition, code will never reach line 91
> where SingleTupleExtractor is created therefore there will be no extractor.
> Solution would be to catch all types of exceptions and throw those exceptions
> which will be detected by KC as critical errors. Alternatively, put() method
> should throw an exception is stopped is true.
> * When put() method is called, if there is no key and no extractor, in line
> 121 we see that the error is logged but exception is not thrown so KC thinks
> that the message was successfully streamed. Here, ConnectException should be
> thrown. If users want the current behavior (which is to stream Kafka messages
> with key and skip those without key), they can set option errors.tolerance =
> all in connector config. [3]
> [1]
> [https://docs.confluent.io/platform/current/connect/references/restapi.html]
> [2]
> [https://kafka.apache.org/34/javadoc/org/apache/kafka/connect/sink/SinkTask.html]
> [3]
> [https://www.confluent.io/blog/kafka-connect-deep-dive-error-handling-dead-letter-queues/]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-19459) Kafka Connect IgniteSinkConnector drops messages in case of error
[
https://issues.apache.org/jira/browse/IGNITE-19459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724257#comment-17724257
]
Vedran Ljubovic commented on IGNITE-19459:
--
After some experiments, I have found that simply moving extractor
initialization before others fixes the problem. Apparently
StreamerContext.getStreamer() can sometimes take a few seconds, kc starts
streaming messages before it is done and these messages will be dropped because
the key is null. Patch is included.
> Kafka Connect IgniteSinkConnector drops messages in case of error
> -
>
> Key: IGNITE-19459
> URL: https://issues.apache.org/jira/browse/IGNITE-19459
> Project: Ignite
> Issue Type: Bug
> Components: extensions
>Affects Versions: 2.15
>Reporter: Vedran Ljubovic
>Priority: Major
> Attachments: event_dropping.patch
>
>
> We are using Kafka Connect (KC) to stream messages from Kafka to Ignite.
> Since the Kafka topic is using null key, we have created a custom
> SingleTupleExtractor to generate keys from payload. This works very well when
> everything is ok. However, if there are any kind of issues with starting a
> cache on Ignite (such as if cluster state is inactive or if cache has
> lostParts), we expect KC to fail to start. Instead, KC will start and appear
> to be running, and the messages will be dropped - which means that once the
> problems are removed, KC will not attempt to resend the messages even after
> restart! This for us is unacceptable, we believe that the system should be
> reliable and fault-tolerant.
> In logs we notice errors such as:
> {code:java}
> Failed to stream a record with null key! {code}
> which is useless since we do have a SingleTupleExtractor for this purpose and
> we can see that it isn't being called at all!
> When KC REST API [1] is used, we find the state is RUNNING which means that
> we have no way to detect this error other than parsing the logs which is
> unreliable.
> Upon investigating this issue, we found the following:
> * Ignite connection and IgniteDataStreamer are declared as private static
> final fields of an inner class, they will be initialized when calling the
> start() method of IgniteSinkConnector. From KC docs [2], we conclude that
> method initialize() should be overloaded and the connections created there,
> also that appropriate exception types should be thrown so that KC knows that
> connection has failed and terminate the task/connector.
> * When start() method is called, StreamerContext.getStreamer() in line 72
> will fail with exception. This exception is not handled by KC so it doesn't
> know that task failed to start. In addition, code will never reach line 91
> where SingleTupleExtractor is created therefore there will be no extractor.
> Solution would be to catch all types of exceptions and throw those exceptions
> which will be detected by KC as critical errors. Alternatively, put() method
> should throw an exception is stopped is true.
> * When put() method is called, if there is no key and no extractor, in line
> 121 we see that the error is logged but exception is not thrown so KC thinks
> that the message was successfully streamed. Here, ConnectException should be
> thrown. If users want the current behavior (which is to stream Kafka messages
> with key and skip those without key), they can set option errors.tolerance =
> all in connector config. [3]
> [1]
> [https://docs.confluent.io/platform/current/connect/references/restapi.html]
> [2]
> [https://kafka.apache.org/34/javadoc/org/apache/kafka/connect/sink/SinkTask.html]
> [3]
> [https://www.confluent.io/blog/kafka-connect-deep-dive-error-handling-dead-letter-queues/]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-19527) .NET: Thin 3.0: Add nunit.analyzers
[ https://issues.apache.org/jira/browse/IGNITE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724264#comment-17724264 ] Pavel Tupitsyn commented on IGNITE-19527: - Merged to main: be922061aea9d7b61b23720fab21b7595baf58d2 > .NET: Thin 3.0: Add nunit.analyzers > --- > > Key: IGNITE-19527 > URL: https://issues.apache.org/jira/browse/IGNITE-19527 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET, ignite-3 > Fix For: 3.0.0-beta2 > > > Add https://github.com/nunit/nunit.analyzers to the project for improved > static analysis of the test code. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-19445) NPE during transaction recovery
[
https://issues.apache.org/jira/browse/IGNITE-19445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17722907#comment-17722907
]
Anton Vinogradov edited comment on IGNITE-19445 at 5/19/23 12:58 PM:
-
Checked that 2 threads may work with the same GridIntList simultaneously, and
it's true.
Typical case:
{noformat}
"sys-stripe-0-#68%dht.TxTest1%@5510" prio=5 tid=0x56 nid=NA sleeping
java.lang.Thread.State: TIMED_WAITING
at
org.apache.ignite.internal.util.GridIntList.add(GridIntList.java:189)
==> ADDITION!
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxStateImpl.addActiveCache(IgniteTxStateImpl.java:260)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.addActiveCache(IgniteTxLocalAdapter.java:1365)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.addEntry(GridDhtTxLocalAdapter.java:500)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.prepareAsync(GridDhtTxLocal.java:389)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:652)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:449)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest0(IgniteTxHandler.java:203)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest(IgniteTxHandler.java:177)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$000(IgniteTxHandler.java:137)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:225)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:223)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1164)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:605)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:406)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:324)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:112)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:314)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1907)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1528)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$5300(GridIoManager.java:243)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.execute(GridIoManager.java:1421)
at
org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:55)
at
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:637)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
at java.lang.Thread.run(Thread.java:750)
"sys-#131%dht.TxTest1%@5876" prio=5 tid=0x96 nid=NA runnable
java.lang.Thread.State: RUNNABLE
at
org.apache.ignite.internal.util.GridIntList.get(GridIntList.java:277)
==> GET!
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxStateImpl.storeWriteThrough(IgniteTxStateImpl.java:339)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.storeWriteThrough(IgniteTxAdapter.java:517)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager$TxRecoveryInitRunnable.run(IgniteTxManager.java:3340)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7487)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:649)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
{noformat}
was (Author: av):
Checked that 2 threads may work with the same GridIntList, and it's true.
Typical case:
{noformat}
"sys-stripe-0-#68%dht.TxTest1%@5510" prio=5 tid=0x56 nid=NA sleeping
java.lang.Thread.Sta
[jira] [Commented] (IGNITE-19458) Kafka Connect IgniteSinkConnector multiple topics & caches do not work
[ https://issues.apache.org/jira/browse/IGNITE-19458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724269#comment-17724269 ] Vedran Ljubovic commented on IGNITE-19458: -- Code in attachment works for me. > Kafka Connect IgniteSinkConnector multiple topics & caches do not work > -- > > Key: IGNITE-19458 > URL: https://issues.apache.org/jira/browse/IGNITE-19458 > Project: Ignite > Issue Type: Bug > Components: extensions >Affects Versions: 2.15 >Reporter: Vedran Ljubovic >Priority: Major > Attachments: IgniteSinkTask.java, no_static_final.patch > > > We are using Kafka Connect to stream 4 topics into 4 separate caches. We are > presently forced to use 4 separate instances of Kafka Connect because when 4 > connectors are passed to one instance all messages will be streamed into the > same (first) Ignite cache. Multiple Connect instances work, but in the future > we plan to add more topics/caches and this will affect memory usage as well > as our ability to distribute workload in a cluster. > I think that the reason for this is in IgniteSinkTask.java where > IgniteDataStreamer is declared as private static final attribute of inner > class, which means that it will be initialized with the first passed cache > and cannot be changed later. > I'm not sure which is the best approach performance wise, but possible > solutions would be: > * It should be non-static attribute of IgniteSinkTask (different streamer > for each task) > * It should be created in IgniteSinkConnector and passed to IgniteSinkTask > (one streamer per connector) > * Have a Map of cacheNames to Streamers which ensures that only as many > Streamers will be created as there are different caches used. > Also I notice that cacheName, SingleTupleExtractor, stopped boolean, are all > declared as static which cause various problems when using multiple > connectors (messages being processed with wrong SimpleTupleExtractor etc.) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19458) Kafka Connect IgniteSinkConnector multiple topics & caches do not work
[ https://issues.apache.org/jira/browse/IGNITE-19458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vedran Ljubovic updated IGNITE-19458: - Attachment: no_static_final.patch > Kafka Connect IgniteSinkConnector multiple topics & caches do not work > -- > > Key: IGNITE-19458 > URL: https://issues.apache.org/jira/browse/IGNITE-19458 > Project: Ignite > Issue Type: Bug > Components: extensions >Affects Versions: 2.15 >Reporter: Vedran Ljubovic >Priority: Major > Attachments: IgniteSinkTask.java, no_static_final.patch > > > We are using Kafka Connect to stream 4 topics into 4 separate caches. We are > presently forced to use 4 separate instances of Kafka Connect because when 4 > connectors are passed to one instance all messages will be streamed into the > same (first) Ignite cache. Multiple Connect instances work, but in the future > we plan to add more topics/caches and this will affect memory usage as well > as our ability to distribute workload in a cluster. > I think that the reason for this is in IgniteSinkTask.java where > IgniteDataStreamer is declared as private static final attribute of inner > class, which means that it will be initialized with the first passed cache > and cannot be changed later. > I'm not sure which is the best approach performance wise, but possible > solutions would be: > * It should be non-static attribute of IgniteSinkTask (different streamer > for each task) > * It should be created in IgniteSinkConnector and passed to IgniteSinkTask > (one streamer per connector) > * Have a Map of cacheNames to Streamers which ensures that only as many > Streamers will be created as there are different caches used. > Also I notice that cacheName, SingleTupleExtractor, stopped boolean, are all > declared as static which cause various problems when using multiple > connectors (messages being processed with wrong SimpleTupleExtractor etc.) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19458) Kafka Connect IgniteSinkConnector multiple topics & caches do not work
[ https://issues.apache.org/jira/browse/IGNITE-19458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vedran Ljubovic updated IGNITE-19458: - Attachment: IgniteSinkTask.java > Kafka Connect IgniteSinkConnector multiple topics & caches do not work > -- > > Key: IGNITE-19458 > URL: https://issues.apache.org/jira/browse/IGNITE-19458 > Project: Ignite > Issue Type: Bug > Components: extensions >Affects Versions: 2.15 >Reporter: Vedran Ljubovic >Priority: Major > Attachments: IgniteSinkTask.java, no_static_final.patch > > > We are using Kafka Connect to stream 4 topics into 4 separate caches. We are > presently forced to use 4 separate instances of Kafka Connect because when 4 > connectors are passed to one instance all messages will be streamed into the > same (first) Ignite cache. Multiple Connect instances work, but in the future > we plan to add more topics/caches and this will affect memory usage as well > as our ability to distribute workload in a cluster. > I think that the reason for this is in IgniteSinkTask.java where > IgniteDataStreamer is declared as private static final attribute of inner > class, which means that it will be initialized with the first passed cache > and cannot be changed later. > I'm not sure which is the best approach performance wise, but possible > solutions would be: > * It should be non-static attribute of IgniteSinkTask (different streamer > for each task) > * It should be created in IgniteSinkConnector and passed to IgniteSinkTask > (one streamer per connector) > * Have a Map of cacheNames to Streamers which ensures that only as many > Streamers will be created as there are different caches used. > Also I notice that cacheName, SingleTupleExtractor, stopped boolean, are all > declared as static which cause various problems when using multiple > connectors (messages being processed with wrong SimpleTupleExtractor etc.) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-16160) [ignite-extensions] Failed CdcKafkaReplicationTest.testActivePassiveReplication
[
https://issues.apache.org/jira/browse/IGNITE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov resolved IGNITE-16160.
Resolution: Incomplete
I'm closing issue as incomplete, because log of the build was deleted and
stacktrace does not have root case of failure.
Problem looks similar to IGNITE-18995, where you can see root cause of such
future failures in the description of issue.
> [ignite-extensions] Failed
> CdcKafkaReplicationTest.testActivePassiveReplication
> ---
>
> Key: IGNITE-16160
> URL: https://issues.apache.org/jira/browse/IGNITE-16160
> Project: Ignite
> Issue Type: Test
> Components: extensions
>Reporter: Sergei Ryzhov
>Priority: Major
> Labels: IEP-59, ise
>
> https://ci2.ignite.apache.org/buildConfiguration/IgniteExtensions_Tests_Cdc?branch=%3Cdefault%3E&mode=builds#all-projects
> Log:
> {code:java}
> [2022-04-16
> 06:13:32,532][ERROR][async-runnable-runner-1][IgniteTestResources] Cdc error
> class org.apache.ignite.IgniteInterruptedException: Got interrupted
> while waiting for future to complete.
> at
> org.apache.ignite.internal.util.IgniteUtils$3.apply(IgniteUtils.java:947)
> at
> org.apache.ignite.internal.util.IgniteUtils$3.apply(IgniteUtils.java:945)
> at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1130)
> at org.apache.ignite.Ignition.start(Ignition.java:331)
> at
> org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.start(IgniteToIgniteCdcStreamer.java:114)
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.start(WalRecordsConsumer.java:148)
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:283)
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:230)
> at
> org.apache.ignite.cdc.CdcIgniteToIgniteReplicationTest.lambda$igniteToIgnite$0(CdcIgniteToIgniteReplicationTest.java:83)
> at
> org.apache.ignite.testframework.GridTestUtils$RunnableX.run(GridTestUtils.java:2569)
> at
> org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$3(GridTestUtils.java:1161)
> at
> org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$4(GridTestUtils.java:1217)
> at
> org.apache.ignite.testframework.GridTestUtils$7.call(GridTestUtils.java:1538)
> at
> org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:88)
> [2022-04-16
> 06:13:32,550][ERROR][async-runnable-runner-1][IgniteTestResources] Cdc error
> class org.apache.ignite.IgniteException: sleep interrupted
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:414)
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:286)
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:230)
> at
> org.apache.ignite.cdc.CdcIgniteToIgniteReplicationTest.lambda$igniteToIgnite$0(CdcIgniteToIgniteReplicationTest.java:83)
> at
> org.apache.ignite.testframework.GridTestUtils$RunnableX.run(GridTestUtils.java:2569)
> at
> org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$3(GridTestUtils.java:1161)
> at
> org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$4(GridTestUtils.java:1217)
> at
> org.apache.ignite.testframework.GridTestUtils$7.call(GridTestUtils.java:1538)
> at
> org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:88)
> Caused by: class
> org.apache.ignite.internal.IgniteInterruptedCheckedException: sleep
> interrupted
> at
> org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:8252)
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:410)
> ... 8 more
> Caused by: java.lang.InterruptedException: sleep interrupted
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:8247)
> ... 9 more
> [2022-04-16
> 06:13:32,564][ERROR][async-runnable-runner-1][IgniteTestResources] Cdc error
> class org.apache.ignite.IgniteException: sleep interrupted
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:414)
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:286)
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:230)
> at
> org.apache.ignite.cdc.CdcIgniteToIgniteReplicationTest.lambda$igniteToIgnite$0(CdcIgniteToIgniteReplicationTest.java:83)
> at
> org.apache.ignite.testframework.GridTestUtils$RunnableX.run(GridTestUtils.java:2569)
> at
> org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$3(GridTestUtils.java:1161
[jira] [Created] (IGNITE-19530) Reduce size of configuration keys
Denis Chudov created IGNITE-19530:
-
Summary: Reduce size of configuration keys
Key: IGNITE-19530
URL: https://issues.apache.org/jira/browse/IGNITE-19530
Project: Ignite
Issue Type: Improvement
Reporter: Denis Chudov
The distributed configuration keys are byte arrays formed from strings that
contain some constant prefixes, postfixes, delimiters and identificators,
mostly UUIDs. Example of the configuration key for default value provider type
of table column:
{{dst-cfg.table.tables.d7b99c6a-de10-454d-9370-38d18b65e9c0.columns.d8482dae-cfb8-42b8-a759-9727dd3763a6.defaultValueProvider.type}}
It contains 2 UUIDs in string representation. Unfortunately, there are several
configuration entries for each table column (having similar keys) and besides
that about a dozen of keys for table itself.
As a result, configuration keys take 68% of a meta storage message related to
table creation (for one node cluster, for a table of 2 columns and 25
partitions) which creates excessive load on meta storage raft group in case of
mass table creation (see IGNITE-19275 )
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19530) Reduce size of configuration keys
[
https://issues.apache.org/jira/browse/IGNITE-19530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denis Chudov updated IGNITE-19530:
--
Description:
*Motivation*
The distributed configuration keys are byte arrays formed from strings that
contain some constant prefixes, postfixes, delimiters and identificators,
mostly UUIDs. Example of the configuration key for default value provider type
of table column:
{{dst-cfg.table.tables.d7b99c6a-de10-454d-9370-38d18b65e9c0.columns.d8482dae-cfb8-42b8-a759-9727dd3763a6.defaultValueProvider.type}}
It contains 2 UUIDs in string representation. Unfortunately, there are several
configuration entries for each table column (having similar keys) and besides
that about a dozen of keys for table itself.
As a result, configuration keys take 68% of a meta storage message related to
table creation (for one node cluster, for a table of 2 columns and 25
partitions) which creates excessive load on meta storage raft group in case of
mass table creation (see IGNITE-19275 )
*Definition of done*
We should get rid of string representation of UUIDs in configuration keys,
UUIDs should be written as 16 bytes each into byte array directly. Also, string
constants should be reduced (or even replaced to constants consisting of few
bytes) because there is no need to keep them human readable.
was:
The distributed configuration keys are byte arrays formed from strings that
contain some constant prefixes, postfixes, delimiters and identificators,
mostly UUIDs. Example of the configuration key for default value provider type
of table column:
{{dst-cfg.table.tables.d7b99c6a-de10-454d-9370-38d18b65e9c0.columns.d8482dae-cfb8-42b8-a759-9727dd3763a6.defaultValueProvider.type}}
It contains 2 UUIDs in string representation. Unfortunately, there are several
configuration entries for each table column (having similar keys) and besides
that about a dozen of keys for table itself.
As a result, configuration keys take 68% of a meta storage message related to
table creation (for one node cluster, for a table of 2 columns and 25
partitions) which creates excessive load on meta storage raft group in case of
mass table creation (see IGNITE-19275 )
> Reduce size of configuration keys
> -
>
> Key: IGNITE-19530
> URL: https://issues.apache.org/jira/browse/IGNITE-19530
> Project: Ignite
> Issue Type: Improvement
>Reporter: Denis Chudov
>Priority: Major
> Labels: ignite-3
>
> *Motivation*
> The distributed configuration keys are byte arrays formed from strings that
> contain some constant prefixes, postfixes, delimiters and identificators,
> mostly UUIDs. Example of the configuration key for default value provider
> type of table column:
> {{dst-cfg.table.tables.d7b99c6a-de10-454d-9370-38d18b65e9c0.columns.d8482dae-cfb8-42b8-a759-9727dd3763a6.defaultValueProvider.type}}
> It contains 2 UUIDs in string representation. Unfortunately, there are
> several configuration entries for each table column (having similar keys) and
> besides that about a dozen of keys for table itself.
> As a result, configuration keys take 68% of a meta storage message related to
> table creation (for one node cluster, for a table of 2 columns and 25
> partitions) which creates excessive load on meta storage raft group in case
> of mass table creation (see IGNITE-19275 )
> *Definition of done*
> We should get rid of string representation of UUIDs in configuration keys,
> UUIDs should be written as 16 bytes each into byte array directly. Also,
> string constants should be reduced (or even replaced to constants consisting
> of few bytes) because there is no need to keep them human readable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19531) Switch table IDs from UUIDs to integers
Roman Puchkovskiy created IGNITE-19531: -- Summary: Switch table IDs from UUIDs to integers Key: IGNITE-19531 URL: https://issues.apache.org/jira/browse/IGNITE-19531 Project: Ignite Issue Type: Improvement Reporter: Roman Puchkovskiy Assignee: Roman Puchkovskiy Fix For: 3.0.0-beta2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19491) Proper utilisation of a distribution zone manager's local states.
[
https://issues.apache.org/jira/browse/IGNITE-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-19491:
-
Epic Link: IGNITE-19170
> Proper utilisation of a distribution zone manager's local states.
> -
>
> Key: IGNITE-19491
> URL: https://issues.apache.org/jira/browse/IGNITE-19491
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> In distribution zone manager we have several local entities that hold some
> states.
> * DistributionZoneManager.ZoneState#topologyAugmentationMap
> * DistributionZoneManager#nodesAttributes
> Currently {{DistributionZoneManager.ZoneState#topologyAugmentationMap}} is
> being cleaned only on a node, that successfully writes data nodes, which
> seems to be incorrect, because for example a node, that never writes data
> nodes, will never clean up its state.
> Need to find a proper solution for states utilisation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19491) Proper utilisation of a distribution zone manager's local states.
[
https://issues.apache.org/jira/browse/IGNITE-19491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-19491:
-
Description:
In distribution zone manager we have several local entities that hold some
states.
* DistributionZoneManager.ZoneState#topologyAugmentationMap
* DistributionZoneManager#nodesAttributes
Currently {{DistributionZoneManager.ZoneState#topologyAugmentationMap}} is
being cleaned only on a node, that successfully writes data nodes, which seems
to be incorrect, because for example a node, that never writes data nodes, will
never clean up its state.
Need to find a proper solution for states utilisation.
UPD:
Proposed solution:
ZoneState#topologyAugmentationMap should be cleared when we read scaleUp/Down
keys in saveDataNodesToMetaStorageOnScaleUp/Down methods,
when we retrieved this keys, it is safe to call
{{zoneState.cleanUp(Math.min(scaleDownTriggerRevision,
scaleUpTriggerRevision))}}, but we also need to change semantic of {{cleanUp}}
method, so we need to call {{topologyAugmentationMap.headMap(toKey,
false).clear()}} with false flag, to not include the passed key and not to
remove it from the tree.
was:
In distribution zone manager we have several local entities that hold some
states.
* DistributionZoneManager.ZoneState#topologyAugmentationMap
* DistributionZoneManager#nodesAttributes
Currently {{DistributionZoneManager.ZoneState#topologyAugmentationMap}} is
being cleaned only on a node, that successfully writes data nodes, which seems
to be incorrect, because for example a node, that never writes data nodes, will
never clean up its state.
Need to find a proper solution for states utilisation.
> Proper utilisation of a distribution zone manager's local states.
> -
>
> Key: IGNITE-19491
> URL: https://issues.apache.org/jira/browse/IGNITE-19491
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> In distribution zone manager we have several local entities that hold some
> states.
> * DistributionZoneManager.ZoneState#topologyAugmentationMap
> * DistributionZoneManager#nodesAttributes
> Currently {{DistributionZoneManager.ZoneState#topologyAugmentationMap}} is
> being cleaned only on a node, that successfully writes data nodes, which
> seems to be incorrect, because for example a node, that never writes data
> nodes, will never clean up its state.
> Need to find a proper solution for states utilisation.
> UPD:
> Proposed solution:
> ZoneState#topologyAugmentationMap should be cleared when we read scaleUp/Down
> keys in saveDataNodesToMetaStorageOnScaleUp/Down methods,
> when we retrieved this keys, it is safe to call
> {{zoneState.cleanUp(Math.min(scaleDownTriggerRevision,
> scaleUpTriggerRevision))}}, but we also need to change semantic of
> {{cleanUp}} method, so we need to call
> {{topologyAugmentationMap.headMap(toKey, false).clear()}} with false flag, to
> not include the passed key and not to remove it from the tree.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19532) Introduce happends before between meta storage safe time local publication and completion of corresponding meta storage listners
Alexander Lapin created IGNITE-19532: Summary: Introduce happends before between meta storage safe time local publication and completion of corresponding meta storage listners Key: IGNITE-19532 URL: https://issues.apache.org/jira/browse/IGNITE-19532 Project: Ignite Issue Type: Improvement Reporter: Alexander Lapin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19532) Introduce happends before relation between local meta storage safe time publication and completion of corresponding meta storage listners
[ https://issues.apache.org/jira/browse/IGNITE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-19532: - Summary: Introduce happends before relation between local meta storage safe time publication and completion of corresponding meta storage listners (was: Introduce happends before relation between meta storage safe time local publication and completion of corresponding meta storage listners) > Introduce happends before relation between local meta storage safe time > publication and completion of corresponding meta storage listners > - > > Key: IGNITE-19532 > URL: https://issues.apache.org/jira/browse/IGNITE-19532 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > > Introduce happens before relation between meta storage safe time local > publication and completion of corresponding meta storage listeners -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19532) Introduce happends before relation between meta storage safe time local publication and completion of corresponding meta storage listners
[ https://issues.apache.org/jira/browse/IGNITE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-19532: - Summary: Introduce happends before relation between meta storage safe time local publication and completion of corresponding meta storage listners (was: Introduce happends before between meta storage safe time local publication and completion of corresponding meta storage listners) > Introduce happends before relation between meta storage safe time local > publication and completion of corresponding meta storage listners > - > > Key: IGNITE-19532 > URL: https://issues.apache.org/jira/browse/IGNITE-19532 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19532) Introduce happends before relation between meta storage safe time local publication and completion of corresponding meta storage listners
[ https://issues.apache.org/jira/browse/IGNITE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-19532: - Description: Introduce happens before relation between meta storage safe time local publication and completion of corresponding meta storage listeners > Introduce happends before relation between meta storage safe time local > publication and completion of corresponding meta storage listners > - > > Key: IGNITE-19532 > URL: https://issues.apache.org/jira/browse/IGNITE-19532 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > > Introduce happens before relation between meta storage safe time local > publication and completion of corresponding meta storage listeners -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19532) Introduce happends before relation between local meta storage safe time publication and completion of corresponding meta storage listners
[
https://issues.apache.org/jira/browse/IGNITE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-19532:
-
Description:
h3. Motivation
Let's assume that we have a component named LeaseTracker that locally stores
leases in a map though meta storage updates
{code:java}
public class LeaseTracker implements PlacementDriver {
/** Leases cache. */
private final Map leases;
...
private class UpdateListener implements WatchListener {
@Override
public CompletableFuture onUpdate(WatchEvent event) {
for (EntryEvent entry : event.entryEvents()) {
...
Lease lease = fromBytes(msEntry.value());
leases.put(grpId, lease);
...
return completedFuture(null);
}
}
...{code}
was:Introduce happens before relation between meta storage safe time local
publication and completion of corresponding meta storage listeners
> Introduce happends before relation between local meta storage safe time
> publication and completion of corresponding meta storage listners
> -
>
> Key: IGNITE-19532
> URL: https://issues.apache.org/jira/browse/IGNITE-19532
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>
> h3. Motivation
> Let's assume that we have a component named LeaseTracker that locally stores
> leases in a map though meta storage updates
> {code:java}
> public class LeaseTracker implements PlacementDriver {
> /** Leases cache. */
> private final Map leases;
> ...
> private class UpdateListener implements WatchListener {
> @Override
> public CompletableFuture onUpdate(WatchEvent event) {
> for (EntryEvent entry : event.entryEvents()) {
> ...
> Lease lease = fromBytes(msEntry.value());
> leases.put(grpId, lease);
> ...
> return completedFuture(null);
> }
> }
> ...{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19532) Introduce happends before relation between local meta storage safe time publication and completion of corresponding meta storage listners
[
https://issues.apache.org/jira/browse/IGNITE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-19532:
-
Description:
h3. Motivation
Let's assume that we have a component named LeaseTracker that locally stores
leases in a map though meta storage updates
{code:java}
public class LeaseTracker implements PlacementDriver {
/** Leases cache. */
private final Map leases;
...
private class UpdateListener implements WatchListener {
@Override
public CompletableFuture onUpdate(WatchEvent event) {
for (EntryEvent entry : event.entryEvents()) {
...
Lease lease = fromBytes(msEntry.value());
leases.put(grpId, lease);
...
return completedFuture(null);
}
}
...{code}
and we want to await lease in a meta storage safe time bounded way.
{code:java}
public CompletableFuture getPrimaryReplica(ReplicationGroupId
replicationGroupId, HybridTimestamp timestamp) {
...
return msManager.clusterTime().waitFor(timestamp).thenApply(() -> {
...
Lease lease0 = leases.get(replicationGroupId);
if (lease.getExpirationTime().after(timestamp)) {
return lease0;
} else {
return null;
}
...
}));
} finally {
busyLock.leaveBusy();
}
} {code}
Currently we may
was:
h3. Motivation
Let's assume that we have a component named LeaseTracker that locally stores
leases in a map though meta storage updates
{code:java}
public class LeaseTracker implements PlacementDriver {
/** Leases cache. */
private final Map leases;
...
private class UpdateListener implements WatchListener {
@Override
public CompletableFuture onUpdate(WatchEvent event) {
for (EntryEvent entry : event.entryEvents()) {
...
Lease lease = fromBytes(msEntry.value());
leases.put(grpId, lease);
...
return completedFuture(null);
}
}
...{code}
> Introduce happends before relation between local meta storage safe time
> publication and completion of corresponding meta storage listners
> -
>
> Key: IGNITE-19532
> URL: https://issues.apache.org/jira/browse/IGNITE-19532
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>
> h3. Motivation
> Let's assume that we have a component named LeaseTracker that locally stores
> leases in a map though meta storage updates
> {code:java}
> public class LeaseTracker implements PlacementDriver {
> /** Leases cache. */
> private final Map leases;
> ...
> private class UpdateListener implements WatchListener {
> @Override
> public CompletableFuture onUpdate(WatchEvent event) {
> for (EntryEvent entry : event.entryEvents()) {
> ...
> Lease lease = fromBytes(msEntry.value());
> leases.put(grpId, lease);
> ...
> return completedFuture(null);
> }
> }
> ...{code}
> and we want to await lease in a meta storage safe time bounded way.
> {code:java}
> public CompletableFuture getPrimaryReplica(ReplicationGroupId
> replicationGroupId, HybridTimestamp timestamp) {
> ...
> return msManager.clusterTime().waitFor(timestamp).thenApply(() -> {
> ...
> Lease lease0 = leases.get(replicationGroupId);
> if (lease.getExpirationTime().after(timestamp)) {
> return lease0;
> } else {
> return null;
> }
> ...
> }));
> } finally {
> busyLock.leaveBusy();
> }
> } {code}
> Currently we may
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19532) Introduce happends before relation between local meta storage safe time publication and completion of corresponding meta storage listners
[
https://issues.apache.org/jira/browse/IGNITE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-19532:
-
Description:
h3. Motivation
Let's assume that we have a component named LeaseTracker that locally stores
leases in a map though meta storage updates
{code:java}
public class LeaseTracker implements PlacementDriver {
/** Leases cache. */
private final Map leases;
...
private class UpdateListener implements WatchListener {
@Override
public CompletableFuture onUpdate(WatchEvent event) {
for (EntryEvent entry : event.entryEvents()) {
...
Lease lease = fromBytes(msEntry.value());
leases.put(grpId, lease);
...
return completedFuture(null);
}
}
...{code}
and we want to await lease in a meta storage safe time bounded way.
{code:java}
public CompletableFuture getPrimaryReplica(ReplicationGroupId
replicationGroupId, HybridTimestamp timestamp) {
...
return msManager.clusterTime().waitFor(timestamp).thenApply(() -> {
...
Lease lease0 = leases.get(replicationGroupId);
if (lease.getExpirationTime().after(timestamp)) {
return lease0;
} else {
return null;
}
...
}
} {code}
Currently that won't work properly
because
{code:java}
msManager.clusterTime().waitFor(timestamp) {code}
will be completed before local leases will be populated with data through meta
storage events.
Thus it'll be nice to complete clusterTime().waitFor(timestamp) futute (and
generally speaking publish safe time) after all corresponding ms listeners are
completed.
was:
h3. Motivation
Let's assume that we have a component named LeaseTracker that locally stores
leases in a map though meta storage updates
{code:java}
public class LeaseTracker implements PlacementDriver {
/** Leases cache. */
private final Map leases;
...
private class UpdateListener implements WatchListener {
@Override
public CompletableFuture onUpdate(WatchEvent event) {
for (EntryEvent entry : event.entryEvents()) {
...
Lease lease = fromBytes(msEntry.value());
leases.put(grpId, lease);
...
return completedFuture(null);
}
}
...{code}
and we want to await lease in a meta storage safe time bounded way.
{code:java}
public CompletableFuture getPrimaryReplica(ReplicationGroupId
replicationGroupId, HybridTimestamp timestamp) {
...
return msManager.clusterTime().waitFor(timestamp).thenApply(() -> {
...
Lease lease0 = leases.get(replicationGroupId);
if (lease.getExpirationTime().after(timestamp)) {
return lease0;
} else {
return null;
}
...
}));
} finally {
busyLock.leaveBusy();
}
} {code}
Currently we may
> Introduce happends before relation between local meta storage safe time
> publication and completion of corresponding meta storage listners
> -
>
> Key: IGNITE-19532
> URL: https://issues.apache.org/jira/browse/IGNITE-19532
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>
> h3. Motivation
> Let's assume that we have a component named LeaseTracker that locally stores
> leases in a map though meta storage updates
> {code:java}
> public class LeaseTracker implements PlacementDriver {
> /** Leases cache. */
> private final Map leases;
> ...
> private class UpdateListener implements WatchListener {
> @Override
> public CompletableFuture onUpdate(WatchEvent event) {
> for (EntryEvent entry : event.entryEvents()) {
> ...
> Lease lease = fromBytes(msEntry.value());
> leases.put(grpId, lease);
> ...
> return completedFuture(null);
> }
> }
> ...{code}
> and we want to await lease in a meta storage safe time bounded way.
> {code:java}
> public CompletableFuture getPrimaryReplica(ReplicationGroupId
> replicationGroupId, HybridTimestamp timestamp) {
> ...
> return msManager.clusterTime().waitFor(timestamp).thenApply(() -> {
> ...
> Lease lease0 = leases.get(replicationGroupId);
> if (lease.getExpirationTime().after(timestamp)) {
> return lease
[jira] [Commented] (IGNITE-19389) Make lock acquiring timeout message more certain.
[
https://issues.apache.org/jira/browse/IGNITE-19389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724352#comment-17724352
]
Ignite TC Bot commented on IGNITE-19389:
{panel:title=Branch: [pull/10726/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10726/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *--> Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7181278&buildTypeId=IgniteTests24Java8_RunAll]
> Make lock acquiring timeout message more certain.
> -
>
> Key: IGNITE-19389
> URL: https://issues.apache.org/jira/browse/IGNITE-19389
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Vladimir Steshin
>Priority: Minor
>
> Suggestion: make text of lock acquiring timeout message more certain if there
> is no attempt to acquire like: "_Didn't try to take lock for transaction ...
> because tx has already timeouted_" instead of common "_Failed to acquire lock
> within provided timeout for transaction_"
> We've met a case when user sees lock acquiring timeout in transaction but is
> confused in researching the cause. He expected and tried to find concurrent
> transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't
> even launched. The timeout has already happened. Stacktrace indicates the
> problem, but it is not the best solution to a user.
> The stacktrace:
> {code:java}
> org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException:
> Failed to acquire lock within provided timeout for transaction
> [timeout=3,
> tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
> xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
> nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
> [topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
> concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
> invalidate=false, rollbackOnly=false,
> nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
> startTime=1673243469168, duration=30255]]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:802)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-19389) Make lock acquiring timeout message more certain.
[
https://issues.apache.org/jira/browse/IGNITE-19389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Steshin reassigned IGNITE-19389:
-
Assignee: Vladimir Steshin
> Make lock acquiring timeout message more certain.
> -
>
> Key: IGNITE-19389
> URL: https://issues.apache.org/jira/browse/IGNITE-19389
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Minor
>
> Suggestion: make text of lock acquiring timeout message more certain if there
> is no attempt to acquire like: "_Didn't try to take lock for transaction ...
> because tx has already timeouted_" instead of common "_Failed to acquire lock
> within provided timeout for transaction_"
> We've met a case when user sees lock acquiring timeout in transaction but is
> confused in researching the cause. He expected and tried to find concurrent
> transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't
> even launched. The timeout has already happened. Stacktrace indicates the
> problem, but it is not the best solution to a user.
> The stacktrace:
> {code:java}
> org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException:
> Failed to acquire lock within provided timeout for transaction
> [timeout=3,
> tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
> xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
> nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
> [topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
> concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
> invalidate=false, rollbackOnly=false,
> nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
> startTime=1673243469168, duration=30255]]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:802)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19389) Make lock acquiring timeout message more certain.
[
https://issues.apache.org/jira/browse/IGNITE-19389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Petrov updated IGNITE-19389:
Ignite Flags: (was: Docs Required,Release Notes Required)
> Make lock acquiring timeout message more certain.
> -
>
> Key: IGNITE-19389
> URL: https://issues.apache.org/jira/browse/IGNITE-19389
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Minor
> Fix For: 2.16
>
>
> Suggestion: make text of lock acquiring timeout message more certain if there
> is no attempt to acquire like: "_Didn't try to take lock for transaction ...
> because tx has already timeouted_" instead of common "_Failed to acquire lock
> within provided timeout for transaction_"
> We've met a case when user sees lock acquiring timeout in transaction but is
> confused in researching the cause. He expected and tried to find concurrent
> transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't
> even launched. The timeout has already happened. Stacktrace indicates the
> problem, but it is not the best solution to a user.
> The stacktrace:
> {code:java}
> org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException:
> Failed to acquire lock within provided timeout for transaction
> [timeout=3,
> tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
> xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
> nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
> [topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
> concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
> invalidate=false, rollbackOnly=false,
> nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
> startTime=1673243469168, duration=30255]]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:802)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19389) Make lock acquiring timeout message more certain.
[
https://issues.apache.org/jira/browse/IGNITE-19389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Petrov updated IGNITE-19389:
Fix Version/s: 2.16
> Make lock acquiring timeout message more certain.
> -
>
> Key: IGNITE-19389
> URL: https://issues.apache.org/jira/browse/IGNITE-19389
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Minor
> Fix For: 2.16
>
>
> Suggestion: make text of lock acquiring timeout message more certain if there
> is no attempt to acquire like: "_Didn't try to take lock for transaction ...
> because tx has already timeouted_" instead of common "_Failed to acquire lock
> within provided timeout for transaction_"
> We've met a case when user sees lock acquiring timeout in transaction but is
> confused in researching the cause. He expected and tried to find concurrent
> transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't
> even launched. The timeout has already happened. Stacktrace indicates the
> problem, but it is not the best solution to a user.
> The stacktrace:
> {code:java}
> org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException:
> Failed to acquire lock within provided timeout for transaction
> [timeout=3,
> tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
> xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
> nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
> [topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
> concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
> invalidate=false, rollbackOnly=false,
> nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
> startTime=1673243469168, duration=30255]]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:802)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-19389) Make lock acquiring timeout message more certain.
[
https://issues.apache.org/jira/browse/IGNITE-19389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724374#comment-17724374
]
Mikhail Petrov commented on IGNITE-19389:
-
[~vladsz83] Thank you for the contribution.
> Make lock acquiring timeout message more certain.
> -
>
> Key: IGNITE-19389
> URL: https://issues.apache.org/jira/browse/IGNITE-19389
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Minor
> Fix For: 2.16
>
>
> Suggestion: make text of lock acquiring timeout message more certain if there
> is no attempt to acquire like: "_Didn't try to take lock for transaction ...
> because tx has already timeouted_" instead of common "_Failed to acquire lock
> within provided timeout for transaction_"
> We've met a case when user sees lock acquiring timeout in transaction but is
> confused in researching the cause. He expected and tried to find concurrent
> transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't
> even launched. The timeout has already happened. Stacktrace indicates the
> problem, but it is not the best solution to a user.
> The stacktrace:
> {code:java}
> org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException:
> Failed to acquire lock within provided timeout for transaction
> [timeout=3,
> tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
> xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
> nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
> [topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
> concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
> invalidate=false, rollbackOnly=false,
> nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
> startTime=1673243469168, duration=30255]]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:802)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19389) Make lock acquiring timeout message more certain.
[
https://issues.apache.org/jira/browse/IGNITE-19389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Steshin updated IGNITE-19389:
--
Ignite Flags: Release Notes Required
> Make lock acquiring timeout message more certain.
> -
>
> Key: IGNITE-19389
> URL: https://issues.apache.org/jira/browse/IGNITE-19389
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.12
>Reporter: Vladimir Steshin
>Assignee: Vladimir Steshin
>Priority: Minor
> Fix For: 2.16
>
>
> Suggestion: make text of lock acquiring timeout message more certain if there
> is no attempt to acquire like: "_Didn't try to take lock for transaction ...
> because tx has already timeouted_" instead of common "_Failed to acquire lock
> within provided timeout for transaction_"
> We've met a case when user sees lock acquiring timeout in transaction but is
> confused in researching the cause. He expected and tried to find concurrent
> transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't
> even launched. The timeout has already happened. Stacktrace indicates the
> problem, but it is not the best solution to a user.
> The stacktrace:
> {code:java}
> org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException:
> Failed to acquire lock within provided timeout for transaction
> [timeout=3,
> tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
> xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
> nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
> [topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
> concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
> invalidate=false, rollbackOnly=false,
> nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
> startTime=1673243469168, duration=30255]]
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
> ~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:802)
> [ignite-core-2.12.0-p6.jar:2.12.0-p6]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-19381) TimestampAware messages sometimes lacks timestamps
[
https://issues.apache.org/jira/browse/IGNITE-19381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724409#comment-17724409
]
Vladislav Pyatkov commented on IGNITE-19381:
Merged 22f6d0d1d3c74f40af39ca5f540ccc73abddbae4
> TimestampAware messages sometimes lacks timestamps
> --
>
> Key: IGNITE-19381
> URL: https://issues.apache.org/jira/browse/IGNITE-19381
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Blocker
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Motivation
> While HybridTimestamp refactoring [~ibessonov] noticed that RW related
> messages might have null as a timestamp that seems to be incorrect. Probably
> we've missed
> {code:java}
> .timestampLong(clock.nowLong());{code}
> within some RW messages, e.g. while building
> ReadWriteScanRetrieveBatchReplicaRequest.
> h3. Definition of done
> Thus it's required to:
> * add timestampLong() wherever needed.
> * add assertion that will check that TimestampAware messages have @NotNull
> timestamp field.
> * replace "nullableHybridTimestamp(...)" call with "hybridTimestamp(...),
> etc.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
