[ignite-3] branch ignite-19942 updated (beadc36d2b -> 8a39a398e6)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19942 in repository https://gitbox.apache.org/repos/asf/ignite-3.git from beadc36d2b minor. add 8a39a398e6 fix tests No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/ignite/internal/index/IndexManager.java | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
[ignite-3] branch ignite-19903 updated (1b7652f6af -> f9730731f3)
This is an automated email from the ASF dual-hosted git repository.
sdanilov pushed a change to branch ignite-19903
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
omit 1b7652f6af Review fixes
omit 0c46366f3c .
omit 50f4d965e5 .
omit 43dbdb2571 .
omit b90793d3d0 .
omit 8278f70a23 .
omit 30de0fb088 .
omit 593f061a70 .
omit 463d9e00ba .
omit 2b40a6ee39 .
omit f97d612f92 IGNITE-19903 Fix recovery descriptor race condition
add f8a280e514 IGNITE-19954 .NET: Work around tx conflicts in
TestAutoFlushFrequency (#2309)
add 0a6e79264b IGNITE-19948 Add authenticated username to the prompt
(#2304)
add 7a5fb6bc54 IGNITE-19941 Sql. Test
ExecutionServiceImplTest.testCancelOnInitiator is flaky (#2305)
add e741560fd3 IGNITE-19927 Improve test coverage for CREATE TABLE (#2297)
add fef7a24c2a IGNITE-19812 Do not swallow exceptions during start of a
partition (#2240)
add b48b5c34af IGNITE-19903 Fix recovery descriptor race condition
add c3a6c7b8e7 .
add e5c3200f0d .
add bc6227408f .
add 351e73e065 .
add b05aeb7f2d .
add 7fbf3723d8 .
add e9b7e133e8 .
add 597cf38489 .
add bbb980422a .
add 596cfddc98 Review fixes
add f9730731f3 Review fixes
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (1b7652f6af)
\
N -- N -- N refs/heads/ignite-19903 (f9730731f3)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../internal/catalog/CatalogServiceImpl.java | 66 ++-
.../internal/catalog/CatalogServiceSelfTest.java | 220 ++---
.../ItConnectToClusterTestBase.java| 10 +-
...tConnectWithBasicAuthenticationCommandTest.java | 102 ++
.../commands/questions/ItConnectToClusterTest.java | 1 +
.../questions/ItConnectToSslClusterTest.java | 1 +
.../repl/executor/ItIgnitePicocliCommandsTest.java | 2 +-
.../internal/cli/call/connect/ConnectCall.java | 38 +++-
.../handler/IgniteCliApiExceptionHandler.java | 8 +-
.../ignite/internal/cli/core/repl/SessionInfo.java | 9 +-
.../cli/core/repl/prompt/ReplPromptProvider.java | 13 +-
.../internal/cli/core/rest/ApiClientFactory.java | 39 +++-
.../core/repl/SessionDefaultValueProviderTest.java | 2 +-
.../filter/DynamicCompleterFilterTest.java | 3 +-
.../recovery/RecoveryClientHandshakeManager.java | 2 +
.../recovery/RecoveryServerHandshakeManager.java | 2 +
.../dotnet/Apache.Ignite.Tests/MetricsTests.cs | 2 +-
.../Apache.Ignite.Tests/Table/DataStreamerTests.cs | 7 +-
.../dotnet/Apache.Ignite.Tests/TestUtils.cs| 16 +-
.../dotnet/Apache.Ignite/Internal/ClientSocket.cs | 2 +-
.../runner/app/ItIgniteNodeRestartTest.java| 16 +-
.../internal/sql/engine/ItCreateTableDdlTest.java | 41 +++-
.../streamer/ItAbstractDataStreamerTest.java | 1 +
.../schema/configuration/TableValidatorImpl.java | 2 +-
.../sql/engine/exec/ExecutionServiceImplTest.java | 56 --
.../internal/table/distributed/TableManager.java | 26 +--
26 files changed, 554 insertions(+), 133 deletions(-)
rename
modules/cli/src/integrationTest/java/org/apache/ignite/internal/cli/commands/{questions
=> }/ItConnectToClusterTestBase.java (90%)
create mode 100644
modules/cli/src/integrationTest/java/org/apache/ignite/internal/cli/commands/connect/ItConnectWithBasicAuthenticationCommandTest.java
[ignite] branch master updated: IGNITE-19158 Improved message about received partition file during snapshot restore (#10762)
This is an automated email from the ASF dual-hosted git repository. mpetrov pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/ignite.git The following commit(s) were added to refs/heads/master by this push: new a107d200789 IGNITE-19158 Improved message about received partition file during snapshot restore (#10762) a107d200789 is described below commit a107d2007894363ffefdac27ee9e209227652c94 Author: Julia Bakulina <96814257+j-bak...@users.noreply.github.com> AuthorDate: Wed Jul 12 18:19:55 2023 +0300 IGNITE-19158 Improved message about received partition file during snapshot restore (#10762) --- .../ignite/internal/managers/communication/GridIoManager.java| 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java b/modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java index 1fc734d3f12..4433000c34c 100644 --- a/modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java +++ b/modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java @@ -31,7 +31,6 @@ import java.net.Socket; import java.net.SocketTimeoutException; import java.nio.channels.Channel; import java.nio.channels.FileChannel; -import java.nio.channels.ReadableByteChannel; import java.nio.channels.SocketChannel; import java.nio.channels.WritableByteChannel; import java.time.Instant; @@ -2917,7 +2916,7 @@ public class GridIoManager extends
[ignite-3] branch main updated: IGNITE-19812 Do not swallow exceptions during start of a partition (#2240)
This is an automated email from the ASF dual-hosted git repository.
sdanilov pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/main by this push:
new fef7a24c2a IGNITE-19812 Do not swallow exceptions during start of a
partition (#2240)
fef7a24c2a is described below
commit fef7a24c2a029cac720d2fea3815c2a70a86b72f
Author: Roman Puchkovskiy
AuthorDate: Wed Jul 12 19:19:47 2023 +0400
IGNITE-19812 Do not swallow exceptions during start of a partition (#2240)
---
.../runner/app/ItIgniteNodeRestartTest.java| 16 -
.../internal/table/distributed/TableManager.java | 26 +-
2 files changed, 26 insertions(+), 16 deletions(-)
diff --git
a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
index 47f2d5f4f7..7a9b323950 100644
---
a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
+++
b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
@@ -28,7 +28,6 @@ import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
-import static org.mockito.Mockito.mock;
import java.io.IOException;
import java.nio.file.Files;
@@ -313,6 +312,12 @@ public class ItIgniteNodeRestartTest extends
BaseIgniteRestartTest {
Consumer>> registry = (c) ->
metaStorageMgr.registerRevisionUpdateListener(c::apply);
+var baselineManager = new BaselineManager(
+clusterCfgMgr,
+metaStorageMgr,
+clusterSvc
+);
+
DataStorageModules dataStorageModules = new
DataStorageModules(ServiceLoader.load(DataStorageModule.class));
Path storagePath = getPartitionsStorePath(dir);
@@ -362,7 +367,7 @@ public class ItIgniteNodeRestartTest extends
BaseIgniteRestartTest {
replicaMgr,
lockManager,
replicaService,
-mock(BaselineManager.class),
+baselineManager,
clusterSvc.topologyService(),
txManager,
dataStorageManager,
@@ -416,6 +421,7 @@ public class ItIgniteNodeRestartTest extends
BaseIgniteRestartTest {
cmgManager,
replicaMgr,
txManager,
+baselineManager,
metaStorageMgr,
clusterCfgMgr,
dataStorageManager,
@@ -967,7 +973,7 @@ public class ItIgniteNodeRestartTest extends
BaseIgniteRestartTest {
* Checks that the table created in cluster of 2 nodes, is recovered on a
node after restart of this node.
*/
@Test
-public void testRecoveryOnOneNode() throws InterruptedException {
+public void testRecoveryOnOneNode() {
IgniteImpl ignite = startNode(0);
IgniteImpl node = startNode(1);
@@ -1048,7 +1054,7 @@ public class ItIgniteNodeRestartTest extends
BaseIgniteRestartTest {
*/
@Test
@Disabled(value = "https://issues.apache.org/jira/browse/IGNITE-18919;)
-public void testMetastorageStop() throws InterruptedException {
+public void testMetastorageStop() {
int cfgGap = 4;
List nodes = startNodes(3);
@@ -1182,7 +1188,7 @@ public class ItIgniteNodeRestartTest extends
BaseIgniteRestartTest {
* @param name Table name.
* @param replicas Replica factor.
*/
-private void createTableWithData(List nodes, String name, int
replicas) throws InterruptedException {
+private void createTableWithData(List nodes, String name, int
replicas) {
createTableWithData(nodes, name, replicas, 2);
}
diff --git
a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index 9e038c48f0..e7bfde3fd0 100644
---
a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++
b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -141,6 +141,7 @@ import org.apache.ignite.internal.schema.event.SchemaEvent;
import org.apache.ignite.internal.schema.event.SchemaEventParameters;
import org.apache.ignite.internal.storage.DataStorageManager;
import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.RowId;
import org.apache.ignite.internal.storage.StorageException;
import org.apache.ignite.internal.storage.engine.MvTableStorage;
import
[ignite-3] branch ignite-19942 updated (60b1709a2a -> beadc36d2b)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19942 in repository https://gitbox.apache.org/repos/asf/ignite-3.git omit 60b1709a2a minor. add beadc36d2b minor. This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (60b1709a2a) \ N -- N -- N refs/heads/ignite-19942 (beadc36d2b) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java | 6 +- 1 file changed, 1 insertion(+), 5 deletions(-)
[ignite-3] branch ignite-19942 updated (8a7a6ab93e -> 60b1709a2a)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19942 in repository https://gitbox.apache.org/repos/asf/ignite-3.git from 8a7a6ab93e Mirror Index changes from Config to Catalog add 60b1709a2a minor. No new revisions were added by this update. Summary of changes: .../internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java | 5 - 1 file changed, 4 insertions(+), 1 deletion(-)
[ignite-3] branch ignite-19942 updated (d975dca13c -> 8a7a6ab93e)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19942 in repository https://gitbox.apache.org/repos/asf/ignite-3.git omit d975dca13c Mirror Index changes from Config to Catalog omit d93e2f0207 Mirror Table changes from Config to Catalog. omit fd79660fb4 Mirror DistributionZone changes from Config to Catalog. add 1910afa0b1 IGNITE-18959 Placement driver's local map cleanup on assingments removal (#2292) add 7e1e0cf74c IGNITE-19914 .NET: Fix colocation column order (#2308) add d305b4f153 IGNITE-19801 Move ConfigurationStorageRevisionListener to test fixtures (#2307) add f8a280e514 IGNITE-19954 .NET: Work around tx conflicts in TestAutoFlushFrequency (#2309) add 0a6e79264b IGNITE-19948 Add authenticated username to the prompt (#2304) add 7a5fb6bc54 IGNITE-19941 Sql. Test ExecutionServiceImplTest.testCancelOnInitiator is flaky (#2305) add e741560fd3 IGNITE-19927 Improve test coverage for CREATE TABLE (#2297) add 2e08b78f77 Mirror DistributionZone changes from Config to Catalog. add 9bddcb851e Mirror Table changes from Config to Catalog. add 8a7a6ab93e Mirror Index changes from Config to Catalog This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (d975dca13c) \ N -- N -- N refs/heads/ignite-19942 (8a7a6ab93e) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../internal/catalog/CatalogServiceImpl.java | 64 ++- .../internal/catalog/CatalogServiceSelfTest.java | 128 + .../ItConnectToClusterTestBase.java| 10 +- ...tConnectWithBasicAuthenticationCommandTest.java | 102 .../commands/questions/ItConnectToClusterTest.java | 1 + .../questions/ItConnectToSslClusterTest.java | 1 + .../repl/executor/ItIgnitePicocliCommandsTest.java | 2 +- .../internal/cli/call/connect/ConnectCall.java | 38 +- .../handler/IgniteCliApiExceptionHandler.java | 8 +- .../ignite/internal/cli/core/repl/SessionInfo.java | 9 +- .../cli/core/repl/prompt/ReplPromptProvider.java | 13 ++- .../internal/cli/core/rest/ApiClientFactory.java | 39 +-- .../core/repl/SessionDefaultValueProviderTest.java | 2 +- .../filter/DynamicCompleterFilterTest.java | 3 +- .../configuration/ConfigurationChanger.java| 69 --- .../configuration/ConfigurationRegistry.java | 57 + .../storage/ConfigurationStorageListener.java | 9 +- .../notifications/ConfigurationListenerTest.java | 105 + .../configuration/TestConfigurationChanger.java| 5 - .../ConfigurationStorageRevisionListener.java | 1 - ...ConfigurationStorageRevisionListenerHolder.java | 0 .../internal/metastorage/MetaStorageManager.java | 10 ++ .../ignite/internal/metastorage/WatchListener.java | 11 -- .../metastorage/impl/MetaStorageManagerImpl.java | 2 + .../ignite/internal/metastorage/server/Watch.java | 9 -- .../metastorage/server/WatchProcessorTest.java | 5 - .../placementdriver/AssignmentsTracker.java| 2 +- .../negotiation/LeaseNegotiator.java | 11 +- .../SerializerHandlerBenchmarksBase.cs | 17 ++- .../dotnet/Apache.Ignite.Tests/MetricsTests.cs | 2 +- .../Proto/ColocationHashTests.cs | 61 -- .../Apache.Ignite.Tests/Table/DataStreamerTests.cs | 7 +- .../Serialization/ObjectSerializerHandlerTests.cs | 15 ++- .../dotnet/Apache.Ignite.Tests/TestUtils.cs| 16 ++- .../dotnet/Apache.Ignite/Internal/ClientSocket.cs | 2 +- .../Proto/BinaryTuple/BinaryTupleBuilder.cs| 125 .../BinaryTuple/IHashedColumnIndexProvider.cs | 11 +- .../Apache.Ignite/Internal/Proto/HashUtils.cs | 2 +- .../Apache.Ignite/Internal/Table/DataStreamer.cs | 2 +- .../dotnet/Apache.Ignite/Internal/Table/Schema.cs | 9 +- .../Serialization/IRecordSerializerHandler.cs | 2 +- .../dotnet/Apache.Ignite/Internal/Table/Table.cs | 15 ++- .../ignite/internal/BaseIgniteRestartTest.java | 4 +- .../ItDistributedConfigurationPropertiesTest.java | 39 +++ .../ItDistributedConfigurationStorageTest.java | 12 +- .../runner/app/ItIgniteNodeRestartTest.java| 2 +- .../internal/sql/api/ItSqlAsynchronousApiTest.java | 2 +-
[ignite-3] branch ignite-19927 deleted (was 0b9b2451ca)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19927 in repository https://gitbox.apache.org/repos/asf/ignite-3.git was 0b9b2451ca Minors. The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-3] branch main updated (7a5fb6bc54 -> e741560fd3)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/ignite-3.git from 7a5fb6bc54 IGNITE-19941 Sql. Test ExecutionServiceImplTest.testCancelOnInitiator is flaky (#2305) add e741560fd3 IGNITE-19927 Improve test coverage for CREATE TABLE (#2297) No new revisions were added by this update. Summary of changes: .../internal/catalog/CatalogServiceImpl.java | 66 ++- .../internal/catalog/CatalogServiceSelfTest.java | 220 ++--- .../internal/sql/engine/ItCreateTableDdlTest.java | 41 +++- .../schema/configuration/TableValidatorImpl.java | 2 +- 4 files changed, 255 insertions(+), 74 deletions(-)
[ignite-3] branch main updated (7e1e0cf74c -> d305b4f153)
This is an automated email from the ASF dual-hosted git repository.
ibessonov pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
from 7e1e0cf74c IGNITE-19914 .NET: Fix colocation column order (#2308)
add d305b4f153 IGNITE-19801 Move ConfigurationStorageRevisionListener to
test fixtures (#2307)
No new revisions were added by this update.
Summary of changes:
.../configuration/ConfigurationChanger.java| 69 +-
.../configuration/ConfigurationRegistry.java | 57 +--
.../storage/ConfigurationStorageListener.java | 9 +-
.../notifications/ConfigurationListenerTest.java | 105 +
.../configuration/TestConfigurationChanger.java| 5 -
.../ConfigurationStorageRevisionListener.java | 1 -
...ConfigurationStorageRevisionListenerHolder.java | 0
.../ignite/internal/metastorage/WatchListener.java | 11 ---
.../ignite/internal/metastorage/server/Watch.java | 9 --
.../metastorage/server/WatchProcessorTest.java | 5 -
.../ignite/internal/BaseIgniteRestartTest.java | 4 +-
.../ItDistributedConfigurationPropertiesTest.java | 39 +++-
.../ItDistributedConfigurationStorageTest.java | 12 +--
.../runner/app/ItIgniteNodeRestartTest.java| 2 +-
.../storage/DistributedConfigurationStorage.java | 7 --
.../storage/ConfigurationStorageTest.java | 12 +--
16 files changed, 47 insertions(+), 300 deletions(-)
rename modules/configuration/src/{main =>
testFixtures}/java/org/apache/ignite/internal/configuration/notifications/ConfigurationStorageRevisionListener.java
(96%)
rename modules/configuration/src/{main =>
testFixtures}/java/org/apache/ignite/internal/configuration/notifications/ConfigurationStorageRevisionListenerHolder.java
(100%)
[ignite] branch IGNITE-17700__realtime_cdc updated: IGNITE-19622 Add realtime CDC buffer (#10778)
This is an automated email from the ASF dual-hosted git repository.
timoninmaxim pushed a commit to branch IGNITE-17700__realtime_cdc
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/IGNITE-17700__realtime_cdc by
this push:
new 1c2cf8dd0a4 IGNITE-19622 Add realtime CDC buffer (#10778)
1c2cf8dd0a4 is described below
commit 1c2cf8dd0a426e88d9b70cc7d15db71f9edd694c
Author: Maksim Timonin
AuthorDate: Wed Jul 12 13:47:01 2023 +0300
IGNITE-19622 Add realtime CDC buffer (#10778)
---
.../org/apache/ignite/IgniteSystemProperties.java | 8 +
.../configuration/DataStorageConfiguration.java| 55 +
.../cache/persistence/cdc/CdcBuffer.java | 164 +
.../cache/persistence/cdc/CdcBufferConsumer.java | 33 +++
.../cache/persistence/cdc/CdcProcessor.java| 81 +++
.../cache/persistence/cdc/CdcWorker.java | 93
.../persistence/wal/FileWriteAheadLogManager.java | 19 +-
.../wal/filehandle/FileHandleManagerFactory.java | 6 +
.../wal/filehandle/FileHandleManagerImpl.java | 21 +-
.../wal/filehandle/FileWriteHandleImpl.java| 17 +-
.../wal/filehandle/FsyncFileHandleManagerImpl.java | 11 +-
.../wal/filehandle/FsyncFileWriteHandle.java | 16 +-
.../cdc/RealtimeCdcBufferDedicatedWorkerTest.java | 42
.../persistence/cdc/RealtimeCdcBufferSelfTest.java | 113 +
.../persistence/cdc/RealtimeCdcBufferTest.java | 259 +
.../cdc/RealtimeCdcBufferWalMmapDisabledTest.java | 27 +++
.../ignite/testsuites/IgnitePdsTestSuite2.java | 10 +
17 files changed, 966 insertions(+), 9 deletions(-)
diff --git
a/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java
b/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java
index 5fb101aebce..eaabed6607a 100644
--- a/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java
+++ b/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java
@@ -93,6 +93,7 @@ import static
org.apache.ignite.internal.processors.cache.distributed.dht.topolo
import static
org.apache.ignite.internal.processors.cache.mvcc.MvccCachingManager.DFLT_MVCC_TX_SIZE_CACHING_THRESHOLD;
import static
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.DFLT_DEFRAGMENTATION_REGION_SIZE_PERCENTAGE;
import static
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.DFLT_PDS_WAL_REBALANCE_THRESHOLD;
+import static
org.apache.ignite.internal.processors.cache.persistence.cdc.CdcWorker.DFLT_POLL_CDC_BUF_THROTTLING_TIMEOUT;
import static
org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointHistory.DFLT_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE;
import static
org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointMarkersStorage.DFLT_IGNITE_CHECKPOINT_MAP_SNAPSHOT_THRESHOLD;
import static
org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointWorkflow.DFLT_CHECKPOINT_PARALLEL_SORT_THRESHOLD;
@@ -1531,6 +1532,13 @@ public final class IgniteSystemProperties {
type = Long.class, defaults = "" + DFLT_UNWIND_THROTTLING_TIMEOUT)
public static final String IGNITE_UNWIND_THROTTLING_TIMEOUT =
"IGNITE_UNWIND_THROTTLING_TIMEOUT";
+/**
+ * Throttling timeout in millis for polling CDC buffer in realtime mode.
Default is 100 ms.
+ */
+@SystemProperty(value = "Throttling timeout in millis for polling CDC
buffer in realtime mode", type = Long.class,
+defaults = "" + DFLT_POLL_CDC_BUF_THROTTLING_TIMEOUT)
+public static final String IGNITE_THROTTLE_POLL_CDC_BUF =
"IGNITE_THROTTLE_POLL_CDC_BUF";
+
/**
* Threshold for throttling operations logging.
*/
diff --git
a/modules/core/src/main/java/org/apache/ignite/configuration/DataStorageConfiguration.java
b/modules/core/src/main/java/org/apache/ignite/configuration/DataStorageConfiguration.java
index 8daa97a1c7b..86ed3c4e3ea 100644
---
a/modules/core/src/main/java/org/apache/ignite/configuration/DataStorageConfiguration.java
+++
b/modules/core/src/main/java/org/apache/ignite/configuration/DataStorageConfiguration.java
@@ -20,6 +20,7 @@ package org.apache.ignite.configuration;
import java.io.Serializable;
import java.util.zip.Deflater;
import org.apache.ignite.IgniteSystemProperties;
+import
org.apache.ignite.internal.processors.cache.persistence.cdc.CdcBufferConsumer;
import
org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory;
import
org.apache.ignite.internal.processors.cache.persistence.file.FileIOFactory;
import
org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory;
@@ -252,6 +253,14 @@ public class DataStorageConfiguration implements
Serializable {
@IgniteExperimental
private long cdcWalDirMaxSize = DFLT_CDC_WAL_DIRECTORY_MAX_SIZE;
+
[ignite-3] branch main updated: IGNITE-19914 .NET: Fix colocation column order (#2308)
This is an automated email from the ASF dual-hosted git repository.
ptupitsyn pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/main by this push:
new 7e1e0cf74c IGNITE-19914 .NET: Fix colocation column order (#2308)
7e1e0cf74c is described below
commit 7e1e0cf74c741a8d9c7c9e678535a4ca04fb8dc9
Author: Pavel Tupitsyn
AuthorDate: Wed Jul 12 13:46:16 2023 +0300
IGNITE-19914 .NET: Fix colocation column order (#2308)
Colocation key column order can be different from the schema column order,
for example:
```sql
create table test(id integer, id0 bigint, id1 varchar, primary key(id,
id0)) colocate by (id1, id0)
```
Instead of computing the hash as we go in `BinaryTupleBuilder`, store
individual hashes in the right order into a preallocated area in the buffer and
combine them later.
---
.../SerializerHandlerBenchmarksBase.cs | 17 ++-
.../Proto/ColocationHashTests.cs | 61 --
.../Serialization/ObjectSerializerHandlerTests.cs | 15 ++-
.../Proto/BinaryTuple/BinaryTupleBuilder.cs| 125 +
.../BinaryTuple/IHashedColumnIndexProvider.cs | 11 +-
.../Apache.Ignite/Internal/Proto/HashUtils.cs | 2 +-
.../Apache.Ignite/Internal/Table/DataStreamer.cs | 2 +-
.../dotnet/Apache.Ignite/Internal/Table/Schema.cs | 9 +-
.../Serialization/IRecordSerializerHandler.cs | 2 +-
.../dotnet/Apache.Ignite/Internal/Table/Table.cs | 15 ++-
10 files changed, 184 insertions(+), 75 deletions(-)
diff --git
a/modules/platforms/dotnet/Apache.Ignite.Benchmarks/Table/Serialization/SerializerHandlerBenchmarksBase.cs
b/modules/platforms/dotnet/Apache.Ignite.Benchmarks/Table/Serialization/SerializerHandlerBenchmarksBase.cs
index 02bc0f63f1..bc94378f00 100644
---
a/modules/platforms/dotnet/Apache.Ignite.Benchmarks/Table/Serialization/SerializerHandlerBenchmarksBase.cs
+++
b/modules/platforms/dotnet/Apache.Ignite.Benchmarks/Table/Serialization/SerializerHandlerBenchmarksBase.cs
@@ -44,12 +44,17 @@ namespace Apache.Ignite.Benchmarks.Table.Serialization
[nameof(Car.Seats)] = Object.Seats
};
-internal static readonly Schema Schema = new(1, 1, 1, new[]
-{
-new Column(nameof(Car.Id), ColumnType.Uuid, IsNullable: false,
ColocationIndex: 0, IsKey: true, SchemaIndex: 0, Scale: 0, Precision: 0),
-new Column(nameof(Car.BodyType), ColumnType.String, IsNullable:
false, ColocationIndex: -1, IsKey: false, SchemaIndex: 1, Scale: 0, Precision:
0),
-new Column(nameof(Car.Seats), ColumnType.Int32, IsNullable: false,
ColocationIndex: -1, IsKey: false, SchemaIndex: 2, Scale: 0, Precision: 0)
-});
+internal static readonly Schema Schema = new(
+Version: 1,
+TableId: 1,
+KeyColumnCount: 1,
+ColocationColumnCount: 1,
+Columns: new[]
+{
+new Column(nameof(Car.Id), ColumnType.Uuid, IsNullable: false,
ColocationIndex: 0, IsKey: true, SchemaIndex: 0, Scale: 0, Precision: 0),
+new Column(nameof(Car.BodyType), ColumnType.String,
IsNullable: false, ColocationIndex: -1, IsKey: false, SchemaIndex: 1, Scale: 0,
Precision: 0),
+new Column(nameof(Car.Seats), ColumnType.Int32, IsNullable:
false, ColocationIndex: -1, IsKey: false, SchemaIndex: 2, Scale: 0, Precision:
0)
+});
internal static readonly byte[] SerializedData = GetSerializedData();
diff --git
a/modules/platforms/dotnet/Apache.Ignite.Tests/Proto/ColocationHashTests.cs
b/modules/platforms/dotnet/Apache.Ignite.Tests/Proto/ColocationHashTests.cs
index c63da30322..d0187bd90f 100644
--- a/modules/platforms/dotnet/Apache.Ignite.Tests/Proto/ColocationHashTests.cs
+++ b/modules/platforms/dotnet/Apache.Ignite.Tests/Proto/ColocationHashTests.cs
@@ -29,6 +29,7 @@ using Ignite.Compute;
using Ignite.Sql;
using Ignite.Table;
using Internal.Buffers;
+using Internal.Proto;
using Internal.Proto.BinaryTuple;
using Internal.Proto.MsgPack;
using Internal.Table;
@@ -41,7 +42,11 @@ using NUnit.Framework;
///
public class ColocationHashTests : IgniteTestsBase
{
-private const string ColocationHashJob =
"org.apache.ignite.internal.runner.app.PlatformTestNodeRunner$ColocationHashJob";
+private const string PlatformTestNodeRunner =
"org.apache.ignite.internal.runner.app.PlatformTestNodeRunner";
+
+private const string ColocationHashJob = PlatformTestNodeRunner +
"$ColocationHashJob";
+
+private const string TableRowColocationHashJob = PlatformTestNodeRunner +
"$TableRowColocationHashJob";
private static readonly object[] TestCases =
{
@@ -161,16 +166,56 @@ public class ColocationHashTests : IgniteTestsBase
}
}
+[Test]
+public async Task TestCustomColocationColumnOrder([Values(true, false)]
bool
[ignite-3] branch main updated: IGNITE-18959 Placement driver's local map cleanup on assingments removal (#2292)
This is an automated email from the ASF dual-hosted git repository.
sk0x50 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/main by this push:
new 1910afa0b1 IGNITE-18959 Placement driver's local map cleanup on
assingments removal (#2292)
1910afa0b1 is described below
commit 1910afa0b1aec6b0f97330532303f237f75db029
Author: Denis Chudov
AuthorDate: Wed Jul 12 13:38:09 2023 +0300
IGNITE-18959 Placement driver's local map cleanup on assingments removal
(#2292)
---
.../ignite/internal/metastorage/MetaStorageManager.java | 10 ++
.../internal/metastorage/impl/MetaStorageManagerImpl.java | 2 ++
.../internal/placementdriver/AssignmentsTracker.java | 2 +-
.../placementdriver/negotiation/LeaseNegotiator.java | 11 +--
.../java/org/apache/ignite/internal/app/IgniteImpl.java | 12 ++--
.../ignite/internal/table/distributed/TableManager.java | 15 +++
6 files changed, 39 insertions(+), 13 deletions(-)
diff --git
a/modules/metastorage-api/src/main/java/org/apache/ignite/internal/metastorage/MetaStorageManager.java
b/modules/metastorage-api/src/main/java/org/apache/ignite/internal/metastorage/MetaStorageManager.java
index fb37f7f1ab..54453fbff0 100644
---
a/modules/metastorage-api/src/main/java/org/apache/ignite/internal/metastorage/MetaStorageManager.java
+++
b/modules/metastorage-api/src/main/java/org/apache/ignite/internal/metastorage/MetaStorageManager.java
@@ -131,6 +131,16 @@ public interface MetaStorageManager extends
IgniteComponent {
*/
CompletableFuture putAll(Map vals);
+/**
+ * Removes an entry for the given key.
+ */
+CompletableFuture remove(ByteArray key);
+
+/**
+ * Removes entries for given keys.
+ */
+CompletableFuture removeAll(Set keys);
+
/**
* Retrieves entries for the given key prefix in lexicographic order.
Shortcut for {@link #prefix(ByteArray, long)} where
* {@code revUpperBound = LATEST_REVISION}.
diff --git
a/modules/metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/MetaStorageManagerImpl.java
b/modules/metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/MetaStorageManagerImpl.java
index 10ec576f1a..8d810e6044 100644
---
a/modules/metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/MetaStorageManagerImpl.java
+++
b/modules/metastorage/src/main/java/org/apache/ignite/internal/metastorage/impl/MetaStorageManagerImpl.java
@@ -638,6 +638,7 @@ public class MetaStorageManagerImpl implements
MetaStorageManager {
*
* @see MetaStorageService#remove(ByteArray)
*/
+@Override
public CompletableFuture remove(ByteArray key) {
if (!busyLock.enterBusy()) {
return CompletableFuture.failedFuture(new NodeStoppingException());
@@ -672,6 +673,7 @@ public class MetaStorageManagerImpl implements
MetaStorageManager {
*
* @see MetaStorageService#removeAll(Set)
*/
+@Override
public CompletableFuture removeAll(Set keys) {
if (!busyLock.enterBusy()) {
return CompletableFuture.failedFuture(new NodeStoppingException());
diff --git
a/modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/AssignmentsTracker.java
b/modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/AssignmentsTracker.java
index cc197a44a5..c8a3542421 100644
---
a/modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/AssignmentsTracker.java
+++
b/modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/AssignmentsTracker.java
@@ -144,7 +144,7 @@ public class AssignmentsTracker {
var replicationGrpId = TablePartitionId.fromString(
new String(evt.newEntry().key(),
StandardCharsets.UTF_8).replace(STABLE_ASSIGNMENTS_PREFIX, ""));
-if (evt.newEntry().empty()) {
+if (evt.newEntry().tombstone()) {
groupAssignments.remove(replicationGrpId);
} else {
Set prevAssignment =
groupAssignments.put(replicationGrpId,
ByteUtils.fromBytes(evt.newEntry().value()));
diff --git
a/modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/negotiation/LeaseNegotiator.java
b/modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/negotiation/LeaseNegotiator.java
index dbd3326fc5..ffcd828fa8 100644
---
a/modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/negotiation/LeaseNegotiator.java
+++
b/modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/negotiation/LeaseNegotiator.java
@@ -22,6 +22,8 @@ import static
org.apache.ignite.internal.placementdriver.negotiation.LeaseAgreem
import java.util.Map;
import
[ignite-3] branch catalog-feature updated (5b1b2d3b37 -> 4d4c9e8748)
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a change to branch catalog-feature
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
from 5b1b2d3b37 IGNITE-19932 Simplify handleChangePendingAssignmentEvent in
TableManager (#2299)
add 37a0ee3831 IGNITE-19865 Fix propagating trace identifier (#2291)
add d13553ebda IGNITE-19688 use waitAndGetLeader instead of waitLeader and
getLeader (#2259)
add 37651e2928 IGNITE-19212 ODBC: Implement basic query execution (#2284)
add 00181c783f IGNITE-19810 Remove excessive BinaryRow buffer copying
(#2295)
add 61864e4634 IGNITE-19946 Implement batch log updates for RAFT. (#2273)
add 87afd9d8c8 IGNITE-19205 ODBC data buffers (#2303)
add 8303cc4f02 IGNITE-19935 Allow combining individual colocation hashes
(#2302)
add 39a7c6800d IGNITE-19924 Test shouldn't guess id of newly created
catalog object (#2296)
add 4d4c9e8748 IGNITE-19778 Restore components state on metastorage
recovery (#2266)
No new revisions were added by this update.
Summary of changes:
.idea/inspectionProfiles/Project_Default.xml | 1 -
.../apache/ignite/lang/IgniteCheckedException.java | 9 +-
.../org/apache/ignite/lang/IgniteException.java| 9 +-
.../apache/ignite/lang/IgniteExceptionUtils.java | 6 +-
.../apache/ignite/lang/IgniteExceptionTest.java| 100 ++-
.../internal/binarytuple/BinaryTupleCommon.java| 6 +
.../internal/catalog/storage/UpdateLogImpl.java| 2 +-
.../internal/catalog/CatalogServiceSelfTest.java | 207 +++--
.../ignite/client/PartitionAwarenessTest.java | 359
.../configuration/ConfigurationChanger.java| 5 -
.../storage/ConfigurationStorage.java | 10 -
.../storage/TestConfigurationStorage.java | 5 -
.../org/apache/ignite/internal/util/ByteUtils.java | 30 -
.../ignite/internal/util/ExceptionUtils.java | 30 +
.../ignite/internal/util/HashCalculator.java | 206 -
.../org/apache/ignite/internal/util/HashUtils.java | 52 +-
.../java/org/apache/ignite/lang/ByteArray.java | 7 +
.../lang/IgniteInternalCheckedException.java | 9 +-
.../ignite/lang/IgniteInternalException.java | 9 +-
.../distributionzones/DistributionZoneManager.java | 2 +-
.../BaseDistributionZoneManagerTest.java | 2 +-
.../internal/metastorage/MetaStorageManager.java | 23 +-
.../impl/ItMetaStorageManagerImplTest.java | 71 --
...MetaStorageSafeTimePropagationAbstractTest.java | 2 +-
.../metastorage/impl/ItMetaStorageWatchTest.java | 6 +-
.../metastorage/impl/MetaStorageManagerImpl.java | 40 +-
.../server/persistence/RocksDbKeyValueStorage.java | 6 +-
.../server/BasicOperationsKeyValueStorageTest.java | 6 +-
.../server/SimpleInMemoryKeyValueStorage.java | 6 +-
.../pagememory/util/PartitionlessLinks.java| 2 +-
modules/platforms/cpp/ignite/client/CMakeLists.txt | 4 +
.../cpp/ignite/client/detail/cluster_connection.h | 2 +-
.../cpp/ignite/client/detail/node_connection.cpp | 18 +-
.../cpp/ignite/client/detail/node_connection.h | 2 +-
.../cpp/ignite/client/detail/sql/result_set_impl.h | 10 +-
.../platforms/cpp/ignite/client/detail/utils.cpp | 105 +--
.../platforms/cpp/ignite/common/big_decimal.cpp| 8 +-
modules/platforms/cpp/ignite/common/big_decimal.h | 11 +-
modules/platforms/cpp/ignite/common/big_integer.h | 2 +-
modules/platforms/cpp/ignite/network/data_buffer.h | 3 +
modules/platforms/cpp/ignite/odbc/CMakeLists.txt | 2 +
.../ignite/odbc/app/application_data_buffer.cpp| 441 -
.../cpp/ignite/odbc/app/application_data_buffer.h | 17 +-
.../odbc/app/application_data_buffer_test.cpp | 988 +
.../platforms/cpp/ignite/odbc/app/parameter.cpp| 226 -
modules/platforms/cpp/ignite/odbc/app/parameter.h | 33 +-
.../cpp/ignite/odbc/app/parameter_set.cpp | 50 +-
.../platforms/cpp/ignite/odbc/app/parameter_set.h | 8 +
.../cpp/ignite/odbc/config/configuration.cpp | 12 +
.../cpp/ignite/odbc/config/configuration.h | 19 +-
.../cpp/ignite/odbc/diagnostic/diagnosable.h | 28 +
.../ignite/odbc/diagnostic/diagnosable_adapter.cpp | 1 +
.../platforms/cpp/ignite/odbc/meta/column_meta.cpp | 15 +-
.../platforms/cpp/ignite/odbc/meta/column_meta.h | 11 +-
.../query/cursor.h}| 69 +-
.../platforms/cpp/ignite/odbc/query/data_query.cpp | 360
.../platforms/cpp/ignite/odbc/query/data_query.h | 212 +
.../platforms/cpp/ignite/odbc/sql_connection.cpp | 71 +-
modules/platforms/cpp/ignite/odbc/sql_connection.h | 97 ++
.../platforms/cpp/ignite/odbc/sql_statement.cpp| 12 +-
modules/platforms/cpp/ignite/odbc/utility.cpp | 3 +
.../{client/detail => protocol}/client_operation.h | 0
modules/platforms/cpp/ignite/protocol/utils.h | 26 +
[ignite-3] branch ignite-19927 updated (e72cc68840 -> af8b39593e)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19927 in repository https://gitbox.apache.org/repos/asf/ignite-3.git omit e72cc68840 Styles. omit 91973f02c4 Improve test coverage for CREATE TABLE. add 96aaa46976 IGNITE-19609 Java client: Add data streamer metrics (#2298) add 5b1b2d3b37 IGNITE-19932 Simplify handleChangePendingAssignmentEvent in TableManager (#2299) add 37a0ee3831 IGNITE-19865 Fix propagating trace identifier (#2291) add d13553ebda IGNITE-19688 use waitAndGetLeader instead of waitLeader and getLeader (#2259) add 37651e2928 IGNITE-19212 ODBC: Implement basic query execution (#2284) add 00181c783f IGNITE-19810 Remove excessive BinaryRow buffer copying (#2295) add 61864e4634 IGNITE-19946 Implement batch log updates for RAFT. (#2273) add 87afd9d8c8 IGNITE-19205 ODBC data buffers (#2303) add 8303cc4f02 IGNITE-19935 Allow combining individual colocation hashes (#2302) add 39a7c6800d IGNITE-19924 Test shouldn't guess id of newly created catalog object (#2296) add 4d4c9e8748 IGNITE-19778 Restore components state on metastorage recovery (#2266) add af8b39593e Improve test coverage for CREATE TABLE. This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (e72cc68840) \ N -- N -- N refs/heads/ignite-19927 (af8b39593e) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .idea/inspectionProfiles/Project_Default.xml | 1 - .../apache/ignite/lang/IgniteCheckedException.java | 9 +- .../org/apache/ignite/lang/IgniteException.java| 9 +- .../apache/ignite/lang/IgniteExceptionUtils.java | 6 +- .../apache/ignite/lang/IgniteExceptionTest.java| 100 ++- .../internal/binarytuple/BinaryTupleCommon.java| 6 + .../internal/catalog/CatalogServiceImpl.java | 17 +- .../internal/catalog/storage/UpdateLogImpl.java| 2 +- .../internal/catalog/CatalogServiceSelfTest.java | 304 --- .../ignite/internal/client/ClientMetricSource.java | 107 ++- .../ignite/internal/client/ReliableChannel.java| 9 + .../internal/client/table/ClientDataStreamer.java | 5 +- .../client/table/ClientKeyValueBinaryView.java | 1 + .../apache/ignite/client/ClientMetricsTest.java| 45 + .../ignite/client/PartitionAwarenessTest.java | 359 .../java/org/apache/ignite/client/TestServer.java | 12 + .../configuration/ConfigurationChanger.java| 5 - .../storage/ConfigurationStorage.java | 10 - .../storage/TestConfigurationStorage.java | 5 - .../internal/streamer/StreamerMetricSink.java} | 32 +- .../internal/streamer/StreamerSubscriber.java | 39 +- .../org/apache/ignite/internal/util/ByteUtils.java | 30 - .../ignite/internal/util/ExceptionUtils.java | 30 + .../ignite/internal/util/HashCalculator.java | 206 - .../org/apache/ignite/internal/util/HashUtils.java | 52 +- .../java/org/apache/ignite/lang/ByteArray.java | 7 + .../lang/IgniteInternalCheckedException.java | 9 +- .../ignite/lang/IgniteInternalException.java | 9 +- .../distributionzones/DistributionZoneManager.java | 2 +- .../BaseDistributionZoneManagerTest.java | 2 +- .../internal/metastorage/MetaStorageManager.java | 23 +- .../impl/ItMetaStorageManagerImplTest.java | 71 -- ...MetaStorageSafeTimePropagationAbstractTest.java | 2 +- .../metastorage/impl/ItMetaStorageWatchTest.java | 6 +- .../metastorage/impl/MetaStorageManagerImpl.java | 40 +- .../server/persistence/RocksDbKeyValueStorage.java | 6 +- .../server/BasicOperationsKeyValueStorageTest.java | 6 +- .../server/SimpleInMemoryKeyValueStorage.java | 6 +- .../pagememory/util/PartitionlessLinks.java| 2 +- modules/platforms/cpp/ignite/client/CMakeLists.txt | 4 + .../cpp/ignite/client/detail/cluster_connection.h | 2 +- .../cpp/ignite/client/detail/node_connection.cpp | 18 +- .../cpp/ignite/client/detail/node_connection.h | 2 +- .../cpp/ignite/client/detail/sql/result_set_impl.h | 10 +- .../platforms/cpp/ignite/client/detail/utils.cpp | 105 +-- .../platforms/cpp/ignite/common/big_decimal.cpp| 8 +- modules/platforms/cpp/ignite/common/big_decimal.h | 11 +- modules/platforms/cpp/ignite/common/big_integer.h | 2 +-
[ignite-3] branch ignite-19499 updated (303bb29ed4 -> 337217dcd5)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19499 in repository https://gitbox.apache.org/repos/asf/ignite-3.git omit 303bb29ed4 Mirror Index changes from Config to Catalog omit 9f9e2d1be0 Fix invalid default type scale/precision. Fix tests. omit 3f351f50a3 Mirror Table changes from Config to Catalog. omit 25386d11a9 Mirror DistributionZone changes from Config to Catalog. omit e5745d9ed8 Test shouldn't guess id of newly created catalog object. omit a2ea7876ed Improve test coverage for CREATE TABLE. add 37651e2928 IGNITE-19212 ODBC: Implement basic query execution (#2284) add 00181c783f IGNITE-19810 Remove excessive BinaryRow buffer copying (#2295) add 61864e4634 IGNITE-19946 Implement batch log updates for RAFT. (#2273) add 87afd9d8c8 IGNITE-19205 ODBC data buffers (#2303) add 8303cc4f02 IGNITE-19935 Allow combining individual colocation hashes (#2302) add 39a7c6800d IGNITE-19924 Test shouldn't guess id of newly created catalog object (#2296) add 4d4c9e8748 IGNITE-19778 Restore components state on metastorage recovery (#2266) add 2670bf4669 Improve test coverage for CREATE TABLE. add b8fb21adef Mirror DistributionZone changes from Config to Catalog. add 1c026b5863 Mirror Table changes from Config to Catalog. add f2007fff43 Fix invalid default type scale/precision. Fix tests. add 337217dcd5 Mirror Index changes from Config to Catalog This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (303bb29ed4) \ N -- N -- N refs/heads/ignite-19499 (337217dcd5) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .idea/inspectionProfiles/Project_Default.xml | 1 - .../internal/binarytuple/BinaryTupleCommon.java| 6 + .../apache/ignite/internal/catalog/Catalog.java| 31 - .../internal/catalog/CatalogServiceImpl.java | 13 +- .../internal/catalog/commands/CatalogUtils.java| 36 +- .../internal/catalog/storage/UpdateLogImpl.java| 2 +- .../internal/catalog/CatalogServiceSelfTest.java | 25 +- .../ignite/client/PartitionAwarenessTest.java | 359 .../configuration/ConfigurationChanger.java| 5 - .../storage/ConfigurationStorage.java | 10 - .../storage/TestConfigurationStorage.java | 5 - .../org/apache/ignite/internal/util/ByteUtils.java | 30 - .../ignite/internal/util/HashCalculator.java | 206 - .../org/apache/ignite/internal/util/HashUtils.java | 52 +- .../java/org/apache/ignite/lang/ByteArray.java | 7 + .../distributionzones/DistributionZoneManager.java | 2 +- .../BaseDistributionZoneManagerTest.java | 2 +- .../internal/metastorage/MetaStorageManager.java | 23 +- .../impl/ItMetaStorageManagerImplTest.java | 71 -- ...MetaStorageSafeTimePropagationAbstractTest.java | 2 +- .../metastorage/impl/ItMetaStorageWatchTest.java | 6 +- .../metastorage/impl/MetaStorageManagerImpl.java | 40 +- .../server/persistence/RocksDbKeyValueStorage.java | 6 +- .../server/BasicOperationsKeyValueStorageTest.java | 6 +- .../server/SimpleInMemoryKeyValueStorage.java | 6 +- .../pagememory/util/PartitionlessLinks.java| 2 +- modules/platforms/cpp/ignite/client/CMakeLists.txt | 4 + .../cpp/ignite/client/detail/cluster_connection.h | 2 +- .../cpp/ignite/client/detail/node_connection.cpp | 18 +- .../cpp/ignite/client/detail/node_connection.h | 2 +- .../cpp/ignite/client/detail/sql/result_set_impl.h | 10 +- .../platforms/cpp/ignite/client/detail/utils.cpp | 105 +-- .../platforms/cpp/ignite/common/big_decimal.cpp| 8 +- modules/platforms/cpp/ignite/common/big_decimal.h | 11 +- modules/platforms/cpp/ignite/common/big_integer.h | 2 +- modules/platforms/cpp/ignite/network/data_buffer.h | 3 + modules/platforms/cpp/ignite/odbc/CMakeLists.txt | 2 + .../ignite/odbc/app/application_data_buffer.cpp| 441 - .../cpp/ignite/odbc/app/application_data_buffer.h | 17 +- .../odbc/app/application_data_buffer_test.cpp | 988 + .../platforms/cpp/ignite/odbc/app/parameter.cpp| 226 - modules/platforms/cpp/ignite/odbc/app/parameter.h | 33 +- .../cpp/ignite/odbc/app/parameter_set.cpp | 50 +- .../platforms/cpp/ignite/odbc/app/parameter_set.h | 8 +
[ignite-3] branch ignite-19903 updated (c178849ed3 -> b3e97592ae)
This is an automated email from the ASF dual-hosted git repository. sdanilov pushed a change to branch ignite-19903 in repository https://gitbox.apache.org/repos/asf/ignite-3.git omit c178849ed3 . omit 30bfabe82a . omit 0a9b08e837 . omit db2de8f949 . omit 2a1245a125 . omit f84f2ce38e . omit 4f847a5718 . omit 116b7e6ada . omit de59bc7fd4 IGNITE-19903 Fix recovery descriptor race condition add 8586df07f8 IGNITE-19578 Decrease count of lease messages to meta storage (#2141) add 96aaa46976 IGNITE-19609 Java client: Add data streamer metrics (#2298) add 5b1b2d3b37 IGNITE-19932 Simplify handleChangePendingAssignmentEvent in TableManager (#2299) add 37a0ee3831 IGNITE-19865 Fix propagating trace identifier (#2291) add d13553ebda IGNITE-19688 use waitAndGetLeader instead of waitLeader and getLeader (#2259) add 37651e2928 IGNITE-19212 ODBC: Implement basic query execution (#2284) add 00181c783f IGNITE-19810 Remove excessive BinaryRow buffer copying (#2295) add 61864e4634 IGNITE-19946 Implement batch log updates for RAFT. (#2273) add 87afd9d8c8 IGNITE-19205 ODBC data buffers (#2303) add 8303cc4f02 IGNITE-19935 Allow combining individual colocation hashes (#2302) add 39a7c6800d IGNITE-19924 Test shouldn't guess id of newly created catalog object (#2296) add 4d4c9e8748 IGNITE-19778 Restore components state on metastorage recovery (#2266) add 3ee17160cc IGNITE-19903 Fix recovery descriptor race condition add caa26c28d7 . add 9144b2dded . add 4c74b624fe . add 4d2cb4b2ba . add 90d0db9f3c . add f9cf3c443b . add b779f2f975 . add 1eb013 . add b3e97592ae . This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (c178849ed3) \ N -- N -- N refs/heads/ignite-19903 (b3e97592ae) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .idea/inspectionProfiles/Project_Default.xml | 1 - .../apache/ignite/lang/IgniteCheckedException.java | 9 +- .../org/apache/ignite/lang/IgniteException.java| 9 +- .../apache/ignite/lang/IgniteExceptionUtils.java | 6 +- .../apache/ignite/lang/IgniteExceptionTest.java| 100 ++- .../internal/binarytuple/BinaryTupleCommon.java| 6 + .../internal/catalog/storage/UpdateLogImpl.java| 2 +- .../internal/catalog/CatalogServiceSelfTest.java | 207 +++-- .../ignite/internal/client/ClientMetricSource.java | 107 ++- .../ignite/internal/client/ReliableChannel.java| 9 + .../internal/client/table/ClientDataStreamer.java | 5 +- .../client/table/ClientKeyValueBinaryView.java | 1 + .../apache/ignite/client/ClientMetricsTest.java| 45 + .../ignite/client/PartitionAwarenessTest.java | 359 .../java/org/apache/ignite/client/TestServer.java | 12 + .../configuration/ConfigurationChanger.java| 5 - .../storage/ConfigurationStorage.java | 10 - .../storage/TestConfigurationStorage.java | 5 - .../internal/streamer/StreamerMetricSink.java} | 32 +- .../internal/streamer/StreamerSubscriber.java | 39 +- .../org/apache/ignite/internal/util/ByteUtils.java | 30 - .../ignite/internal/util/ExceptionUtils.java | 30 + .../ignite/internal/util/HashCalculator.java | 206 - .../org/apache/ignite/internal/util/HashUtils.java | 52 +- .../apache/ignite/internal/util/IgniteUtils.java | 53 ++ .../java/org/apache/ignite/lang/ByteArray.java | 7 + .../lang/IgniteInternalCheckedException.java | 9 +- .../ignite/lang/IgniteInternalException.java | 9 +- .../distributionzones/DistributionZoneManager.java | 2 +- .../BaseDistributionZoneManagerTest.java | 2 +- .../internal/metastorage/MetaStorageManager.java | 23 +- .../impl/ItMetaStorageManagerImplTest.java | 71 -- ...MetaStorageSafeTimePropagationAbstractTest.java | 2 +- .../metastorage/impl/ItMetaStorageWatchTest.java | 6 +- .../metastorage/impl/MetaStorageManagerImpl.java | 40 +- .../server/persistence/RocksDbKeyValueStorage.java | 6 +- .../server/BasicOperationsKeyValueStorageTest.java | 6 +- .../server/SimpleInMemoryKeyValueStorage.java | 6 +- .../recovery/RecoveryClientHandshakeManager.java | 2 +- .../network/recovery/RecoveryDescriptor.java | 20 +- .../recovery/RecoveryServerHandshakeManager.java | 2 +-
[ignite-3] branch main updated: IGNITE-19778 Restore components state on metastorage recovery (#2266)
This is an automated email from the ASF dual-hosted git repository.
sdanilov pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/main by this push:
new 4d4c9e8748 IGNITE-19778 Restore components state on metastorage
recovery (#2266)
4d4c9e8748 is described below
commit 4d4c9e8748bd16e079757cc4f4beac6b8977e3c0
Author: Semyon Danilov
AuthorDate: Wed Jul 12 12:03:53 2023 +0400
IGNITE-19778 Restore components state on metastorage recovery (#2266)
---
.../internal/catalog/storage/UpdateLogImpl.java| 2 +-
.../configuration/ConfigurationChanger.java| 5 -
.../storage/ConfigurationStorage.java | 10 -
.../storage/TestConfigurationStorage.java | 5 -
.../java/org/apache/ignite/lang/ByteArray.java | 7 +
.../distributionzones/DistributionZoneManager.java | 2 +-
.../BaseDistributionZoneManagerTest.java | 2 +-
.../internal/metastorage/MetaStorageManager.java | 23 +-
.../impl/ItMetaStorageManagerImplTest.java | 71 --
...MetaStorageSafeTimePropagationAbstractTest.java | 2 +-
.../metastorage/impl/ItMetaStorageWatchTest.java | 6 +-
.../metastorage/impl/MetaStorageManagerImpl.java | 40 ++--
.../server/persistence/RocksDbKeyValueStorage.java | 6 +-
.../server/BasicOperationsKeyValueStorageTest.java | 6 +-
.../server/SimpleInMemoryKeyValueStorage.java | 6 +-
.../ignite/internal/BaseIgniteRestartTest.java | 66 +-
.../ItDistributedConfigurationPropertiesTest.java | 4 +-
.../ItDistributedConfigurationStorageTest.java | 4 +-
.../storage/ItRebalanceDistributedTest.java| 68 +++---
.../zones/ItDistributionZonesFilterTest.java | 7 +
...niteDistributionZoneManagerNodeRestartTest.java | 5 +-
.../runner/app/ItIgniteNodeRestartTest.java| 9 +-
.../org/apache/ignite/internal/app/IgniteImpl.java | 37 +---
.../storage/DistributedConfigurationStorage.java | 134 +--
.../storage/LocalConfigurationStorage.java | 5 -
.../storage/LocalFileConfigurationStorage.java | 5 -
.../recovery/ConfigurationCatchUpListener.java | 117 --
.../recovery/RecoveryCompletionFutureFactory.java | 51 -
.../DistributedConfigurationCatchUpTest.java | 244 -
.../DistributedConfigurationStorageTest.java | 8 +-
.../internal/table/distributed/TableManager.java | 12 +-
31 files changed, 189 insertions(+), 780 deletions(-)
diff --git
a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLogImpl.java
b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLogImpl.java
index f627c51118..1b7994fabc 100644
---
a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLogImpl.java
+++
b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLogImpl.java
@@ -157,7 +157,7 @@ public class UpdateLogImpl implements UpdateLog {
// TODO: IGNITE-19790 Read range from metastore
while (true) {
ByteArray key = CatalogKey.update(ver++);
-Entry entry = metastore.getLocally(key.bytes(), appliedRevision);
+Entry entry = metastore.getLocally(key, appliedRevision);
if (entry.empty() || entry.tombstone()) {
break;
diff --git
a/modules/configuration/src/main/java/org/apache/ignite/internal/configuration/ConfigurationChanger.java
b/modules/configuration/src/main/java/org/apache/ignite/internal/configuration/ConfigurationChanger.java
index 5a01225087..6b115f5995 100644
---
a/modules/configuration/src/main/java/org/apache/ignite/internal/configuration/ConfigurationChanger.java
+++
b/modules/configuration/src/main/java/org/apache/ignite/internal/configuration/ConfigurationChanger.java
@@ -644,11 +644,6 @@ public abstract class ConfigurationChanger implements
DynamicConfigurationChange
rwLock.writeLock().unlock();
}
-// Save revisions for recovery.
-// We execute synchronously to avoid a race between
notifications about updating the Meta Storage and updating the revision
-// of the Meta Storage.
-storage.writeConfigurationRevision(oldStorageRoots.version,
newStorageRoots.version);
-
long notificationNumber =
notificationListenerCnt.incrementAndGet();
CompletableFuture notificationFuture;
diff --git
a/modules/configuration/src/main/java/org/apache/ignite/internal/configuration/storage/ConfigurationStorage.java
b/modules/configuration/src/main/java/org/apache/ignite/internal/configuration/storage/ConfigurationStorage.java
index 7315dfd51b..d9663eb284 100644
---
a/modules/configuration/src/main/java/org/apache/ignite/internal/configuration/storage/ConfigurationStorage.java
+++
