[ignite-3] branch ignite-19499 updated (94b7d17687 -> 09b385aec1)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19499 in repository https://gitbox.apache.org/repos/asf/ignite-3.git omit 94b7d17687 Switch other managers to Catalog events. omit d375832959 Switch TableManager to Catalog events. add 307aa7529d Get rid of DdlCommandHandlerWrapper. Get rid of unused methods. Fix styles. add 1f89c85941 Switch TableManager to Catalog events. add 09b385aec1 Switch other managers to Catalog events. This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (94b7d17687) \ N -- N -- N refs/heads/ignite-19499 (09b385aec1) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../apache/ignite/internal/index/IndexManager.java | 123 ++- .../ignite/internal/index/IndexManagerTest.java| 3 +- .../ignite/internal/index/ItIndexManagerTest.java | 6 +- .../ignite/internal/schema/SchemaManager.java | 111 ++ .../internal/sql/engine/SqlQueryProcessor.java | 4 +- .../sql/engine/exec/ddl/DdlCommandHandler.java | 402 ++--- .../engine/exec/ddl/DdlCommandHandlerWrapper.java | 100 - .../DdlCommandHandlerExceptionHandlingTest.java| 2 +- .../ddl/DistributionZoneDdlCommandHandlerTest.java | 6 +- .../internal/table/distributed/TableManager.java | 67 +--- .../table/distributed/TableManagerTest.java| 3 - 11 files changed, 113 insertions(+), 714 deletions(-) delete mode 100644 modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
[ignite-3] branch ignite-19942 updated (3d1e168975 -> 307aa7529d)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19942 in repository https://gitbox.apache.org/repos/asf/ignite-3.git from 3d1e168975 Styles. add 307aa7529d Get rid of DdlCommandHandlerWrapper. Get rid of unused methods. Fix styles. No new revisions were added by this update. Summary of changes: .../apache/ignite/internal/index/IndexManager.java | 126 ++- .../ignite/internal/index/IndexManagerTest.java| 3 +- .../ignite/internal/index/ItIndexManagerTest.java | 6 +- .../internal/sql/engine/SqlQueryProcessor.java | 4 +- .../sql/engine/exec/ddl/DdlCommandHandler.java | 402 ++--- .../engine/exec/ddl/DdlCommandHandlerWrapper.java | 100 - .../DdlCommandHandlerExceptionHandlingTest.java| 2 +- .../ddl/DistributionZoneDdlCommandHandlerTest.java | 6 +- .../internal/table/distributed/TableManager.java | 41 +-- 9 files changed, 60 insertions(+), 630 deletions(-) delete mode 100644 modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
[ignite-3] branch ignite-19499 updated (ce3dc7abf0 -> 94b7d17687)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19499 in repository https://gitbox.apache.org/repos/asf/ignite-3.git discard ce3dc7abf0 wip. discard 210d54afba wip discard 67bdc86dcf wip add d375832959 Switch TableManager to Catalog events. add 94b7d17687 Switch other managers to Catalog events. This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (ce3dc7abf0) \ N -- N -- N refs/heads/ignite-19499 (94b7d17687) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../internal/catalog/CatalogServiceImpl.java | 15 +++ .../distributionzones/DistributionZoneManager.java | 2 +- .../apache/ignite/internal/index/IndexManager.java | 50 -- .../internal/schema/CatalogDescriptorUtils.java| 3 ++ .../ignite/internal/schema/SchemaManager.java | 6 +-- .../internal/table/distributed/TableManager.java | 36 6 files changed, 58 insertions(+), 54 deletions(-)
[ignite-3] branch ignite-19499 updated (337217dcd5 -> ce3dc7abf0)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19499 in repository https://gitbox.apache.org/repos/asf/ignite-3.git omit 337217dcd5 Mirror Index changes from Config to Catalog omit f2007fff43 Fix invalid default type scale/precision. Fix tests. omit 1c026b5863 Mirror Table changes from Config to Catalog. omit b8fb21adef Mirror DistributionZone changes from Config to Catalog. omit 2670bf4669 Improve test coverage for CREATE TABLE. add 1910afa0b1 IGNITE-18959 Placement driver's local map cleanup on assingments removal (#2292) add 7e1e0cf74c IGNITE-19914 .NET: Fix colocation column order (#2308) add d305b4f153 IGNITE-19801 Move ConfigurationStorageRevisionListener to test fixtures (#2307) add f8a280e514 IGNITE-19954 .NET: Work around tx conflicts in TestAutoFlushFrequency (#2309) add 0a6e79264b IGNITE-19948 Add authenticated username to the prompt (#2304) add 7a5fb6bc54 IGNITE-19941 Sql. Test ExecutionServiceImplTest.testCancelOnInitiator is flaky (#2305) add e741560fd3 IGNITE-19927 Improve test coverage for CREATE TABLE (#2297) add fef7a24c2a IGNITE-19812 Do not swallow exceptions during start of a partition (#2240) add 415b262b75 IGNITE-19859 Fix testAutoFlushByTimer flakiness (#2313) add 48ca4d4632 IGNITE-19963 Move completion of ClockWaiter futures to a special thread pool (#2310) add 527b7b27d5 IGNITE-19903 Fix recovery descriptor race condition (#2283) add 5e12216c50 Mirror DistributionZone changes from Config to Catalog. add 4bb103e854 Mirror Table changes from Config to Catalog. add bdffc3478b Mirror Index changes from Config to Catalog add 64785bcd2e minor. add 0b275e9c6a fix tests add 9a2248aa21 Styles. add 3d1e168975 Styles. add 67bdc86dcf wip add 210d54afba wip add ce3dc7abf0 wip. This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (337217dcd5) \ N -- N -- N refs/heads/ignite-19499 (ce3dc7abf0) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../internal/catalog/CatalogServiceImpl.java | 29 ++- .../ignite/internal/catalog/ClockWaiter.java | 95 ++--- .../internal/catalog/commands/CatalogUtils.java| 1 + .../internal/catalog/CatalogServiceSelfTest.java | 39 ++-- .../ignite/internal/catalog/ClockWaiterTest.java | 3 +- .../ItConnectToClusterTestBase.java| 10 +- ...tConnectWithBasicAuthenticationCommandTest.java | 102 + .../commands/questions/ItConnectToClusterTest.java | 1 + .../questions/ItConnectToSslClusterTest.java | 1 + .../repl/executor/ItIgnitePicocliCommandsTest.java | 2 +- .../internal/cli/call/connect/ConnectCall.java | 38 +++- .../handler/IgniteCliApiExceptionHandler.java | 8 +- .../ignite/internal/cli/core/repl/SessionInfo.java | 9 +- .../cli/core/repl/prompt/ReplPromptProvider.java | 13 +- .../internal/cli/core/rest/ApiClientFactory.java | 39 +++- .../core/repl/SessionDefaultValueProviderTest.java | 2 +- .../filter/DynamicCompleterFilterTest.java | 3 +- .../configuration/ConfigurationChanger.java| 69 +++ .../configuration/ConfigurationRegistry.java | 57 + .../storage/ConfigurationStorageListener.java | 9 +- .../notifications/ConfigurationListenerTest.java | 105 +- .../configuration/TestConfigurationChanger.java| 5 - .../ConfigurationStorageRevisionListener.java | 1 - ...ConfigurationStorageRevisionListenerHolder.java | 0 .../ignite/internal/future/OrderingFuture.java | 63 +- .../ignite/internal/hlc/ClockUpdateListener.java | 12 +- .../ignite/internal/hlc/HybridClockImpl.java | 24 ++- .../apache/ignite/internal/HybridClockTest.java| 8 +- .../apache/ignite/internal/TestHybridClock.java| 2 - .../CompletableFutureCompletedMatcher.java | 61 ++ .../distributionzones/DistributionZoneManager.java | 171 --- .../apache/ignite/internal/index/IndexManager.java | 229 ++--- .../ignite/internal/index/IndexManagerTest.java| 8 +- .../internal/metastorage/MetaStorageManager.java | 10 + .../ignite/internal/metastorage/WatchListener.java | 11 - .../metastorage/impl/MetaStorageManagerImpl.java | 2 +
[ignite-3] branch ignite-19942 updated (9a2248aa21 -> 3d1e168975)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19942 in repository https://gitbox.apache.org/repos/asf/ignite-3.git from 9a2248aa21 Styles. add 3d1e168975 Styles. No new revisions were added by this update. Summary of changes: .../distributionzones/DistributionZoneManager.java | 8 +-- .../apache/ignite/internal/index/IndexManager.java | 78 -- .../ignite/internal/index/IndexManagerTest.java| 8 +-- .../ignite/internal/index/ItIndexManagerTest.java | 2 +- .../engine/exec/ddl/DdlCommandHandlerWrapper.java | 18 + 5 files changed, 34 insertions(+), 80 deletions(-)
[ignite-3] branch ignite-19942 updated (0b275e9c6a -> 9a2248aa21)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch ignite-19942 in repository https://gitbox.apache.org/repos/asf/ignite-3.git from 0b275e9c6a fix tests add 9a2248aa21 Styles. No new revisions were added by this update. Summary of changes: .../distributionzones/DistributionZoneManager.java | 175 - 1 file changed, 98 insertions(+), 77 deletions(-)
[ignite-3] 03/05: Mirror Index changes from Config to Catalog
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19942
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit bdffc3478b3659163f05c77cc4dacd22b0e62e35
Author: amashenkov
AuthorDate: Tue Jul 11 19:30:31 2023 +0300
Mirror Index changes from Config to Catalog
---
.../internal/catalog/commands/CatalogUtils.java| 1 -
.../apache/ignite/internal/index/IndexManager.java | 140 -
.../ignite/internal/index/IndexManagerTest.java| 90 +
.../ignite/internal/index/ItIndexManagerTest.java | 23 ++--
.../runner/app/ItIgniteNodeRestartTest.java| 8 +-
.../org/apache/ignite/internal/app/IgniteImpl.java | 2 +-
.../sql/engine/exec/ddl/DdlCommandHandler.java | 2 +-
.../engine/exec/ddl/DdlCommandHandlerWrapper.java | 37 +++---
.../sql/engine/exec/MockedStructuresTest.java | 17 ++-
.../internal/table/distributed/TableManager.java | 8 +-
.../table/distributed/TableManagerTest.java| 11 +-
11 files changed, 272 insertions(+), 67 deletions(-)
diff --git
a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
index d157cbaba2..7d53598e23 100644
---
a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
+++
b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
@@ -208,5 +208,4 @@ public class CatalogUtils {
return 0;
}
}
-
}
diff --git
a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
index a9bbdf6c45..f7d4377ce8 100644
---
a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
+++
b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
@@ -36,6 +36,10 @@ import java.util.function.Function;
import org.apache.ignite.configuration.NamedListView;
import
org.apache.ignite.configuration.notifications.ConfigurationNamedListListener;
import
org.apache.ignite.configuration.notifications.ConfigurationNotificationEvent;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
+import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
+import org.apache.ignite.internal.catalog.commands.DropIndexParams;
import org.apache.ignite.internal.catalog.descriptors.CatalogColumnCollation;
import
org.apache.ignite.internal.catalog.descriptors.CatalogHashIndexDescriptor;
import
org.apache.ignite.internal.catalog.descriptors.CatalogIndexColumnDescriptor;
@@ -59,6 +63,7 @@ import
org.apache.ignite.internal.schema.configuration.TableView;
import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
import org.apache.ignite.internal.schema.configuration.TablesView;
import org.apache.ignite.internal.schema.configuration.index.HashIndexChange;
+import org.apache.ignite.internal.schema.configuration.index.SortedIndexChange;
import org.apache.ignite.internal.schema.configuration.index.TableIndexChange;
import
org.apache.ignite.internal.schema.configuration.index.TableIndexConfiguration;
import org.apache.ignite.internal.schema.configuration.index.TableIndexView;
@@ -73,6 +78,7 @@ import org.apache.ignite.internal.util.IgniteSpinBusyLock;
import org.apache.ignite.internal.util.StringUtils;
import org.apache.ignite.lang.ErrorGroups;
import org.apache.ignite.lang.ErrorGroups.Common;
+import org.apache.ignite.lang.IgniteException;
import org.apache.ignite.lang.IgniteInternalException;
import org.apache.ignite.lang.IndexAlreadyExistsException;
import org.apache.ignite.lang.IndexNotFoundException;
@@ -96,6 +102,9 @@ public class IndexManager extends Producer imp
/** Table manager. */
private final TableManager tableManager;
+/** Catalog manager. */
+private final CatalogManager catalogManager;
+
/** Busy lock to stop synchronously. */
private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
@@ -106,17 +115,20 @@ public class IndexManager extends Producer imp
* Constructor.
*
* @param tablesCfg Tables and indexes configuration.
+ * @param catalogManager Catalog manager.
* @param schemaManager Schema manager.
* @param tableManager Table manager.
*/
public IndexManager(
TablesConfiguration tablesCfg,
+CatalogManager catalogManager,
SchemaManager schemaManager,
TableManager tableManager
) {
this.tablesCfg = Objects.requireNonNull(tablesCfg, "tablesCfg");
this.schemaManager = Objects.requireNonNull(schemaManager,
"schemaManager");
this.tableManager = tableManager;
+
[ignite-3] 01/05: Mirror DistributionZone changes from Config to Catalog.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19942
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 5e12216c50751de41e261d9aba76393c3b0f02ec
Author: amashenkov
AuthorDate: Tue Jul 4 23:29:33 2023 +0300
Mirror DistributionZone changes from Config to Catalog.
---
.../internal/catalog/CatalogServiceImpl.java | 6 +-
.../catalog/descriptors/CatalogZoneDescriptor.java | 3 +
modules/distribution-zones/build.gradle| 2 +
.../distributionzones/DistributionZoneManager.java | 185 +
.../BaseDistributionZoneManagerTest.java | 1 +
...ibutionZoneManagerConfigurationChangesTest.java | 1 +
.../DistributionZoneManagerTest.java | 1 +
.../DistributionZoneMockTest.java | 2 +
.../DistributionZonesTestUtil.java | 34
.../storage/ItRebalanceDistributedTest.java| 1 +
...niteDistributionZoneManagerNodeRestartTest.java | 2 +
.../runner/app/ItIgniteNodeRestartTest.java| 15 +-
.../org/apache/ignite/internal/app/IgniteImpl.java | 21 +--
.../engine/exec/ddl/DdlCommandHandlerWrapper.java | 34
.../DdlCommandHandlerExceptionHandlingTest.java| 28 ++--
15 files changed, 196 insertions(+), 140 deletions(-)
diff --git
a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java
b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java
index a8e3e30258..09fad23701 100644
---
a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java
+++
b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogServiceImpl.java
@@ -150,18 +150,18 @@ public class CatalogServiceImpl extends
Producer fut = new CompletableFuture<>();
-int[] zoneIdContainer = new int[1];
-
-zonesConfiguration.change(zonesChange ->
zonesChange.changeDistributionZones(zonesListChange -> {
-try {
-zonesListChange.create(distributionZoneCfg.name(),
zoneChange -> {
-if (distributionZoneCfg.partitions() != null) {
-
zoneChange.changePartitions(distributionZoneCfg.partitions());
-}
-
-if (distributionZoneCfg.dataStorageChangeConsumer() ==
null) {
-zoneChange.changeDataStorage(ch ->
ch.convert(zonesConfiguration.defaultDataStorage().value()));
-} else {
-
zoneChange.changeDataStorage(distributionZoneCfg.dataStorageChangeConsumer());
-}
-
-if (distributionZoneCfg.replicas() != null) {
-
zoneChange.changeReplicas(distributionZoneCfg.replicas());
-}
-
-if (distributionZoneCfg.filter() != null) {
-
zoneChange.changeFilter(distributionZoneCfg.filter());
-}
-
-if (distributionZoneCfg.dataNodesAutoAdjust() != null)
{
-
zoneChange.changeDataNodesAutoAdjust(distributionZoneCfg.dataNodesAutoAdjust());
-}
-
-if (distributionZoneCfg.dataNodesAutoAdjustScaleUp()
== null) {
-if (distributionZoneCfg.dataNodesAutoAdjust() !=
null) {
-
zoneChange.changeDataNodesAutoAdjustScaleUp(INFINITE_TIMER_VALUE);
-}
+catalogManager.createDistributionZone(CreateZoneParams.builder()
+.zoneName(distributionZoneCfg.name())
+.partitions(distributionZoneCfg.partitions())
+.filter(distributionZoneCfg.filter())
+.replicas(distributionZoneCfg.replicas())
+
.dataNodesAutoAdjust(distributionZoneCfg.dataNodesAutoAdjust())
+
.dataNodesAutoAdjustScaleUp(distributionZoneCfg.dataNodesAutoAdjustScaleUp())
+
.dataNodesAutoAdjustScaleDown(distributionZoneCfg.dataNodesAutoAdjustScaleDown())
+.build())
+.thenApply(ignore ->
catalogManager.zone(distributionZoneCfg.name(), Long.MAX_VALUE).id())
+.thenCompose(zoneId ->
+zonesConfiguration.change(zonesChange ->
zonesChange.changeDistributionZones(zonesListChange -> {
+try {
+
zonesListChange.create(distributionZoneCfg.name(), zoneChange -> {
+if (distributionZoneCfg.partitions()
!= null) {
+
zoneChange.changePartitions(distributionZoneCfg.partitions());
+
[ignite-3] 02/05: Mirror Table changes from Config to Catalog.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19942
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 4bb103e854b486a6fe59e137bea9bde13092a218
Author: amashenkov
AuthorDate: Fri Jul 7 15:40:08 2023 +0300
Mirror Table changes from Config to Catalog.
---
.../internal/catalog/commands/CatalogUtils.java| 69 ++-
.../internal/catalog/commands/ColumnParams.java| 6 +-
.../internal/catalog/CatalogServiceSelfTest.java | 2 +-
.../storage/ItRebalanceDistributedTest.java| 32 ++--
.../inmemory/ItRaftStorageVolatilityTest.java | 12 +-
.../raftsnapshot/ItTableRaftSnapshotsTest.java | 2 +-
.../runner/app/ItIgniteNodeRestartTest.java| 5 +-
.../runner/app/ItSchemaChangeKvViewTest.java | 2 +
.../runner/app/ItSchemaChangeTableViewTest.java| 1 +
.../runner/app/ItTableApiContractTest.java | 124 ++--
.../internal/runner/app/ItTablesApiTest.java | 56 +++---
.../runner/app/PlatformTestNodeRunner.java | 18 +-
.../ignite/internal/sql/api/ItCommonApiTest.java | 6 +-
.../internal/sql/api/ItSqlAsynchronousApiTest.java | 22 +--
.../internal/sql/api/ItSqlSynchronousApiTest.java | 24 +--
.../internal/sql/engine/ItCreateTableDdlTest.java | 30 +--
.../ignite/internal/table/ItRoReadsTest.java | 24 ++-
.../org/apache/ignite/internal/app/IgniteImpl.java | 1 +
modules/schema/build.gradle| 1 +
.../internal/schema/CatalogDescriptorUtils.java| 111 +++
.../testutils/SchemaToCatalogParamsConverter.java | 208 +
.../sql/engine/exec/ddl/DdlCommandHandler.java | 2 +-
.../engine/exec/ddl/DdlCommandHandlerWrapper.java | 42 ++---
.../exec/ddl/DdlToCatalogCommandConverter.java | 4 +
.../sql/engine/exec/MockedStructuresTest.java | 8 +
.../internal/table/distributed/TableManager.java | 204 +++-
.../table/distributed/TableManagerTest.java| 72 ---
27 files changed, 813 insertions(+), 275 deletions(-)
diff --git
a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
index 7823b09d47..d157cbaba2 100644
---
a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
+++
b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
@@ -38,6 +38,31 @@ import org.apache.ignite.sql.ColumnType;
* Catalog utils.
*/
public class CatalogUtils {
+/**
+ * Default TIMESTAMP type precision: microseconds.
+ *
+ * SQL`16 part 2 section 6.1 syntax rule 36
+ */
+public static final int DEFAULT_TIMESTAMP_PRECISION = 6;
+/**
+ * Default TIME type precision: seconds.
+ *
+ * SQL`16 part 2 section 6.1 syntax rule 36
+ */
+public static final int DEFAULT_TIME_PRECISION = 0;
+/**
+ * Default DECIMAL precision is implementation-defined.
+ *
+ * SQL`16 part 2 section 6.1 syntax rule 25
+ */
+public static final int DEFAULT_DECIMAL_PRECISION = 19;
+/**
+ * Default DECIMAL scale is implementation-defined.
+ *
+ * SQL`16 part 2 section 6.1 syntax rule 25
+ */
+public static final int DEFAULT_DECIMAL_SCALE = 3;
+
private static final Map>
ALTER_COLUMN_TYPE_TRANSITIONS = new EnumMap<>(ColumnType.class);
static {
@@ -125,9 +150,9 @@ public class CatalogUtils {
* @return Column descriptor.
*/
public static CatalogTableColumnDescriptor fromParams(ColumnParams params)
{
-int precision = params.precision() != null ? params.precision() : 0;
-int scale = params.scale() != null ? params.scale() : 0;
-int length = params.length() != null ? params.length() : 0;
+int precision = params.precision() != null ? params.precision() :
defaultPrecision(params.type());
+int scale = params.scale() != null ? params.scale() :
defaultScale(params.type());
+int length = params.length() != null ? params.length() :
defaultLength(params.type());
DefaultValue defaultValue = params.defaultValueDefinition();
return new CatalogTableColumnDescriptor(params.name(), params.type(),
params.nullable(),
@@ -146,4 +171,42 @@ public class CatalogUtils {
return supportedTransitions != null &&
supportedTransitions.contains(target);
}
+
+private static int defaultPrecision(ColumnType columnType) {
+//TODO IGNITE-19938: Add REAL,FLOAT and DOUBLE precision. See SQL`16
part 2 section 6.1 syntax rule 29-31
+switch (columnType) {
+case NUMBER:
+case DECIMAL:
+return DEFAULT_DECIMAL_PRECISION;
+case TIME:
+return DEFAULT_TIME_PRECISION;
+case TIMESTAMP:
+case DATETIME:
+
[ignite-3] 05/05: fix tests
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a commit to branch ignite-19942 in repository https://gitbox.apache.org/repos/asf/ignite-3.git commit 0b275e9c6a66fce4f1a7e148ff6bec97c2403004 Author: amashenkov AuthorDate: Wed Jul 12 19:46:26 2023 +0300 fix tests --- .../src/main/java/org/apache/ignite/internal/index/IndexManager.java | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java index f7d4377ce8..2944ef3f6e 100644 --- a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java +++ b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java @@ -154,11 +154,10 @@ public class IndexManager extends Producer imp .toArray(String[]::new); String pkName = table.name() + "_PK"; - -int pkIndexId = catalogManager.index(pkName, Long.MAX_VALUE).id(); +int pkId = table.tableId() + 1; // Update config bypassing the Catalog, because PK has just been created in Catalog with the table. -return createIndexInternal(pkIndexId, "PUBLIC", pkName, table.name(), false, +return createIndexInternal(pkId, "PUBLIC", pkName, table.name(), false, change -> change.changeUniq(true).convert(HashIndexChange.class) .changeColumnNames(pkColumns) );
[ignite-3] 04/05: minor.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19942
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 64785bcd2e7606a866d4926920c961a29f2d9554
Author: amashenkov
AuthorDate: Wed Jul 12 18:17:29 2023 +0300
minor.
---
.../internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java | 1 -
1 file changed, 1 deletion(-)
diff --git
a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java
b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java
index 1100d7ec1d..3e958d982b 100644
---
a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java
+++
b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java
@@ -212,7 +212,6 @@ class DdlToCatalogCommandConverter {
}
private static ColumnParams convert(ColumnDefinition def) {
-RelDataType type = def.type();
return ColumnParams.builder()
.name(def.name())
.type(TypeUtils.columnType(def.type()))
[ignite-3] branch ignite-19942 updated (8a39a398e6 -> 0b275e9c6a)
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a change to branch ignite-19942
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
omit 8a39a398e6 fix tests
omit beadc36d2b minor.
omit 8a7a6ab93e Mirror Index changes from Config to Catalog
omit 9bddcb851e Mirror Table changes from Config to Catalog.
omit 2e08b78f77 Mirror DistributionZone changes from Config to Catalog.
add fef7a24c2a IGNITE-19812 Do not swallow exceptions during start of a
partition (#2240)
add 415b262b75 IGNITE-19859 Fix testAutoFlushByTimer flakiness (#2313)
add 48ca4d4632 IGNITE-19963 Move completion of ClockWaiter futures to a
special thread pool (#2310)
add 527b7b27d5 IGNITE-19903 Fix recovery descriptor race condition (#2283)
new 5e12216c50 Mirror DistributionZone changes from Config to Catalog.
new 4bb103e854 Mirror Table changes from Config to Catalog.
new bdffc3478b Mirror Index changes from Config to Catalog
new 64785bcd2e minor.
new 0b275e9c6a fix tests
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8a39a398e6)
\
N -- N -- N refs/heads/ignite-19942 (0b275e9c6a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../ignite/internal/catalog/ClockWaiter.java | 95 +
.../ignite/internal/catalog/ClockWaiterTest.java | 3 +-
.../ignite/internal/future/OrderingFuture.java | 63 +-
.../ignite/internal/hlc/ClockUpdateListener.java | 12 ++-
.../ignite/internal/hlc/HybridClockImpl.java | 24 --
.../apache/ignite/internal/HybridClockTest.java| 8 +-
.../apache/ignite/internal/TestHybridClock.java| 2 -
.../CompletableFutureCompletedMatcher.java | 61 ++
.../network/netty/ItConnectionManagerTest.java | 97 ++
.../scalecube/ItScaleCubeNetworkMessagingTest.java | 37 +++--
...ion.java => ChannelAlreadyExistsException.java} | 27 +++---
.../network/handshake/HandshakeException.java | 4 +-
.../network/netty/ChannelCreationListener.java}| 14 ++--
.../internal/network/netty/ConnectionManager.java | 50 +--
.../netty/DefaultRecoveryDescriptorProvider.java | 14 +---
.../network/netty/InboundRecoveryHandler.java | 8 ++
.../ignite/internal/network/netty/NettySender.java | 9 +-
.../ignite/internal/network/netty/NettyServer.java | 8 --
.../network/recovery/HandshakeTieBreaker.java | 40 +
.../recovery/RecoveryClientHandshakeManager.java | 92 +++-
.../network/recovery/RecoveryDescriptor.java | 46 ++
.../recovery/RecoveryDescriptorProvider.java | 3 +-
.../recovery/RecoveryServerHandshakeManager.java | 96 -
.../ignite/network/DefaultMessagingService.java| 26 +-
.../internal/network/netty/NettyServerTest.java| 8 +-
.../network/netty/RecoveryHandshakeTest.java | 53 +++-
.../network/DefaultMessagingServiceTest.java | 3 +-
.../placementdriver/leases/LeaseTracker.java | 1 -
.../runner/app/ItIgniteNodeRestartTest.java| 16 ++--
.../streamer/ItAbstractDataStreamerTest.java | 16 ++--
.../internal/table/distributed/TableManager.java | 26 +++---
31 files changed, 722 insertions(+), 240 deletions(-)
create mode 100644
modules/core/src/testFixtures/java/org/apache/ignite/internal/testframework/matchers/CompletableFutureCompletedMatcher.java
copy
modules/network/src/main/java/org/apache/ignite/internal/network/handshake/{HandshakeException.java
=> ChannelAlreadyExistsException.java} (61%)
copy
modules/{cli/src/main/java/org/apache/ignite/internal/cli/config/ConfigManagerProvider.java
=>
network/src/main/java/org/apache/ignite/internal/network/netty/ChannelCreationListener.java}
(76%)
create mode 100644
modules/network/src/main/java/org/apache/ignite/internal/network/recovery/HandshakeTieBreaker.java
[ignite-3] branch catalog-feature updated (4d4c9e8748 -> 527b7b27d5)
This is an automated email from the ASF dual-hosted git repository. amashenkov pushed a change to branch catalog-feature in repository https://gitbox.apache.org/repos/asf/ignite-3.git from 4d4c9e8748 IGNITE-19778 Restore components state on metastorage recovery (#2266) add 1910afa0b1 IGNITE-18959 Placement driver's local map cleanup on assingments removal (#2292) add 7e1e0cf74c IGNITE-19914 .NET: Fix colocation column order (#2308) add d305b4f153 IGNITE-19801 Move ConfigurationStorageRevisionListener to test fixtures (#2307) add f8a280e514 IGNITE-19954 .NET: Work around tx conflicts in TestAutoFlushFrequency (#2309) add 0a6e79264b IGNITE-19948 Add authenticated username to the prompt (#2304) add 7a5fb6bc54 IGNITE-19941 Sql. Test ExecutionServiceImplTest.testCancelOnInitiator is flaky (#2305) add e741560fd3 IGNITE-19927 Improve test coverage for CREATE TABLE (#2297) add fef7a24c2a IGNITE-19812 Do not swallow exceptions during start of a partition (#2240) add 415b262b75 IGNITE-19859 Fix testAutoFlushByTimer flakiness (#2313) add 48ca4d4632 IGNITE-19963 Move completion of ClockWaiter futures to a special thread pool (#2310) add 527b7b27d5 IGNITE-19903 Fix recovery descriptor race condition (#2283) No new revisions were added by this update. Summary of changes: .../internal/catalog/CatalogServiceImpl.java | 66 ++- .../ignite/internal/catalog/ClockWaiter.java | 95 + .../internal/catalog/CatalogServiceSelfTest.java | 220 ++--- .../ignite/internal/catalog/ClockWaiterTest.java | 3 +- .../ItConnectToClusterTestBase.java| 10 +- ...tConnectWithBasicAuthenticationCommandTest.java | 102 ++ .../commands/questions/ItConnectToClusterTest.java | 1 + .../questions/ItConnectToSslClusterTest.java | 1 + .../repl/executor/ItIgnitePicocliCommandsTest.java | 2 +- .../internal/cli/call/connect/ConnectCall.java | 38 +++- .../handler/IgniteCliApiExceptionHandler.java | 8 +- .../ignite/internal/cli/core/repl/SessionInfo.java | 9 +- .../cli/core/repl/prompt/ReplPromptProvider.java | 13 +- .../internal/cli/core/rest/ApiClientFactory.java | 39 +++- .../core/repl/SessionDefaultValueProviderTest.java | 2 +- .../filter/DynamicCompleterFilterTest.java | 3 +- .../configuration/ConfigurationChanger.java| 69 +++ .../configuration/ConfigurationRegistry.java | 57 +- .../storage/ConfigurationStorageListener.java | 9 +- .../notifications/ConfigurationListenerTest.java | 105 +- .../configuration/TestConfigurationChanger.java| 5 - .../ConfigurationStorageRevisionListener.java | 1 - ...ConfigurationStorageRevisionListenerHolder.java | 0 .../ignite/internal/future/OrderingFuture.java | 63 +- .../ignite/internal/hlc/ClockUpdateListener.java | 12 +- .../ignite/internal/hlc/HybridClockImpl.java | 24 ++- .../apache/ignite/internal/HybridClockTest.java| 8 +- .../apache/ignite/internal/TestHybridClock.java| 2 - .../CompletableFutureCompletedMatcher.java | 61 ++ .../internal/metastorage/MetaStorageManager.java | 10 + .../ignite/internal/metastorage/WatchListener.java | 11 -- .../metastorage/impl/MetaStorageManagerImpl.java | 2 + .../ignite/internal/metastorage/server/Watch.java | 9 - .../metastorage/server/WatchProcessorTest.java | 5 - .../network/netty/ItConnectionManagerTest.java | 97 + .../scalecube/ItScaleCubeNetworkMessagingTest.java | 37 +++- ...ion.java => ChannelAlreadyExistsException.java} | 27 ++- .../network/handshake/HandshakeException.java | 4 +- .../network/netty/ChannelCreationListener.java}| 14 +- .../internal/network/netty/ConnectionManager.java | 50 ++--- .../netty/DefaultRecoveryDescriptorProvider.java | 14 +- .../network/netty/InboundRecoveryHandler.java | 8 + .../ignite/internal/network/netty/NettySender.java | 9 +- .../ignite/internal/network/netty/NettyServer.java | 8 - .../network/recovery/HandshakeTieBreaker.java | 40 .../recovery/RecoveryClientHandshakeManager.java | 92 ++--- .../network/recovery/RecoveryDescriptor.java | 46 + .../recovery/RecoveryDescriptorProvider.java | 3 +- .../recovery/RecoveryServerHandshakeManager.java | 96 ++--- .../ignite/network/DefaultMessagingService.java| 26 ++- .../internal/network/netty/NettyServerTest.java| 8 +- .../network/netty/RecoveryHandshakeTest.java | 53 ++--- .../network/DefaultMessagingServiceTest.java | 3 +- .../placementdriver/AssignmentsTracker.java| 2 +- .../placementdriver/leases/LeaseTracker.java | 1 - .../negotiation/LeaseNegotiator.java | 11 +- .../SerializerHandlerBenchmarksBase.cs | 17 +- .../dotnet/Apache.Ignite.Tests/MetricsTests.cs | 2 +-
[ignite] branch IGNITE-17700__realtime_cdc updated: IGNITE-19675 [IEP-104] Implement WALIterator over ByteBuffer (#10798)
This is an automated email from the ASF dual-hosted git repository.
timoninmaxim pushed a commit to branch IGNITE-17700__realtime_cdc
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/IGNITE-17700__realtime_cdc by
this push:
new 9b8cf4f1ec0 IGNITE-19675 [IEP-104] Implement WALIterator over
ByteBuffer (#10798)
9b8cf4f1ec0 is described below
commit 9b8cf4f1ec0fd2764a9e43a6aabe783782e54180
Author: yurinaryshkin <135707807+yurinarysh...@users.noreply.github.com>
AuthorDate: Thu Jul 13 15:19:18 2023 +0300
IGNITE-19675 [IEP-104] Implement WALIterator over ByteBuffer (#10798)
---
.../pagemem/wal/record/FilteredRecord.java | 5 +-
...or.java => AbstractFileWalRecordsIterator.java} | 47 +-
.../wal/AbstractWalRecordsIteratorAdapter.java | 78 +++
.../persistence/wal/ByteBufferBackedDataInput.java | 10 +
.../wal/ByteBufferBackedDataInputImpl.java | 14 +
.../persistence/wal/ByteBufferWalIterator.java | 132 +
.../persistence/wal/Crc32CheckingDataInput.java| 200 +++
.../cache/persistence/wal/FileDescriptor.java | 2 +-
.../persistence/wal/FileWriteAheadLogManager.java | 4 +-
.../wal/SingleSegmentLogicalRecordsIterator.java | 2 +-
.../cache/persistence/wal/io/FileInput.java| 221 +--
.../cache/persistence/wal/io/SimpleFileInput.java | 8 -
.../wal/reader/StandaloneWalRecordsIterator.java | 4 +-
.../wal/serializer/RecordSerializer.java | 4 +-
.../wal/serializer/RecordV1Serializer.java | 11 +-
.../wal/serializer/RecordV2Serializer.java | 3 +-
.../main/resources/META-INF/classnames.properties | 4 +-
.../db/wal/crc/IgniteDataIntegrityTests.java | 4 +-
.../persistence/wal/ByteBufferWalIteratorTest.java | 639 +
.../ignite/testsuites/IgnitePdsTestSuite5.java | 3 +
20 files changed, 1101 insertions(+), 294 deletions(-)
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/FilteredRecord.java
b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/FilteredRecord.java
index e077e5accc2..915353ebf2a 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/FilteredRecord.java
+++
b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/FilteredRecord.java
@@ -16,12 +16,9 @@
*/
package org.apache.ignite.internal.pagemem.wal.record;
-import
org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator;
-
/**
* Special type of WAL record. Shouldn't be stored in file.
- * Returned by deserializer if next record is not matched by filter.
Automatically handled by
- * {@link AbstractWalRecordsIterator}.
+ * Returned by deserializer if next record is not matched by filter.
*/
public class FilteredRecord extends WALRecord {
/** Instance. */
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/AbstractWalRecordsIterator.java
b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/AbstractFileWalRecordsIterator.java
similarity index 91%
rename from
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/AbstractWalRecordsIterator.java
rename to
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/AbstractFileWalRecordsIterator.java
index 926bce8fe39..4dcb55ed368 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/AbstractWalRecordsIterator.java
+++
b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/AbstractFileWalRecordsIterator.java
@@ -25,7 +25,6 @@ import java.nio.ByteOrder;
import java.util.Optional;
import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.IgniteLogger;
-import org.apache.ignite.internal.pagemem.wal.WALIterator;
import org.apache.ignite.internal.pagemem.wal.record.WALRecord;
import org.apache.ignite.internal.processors.cache.GridCacheSharedContext;
import org.apache.ignite.internal.processors.cache.persistence.file.FileIO;
@@ -36,7 +35,6 @@ import
org.apache.ignite.internal.processors.cache.persistence.wal.io.SegmentIO;
import
org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordSerializer;
import
org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordSerializerFactory;
import
org.apache.ignite.internal.processors.cache.persistence.wal.serializer.SegmentHeader;
-import org.apache.ignite.internal.util.GridCloseableIteratorAdapter;
import org.apache.ignite.internal.util.typedef.P2;
import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.lang.IgniteBiTuple;
@@ -49,22 +47,10 @@ import static
org.apache.ignite.internal.processors.cache.persistence.wal.serial
* Iterator over WAL segments. This abstract class
[ignite-3] branch ignite-15655-new-network deleted (was c754fcd31e)
This is an automated email from the ASF dual-hosted git repository. sdanilov pushed a change to branch ignite-15655-new-network in repository https://gitbox.apache.org/repos/asf/ignite-3.git was c754fcd31e IGNITE-15655 remove wrong changes This change permanently discards the following revisions: discard c754fcd31e IGNITE-15655 remove wrong changes discard b006a55b1f IGNITE-15655 enable all tests
[ignite-3] branch ignite-19778 deleted (was bbed151017)
This is an automated email from the ASF dual-hosted git repository. sdanilov pushed a change to branch ignite-19778 in repository https://gitbox.apache.org/repos/asf/ignite-3.git was bbed151017 Add todo The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-3] branch ignite-19903 deleted (was d9290c8083)
This is an automated email from the ASF dual-hosted git repository. sdanilov pushed a change to branch ignite-19903 in repository https://gitbox.apache.org/repos/asf/ignite-3.git was d9290c8083 Review fixes The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[ignite-3] branch main updated: IGNITE-19903 Fix recovery descriptor race condition (#2283)
This is an automated email from the ASF dual-hosted git repository.
sdanilov pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/main by this push:
new 527b7b27d5 IGNITE-19903 Fix recovery descriptor race condition (#2283)
527b7b27d5 is described below
commit 527b7b27d5cfeb705ff38a0f73fae0a6500ffcb7
Author: Semyon Danilov
AuthorDate: Thu Jul 13 15:50:43 2023 +0400
IGNITE-19903 Fix recovery descriptor race condition (#2283)
---
.../ignite/internal/future/OrderingFuture.java | 63 +-
.../CompletableFutureCompletedMatcher.java | 61 ++
.../network/netty/ItConnectionManagerTest.java | 97 ++
.../scalecube/ItScaleCubeNetworkMessagingTest.java | 37 +++--
...ion.java => ChannelAlreadyExistsException.java} | 27 +++---
.../network/handshake/HandshakeException.java | 4 +-
.../ChannelCreationListener.java} | 26 ++
.../internal/network/netty/ConnectionManager.java | 50 +--
.../netty/DefaultRecoveryDescriptorProvider.java | 14 +---
.../network/netty/InboundRecoveryHandler.java | 8 ++
.../ignite/internal/network/netty/NettySender.java | 9 +-
.../ignite/internal/network/netty/NettyServer.java | 8 --
.../network/recovery/HandshakeTieBreaker.java | 40 +
.../recovery/RecoveryClientHandshakeManager.java | 92 +++-
.../network/recovery/RecoveryDescriptor.java | 46 ++
.../recovery/RecoveryDescriptorProvider.java | 3 +-
.../recovery/RecoveryServerHandshakeManager.java | 96 -
.../ignite/network/DefaultMessagingService.java| 26 +-
.../internal/network/netty/NettyServerTest.java| 8 +-
.../network/netty/RecoveryHandshakeTest.java | 53 +++-
.../network/DefaultMessagingServiceTest.java | 3 +-
.../placementdriver/leases/LeaseTracker.java | 1 -
22 files changed, 592 insertions(+), 180 deletions(-)
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/future/OrderingFuture.java
b/modules/core/src/main/java/org/apache/ignite/internal/future/OrderingFuture.java
index 6aa95c04ff..40b7821f07 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/future/OrderingFuture.java
+++
b/modules/core/src/main/java/org/apache/ignite/internal/future/OrderingFuture.java
@@ -29,6 +29,7 @@ import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.function.BiConsumer;
+import java.util.function.BiFunction;
import java.util.function.Function;
import org.jetbrains.annotations.Nullable;
@@ -299,8 +300,8 @@ public class OrderingFuture {
private static void acceptQuietly(BiConsumer action, T result, Throwable ex) {
try {
action.accept(result, ex);
-} catch (Exception e) {
-// ignore
+} catch (Exception ignored) {
+// No-op.
}
}
@@ -337,6 +338,41 @@ public class OrderingFuture {
}
}
+/**
+ * Adds a mapping function that gets executed as soon as this future gets
completed for any reason. The function will accept both result
+ * and exception and return a future with the result of the function's
execution.
+ *
+ * @param mapper The function to use to compute the value of the returned
OrderingFuture.
+ * @return The new OrderingFuture.
+ * @see CompletableFuture#handle(BiFunction)
+ */
+public OrderingFuture handle(BiFunction mapper) {
+Handle dependent = null;
+
+while (true) {
+State prevState = state;
+
+if (prevState.completionQueueProcessed()) {
+try {
+U mappingResult = mapper.apply(prevState.result,
prevState.exception);
+
+return completedFuture(mappingResult);
+} catch (Throwable t) {
+return failedFuture(t);
+}
+}
+
+if (dependent == null) {
+dependent = new Handle<>(new OrderingFuture<>(), mapper);
+}
+State newState = prevState.enqueueDependent(dependent);
+
+if (replaceState(prevState, newState)) {
+return dependent.resultFuture;
+}
+}
+}
+
private static CompletionException wrapWithCompletionException(Throwable
ex) {
return ex instanceof CompletionException ? (CompletionException) ex :
new CompletionException(ex);
}
@@ -463,6 +499,25 @@ public class OrderingFuture {
}
}
+private static class Handle implements DependentAction {
+private final OrderingFuture resultFuture;
+private final BiFunction action;
+
+private Handle(OrderingFuture resultFuture, BiFunction action) {
+this.resultFuture = resultFuture;
+
[ignite] branch IGNITE-17700__realtime_cdc updated: IGNITE-19639 Add realtime CDC records (#10840)
This is an automated email from the ASF dual-hosted git repository.
timoninmaxim pushed a commit to branch IGNITE-17700__realtime_cdc
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/IGNITE-17700__realtime_cdc by
this push:
new 213d0951c14 IGNITE-19639 Add realtime CDC records (#10840)
213d0951c14 is described below
commit 213d0951c14344629223b16e6e114067fbfddf78
Author: Maksim Timonin
AuthorDate: Thu Jul 13 14:12:30 2023 +0300
IGNITE-19639 Add realtime CDC records (#10840)
---
.../wal/record/RealtimeCdcRecord.java} | 21 ++---
.../wal/record/RealtimeCdcStopRecord.java} | 21 ++---
.../internal/pagemem/wal/record/WALRecord.java | 8 +-
.../cache/persistence/cdc/CdcBuffer.java | 16 +++-
.../cache/persistence/cdc/CdcBufferConsumer.java | 3 +-
.../cache/persistence/cdc/CdcProcessor.java| 9 +--
.../cache/persistence/cdc/CdcWorker.java | 55 ++---
.../wal/serializer/RecordDataV1Serializer.java | 16
.../persistence/cdc/RealtimeCdcBufferTest.java | 90 +++---
9 files changed, 178 insertions(+), 61 deletions(-)
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/cdc/CdcBufferConsumer.java
b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/RealtimeCdcRecord.java
similarity index 69%
copy from
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/cdc/CdcBufferConsumer.java
copy to
modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/RealtimeCdcRecord.java
index 67cd422fb0f..5a9e68067ab 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/cdc/CdcBufferConsumer.java
+++
b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/RealtimeCdcRecord.java
@@ -15,19 +15,12 @@
* limitations under the License.
*/
-package org.apache.ignite.internal.processors.cache.persistence.cdc;
+package org.apache.ignite.internal.pagemem.wal.record;
-import java.nio.ByteBuffer;
-
-/** Mock for Realtime CDC buffer consumer. */
-public interface CdcBufferConsumer {
-/**
- * Consumes raw WAL data.
- *
- * @param data Raw data to consume.
- */
-public void consume(ByteBuffer data);
-
-/** */
-public void close();
+/** */
+public class RealtimeCdcRecord extends WALRecord {
+/** {@inheritDoc} */
+@Override public RecordType type() {
+return RecordType.REALTIME_CDC_RECORD;
+}
}
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/cdc/CdcBufferConsumer.java
b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/RealtimeCdcStopRecord.java
similarity index 69%
copy from
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/cdc/CdcBufferConsumer.java
copy to
modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/RealtimeCdcStopRecord.java
index 67cd422fb0f..5b89bcd564a 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/cdc/CdcBufferConsumer.java
+++
b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/RealtimeCdcStopRecord.java
@@ -15,19 +15,12 @@
* limitations under the License.
*/
-package org.apache.ignite.internal.processors.cache.persistence.cdc;
+package org.apache.ignite.internal.pagemem.wal.record;
-import java.nio.ByteBuffer;
-
-/** Mock for Realtime CDC buffer consumer. */
-public interface CdcBufferConsumer {
-/**
- * Consumes raw WAL data.
- *
- * @param data Raw data to consume.
- */
-public void consume(ByteBuffer data);
-
-/** */
-public void close();
+/** */
+public class RealtimeCdcStopRecord extends WALRecord {
+/** {@inheritDoc} */
+@Override public RecordType type() {
+return RecordType.REALTIME_STOP_CDC_RECORD;
+}
}
diff --git
a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/WALRecord.java
b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/WALRecord.java
index 3457251e0f4..02f4125215e 100644
---
a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/WALRecord.java
+++
b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/WALRecord.java
@@ -291,7 +291,13 @@ public abstract class WALRecord {
INCREMENTAL_SNAPSHOT_FINISH_RECORD(77, LOGICAL),
/** CDC data record. */
-CDC_DATA_RECORD(78, CUSTOM);
+CDC_DATA_RECORD(78, CUSTOM),
+
+/** Realtime CDC record. */
+REALTIME_CDC_RECORD(79, CUSTOM),
+
+/** Realtime CDC record. */
+REALTIME_STOP_CDC_RECORD(80, CUSTOM);
/** Index for serialization. Should be consistent throughout all
versions. */
private final int idx;
diff --git
[ignite-3] branch main updated: IGNITE-19963 Move completion of ClockWaiter futures to a special thread pool (#2310)
This is an automated email from the ASF dual-hosted git repository.
sdanilov pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/main by this push:
new 48ca4d4632 IGNITE-19963 Move completion of ClockWaiter futures to a
special thread pool (#2310)
48ca4d4632 is described below
commit 48ca4d4632d8dc2d55fbde4bbfa7a6c52a841b30
Author: Roman Puchkovskiy
AuthorDate: Thu Jul 13 14:00:32 2023 +0400
IGNITE-19963 Move completion of ClockWaiter futures to a special thread
pool (#2310)
---
.../ignite/internal/catalog/ClockWaiter.java | 95 ++
.../ignite/internal/catalog/ClockWaiterTest.java | 3 +-
.../ignite/internal/hlc/ClockUpdateListener.java | 12 ++-
.../ignite/internal/hlc/HybridClockImpl.java | 24 --
.../apache/ignite/internal/HybridClockTest.java| 8 +-
.../apache/ignite/internal/TestHybridClock.java| 2 -
6 files changed, 96 insertions(+), 48 deletions(-)
diff --git
a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/ClockWaiter.java
b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/ClockWaiter.java
index 8c5c65df22..7141237fbd 100644
---
a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/ClockWaiter.java
+++
b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/ClockWaiter.java
@@ -17,14 +17,18 @@
package org.apache.ignite.internal.catalog;
+import static java.util.concurrent.CompletableFuture.completedFuture;
import static java.util.concurrent.CompletableFuture.failedFuture;
import java.util.concurrent.CancellationException;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionException;
+import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
+import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import org.apache.ignite.internal.hlc.ClockUpdateListener;
@@ -35,6 +39,7 @@ import org.apache.ignite.internal.logger.Loggers;
import org.apache.ignite.internal.manager.IgniteComponent;
import org.apache.ignite.internal.thread.NamedThreadFactory;
import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.internal.util.IgniteUtils;
import org.apache.ignite.internal.util.PendingComparableValuesTracker;
import org.apache.ignite.internal.util.TrackerClosedException;
import org.apache.ignite.lang.NodeStoppingException;
@@ -59,18 +64,39 @@ public class ClockWaiter implements IgniteComponent {
private final ClockUpdateListener updateListener = this::onUpdate;
+private final Runnable triggerClockUpdate = this::triggerTrackerUpdate;
+
+/** Executor on which short-lived tasks are scheduled that are needed to
timely complete awaiting futures. */
private volatile ScheduledExecutorService scheduler;
+/** Executor that executes completion of futures returned to the user, so
it might take arbitrarily heavy operations. */
+private final ExecutorService futureExecutor;
+
+/**
+ * Creates a new {@link ClockWaiter}.
+ *
+ * @param nodeName Name of the current Ignite node.
+ * @param clock Clock to look at.
+ */
public ClockWaiter(String nodeName, HybridClock clock) {
this.nodeName = nodeName;
this.clock = clock;
+
+futureExecutor = new ThreadPoolExecutor(
+0,
+4,
+1,
+TimeUnit.MINUTES,
+new LinkedBlockingQueue<>(),
+new NamedThreadFactory(nodeName +
"-clock-waiter-future-executor", LOG)
+);
}
@Override
public void start() {
clock.addUpdateListener(updateListener);
-scheduler = Executors.newSingleThreadScheduledExecutor(new
NamedThreadFactory(nodeName + "-clock-waiter", LOG));
+scheduler = Executors.newSingleThreadScheduledExecutor(new
NamedThreadFactory(nodeName + "-clock-waiter-scheduler", LOG));
}
@Override
@@ -91,6 +117,9 @@ public class ClockWaiter implements IgniteComponent {
// user-facing futures we return from the tracker), but we don't need
them for anything else,
// so it's simpler to just use shutdownNow().
scheduler.shutdownNow();
+
+IgniteUtils.shutdownAndAwaitTermination(futureExecutor, 10,
TimeUnit.SECONDS);
+
scheduler.awaitTermination(10, TimeUnit.SECONDS);
}
@@ -109,6 +138,9 @@ public class ClockWaiter implements IgniteComponent {
/**
* Wait for the clock to reach the given timestamp.
*
+ * If completion of the returned future triggers some I/O operations or
causes the code to block, it is highly
+ * recommended to execute those
[ignite-3] branch ignite-19903 updated (f9730731f3 -> d9290c8083)
This is an automated email from the ASF dual-hosted git repository. sdanilov pushed a change to branch ignite-19903 in repository https://gitbox.apache.org/repos/asf/ignite-3.git from f9730731f3 Review fixes add d9290c8083 Review fixes No new revisions were added by this update. Summary of changes: .../CompletableFutureCompletedMatcher.java | 5 +- .../network/netty/ItConnectionManagerTest.java | 7 +- .../network/recovery/HandshakeTieBreaker.java | 40 + .../recovery/RecoveryClientHandshakeManager.java | 85 ++- .../recovery/RecoveryServerHandshakeManager.java | 99 +++--- 5 files changed, 141 insertions(+), 95 deletions(-) create mode 100644 modules/network/src/main/java/org/apache/ignite/internal/network/recovery/HandshakeTieBreaker.java
[ignite-3] branch main updated: IGNITE-19859 Fix testAutoFlushByTimer flakiness (#2313)
This is an automated email from the ASF dual-hosted git repository.
ptupitsyn pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
The following commit(s) were added to refs/heads/main by this push:
new 415b262b75 IGNITE-19859 Fix testAutoFlushByTimer flakiness (#2313)
415b262b75 is described below
commit 415b262b752de54633fcd03b8b9a4565a97a96fa
Author: Pavel Tupitsyn
AuthorDate: Thu Jul 13 11:08:39 2023 +0300
IGNITE-19859 Fix testAutoFlushByTimer flakiness (#2313)
Increase timeout, use read-only tx instead of try-catch for concurrent
reads.
---
.../internal/streamer/ItAbstractDataStreamerTest.java| 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git
a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/streamer/ItAbstractDataStreamerTest.java
b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/streamer/ItAbstractDataStreamerTest.java
index 85c36d5e4e..6e26755ace 100644
---
a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/streamer/ItAbstractDataStreamerTest.java
+++
b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/streamer/ItAbstractDataStreamerTest.java
@@ -38,6 +38,7 @@ import org.apache.ignite.table.RecordView;
import org.apache.ignite.table.Table;
import org.apache.ignite.table.Tuple;
import org.apache.ignite.table.mapper.Mapper;
+import org.apache.ignite.tx.TransactionOptions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -142,16 +143,15 @@ public abstract class ItAbstractDataStreamerTest extends
ClusterPerClassIntegrat
publisher.submit(tuple(1, "foo"));
assertTrue(waitForCondition(() -> {
+@SuppressWarnings("resource")
+var tx = ignite().transactions().begin(new
TransactionOptions().readOnly(true));
+
try {
-return view.get(null, tupleKey(1)) != null;
-} catch (Exception e) {
-// TODO IGNITE-19824: Remove try-catch.
-// Ignore tx conflicts caused by deadlock detection mechanism
issues.
-// noinspection CallToPrintStackTrace
-e.printStackTrace();
-return false;
+return view.get(tx, tupleKey(1)) != null;
+} finally {
+tx.rollback();
}
-}, 1000));
+}, 50, 5000));
}
@Test
