[jira] [Updated] (IGNITE-18488) SQL: SUBSTRING function does not support NULL values
[
https://issues.apache.org/jira/browse/IGNITE-18488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky updated IGNITE-18488:
Ignite Flags: (was: Docs Required,Release Notes Required)
> SQL: SUBSTRING function does not support NULL values
>
>
> Key: IGNITE-18488
> URL: https://issues.apache.org/jira/browse/IGNITE-18488
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Andrey Khitrin
>Assignee: Andrey Mashenkov
>Priority: Major
> Labels: ignite-3, sql
> Fix For: 3.0.0-beta2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ANSI99 SQL specification ("6.18 ") says the following:
> {code:java}
> 3) If is specified, then:
> a) Let C be the value of the , ..., and let S
> be the value of the .
> b) If is specified, then let L be the value of length> ...
> c) If either C, S, or L is the null value, then the result of the
> is the null value.
> {code}
> So, we should expect the following behavior:
> {code:sql}
> SUBSTRING('text' FROM 1 FOR NULL) -> NULL
> SUBSTRING('text' FROM NULL FOR 2) -> NULL
> SUBSTRING(NULL FROM 1 FOR 2) -> NULL
> {code}
> Instead, we got errors for these queries:
> {code:sql}
> sql-cli> SELECT SUBSTRING('text' FROM 1 FOR NULL);
> SQL query execution error
> Exception while executing query [query=SELECT SUBSTRING('text' FROM 1 FOR
> NULL);]. Error message:From line 1, column 8 to line 1, column 40: Cannot
> apply 'SUBSTRING' to arguments of type 'SUBSTRING( FROM
> FOR )'. Supported form(s): 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> sql-cli> SELECT SUBSTRING('text' FROM NULL FOR 2);
> SQL query execution error
> Exception while executing query [query=SELECT SUBSTRING('text' FROM NULL FOR
> 2);]. Error message:From line 1, column 8 to line 1, column 40: Cannot apply
> 'SUBSTRING' to arguments of type 'SUBSTRING( FROM FOR
> )'. Supported form(s): 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> sql-cli> SELECT SUBSTRING(NULL FROM 1 FOR 2);
> (client freezes, no answer at all)
> {code}
> *Important*: the last query causes {{NullPointerException}} within DB:
> {code:java}
> 2022-12-30 07:32:46:210 +
> [WARNING][node1-srv-worker-1][ClientInboundMessageHandler] Error processing
> client request
> java.lang.NullPointerException
> at
> org.apache.ignite.internal.sql.SqlColumnTypeConverter.columnTypeToClass(SqlColumnTypeConverter.java:42)
> at
> org.apache.ignite.client.handler.JdbcQueryCursorHandlerImpl.createColumnMetadata(JdbcQueryCursorHandlerImpl.java:162)
> at
> java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
> at
> java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
> at
> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
> at
> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
> at
> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
> at
> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
> at
> org.apache.ignite.client.handler.JdbcQueryCursorHandlerImpl.queryMetadataAsync(JdbcQueryCursorHandlerImpl.java:133)
> at
> org.apache.ignite.client.handler.requests.jdbc.ClientJdbcQueryMetadataRequest.process(ClientJdbcQueryMetadataRequest.java:47)
> at
> org.apache.ignite.client.handler.ClientInboundMessageHandler.processOperation(ClientInboundMessageHandler.java:454)
> at
> org.apache.ignite.client.handler.ClientInboundMessageHandler.processOperation(ClientInboundMessageHandler.java:336)
> at
> org.apache.ignite.client.handler.ClientInboundMessageHandler.channelRead(ClientInboundMessageHandler.java:187)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> at
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327)
> at
>
[jira] [Assigned] (IGNITE-19341) SQL: SUBSTRING function does not support NULL values (try 2)
[
https://issues.apache.org/jira/browse/IGNITE-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky reassigned IGNITE-19341:
---
Assignee: Evgeny Stanilovsky
> SQL: SUBSTRING function does not support NULL values (try 2)
>
>
> Key: IGNITE-19341
> URL: https://issues.apache.org/jira/browse/IGNITE-19341
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Andrey Khitrin
>Assignee: Evgeny Stanilovsky
>Priority: Major
> Labels: calcite2-required, ignite-3, sql
>
> ANSI99 SQL specification ("6.18 ") says the following:
> {code:java}
> 3) If is specified, then:
> a) Let C be the value of the , ..., and let S
> be the value of the .
> b) If is specified, then let L be the value of length> ...
> c) If either C, S, or L is the null value, then the result of the
> is the null value.
> {code}
> So, we should expect the following behavior:
> {code:sql}
> SUBSTRING('text' FROM 1 FOR NULL) -> NULL
> SUBSTRING('text' FROM NULL FOR 2) -> NULL
> SUBSTRING(NULL FROM 1 FOR 2) -> NULL
> {code}
> Instead, we got errors for these queries:
> {code:sql}
> sql-cli> SELECT SUBSTRING('text' FROM 1 FOR NULL);
> SQL query execution error
> Exception while executing query [query=SELECT SUBSTRING('text' FROM 1 FOR
> NULL);]. Error message:From line 1, column 8
> to line 1, column 40: Cannot apply 'SUBSTRING' to arguments of type
> 'SUBSTRING( FROM FOR )'. Supported form(s):
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> sql-cli> SELECT SUBSTRING('text' FROM NULL FOR 2);
> SQL query execution error
> Exception while executing query [query=SELECT SUBSTRING('text' FROM NULL FOR
> 2);]. Error message:From line 1, column 8
> to line 1, column 40: Cannot apply 'SUBSTRING' to arguments of type
> 'SUBSTRING( FROM FOR )'. Supported form(s):
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> {code}
> Only such request works fine:
> {code:sql}
> sql-cli> SELECT SUBSTRING(NULL FROM 1 FOR 2);
> ╔═╗
> ║ EXPR$0 ║
> ╠═╣
> ║ null║
> ╚═╝
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-18955) Add the ability to use filters when data nodes are calculated
[
https://issues.apache.org/jira/browse/IGNITE-18955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723541#comment-17723541
]
Alexander Lapin commented on IGNITE-18955:
--
[~maliev] LGTM, thanks!
> Add the ability to use filters when data nodes are calculated
> -
>
> Key: IGNITE-18955
> URL: https://issues.apache.org/jira/browse/IGNITE-18955
> Project: Ignite
> Issue Type: Improvement
>Reporter: Mirza Aliev
>Assignee: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> {*}Motivation{*}:
> We need to be able to use filters when data nodes are recalculated
> *Definition of done:*
> * Filters are applied when data nodes are recalculated
> *Implementation details:*
> When we write data nodes to metastorage, we extend existing Map with data
> nodes and instead of saving only nodes' names, we store new structure called
> {{NodeWithAttributes}}, where we store name and attributes of the node.
> Currently, when we read data nodes, we convert existing map with node
> counters to a Set of nodes, which counters are more than 0. Now we also will
> apply filter, so as a result data nodes will be filtered with the according
> filter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-18955) Add the ability to use filters when data nodes are calculated
[
https://issues.apache.org/jira/browse/IGNITE-18955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-18955:
-
Reviewer: Alexander Lapin
> Add the ability to use filters when data nodes are calculated
> -
>
> Key: IGNITE-18955
> URL: https://issues.apache.org/jira/browse/IGNITE-18955
> Project: Ignite
> Issue Type: Improvement
>Reporter: Mirza Aliev
>Assignee: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> {*}Motivation{*}:
> We need to be able to use filters when data nodes are recalculated
> *Definition of done:*
> * Filters are applied when data nodes are recalculated
> *Implementation details:*
> When we write data nodes to metastorage, we extend existing Map with data
> nodes and instead of saving only nodes' names, we store new structure called
> {{NodeWithAttributes}}, where we store name and attributes of the node.
> Currently, when we read data nodes, we convert existing map with node
> counters to a Set of nodes, which counters are more than 0. Now we also will
> apply filter, so as a result data nodes will be filtered with the according
> filter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19389) Make lock acquiring timeout message more certain.
[
https://issues.apache.org/jira/browse/IGNITE-19389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Steshin updated IGNITE-19389:
--
Description:
Suggestion: make text of lock acquiring timeout message more certain if there
is no attempt to acquire like: "_Didn't try to take lock for transaction ...
because tx has already timeouted_" instead of common "_Failed to acquire lock
within provided timeout for transaction_"
We've met a case when user sees lock acquiring timeout in transaction but is
confused in researching the cause. He expected and tried to find concurrent
transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't even
launched. The timeout has already happened. Stacktrace indicates the problem,
but it is not the best solution to a user.
The stacktrace:
{code:java}
org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException: Failed
to acquire lock within provided timeout for transaction [timeout=3,
tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
[topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
invalidate=false, rollbackOnly=false,
nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
startTime=1673243469168, duration=30255]]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:802)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
{code}
was:
Suggestion: make text of lock acquiring timeout message more certain if there
is no attempt to acquire lock: like "_Didn't try to take lock for transaction
.. because tx hax already timeouted_" instead of common "_Failed to acquire
lock within provided timeout for transaction_"
We've met a case when user sees lock acquiring timeout in transaction but is
confused in researching the cause. He expected and tried to find concurrent
transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't even
launched. The timeout has already happened. Stacktrace indicates the problem,
but it is not the best solution to a user.
The stacktrace:
{code:java}
org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException: Failed
to acquire lock within provided timeout for transaction [timeout=3,
tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
[topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
invalidate=false, rollbackOnly=false,
nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
startTime=1673243469168, duration=30255]]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
[jira] [Updated] (IGNITE-19504) Sql. Align difference between ignite and calcite runtime available functions.
[
https://issues.apache.org/jira/browse/IGNITE-19504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky updated IGNITE-19504:
Description:
After [1] is done, we can proceed with functionality alignment.
IgniteSqlOperatorTable declare runtime available functions, thus undeclared
functionality is not available for now, for example :
{noformat}
select left('Chicago', 5)
{noformat}
will throw: NoSuchMethodException, the goal of this issue is functionality
aligment for such of undeclared functions and append additional tests.
Need to register :
SqlLibraryOperators.ENDS_WITH
SqlLibraryOperators.STARTS_WITH
[1] https://issues.apache.org/jira/browse/IGNITE-19504
was:
After [1] is done, we can proceed with functionality alignment.
IgniteSqlOperatorTable declare runtime available functions, thus undeclared
functionality is not available for now, for example :
{noformat}
select left('Chicago', 5)
{noformat}
will throw: NoSuchMethodException, the goal of this issue is functionality
aligment for such of undeclared functions and append additional tests.
[1] https://issues.apache.org/jira/browse/IGNITE-19504
> Sql. Align difference between ignite and calcite runtime available functions.
> -
>
> Key: IGNITE-19504
> URL: https://issues.apache.org/jira/browse/IGNITE-19504
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Evgeny Stanilovsky
>Priority: Major
> Labels: calcite2-required, calcite3-required, ignite-3
>
> After [1] is done, we can proceed with functionality alignment.
> IgniteSqlOperatorTable declare runtime available functions, thus undeclared
> functionality is not available for now, for example :
> {noformat}
> select left('Chicago', 5)
> {noformat}
> will throw: NoSuchMethodException, the goal of this issue is functionality
> aligment for such of undeclared functions and append additional tests.
> Need to register :
> SqlLibraryOperators.ENDS_WITH
> SqlLibraryOperators.STARTS_WITH
> [1] https://issues.apache.org/jira/browse/IGNITE-19504
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19173) Get rid BinaryObject restoration on backups during the put operations
[ https://issues.apache.org/jira/browse/IGNITE-19173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-19173: -- Labels: iep-97 (was: iep-101 iep-97) > Get rid BinaryObject restoration on backups during the put operations > - > > Key: IGNITE-19173 > URL: https://issues.apache.org/jira/browse/IGNITE-19173 > Project: Ignite > Issue Type: Improvement >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Labels: iep-97 > > Currently, array restoration is necessary on backups on put operation because > typeId is required to wait for the proper metadata. > But, typeId can be send (on put) separately to avoid useless restoration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19512) Support pagination in CLI
Aleksandr created IGNITE-19512: -- Summary: Support pagination in CLI Key: IGNITE-19512 URL: https://issues.apache.org/jira/browse/IGNITE-19512 Project: Ignite Issue Type: Improvement Components: cli Reporter: Aleksandr After IGNITE-19511 is done, we have to support the pagination on CLI. The pagination should be optional. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19511) Add pagination to REST
Aleksandr created IGNITE-19511: -- Summary: Add pagination to REST Key: IGNITE-19511 URL: https://issues.apache.org/jira/browse/IGNITE-19511 Project: Ignite Issue Type: Improvement Components: rest Reporter: Aleksandr To enhance the performance and usability of our endpoints (deployment units API, metrics, etc), it is recommended to integrate pagination. - list all endpoints requiring pagination support. - develop a universal pagination mechanism applicable across all designated endpoints. Please note that this task does not include the modification of CLI. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19341) SQL: SUBSTRING function does not support NULL values (try 2)
[
https://issues.apache.org/jira/browse/IGNITE-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Mashenkov reassigned IGNITE-19341:
-
Assignee: (was: Andrey Mashenkov)
> SQL: SUBSTRING function does not support NULL values (try 2)
>
>
> Key: IGNITE-19341
> URL: https://issues.apache.org/jira/browse/IGNITE-19341
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Andrey Khitrin
>Priority: Major
> Labels: calcite2-required, ignite-3, sql
>
> ANSI99 SQL specification ("6.18 ") says the following:
> {code:java}
> 3) If is specified, then:
> a) Let C be the value of the , ..., and let S
> be the value of the .
> b) If is specified, then let L be the value of length> ...
> c) If either C, S, or L is the null value, then the result of the
> is the null value.
> {code}
> So, we should expect the following behavior:
> {code:sql}
> SUBSTRING('text' FROM 1 FOR NULL) -> NULL
> SUBSTRING('text' FROM NULL FOR 2) -> NULL
> SUBSTRING(NULL FROM 1 FOR 2) -> NULL
> {code}
> Instead, we got errors for these queries:
> {code:sql}
> sql-cli> SELECT SUBSTRING('text' FROM 1 FOR NULL);
> SQL query execution error
> Exception while executing query [query=SELECT SUBSTRING('text' FROM 1 FOR
> NULL);]. Error message:From line 1, column 8
> to line 1, column 40: Cannot apply 'SUBSTRING' to arguments of type
> 'SUBSTRING( FROM FOR )'. Supported form(s):
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> sql-cli> SELECT SUBSTRING('text' FROM NULL FOR 2);
> SQL query execution error
> Exception while executing query [query=SELECT SUBSTRING('text' FROM NULL FOR
> 2);]. Error message:From line 1, column 8
> to line 1, column 40: Cannot apply 'SUBSTRING' to arguments of type
> 'SUBSTRING( FROM FOR )'. Supported form(s):
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> {code}
> Only such request works fine:
> {code:sql}
> sql-cli> SELECT SUBSTRING(NULL FROM 1 FOR 2);
> ╔═╗
> ║ EXPR$0 ║
> ╠═╣
> ║ null║
> ╚═╝
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19460) Implement missed DDL commands with using Catalog
[ https://issues.apache.org/jira/browse/IGNITE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19460: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Implement missed DDL commands with using Catalog > - > > Key: IGNITE-19460 > URL: https://issues.apache.org/jira/browse/IGNITE-19460 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Earlier in IGNITE-19081 we introduce Catalog events. > Let's add table altering, index creation/dropping commands and them to the > CatalogManager. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19112) Get rid of MvTableStorage#getOrCreateIndex
[ https://issues.apache.org/jira/browse/IGNITE-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19112: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Get rid of MvTableStorage#getOrCreateIndex > -- > > Key: IGNITE-19112 > URL: https://issues.apache.org/jira/browse/IGNITE-19112 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Get rid of methods: > * org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateIndex > * > org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateHashIndex > * > org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateSortedIndex > and make new ones: > * org.apache.ignite.internal.storage.engine.MvTableStorage#createIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#createHashIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getHashIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#createSortedIndex > * org.apache.ignite.internal.storage.engine.MvTableStorage#getSortedIndex -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18536) Schema synchronization feature and Catalog feature integration
[ https://issues.apache.org/jira/browse/IGNITE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-18536: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Schema synchronization feature and Catalog feature integration > -- > > Key: IGNITE-18536 > URL: https://issues.apache.org/jira/browse/IGNITE-18536 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > TBD. > Metadata synchronization should be used, implemented (most likely) on top of > "transactions in the past", defined in the transactions IEP (see "Lock-free > RW Transactions" in > https://cwiki.apache.org/confluence/display/IGNITE/IEP-91:+Transaction+protocol). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18236) Cache objects transformation
[ https://issues.apache.org/jira/browse/IGNITE-18236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-18236: - Labels: iep-97 important ise (was: iep-101 important ise) > Cache objects transformation > > > Key: IGNITE-18236 > URL: https://issues.apache.org/jira/browse/IGNITE-18236 > Project: Ignite > Issue Type: Improvement >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Labels: iep-97, important, ise > Fix For: 2.15 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > See > [IEP-97|https://cwiki.apache.org/confluence/display/IGNITE/IEP-97+Cache+objects+transformation] > for details. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19173) Get rid BinaryObject restoration on backups during the put operations
[ https://issues.apache.org/jira/browse/IGNITE-19173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-19173: - Labels: iep-101 iep-97 (was: iep-101) > Get rid BinaryObject restoration on backups during the put operations > - > > Key: IGNITE-19173 > URL: https://issues.apache.org/jira/browse/IGNITE-19173 > Project: Ignite > Issue Type: Improvement >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Labels: iep-101, iep-97 > > Currently, array restoration is necessary on backups on put operation because > typeId is required to wait for the proper metadata. > But, typeId can be send (on put) separately to avoid useless restoration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19082) Describe Catalog operation flow in README and cleanup dead code.
[ https://issues.apache.org/jira/browse/IGNITE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19082: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Describe Catalog operation flow in README and cleanup dead code. > > > Key: IGNITE-19082 > URL: https://issues.apache.org/jira/browse/IGNITE-19082 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Let's > * ensure Catalog is used by default as a single schema management point by > TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. > * Describe CatalogService operations in README.md > * drop schema related code from configuration. > * drop outdated code from TableManager, IndexManager, SqlSchemaManager, > SchemaRegistry. > * make a PR for merging feature branch to main (if applicable). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18537) Implement missed create/alter/drop Catalog operations
[ https://issues.apache.org/jira/browse/IGNITE-18537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-18537: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Implement missed create/alter/drop Catalog operations > - > > Key: IGNITE-18537 > URL: https://issues.apache.org/jira/browse/IGNITE-18537 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > Every CRETE/ALTER/DROP operation should trigger a new Catalog version. > We already have CREATE TABLE DDL operation implemented. Let's implement > others in similar way. > Please keep in mind that Distribution zones is out of scope now. -the > affinity is not the part of the schema definition. Data storage parameters > are not the part of it as well.- -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19081) Introduce Catalog events API.
[ https://issues.apache.org/jira/browse/IGNITE-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19081: - Epic Link: IGNITE-19502 (was: IGNITE-17766) > Introduce Catalog events API. > - > > Key: IGNITE-19081 > URL: https://issues.apache.org/jira/browse/IGNITE-19081 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > Catalog service should trigger events when it's state is changed. > Other components should be able to subscribe on these events. > Let's > * extends CatalogService from Producer abstract class > * add events for supported DDL commands > * write tests on Catalog events > Note: Let's make these changes aside and avoid breaking the current > implementation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18535) Define new classes for versioned tables/indexes schemas
[
https://issues.apache.org/jira/browse/IGNITE-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Chugunov updated IGNITE-18535:
-
Epic Link: IGNITE-19502 (was: IGNITE-17766)
> Define new classes for versioned tables/indexes schemas
> ---
>
> Key: IGNITE-18535
> URL: https://issues.apache.org/jira/browse/IGNITE-18535
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Ivan Bessonov
>Assignee: Andrey Mashenkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Current approach with schema management is faulty and can't support indexes.
> On top of that, it doesn't allow us to truly have multi-versioned historical
> data. Once the table is removed, it's removed for good, meaning that
> "current" RO transactions will not be able to finish. This is not acceptable.
> h3. Schema definitions
> What we need to have is the following:
> {code:java}
> SchemaDefinitions = map {version -> SchemaDefinition}
> SchemaDefinition = {timestamp, set {TableDefinition}, set{IndexDefinition}}
> TableDefinition = {name, id, array[ColumnDefinition], ...}
> IndexDefinition = {name, id, tableId, state, array[IdxColumnDefinition],
> ...}{code}
> Schema must be versioned, that's the first point. Well, it's already
> versioned in "main", here I mean the global versioning to tie everything to
> transactions and management of SQL indexes.
> Each definition correspond to a time period, where it represents the "actual"
> state of things. It must be used for RO queries, for example. RW transactions
> always use LATEST schema, obviously.
> Now, the meaning of defined values:
> * version - a simple auto-incrementing integer value;
> * "timestamp" - the schema is considered to be valid from this timestamp
> until the timestamp of "next" version (or "inifinity" if the next version
> doesn't yet exist);
> * most of tables and indexes properties are self-explanatory;
> * index state - RO or RW. We should differentiate the indexes that are not
> yet built frome indexes that are fully available.
> Currently, it's not too clear where to store this structure. The problem lies
> in the realm of metadata synchronization, that's not yet designed. But the
> thing is that all nodes must eventually have an up-to-date state and every
> data/index update must be consistent with the version that belongs to a
> current operation's timestamp.
> There are two likely candidates - Meta-Storage or Configuration. We'll figure
> it out later.
> h3. Seralization / storage
> It would be convenient to only store the oldest version + the collection of
> diffs. Every node would unpack that locally, but we would save a lot on the
> storage space in meta-storage in case when user has a lot of tables/indexes.
> This approach would also be beneficial for another reason: we need to know,
> what's changed between versions. It may be hard to calculate if all that we
> have are definitions themselves.
> h3. General thoughts
> This may be a good place to start using integer tableId and indexId more
> often. UUIDs are too much. What's good is that "serializability" of schemas
> gives us easy way of generating integer ids, just like it's don right now
> with configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-19506) Use data nodes from DistributionZoneManager with a causality token instead of BaselineManager#nodes
[ https://issues.apache.org/jira/browse/IGNITE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin reassigned IGNITE-19506: Assignee: Sergey Uttsel > Use data nodes from DistributionZoneManager with a causality token instead of > BaselineManager#nodes > --- > > Key: IGNITE-19506 > URL: https://issues.apache.org/jira/browse/IGNITE-19506 > Project: Ignite > Issue Type: Bug >Reporter: Sergey Uttsel >Assignee: Sergey Uttsel >Priority: Major > Labels: ignite-3 > > h3. *Motivation* > Need to use data nodes from DistributionZoneManager instead of > BaselineManager#nodes in: > # DistributionZoneRebalanceEngine#onUpdateReplicas > # TableManager#createAssignmentsSwitchRebalanceListener > We need to get data nodes consistently so we need to use revision of > configuration events and a meta storage events as causality token. Also need > to use VersionedValue to save data nodes with causality token. > h3. *Definition of Done* > DistributionZoneRebalanceEngine#onUpdateReplicas and > TableManager#createAssignmentsSwitchRebalanceListener use data nodes from > DistributionZoneManager -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19170) Distribution Zones [Part3]. Data collocation.
[ https://issues.apache.org/jira/browse/IGNITE-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-19170: Summary: Distribution Zones [Part3]. Data collocation. (was: Distribution Zones [Part3]) > Distribution Zones [Part3]. Data collocation. > - > > Key: IGNITE-19170 > URL: https://issues.apache.org/jira/browse/IGNITE-19170 > Project: Ignite > Issue Type: Epic >Reporter: Mirza Aliev >Priority: Major > Labels: ignite-3 > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16288) Async API for the configuration
[ https://issues.apache.org/jira/browse/IGNITE-16288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-16288: - Epic Link: (was: IGNITE-17766) > Async API for the configuration > --- > > Key: IGNITE-16288 > URL: https://issues.apache.org/jira/browse/IGNITE-16288 > Project: Ignite > Issue Type: Task >Reporter: Semyon Danilov >Priority: Major > Labels: ignite-3, tech-debt > Fix For: 3.0.0-beta2 > > > Currently, configuration API is blocking, which creates a limitation for a > call site -- we can't call configuration methods from network threads. E.g. > TableManager#tablesAsyncInternal supposed to be non-blocking, but it does use > configuration API, so it had to be used via CompletableFuture#supplyAsync. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19488) RemoteFragmentExecutionException when inserting more than 30 000 rows into one table
[
https://issues.apache.org/jira/browse/IGNITE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723474#comment-17723474
]
Igor commented on IGNITE-19488:
---
[~xtern] there is the code to reproduce the error
https://github.com/Lunigorn/ignite3test/tree/rows-capacity-test
> RemoteFragmentExecutionException when inserting more than 30 000 rows into
> one table
>
>
> Key: IGNITE-19488
> URL: https://issues.apache.org/jira/browse/IGNITE-19488
> Project: Ignite
> Issue Type: Bug
> Components: jdbc, sql
>Reporter: Igor
>Priority: Critical
> Labels: ignite-3
> Attachments: logs.zip
>
>
> h1. Steps to reproduce
> Ignite 3 main branch commit 45380a6c802203dab0d72bd1eb9fb202b2a345b0
> # Create table with 5 columns
> # Insert into table rows by batches 1000 rows each batch.
> # Repeat previous step untill exception is thrown.
> h1. Expected behaviour
> Created more than 30 000 rows.
> h1. Actual behaviour
> An exception after 29 000 rows are inserted:
> {code:java}
> Exception while executing query [query=SELECT COUNT(*) FROM
> rows_capacity_table]. Error message:IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 Query remote fragment execution
> failed: nodeName=TablesAmountCapacityTest_cluster_0,
> queryId=ecd14026-5366-4ee2-b73a-f38757d3ba4f, fragmentId=1561,
> originalMessage=IGN-CMN-1 TraceId:24c93463-f078-410a-8831-36b5c549a907
> java.sql.SQLException: Exception while executing query [query=SELECT COUNT(*)
> FROM rows_capacity_table]. Error message:IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 Query remote fragment execution
> failed: nodeName=TablesAmountCapacityTest_cluster_0,
> queryId=ecd14026-5366-4ee2-b73a-f38757d3ba4f, fragmentId=1561,
> originalMessage=IGN-CMN-1 TraceId:24c93463-f078-410a-8831-36b5c549a907
> at
> org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
> at
> org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:149)
> at
> org.apache.ignite.internal.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:108)
> {code}
> Logs are in the attachment.
> [^logs.zip]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-18747) Sql. Provide commands and handlers for alter index name operation
[ https://issues.apache.org/jira/browse/IGNITE-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-18747: - Epic Link: (was: IGNITE-17766) > Sql. Provide commands and handlers for alter index name operation > - > > Key: IGNITE-18747 > URL: https://issues.apache.org/jira/browse/IGNITE-18747 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Yury Gerzhedovich >Priority: Major > Labels: ignite-3 > > After implementing IGNITE-16196 need to provide DDL command and handler to > altering index name, as well as translation to the command from AST > representation. > As a result, we will be able to translate AST to a command (see > DdlSqlToCommandConverter) and execute this command in order to apply changes > to configuration (see DdlCommandHandler). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16196) Sql. Extend SQL grammar with ALTER INDEX RENAME statement
[ https://issues.apache.org/jira/browse/IGNITE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-16196: - Epic Link: (was: IGNITE-17766) > Sql. Extend SQL grammar with ALTER INDEX RENAME statement > - > > Key: IGNITE-16196 > URL: https://issues.apache.org/jira/browse/IGNITE-16196 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 3.0.0-alpha3 >Reporter: Taras Ledkov >Priority: Minor > Labels: ignite-3 > > As we can CREATE and DROP INDEX need to provide the ability to alter a name > of an index by DDL commands. > Let's extend SQL grammar with the following syntax: > ALTER INDEX [ IF EXISTS ] RENAME TO -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19501) SchemaManager should use CatalogService for building BinaryRow descriptors
[ https://issues.apache.org/jira/browse/IGNITE-19501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19501: - Epic Link: IGNITE-19502 > SchemaManager should use CatalogService for building BinaryRow descriptors > -- > > Key: IGNITE-19501 > URL: https://issues.apache.org/jira/browse/IGNITE-19501 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, SchemaManager uses configuration data to create > BinaryRow/BinaryTuple descriptors. > Let's make SchemaManager and SchemaRegistry using Catalog data instead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19500) IndexManager should listen CatalogService events instead of configuration
[ https://issues.apache.org/jira/browse/IGNITE-19500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19500: - Epic Link: IGNITE-19502 > IndexManager should listen CatalogService events instead of configuration > - > > Key: IGNITE-19500 > URL: https://issues.apache.org/jira/browse/IGNITE-19500 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, IndexManager listens configuration events to create internal > structures. > Let's make IndexManager listens CatalogService events instead. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19493) Sql. Query flow refactoring.
[ https://issues.apache.org/jira/browse/IGNITE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19493: - Epic Link: IGNITE-19502 > Sql. Query flow refactoring. > > > Key: IGNITE-19493 > URL: https://issues.apache.org/jira/browse/IGNITE-19493 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from > SqlQueryManager, then create query context, then start implicit transaction > if needed. > SqlQueryManager doesn't aware of schema versions. > Let's > * add tx timestamp to the interface, it is ok to ignore the tx timestamp > parameter in current implementation unless a new one based on Catalog will be > implemented in IGNITE-19496. > * rework current flow to start implicit transaction first, then get a schema > based on Tx timestamp, then create a query context. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19499) TableManager should listen CatalogService events instead of configuration
[ https://issues.apache.org/jira/browse/IGNITE-19499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19499: - Epic Link: IGNITE-19502 > TableManager should listen CatalogService events instead of configuration > - > > Key: IGNITE-19499 > URL: https://issues.apache.org/jira/browse/IGNITE-19499 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, TableManager listens configuration events to create internal > structures. > Let's make TableManager listens CatalogService events instead. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19497) Sql. Switch to SqlSchemaManager implementation on top of CatalogService
[ https://issues.apache.org/jira/browse/IGNITE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19497: - Epic Link: IGNITE-19502 > Sql. Switch to SqlSchemaManager implementation on top of CatalogService > > > Key: IGNITE-19497 > URL: https://issues.apache.org/jira/browse/IGNITE-19497 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > Let's switch to a new SqlSchemaManager from IGNITE-19496, which uses Catalog > events, and drop old implementation. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19493) Sql. Query flow refactoring.
[ https://issues.apache.org/jira/browse/IGNITE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19493: - Description: As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from SqlQueryManager, then create query context, then start implicit transaction if needed. SqlQueryManager doesn't aware of schema versions. Let's * add tx timestamp to the interface, it is ok to ignore the tx timestamp parameter in current implementation unless a new one based on Catalog will be implemented in IGNITE-19496. * rework current flow to start implicit transaction first, then get a schema based on Tx timestamp, then create a query context. was: As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from SqlQueryMaanger, then create query context, then start implicit transaction if needed. SqlQueryManager doesn't aware of schema versions. Let's * add tx timestamp to the interface, it is ok to ignore the tx timestamp parameter in current implementation unless a new one based on Catalog will be implemented in IGNITE-19496. * rework current flow to start implicit transaction first, then get a schema based on Tx timestamp, then create a query context. > Sql. Query flow refactoring. > > > Key: IGNITE-19493 > URL: https://issues.apache.org/jira/browse/IGNITE-19493 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from > SqlQueryManager, then create query context, then start implicit transaction > if needed. > SqlQueryManager doesn't aware of schema versions. > Let's > * add tx timestamp to the interface, it is ok to ignore the tx timestamp > parameter in current implementation unless a new one based on Catalog will be > implemented in IGNITE-19496. > * rework current flow to start implicit transaction first, then get a schema > based on Tx timestamp, then create a query context. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19496) Sql. Implement SqlSchemaManager on top of CatalogService
[ https://issues.apache.org/jira/browse/IGNITE-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19496: - Epic Link: IGNITE-19502 > Sql. Implement SqlSchemaManager on top of CatalogService > > > Key: IGNITE-19496 > URL: https://issues.apache.org/jira/browse/IGNITE-19496 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, SqlSchemaManager listens configuration events to compose a new > schema, and does not support schema versioning. > Let's implement SqlSchemaManager on top of CatalogService, which is already > aware of schema versions. > The new implementation may be just a caching adapter for CatalogService, > which enriches Catalog with metadata that can be used by Calcite engine. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19483) Transform TableManager and IndexManager to internally work against Catalog event types
[ https://issues.apache.org/jira/browse/IGNITE-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19483: - Epic Link: IGNITE-19502 > Transform TableManager and IndexManager to internally work against Catalog > event types > -- > > Key: IGNITE-19483 > URL: https://issues.apache.org/jira/browse/IGNITE-19483 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Currently, when an event like 'table was added to configuration' happens, the > listener pools the table config by itself and then uses it to create the > table. > This should be changed: the table configuration object should be converted to > an object from Catalog domain and pushed to the listeners. > Same should be done to indices. > Requires investigation. > Also, we need to stop passing configuration to deeply-nested components (like > storages). Also requires investigation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19482) Add a method to InternalTransaction for getting initial schema timestamp
[ https://issues.apache.org/jira/browse/IGNITE-19482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19482: - Epic Link: IGNITE-19502 > Add a method to InternalTransaction for getting initial schema timestamp > > > Key: IGNITE-19482 > URL: https://issues.apache.org/jira/browse/IGNITE-19482 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > The method must return readTs for an RO transaction and beginTs for an RW > transaction. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19510) Deprecated classes/methods/constructors in codebase of AI 3.0
Yury Gerzhedovich created IGNITE-19510: -- Summary: Deprecated classes/methods/constructors in codebase of AI 3.0 Key: IGNITE-19510 URL: https://issues.apache.org/jira/browse/IGNITE-19510 Project: Ignite Issue Type: Improvement Reporter: Yury Gerzhedovich Fix For: 3.0.0-beta2 The number of deprecated entities in code AI 3.0 is variable but high, right now the number is *36* . Need to get rid of most of the deprecation in the code. The ticket could be treated as umbrella for such acivity. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19428) Implement placement driver API
[
https://issues.apache.org/jira/browse/IGNITE-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-19428:
-
Reviewer: Denis Chudov
> Implement placement driver API
> --
>
> Key: IGNITE-19428
> URL: https://issues.apache.org/jira/browse/IGNITE-19428
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> In order to implement RW part of tx protocol it's required to check whether
> we touch same primary replica that was previously enlisted and await primary
> replica appearance on group readiness. Currently it's implemented using same
> raft-leader check on top of retry logic with throttling which is inefficient
> and doesn't support overall idea of hiding replication protocol details under
> replicas umbrella.
> h3. Definition of done
> Long story short, it's required to implement following API (exact one or
> similar)
> {code:java}
> /**
> * Returns a future for the primary replica for the specified replication
> group whose expiration time (the right border of the
> * corresponding lease interval) is greater than or equal to the
> timestamp passed as a parameter. Please pay attention that there are
> * no restriction on the lease start time (left border), it can either be
> less or greater than or equal to proposed timestamp.
> * Given method will await for an appropriate primary replica appearance
> if there's no already existing one. Such awaiting logic is
> * unbounded, so it's mandatory to use explicit await termination like
> {@code orTimeout}.
> *
> * @param replicationGroupId Replication group id.
> * @param timestamp Timestamp reference value.
> * @return Primary replica future.
> */
> public CompletableFuture
> awaitPrimaryReplica(ReplicationGroupId replicationGroupId, HybridTimestamp
> timestamp) {
> return CompletableFuture.completedFuture(null);
> }
> /**
> * Same as {@link
> PlacementDriverManager#awaitPrimaryReplica(ReplicationGroupId,
> HybridTimestamp)} despite the fact that given method
> * won't await for an appropriate primary replica appearance if there's
> no already existing one, in that case null will be returned.
> *
> * @param replicationGroupId Replication group id.
> * @param timestamp Timestamp reference value.
> * @return Primary replica future.
> */
> public CompletableFuture
> getPrimaryReplica(ReplicationGroupId replicationGroupId, HybridTimestamp
> timestamp) {
> return CompletableFuture.completedFuture(null);
> } {code}
> h3. Implementation Notes
> Seems reasonable to reuse PendingComparableValuesTracker logic in order to
> implement aforementioned API. Some PendingComparableValuesTracker
> enhancement are required though, see
> https://issues.apache.org/jira/browse/IGNITE-18859 for more details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19509) Get rid of reading configuration when calling methods MvTableStorage#getOrCreate*Index
Kirill Tkalenko created IGNITE-19509: Summary: Get rid of reading configuration when calling methods MvTableStorage#getOrCreate*Index Key: IGNITE-19509 URL: https://issues.apache.org/jira/browse/IGNITE-19509 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko Fix For: 3.0.0-beta2 Get rid of reading configuration when calling *org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreate*Index*, instead replace it with directly passing the *IndexDescription* of the indexes to the methods. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19509) Get rid of reading configuration when calling methods MvTableStorage#getOrCreate*Index
[ https://issues.apache.org/jira/browse/IGNITE-19509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko reassigned IGNITE-19509: Assignee: Kirill Tkalenko > Get rid of reading configuration when calling methods > MvTableStorage#getOrCreate*Index > -- > > Key: IGNITE-19509 > URL: https://issues.apache.org/jira/browse/IGNITE-19509 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Get rid of reading configuration when calling > *org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreate*Index*, > instead replace it with directly passing the *IndexDescription* of the > indexes to the methods. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19476) Implement unit deploy on demand API
[ https://issues.apache.org/jira/browse/IGNITE-19476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Pochatkin reassigned IGNITE-19476: -- Assignee: Mikhail Pochatkin > Implement unit deploy on demand API > --- > > Key: IGNITE-19476 > URL: https://issues.apache.org/jira/browse/IGNITE-19476 > Project: Ignite > Issue Type: New Feature >Reporter: Mikhail Pochatkin >Assignee: Mikhail Pochatkin >Priority: Major > Labels: ignite-3 > > If code execution is initiated on some node (due to a Compute API invocation) > and this target node doesn’t have a required deployment unit it could be > requested from some node which already has the required deployment unit. It > means that the required deployment unit has clusterDURecord.status == > DEPLOYED and there is at least one node where nodeDURecord.status == > DEPLOYED. If this conditions are met the following steps must be performed: > > # Add meta storage record for target node and deployment unit where > nodeDURecord.status == UPLOADING and nodeDURecord.depOpId == > clusterDUStatus.depOpId. The node should prevent concurrent races between > different threads. It’s achievable without any meta storage operations > because there is no other node which can initiate the on-demand deployment > process.. > Note that if clusterDURecord.status == DEPLOYED && nodeDURecord.status == > UPLOADING => deployment on-demand is in progress for a given deployment unit > and given node. > As soon as the deployment unit is uploaded to the target node, > nodeDURecord.status must be changed to DEPLOYED value. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-18832) Option to assign specific name to thin client and capturing the name in sys.client_connections
[
https://issues.apache.org/jira/browse/IGNITE-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723461#comment-17723461
]
Ignite TC Bot commented on IGNITE-18832:
{panel:title=Branch: [pull/10719/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10719/head] Base: [master] : New Tests
(5)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Queries 3 (lazy=true){color} [[tests
2|https://ci2.ignite.apache.org/viewLog.html?buildId=7177795]]
* {color:#013220}IgniteBinaryCacheQueryLazyTestSuite3:
SqlSystemViewsSelfTest.testClientConnectionViews - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryLazyTestSuite3:
SystemViewSelfTest.testClientConnectionAttributes - PASSED{color}
{color:#8b}Queries 3{color} [[tests
2|https://ci2.ignite.apache.org/viewLog.html?buildId=7177794]]
* {color:#013220}IgniteBinaryCacheQueryTestSuite3:
SqlSystemViewsSelfTest.testClientConnectionViews - PASSED{color}
* {color:#013220}IgniteBinaryCacheQueryTestSuite3:
SystemViewSelfTest.testClientConnectionAttributes - PASSED{color}
{color:#8b}Cache 13{color} [[tests
1|https://ci2.ignite.apache.org/viewLog.html?buildId=7177720]]
* {color:#013220}IgniteCacheTestSuite13:
SystemViewSelfTest.testClientConnectionAttributes - PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7177822buildTypeId=IgniteTests24Java8_RunAll]
> Option to assign specific name to thin client and capturing the name in
> sys.client_connections
> --
>
> Key: IGNITE-18832
> URL: https://issues.apache.org/jira/browse/IGNITE-18832
> Project: Ignite
> Issue Type: Improvement
> Components: thin client
>Affects Versions: 2.12, 2.13, 2.11.1, 2.14
>Reporter: Gangaiah
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: IEP-35
>
>
> System view client_connections doesn't have name column to capture client
> name, it should have the column to track the thin client connections.. It's
> difficult to pin point the client where remote address same for multiple
> clients in production scenarios.
> https://lists.apache.org/thread/joy6vjopq2zovkj768hnhfvmd44vhs4b
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-19505) Design distribution zone based partition replication
[ https://issues.apache.org/jira/browse/IGNITE-19505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin reassigned IGNITE-19505: Assignee: Kirill Gusakov > Design distribution zone based partition replication > > > Key: IGNITE-19505 > URL: https://issues.apache.org/jira/browse/IGNITE-19505 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Assignee: Kirill Gusakov >Priority: Major > > *Motivation* > The current data layout is builded around the table partitions. Each table > partition forms the group of replicas with the RAFT protocol (pluggable > protocol in future). This layout was great up to the moment, when we decided > to introduce the support of collocation for any chosen group of tables. > After some rounds of discussions we decided to use the distribution zone as > the scope of collocation. So, the partition will be managed by the > distribution zone associated entity, instead of the table one. > *Definition of done* > Under this issue we need to provide the design document, which will describe > the process for migration the current state to the: > - Table partitions must be the part (one-to-one relation) of the zone > partition. > - The data layout for partitions of different tables inside the one zone > partition - must be described as well. > - Tables, which share the one distribution zone will be collocated (the same > key must be stored in the same distribution zone partition) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19508) Design and breakdown tx on unstable topology topic.
Alexander Lapin created IGNITE-19508: Summary: Design and breakdown tx on unstable topology topic. Key: IGNITE-19508 URL: https://issues.apache.org/jira/browse/IGNITE-19508 Project: Ignite Issue Type: Task Reporter: Alexander Lapin h3. Motivation General approach of tx recovery/tx on unstable topology is available in described in [https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol#IEP91:Transactionprotocol-Recovery|IEP-91] Thus it's required to prepare full set of tickets in order to handle * Data node (primaryReplica|majority) failure. * Commit partition failure. * Tx coordinator failure. except disaster recovery and graceful primary replica switch, that will be covered in separate topics. h3. Definition of Done * Corresponding epic is created. * Epic is populated with full set of tasks. * Tasks are estimated and assigned to sprints. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19507) [TC Bot] Doesn't send messages to Slack
Sergey Uttsel created IGNITE-19507:
--
Summary: [TC Bot] Doesn't send messages to Slack
Key: IGNITE-19507
URL: https://issues.apache.org/jira/browse/IGNITE-19507
Project: Ignite
Issue Type: Bug
Reporter: Sergey Uttsel
TC Bot doesn't send messages to Slack. For example:
* Open [https://mtcga.gridgain.com/monitoring.html]
* Press "Send" button for Test Slack notification.
* {*}Expected{*}: new message in Slack chat.
* {*}Actual{*}: no messages.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19505) Design distribution zone based partition replication
[ https://issues.apache.org/jira/browse/IGNITE-19505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-19505: Description: *Motivation* The current data layout is builded around the table partitions. Each table partition forms the group of replicas with the RAFT protocol (pluggable protocol in future). This layout was great up to the moment, when we decided to introduce the support of collocation for any chosen group of tables. After some rounds of discussions we decided to use the distribution zone as the scope of collocation. So, the partition will be managed by the distribution zone associated entity, instead of the table one. *Definition of done* Under this issue we need to provide the design document, which will describe the process for migration the current state to the: - Table partitions must be the part (one-to-one relation) of the zone partition. - The data layout for partitions of different tables inside the one zone partition - must be described as well. - Tables, which share the one distribution zone will be collocated (the same key must be stored in the same distribution zone partition) was: *Motivation* The current data layout is builded around the table partitions. Each table partition forms the group of replicas with the RAFT protocol (pluggable protocol in future). This layout was great up to the moment, when we decided to introduce the support of collocation for any chosen group of tables. After some rounds of discussions we decided to use the distribution zone as the scope of collocation. So, the partition will be managed by the distribution zone entity, instead of the table one. *Definition of done* Under this issue we need to provide the design document, which will describe the process for migration the current state to the: - Table partitions must be the part (one-to-one relation) of the zone partition. - The data layout for partitions of different tables inside the one zone partition - must be described as well. - Tables, which share the one distribution zone will be collocated (the same key must be stored in the same distribution zone partition) > Design distribution zone based partition replication > > > Key: IGNITE-19505 > URL: https://issues.apache.org/jira/browse/IGNITE-19505 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Priority: Major > > *Motivation* > The current data layout is builded around the table partitions. Each table > partition forms the group of replicas with the RAFT protocol (pluggable > protocol in future). This layout was great up to the moment, when we decided > to introduce the support of collocation for any chosen group of tables. > After some rounds of discussions we decided to use the distribution zone as > the scope of collocation. So, the partition will be managed by the > distribution zone associated entity, instead of the table one. > *Definition of done* > Under this issue we need to provide the design document, which will describe > the process for migration the current state to the: > - Table partitions must be the part (one-to-one relation) of the zone > partition. > - The data layout for partitions of different tables inside the one zone > partition - must be described as well. > - Tables, which share the one distribution zone will be collocated (the same > key must be stored in the same distribution zone partition) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19506) Use data nodes from DistributionZoneManager with a causality token instead of BaselineManager#nodes
[ https://issues.apache.org/jira/browse/IGNITE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Uttsel updated IGNITE-19506: --- Description: h3. *Motivation* Need to use data nodes from DistributionZoneManager instead of BaselineManager#nodes in: # DistributionZoneRebalanceEngine#onUpdateReplicas # TableManager#createAssignmentsSwitchRebalanceListener We need to get data nodes consistently so we need to use revision of configuration events and a meta storage events as causality token. Also need to use VersionedValue to save data nodes with causality token. h3. *Definition of Done* DistributionZoneRebalanceEngine#onUpdateReplicas and TableManager#createAssignmentsSwitchRebalanceListener use data nodes from DistributionZoneManager was: h3. *Motivation* Need to use data nodes from DistributionZoneManager instead of BaselineManager#nodes in: # DistributionZoneRebalanceEngine#onUpdateReplicas # TableManager#createAssignmentsSwitchRebalanceListener We need to get data nodes consistently so we need to use revision of configuration events and a meta storage events as causality token. Also need to use VersionedValue to save data nodes with causality token. h3. *Definition of Done* DistributionZoneRebalanceEngine#onUpdateReplicas and TableManager#createAssignmentsSwitchRebalanceListener use data nodes from DistributionZoneManager > Use data nodes from DistributionZoneManager with a causality token instead of > BaselineManager#nodes > --- > > Key: IGNITE-19506 > URL: https://issues.apache.org/jira/browse/IGNITE-19506 > Project: Ignite > Issue Type: Bug >Reporter: Sergey Uttsel >Priority: Major > Labels: ignite-3 > > h3. *Motivation* > Need to use data nodes from DistributionZoneManager instead of > BaselineManager#nodes in: > # DistributionZoneRebalanceEngine#onUpdateReplicas > # TableManager#createAssignmentsSwitchRebalanceListener > We need to get data nodes consistently so we need to use revision of > configuration events and a meta storage events as causality token. Also need > to use VersionedValue to save data nodes with causality token. > h3. *Definition of Done* > DistributionZoneRebalanceEngine#onUpdateReplicas and > TableManager#createAssignmentsSwitchRebalanceListener use data nodes from > DistributionZoneManager -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19505) Design distribution zone based partition replication
[ https://issues.apache.org/jira/browse/IGNITE-19505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-19505: Description: *Motivation* The current data layout is builded around the table partitions. Each table partition forms the group of replicas with the RAFT protocol (pluggable protocol in future). This layout was great up to the moment, when we decided to introduce the support of collocation for any chosen group of tables. After some rounds of discussions we decided to use the distribution zone as the scope of collocation. So, the partition will be managed by the distribution zone entity, instead of the table one. *Definition of done* Under this issue we need to provide the design document, which will describe the process for migration the current state to the: - Table partitions must be the part (one-to-one relation) of the zone partition. - The data layout for partitions of different tables inside the one zone partition - must be described as well. - Tables, which share the one distribution zone will be collocated (the same key must be stored in the same distribution zone partition) was: *Motivation* Current data layout is builded around the table partitions. Each table partition forms the group of replicas with the RAFT protocol (pluggable protocol in future). This layout was great up to the moment, when we decided to introduce the support of collocation for any chosen group of tables. After some rounds of discussions we decided to use the distribution zone as the scope of collocation. So, the partition will be managed by the distribution zone entity, instead of the table one. *Definition of done* Under this issue we need to provide the design document, which will describe the process for migration the current state to the: - Table partitions must be the part (one-to-one relation) of the zone partition. - The data layout for partitions of different tables inside the one zone partition - must be described as well. - Tables, which share the one distribution zone will be collocated (the same key must be stored in the same distribution zone partition) > Design distribution zone based partition replication > > > Key: IGNITE-19505 > URL: https://issues.apache.org/jira/browse/IGNITE-19505 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Priority: Major > > *Motivation* > The current data layout is builded around the table partitions. Each table > partition forms the group of replicas with the RAFT protocol (pluggable > protocol in future). This layout was great up to the moment, when we decided > to introduce the support of collocation for any chosen group of tables. > After some rounds of discussions we decided to use the distribution zone as > the scope of collocation. So, the partition will be managed by the > distribution zone entity, instead of the table one. > *Definition of done* > Under this issue we need to provide the design document, which will describe > the process for migration the current state to the: > - Table partitions must be the part (one-to-one relation) of the zone > partition. > - The data layout for partitions of different tables inside the one zone > partition - must be described as well. > - Tables, which share the one distribution zone will be collocated (the same > key must be stored in the same distribution zone partition) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19505) Design distribution zone based partition replication
[ https://issues.apache.org/jira/browse/IGNITE-19505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-19505: Description: *Motivation* Current data layout is builded around the table partitions. Each table partition forms the group of replicas with the RAFT protocol (pluggable protocol in future). This layout was great up to the moment, when we decided to introduce the support of collocation for any chosen group of tables. After some rounds of discussions we decided to use the distribution zone as the scope of collocation. So, the partition will be managed by the distribution zone entity, instead of the table one. *Definition of done* Under this issue we need to provide the design document, which will describe the process for migration the current state to the: - Table partitions must be the part (one-to-one relation) of the zone partition. - The data layout for partitions of different tables inside the one zone partition - must be described as well. - Tables, which share the one distribution zone will be collocated (the same key must be stored in the same distribution zone partition) > Design distribution zone based partition replication > > > Key: IGNITE-19505 > URL: https://issues.apache.org/jira/browse/IGNITE-19505 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Priority: Major > > *Motivation* > Current data layout is builded around the table partitions. Each table > partition forms the group of replicas with the RAFT protocol (pluggable > protocol in future). This layout was great up to the moment, when we decided > to introduce the support of collocation for any chosen group of tables. > After some rounds of discussions we decided to use the distribution zone as > the scope of collocation. So, the partition will be managed by the > distribution zone entity, instead of the table one. > *Definition of done* > Under this issue we need to provide the design document, which will describe > the process for migration the current state to the: > - Table partitions must be the part (one-to-one relation) of the zone > partition. > - The data layout for partitions of different tables inside the one zone > partition - must be described as well. > - Tables, which share the one distribution zone will be collocated (the same > key must be stored in the same distribution zone partition) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19457) Review nullability of network messages' fields
[
https://issues.apache.org/jira/browse/IGNITE-19457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin reassigned IGNITE-19457:
Assignee: Alexander Lapin
> Review nullability of network messages' fields
> --
>
> Key: IGNITE-19457
> URL: https://issues.apache.org/jira/browse/IGNITE-19457
> Project: Ignite
> Issue Type: Improvement
>Reporter: Semyon Danilov
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> After implementing https://issues.apache.org/jira/browse/IGNITE-19437
> occurred that following list of fields in messages are null despite the fact
> that probably they should be @NotNull, probably that means that we've forgot
> to add corresponding setters while building such methods.
> * PlacementDriverMessage, ReplicaRequest:
> {code:java}
> groupId{code}
> * SnapshotRequestMessage:
> {code:java}
> id{code}
> * ReadWriteMultiRowReplicaRequest:
> {code:java}
> commitPartitionId{code}
> * ReadWriteReplicaRequest:
> {code:java}
> transactionId{code}
> * ReadWriteSingleRowReplicaRequest:
> {code:java}
> commitPartitionId{code}
> h3. Definition of Done
> * For each case null-ability verified and either comment is written
> explaining why it's valid to have null value or setter is added and
> @Nullable is removed.
> * Tests not needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19506) Use data nodes from DistributionZoneManager with a causality token instead of BaselineManager#nodes
[ https://issues.apache.org/jira/browse/IGNITE-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Uttsel updated IGNITE-19506: --- Labels: ignite-3 (was: ) > Use data nodes from DistributionZoneManager with a causality token instead of > BaselineManager#nodes > --- > > Key: IGNITE-19506 > URL: https://issues.apache.org/jira/browse/IGNITE-19506 > Project: Ignite > Issue Type: Bug >Reporter: Sergey Uttsel >Priority: Major > Labels: ignite-3 > > h3. *Motivation* > Need to use data nodes from DistributionZoneManager instead of > BaselineManager#nodes in: > # DistributionZoneRebalanceEngine#onUpdateReplicas > # TableManager#createAssignmentsSwitchRebalanceListener > We need to get data nodes consistently so we need to use revision of > configuration events and a meta storage events as causality token. Also need > to use > VersionedValue to save data nodes with causality token. > h3. *Definition of Done* > DistributionZoneRebalanceEngine#onUpdateReplicas and > TableManager#createAssignmentsSwitchRebalanceListener use data nodes from > DistributionZoneManager -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19506) Use data nodes from DistributionZoneManager with a causality token instead of BaselineManager#nodes
Sergey Uttsel created IGNITE-19506: -- Summary: Use data nodes from DistributionZoneManager with a causality token instead of BaselineManager#nodes Key: IGNITE-19506 URL: https://issues.apache.org/jira/browse/IGNITE-19506 Project: Ignite Issue Type: Bug Reporter: Sergey Uttsel h3. *Motivation* Need to use data nodes from DistributionZoneManager instead of BaselineManager#nodes in: # DistributionZoneRebalanceEngine#onUpdateReplicas # TableManager#createAssignmentsSwitchRebalanceListener We need to get data nodes consistently so we need to use revision of configuration events and a meta storage events as causality token. Also need to use VersionedValue to save data nodes with causality token. h3. *Definition of Done* DistributionZoneRebalanceEngine#onUpdateReplicas and TableManager#createAssignmentsSwitchRebalanceListener use data nodes from DistributionZoneManager -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19505) Design distribution zone based partition replication
[ https://issues.apache.org/jira/browse/IGNITE-19505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-19505: Ignite Flags: (was: Docs Required,Release Notes Required) > Design distribution zone based partition replication > > > Key: IGNITE-19505 > URL: https://issues.apache.org/jira/browse/IGNITE-19505 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19504) Sql. Align difference between ignite and calcite runtime available functions.
[
https://issues.apache.org/jira/browse/IGNITE-19504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky updated IGNITE-19504:
Description:
After [1] is done, we can proceed with functionality alignment.
IgniteSqlOperatorTable declare runtime available functions, thus undeclared
functionality is not available for now, for example :
{noformat}
select left('Chicago', 5)
{noformat}
will throw: NoSuchMethodException, the goal of this issue is functionality
aligment for such of undeclared functions and append additional tests.
[1] https://issues.apache.org/jira/browse/IGNITE-19504
was:
IgniteSqlOperatorTable declare runtime available functions, thus undeclared
functionality is not available for now, for example :
{noformat}
select left('Chicago', 5)
{noformat}
will throw: NoSuchMethodException, the goal of this issue is functionality
aligment for such of undeclared functions and append additional tests.
> Sql. Align difference between ignite and calcite runtime available functions.
> -
>
> Key: IGNITE-19504
> URL: https://issues.apache.org/jira/browse/IGNITE-19504
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Evgeny Stanilovsky
>Priority: Major
> Labels: calcite2-required, calcite3-required, ignite-3
>
> After [1] is done, we can proceed with functionality alignment.
> IgniteSqlOperatorTable declare runtime available functions, thus undeclared
> functionality is not available for now, for example :
> {noformat}
> select left('Chicago', 5)
> {noformat}
> will throw: NoSuchMethodException, the goal of this issue is functionality
> aligment for such of undeclared functions and append additional tests.
> [1] https://issues.apache.org/jira/browse/IGNITE-19504
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19505) Design distribution zone based partition replication
Kirill Gusakov created IGNITE-19505: --- Summary: Design distribution zone based partition replication Key: IGNITE-19505 URL: https://issues.apache.org/jira/browse/IGNITE-19505 Project: Ignite Issue Type: Task Reporter: Kirill Gusakov -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19389) Make lock acquiring timeout message more certain.
[
https://issues.apache.org/jira/browse/IGNITE-19389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Steshin updated IGNITE-19389:
--
Description:
Suggestion: make text of lock acquiring timeout message more certain if there
is no attempt to acquire lock: like "_Didn't try to take lock for transaction
.. because tx hax already timeouted_" instead of common "_Failed to acquire
lock within provided timeout for transaction_"
We've met a case when user sees lock acquiring timeout in transaction but is
confused in researching the cause. He expected and tried to find concurrent
transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't even
launched. The timeout has already happened. Stacktrace indicates the problem,
but it is not the best solution to a user.
The stacktrace:
{code:java}
org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException: Failed
to acquire lock within provided timeout for transaction [timeout=3,
tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
[topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
invalidate=false, rollbackOnly=false,
nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
startTime=1673243469168, duration=30255]]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:802)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
{code}
was:
Suggestion: make text of lock acquiring timeout message more certain if there
is no attempt to acquire lock: like "Didn't try to take lock for transaction ..
because tx hax already timeouted" instead of common "Failed to acquire lock
within provided timeout for transaction"
We've met a case when user sees lock acquiring timeout in transaction but is
confused in researching the cause. He expected and tried to find concurrent
transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't even
launched. The timeout has already happened. Stacktrace indicates the problem,
but it is not the best solution to a user.
The stacktrace:
{code:java}
org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException: Failed
to acquire lock within provided timeout for transaction [timeout=3,
tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
[topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
invalidate=false, rollbackOnly=false,
nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
startTime=1673243469168, duration=30255]]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
[jira] [Updated] (IGNITE-19389) Make lock acquiring timeout message more certain.
[
https://issues.apache.org/jira/browse/IGNITE-19389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Steshin updated IGNITE-19389:
--
Description:
Suggestion: make text of lock acquiring timeout message more certain if there
is no attempt to acquire lock: like "Didn't try to take lock for transaction ..
because tx hax already timeouted" instead of common "Failed to acquire lock
within provided timeout for transaction"
We've met a case when user sees lock acquiring timeout in transaction but is
confused in researching the cause. He expected and tried to find concurrent
transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't even
launched. The timeout has already happened. Stacktrace indicates the problem,
but it is not the best solution to a user.
The stacktrace:
{code:java}
org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException: Failed
to acquire lock within provided timeout for transaction [timeout=3,
tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
[topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
invalidate=false, rollbackOnly=false,
nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
startTime=1673243469168, duration=30255]]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:802)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
{code}
was:
Suggestion: make text of lock acquiring timeout message more certain if there
is no attempt to acquire lock
We've met a case when user sees lock acquiring timeout in transaction but is
confused in researching the cause. He expected and tried to find concurrent
transaction, deadlocks, hanging locks, etc. Actually, lock attempt hasn't even
launched. The timeout has already happened. Stacktrace indicates the problem,
but it is not the best solution to a user.
The stacktrace:
{code:java}
org.apache.ignite.internal.transactions.IgniteTxTimeoutCheckedException: Failed
to acquire lock within provided timeout for transaction [timeout=3,
tx=GridDhtTxLocal[xid=f5c28b36881--10c7-fe43--0016,
xidVersion=GridCacheVersion [topVer=281542211, order=1685300194399,
nodeOrder=22, dataCenterId=0], nearXidVersion=GridCacheVersion
[topVer=281542211, order=1685300194398, nodeOrder=552, dataCenterId=0],
concurrency=PESSIMISTIC, isolation=REPEATABLE_READ, state=ACTIVE,
invalidate=false, rollbackOnly=false,
nodeId=0693b588-3677-45d8-b1a2-06aa03fa4ed8, timeout=3,
startTime=1673243469168, duration=30255]]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.timeoutException(IgniteTxAdapter.java:785)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:710)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:660)
~[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1239)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:824)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:802)
[ignite-core-2.12.0-p6.jar:2.12.0-p6]
{code}
> Make lock
[jira] [Updated] (IGNITE-15003) Calcite engine. Function argument is not casted implicitly
[
https://issues.apache.org/jira/browse/IGNITE-15003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky updated IGNITE-15003:
Ignite Flags: (was: Docs Required,Release Notes Required)
> Calcite engine. Function argument is not casted implicitly
> --
>
> Key: IGNITE-15003
> URL: https://issues.apache.org/jira/browse/IGNITE-15003
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Taras Ledkov
>Assignee: Vladimir Steshin
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Steps to reproduce:
> {code}
> CREATE TABLE strings(a VARCHAR, b BIGINT)
> INSERT INTO STRINGS VALUES ('abc', 1)
> SELECT LEFT(a, b) FROM strings
> {code}
> The query fails with error:
> {code}
> java.lang.RuntimeException: while resolving method 'left[class
> java.lang.String, long]' in class class
> org.apache.calcite.runtime.SqlFunctions
> at
> org.apache.calcite.adapter.enumerable.EnumUtils.call(EnumUtils.java:666)
> at
> org.apache.calcite.adapter.enumerable.EnumUtils.call(EnumUtils.java:627)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexImpTable$MethodImplementor.implementSafe(RexImpTable.java:993)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexImpTable$AbstractRexCallImplementor.genValueStatement(RexImpTable.java:1946)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexImpTable$AbstractRexCallImplementor.implement(RexImpTable.java:1909)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.visitCall(RexToLixTranslator.java:978)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.visitCall(RexToLixTranslator.java:78)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:174)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:873)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:78)
> at org.apache.calcite.rex.RexLocalRef.accept(RexLocalRef.java:75)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.translate(RexToLixTranslator.java:204)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.translate(RexToLixTranslator.java:197)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.translateList(RexToLixTranslator.java:750)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.translateProjects(RexToLixTranslator.java:178)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.compile(ExpressionFactoryImpl.java:300)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.lambda$scalar$4(ExpressionFactoryImpl.java:263)
> at
> java.base/java.util.concurrent.ConcurrentMap.computeIfAbsent(ConcurrentMap.java:330)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.scalar(ExpressionFactoryImpl.java:263)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.project(ExpressionFactoryImpl.java:220)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.visit(LogicalRelImplementor.java:320)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.visit(LogicalRelImplementor.java:109)
> at
> org.apache.ignite.internal.processors.query.calcite.rel.IgniteTableScan.accept(IgniteTableScan.java:124)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.visit(LogicalRelImplementor.java:657)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.visit(LogicalRelImplementor.java:667)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.visit(LogicalRelImplementor.java:161)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.visit(LogicalRelImplementor.java:109)
> at
> org.apache.ignite.internal.processors.query.calcite.rel.IgniteSender.accept(IgniteSender.java:97)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.visit(LogicalRelImplementor.java:657)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.LogicalRelImplementor.go(LogicalRelImplementor.java:672)
> at
>
[jira] [Created] (IGNITE-19504) Sql. Align difference between ignite and calcite runtime available functions.
Evgeny Stanilovsky created IGNITE-19504:
---
Summary: Sql. Align difference between ignite and calcite runtime
available functions.
Key: IGNITE-19504
URL: https://issues.apache.org/jira/browse/IGNITE-19504
Project: Ignite
Issue Type: Improvement
Components: sql
Affects Versions: 3.0.0-beta1
Reporter: Evgeny Stanilovsky
IgniteSqlOperatorTable declare runtime available functions, thus undeclared
functionality is not available for now, for example :
{noformat}
select left('Chicago', 5)
{noformat}
will throw: NoSuchMethodException, the goal of this issue is functionality
aligment for such of undeclared functions and append additional tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19503) Sql. UUID. Some expressions and statements with types from incompatible families are not rejected.
[
https://issues.apache.org/jira/browse/IGNITE-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-19503:
--
Description:
Type compatibility checks for UUID in some expression/statement are not
performed at validation stage:
INTEGER vs UUID:
{code:java}
@Test
public void testInsert() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// / class java.util.UUID cannot be cast to class java.lang.Integer
sql("INSERT INTO t1 VALUES(1,
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
@Test
public void testUpdate() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
sql("INSERT INTO t1 VALUES(1, 1)");
// class java.util.UUID cannot be cast to class java.lang.Integer
sql("UPDATE t1 SET int_col
='46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
}
{code}
INTEGER vs VARBINARY
{code:java}
@Test
public void testInsert2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// Column 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("INSERT INTO t1 VALUES(1, x'010203')");
}
@Test
public void testUpdate2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// column 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("UPDATE t1 SET int_col = x'010203'");
}
{code}
Expressions:
{code:java}
@Test
public void testExprs() {
//sql("SELECT 1 in ('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
// Binary comparison operations are rejected by validator: OK
// From line 1, column 1 to line 1, column 55: Invalid types for
comparison: INTEGER NOT NULL = UUID NOT NULL
sql("SELECT 1 = '46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
// CASE: OK
// Illegal mixing of types in CASE or COALESCE statement
sql("SELECT CASE int_col WHEN 1 THEN
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID ELSE '2' END FROM t1");
// Arithmetic
/*
java.lang.IllegalArgumentException: Cannot infer return type for +;
operand types: [INTEGER, UUID]
at
org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:541)
at
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:504)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:605)
at
org.apache.calcite.sql.SqlBinaryOperator.deriveType(SqlBinaryOperator.java:178)
*/
sql("SELECT 1 + '46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
// Runtime: Cannot convert 46138242-b771-4d8b-ad26-2b3fcee5f11d to int
sql("SELECT 1 IN ('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
// Line 8, Column 281: Cannot cast "java.util.UUID" to
"org.apache.calcite.avatica.util.ByteString"
//Caused by: org.codehaus.commons.compiler.CompileException: Line 8,
Column 281: Cannot cast "java.util.UUID" to
"org.apache.calcite.avatica.util.ByteString"
//at
org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:13014)
sql("SELECT x'010203' IN
('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
{code}
This happens because ANY type is used as a type family for UUID/custom data
types and ANY can be cast to/from all other data types see SqlTypeUtil and
other calcite classes.
When fixing INSERT statement we should remember that calcite's DEFAULT
expression has ANY type as well.
was:
Type compatibility checks for UUID in some expression/statement are not
performed at validation stage:
INTEGER vs UUID:
{code:java}
@Test
public void testInsert() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// / class java.util.UUID cannot be cast to class java.lang.Integer
sql("INSERT INTO t1 VALUES(1,
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
@Test
public void testUpdate() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
sql("INSERT INTO t1 VALUES(1, 1)");
// class java.util.UUID cannot be cast to class java.lang.Integer
sql("UPDATE t1 SET int_col
='46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
}
{code}
INTEGER vs VARBINARY
{code:java}
@Test
public void testInsert2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// Column 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("INSERT INTO t1 VALUES(1, x'010203')");
}
@Test
public void testUpdate2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
//sql("INSERT INTO t1 VALUES(1, 1)");
// olumn 33: Cannot assign to
[jira] [Updated] (IGNITE-19503) Sql. UUID. Some expressions and statements with types from incompatible families are not rejected.
[
https://issues.apache.org/jira/browse/IGNITE-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-19503:
--
Description:
Type compatibility checks for UUID in some expression/statement are not
performed at validation stage:
INTEGER vs UUID:
{code:java}
@Test
public void testInsert() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// / class java.util.UUID cannot be cast to class java.lang.Integer
sql("INSERT INTO t1 VALUES(1,
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
@Test
public void testUpdate() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
sql("INSERT INTO t1 VALUES(1, 1)");
// class java.util.UUID cannot be cast to class java.lang.Integer
sql("UPDATE t1 SET int_col
='46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
}
{code}
INTEGER vs VARBINARY
{code:java}
@Test
public void testInsert2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// Column 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("INSERT INTO t1 VALUES(1, x'010203')");
}
@Test
public void testUpdate2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
//sql("INSERT INTO t1 VALUES(1, 1)");
// olumn 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("UPDATE t1 SET int_col = x'010203'");
}
{code}
Expressions:
{code:java}
@Test
public void testExprs() {
//sql("SELECT 1 in ('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
// Binary comparison operations are rejected by validator: OK
// From line 1, column 1 to line 1, column 55: Invalid types for
comparison: INTEGER NOT NULL = UUID NOT NULL
sql("SELECT 1 = '46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
// CASE: OK
// Illegal mixing of types in CASE or COALESCE statement
sql("SELECT CASE int_col WHEN 1 THEN
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID ELSE '2' END FROM t1");
// Arithmetic
/*
java.lang.IllegalArgumentException: Cannot infer return type for +;
operand types: [INTEGER, UUID]
at
org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:541)
at
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:504)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:605)
at
org.apache.calcite.sql.SqlBinaryOperator.deriveType(SqlBinaryOperator.java:178)
*/
sql("SELECT 1 + '46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
// Runtime: Cannot convert 46138242-b771-4d8b-ad26-2b3fcee5f11d to int
sql("SELECT 1 IN ('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
// Line 8, Column 281: Cannot cast "java.util.UUID" to
"org.apache.calcite.avatica.util.ByteString"
//Caused by: org.codehaus.commons.compiler.CompileException: Line 8,
Column 281: Cannot cast "java.util.UUID" to
"org.apache.calcite.avatica.util.ByteString"
//at
org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:13014)
sql("SELECT x'010203' IN
('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
{code}
This happens because ANY type is used as a type family for UUID/custom data
types and ANY can be cast to/from all other data types see SqlTypeUtil and
other calcite classes.
When fixing INSERT statement we should remember that calcite's DEFAULT
expression has ANY type as well.
was:
Type compatibility checks for UUID in some expression/statement are not
performed at validation stage:
INTEGER vs UUID:
{code:java}
@Test
public void testInsert() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// / class java.util.UUID cannot be cast to class java.lang.Integer
sql("INSERT INTO t1 VALUES(1,
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
@Test
public void testUpdate() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
sql("INSERT INTO t1 VALUES(1, 1)");
// class java.util.UUID cannot be cast to class java.lang.Integer
sql("UPDATE t1 SET int_col
='46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
}
{code}
INTEGER vs VARBINARY
{code:java}
@Test
public void testInsert2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// Column 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("INSERT INTO t1 VALUES(1, x'010203')");
}
@Test
public void testUpdate2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
//sql("INSERT INTO t1 VALUES(1,
[jira] [Updated] (IGNITE-19503) Sql. UUID. Some expressions and statements with types from incompatible families are not rejected.
[
https://issues.apache.org/jira/browse/IGNITE-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-19503:
--
Description:
Type compatibility checks for UUID in some expression/statement are not
performed at validation stage:
INTEGER vs UUID:
{code:java}
@Test
public void testInsert() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// / class java.util.UUID cannot be cast to class java.lang.Integer
sql("INSERT INTO t1 VALUES(1,
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
@Test
public void testUpdate() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
sql("INSERT INTO t1 VALUES(1, 1)");
// class java.util.UUID cannot be cast to class java.lang.Integer
sql("UPDATE t1 SET int_col
='46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
}
{code}
INTEGER vs VARBINARY
{code:java}
@Test
public void testInsert2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// Column 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("INSERT INTO t1 VALUES(1, x'010203')");
}
@Test
public void testUpdate2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
//sql("INSERT INTO t1 VALUES(1, 1)");
// olumn 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("UPDATE t1 SET int_col = x'010203'");
}
{code}
Expressions:
{code:java}
@Test
public void testExprs() {
//sql("SELECT 1 in ('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
// Binary comparison operations are rejected by validator: OK
// From line 1, column 1 to line 1, column 55: Invalid types for
comparison: INTEGER NOT NULL = UUID NOT NULL
sql("SELECT 1 = '46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
// CASE: OK
// Illegal mixing of types in CASE or COALESCE statement
sql("SELECT CASE int_col WHEN 1 THEN
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID ELSE '2' END FROM t1");
// Arithmetic
/*
java.lang.IllegalArgumentException: Cannot infer return type for +;
operand types: [INTEGER, UUID]
at
org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:541)
at
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:504)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:605)
at
org.apache.calcite.sql.SqlBinaryOperator.deriveType(SqlBinaryOperator.java:178)
*/
sql("SELECT 1 + '46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
// Runtime: Cannot convert 46138242-b771-4d8b-ad26-2b3fcee5f11d to int
sql("SELECT 1 IN ('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
// Line 8, Column 281: Cannot cast "java.util.UUID" to
"org.apache.calcite.avatica.util.ByteString"
//Caused by: org.codehaus.commons.compiler.CompileException: Line 8,
Column 281: Cannot cast "java.util.UUID" to
"org.apache.calcite.avatica.util.ByteString"
//at
org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:13014)
sql("SELECT x'010203' IN
('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
{code}
This happens because ANY type is used as a type family for UUID/custom data
types and ANY can be cast to/from all other data types see SqlTypeUtil and
other calcite classes.
This can be fixed for all custom data types by adding a special checks for ANY
type.
When fixing INSERT statement we should remember that calcite's DEFAULT
expression has ANY type as well.
was:
Type compatibility checks for UUID in some expression/statement are not
performed at validation stage:
INTEGER vs UUID:
{code:java}
@Test
public void testInsert() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// / class java.util.UUID cannot be cast to class java.lang.Integer
sql("INSERT INTO t1 VALUES(1,
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
@Test
public void testUpdate() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
sql("INSERT INTO t1 VALUES(1, 1)");
// class java.util.UUID cannot be cast to class java.lang.Integer
sql("UPDATE t1 SET int_col
='46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
}
{code}
INTEGER vs VARBINARY
{code:java}
@Test
public void testInsert2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// Column 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("INSERT INTO t1 VALUES(1, x'010203')");
}
@Test
public void testUpdate2() {
sql("CREATE TABLE t1
[jira] [Commented] (IGNITE-19255) Fix broken unit tests in distribution-zones module
[
https://issues.apache.org/jira/browse/IGNITE-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723428#comment-17723428
]
Alexander Lapin commented on IGNITE-19255:
--
[~Sergey Uttsel] LGTM, thanks!
> Fix broken unit tests in distribution-zones module
> --
>
> Key: IGNITE-19255
> URL: https://issues.apache.org/jira/browse/IGNITE-19255
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksandr Polovtcev
>Assignee: Sergey Uttsel
>Priority: Blocker
> Labels: ignite-3
>
> In IGNITE-19105 I've changed some internal shenanigans of the
> MetaStorageManager (without affecting its API in any way). After that, nearly
> all unit tests in the {{distribution-zones}} module started to fail. Turns
> out it happened because of extensive mock usages that emulate behavior of the
> Meta Storage. So I decided to replace it with the
> {{StandaloneMetaStorageManager}} implementation and all hell broke loose:
> many tests emulate Meta Storage incorrectly, a lot of races appeared, because
> many methods became truly asynchronous.
> This situation is very frustrating: a different component internals were
> changed with no API changes and a completely unrelated module is not longer
> able to pass its tests. Though I fixed most of the failures, some tests are
> still failing and I'm going to try to describe, what's wrong with them:
> *{{DistributionZoneManagerScaleUpTest#testDataNodesPropagationAfterScaleUpTriggeredOnNewCluster}}*
> - this test tests a scenario when we start a node after logical topology was
> updated. I don't know how realistic is this scenario, but the problem is that
> "data nodes" don't get populated with the logical topology nodes on
> {{distributionZoneManager}} start, because {{scheduleTimers}} method, that
> get's invoked from the Meta Storage Watch, doesn't go inside the {{if
> (!addedNodes.isEmpty() && autoAdjustScaleUp != INFINITE_TIMER_VALUE)}} branch.
> *{{DistributionZoneManagerScaleUpTest#testDataNodesPropagationForDefaultZoneAfterScaleUpTriggered}}*
> - same issue as above.
> *{{DistributionZoneManagerScaleUpTest#testDataNodesPropagationForDefaultZoneAfterScaleDownTriggered}}*
> - same issue as above.
> *{{DistributionZoneManagerScaleUpTest#testUpdateZoneScaleUpTriggersDataNodePropagation}}*
> - this test fails with the following assertion error: {_}Expected revision
> that is greater or equal to already seen meta storage events.{_}. This is
> because TestConfigurationStorage does not use the same revision as the Meta
> Storage, therefore their revisions can't be compared directly. This should
> either be converted to an integration test or it should use
> `DistributedConfigurationStrorage` instead.
> (ticket is created https://issues.apache.org/jira/browse/IGNITE-19342)
> *{{DistributionZoneManagerScaleUpTest#testUpdateZoneScaleDownTriggersDataNodePropagation}}*
> - same issue as above. (ticket is created
> https://issues.apache.org/jira/browse/IGNITE-19342)
> *{{DistributionZoneManagerScaleUpTest#testDropZoneDoNotPropagateDataNodesAfterScaleUp}}*
> - this test is flaky, because notifications from test configuration storage
> and from Meta Storage Watches are not related to each other (unlike real-life
> Distributed Configuration Storage which is built on top of Watches), so
> notifications from the configuration storage and Meta Storage can arrive in a
> undetermined order. (ticket is created
> https://issues.apache.org/jira/browse/IGNITE-19342)
> *{{DistributionZoneManagerScaleUpTest#testDropZoneDoNotPropagateDataNodesAfterScaleDown}}*
> - same issue as above.
> (ticket is created https://issues.apache.org/jira/browse/IGNITE-19342)
> *{{DistributionZoneManagerWatchListenerTest#testDataNodesOfDefaultZoneUpdatedOnWatchListenerEvent}}*
> - this test is flaky, probably due to some races between Watch and
> Configuration Listener execution (sometimes a retry on {{invoke}} happens and
> {{Mockito#verify}} fails). (ticket is created
> https://issues.apache.org/jira/browse/IGNITE-19342)
>
> *New tests* from
> [https://github.com/gridgain/apache-ignite-3/tree/ignite-18756]
> *DistributionZoneAwaitDataNodesTest#testRemoveZoneWhileAwaitingDataNodes* -
> this test must remove the zone after MetastorageTopologyListener updates the
> topVerTracker and before
> MetastorageDataNodesListener updates
> scaleUpRevisionTracker/scaleDownRevisionTracker. Now it's impossible to do it
> with StandaloneMetaStorageManager.
> (https://issues.apache.org/jira/browse/IGNITE-19343)
> *DistributionZoneAwaitDataNodesTest#testScaleUpScaleDownAreChangedWhileAwaitingDataNodes*
> - same issue as above but here we need to update scaleUp and scaleDown
> instead of removing the zone.
> (https://issues.apache.org/jira/browse/IGNITE-19343)
--
This message was
[jira] [Updated] (IGNITE-19255) Fix broken unit tests in distribution-zones module
[
https://issues.apache.org/jira/browse/IGNITE-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-19255:
-
Reviewer: Alexander Lapin
> Fix broken unit tests in distribution-zones module
> --
>
> Key: IGNITE-19255
> URL: https://issues.apache.org/jira/browse/IGNITE-19255
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksandr Polovtcev
>Assignee: Sergey Uttsel
>Priority: Blocker
> Labels: ignite-3
>
> In IGNITE-19105 I've changed some internal shenanigans of the
> MetaStorageManager (without affecting its API in any way). After that, nearly
> all unit tests in the {{distribution-zones}} module started to fail. Turns
> out it happened because of extensive mock usages that emulate behavior of the
> Meta Storage. So I decided to replace it with the
> {{StandaloneMetaStorageManager}} implementation and all hell broke loose:
> many tests emulate Meta Storage incorrectly, a lot of races appeared, because
> many methods became truly asynchronous.
> This situation is very frustrating: a different component internals were
> changed with no API changes and a completely unrelated module is not longer
> able to pass its tests. Though I fixed most of the failures, some tests are
> still failing and I'm going to try to describe, what's wrong with them:
> *{{DistributionZoneManagerScaleUpTest#testDataNodesPropagationAfterScaleUpTriggeredOnNewCluster}}*
> - this test tests a scenario when we start a node after logical topology was
> updated. I don't know how realistic is this scenario, but the problem is that
> "data nodes" don't get populated with the logical topology nodes on
> {{distributionZoneManager}} start, because {{scheduleTimers}} method, that
> get's invoked from the Meta Storage Watch, doesn't go inside the {{if
> (!addedNodes.isEmpty() && autoAdjustScaleUp != INFINITE_TIMER_VALUE)}} branch.
> *{{DistributionZoneManagerScaleUpTest#testDataNodesPropagationForDefaultZoneAfterScaleUpTriggered}}*
> - same issue as above.
> *{{DistributionZoneManagerScaleUpTest#testDataNodesPropagationForDefaultZoneAfterScaleDownTriggered}}*
> - same issue as above.
> *{{DistributionZoneManagerScaleUpTest#testUpdateZoneScaleUpTriggersDataNodePropagation}}*
> - this test fails with the following assertion error: {_}Expected revision
> that is greater or equal to already seen meta storage events.{_}. This is
> because TestConfigurationStorage does not use the same revision as the Meta
> Storage, therefore their revisions can't be compared directly. This should
> either be converted to an integration test or it should use
> `DistributedConfigurationStrorage` instead.
> (ticket is created https://issues.apache.org/jira/browse/IGNITE-19342)
> *{{DistributionZoneManagerScaleUpTest#testUpdateZoneScaleDownTriggersDataNodePropagation}}*
> - same issue as above. (ticket is created
> https://issues.apache.org/jira/browse/IGNITE-19342)
> *{{DistributionZoneManagerScaleUpTest#testDropZoneDoNotPropagateDataNodesAfterScaleUp}}*
> - this test is flaky, because notifications from test configuration storage
> and from Meta Storage Watches are not related to each other (unlike real-life
> Distributed Configuration Storage which is built on top of Watches), so
> notifications from the configuration storage and Meta Storage can arrive in a
> undetermined order. (ticket is created
> https://issues.apache.org/jira/browse/IGNITE-19342)
> *{{DistributionZoneManagerScaleUpTest#testDropZoneDoNotPropagateDataNodesAfterScaleDown}}*
> - same issue as above.
> (ticket is created https://issues.apache.org/jira/browse/IGNITE-19342)
> *{{DistributionZoneManagerWatchListenerTest#testDataNodesOfDefaultZoneUpdatedOnWatchListenerEvent}}*
> - this test is flaky, probably due to some races between Watch and
> Configuration Listener execution (sometimes a retry on {{invoke}} happens and
> {{Mockito#verify}} fails). (ticket is created
> https://issues.apache.org/jira/browse/IGNITE-19342)
>
> *New tests* from
> [https://github.com/gridgain/apache-ignite-3/tree/ignite-18756]
> *DistributionZoneAwaitDataNodesTest#testRemoveZoneWhileAwaitingDataNodes* -
> this test must remove the zone after MetastorageTopologyListener updates the
> topVerTracker and before
> MetastorageDataNodesListener updates
> scaleUpRevisionTracker/scaleDownRevisionTracker. Now it's impossible to do it
> with StandaloneMetaStorageManager.
> (https://issues.apache.org/jira/browse/IGNITE-19343)
> *DistributionZoneAwaitDataNodesTest#testScaleUpScaleDownAreChangedWhileAwaitingDataNodes*
> - same issue as above but here we need to update scaleUp and scaleDown
> instead of removing the zone.
> (https://issues.apache.org/jira/browse/IGNITE-19343)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19503) Sql. UUID. Some expressions and statements with types from incompatible families are not rejected.
[
https://issues.apache.org/jira/browse/IGNITE-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-19503:
--
Description:
Type compatibility checks for UUID in some expression/statement are not
performed at validation stage:
INTEGER vs UUID:
{code:java}
@Test
public void testInsert() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// / class java.util.UUID cannot be cast to class java.lang.Integer
sql("INSERT INTO t1 VALUES(1,
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
@Test
public void testUpdate() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
sql("INSERT INTO t1 VALUES(1, 1)");
// class java.util.UUID cannot be cast to class java.lang.Integer
sql("UPDATE t1 SET int_col
='46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
}
{code}
INTEGER vs VARBINARY
{code:java}
@Test
public void testInsert2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// Column 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("INSERT INTO t1 VALUES(1, x'010203')");
}
@Test
public void testUpdate2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
//sql("INSERT INTO t1 VALUES(1, 1)");
// olumn 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("UPDATE t1 SET int_col = x'010203'");
}
{code}
Expressions:
{code:java}
@Test
public void testExprs() {
//sql("SELECT 1 in ('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
// Binary comparison operations are rejected by validator: OK
// From line 1, column 1 to line 1, column 55: Invalid types for
comparison: INTEGER NOT NULL = UUID NOT NULL
sql("SELECT 1 = '46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
// CASE: OK
// Illegal mixing of types in CASE or COALESCE statement
sql("SELECT CASE int_col WHEN 1 THEN
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID ELSE '2' END FROM t1");
// Arithmetic
/*
java.lang.IllegalArgumentException: Cannot infer return type for +;
operand types: [INTEGER, UUID]
at
org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:541)
at
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:504)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:605)
at
org.apache.calcite.sql.SqlBinaryOperator.deriveType(SqlBinaryOperator.java:178)
*/
sql("SELECT 1 + '46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
// Runtime: Cannot convert 46138242-b771-4d8b-ad26-2b3fcee5f11d to int
sql("SELECT 1 IN ('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
// Line 8, Column 281: Cannot cast "java.util.UUID" to
"org.apache.calcite.avatica.util.ByteString"
//Caused by: org.codehaus.commons.compiler.CompileException: Line 8,
Column 281: Cannot cast "java.util.UUID" to
"org.apache.calcite.avatica.util.ByteString"
//at
org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:13014)
sql("SELECT x'010203' IN
('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
{code}
This happens because ANY type is used as a type family for UUID/a custom data
type and ANY can be cast to/from all other data types see SqlTypeUtil and other
calcite classes.
This can be fixed for all custom data types by adding a special checks for ANY
type.
When fixing INSERT statement we should remember that calcite's DEFAULT
expression has ANY type as well.
was:
Type compatibility checks for UUID in some expression/statement are not
performed at validation stage:
INTEGER vs UUID:
{code:java}
@Test
public void testInsert() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// / class java.util.UUID cannot be cast to class java.lang.Integer
sql("INSERT INTO t1 VALUES(1,
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
@Test
public void testUpdate() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
sql("INSERT INTO t1 VALUES(1, 1)");
// class java.util.UUID cannot be cast to class java.lang.Integer
sql("UPDATE t1 SET int_col
='46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
}
{code}
INTEGER vs VARBINARY
{code:java}
@Test
public void testInsert2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// Column 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("INSERT INTO t1 VALUES(1, x'010203')");
}
@Test
public void testUpdate2() {
sql("CREATE TABLE t1
[jira] [Comment Edited] (IGNITE-19307) Sql. Set operations: Conversion to relational algebra failed to preserve datatypes.
[
https://issues.apache.org/jira/browse/IGNITE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723421#comment-17723421
]
Maksim Zhuravkov edited comment on IGNITE-19307 at 5/17/23 11:41 AM:
-
This is issue for set operations is also mentioned in
https://issues.apache.org/jira/browse/CALCITE-5130
was (Author: JIRAUSER298618):
This is issue is by caused by https://issues.apache.org/jira/browse/CALCITE-5130
> Sql. Set operations: Conversion to relational algebra failed to preserve
> datatypes.
> ---
>
> Key: IGNITE-19307
> URL: https://issues.apache.org/jira/browse/IGNITE-19307
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Minor
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> The following query fails with "Conversion to relational algebra failed to
> preserve datatypes"
> {code:java}
> @Test
> public void test() {
>sql("CREATE TABLE tt (id integer primary key, test_key integer)");
>sql("INSERT INTO tt VALUES(1, 100)");
>assertQuery("SELECT test_key FROM tt UNION ALL SELECT '1000'").check();
> }
> {code}
> Although the same expression with literal works as expected:
> {code:java}
> @Test
> public void test2() {
>assertQuery("SELECT 100 UNION ALL SELECT '1000'").check();
> }
> {code}
> Error:
> {code:java}
> java.lang.AssertionError: Conversion to relational algebra failed to preserve
> datatypes:
> validated type:
> RecordType(VARCHAR CHARACTER SET "UTF-8" NOT NULL TEST_KEY) NOT NULL
> converted type:
> RecordType(VARCHAR CHARACTER SET "UTF-8" EXPR$0) NOT NULL
> rel:
> LogicalUnion(all=[true])
> LogicalProject(EXPR$0=[CAST($1):VARCHAR CHARACTER SET "UTF-8"])
> IgniteLogicalTableScan(table=[[PUBLIC, TT]])
> LogicalValues(tuples=[[{ _UTF-8'1000' }]])
> {code}
> When both side of set operation are columns query also works:
> {code:java}
> @Test
> public void test3() {
> sql("CREATE TABLE tt (id integer primary key, test_key integer)");
> sql("CREATE TABLE ttt (id integer primary key, test_key varchar)");
> sql("INSERT INTO tt VALUES(1, 100)");
> sql("INSERT INTO ttt VALUES(1, '200')");
> assertQuery("SELECT test_key FROM tt UNION ALL SELECT test_key FROM
> ttt").check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-19307) Sql. Set operations: Conversion to relational algebra failed to preserve datatypes.
[
https://issues.apache.org/jira/browse/IGNITE-19307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723421#comment-17723421
]
Maksim Zhuravkov commented on IGNITE-19307:
---
This is issue is by caused by https://issues.apache.org/jira/browse/CALCITE-5130
> Sql. Set operations: Conversion to relational algebra failed to preserve
> datatypes.
> ---
>
> Key: IGNITE-19307
> URL: https://issues.apache.org/jira/browse/IGNITE-19307
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Minor
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> The following query fails with "Conversion to relational algebra failed to
> preserve datatypes"
> {code:java}
> @Test
> public void test() {
>sql("CREATE TABLE tt (id integer primary key, test_key integer)");
>sql("INSERT INTO tt VALUES(1, 100)");
>assertQuery("SELECT test_key FROM tt UNION ALL SELECT '1000'").check();
> }
> {code}
> Although the same expression with literal works as expected:
> {code:java}
> @Test
> public void test2() {
>assertQuery("SELECT 100 UNION ALL SELECT '1000'").check();
> }
> {code}
> Error:
> {code:java}
> java.lang.AssertionError: Conversion to relational algebra failed to preserve
> datatypes:
> validated type:
> RecordType(VARCHAR CHARACTER SET "UTF-8" NOT NULL TEST_KEY) NOT NULL
> converted type:
> RecordType(VARCHAR CHARACTER SET "UTF-8" EXPR$0) NOT NULL
> rel:
> LogicalUnion(all=[true])
> LogicalProject(EXPR$0=[CAST($1):VARCHAR CHARACTER SET "UTF-8"])
> IgniteLogicalTableScan(table=[[PUBLIC, TT]])
> LogicalValues(tuples=[[{ _UTF-8'1000' }]])
> {code}
> When both side of set operation are columns query also works:
> {code:java}
> @Test
> public void test3() {
> sql("CREATE TABLE tt (id integer primary key, test_key integer)");
> sql("CREATE TABLE ttt (id integer primary key, test_key varchar)");
> sql("INSERT INTO tt VALUES(1, 100)");
> sql("INSERT INTO ttt VALUES(1, '200')");
> assertQuery("SELECT test_key FROM tt UNION ALL SELECT test_key FROM
> ttt").check();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19503) Sql. UUID. Some expressions and statements with types from incompatible families are not rejected.
Maksim Zhuravkov created IGNITE-19503:
-
Summary: Sql. UUID. Some expressions and statements with types
from incompatible families are not rejected.
Key: IGNITE-19503
URL: https://issues.apache.org/jira/browse/IGNITE-19503
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: 3.0.0-beta1
Reporter: Maksim Zhuravkov
Type compatibility checks for UUID in some expression/statement are not
performed at validation stage:
INTEGER vs UUID:
{code:java}
@Test
public void testInsert() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// / class java.util.UUID cannot be cast to class java.lang.Integer
sql("INSERT INTO t1 VALUES(1,
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
@Test
public void testUpdate() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
sql("INSERT INTO t1 VALUES(1, 1)");
// class java.util.UUID cannot be cast to class java.lang.Integer
sql("UPDATE t1 SET int_col
='46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
}
{code}
INTEGER vs VARBINARY
{code:java}
@Test
public void testInsert2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
// Column 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("INSERT INTO t1 VALUES(1, x'010203')");
}
@Test
public void testUpdate2() {
sql("CREATE TABLE t1 (key INTEGER PRIMARY KEY , int_col INTEGER)");
//sql("INSERT INTO t1 VALUES(1, 1)");
// olumn 33: Cannot assign to target field 'INT_COL' of type INTEGER
from source field 'EXPR$0' of type BINARY(3)
sql("UPDATE t1 SET int_col = x'010203'");
}
{code}
Expressions:
{code:java}
@Test
public void testExprs() {
//sql("SELECT 1 in ('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
// Binary comparison operations are rejected by validator: OK
// From line 1, column 1 to line 1, column 55: Invalid types for
comparison: INTEGER NOT NULL = UUID NOT NULL
sql("SELECT 1 = '46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
// CASE: OK
// Illegal mixing of types in CASE or COALESCE statement
sql("SELECT CASE int_col WHEN 1 THEN
'46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID ELSE '2' END FROM t1");
// Arithmetic
/*
java.lang.IllegalArgumentException: Cannot infer return type for +;
operand types: [INTEGER, UUID]
at
org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:541)
at
org.apache.calcite.sql.SqlOperator.validateOperands(SqlOperator.java:504)
at org.apache.calcite.sql.SqlOperator.deriveType(SqlOperator.java:605)
at
org.apache.calcite.sql.SqlBinaryOperator.deriveType(SqlBinaryOperator.java:178)
*/
sql("SELECT 1 + '46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID");
// Runtime: Cannot convert 46138242-b771-4d8b-ad26-2b3fcee5f11d to int
sql("SELECT 1 IN ('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
// Line 8, Column 281: Cannot cast "java.util.UUID" to
"org.apache.calcite.avatica.util.ByteString"
//Caused by: org.codehaus.commons.compiler.CompileException: Line 8,
Column 281: Cannot cast "java.util.UUID" to
"org.apache.calcite.avatica.util.ByteString"
//at
org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:13014)
sql("SELECT x'010203' IN
('46138242-b771-4d8b-ad26-2b3fcee5f11d'::UUID)");
}
{code}
This happens because ANY type is used as a type family for UUID/a custom data
type.
This can be fixed for all custom data types by adding a special checks for ANY
type.
When fixing INSERT statement we should remember that calcite's DEFAULT
expression has ANY type as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-19407) Sql. Introduce DDL command for ALTER COLUMN TYPE.
[
https://issues.apache.org/jira/browse/IGNITE-19407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin reassigned IGNITE-19407:
-
Assignee: Pavel Pereslegin
> Sql. Introduce DDL command for ALTER COLUMN TYPE.
> -
>
> Key: IGNITE-19407
> URL: https://issues.apache.org/jira/browse/IGNITE-19407
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Andrey Mashenkov
>Assignee: Pavel Pereslegin
>Priority: Major
> Labels: ignite-3
>
> Introduce DDL command class.
> Implement command validation logic.
> Pass command to the CatalogService via DdlHandler.
> Validator should allow changing column DEFAULT and changing column type for
> the next cases
> {noformat}
> INT8 -> INT16 -> INT32 -> INT64
> FLOAT -> DOUBLE
> DECIMAL(10,5) -> DECIMAL(15, 5) // increasing precision only
> VARCHAR(10) -> VARCHAR(20)
> VARBINARY(10) -> VARBINARY(20)
>NOT NULL -> NULLABLE
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-19483) Transform TableManager and IndexManager to internally work against Catalog event types
[ https://issues.apache.org/jira/browse/IGNITE-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy reassigned IGNITE-19483: -- Assignee: Roman Puchkovskiy > Transform TableManager and IndexManager to internally work against Catalog > event types > -- > > Key: IGNITE-19483 > URL: https://issues.apache.org/jira/browse/IGNITE-19483 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Currently, when an event like 'table was added to configuration' happens, the > listener pools the table config by itself and then uses it to create the > table. > This should be changed: the table configuration object should be converted to > an object from Catalog domain and pushed to the listeners. > Same should be done to indices. > Requires investigation. > Also, we need to stop passing configuration to deeply-nested components (like > storages). Also requires investigation. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19496) Sql. Implement SqlSchemaManager on top of CatalogService
[ https://issues.apache.org/jira/browse/IGNITE-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19496: -- Summary: Sql. Implement SqlSchemaManager on top of CatalogService (was: Implement SqlSchemaManager on top of CatalogService) > Sql. Implement SqlSchemaManager on top of CatalogService > > > Key: IGNITE-19496 > URL: https://issues.apache.org/jira/browse/IGNITE-19496 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, SqlSchemaManager listens configuration events to compose a new > schema, and does not support schema versioning. > Let's implement SqlSchemaManager on top of CatalogService, which is already > aware of schema versions. > The new implementation may be just a caching adapter for CatalogService, > which enriches Catalog with metadata that can be used by Calcite engine. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19497) Sql. Switch to SqlSchemaManager implementation on top of CatalogService
[ https://issues.apache.org/jira/browse/IGNITE-19497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19497: -- Summary: Sql. Switch to SqlSchemaManager implementation on top of CatalogService (was: Switch to SqlSchemaManager implementation on top of CatalogService ) > Sql. Switch to SqlSchemaManager implementation on top of CatalogService > > > Key: IGNITE-19497 > URL: https://issues.apache.org/jira/browse/IGNITE-19497 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > Let's switch to a new SqlSchemaManager from IGNITE-19496, which uses Catalog > events, and drop old implementation. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19492) Sql. Support rename operation for Catalog objects
[ https://issues.apache.org/jira/browse/IGNITE-19492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19492: -- Summary: Sql. Support rename operation for Catalog objects (was: Support rename operation for Catalog objects.) > Sql. Support rename operation for Catalog objects > - > > Key: IGNITE-19492 > URL: https://issues.apache.org/jira/browse/IGNITE-19492 > Project: Ignite > Issue Type: Task > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > Umbrella ticket -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-19488) RemoteFragmentExecutionException when inserting more than 30 000 rows into one table
[
https://issues.apache.org/jira/browse/IGNITE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723404#comment-17723404
]
Pavel Pereslegin edited comment on IGNITE-19488 at 5/17/23 10:29 AM:
-
[~lunigorn], please describe the table you created in more detail, and even
better, attach a complete reproducer, which you used to catch this issue.
was (Author: xtern):
[~lunigorn], please describe the table you created in more detail, and even
better, attach a complete reproducer for this issue.
> RemoteFragmentExecutionException when inserting more than 30 000 rows into
> one table
>
>
> Key: IGNITE-19488
> URL: https://issues.apache.org/jira/browse/IGNITE-19488
> Project: Ignite
> Issue Type: Bug
> Components: jdbc, sql
>Reporter: Igor
>Priority: Critical
> Labels: ignite-3
> Attachments: logs.zip
>
>
> h1. Steps to reproduce
> Ignite 3 main branch commit 45380a6c802203dab0d72bd1eb9fb202b2a345b0
> # Create table with 5 columns
> # Insert into table rows by batches 1000 rows each batch.
> # Repeat previous step untill exception is thrown.
> h1. Expected behaviour
> Created more than 30 000 rows.
> h1. Actual behaviour
> An exception after 29 000 rows are inserted:
> {code:java}
> Exception while executing query [query=SELECT COUNT(*) FROM
> rows_capacity_table]. Error message:IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 Query remote fragment execution
> failed: nodeName=TablesAmountCapacityTest_cluster_0,
> queryId=ecd14026-5366-4ee2-b73a-f38757d3ba4f, fragmentId=1561,
> originalMessage=IGN-CMN-1 TraceId:24c93463-f078-410a-8831-36b5c549a907
> java.sql.SQLException: Exception while executing query [query=SELECT COUNT(*)
> FROM rows_capacity_table]. Error message:IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 Query remote fragment execution
> failed: nodeName=TablesAmountCapacityTest_cluster_0,
> queryId=ecd14026-5366-4ee2-b73a-f38757d3ba4f, fragmentId=1561,
> originalMessage=IGN-CMN-1 TraceId:24c93463-f078-410a-8831-36b5c549a907
> at
> org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
> at
> org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:149)
> at
> org.apache.ignite.internal.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:108)
> {code}
> Logs are in the attachment.
> [^logs.zip]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19502) Separate component to keep database schema and manage changes in it
[ https://issues.apache.org/jira/browse/IGNITE-19502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Chugunov updated IGNITE-19502: - Description: Right now database schema information in Ignite 3.0 is stored and treated as regular configuration. All business processes related to schema changes are distributed via internal mechanisms of configuration engine and orchestrated ad-hoc by consumer components like TableManager or IndexManager. But there is a strong evidence that API and guarantees of configuration engine don't satisfy important requirements that emerge in the field of db schema management. So we need to stop treating schema as any other configuration and move it to a separate component called Catalog Service. This component will provide high-level API to modify the schema and will take responsibility for all other necessary processes like synchronizing schema changes between nodes, managing schema change events, notifying all necessary components and providing orchestration between them. was: Right now database schema information in Ignite 3.0 is stored and treated as regular configuration. All business processes related to schema changes are distributed via internal mechanisms of configuration engine and orchestrated ad-hoc by consumer components like TableManager or IndexManager. But there is a strong evidence that API and guarantees of configuration engine don't satisfy important requirements that emerge in the field of db schema management. So we need to stop treating schema as any other configuration and move it to a separate component called Catalog Service. This component will provide high-level API to modify the schema and take responsibility for all other necessary processes like synchronizing schema changes between nodes, managing schema change events, notifying all necessary components and providing orchestration between them. > Separate component to keep database schema and manage changes in it > --- > > Key: IGNITE-19502 > URL: https://issues.apache.org/jira/browse/IGNITE-19502 > Project: Ignite > Issue Type: Epic > Components: persistence, sql >Reporter: Sergey Chugunov >Priority: Major > Labels: ignite-3 > > Right now database schema information in Ignite 3.0 is stored and treated as > regular configuration. All business processes related to schema changes are > distributed via internal mechanisms of configuration engine and orchestrated > ad-hoc by consumer components like TableManager or IndexManager. > But there is a strong evidence that API and guarantees of configuration > engine don't satisfy important requirements that emerge in the field of db > schema management. > So we need to stop treating schema as any other configuration and move it to > a separate component called Catalog Service. > This component will provide high-level API to modify the schema and will take > responsibility for all other necessary processes like synchronizing schema > changes between nodes, managing schema change events, notifying all necessary > components and providing orchestration between them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19502) Separate component to keep database schema and manage changes in it
Sergey Chugunov created IGNITE-19502: Summary: Separate component to keep database schema and manage changes in it Key: IGNITE-19502 URL: https://issues.apache.org/jira/browse/IGNITE-19502 Project: Ignite Issue Type: Epic Components: persistence, sql Reporter: Sergey Chugunov Right now database schema information in Ignite 3.0 is stored and treated as regular configuration. All business processes related to schema changes are distributed via internal mechanisms of configuration engine and orchestrated ad-hoc by consumer components like TableManager or IndexManager. But there is a strong evidence that API and guarantees of configuration engine don't satisfy important requirements that emerge in the field of db schema management. So we need to stop treating schema as any other configuration and move it to a separate component called Catalog Service. This component will provide high-level API to modify the schema and take responsibility for all other necessary processes like synchronizing schema changes between nodes, managing schema change events, notifying all necessary components and providing orchestration between them. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19488) RemoteFragmentExecutionException when inserting more than 30 000 rows into one table
[
https://issues.apache.org/jira/browse/IGNITE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723404#comment-17723404
]
Pavel Pereslegin commented on IGNITE-19488:
---
[~lunigorn], please describe the table you created in more detail, and even
better, attach a complete reproducer for this issue.
> RemoteFragmentExecutionException when inserting more than 30 000 rows into
> one table
>
>
> Key: IGNITE-19488
> URL: https://issues.apache.org/jira/browse/IGNITE-19488
> Project: Ignite
> Issue Type: Bug
> Components: jdbc, sql
>Reporter: Igor
>Priority: Critical
> Labels: ignite-3
> Attachments: logs.zip
>
>
> h1. Steps to reproduce
> Ignite 3 main branch commit 45380a6c802203dab0d72bd1eb9fb202b2a345b0
> # Create table with 5 columns
> # Insert into table rows by batches 1000 rows each batch.
> # Repeat previous step untill exception is thrown.
> h1. Expected behaviour
> Created more than 30 000 rows.
> h1. Actual behaviour
> An exception after 29 000 rows are inserted:
> {code:java}
> Exception while executing query [query=SELECT COUNT(*) FROM
> rows_capacity_table]. Error message:IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 Query remote fragment execution
> failed: nodeName=TablesAmountCapacityTest_cluster_0,
> queryId=ecd14026-5366-4ee2-b73a-f38757d3ba4f, fragmentId=1561,
> originalMessage=IGN-CMN-1 TraceId:24c93463-f078-410a-8831-36b5c549a907
> java.sql.SQLException: Exception while executing query [query=SELECT COUNT(*)
> FROM rows_capacity_table]. Error message:IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 IGN-CMN-1
> TraceId:24c93463-f078-410a-8831-36b5c549a907 Query remote fragment execution
> failed: nodeName=TablesAmountCapacityTest_cluster_0,
> queryId=ecd14026-5366-4ee2-b73a-f38757d3ba4f, fragmentId=1561,
> originalMessage=IGN-CMN-1 TraceId:24c93463-f078-410a-8831-36b5c549a907
> at
> org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
> at
> org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:149)
> at
> org.apache.ignite.internal.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:108)
> {code}
> Logs are in the attachment.
> [^logs.zip]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19460) Implement missed DDL commands with using Catalog
[ https://issues.apache.org/jira/browse/IGNITE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19460: -- Description: Earlier in IGNITE-19081 we introduce Catalog events. Let's add table altering, index creation/dropping commands and them to the CatalogManager. was: Earlier in IGNITE-19081 we introduce Catalog events. Let's support table creation/dropping with using Catalog events. > Implement missed DDL commands with using Catalog > - > > Key: IGNITE-19460 > URL: https://issues.apache.org/jira/browse/IGNITE-19460 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Earlier in IGNITE-19081 we introduce Catalog events. > Let's add table altering, index creation/dropping commands and them to the > CatalogManager. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19460) Implement missed DDL commands with using Catalog
[ https://issues.apache.org/jira/browse/IGNITE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19460: -- Summary: Implement missed DDL commands with using Catalog (was: Implement DDL command with using Catalog events.) > Implement missed DDL commands with using Catalog > - > > Key: IGNITE-19460 > URL: https://issues.apache.org/jira/browse/IGNITE-19460 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Earlier in IGNITE-19081 we introduce Catalog events. > Let's support table creation/dropping with using Catalog events. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19082) Describe Catalog operation flow in README and cleanup dead code.
[ https://issues.apache.org/jira/browse/IGNITE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov reassigned IGNITE-19082: - Assignee: (was: Andrey Mashenkov) > Describe Catalog operation flow in README and cleanup dead code. > > > Key: IGNITE-19082 > URL: https://issues.apache.org/jira/browse/IGNITE-19082 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Let's > * ensure Catalog is used by default as a single schema management point by > TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. > * Describe CatalogService operations in README.md > * drop schema related code from configuration. > * drop outdated code from TableManager, IndexManager, SqlSchemaManager, > SchemaRegistry. > * make a PR for merging feature branch to main (if applicable). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19082) Describe Catalog operation flow in README and cleanup dead code.
[ https://issues.apache.org/jira/browse/IGNITE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19082: -- Description: Let's * ensure Catalog is used by default as a single schema management point by TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * Describe CatalogService operations in README.md * drop schema related code from configuration. * drop outdated code from TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * make a PR for merging feature branch to main (if applicable). was: Let's * use Catalog by default as a single schema management point. * switch TableManager, IndexManager, SqlSchemaManager, SchemaRegistry to implementations that uses Catalog events. * drop schema related code from configuration. * drop outdated code from TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. * describe Catalog flow in README.md * make a PR for merging feature branch to main (if applicable). > Describe Catalog operation flow in README and cleanup dead code. > > > Key: IGNITE-19082 > URL: https://issues.apache.org/jira/browse/IGNITE-19082 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Let's > * ensure Catalog is used by default as a single schema management point by > TableManager, IndexManager, SqlSchemaManager, SchemaRegistry. > * Describe CatalogService operations in README.md > * drop schema related code from configuration. > * drop outdated code from TableManager, IndexManager, SqlSchemaManager, > SchemaRegistry. > * make a PR for merging feature branch to main (if applicable). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19082) Describe Catalog operation flow in README and cleanup dead code.
[ https://issues.apache.org/jira/browse/IGNITE-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19082: -- Summary: Describe Catalog operation flow in README and cleanup dead code. (was: Use Catalog for schema management and cleanup dead code.) > Describe Catalog operation flow in README and cleanup dead code. > > > Key: IGNITE-19082 > URL: https://issues.apache.org/jira/browse/IGNITE-19082 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Let's > * use Catalog by default as a single schema management point. > * switch TableManager, IndexManager, SqlSchemaManager, SchemaRegistry to > implementations that uses Catalog events. > * drop schema related code from configuration. > * drop outdated code from TableManager, IndexManager, SqlSchemaManager, > SchemaRegistry. > * describe Catalog flow in README.md > * make a PR for merging feature branch to main (if applicable). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19501) SchemaManager should use CatalogService for building BinaryRow descriptors
Andrey Mashenkov created IGNITE-19501: - Summary: SchemaManager should use CatalogService for building BinaryRow descriptors Key: IGNITE-19501 URL: https://issues.apache.org/jira/browse/IGNITE-19501 Project: Ignite Issue Type: Improvement Reporter: Andrey Mashenkov As of now, SchemaManager uses configuration data to create BinaryRow/BinaryTuple descriptors. Let's make SchemaManager and SchemaRegistry using Catalog data instead. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19454) Validate backwards schema compatibility
[ https://issues.apache.org/jira/browse/IGNITE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Puchkovskiy updated IGNITE-19454: --- Reviewer: Semyon Danilov [~sdanilov] could you please take a look? > Validate backwards schema compatibility > --- > > Key: IGNITE-19454 > URL: https://issues.apache.org/jira/browse/IGNITE-19454 > Project: Ignite > Issue Type: Improvement >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > See description in IGNITE-19228 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19499) TableManager should listen CatalogService events instead of configuration
[ https://issues.apache.org/jira/browse/IGNITE-19499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19499: -- Summary: TableManager should listen CatalogService events instead of configuration (was: TableManager should listen CatalogSevice events instead of configuration) > TableManager should listen CatalogService events instead of configuration > - > > Key: IGNITE-19499 > URL: https://issues.apache.org/jira/browse/IGNITE-19499 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, TableManager listens configuration events to create internal > structures. > Let's make TableManager listens CatalogService events instead. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19499) TableManager should listen CatalogSevice events instead of configuration
[ https://issues.apache.org/jira/browse/IGNITE-19499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19499: -- Description: As of now, TableManager listens configuration events to create internal structures. Let's make TableManager listens CatalogService events instead. Note: Some tests may fails due to changed guarantees and related ticked incompletion. So, let's do this in a separate feature branch. was: As of now, SqlSchemaManager listens configuration events to creates internal structures. Let's TableManager listens CatalogService events instead. Note: Some tests may fails due to changed guarantees and related ticked incompletion. So, let's do this in a separate feature branch. > TableManager should listen CatalogSevice events instead of configuration > > > Key: IGNITE-19499 > URL: https://issues.apache.org/jira/browse/IGNITE-19499 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, TableManager listens configuration events to create internal > structures. > Let's make TableManager listens CatalogService events instead. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19393) Java thin 3.0: testAccessLockedKeyTimesOut is flaky
[
https://issues.apache.org/jira/browse/IGNITE-19393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-19393:
-
Priority: Blocker (was: Major)
> Java thin 3.0: testAccessLockedKeyTimesOut is flaky
> ---
>
> Key: IGNITE-19393
> URL: https://issues.apache.org/jira/browse/IGNITE-19393
> Project: Ignite
> Issue Type: Bug
> Components: thin client
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Alexander Lapin
>Priority: Blocker
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> java.lang.AssertionError:
> Expected: a string containing "Replication is timed out"
> but: was "IGN-TX-4 TraceId:d54813d7-793a-4d81-9456-b7140cee2209 IGN-TX-4
> TraceId:d54813d7-793a-4d81-9456-b7140cee2209 IGN-TX-4
> TraceId:d54813d7-793a-4d81-9456-b7140cee2209 Failed to acquire a lock due to
> a conflict [txId=0187-db61-2a99--bb66f319,
> conflictingWaiter=WaiterImpl [txId=0187-db61-2a99--bb66f318,
> intendedLockMode=null, lockMode=X, ex=null, isDone=true]]"
> java.lang.AssertionError:
> Expected: a string containing "Replication is timed out"
> but: was "IGN-TX-4 TraceId:d54813d7-793a-4d81-9456-b7140cee2209 IGN-TX-4
> TraceId:d54813d7-793a-4d81-9456-b7140cee2209 IGN-TX-4
> TraceId:d54813d7-793a-4d81-9456-b7140cee2209 Failed to acquire a lock due to
> a conflict [txId=0187-db61-2a99--bb66f319,
> conflictingWaiter=WaiterImpl [txId=0187-db61-2a99--bb66f318,
> intendedLockMode=null, lockMode=X, ex=null, isDone=true]]"
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
> at
> org.apache.ignite.internal.runner.app.client.ItThinClientTransactionsTest.testAccessLockedKeyTimesOut(ItThinClientTransactionsTest.java:200)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
> at
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
> at
> org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
> at
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
> at
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
> at
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
> at
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
> at
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
> at
>
[jira] [Updated] (IGNITE-19500) IndexManager should listen CatalogService events instead of configuration
[ https://issues.apache.org/jira/browse/IGNITE-19500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19500: -- Description: As of now, IndexManager listens configuration events to create internal structures. Let's make IndexManager listens CatalogService events instead. Note: Some tests may fails due to changed guarantees and related ticked incompletion. So, let's do this in a separate feature branch. was: As of now, IndexManager listens configuration events to create internal structures. Let's IndexManager listens CatalogService events instead. Note: Some tests may fails due to changed guarantees and related ticked incompletion. So, let's do this in a separate feature branch. > IndexManager should listen CatalogService events instead of configuration > - > > Key: IGNITE-19500 > URL: https://issues.apache.org/jira/browse/IGNITE-19500 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, IndexManager listens configuration events to create internal > structures. > Let's make IndexManager listens CatalogService events instead. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19500) IndexManager should listen CatalogService events instead of configuration
[ https://issues.apache.org/jira/browse/IGNITE-19500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19500: -- Description: As of now, IndexManager listens configuration events to create internal structures. Let's IndexManager listens CatalogService events instead. Note: Some tests may fails due to changed guarantees and related ticked incompletion. So, let's do this in a separate feature branch. was: As of now, SqlSchemaManager listens configuration events to creates internal structures. Let's TableManager listens CatalogService events instead. Note: Some tests may fails due to changed guarantees and related ticked incompletion. So, let's do this in a separate feature branch. > IndexManager should listen CatalogService events instead of configuration > - > > Key: IGNITE-19500 > URL: https://issues.apache.org/jira/browse/IGNITE-19500 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, IndexManager listens configuration events to create internal > structures. > Let's IndexManager listens CatalogService events instead. > Note: Some tests may fails due to changed guarantees and related ticked > incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19500) IndexManager should listen CatalogService events instead of configuration
Andrey Mashenkov created IGNITE-19500: - Summary: IndexManager should listen CatalogService events instead of configuration Key: IGNITE-19500 URL: https://issues.apache.org/jira/browse/IGNITE-19500 Project: Ignite Issue Type: Improvement Reporter: Andrey Mashenkov As of now, SqlSchemaManager listens configuration events to creates internal structures. Let's TableManager listens CatalogService events instead. Note: Some tests may fails due to changed guarantees and related ticked incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19499) TableManager should listen CatalogSevice events instead of configuration
Andrey Mashenkov created IGNITE-19499: - Summary: TableManager should listen CatalogSevice events instead of configuration Key: IGNITE-19499 URL: https://issues.apache.org/jira/browse/IGNITE-19499 Project: Ignite Issue Type: Improvement Reporter: Andrey Mashenkov As of now, SqlSchemaManager listens configuration events to creates internal structures. Let's TableManager listens CatalogService events instead. Note: Some tests may fails due to changed guarantees and related ticked incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19498) Start building indexes only once
Kirill Tkalenko created IGNITE-19498: Summary: Start building indexes only once Key: IGNITE-19498 URL: https://issues.apache.org/jira/browse/IGNITE-19498 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko Fix For: 3.0.0-beta2 As part of IGNITE-19177, the building of indexes moves from the *IndexManager* to the replica, because of this, a race may occur when the replica becomes the leader, and we need to start listening to the creation of indexes in the configuration and reading previous indexes from the configuration. Therefore, we need to figure out how we can avoid the double start of index building, maybe this is not a problem and later we will just close the ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-19434) Calcite Engine : Does not support EXPLAIN command.
[ https://issues.apache.org/jira/browse/IGNITE-19434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723396#comment-17723396 ] YuJue Li edited comment on IGNITE-19434 at 5/17/23 9:57 AM: Yes, I know this grammar. You can take a look at IGNITE-12909. I wonder if it is necessary to maintain syntax consistency between Calcite based engine and H2 based engine? MySQL and PostgreSQL and SQL Server, include H2, the syntax for executing plan is explain select.. Alternatively, support both grammars is also an option. was (Author: liyuj): Yes, I know this grammar. You can take a look at IGNITE-12909. I wonder if it is necessary to maintain syntax consistency between Calcite based engine and H2 based engine? MySQL and PostgreSQL and SQL Server, include H2, the syntax for executing plan is explain select.. > Calcite Engine : Does not support EXPLAIN command. > -- > > Key: IGNITE-19434 > URL: https://issues.apache.org/jira/browse/IGNITE-19434 > Project: Ignite > Issue Type: New Feature > Components: sql >Affects Versions: 2.15 >Reporter: YuJue Li >Priority: Major > Fix For: 2.16 > > > java.sql.SQLException: Failed to parse query. Encountered "explain select" at > line 1, column 1. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19497) Switch to SqlSchemaManager implementation on top of CatalogService
Andrey Mashenkov created IGNITE-19497: - Summary: Switch to SqlSchemaManager implementation on top of CatalogService Key: IGNITE-19497 URL: https://issues.apache.org/jira/browse/IGNITE-19497 Project: Ignite Issue Type: Improvement Components: sql Reporter: Andrey Mashenkov Let's switch to a new SqlSchemaManager from IGNITE-19496, which uses Catalog events, and drop old implementation. Note: Some tests may fails due to changed guarantees and related ticked incompletion. So, let's do this in a separate feature branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-19434) Calcite Engine : Does not support EXPLAIN command.
[ https://issues.apache.org/jira/browse/IGNITE-19434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723396#comment-17723396 ] YuJue Li edited comment on IGNITE-19434 at 5/17/23 9:51 AM: Yes, I know this grammar. You can take a look at IGNITE-12909. I wonder if it is necessary to maintain syntax consistency between Calcite based engine and H2 based engine? MySQL and PostgreSQL and SQL Server, include H2, the syntax for executing plan is explain select.. was (Author: liyuj): Yes, I know this grammar. You can take a look at IGNITE-12909. I wonder if it is necessary to maintain syntax consistency between Calcite based engine and H2 based engine?erv MySQL and PostgreSQL and SQL Server, include H2, the syntax for executing plan is explain select.. > Calcite Engine : Does not support EXPLAIN command. > -- > > Key: IGNITE-19434 > URL: https://issues.apache.org/jira/browse/IGNITE-19434 > Project: Ignite > Issue Type: New Feature > Components: sql >Affects Versions: 2.15 >Reporter: YuJue Li >Priority: Major > Fix For: 2.16 > > > java.sql.SQLException: Failed to parse query. Encountered "explain select" at > line 1, column 1. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19493) Sql. Query flow refactoring.
[ https://issues.apache.org/jira/browse/IGNITE-19493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-19493: -- Description: As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from SqlQueryMaanger, then create query context, then start implicit transaction if needed. SqlQueryManager doesn't aware of schema versions. Let's * add tx timestamp to the interface, it is ok to ignore the tx timestamp parameter in current implementation unless a new one based on Catalog will be implemented in IGNITE-19496. * rework current flow to start implicit transaction first, then get a schema based on Tx timestamp, then create a query context. was: As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from SqlQueryMaanger, then create query context, then start implicit transaction if needed. SqlQueryManager doesn't aware of schema versions. Let's * add tx timestamp to the interface, it is ok to ignore the tx timestamp parameter in current implementation unless a new one based on Catalog will be implemented. * rework current flow to start implicit transaction first, then get a schema based on Tx timestamp, then create a query context. > Sql. Query flow refactoring. > > > Key: IGNITE-19493 > URL: https://issues.apache.org/jira/browse/IGNITE-19493 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > > As of now, in SqlQueryProcessor.querySingle0(), we get SQL SchemaPlus from > SqlQueryMaanger, then create query context, then start implicit transaction > if needed. > SqlQueryManager doesn't aware of schema versions. > Let's > * add tx timestamp to the interface, it is ok to ignore the tx timestamp > parameter in current implementation unless a new one based on Catalog will be > implemented in IGNITE-19496. > * rework current flow to start implicit transaction first, then get a schema > based on Tx timestamp, then create a query context. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19434) Calcite Engine : Does not support EXPLAIN command.
[ https://issues.apache.org/jira/browse/IGNITE-19434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723396#comment-17723396 ] YuJue Li commented on IGNITE-19434: --- Yes, I know this grammar. You can take a look at IGNITE-12909. I wonder if it is necessary to maintain syntax consistency between Calcite based engine and H2 based engine?erv MySQL and PostgreSQL and SQL Server, include H2, the syntax for executing plan is explain select.. > Calcite Engine : Does not support EXPLAIN command. > -- > > Key: IGNITE-19434 > URL: https://issues.apache.org/jira/browse/IGNITE-19434 > Project: Ignite > Issue Type: New Feature > Components: sql >Affects Versions: 2.15 >Reporter: YuJue Li >Priority: Major > Fix For: 2.16 > > > java.sql.SQLException: Failed to parse query. Encountered "explain select" at > line 1, column 1. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-19496) Implement SqlSchemaManager on top of CatalogService
Andrey Mashenkov created IGNITE-19496: - Summary: Implement SqlSchemaManager on top of CatalogService Key: IGNITE-19496 URL: https://issues.apache.org/jira/browse/IGNITE-19496 Project: Ignite Issue Type: Improvement Components: sql Reporter: Andrey Mashenkov As of now, SqlSchemaManager listens configuration events to compose a new schema, and does not support schema versioning. Let's implement SqlSchemaManager on top of CatalogService, which is already aware of schema versions. The new implementation may be just a caching adapter for CatalogService, which enriches Catalog with metadata that can be used by Calcite engine. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19403) Watch listeners must be deployed after the zone manager starts
[
https://issues.apache.org/jira/browse/IGNITE-19403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-19403:
-
Epic Link: (was: IGNITE-18528)
> Watch listeners must be deployed after the zone manager starts
> --
>
> Key: IGNITE-19403
> URL: https://issues.apache.org/jira/browse/IGNITE-19403
> Project: Ignite
> Issue Type: Test
>Reporter: Sergey Uttsel
>Priority: Major
> Labels: ignite-3
>
> h3. *Motivation*
> A method
> {{DistributionZonesTestUtil#deployWatchesAndUpdateMetaStorageRevision}} is
> used in tests related to a distribution zone manager to increase a meta
> storage applied revision before distribution manager starts. The method
> breaks invariant: zone manager must be started before
> metaStorageManager.deployWatches() is invoked. Need to do proper solution for
> increasing the applied revision.
>
> The first approach to fix it is to invoke methods in this order
>
> {code:java}
> vaultManager.put(new ByteArray("applied_revision"), longToBytes(1)).get();
> metaStorageManager.start();
> distributionZoneManager.start();
> metaStorageManager.deployWatches();
> {code}
>
> First we put applied_revision. Then start metaStorageManager and
> metaStorageManager. Then deploy watches.
> The disadvantage of this method is that th ByteArray("applied_revision") is
> an internal part of the implementation.
>
> The second way is a restart all of the components used in the test to
> simulate the node restart. In this case, after the zones manager's restart,
> the revision will be greater than zero.
> h3. *Definition of Done*
> The {{deployWatchesAndUpdateMetaStorageRevision}} is replaced by proper
> solution. Need to try approach with restart of all components.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-19061) Node restart rewrites data nodes with too actual nodes
[
https://issues.apache.org/jira/browse/IGNITE-19061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-19061:
-
Description:
Currently, in the {{DistributionZoneManager.start}} method we have a logic that
initialises data nodes in Meta Storage. It takes logical topology from Vault
and write it to the Meta Storage as a data nodes.
This behaviour is not correct, lets consider the scenario:
# We have cluster [A, B, C], new node E was added to the logical topology,
scale up timer was scheduled to add that node to a data nodes of some zone
Zone_1 after 1 hour.
# Right after that node A from cluster was restarted, so it will rewrite data
nodes for Zone_1 according the logic from the {{DistributionZoneManager.start}}
# Data nodes of Zone_1 become [A, B, C, E], but user was expecting, that data
nodes for Zone_1 will be changed after 1 hour.
UPD:
Current ticket is reclassified to the ticket for design, so the proper solution
for the restart and restoring timers must be provided.
was:
Currently, in the {{DistributionZoneManager.start}} method we have a logic that
initialises data nodes in Meta Storage. It takes logical topology from Vault
and write it to the Meta Storage as a data nodes.
This behaviour is not correct, lets consider the scenario:
# We have cluster [A, B, C], new node E was added to the logical topology,
scale up timer was scheduled to add that node to a data nodes of some zone
Zone_1 after 1 hour.
# Right after that node A from cluster was restarted, so it will rewrite data
nodes for Zone_1 according the logic from the {{DistributionZoneManager.start}}
# Data nodes of Zone_1 become [A, B, C, E], but user was expecting, that data
nodes for Zone_1 will be changed after 1 hour.
> Node restart rewrites data nodes with too actual nodes
> --
>
> Key: IGNITE-19061
> URL: https://issues.apache.org/jira/browse/IGNITE-19061
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> Currently, in the {{DistributionZoneManager.start}} method we have a logic
> that initialises data nodes in Meta Storage. It takes logical topology from
> Vault and write it to the Meta Storage as a data nodes.
> This behaviour is not correct, lets consider the scenario:
> # We have cluster [A, B, C], new node E was added to the logical topology,
> scale up timer was scheduled to add that node to a data nodes of some zone
> Zone_1 after 1 hour.
> # Right after that node A from cluster was restarted, so it will rewrite data
> nodes for Zone_1 according the logic from the
> {{DistributionZoneManager.start}}
> # Data nodes of Zone_1 become [A, B, C, E], but user was expecting, that data
> nodes for Zone_1 will be changed after 1 hour.
> UPD:
> Current ticket is reclassified to the ticket for design, so the proper
> solution for the restart and restoring timers must be provided.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-19495) SQL onheap cache stores row on per-cache basis, but access these rows on per-index basis
Aleksey Plekhanov created IGNITE-19495:
--
Summary: SQL onheap cache stores row on per-cache basis, but
access these rows on per-index basis
Key: IGNITE-19495
URL: https://issues.apache.org/jira/browse/IGNITE-19495
Project: Ignite
Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov
If there are two indexes on the same cache and sqlOnHeapCache property is
enabled there can be node failures such as:
{noformat}
org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
B+Tree is corrupted [groupId=-917681634, pageIds=[844420635164678],
cacheId=-917681634, cacheName=..., indexName=_key_PK, msg=Runtime failure on
row: Row@71aeb1b9[ ... ][ ]]
...
Caused by: org.apache.ignite.IgniteException: Failed to store new index row.
...
Caused by: java.lang.UnsupportedOperationException: 13 cannot be used for
inline type 19
{noformat}
Root cause: {{IndexRowCache}} is created per cache group (see
{{IndexRowCacheRegistry#onCacheRegistered}}), but stores {{IndexRowImpl}}
binded to particular index (see {{InlineIndexTree#createIndexRow}}). In case,
when another index requires the same data row, it gets row from the onheap
cache with the wrong index row handler.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-18879) Leaseholder candidates balancing
[
https://issues.apache.org/jira/browse/IGNITE-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-18879:
-
Epic Link: (was: IGNITE-18524)
> Leaseholder candidates balancing
>
>
> Key: IGNITE-18879
> URL: https://issues.apache.org/jira/browse/IGNITE-18879
> Project: Ignite
> Issue Type: Improvement
>Reporter: Denis Chudov
>Priority: Major
> Labels: ignite-3
>
> *Motivation*
> Primary replicas (leaseholders) should be evenly distributed over cluster to
> balance the transactional load between nodes. As the placement driver assigns
> primary replicas, balancing the primary replicas is also it's responsibility.
> Naive implementation of balancing should choose a node as leaseholder
> candidate in a way to save even lease distribution over all nodes. In real
> cluster, it may take into account slow nodes, hot table records, etc. If
> lease candidate declines LeaseGrantMessage from placement driver, the
> balancer should make decision to choose another candidate for given primary
> replica or enforce the previously chosen. So the balancing algorith should be
> pluggable, so that we could have ability to improve/replace/compare it with
> others.
> *Definition of done*
> Introduced interface for lease candidates balancer, and a simple
> implementation sustaining even lease distribution, which is used by placement
> driver by default. No public or internal configuration needed on this stage.
> *Implementation notes*
> Lease candidates balancer should have at least 2 methods:
> - {_}get(group, ignoredNodes){_}: returns candidate for the given group, a
> node from ignoredNodes set can't be chosen as a candidate
> - {_}considerRedirectProposal(group, candidate, proposedCandidate){_}:
> processes redirect proposal for given group provided by given candidate
> (previously chosen using _get_ method), proposedCandidate is the alternative
> candidate. Returns candidate that should be enforced by placement driver.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
