date:20221012

YuJue Li created IGNITE-17892:
-

 Summary: Ignite doc: cdcEnabled parameter is in the 
DataRegionConfiguration
 Key: IGNITE-17892
 URL: https://issues.apache.org/jira/browse/IGNITE-17892
 Project: Ignite
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.14
Reporter: YuJue Li
 Fix For: 2.15


[https://ignite.apache.org/docs/latest/persistence/change-data-capture|https://ignite.apache.org/docs/latest/persistence/change-data-capture#ignite-node]

DataStorageConfiguration#cdcEnabled

should be:
DataRegionConfiguration#cdcEnabled



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-17892) Ignite doc: cdcEnabled parameter is in the DataRegionConfiguration



 [ 
https://issues.apache.org/jira/browse/IGNITE-17892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

YuJue Li reassigned IGNITE-17892:
-

Assignee: YuJue Li

> Ignite doc: cdcEnabled parameter is in the DataRegionConfiguration
> --
>
> Key: IGNITE-17892
> URL: https://issues.apache.org/jira/browse/IGNITE-17892
> Project: Ignite
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 2.14
>Reporter: YuJue Li
>Assignee: YuJue Li
>Priority: Minor
> Fix For: 2.15
>
>
> [https://ignite.apache.org/docs/latest/persistence/change-data-capture|https://ignite.apache.org/docs/latest/persistence/change-data-capture#ignite-node]
> DataStorageConfiguration#cdcEnabled
> should be:
> DataRegionConfiguration#cdcEnabled



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17876) .NET: Thin 3.0: Single column mapping



[ 
https://issues.apache.org/jira/browse/IGNITE-17876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616816#comment-17616816
 ] 

Pavel Tupitsyn commented on IGNITE-17876:
-

Merged to main: 11387af5258c69843ef083f2cedadd18b3cfa7c0

> .NET: Thin 3.0: Single column mapping
> -
>
> Key: IGNITE-17876
> URL: https://issues.apache.org/jira/browse/IGNITE-17876
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Allow single-column mapping:
> {code}
> var view = Table.GetRecordView();
> await view.UpsertAsync(null, 123);
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17876) .NET: Thin 3.0: Single column mapping

2022-10-12 Thread Igor Sapego (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-17876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616815#comment-17616815
 ] 

Igor Sapego commented on IGNITE-17876:
--

[~ptupitsyn] looks good to me

> .NET: Thin 3.0: Single column mapping
> -
>
> Key: IGNITE-17876
> URL: https://issues.apache.org/jira/browse/IGNITE-17876
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-alpha6
>
>
> Allow single-column mapping:
> {code}
> var view = Table.GetRecordView();
> await view.UpsertAsync(null, 123);
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-17786) Add --verbose option to all commands

2022-10-12 Thread Vadim Pakhnushev (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-17786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vadim Pakhnushev reassigned IGNITE-17786:
-

Assignee: Vadim Pakhnushev

> Add --verbose option to all commands
> 
>
> Key: IGNITE-17786
> URL: https://issues.apache.org/jira/browse/IGNITE-17786
> Project: Ignite
>  Issue Type: Improvement
>  Components: cli
>Reporter: Aleksandr
>Assignee: Vadim Pakhnushev
>Priority: Major
>  Labels: ignite-3
>
> Users of CLI should have the ability to go deeper if they face issues with 
> CLI.  
> The goal of this ticket is to add {{--verbose}} option to all commands and 
> print additional debug information to the terminal if this flag is set to 
> true.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-13022) Calcite integration. Merge index conditions for the same field.

2022-10-12 Thread Ignite TC Bot (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-13022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616795#comment-17616795
 ] 

Ignite TC Bot commented on IGNITE-13022:


{panel:title=Branch: [pull/10306/head] Base: [master] : Possible Blockers 
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Control Utility 2{color} [[tests 
1|https://ci2.ignite.apache.org/viewLog.html?buildId=6816459]]
* IgniteControlUtilityTestSuite2: 
GridCommandHandlerDefragmentationTest.testDefragmentationStatus - History for 
base branch is absent.

{panel}
{panel:title=Branch: [pull/10306/head] Base: [master] : New Tests 
(13)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Calcite SQL{color} [[tests 
13|https://ci2.ignite.apache.org/viewLog.html?buildId=6813457]]
* {color:#013220}IgniteCalciteTestSuite: 
IndexSearchBoundsPlannerTest.testBoundsMerge - PASSED{color}
* {color:#013220}IgniteCalciteTestSuite: 
IndexSearchBoundsPlannerTest.testBoundsOneFieldSearchRangeOptimization - 
PASSED{color}
* {color:#013220}IgniteCalciteTestSuite: 
IndexSearchBoundsPlannerTest.testBoundsSeveralFieldsSearch - PASSED{color}
* {color:#013220}IgniteCalciteTestSuite: 
IndexSearchBoundsPlannerTest.testBoundsMaxComplexity - PASSED{color}
* {color:#013220}IgniteCalciteTestSuite: 
IndexSearchBoundsPlannerTest.testBoundsOneFieldSearchWithNull - PASSED{color}
* {color:#013220}IgniteCalciteTestSuite: 
IndexSearchBoundsPlannerTest.testBoundsOneFieldSearchDeduplication - 
PASSED{color}
* {color:#013220}IgniteCalciteTestSuite: 
IndexSearchBoundsPlannerTest.testBoundsOneFieldSearch - PASSED{color}
* {color:#013220}IgniteCalciteTestSuite: 
IndexSearchBoundsPlannerTest.testBoundsDescOrdering - PASSED{color}
* {color:#013220}IgniteCalciteTestSuite: 
IndexSearchBoundsPlannerTest.testBoundsWithCorrelate - PASSED{color}
* {color:#013220}IgniteCalciteTestSuite: 
CalciteBasicSecondaryIndexIntegrationTest.testIndexBoundsMerge - PASSED{color}
* {color:#013220}IgniteCalciteTestSuite: 
IndexSearchBoundsPlannerTest.testBoundsTypeConversion - PASSED{color}
... and 2 new tests

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=6813537&buildTypeId=IgniteTests24Java8_RunAll]

> Calcite integration. Merge index conditions for the same field.
> ---
>
> Key: IGNITE-13022
> URL: https://issues.apache.org/jira/browse/IGNITE-13022
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Roman Kondakov
>Assignee: Aleksey Plekhanov
>Priority: Minor
>  Labels: calcite2-required, calcite3-required
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Index scans should be able to merge index conditions. For example query
> {code:java}
> SELECT * FROM tbl WHERE a<5 AND a<10
> {code}
> should be reduced to
>  
> {code:java}
> SELECT * FROM tbl WHERE a<5
> {code}
> Parameters should be handled in a more tricky way:
> {code:java}
> SELECT * FROM tbl WHERE a {code}
> can be rewritten as
> {code:java}
> SELECT * FROM tbl WHERE a where the expression {{MIN(?1, ?2)}} should be evaluated right before the 
> execution when parameters are known.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17876) .NET: Thin 3.0: Single column mapping



 [ 
https://issues.apache.org/jira/browse/IGNITE-17876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-17876:

Description: 
Allow single-column mapping:

{code}
var view = Table.GetRecordView();
await view.UpsertAsync(null, 123);
{code}

  was:
Allow single-column mapping:

{code}
var view = Table.GetRecordView();
await view.UpsertAsync(null, 123);
{code}


> .NET: Thin 3.0: Single column mapping
> -
>
> Key: IGNITE-17876
> URL: https://issues.apache.org/jira/browse/IGNITE-17876
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-alpha6
>
>
> Allow single-column mapping:
> {code}
> var view = Table.GetRecordView();
> await view.UpsertAsync(null, 123);
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-17754) Display useful information for the user on node start



 [ 
https://issues.apache.org/jira/browse/IGNITE-17754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr reassigned IGNITE-17754:
--

Assignee: Aleksandr

> Display useful information for the user on node start
> -
>
> Key: IGNITE-17754
> URL: https://issues.apache.org/jira/browse/IGNITE-17754
> Project: Ignite
>  Issue Type: Task
>  Components: cli, rest
>Reporter: Aleksandr
>Assignee: Aleksandr
>Priority: Major
>  Labels: ignite-3
>
> There should be a clear way for the user how to connect to the node from cli 
> after it is started. Now there is nothing displayed after the node started. 
> I would like to recommend displaying something like: "Ignite 3 node started. 
> Use ignite3-cli command to connect to the node. 'ignite connect 
> http://localhost:10300";. 
> Note that there is no easy way to understand what port REST server is running 
> on. So, this is part of this ticket to develop the mechanism of propagating 
> the REST port.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17891) Add "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm args



 [ 
https://issues.apache.org/jira/browse/IGNITE-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

YuJue Li updated IGNITE-17891:
--
Attachment: 0001-IGNITE-17891-Add-add-opens-java.base-java.util.concu.patch

> Add "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm 
> args
> ---
>
> Key: IGNITE-17891
> URL: https://issues.apache.org/jira/browse/IGNITE-17891
> Project: Ignite
>  Issue Type: Bug
>  Components: cli
>Affects Versions: 2.14
> Environment: jdk 17
>Reporter: YuJue Li
>Assignee: YuJue Li
>Priority: Minor
> Fix For: 2.15
>
> Attachments: 
> 0001-IGNITE-17891-Add-add-opens-java.base-java.util.concu.patch, 
> CacheEventsExample.java
>
>
> Add \{-}"{-}-add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to 
> jvm args in JDK17 env
> start 2 nodes with ignite.sh
> then run the reproducer.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-15245) JDBC connection leak with cache.invoke() over cache store with external JDBC storage

2022-10-12 Thread Ignite TC Bot (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-15245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616777#comment-17616777
 ] 

Ignite TC Bot commented on IGNITE-15245:


{panel:title=Branch: [pull/10302/head] Base: [master] : Possible Blockers 
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Control Utility 2{color} [[tests 
1|https://ci2.ignite.apache.org/viewLog.html?buildId=6816585]]
* IgniteControlUtilityTestSuite2: 
GridCommandHandlerDefragmentationTest.testDefragmentationStatus - History for 
base branch is absent.

{panel}
{panel:title=Branch: [pull/10302/head] Base: [master] : New Tests 
(436)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Control Utility 2{color} [[tests 
429|https://ci2.ignite.apache.org/viewLog.html?buildId=6816585]]
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testAtomicAndTxValueAndVersion[strategy=REMOVE,
 explicitGrp=false, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testAtomicAndTxVersionOnly[strategy=RELATIVE_MAJORITY,
 explicitGrp=true, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testCacheFilter[strategy=RELATIVE_MAJORITY, 
explicitGrp=true, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testRepairNonExistentCache[strategy=RELATIVE_MAJORITY,
 explicitGrp=true, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testAtomicAndTxValueAndVersion[strategy=REMOVE,
 explicitGrp=true, callByGrp=true] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testAtomicAndTxVersionOnly[strategy=REMOVE, 
explicitGrp=false, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testCacheFilter[strategy=REMOVE, 
explicitGrp=false, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testRepairNonExistentCache[strategy=REMOVE, 
explicitGrp=false, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testAtomicAndTxValueAndVersion[strategy=REMOVE,
 explicitGrp=true, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testAtomicAndTxVersionOnly[strategy=REMOVE, 
explicitGrp=true, callByGrp=true] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: 
GridCommandHandlerConsistencyTest.testCacheFilter[strategy=REMOVE, 
explicitGrp=true, callByGrp=true] - PASSED{color}
... and 418 new tests

{color:#8b}Platform .NET (Windows){color} [[tests 
2|https://ci2.ignite.apache.org/viewLog.html?buildId=6816545]]
* {color:#013220}exe: 
ClientConnectionTest.TestExceptionInRetryPolicyPropagatesToCaller(True) - 
PASSED{color}
* {color:#013220}exe: 
ClientConnectionTest.TestExceptionInRetryPolicyPropagatesToCaller(False) - 
PASSED{color}

{color:#8b}Cache 1{color} [[tests 
1|https://ci2.ignite.apache.org/viewLog.html?buildId=6816472]]
* {color:#013220}IgniteBinaryCacheTestSuite: 
CacheJdbcPojoWriteBehindConnectionLeakTest.testInvoke - PASSED{color}

{color:#8b}Thin Client: Java{color} [[tests 
4|https://ci2.ignite.apache.org/viewLog.html?buildId=6816566]]
* {color:#013220}ClientTestSuite: 
ReliabilityTestPartitionAwareAsync.testExceptionInRetryPolicyPropagatesToCaller 
- PASSED{color}
* {color:#013220}ClientTestSuite: 
ReliabilityTestPartitionAware.testExceptionInRetryPolicyPropagatesToCaller - 
PASSED{color}
* {color:#013220}ClientTestSuite: 
ReliabilityTestAsync.testExceptionInRetryPolicyPropagatesToCaller - 
PASSED{color}
* {color:#013220}ClientTestSuite: 
ReliabilityTest.testExceptionInRetryPolicyPropagatesToCaller - PASSED{color}

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=6816577&buildTypeId=IgniteTests24Java8_RunAll]

> JDBC connection leak with cache.invoke() over cache store with external JDBC 
> storage
> 
>
> Key: IGNITE-15245
> URL: https://issues.apache.org/jira/browse/IGNITE-15245
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 2.10
>Reporter: Ilya Korol
>Assignee: Pavel Pereslegin
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Given following snippet:
> {code:java}
> try (Transaction tx = 
> ignite.transactions().txStart(TransactionConcurrency.PESSIMISTIC, 
> TransactionIsolation.REPEATABLE_READ)) {
> cache.invoke(pojo.getId(), en

[jira] [Updated] (IGNITE-17891) Add "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm args



 [ 
https://issues.apache.org/jira/browse/IGNITE-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

YuJue Li updated IGNITE-17891:
--
Summary: Add 
"--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm args  
(was: Add "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" and 
"--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm args)

> Add "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm 
> args
> ---
>
> Key: IGNITE-17891
> URL: https://issues.apache.org/jira/browse/IGNITE-17891
> Project: Ignite
>  Issue Type: Bug
>  Components: cli
>Affects Versions: 2.14
> Environment: jdk 17
>Reporter: YuJue Li
>Assignee: YuJue Li
>Priority: Minor
> Fix For: 2.15
>
> Attachments: CacheEventsExample.java
>
>
> Add "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" and 
> "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm args 
> in JDK17 env
> start 2 nodes with ignite.sh
> then run the reproducer.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17891) Add "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm args



 [ 
https://issues.apache.org/jira/browse/IGNITE-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

YuJue Li updated IGNITE-17891:
--
Description: 
Add \{-}"{-}-add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to 
jvm args in JDK17 env

start 2 nodes with ignite.sh

then run the reproducer.

  was:
Add "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" and 
"--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm args in 
JDK17 env

start 2 nodes with ignite.sh

then run the reproducer.


> Add "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm 
> args
> ---
>
> Key: IGNITE-17891
> URL: https://issues.apache.org/jira/browse/IGNITE-17891
> Project: Ignite
>  Issue Type: Bug
>  Components: cli
>Affects Versions: 2.14
> Environment: jdk 17
>Reporter: YuJue Li
>Assignee: YuJue Li
>Priority: Minor
> Fix For: 2.15
>
> Attachments: CacheEventsExample.java
>
>
> Add \{-}"{-}-add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to 
> jvm args in JDK17 env
> start 2 nodes with ignite.sh
> then run the reproducer.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17891) Add "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" and "--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm args

YuJue Li created IGNITE-17891:
-

 Summary: Add 
"--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" and 
"--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm args
 Key: IGNITE-17891
 URL: https://issues.apache.org/jira/browse/IGNITE-17891
 Project: Ignite
  Issue Type: Bug
  Components: cli
Affects Versions: 2.14
 Environment: jdk 17
Reporter: YuJue Li
Assignee: YuJue Li
 Fix For: 2.15
 Attachments: CacheEventsExample.java

Add "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED" and 
"--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED" to jvm args in 
JDK17 env

start 2 nodes with ignite.sh

then run the reproducer.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17160) Minor improvements in index-reader

2022-10-12 Thread Dmitry Pavlov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-17160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Pavlov updated IGNITE-17160:
---
Labels: ise  (was: )

> Minor improvements in index-reader
> --
>
> Key: IGNITE-17160
> URL: https://issues.apache.org/jira/browse/IGNITE-17160
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Priority: Trivial
>  Labels: ise
> Fix For: 2.14
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Code of index reader contains many compiler and IDE warnings that can be 
> easily fixed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17865) Disable partition ranges in log messages about rebalance or PME

2022-10-12 Thread Ilya Shishkov (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-17865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616657#comment-17616657
 ] 

Ilya Shishkov commented on IGNITE-17865:


[~slava.koptilin], hi!

I understand, that blowing up of logs is not a god idea, but are there any 
practical results, when such optimization can significantly reduce size of 
message and logs?
As I see, most of time lists of partitions in these messages does not form a 
continuous sequence, and even if so, only 2, and less often 3 and more 
partitions are forms continuous sequence. And for two partitions real economy 
is a whitespace symbol.

> Disable partition ranges in log messages about rebalance or PME
> ---
>
> Key: IGNITE-17865
> URL: https://issues.apache.org/jira/browse/IGNITE-17865
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Ilya Shishkov
>Priority: Minor
>  Labels: ise, newbie
>
> When you need to analyze cases of partitions inconsistency, full list of 
> partitions is needed, but there is an optimization which replaces list of 
> consecutive partitions with ranges. So, as you can see below, we can not find 
> partition 2 by simple text search:
> {quote}
> 2021-08-11 23:12:11.338 [WARN 
> ]\[sys-#213\][org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl]
>  Partitions have been scheduled for rebalancing due to outdated update 
> counter [grp=group, readyTopVer=AffinityTopologyVersion [topVer=33, 
> minorTopVer=0], topVer=AffinityTopologyVersion [topVer=33, minorTopVer=0], 
> nodeId=8e05997a-4ff7-4fdb-9480-cfbcefa8bbf5, 
> partsFull=[{color:red}*1-3*{color}, ...], partsHistorical=[]]
> {quote}
> {quote}
> 2021-08-11 23:12:11.338 [WARN 
> ]\[sys-#213\][org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture]
>  Partitions weren't present in any history reservation: [[[grp=grp2 
> part=[[{color:red}*1-3*{color}]]]
> {quote}
> We need to remove this optimization, because it can lead to mistakes in logs 
> analysis.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17878) RocksDB sorted indexes ignore partitions during the scan

2022-10-12 Thread Kirill Tkalenko (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kirill Tkalenko updated IGNITE-17878:
-
Reviewer: Kirill Tkalenko

> RocksDB sorted indexes ignore partitions during the scan
> 
>
> Key: IGNITE-17878
> URL: https://issues.apache.org/jira/browse/IGNITE-17878
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> If one of bounds is null, scan will return results from some other partitions 
> as well.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17878) RocksDB sorted indexes ignore partitions during the scan

2022-10-12 Thread Kirill Tkalenko (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kirill Tkalenko updated IGNITE-17878:
-
Fix Version/s: 3.0.0-alpha6

> RocksDB sorted indexes ignore partitions during the scan
> 
>
> Key: IGNITE-17878
> URL: https://issues.apache.org/jira/browse/IGNITE-17878
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> If one of bounds is null, scan will return results from some other partitions 
> as well.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17489) Packaging: Brew package



[ 
https://issues.apache.org/jira/browse/IGNITE-17489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616501#comment-17616501
 ] 

Aleksandr commented on IGNITE-17489:


class Ignite3db < Formula
  desc ""
  homepage ""
  url "file:///../build/distributions/ignite3-db-3.0.0-SNAPSHOT.zip"
  sha256 "a1b299cb2d81a168b89f7d9af133923c2463b547bf9e37eea634ab96defca2d9"
  license "Apache 2.0"


  def install
lib.install Dir["lib/*.jar"]
bin.install "bin/ignite3-db.sh"
(prefix/"etc/").install Dir["etc/*"]
(var/"log/#{name}").mkpath
(var/"#{name}/work").mkpath

inreplace "#{bin}/ignite3-db.sh" do |s|
  s.gsub! "  exec ${CMD} >>${LOG_OUT_FILE:-/dev/null} 2>&1 < /dev/null & 
jobs -p > ${IGNITE_HOME}/pid", "  exec ${CMD}"
end

inreplace "#{prefix/"etc"}/bootstrap-config" do |s|
  s.gsub! "WORK_PATH=$IGNITE_HOME/work", "WORK_PATH=#{var}/#{name}/work"
end

inreplace "#{prefix/"etc"}/ignite.java.util.logging.properties" do |s|
  s.gsub! "java.util.logging.FileHandler.pattern = 
@LOG_DIR@/ignite3db-%g.log", "java.util.logging.FileHandler.pattern = 
#{var}/log/#{name}/ignite3db-%g.log"
end
  end

  test do
system "true"
  end

  def ignite3_db_log_path
var/"log/ignite3-db/#{name}.log"
  end

  service do
#environment_variables IGNITE_HOME: opt_prefix
run [opt_bin/"ignite3-db.sh", "start"]
keep_alive true
log_path f.ignite3_db_log_path
error_log_path f.ignite3_db_log_path
working_dir opt_prefix
  end
end

> Packaging: Brew package
> ---
>
> Key: IGNITE-17489
> URL: https://issues.apache.org/jira/browse/IGNITE-17489
> Project: Ignite
>  Issue Type: New Feature
>  Components: build
>Reporter: Mikhail Pochatkin
>Assignee: Aleksandr
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17489) Packaging: Brew package



[ 
https://issues.apache.org/jira/browse/IGNITE-17489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616500#comment-17616500
 ] 

Aleksandr commented on IGNITE-17489:


class Ignite3cli < Formula
  desc ""
  homepage ""
  url "file:///packaging/build/distributions/ignite3-cli-3.0.0-SNAPSHOT.zip"
  sha256 "a53bf7c02f958ca9cb1b54b216139196b2044152b7a49b551ea5d16a090a6408"

  license "Apache 2.0"

  def install
rm_f Dir["bin/*.bat"]

lib.install Dir["lib/*.jar"]
bin.install "bin/ignite3cli"
bash_completion.install "bin/ignite_completion.sh" => "ignite3cli"
  end

  def caveats
<<~EOS
  Add the following line to your ~/.bash_profile or ~/.zshrc:
   source /usr/local/etc/bash_completion.d/ignite3cli
EOS
  end

  test do
system "#{bin}/ignite3cli", "--help"
  end
end

> Packaging: Brew package
> ---
>
> Key: IGNITE-17489
> URL: https://issues.apache.org/jira/browse/IGNITE-17489
> Project: Ignite
>  Issue Type: New Feature
>  Components: build
>Reporter: Mikhail Pochatkin
>Assignee: Aleksandr
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17489) Packaging: Brew package



[ 
https://issues.apache.org/jira/browse/IGNITE-17489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616498#comment-17616498
 ] 

Aleksandr commented on IGNITE-17489:


We have to wait until the ignte3 beta1 is released before submitting the PR to 
the homebrew-core repo. Here is working formulas

> Packaging: Brew package
> ---
>
> Key: IGNITE-17489
> URL: https://issues.apache.org/jira/browse/IGNITE-17489
> Project: Ignite
>  Issue Type: New Feature
>  Components: build
>Reporter: Mikhail Pochatkin
>Assignee: Aleksandr
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17890) Calcite engine. Support index scan on boolean field

2022-10-12 Thread Aleksey Plekhanov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-17890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-17890:
---
Description: 
Currently, if table has index on boolean field it can't be used, since queries 
like
{code:java}
SELECT * FROM tbl WHERE a = TRUE
{code}
Simplified by Calcite to
{code:java}
SELECT * FROM tbl WHERE a {code}
Condition `{{{}a{}}}` is not supported for building search bounds (see 
\{{RexUtils.isSupportedTreeComparison()}}) and such an index can't be used. 

  was:
Currently, if table has index on boolean field it can't be used, since queries 
like
{code:java}
SELECT * FROM tbl WHERE a = TRUE
{code}
Simplified by Calcite to
{code:java}
SELECT * FROM tbl WHERE a {code}
Condition `{{{}a{}}}` is not supported for building search bounds (see 
{{RexUtils.

isSupportedTreeComparison()}}) and such an index can't be used. 


> Calcite engine. Support index scan on boolean field
> ---
>
> Key: IGNITE-17890
> URL: https://issues.apache.org/jira/browse/IGNITE-17890
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Priority: Major
>  Labels: calcite, calcite2-required, calcite3-required
>
> Currently, if table has index on boolean field it can't be used, since 
> queries like
> {code:java}
> SELECT * FROM tbl WHERE a = TRUE
> {code}
> Simplified by Calcite to
> {code:java}
> SELECT * FROM tbl WHERE a {code}
> Condition `{{{}a{}}}` is not supported for building search bounds (see 
> \{{RexUtils.isSupportedTreeComparison()}}) and such an index can't be used. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17890) Calcite engine. Support index scan on boolean field

2022-10-12 Thread Aleksey Plekhanov (Jira)

Aleksey Plekhanov created IGNITE-17890:
--

 Summary: Calcite engine. Support index scan on boolean field
 Key: IGNITE-17890
 URL: https://issues.apache.org/jira/browse/IGNITE-17890
 Project: Ignite
  Issue Type: Improvement
Reporter: Aleksey Plekhanov


Currently, if table has index on boolean field it can't be used, since queries 
like
{code:java}
SELECT * FROM tbl WHERE a = TRUE
{code}
Simplified by Calcite to
{code:java}
SELECT * FROM tbl WHERE a {code}
Condition `{{{}a{}}}` is not supported for building search bounds (see 
{{RexUtils.

isSupportedTreeComparison()}}) and such an index can't be used. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17879) Packaging is broken



 [ 
https://issues.apache.org/jira/browse/IGNITE-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-17879:
---
Labels: ignite-3  (was: )

> Packaging is broken
> ---
>
> Key: IGNITE-17879
> URL: https://issues.apache.org/jira/browse/IGNITE-17879
> Project: Ignite
>  Issue Type: Bug
>  Components: build
>Reporter: Vadim Pakhnushev
>Assignee: Vadim Pakhnushev
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> After integrating IGNITE-17726 and IGNITE-15957 docker container can't start 
> due to the missing config file and deb/rpm packages can start due to the 
> invalid command line arguments passed from start script.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17889) Calcite engine. Avoid full index scans in case of null dynamic parameter

2022-10-12 Thread Aleksey Plekhanov (Jira)

Aleksey Plekhanov created IGNITE-17889:
--

 Summary: Calcite engine. Avoid full index scans in case of null 
dynamic parameter
 Key: IGNITE-17889
 URL: https://issues.apache.org/jira/browse/IGNITE-17889
 Project: Ignite
  Issue Type: Improvement
Reporter: Aleksey Plekhanov


Currently, queries like:
{code:java}
SELECT * FROM tbl WHERE a >= ?
{code}
Should return no rows if dynamic parameter is null, but can be downgraded to 
full index scan in case table have index on column {{a}} (ASCENDING order, 
NULLS FIRST).

We should somehow analyse nulls in search bounds and return empty rows iterator 
for regular field conditions (`=`, `<`, '>`, etc). But also nulls should be 
processed as is in search bounds for conditions like `IS NULL`, `IS NOT NULL`, 
`IS NOT DISTINCT FROM` (the last one not supported currently).  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17881) OrderingFuture should notify dependents asynchronously

2022-10-12 Thread Aleksandr Polovtcev (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-17881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616465#comment-17616465
 ] 

Aleksandr Polovtcev commented on IGNITE-17881:
--

LGTM

> OrderingFuture should notify dependents asynchronously
> --
>
> Key: IGNITE-17881
> URL: https://issues.apache.org/jira/browse/IGNITE-17881
> Project: Ignite
>  Issue Type: Bug
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently, there are the following problems:
>  # Callbacks, when invoked, will see the future as incompleted if they 
> directly call its getNow() or get() methods
>  # If callbacks invoke long or blocking operations, this impedes completion



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-17864) Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE)



 [ 
https://issues.apache.org/jira/browse/IGNITE-17864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy reassigned IGNITE-17864:
--

Assignee: Roman Puchkovskiy

> Optimize scan(HybridTimestamp.MAX_VALUE) and read(HybridTimestamp.MAX_VALUE)
> 
>
> Key: IGNITE-17864
> URL: https://issues.apache.org/jira/browse/IGNITE-17864
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
>
> Please check https://issues.apache.org/jira/browse/IGNITE-17856 for more 
> details about scan(UUID txId) and read(RowId rowId, UUID txId) removal. 
> Within given ticket it's expected that timestamp based scans and reads will 
> be optimized in order to perform HybridTimestamp.MAX_VALUE bouned operations 
> in a most efficient way.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17839) Improve semantics of implicit txn management for async transactions

2022-10-12 Thread Alexander Lapin (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-17839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17839:
-
Reviewer: Alexander Lapin

> Improve semantics of implicit txn management for async transactions
> ---
>
> Key: IGNITE-17839
> URL: https://issues.apache.org/jira/browse/IGNITE-17839
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> There is a usability issue for transaction implicit management API.
> For async transaction tx.commit must be called after all async ops are 
> enlisted to the transaction. This is a responsibility of a user in case of 
> explicit management.
> But, for implicit management, tx.commit must be called automatically. This is 
> not generally not possible for async flows, because by the end of a closure 
> some operations are not yet started.
> The example:
> runInTransactionAsync(tx -> {
> return opAsync().thenComposeAsync(res -> otherOpAsync());
> })
> We can fix this by introducing async context for implicit management and 
> require a user to return last completion stage in the chain, like:
>  CompletableFuture runInTransactionAsync(Function CompletableFuture> clo);



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17839) Improve semantics of implicit txn management for async transactions

2022-10-12 Thread Alexander Lapin (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-17839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616452#comment-17616452
 ] 

Alexander Lapin commented on IGNITE-17839:
--

[~ascherbakov] LGTM

> Improve semantics of implicit txn management for async transactions
> ---
>
> Key: IGNITE-17839
> URL: https://issues.apache.org/jira/browse/IGNITE-17839
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexey Scherbakov
>Assignee: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> There is a usability issue for transaction implicit management API.
> For async transaction tx.commit must be called after all async ops are 
> enlisted to the transaction. This is a responsibility of a user in case of 
> explicit management.
> But, for implicit management, tx.commit must be called automatically. This is 
> not generally not possible for async flows, because by the end of a closure 
> some operations are not yet started.
> The example:
> runInTransactionAsync(tx -> {
> return opAsync().thenComposeAsync(res -> otherOpAsync());
> })
> We can fix this by introducing async context for implicit management and 
> require a user to return last completion stage in the chain, like:
>  CompletableFuture runInTransactionAsync(Function CompletableFuture> clo);



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17871) Use network serialization for RAFT commands

[
https://issues.apache.org/jira/browse/IGNITE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ivan Bessonov updated IGNITE-17871:
---
Description:
h3. Problem

Currently, there are two places where {{Command}} instances are being
serialized:
* ActionRequest - here the command property is marked is Marshallable, meaning
that it will be serialized using a User Object Serialization approach
* Listener - here command is explicitly serialized using JDKMarshaller, for
further handling by RAFT. This is the data that will be written to the Log and
deserialized on followers / learners

What are the problems?

For ActionRequest message, command is expected to be its largest part. And
although writing serialized UOS byte array into a netty socket is faster, then
optimized marshalling, it feels like overall throughput will be smaller. And
the reason is that there's an extra step of converting command into a byte
array, that happens in caller thread.

For serialization in listeners, using JDKMarshaller is both slow and
inefficient in terms of space. Obvious example - network serialization of
SnapshotMeta object, for instance, can be condensed to 8 bytes (assuming we
optimize "writeShort" and change its message type). JDKMarshaller produces 232
bytes. Of course, here most of fields are nulls and real payload will be
bigger, but JDKMarshaller will always lead to more data simply because it has
to store schema meta-information.
h3. Solution

Making Command an implementation of NetworkMessage will solve both of these
problems. ActionRequest will not have its "prepareMarshal" phase, listeners
will have fast and space-efficient serialization algorithm.

Of course, there must be drawbacks. I'll try to explain what I see at the
moment.
* Currently, there's no explicit support for List properties, only Collection.
It is easy to fix
* CMG commands use classes like ClusterNode and IgniteProductVersion. We
should introduce message alternatives
* I saw some enums being used, they are not natively supported at the moment.
There are two options:
** implement native support. I consider this a dangerous path
** store explicit ordinal where it's necessary
* ByteBuffer support would be really nice to have natively. Should be fast to
implement also

One important note: there should be no Marshallable properties in commands,
because we can't persist them. Information about classes' ids is stored in
sessions and can change between sessions. The way to achieve it is to pass a
"null" UOS context into serializator.

Now about serializator: we can have thread-local buffers to write data to. When
write is complete, data is copied as a byte[]. Reading will be done directly
from the byte[].

Possible optimization for ByteBuffers - we can implement them as slices of the
byte[] payload instead of copying sub-arrays. Will save some time and memory.
h3. Plan

Given the volume of changes, I suggest splitting the issue into several parts.
There are multiple sets of commands in Ignite:
* Table commands (5 commands at the moment of writing this text)
* CMG commands (6 commands)
* Metastorage (19)

This list goes in order of complexity. Table commands are very simple. CMG
commands require additional messages for ClusterNodes and such.

Metastorage commands have complicated structures for conditional updates, and
there are many of them.

When all commands are messages, we can safely inherit Command from
NetworkMessage and remove Marshallable from the ActionRequest's field. In
total, this looks like 4 separate issues, 4th one being the current one. List &
ByteBuffer support is already completed in IGNITE-17874.

Of course, implementation of the OptimizedMarshaller should be done here as
well. I'd also provide some ideas to whoever's going to implement the
improvement, there's at least one possible optimization in mind that's hard to
put in words.

There are also tests, they can be migrated in current issue as well.

was:
h3. Problem

What are the problems?

For serialization in listeners, using JDKMarshaller is both slow and
inefficient in terms of space. Obvious example -

[jira] [Created] (IGNITE-17888) Convert Metastorage RAFT commands to NetworkMessage-s

Ivan Bessonov created IGNITE-17888:
--

 Summary: Convert Metastorage RAFT commands to NetworkMessage-s
 Key: IGNITE-17888
 URL: https://issues.apache.org/jira/browse/IGNITE-17888
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov
 Fix For: 3.0.0-alpha6


Please refer to IGNITE-17871 for description



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17887) Convert CMG RAFT commands to NetworkMessage-s

Ivan Bessonov created IGNITE-17887:
--

 Summary: Convert CMG RAFT commands to NetworkMessage-s
 Key: IGNITE-17887
 URL: https://issues.apache.org/jira/browse/IGNITE-17887
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov
 Fix For: 3.0.0-alpha6


Please refer to IGNITE-17871 for description



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17886) Convert Table RAFT commands to NetworkMessage-s



 [ 
https://issues.apache.org/jira/browse/IGNITE-17886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-17886:
---
Description: Please refer to IGNITE-17871 for description  (was: Please 
refer to )

> Convert Table RAFT commands to NetworkMessage-s
> ---
>
> Key: IGNITE-17886
> URL: https://issues.apache.org/jira/browse/IGNITE-17886
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> Please refer to IGNITE-17871 for description



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-17886) Convert Table RAFT commands to NetworkMessage-s



 [ 
https://issues.apache.org/jira/browse/IGNITE-17886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov reassigned IGNITE-17886:
--

Assignee: (was: Ivan Bessonov)

> Convert Table RAFT commands to NetworkMessage-s
> ---
>
> Key: IGNITE-17886
> URL: https://issues.apache.org/jira/browse/IGNITE-17886
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> Please refer to IGNITE-17871 for description



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17886) Convert Table RAFT commands to NetworkMessage-s

Ivan Bessonov created IGNITE-17886:
--

 Summary: Convert Table RAFT commands to NetworkMessage-s
 Key: IGNITE-17886
 URL: https://issues.apache.org/jira/browse/IGNITE-17886
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov
Assignee: Ivan Bessonov
 Fix For: 3.0.0-alpha6






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17886) Convert Table RAFT commands to NetworkMessage-s



 [ 
https://issues.apache.org/jira/browse/IGNITE-17886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-17886:
---
Description: Please refer to 

> Convert Table RAFT commands to NetworkMessage-s
> ---
>
> Key: IGNITE-17886
> URL: https://issues.apache.org/jira/browse/IGNITE-17886
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> Please refer to 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17886) Convert Table RAFT commands to NetworkMessage-s



 [ 
https://issues.apache.org/jira/browse/IGNITE-17886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-17886:
---
Reviewer:   (was: Aleksandr Polovtcev)

> Convert Table RAFT commands to NetworkMessage-s
> ---
>
> Key: IGNITE-17886
> URL: https://issues.apache.org/jira/browse/IGNITE-17886
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17871) Use network serialization for RAFT commands

[
https://issues.apache.org/jira/browse/IGNITE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ivan Bessonov updated IGNITE-17871:
---
Description:
h3. Problem

What are the problems?

Now about serializator: we can have thread-local buffers to write data to. When
write is complete, data is copied as a byte[]. Reading will be done directly
from the byte[].

Possible optimization for ByteBuffers - we can implement them as slices of the
byte[] payload instead of copying sub-arrays. Will save some time and memory.
h3. Plan

This list goes in order of complexity. Table commands are very simple. CMG
commands require additional messages for ClusterNodes and such.

Metastorage commands have complicated structures for conditional updates, and
there are many of them.

was:
h3. Problem

What are the problems?

For serialization in listeners, using JDKMarshaller is both slow and
inefficient in terms of space. Obvious example - network serialization of
SnapshotMeta object, for instance, can be c

[jira] [Updated] (IGNITE-17871) Use network serialization for RAFT commands

[
https://issues.apache.org/jira/browse/IGNITE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ivan Bessonov updated IGNITE-17871:
---
Description:
h3. Problem

What are the problems?

Now about serializator: we can have thread-local buffers to write data to. When
write is complete, data is copied as a byte[]. Reading will be done directly
from the byte[].

Possible optimization for ByteBuffers - we can implement them as slices of the
byte[] payload instead of copying sub-arrays. Will save some time and memory.
h3. Plan

This list goes in order of complexity. Table commands are very simple. CMG
commands require additional messages for ClusterNodes and such.

Metastorage commands have complicated structures for conditional updates, and
there are many of them.

was:
h3. Problem

What are the problems?

[jira] [Updated] (IGNITE-15322) System tasks should run without any explicitly granted permissions

2022-10-12 Thread Mikhail Petrov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-15322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Petrov updated IGNITE-15322:

Description: 
For example, this code needs TASK_EXECUTE permissions.
{code:java}
Affinity affinity = ignite.affinity("TEST");
affinity.mapKeysToNodes(Arrays.asList(1L,100L, 1000L));{code}
This is unexpected behavior, because:
 - the task started implicitly (under the hood), customer should not to know 
about it.
 - this is a system task (not defined by a customer), the tasks needs for a 
normal grid workflow.

Also, I suppose there are any other implicitly tasks, which could lead to 
unexpected behavior (need permissions).

Possible lan to solve this issue:
1. Add mechanism to destinguish whether task class is SYSTEM (part of the 
Ignite codebase) or USER. Here we can reuse SecurityUtils#isSystemType.
2. Add mechanism to detect if task execution was initiated by the user (PUBLIC 
CALL) or by the Ignite system itself (INTERNAL CALL).
3. Add ability to explicitly specify for SYSTEM task/callable/runnable/closure  
what permission should be checked before its execution.
4. Skip permission check for the tasks that are SYSTEM and initiated by the 
Ignite internal code unless permissions are specified explicitly.
5. Restrict execution of SYSTEM tasks initiated by the user directly and for 
which no explicit permissions are specified.

Possible troubles:
1. We have control.sh tasks that are executed via the thin client. Task 
execution through the thin client compute is considered as PUBLIC CALL so we 
should assign some permission to each control.sh task.
To skip creation of a separate permissions for each control.sh task it is 
proposed to group tasks that belongs to the same control.sh command e.g.
control.sh --cache ... tasks will require ADMIN_CACHE permission
control.sh --tx ... tasks will require ADMIN_TX permission 
2. Mentioned SecurityUtils#isSystemType works only for the ignite-core module. 
If some tasks are defined inside other Ignite modules - they will not be 
considered SYSTEM. Currently there are no such task.

  was:
For example, this code needs TASK_EXECUTE permissions.
{code:java}
Affinity affinity = ignite.affinity("TEST");
affinity.mapKeysToNodes(Arrays.asList(1L,100L, 1000L));{code}
This is unexpected behavior, because:
 - the task started implicitly (under the hood), customer should not to know 
about it.
 - this is a system task (not defined by a customer), the tasks needs for a 
normal grid workflow.

Also, I suppose there are any other implicitly tasks, which could lead to 
unexpected behavior (need permissions).

Plan to solve this issue:
1. Add mechanism to destinguish whether task class is SYSTEM (part of the 
Ignite codebase) or USER. Here we can reuse SecurityUtils#isSystemType.
2. Add mechanism to detect if task execution was initiated by the user (PUBLIC 
CALL) or by the Ignite system itself (INTERNAL CALL).
3. Add ability to explicitly specify for SYSTEM task/callable/runnable/closure  
what permission should be checked before its execution.
4. Skip permission check for the tasks that are SYSTEM and initiated by the 
Ignite internal code unless permissions are specified explicitly.
5. Restrict execution of SYSTEM tasks initiated by the user directly and for 
which no explicit permissions are specified.

Possible troubles:
1. We have control.sh tasks that are executed via the thin client. Task 
execution through the thin client compute is considered as PUBLIC CALL so we 
should assign some permission to each control.sh task.
To skip creation of a separate permissions for each control.sh task it is 
proposed to group tasks that belongs to the same control.sh command e.g.
control.sh --cache ... tasks will require ADMIN_CACHE permission
control.sh --tx ... tasks will require ADMIN_TX permission 
2. Mentioned SecurityUtils#isSystemType works only for the ignite-core module. 
If some tasks are defined inside other Ignite modules - they will not be 
considered SYSTEM. Currently there are no such task.


> System tasks should run without any explicitly granted permissions
> --
>
> Key: IGNITE-15322
> URL: https://issues.apache.org/jira/browse/IGNITE-15322
> Project: Ignite
>  Issue Type: Bug
>  Components: compute, security
>Reporter: Ilya Kazakov
>Assignee: Mikhail Petrov
>Priority: Minor
>  Labels: ise
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> For example, this code needs TASK_EXECUTE permissions.
> {code:java}
> Affinity affinity = ignite.affinity("TEST");
> affinity.mapKeysToNodes(Arrays.asList(1L,100L, 1000L));{code}
> This is unexpected behavior, because:
>  - the task started implicitly (under the hood), customer should not to know 
> about it.
>  - this is a system task (not defined by a cu

[jira] [Updated] (IGNITE-15322) System tasks should run without any explicitly granted permissions

2022-10-12 Thread Mikhail Petrov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-15322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Petrov updated IGNITE-15322:

Description: 
For example, this code needs TASK_EXECUTE permissions.
{code:java}
Affinity affinity = ignite.affinity("TEST");
affinity.mapKeysToNodes(Arrays.asList(1L,100L, 1000L));{code}
This is unexpected behavior, because:
 - the task started implicitly (under the hood), customer should not to know 
about it.
 - this is a system task (not defined by a customer), the tasks needs for a 
normal grid workflow.

Also, I suppose there are any other implicitly tasks, which could lead to 
unexpected behavior (need permissions).

Plan to solve this issue:
1. Add mechanism to destinguish whether task class is SYSTEM (part of the 
Ignite codebase) or USER. Here we can reuse SecurityUtils#isSystemType.
2. Add mechanism to detect if task execution was initiated by the user (PUBLIC 
CALL) or by the Ignite system itself (INTERNAL CALL).
3. Add ability to explicitly specify for SYSTEM task/callable/runnable/closure  
what permission should be checked before its execution.
4. Skip permission check for the tasks that are SYSTEM and initiated by the 
Ignite internal code unless permissions are specified explicitly.
5. Restrict execution of SYSTEM tasks initiated by the user directly and for 
which no explicit permissions are specified.

Possible troubles:
1. We have control.sh tasks that are executed via the thin client. Task 
execution through the thin client compute is considered as PUBLIC CALL so we 
should assign some permission to each control.sh task.
To skip creation of a separate permissions for each control.sh task it is 
proposed to group tasks that belongs to the same control.sh command e.g.
control.sh --cache ... tasks will require ADMIN_CACHE permission
control.sh --tx ... tasks will require ADMIN_TX permission 
2. Mentioned SecurityUtils#isSystemType works only for the ignite-core module. 
If some tasks are defined inside other Ignite modules - they will not be 
considered SYSTEM. Currently there are no such task.

  was:
For example, this code needs TASK_EXECUTE permissions.
{code:java}
Affinity affinity = ignite.affinity("TEST");
affinity.mapKeysToNodes(Arrays.asList(1L,100L, 1000L));{code}
This is unexpected behavior, because:
 - the task started implicitly (under the hood), customer should not to know 
about it.
 - this is a system task (not defined by a customer), the tasks needs for a 
normal grid workflow.

Also, I suppose there are any other implicitly tasks, which could lead to 
unexpected behavior (need permissions).


> System tasks should run without any explicitly granted permissions
> --
>
> Key: IGNITE-15322
> URL: https://issues.apache.org/jira/browse/IGNITE-15322
> Project: Ignite
>  Issue Type: Bug
>  Components: compute, security
>Reporter: Ilya Kazakov
>Assignee: Mikhail Petrov
>Priority: Minor
>  Labels: ise
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> For example, this code needs TASK_EXECUTE permissions.
> {code:java}
> Affinity affinity = ignite.affinity("TEST");
> affinity.mapKeysToNodes(Arrays.asList(1L,100L, 1000L));{code}
> This is unexpected behavior, because:
>  - the task started implicitly (under the hood), customer should not to know 
> about it.
>  - this is a system task (not defined by a customer), the tasks needs for a 
> normal grid workflow.
> Also, I suppose there are any other implicitly tasks, which could lead to 
> unexpected behavior (need permissions).
> Plan to solve this issue:
> 1. Add mechanism to destinguish whether task class is SYSTEM (part of the 
> Ignite codebase) or USER. Here we can reuse SecurityUtils#isSystemType.
> 2. Add mechanism to detect if task execution was initiated by the user 
> (PUBLIC CALL) or by the Ignite system itself (INTERNAL CALL).
> 3. Add ability to explicitly specify for SYSTEM 
> task/callable/runnable/closure  what permission should be checked before its 
> execution.
> 4. Skip permission check for the tasks that are SYSTEM and initiated by the 
> Ignite internal code unless permissions are specified explicitly.
> 5. Restrict execution of SYSTEM tasks initiated by the user directly and for 
> which no explicit permissions are specified.
> Possible troubles:
> 1. We have control.sh tasks that are executed via the thin client. Task 
> execution through the thin client compute is considered as PUBLIC CALL so we 
> should assign some permission to each control.sh task.
> To skip creation of a separate permissions for each control.sh task it is 
> proposed to group tasks that belongs to the same control.sh command e.g.
> control.sh --cache ... tasks will require ADMIN_CACHE permission
> control.sh --tx ... tasks will require ADMIN_TX permission 
>

[jira] [Created] (IGNITE-17885) Convert ClusterNode class to interface

Vyacheslav Koptilin created IGNITE-17885:


 Summary: Convert ClusterNode class to interface
 Key: IGNITE-17885
 URL: https://issues.apache.org/jira/browse/IGNITE-17885
 Project: Ignite
  Issue Type: Improvement
Reporter: Vyacheslav Koptilin
 Fix For: 3.0.0-alpha6


It would bi nice convert the _ClusterNode_ class to java interface. It will 
allow to introduce _DetachedNode_ concept that represents an offline node 
(however, such a node can be a part of logical topology), and avoid using 
internal utility classes/methods in the public API.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17884) Ignite interface should expose TopologyService

Vyacheslav Koptilin created IGNITE-17884:


 Summary: Ignite interface should expose TopologyService
 Key: IGNITE-17884
 URL: https://issues.apache.org/jira/browse/IGNITE-17884
 Project: Ignite
  Issue Type: Bug
Reporter: Vyacheslav Koptilin
 Fix For: 3.0.0-alpha6


There is no way to get an instanace of TopologyService via Ignite interface. 

The following methods just duplicate a part of TopologyService and looks like a 
temporary solution:
{code:java}
/**
 * Gets the cluster nodes.
 * NOTE: Temporary API to enable Compute until we have proper Cluster API.
 *
 * @return Collection of cluster nodes.
 */
Collection clusterNodes();

/**
 * Gets the cluster nodes.
 * NOTE: Temporary API to enable Compute until we have proper Cluster API.
 *
 * @return Collection of cluster nodes.
 */
CompletableFuture> clusterNodesAsync();
 {code}
Need just to expose _TopologeService_ instead of these methods.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-17849) Remove filter from partition fullscan



 [ 
https://issues.apache.org/jira/browse/IGNITE-17849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy reassigned IGNITE-17849:
--

Assignee: Roman Puchkovskiy

> Remove filter from partition fullscan
> -
>
> Key: IGNITE-17849
> URL: https://issues.apache.org/jira/browse/IGNITE-17849
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
>
> See these methods:
> {code:java}
> MvPartitionStorage#scan(Predicate, HybridTimestamp)
> MvPartitionStorage#scan(Predicate, UUID){code}
> This parameter turned out to be useless in current design.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17882) Remove org.apache.ignite.binary.BinaryObject from public API



 [ 
https://issues.apache.org/jira/browse/IGNITE-17882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-17882:
-
Fix Version/s: 3.0.0-alpha6

> Remove org.apache.ignite.binary.BinaryObject from public API
> 
>
> Key: IGNITE-17882
> URL: https://issues.apache.org/jira/browse/IGNITE-17882
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vyacheslav Koptilin
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> It does not make sense to have BinaryObject interface in the public API until 
> IGNITE-14316 is resolved (this ticket should introduce the right interface(s) 
> at least).
> For now, let's just remove BinaryObject.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17883) Remove the invoke method from KeyValurView and RecordView interfaces

Vyacheslav Koptilin created IGNITE-17883:


 Summary: Remove the invoke method from KeyValurView and RecordView 
interfaces
 Key: IGNITE-17883
 URL: https://issues.apache.org/jira/browse/IGNITE-17883
 Project: Ignite
  Issue Type: Bug
Affects Versions: 3.0.0-alpha6
Reporter: Vyacheslav Koptilin


_KeyValueView_ and _RecordView_ provide the following methods:
{code:java}
/**
 * Executes invoke processor code against the value associated with the 
provided key.
 *
 * @param tx The transaction or {@code null} to auto commit.
 * @param key A key associated with the value that invoke processor will be 
applied to. The key cannot be {@code null}.
 * @param proc An invocation processor.
 * @param args Optional invoke processor arguments.
 * @param  Invoke processor result type.
 * @return Result of the processing.
 * @see InvokeProcessor
 */
 R invoke(@Nullable Transaction tx, @NotNull K key, 
InvokeProcessor proc, Serializable... args);

/**
 * Asynchronously executes invoke processor code against the value associated 
with the provided key.
 *
 * @param tx The transaction or {@code null} to auto commit.
 * @param key A key associated with the value that invoke processor will be 
applied to. The key cannot be {@code null}.
 * @param proc An invocation processor.
 * @param args Optional invoke processor arguments.
 * @param  Invoke processor result type.
 * @return Future representing pending completion of the operation.
 * @see InvokeProcessor
 */
@NotNull  CompletableFuture invokeAsync(
@Nullable Transaction tx,
@NotNull K key,
InvokeProcessor proc,
Serializable... args);

/**
 * Executes invoke processor code against values associated with the provided 
keys.
 *
 * @param tx The transaction or {@code null} to auto commit.
 * @param  Invoke processor result type.
 * @param keys Ordered collection of keys which values associated with should 
be processed. The keys cannot be {@code null}.
 * @param proc An invocation processor.
 * @param args Optional invoke processor arguments.
 * @return Results of the processing.
 * @see InvokeProcessor
 */
 Map invokeAll(
@Nullable Transaction tx,
@NotNull Collection keys,
InvokeProcessor proc,
Serializable... args);

/**
 * Asynchronously executes invoke processor code against values associated with 
the provided keys.
 *
 * @param tx The transaction or {@code null} to auto commit.
 * @param  Invoke processor result type.
 * @param keys Ordered collection of keys which values associated with should 
be processed. The keys cannot be {@code null}.
 * @param proc An invocation processor.
 * @param args Optional invoke processor arguments.
 * @return Future representing pending completion of the operation.
 * @see InvokeProcessor
 */
@NotNull  CompletableFuture> invokeAllAsync(
@Nullable Transaction tx,
@NotNull Collection keys,
InvokeProcessor proc,
Serializable... args);
 {code}
The main problem with these methods assume that the user defined closure - 
_InvokeProcessor_ must be idempotent. This is a consequence of the limitations 
of the replication protocol. However, the implementation does not have the 
right way to check this requirement. For example, a simple closure that 
increments a value for a key could lead to data inconsistency under some 
circumstances.

For now, _IgniteCompute.executeColocated_ can be used in order to handle these 
needs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17883) Remove the invoke method from KeyValurView and RecordView interfaces



 [ 
https://issues.apache.org/jira/browse/IGNITE-17883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-17883:
-
Fix Version/s: 3.0.0-alpha6

> Remove the invoke method from KeyValurView and RecordView interfaces
> 
>
> Key: IGNITE-17883
> URL: https://issues.apache.org/jira/browse/IGNITE-17883
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0.0-alpha6
>Reporter: Vyacheslav Koptilin
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> _KeyValueView_ and _RecordView_ provide the following methods:
> {code:java}
> /**
>  * Executes invoke processor code against the value associated with the 
> provided key.
>  *
>  * @param tx The transaction or {@code null} to auto commit.
>  * @param key A key associated with the value that invoke processor will be 
> applied to. The key cannot be {@code null}.
>  * @param proc An invocation processor.
>  * @param args Optional invoke processor arguments.
>  * @param  Invoke processor result type.
>  * @return Result of the processing.
>  * @see InvokeProcessor
>  */
>  R invoke(@Nullable Transaction tx, @NotNull K key, 
> InvokeProcessor proc, Serializable... args);
> /**
>  * Asynchronously executes invoke processor code against the value associated 
> with the provided key.
>  *
>  * @param tx The transaction or {@code null} to auto commit.
>  * @param key A key associated with the value that invoke processor will be 
> applied to. The key cannot be {@code null}.
>  * @param proc An invocation processor.
>  * @param args Optional invoke processor arguments.
>  * @param  Invoke processor result type.
>  * @return Future representing pending completion of the operation.
>  * @see InvokeProcessor
>  */
> @NotNull  CompletableFuture invokeAsync(
> @Nullable Transaction tx,
> @NotNull K key,
> InvokeProcessor proc,
> Serializable... args);
> /**
>  * Executes invoke processor code against values associated with the provided 
> keys.
>  *
>  * @param tx The transaction or {@code null} to auto commit.
>  * @param  Invoke processor result type.
>  * @param keys Ordered collection of keys which values associated with should 
> be processed. The keys cannot be {@code null}.
>  * @param proc An invocation processor.
>  * @param args Optional invoke processor arguments.
>  * @return Results of the processing.
>  * @see InvokeProcessor
>  */
>  Map invokeAll(
> @Nullable Transaction tx,
> @NotNull Collection keys,
> InvokeProcessor proc,
> Serializable... args);
> /**
>  * Asynchronously executes invoke processor code against values associated 
> with the provided keys.
>  *
>  * @param tx The transaction or {@code null} to auto commit.
>  * @param  Invoke processor result type.
>  * @param keys Ordered collection of keys which values associated with should 
> be processed. The keys cannot be {@code null}.
>  * @param proc An invocation processor.
>  * @param args Optional invoke processor arguments.
>  * @return Future representing pending completion of the operation.
>  * @see InvokeProcessor
>  */
> @NotNull  CompletableFuture> invokeAllAsync(
> @Nullable Transaction tx,
> @NotNull Collection keys,
> InvokeProcessor proc,
> Serializable... args);
>  {code}
> The main problem with these methods assume that the user defined closure - 
> _InvokeProcessor_ must be idempotent. This is a consequence of the 
> limitations of the replication protocol. However, the implementation does not 
> have the right way to check this requirement. For example, a simple closure 
> that increments a value for a key could lead to data inconsistency under some 
> circumstances.
> For now, _IgniteCompute.executeColocated_ can be used in order to handle 
> these needs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17882) Remove org.apache.ignite.binary.BinaryObject from public API

Vyacheslav Koptilin created IGNITE-17882:


 Summary: Remove org.apache.ignite.binary.BinaryObject from public 
API
 Key: IGNITE-17882
 URL: https://issues.apache.org/jira/browse/IGNITE-17882
 Project: Ignite
  Issue Type: Bug
Reporter: Vyacheslav Koptilin


It does not make sense to have BinaryObject interface in the public API until 
IGNITE-14316 is resolved (this ticket should introduce the right interface(s) 
at least).

For now, let's just remove BinaryObject.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17882) Remove org.apache.ignite.binary.BinaryObject from public API



 [ 
https://issues.apache.org/jira/browse/IGNITE-17882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-17882:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Remove org.apache.ignite.binary.BinaryObject from public API
> 
>
> Key: IGNITE-17882
> URL: https://issues.apache.org/jira/browse/IGNITE-17882
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vyacheslav Koptilin
>Priority: Major
>  Labels: ignite-3
>
> It does not make sense to have BinaryObject interface in the public API until 
> IGNITE-14316 is resolved (this ticket should introduce the right interface(s) 
> at least).
> For now, let's just remove BinaryObject.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (IGNITE-17862) Restore grade build by adding newly added replicas module to the flow



 [ 
https://issues.apache.org/jira/browse/IGNITE-17862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov resolved IGNITE-17862.

Fix Version/s: 3.0.0-alpha6
   Resolution: Fixed

Looks good to me, thank you!

> Restore grade build by adding newly added replicas module to the flow
> -
>
> Key: IGNITE-17862
> URL: https://issues.apache.org/jira/browse/IGNITE-17862
> Project: Ignite
>  Issue Type: Bug
>  Components: build
>Reporter: Alexander Lapin
>Assignee: Mikhail Pochatkin
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> New replicator module, that was added during tx RW implementation, wasn't 
> added to the gradle build scripts and resources.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (IGNITE-17865) Disable partition ranges in log messages about rebalance or PME



[ 
https://issues.apache.org/jira/browse/IGNITE-17865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616395#comment-17616395
 ] 

Vyacheslav Koptilin edited comment on IGNITE-17865 at 10/12/22 12:45 PM:
-

Hello [~shishkovilja] ,

I don't think that removing this optimized output is a good idea. In my humble 
opinion, a tool that is used for analyzing messages should be improved via 
using regex, for instance, instead of blowing up the log.


was (Author: slava.koptilin):
Hello [~shishkovilja] ,

I don't think that removing this optimized output is a good idea. In my humble 
opinion, a tool that is used for analyzing log messages should be improved via 
using regex, for instance, instead of blowing up the log messages.

> Disable partition ranges in log messages about rebalance or PME
> ---
>
> Key: IGNITE-17865
> URL: https://issues.apache.org/jira/browse/IGNITE-17865
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Ilya Shishkov
>Priority: Minor
>  Labels: ise, newbie
>
> When you need to analyze cases of partitions inconsistency, full list of 
> partitions is needed, but there is an optimization which replaces list of 
> consecutive partitions with ranges. So, as you can see below, we can not find 
> partition 2 by simple text search:
> {quote}
> 2021-08-11 23:12:11.338 [WARN 
> ]\[sys-#213\][org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl]
>  Partitions have been scheduled for rebalancing due to outdated update 
> counter [grp=group, readyTopVer=AffinityTopologyVersion [topVer=33, 
> minorTopVer=0], topVer=AffinityTopologyVersion [topVer=33, minorTopVer=0], 
> nodeId=8e05997a-4ff7-4fdb-9480-cfbcefa8bbf5, 
> partsFull=[{color:red}*1-3*{color}, ...], partsHistorical=[]]
> {quote}
> {quote}
> 2021-08-11 23:12:11.338 [WARN 
> ]\[sys-#213\][org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture]
>  Partitions weren't present in any history reservation: [[[grp=grp2 
> part=[[{color:red}*1-3*{color}]]]
> {quote}
> We need to remove this optimization, because it can lead to mistakes in logs 
> analysis.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17865) Disable partition ranges in log messages about rebalance or PME



[ 
https://issues.apache.org/jira/browse/IGNITE-17865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616395#comment-17616395
 ] 

Vyacheslav Koptilin commented on IGNITE-17865:
--

Hello [~shishkovilja] ,

I don't think that removing this optimized output is a good idea. In my humble 
opinion, a tool that is used for analyzing log messages should be improved via 
using regex, for instance, instead of blowing up the log messages.

> Disable partition ranges in log messages about rebalance or PME
> ---
>
> Key: IGNITE-17865
> URL: https://issues.apache.org/jira/browse/IGNITE-17865
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Ilya Shishkov
>Priority: Minor
>  Labels: ise, newbie
>
> When you need to analyze cases of partitions inconsistency, full list of 
> partitions is needed, but there is an optimization which replaces list of 
> consecutive partitions with ranges. So, as you can see below, we can not find 
> partition 2 by simple text search:
> {quote}
> 2021-08-11 23:12:11.338 [WARN 
> ]\[sys-#213\][org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopologyImpl]
>  Partitions have been scheduled for rebalancing due to outdated update 
> counter [grp=group, readyTopVer=AffinityTopologyVersion [topVer=33, 
> minorTopVer=0], topVer=AffinityTopologyVersion [topVer=33, minorTopVer=0], 
> nodeId=8e05997a-4ff7-4fdb-9480-cfbcefa8bbf5, 
> partsFull=[{color:red}*1-3*{color}, ...], partsHistorical=[]]
> {quote}
> {quote}
> 2021-08-11 23:12:11.338 [WARN 
> ]\[sys-#213\][org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture]
>  Partitions weren't present in any history reservation: [[[grp=grp2 
> part=[[{color:red}*1-3*{color}]]]
> {quote}
> We need to remove this optimization, because it can lead to mistakes in logs 
> analysis.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17881) OrderingFuture should notify dependents asynchronously

Roman Puchkovskiy created IGNITE-17881:
--

 Summary: OrderingFuture should notify dependents asynchronously
 Key: IGNITE-17881
 URL: https://issues.apache.org/jira/browse/IGNITE-17881
 Project: Ignite
  Issue Type: Bug
Reporter: Roman Puchkovskiy
Assignee: Roman Puchkovskiy
 Fix For: 3.0.0-alpha6


Currently, there are the following problems:
 # Callbacks, when invoked, will see the future as incompleted if they directly 
call its getNow() or get() methods
 # If callbacks invoke long or blocking operations, this impedes completion



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17206) .NET: Thin client: Add IgniteSet



 [ 
https://issues.apache.org/jira/browse/IGNITE-17206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-17206:

Fix Version/s: 2.14

> .NET: Thin client: Add IgniteSet
> 
>
> Key: IGNITE-17206
> URL: https://issues.apache.org/jira/browse/IGNITE-17206
> Project: Ignite
>  Issue Type: New Feature
>  Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET
> Fix For: 2.14
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Add IgniteSet data structure to .NET thin client.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-17489) Packaging: Brew package



 [ 
https://issues.apache.org/jira/browse/IGNITE-17489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr reassigned IGNITE-17489:
--

Assignee: Aleksandr

> Packaging: Brew package
> ---
>
> Key: IGNITE-17489
> URL: https://issues.apache.org/jira/browse/IGNITE-17489
> Project: Ignite
>  Issue Type: New Feature
>  Components: build
>Reporter: Mikhail Pochatkin
>Assignee: Aleksandr
>Priority: Major
>  Labels: ignite-3
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17878) RocksDB sorted indexes ignore partitions during the scan



 [ 
https://issues.apache.org/jira/browse/IGNITE-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-17878:
---
Description: If one of bounds is null, scan will return results from some 
other partitions as well.

> RocksDB sorted indexes ignore partitions during the scan
> 
>
> Key: IGNITE-17878
> URL: https://issues.apache.org/jira/browse/IGNITE-17878
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> If one of bounds is null, scan will return results from some other partitions 
> as well.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17875) TX counters with gaps must be logged on every PME



 [ 
https://issues.apache.org/jira/browse/IGNITE-17875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-17875:
--
Description: 
TX counters on PME (when all operations are finished) must contain no gaps.
Such partitions should be listed (with counters) with a special warning.
This warning will help to determine that consistency recovery is required.

Currently, we already have a warning "Failed to update the counter ..." which 
is logged when the primary tries to set backup's update counter to its LWM 
which is less than backups's HWM.
But there will be no warning when primary and backups have the same LWM and 
some gaps.

  was:
TX counters on PME (when all operations are finished) must contain no gaps.
Such partitions should be listed (with counters) with a special warning.
This warning will help to determine that consistency recovery is required.

Currently, we already have a warning "Failed to update the counter ..." which 
logged when primary tries to set backup's update counter to it's LWM which is 
less than backups's HWM. 
But there will be no warning when primary and backups has the same LWM and some 
gaps.


> TX counters with gaps must be logged on every PME
> -
>
> Key: IGNITE-17875
> URL: https://issues.apache.org/jira/browse/IGNITE-17875
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Priority: Major
>  Labels: ise
>
> TX counters on PME (when all operations are finished) must contain no gaps.
> Such partitions should be listed (with counters) with a special warning.
> This warning will help to determine that consistency recovery is required.
> Currently, we already have a warning "Failed to update the counter ..." which 
> is logged when the primary tries to set backup's update counter to its LWM 
> which is less than backups's HWM.
> But there will be no warning when primary and backups have the same LWM and 
> some gaps.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17875) TX counters with gaps must be logged on every PME



 [ 
https://issues.apache.org/jira/browse/IGNITE-17875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-17875:
--
Description: 
TX counters on PME (when all operations are finished) must contain no gaps.
Such partitions should be listed (with counters) with a special warning.
This warning will help to determine that consistency recovery is required.

Currently, we already have a warning "Failed to update the counter ..." which 
logged when primary tries to set backup's update counter to it's LWM which is 
less than backups's HWM. 
But there will be no warning when primary and backups has the same LWM and some 
gaps.

  was:
TX counters on PME (when all operations are finished) must contain no gaps.
Such partitions should be listed (with counters) with a special warning.
This warning will help to determine that consistency recovery is required.

Currently, we already have a warning "Failed to update the counter ..." which 
cause when primary tries to set backup's update counter to it's LWM which is 
less than backups's HWM. 
But there will be no warning when primary and backups has the same LWM and some 
gaps.


> TX counters with gaps must be logged on every PME
> -
>
> Key: IGNITE-17875
> URL: https://issues.apache.org/jira/browse/IGNITE-17875
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Priority: Major
>  Labels: ise
>
> TX counters on PME (when all operations are finished) must contain no gaps.
> Such partitions should be listed (with counters) with a special warning.
> This warning will help to determine that consistency recovery is required.
> Currently, we already have a warning "Failed to update the counter ..." which 
> logged when primary tries to set backup's update counter to it's LWM which is 
> less than backups's HWM. 
> But there will be no warning when primary and backups has the same LWM and 
> some gaps.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17875) TX counters with gaps must be logged on every PME



 [ 
https://issues.apache.org/jira/browse/IGNITE-17875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-17875:
--
Description: 
TX counters on PME (when all operations are finished) must contain no gaps.
Such partitions should be listed (with counters) with a special warning.
This warning will help to determine that consistency recovery is required.

Currently, we already have a warning "Failed to update the counter ..." which 
cause when primary tries to set backup's update counter to it's LWM which is 
less than backups's HWM. 
But there will be no warning when primary and backups has the same LWM and some 
gaps.

  was:
TX counters on PME (when all operations are finished) must contain no gaps.
Such partitions should be listed (with counters) with a special warning.
This warning will help to fix the bugs that causes this.


> TX counters with gaps must be logged on every PME
> -
>
> Key: IGNITE-17875
> URL: https://issues.apache.org/jira/browse/IGNITE-17875
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Priority: Major
>  Labels: ise
>
> TX counters on PME (when all operations are finished) must contain no gaps.
> Such partitions should be listed (with counters) with a special warning.
> This warning will help to determine that consistency recovery is required.
> Currently, we already have a warning "Failed to update the counter ..." which 
> cause when primary tries to set backup's update counter to it's LWM which is 
> less than backups's HWM. 
> But there will be no warning when primary and backups has the same LWM and 
> some gaps.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (IGNITE-17369) Snapshot is inconsistent under streamed loading with 'allowOverwrite==false'.

2022-10-12 Thread Vladimir Steshin (Jira)

[
https://issues.apache.org/jira/browse/IGNITE-17369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612278#comment-17612278
]

Vladimir Steshin edited comment on IGNITE-17369 at 10/12/22 10:36 AM:
--

Snapshot can begin work with different state of kin partitions. The shapshot
process waits for the datastreamer futures.
(_GridCacheMvccManager.addDataStreamerFuture()_). The problem is that these
futures are created separately and concurrently on primary and backups nodes by
_IsolatedUpdater_. As result, at the checkpoint some backups might be written
without the primaries. And opposite. There are no updates accepted during
checkpoint. Late streamer updates is not written to snapshoting partitions.
This verification could produce a warninf of active streamer.

Solutions:

1) V1 (PR 10285).
PR brings watching _DataStreamer_ futures in snapshot process. The futures are
created before writing streamer batch on any node. We cannot relay on the
future as on final and consistent write for streamer batch or certain entry.
But we know that datastreamer is in progress at the checkpoint and that it is
on pause. We can invalidate snapshot at this moment.
In theory the solution is not resilent. On streamer batch could've been
entirely written before snapshot. Second batch after. First batch writes
partition on primaries or backups. Second writes the rest. Snapshot is
inconsistent.

2) V2 (PR 10286).
_IsolatedUpdater_ could just notify snapshot process, if exists, that
concurrent inconsistent update is on. A notification of at least one entry on
any node wound be enough. Should work in practice. In theory the solution is
not resilent. On streamer batch could've been entirely written before snapshot.
Second batch after. First batch writes partition on primaries or backups.
Second writes the rest. Snapshot is inconsistent.

3) V3 (PR 10284).
We could mark that _DataStreamer_ is on on any first streamer batch received.
And unmark somehow later. If _DataStreamer_ is marked as active, the snapshot
process could check this mark. Since the mark is set before writting data, it
is set before the datastreamer future which is being waited for in the snapshot
process. This guaraties the mark are visible before the snapshot.

The problem is how to close such mark. When the streaming node left? Node can
live forever. Send special closing request? The streamer node can do not close
streamer at all. Meaning no _close()_ is invoked. Moreoever, _DataStreamer_
works through _CommunicationSPI_. Which doesn't guarantee delivery. We can't be
sure that closing request is delivered and streamer is unmarked on the
accepting node. Do we need to set this mark with a timeout and re-set with next
datastreamer batche? Which timeout? Bind to what?
On closing requests, a rebalance can happen. Should be processed too. Looks
like we need a discovery closing message. Much simpler and reliable.
Also, datastreamer can be canceled. Meaning one batches were written before
snapshot. Other won't ever.

4) V4 (PR 10299).
We could watch streamer is already registered before snapshot and
simultaneously. The problem is that we need to monitor even client at the
snapshot beginning and make sure they answered whether streamer is on. We could
adjust snapshot process so that it would gather client responses at the start
stage. The process is already has snapshot verification routines.

was (Author: vladsz83):
Snapshot can begin work with different state of kin partitions. The shapshot
process waits for the datastreamer futures.
(_GridCacheMvccManager.addDataStreamerFuture()_). The problem is that these
futures are created separately and concurrently on primary and backups nodes by
_IsolatedUpdater_. As result, at the checkpoint some backups might be written
without the primaries. And opposite. There are no updates accepted during
checkpoint. Late streamer updates is not written to snapshoting partitions.

Solutions:

[jira] [Comment Edited] (IGNITE-17369) Snapshot is inconsistent under streamed loading with 'allowOverwrite==false'.

2022-10-12 Thread Vladimir Steshin (Jira)

[
https://issues.apache.org/jira/browse/IGNITE-17369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612278#comment-17612278
]

Vladimir Steshin edited comment on IGNITE-17369 at 10/12/22 10:34 AM:
--

Solutions:

[jira] [Updated] (IGNITE-17880) Topology version must be extended with topology epoch



 [ 
https://issues.apache.org/jira/browse/IGNITE-17880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anton Vinogradov updated IGNITE-17880:
--
Description: 
Epoch must be incremented each time when topology version changed from 0 to 1 
(when the cluster started or restarted).
Each epoch decrease or invariance on join must be logged as a warning.
Epoch must be logged at every topology version change.

This will 
- help to determine how many times the cluster was restarted (and make it 
easier to determine when)
- checks that the part of the cluster which was restarted several times as a 
standalone cluster will never join the rest of the cluster with the lower epoch 
(check some segmentation and management problems)

  was:
Epoch must be incremented each time when topology version changed from 0 to 1 
(when the cluster started or restarted).
Each epoch decrease or invariance on join must be logged as a warning.
Epoch must be logged at every topology version change.

This will 
- help to determine how many times the cluster was restarted (and make it 
easier to determine when)
- checks that the part of the cluster which was restarted sometimes as a 
standalone cluster will never join the rest of the cluster with the lower epoch 
(check some segmentation and management problems)


> Topology version must be extended with topology epoch
> -
>
> Key: IGNITE-17880
> URL: https://issues.apache.org/jira/browse/IGNITE-17880
> Project: Ignite
>  Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Priority: Major
>
> Epoch must be incremented each time when topology version changed from 0 to 1 
> (when the cluster started or restarted).
> Each epoch decrease or invariance on join must be logged as a warning.
> Epoch must be logged at every topology version change.
> This will 
> - help to determine how many times the cluster was restarted (and make it 
> easier to determine when)
> - checks that the part of the cluster which was restarted several times as a 
> standalone cluster will never join the rest of the cluster with the lower 
> epoch (check some segmentation and management problems)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (IGNITE-17369) Snapshot is inconsistent under streamed loading with 'allowOverwrite==false'.

2022-10-12 Thread Vladimir Steshin (Jira)

[
https://issues.apache.org/jira/browse/IGNITE-17369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612278#comment-17612278
]

Vladimir Steshin edited comment on IGNITE-17369 at 10/12/22 10:34 AM:
--

Solutions:

[jira] [Created] (IGNITE-17880) Topology version must be extended with topology epoch

Anton Vinogradov created IGNITE-17880:
-

 Summary: Topology version must be extended with topology epoch
 Key: IGNITE-17880
 URL: https://issues.apache.org/jira/browse/IGNITE-17880
 Project: Ignite
  Issue Type: Sub-task
Reporter: Anton Vinogradov


Epoch must be incremented each time when topology version changed from 0 to 1 
(when the cluster started or restarted).
Each epoch decrease or invariance on join must be logged as a warning.
Epoch must be logged at every topology version change.

This will 
- help to determine how many times the cluster was restarted (and make it 
easier to determine when)
- checks that the part of the cluster which was restarted sometimes as a 
standalone cluster will never join the rest of the cluster with the lower epoch 
(check some segmentation and management problems)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17879) Packaging is broken

2022-10-12 Thread Vadim Pakhnushev (Jira)

Vadim Pakhnushev created IGNITE-17879:
-

 Summary: Packaging is broken
 Key: IGNITE-17879
 URL: https://issues.apache.org/jira/browse/IGNITE-17879
 Project: Ignite
  Issue Type: Bug
  Components: build
Reporter: Vadim Pakhnushev
Assignee: Vadim Pakhnushev


After integrating IGNITE-17726 and IGNITE-15957 docker container can't start 
due to the missing config file and deb/rpm packages can start due to the 
invalid command line arguments passed from start script.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-16355) .NET: Thin 3.0: Support value types in the Table API



[ 
https://issues.apache.org/jira/browse/IGNITE-16355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616320#comment-17616320
 ] 

Pavel Tupitsyn commented on IGNITE-16355:
-

Merged to main: c4c6821c4b1b66e0a7af59e8af4ba665e0656097

> .NET: Thin 3.0: Support value types in the Table API
> 
>
> Key: IGNITE-16355
> URL: https://issues.apache.org/jira/browse/IGNITE-16355
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Table API is constrained to reference types currently:
> {code}
> interface IRecordView where T : class
> {code}
> Remove this constraint. This will require changing all APIs that potentially 
> return null:
> * Get
> * GetAll
> * GetAndUpsert
> * GetAndReplace
> * GetAndDelete
> Single-key APIs can be changed to {{bool TryGet(K, out V)}} format. GetAll - 
> not clear, investigate.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17870) SQL. Do not yield separate query result for comments

2022-10-12 Thread Ignite TC Bot (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-17870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616311#comment-17616311
 ] 

Ignite TC Bot commented on IGNITE-17870:


{panel:title=Branch: [pull/10303/head] Base: [master] : Possible Blockers 
(7)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cache 2{color} [[tests 0 TIMEOUT , Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=6825361]]

{color:#d04437}Cache 6{color} [[tests 0 Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=6825365]]

{color:#d04437}SPI (Discovery){color} [[tests 0 TIMEOUT , Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=6825445]]

{color:#d04437}Cache (Failover) 2{color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=6825372]]
* IgniteCacheFailoverTestSuite2: 
GridCacheAtomicReplicatedFailoverSelfTest.testTopologyChange - Test has low 
fail rate in base branch 0,0% and is not flaky

{color:#d04437}Cache 5{color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=6825364]]
* IgniteCacheTestSuite5: 
CacheLateAffinityAssignmentTest.testBlockedFinishMsgForClient - Test has low 
fail rate in base branch 0,0% and is not flaky

{color:#d04437}Cache 3{color} [[tests 0 TIMEOUT , Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=6825362]]

{color:#d04437}Queries 1 (lazy=true){color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=6825431]]
* IgniteBinaryCacheQueryLazyTestSuite: 
DynamicIndexServerCoordinatorBasicSelfTest.testCreateNoTablePartitionedTransactionalNear
 - Test has low fail rate in base branch 0,0% and is not flaky

{panel}
{panel:title=Branch: [pull/10303/head] Base: [master] : New Tests 
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Queries 1 (lazy=true){color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=6825431]]
* {color:#013220}IgniteBinaryCacheQueryLazyTestSuite: 
SqlParserMultiStatementSelfTest.testEmptyTransaction - PASSED{color}

{color:#8b}Queries 1{color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=6825430]]
* {color:#013220}IgniteBinaryCacheQueryTestSuite: 
SqlParserMultiStatementSelfTest.testEmptyTransaction - PASSED{color}

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6825461&buildTypeId=IgniteTests24Java8_RunAll]

> SQL. Do not yield separate query result for comments
> 
>
> Key: IGNITE-17870
> URL: https://issues.apache.org/jira/browse/IGNITE-17870
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Andrey Mashenkov
>Assignee: Andrey Mashenkov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Motivation
> Currently, lines with commented out lines yield additional query result. For 
> the query below, three results will be returned (result for comment is 
> “updated rows: 0“). You can play with the comments position within the entire 
> query, the behavior with comments is not consistent and clear.
> -- asdf
> SELECT * FROM Test_Person limit 1;
> update Test_Person set name='sdfsdf' where id = ;
> -- come
> -- asdf
> What to do
> Do not yield a separate query result for commented out lines.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17878) RocksDB sorted indexes ignore partitions during the scan



 [ 
https://issues.apache.org/jira/browse/IGNITE-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-17878:
---
Ignite Flags:   (was: Docs Required,Release Notes Required)

> RocksDB sorted indexes ignore partitions during the scan
> 
>
> Key: IGNITE-17878
> URL: https://issues.apache.org/jira/browse/IGNITE-17878
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17878) RocksDB sorted indexes ignore partitions during the scan



 [ 
https://issues.apache.org/jira/browse/IGNITE-17878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-17878:
---
Labels: ignite-3  (was: )

> RocksDB sorted indexes ignore partitions during the scan
> 
>
> Key: IGNITE-17878
> URL: https://issues.apache.org/jira/browse/IGNITE-17878
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-17878) RocksDB sorted indexes ignore partitions during the scan

Ivan Bessonov created IGNITE-17878:
--

 Summary: RocksDB sorted indexes ignore partitions during the scan
 Key: IGNITE-17878
 URL: https://issues.apache.org/jira/browse/IGNITE-17878
 Project: Ignite
  Issue Type: Bug
Reporter: Ivan Bessonov
Assignee: Ivan Bessonov






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17553) Improve Flow framework with flatMap function

2022-10-12 Thread Semyon Danilov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-17553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Semyon Danilov updated IGNITE-17553:

Reviewer: Semyon Danilov

> Improve Flow framework with flatMap function
> 
>
> Key: IGNITE-17553
> URL: https://issues.apache.org/jira/browse/IGNITE-17553
> Project: Ignite
>  Issue Type: Improvement
>  Components: cli, ignite-3
>Reporter: Mikhail Pochatkin
>Assignee: Ivan Gagarkin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17569) Add an option to display plain tables

2022-10-12 Thread Semyon Danilov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-17569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Semyon Danilov updated IGNITE-17569:

Reviewer: Semyon Danilov

> Add an option to display plain tables
> -
>
> Key: IGNITE-17569
> URL: https://issues.apache.org/jira/browse/IGNITE-17569
> Project: Ignite
>  Issue Type: Task
>  Components: cli
>Reporter: Aleksandr
>Assignee: Ivan Gagarkin
>Priority: Major
>  Labels: ignite-3, ignite-3-cli-tool
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Now CLI shows tables that are pretty rendered. This looks great but makes 
> hard to apply in any pipeline. For example, writing an awk script to show all 
> node names from the cluster might be a challange.
> I suggest to add an option (--plain or something) that will make the CLI to 
> display tables with the plain forrmatting. It coud be columns splitted with 
> \t.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17874) Support List and ByteBuffer in message serialization



 [ 
https://issues.apache.org/jira/browse/IGNITE-17874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-17874:
---
Reviewer: Aleksandr Polovtcev  (was: Semyon Danilov)

> Support List and ByteBuffer in message serialization
> 
>
> Key: IGNITE-17874
> URL: https://issues.apache.org/jira/browse/IGNITE-17874
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17569) Add an option to display plain tables

2022-10-12 Thread Semyon Danilov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-17569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Semyon Danilov updated IGNITE-17569:

Description: 
Now CLI shows tables that are pretty rendered. This looks great but makes hard 
to apply in any pipeline. For example, writing an awk script to show all node 
names from the cluster might be a challange.

I suggest to add an option (--plain or something) that will make the CLI to 
display tables with the plain forrmatting. It coud be columns splitted with \t.

  was:
Now CLI shows tables that are pretty rendered. This looks grate but makes hard 
to apply in any pipeline. For example, writing an awk script to show all node 
names from the cluster might be a challange.

I suggest to add an option (--plain or something) that will make the CLI to 
display tables with the plain forrmatting. It coud be columns splitted with \t.


> Add an option to display plain tables
> -
>
> Key: IGNITE-17569
> URL: https://issues.apache.org/jira/browse/IGNITE-17569
> Project: Ignite
>  Issue Type: Task
>  Components: cli
>Reporter: Aleksandr
>Assignee: Ivan Gagarkin
>Priority: Major
>  Labels: ignite-3, ignite-3-cli-tool
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Now CLI shows tables that are pretty rendered. This looks great but makes 
> hard to apply in any pipeline. For example, writing an awk script to show all 
> node names from the cluster might be a challange.
> I suggest to add an option (--plain or something) that will make the CLI to 
> display tables with the plain forrmatting. It coud be columns splitted with 
> \t.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17249) DefaultMessagingService incorrect message sending order



[ 
https://issues.apache.org/jira/browse/IGNITE-17249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616271#comment-17616271
 ] 

Roman Puchkovskiy commented on IGNITE-17249:


Thanks!

> DefaultMessagingService incorrect message sending order
> ---
>
> Key: IGNITE-17249
> URL: https://issues.apache.org/jira/browse/IGNITE-17249
> Project: Ignite
>  Issue Type: Bug
>  Components: networking
>Reporter: Semyon Danilov
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The issue is in the 
> {{private CompletableFuture sendMessage0(NetworkMessage message, String 
> recipientConsistentId, InetSocketAddress addr)}} method.
> {{connectionManager.channel(recipientConsistentId, addr)}} returns a future 
> and we add a "listener" via {{thenCompose}} method. The thing is that it adds 
> a listener to the start of the queue, so when the channel is established the 
> latest listener will be executed first.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-17863) ThreadNameValidationTest fails if run alone (or first in suite)



[ 
https://issues.apache.org/jira/browse/IGNITE-17863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616270#comment-17616270
 ] 

Roman Puchkovskiy commented on IGNITE-17863:


Thanks!

> ThreadNameValidationTest fails if run alone (or first in suite)
> ---
>
> Key: IGNITE-17863
> URL: https://issues.apache.org/jira/browse/IGNITE-17863
> Project: Ignite
>  Issue Type: Bug
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: MakeTeamcityGreenAgain
> Fix For: 2.15
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> This seems to happen because code that configures Log4j logging system 
> triggers creation of a thread factory after the test saves the count of 
> created thread factories, so the test thinks that there is some production 
> code doing so, even though it's an artifact of our test code.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-17759) Need to pass commitTableId and commitPartitionId to MvPartitionStorage#addWrite



 [ 
https://issues.apache.org/jira/browse/IGNITE-17759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin reassigned IGNITE-17759:


Assignee: Denis Chudov

> Need to pass commitTableId and commitPartitionId to 
> MvPartitionStorage#addWrite
> ---
>
> Key: IGNITE-17759
> URL: https://issues.apache.org/jira/browse/IGNITE-17759
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Sergey Uttsel
>Assignee: Denis Chudov
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Currently when PartitionListener invokes MvPartitionStorage#addWrite it 
> passes 
> UUID.randomUUID() as commitTableId and 0 as commitPartitionId. Need pass 
> appropriate values.
>  
> For this:
>  # Need to create
> {code:java}
> class PartitionId {
>     UUID tableId;
>     int partId;
> }{code}
>  # In InternalTableImpl#enlistInTx need to save PartitionId of the first 
> operation to the Transaction.
>  # Need to change {color:#172b4d}Map Long>> enlisted = new ConcurrentSkipListMap<>(){color} to Map IgniteBiTuple> enlisted = new ConcurrentHashMap<>();
>  # Need to change String groupId to PartitionId groupId in all places.
>  # In InternalTableImpl#enlistInTx need to pass PartitionId into 
> ReplicaRequest (ReadWriteSingleRowReplicaRequest, 
> ReadWriteSwapRowReplicaRequest, ReadWriteMultiRowReplicaRequest)
>  # In PartitionReplicaListener need to pass PartitionId from ReplicaRequest 
> to UpdateCommand and UpdateAllCommand.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-17874) Support List and ByteBuffer in message serialization