[jira] [Commented] (IGNITE-15567) Calcite. Support REPEAT functionality or remove it from tests.
[
https://issues.apache.org/jira/browse/IGNITE-15567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794562#comment-17794562
]
Evgeny Stanilovsky commented on IGNITE-15567:
-
REPEAT has different syntax :
{code:java}
SELECT repeat(1, 10*1024)
{code}
works properly well.
> Calcite. Support REPEAT functionality or remove it from tests.
> --
>
> Key: IGNITE-15567
> URL: https://issues.apache.org/jira/browse/IGNITE-15567
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Evgeny Stanilovsky
>Priority: Major
> Labels: calcite, calcite2-required, calcite3-required, ignite-3
>
> Now failed:
> {noformat}
> CREATE TABLE test2 AS SELECT * FROM repeat(1, 10*1024) t1(b), (SELECT 10)
> t2(c);
> {noformat}
> {noformat}
> /join/inner/test_join_duplicates.test[_ignored]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-21035) Sql. Fix type inference for dynamic parameters in operators.
[
https://issues.apache.org/jira/browse/IGNITE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-21035:
--
Description:
Dynamic parameters used in operators require additional type casts to work
properly, since calcite's type checkers use type families that consist of
multiple types.
Several examples:
{code:java}
// val is VARCHAR
INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
// Error:
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
' || '
' || '
{code}
Function call (INTEGER is a type family):
{code:java}
"SELECT substring('asd', ?) FROM t1"
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
line 1, column 8 to line 1, column 26: Ambiguous operator SUBSTRING(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
{code}
Let's fix this behaviour by selecting the most common type in a type family:
* For the first case it means that the validator should select use a type of a
specified argument, when both arguments are not set it should return an error.
* In the second example the validator should for the second one it should
select an INTEGER type.
was:
Dynamic parameters used in operators require additional type casts to work
properly, since calcite's type checkers use type families that consist of
multiple types.
Several examples:
{code:java}
// val is VARCHAR
INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
// Error:
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
' || '
' || '
{code}
Function call (INTEGER is a type family):
{code:java}
"SELECT substring('asd', ?) FROM t1"
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
line 1, column 8 to line 1, column 26: Ambiguous operator SUBSTRING(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
{code}
Let's fix this behaviour by selecting the most common type in a type family:
* For the first case it means that the validator should select use a type of a
specified argument, when both arguments are not set it should return an error.
* In the second example the validator should for the second one it should
select an INTEGER type, the first argument should still require a VARCHAR.
> Sql. Fix type inference for dynamic parameters in operators.
>
>
> Key: IGNITE-21035
> URL: https://issues.apache.org/jira/browse/IGNITE-21035
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Dynamic parameters used in operators require additional type casts to work
> properly, since calcite's type checkers use type families that consist of
> multiple types.
> Several examples:
> {code:java}
> // val is VARCHAR
> INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
> // Error:
> org.apache.ignite.sql.SqlException: IGN-SQL-6
> TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
> line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
> ). Dynamic parameter requires adding explicit type cast. Supported
> form(s):
> ' || '
> ' || '
> {code}
> Function call (INTEGER is a type family):
> {code:java}
> "SELECT substring('asd', ?) FROM t1"
> org.apache.ignite.sql.SqlException: IGN-SQL-6
> TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
> line 1, column 8 to line 1, column 26: Ambiguous operator
> SUBSTRING(, ). Dynamic parameter requires adding explicit
> type cast. Supported form(s):
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> {code}
> Let's fix this behaviour by selecting the most common type in a type family:
> * For the first case it means that the validator should select use a type of
> a speci
[jira] [Updated] (IGNITE-21039) Network performance optimization
[
https://issues.apache.org/jira/browse/IGNITE-21039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Belyak updated IGNITE-21039:
--
Description:
I've run several test to find out the MessagingService performance metrics and
that is what I've found:
{noformat}
TestBoolaMessage 139MB/sec WARD
TestByteaMessage 132MB/sec WARD
TestDoubleaMessage 102MB/sec WARD
TestFloataMessage 132MB/sec WARD
TestDoubleaMessage 130MB/sec WARD
TestLongaMessage 131MB/sec WARD
TestDoubleaMessage 131MB/sec WARD
TestStringaMessage 280MB/sec WARD
TestBoolMessage 11MB/sec WARD WARD
TestByteMessage 12MB/sec WARD
TestDoubleMessage 12MB/sec WARD
TestFloatMessage 13MB/sec WARD
TestIntMessage 12MB/sec WARD
TestLongMessage 11MB/sec WARD
TestShortMessage 12MB/sec WARD
TestStringMessage 18MB/sec WARD
TestBool20Message 15MB/sec WARD
TestByte20Message 12MB/sec WARD
TestDouble20Message 32MB/sec WARD
TestFloat20Message 22MB/sec WARD
TestInt20Message 13MB/sec WARD
TestLong20Message 14MB/sec WARD
TestShort20Message 14MB/sec WARD
TestString20Message 65MB/sec WARD
{noformat}
All messages were sent in the same setup: 2 server nodes, connected with a
*10GBit* interface. *Iperf3* (iperf3 --time 30 --zerocopy --client
192.168.1.126 --omit 3 --interval 1 --length 16384 --window 131072 --parallel 2
--json --version4) shows about *850MB/sec* network throughput. But the *best
AI3* result was only {*}280MB/sec{*}. Upper results use 3 type of messages:
1. {*}TestaMessage{*}: array of 163840 elements (primitive, except
String) of type .
2. {*}TestMessage{*}: single property (primitive, except String) of type
3. {*}Test20Message{*}: 20 property (primitive, except String) of type
All the messages were sent in parallel from the single thread with the window
of 100 messages (right after getting another first ack - the new message were
sent).
It was expected, that network utilization low for the very short messages (like
1 int or 20 int fields), but in comparison with the iperf3 results, the
performance of MessagingService for 163KBytes messages was very low. It became
significantly better only while sending huge array of strings (same string
"{color:#067d17}Test string to check message service performance.{color}").
I've run another butch of tests with 1KB byte[] property in the message in 1
and 8 threads and without send window at all (each thread sends next message
after getting the ack for the previous one):
* *1 thread* and got *37 MBytes/sec*
* *8 threads* and got *63 MBytes/sec* result.
So I suppose there is pretty much contention.
All messages were sent in the followin manner:
{code:java}
private void send(ClusterNode target, NetworkMessage msg) {
messagingService.send(target, msg).handle((v, t) -> {
if (t != null) {
LOG.info("Error while sending huge message", t);
}
if (time() < timeout) {
send(target, msg);
}
}{code}
*
was:
I've run several test to find out the MessagingService performance metrics and
that is what I've found:
{noformat}
TestBoolaMessage 139MB/sec WARD
TestByteaMessage 132MB/sec WARD
TestDoubleaMessage 102MB/sec WARD
TestFloataMessage 132MB/sec WARD
TestDoubleaMessage 130MB/sec WARD
TestLongaMessage 131MB/sec WARD
TestDoubleaMessage 131MB/sec WARD
TestStringaMessage 280MB/sec WARD
TestBoolMessage 11MB/sec WARD WARD
TestByteMessage 12MB/sec WARD
TestDoubleMessage 12MB/sec WARD
TestFloatMessage 13MB/sec WARD
TestIntMessage 12MB/sec WARD
TestLongMessage 11MB/sec WARD
TestShortMessage 12MB/sec WARD
TestStringMessage 18MB/sec WARD
TestBool20Message 15MB/sec WARD
TestByte20Message 12MB/sec WARD
TestDouble20Message 32MB/sec WARD
TestFloat20Message 22MB/sec WARD
TestInt20Message 13MB/sec WARD
TestLong20Message 14MB/sec WARD
TestShort20Message 14MB/sec WARD
TestString20Message 65MB/sec WARD
{noformat}
All messages were sent in the same setup: 2 server nodes, connected with a
*10GBit* interface. *Iperf3* (iperf3 --time 30 --zerocopy --client
192.168.1.126 --omit 3 --interval 1 --length 16384 --window 131072 --parallel 2
--json --version4) shows about *850MB/sec* network throughput. But the *best
AI3* result was only {*}280MB/sec{*}. Upper results use 3 type of messages:
1. {*}TestaMessage{*}: array of 163840 elements (primitive, except
String) of type .
2. {*}TestMessage{*}: single property (primitive, except String) of type
3. {*}Test20Message{*}: 20 property (primitive, except String) of type
All the messages were sent in parallel from the single thread with the window
of 100 messages (right after getting another first ack - the new message were
sent).
It was expected, that network utilization low for the very short messages (like
1 int or 20 int fields), but in comparison with the iperf3 results, the
performance of MessagingService for 163KBytes messages was very low. It became
significantly better only while sending huge array of strings (same string
"{color:#067d17}Test string to chec
[jira] [Updated] (IGNITE-21039) Network performance optimization
[
https://issues.apache.org/jira/browse/IGNITE-21039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Belyak updated IGNITE-21039:
--
Description:
I've run several test to find out the MessagingService performance metrics and
that is what I've found:
{noformat}
TestBoolaMessage 139MB/sec WARD
TestByteaMessage 132MB/sec WARD
TestDoubleaMessage 102MB/sec WARD
TestFloataMessage 132MB/sec WARD
TestDoubleaMessage 130MB/sec WARD
TestLongaMessage 131MB/sec WARD
TestDoubleaMessage 131MB/sec WARD
TestStringaMessage 280MB/sec WARD
TestBoolMessage 11MB/sec WARD WARD
TestByteMessage 12MB/sec WARD
TestDoubleMessage 12MB/sec WARD
TestFloatMessage 13MB/sec WARD
TestIntMessage 12MB/sec WARD
TestLongMessage 11MB/sec WARD
TestShortMessage 12MB/sec WARD
TestStringMessage 18MB/sec WARD
TestBool20Message 15MB/sec WARD
TestByte20Message 12MB/sec WARD
TestDouble20Message 32MB/sec WARD
TestFloat20Message 22MB/sec WARD
TestInt20Message 13MB/sec WARD
TestLong20Message 14MB/sec WARD
TestShort20Message 14MB/sec WARD
TestString20Message 65MB/sec WARD
{noformat}
All messages were sent in the same setup: 2 server nodes, connected with a
*10GBit* interface. *Iperf3* (iperf3 --time 30 --zerocopy --client
192.168.1.126 --omit 3 --interval 1 --length 16384 --window 131072 --parallel 2
--json --version4) shows about *850MB/sec* network throughput. But the *best
AI3* result was only {*}280MB/sec{*}. Upper results use 3 type of messages:
1. {*}TestaMessage{*}: array of 163840 elements (primitive, except
String) of type .
2. {*}TestMessage{*}: single property (primitive, except String) of type
3. {*}Test20Message{*}: 20 property (primitive, except String) of type
All the messages were sent in parallel from the single thread with the window
of 100 messages (right after getting another first ack - the new message were
sent).
It was expected, that network utilization low for the very short messages (like
1 int or 20 int fields), but in comparison with the iperf3 results, the
performance of MessagingService for 163KBytes messages was very low. It became
significantly better only while sending huge array of strings (same string
"{color:#067d17}Test string to check message service performance.{color}").
I've run another butch of tests with 1KB byte[] property in the message in 1
and 8 threads and without send window at all (each thread sends next message
after getting the ack for the previous one):
* *1 thread* and got *37 MBytes/sec*
* *8 threads* and got *63 MBytes/sec* result.
So I suppose there is pretty much contention.
All messages were sent in the followin manner:
{code:java}
private void send(ClusterNode target, NetworkMessage msg) {
messagingService.send(target, msg).handle((v, t) -> {
if (t != null) {
LOG.info("Error while sending huge message", t);
}
if (time() < timeout) {
send(target, msg);
}
}{code}
*
was:
I've run several test to find out the MessagingService performance metrics and
that is what I've found:
{noformat}
TestBoolaMessage 139MB/sec WARD
TestByteaMessage 132MB/sec WARD
TestDoubleaMessage 102MB/sec WARD
TestFloataMessage 132MB/sec WARD
TestDoubleaMessage 130MB/sec WARD
TestLongaMessage 131MB/sec WARD
TestDoubleaMessage 131MB/sec WARD
TestStringaMessage 280MB/sec WARD
TestBoolMessage 11MB/sec WARD WARD
TestByteMessage 12MB/sec WARD
TestDoubleMessage 12MB/sec WARD
TestFloatMessage 13MB/sec WARD
TestIntMessage 12MB/sec WARD
TestLongMessage 11MB/sec WARD
TestShortMessage 12MB/sec WARD
TestStringMessage 18MB/sec WARD
TestBool20Message 15MB/sec WARD
TestByte20Message 12MB/sec WARD
TestDouble20Message 32MB/sec WARD
TestFloat20Message 22MB/sec WARD
TestInt20Message 13MB/sec WARD
TestLong20Message 14MB/sec WARD
TestShort20Message 14MB/sec WARD
TestString20Message 65MB/sec WARD
{noformat}
All messages were sent in the same setup: 2 server nodes, connected with a
*10GBit* interface. *Iperf3* (iperf3 --time 30 --zerocopy --client
192.168.1.126 --omit 3 --interval 1 --length 16384 --window 131072 --parallel 2
--json --version4) shows about *850MB/sec* network throughput. But the *best
AI3* result was only {*}280MB/sec{*}. Upper results use 3 type of messages:
1. {*}TestaMessage{*}: array of 163840 elements (primitive, except
String) of type .
2. {*}TestMessage{*}: single property (primitive, except String) of type
3. {*}Test20Message{*}: 20 property (primitive, except String) of type
All the messages were sent in parallel from the single thread with the window
of 100 messages (right after getting another first ack - the new message were
sent).
It was expected, that network utilization low for the very short messages (like
1 int or 20 int fields), but in comparison with the iperf3 results, the
performance of MessagingService for 163KBytes messages was very low. It became
significantly better only while sending huge array of strings (same string
"{color:#067d17}Test string to chec
[jira] [Updated] (IGNITE-17700) Realtime CDC
[
https://issues.apache.org/jira/browse/IGNITE-17700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Timonin updated IGNITE-17700:
Fix Version/s: 2.16
> Realtime CDC
>
>
> Key: IGNITE-17700
> URL: https://issues.apache.org/jira/browse/IGNITE-17700
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Maksim Timonin
>Priority: Major
> Labels: IEP-104, IEP-59, important, ise
> Fix For: 2.16
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Right now, timeout between CDC event happens and consumer notified about it
> is relatively big.
> It sums from the following:
> * {{{}DataStorageConfiguration#walForceArchiveTimeout{}}}.
> * segment archivation time.
> * {{CdcConfiguration#checkFrequence}}
> * time to consume previous events.
> Advantages of this way described in
> [IEP-59|https://cwiki.apache.org/confluence/display/IGNITE/IEP-59+Change+Data+Capture]
> So if user wants to minimize this timeouts we can implement the following:
> * Collects cdc event inside fixed-size buffer in node.
> * Consume events from buffer by CDCConsumer inside node process.
> * Store WALPointer of last event added to buffer.
> * On buffer overflow stop push events to it.
> ** Fallback to WAL segments iteration
> ** When iteration catch up WAL tail switch back to the buffer.
> * On node failover recover consumer with the cdc state as in ignite-cdc.
> With this approach most of the time CDCConsumer will consume events in near
> realtime fashion.
> Description of the feature:
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-104+CDC%3A+Realtime+mode
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17700) Realtime CDC
[
https://issues.apache.org/jira/browse/IGNITE-17700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Timonin updated IGNITE-17700:
Ignite Flags: (was: Docs Required,Release Notes Required)
> Realtime CDC
>
>
> Key: IGNITE-17700
> URL: https://issues.apache.org/jira/browse/IGNITE-17700
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Maksim Timonin
>Priority: Major
> Labels: IEP-104, IEP-59, important, ise
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Right now, timeout between CDC event happens and consumer notified about it
> is relatively big.
> It sums from the following:
> * {{{}DataStorageConfiguration#walForceArchiveTimeout{}}}.
> * segment archivation time.
> * {{CdcConfiguration#checkFrequence}}
> * time to consume previous events.
> Advantages of this way described in
> [IEP-59|https://cwiki.apache.org/confluence/display/IGNITE/IEP-59+Change+Data+Capture]
> So if user wants to minimize this timeouts we can implement the following:
> * Collects cdc event inside fixed-size buffer in node.
> * Consume events from buffer by CDCConsumer inside node process.
> * Store WALPointer of last event added to buffer.
> * On buffer overflow stop push events to it.
> ** Fallback to WAL segments iteration
> ** When iteration catch up WAL tail switch back to the buffer.
> * On node failover recover consumer with the cdc state as in ignite-cdc.
> With this approach most of the time CDCConsumer will consume events in near
> realtime fashion.
> Description of the feature:
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-104+CDC%3A+Realtime+mode
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21042) Move #toLong to IgniteUtils class
Maksim Timonin created IGNITE-21042: --- Summary: Move #toLong to IgniteUtils class Key: IGNITE-21042 URL: https://issues.apache.org/jira/browse/IGNITE-21042 Project: Ignite Issue Type: Improvement Reporter: Maksim Timonin Assignee: Maksim Timonin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17700) Realtime CDC
[
https://issues.apache.org/jira/browse/IGNITE-17700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794336#comment-17794336
]
Ignite TC Bot commented on IGNITE-17700:
{panel:title=Branch: [pull/11044/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/11044/head] Base: [master] : New Tests
(44)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}PDS 2{color} [[tests
22|https://ci2.ignite.apache.org/viewLog.html?buildId=7646779]]
* {color:#013220}IgnitePdsTestSuite2:
CdcIgniteNodeActiveModeTest.testSwitchToCdcUtilityActiveMode - PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcManagerTest.testProhibitedWalModes[persistentEnabled=false] - PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcManagerTest.testRestartNode[persistentEnabled=false] - PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcManagerTest.testMultipleSegmentContent[persistentEnabled=false] -
PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcManagerTest.testSingleSegmentContent[persistentEnabled=false] - PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcManagerTest.testDisableByProperty[persistentEnabled=false] - PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcManagerTest.testRestartNode[persistentEnabled=true] - PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcManagerTest.testMultipleSegmentContent[persistentEnabled=true] -
PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcManagerTest.testSingleSegmentContent[persistentEnabled=true] - PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcManagerTest.testMultipleSegmentContentWithForceNextSegmentRollover[persistentEnabled=true]
- PASSED{color}
* {color:#013220}IgnitePdsTestSuite2:
CdcManagerTest.testMultipleSegmentContentWithForceNextSegmentRollover[persistentEnabled=false]
- PASSED{color}
... and 11 new tests
{color:#8b}Disk Page Compressions 2{color} [[tests
22|https://ci2.ignite.apache.org/viewLog.html?buildId=7646783]]
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcIgniteNodeActiveModeTest.testSwitchToCdcUtilityActiveMode - PASSED{color}
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcManagerTest.testSingleSegmentContent[persistentEnabled=false] - PASSED{color}
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcManagerTest.testMultipleSegmentContentWithForceNextSegmentRollover[persistentEnabled=false]
- PASSED{color}
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcManagerTest.testMultipleSegmentContentWithForceCurrentSegmentRollover[persistentEnabled=false]
- PASSED{color}
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcManagerTest.testDisableByProperty[persistentEnabled=true] - PASSED{color}
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcManagerTest.testDisableByProperty[persistentEnabled=false] - PASSED{color}
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcManagerTest.testProhibitedWalModes[persistentEnabled=false] - PASSED{color}
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcManagerTest.testRestartNode[persistentEnabled=false] - PASSED{color}
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcManagerTest.testMultipleSegmentContent[persistentEnabled=false] -
PASSED{color}
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcManagerTest.testMultipleSegmentContentWithForceNextSegmentRollover[persistentEnabled=true]
- PASSED{color}
* {color:#013220}IgnitePdsCompressionTestSuite2:
CdcManagerTest.testMultipleSegmentContentWithForceCurrentSegmentRollover[persistentEnabled=true]
- PASSED{color}
... and 11 new tests
{panel}
[TeamCity *--> Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7628651&buildTypeId=IgniteTests24Java8_RunAll]
> Realtime CDC
>
>
> Key: IGNITE-17700
> URL: https://issues.apache.org/jira/browse/IGNITE-17700
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Maksim Timonin
>Priority: Major
> Labels: IEP-104, IEP-59, important, ise
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Right now, timeout between CDC event happens and consumer notified about it
> is relatively big.
> It sums from the following:
> * {{{}DataStorageConfiguration#walForceArchiveTimeout{}}}.
> * segment archivation time.
> * {{CdcConfiguration#checkFrequence}}
> * time to consume previous events.
> Advantages of this way described in
> [IEP-59|https://cwiki.apache.org/confluence/display/IGNITE/IEP-59+Change+Data+Capture]
> So if user wants to minimize this timeouts we can implement the following:
> * Collects cdc event inside fixed-size buffer in node.
> * Consume events from buffer by CDCConsumer inside node process.
> * Store WALPointer of last event added to buffer.
> * On buffer overflow stop push events
[jira] [Created] (IGNITE-21041) FilteredRecord instance must be local to RecordSerializer
Maksim Timonin created IGNITE-21041: --- Summary: FilteredRecord instance must be local to RecordSerializer Key: IGNITE-21041 URL: https://issues.apache.org/jira/browse/IGNITE-21041 Project: Ignite Issue Type: Improvement Reporter: Maksim Timonin Assignee: Maksim Timonin In case of concurrent run WALIterators they concurrent for FilteredRecord#size field. Then FilteredRecord must be singleton in context of single RecordSerializer. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-20811) Refactoring DataInput hierarchy
[ https://issues.apache.org/jira/browse/IGNITE-20811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Timonin resolved IGNITE-20811. - Resolution: Won't Fix > Refactoring DataInput hierarchy > --- > > Key: IGNITE-20811 > URL: https://issues.apache.org/jira/browse/IGNITE-20811 > Project: Ignite > Issue Type: Improvement >Reporter: Maksim Timonin >Assignee: Maksim Timonin >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Current hierarchy should be redesigned, as such methods like "position", > "size" and Crc32CheckingDataInput should be part of ByteBufferedDataInput. > Now they are implemented only for FileInput. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-20824) Base abstract class for WALIterator should not depend on file
[ https://issues.apache.org/jira/browse/IGNITE-20824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Timonin resolved IGNITE-20824. - Resolution: Won't Fix > Base abstract class for WALIterator should not depend on file > - > > Key: IGNITE-20824 > URL: https://issues.apache.org/jira/browse/IGNITE-20824 > Project: Ignite > Issue Type: Improvement >Reporter: Maksim Timonin >Assignee: Maksim Timonin >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Iterator over pure ByteBuffer must be implemented also. It doesn't depend on > files IO. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20971) The Ignite process huge memory overhead for tables creation
[
https://issues.apache.org/jira/browse/IGNITE-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794285#comment-17794285
]
Ivan Bessonov commented on IGNITE-20971:
The fix doesn't cover all issues. The main reason for memory leak is the fact
that node got stuck in a deadlock, and kept trying to execute operations in
background, accumulating big amounts of completable futures, for example.
The deadlock went like this:
* we have slow meta-storage processing
* we receive lease "12:00:00 - 12:00:05"
* current moment is "12:00:10", because previous processed watch event was too
long
* we're attempting to start building indexes
* we request current lease, using "now()" as time
* "now()" will never happen, because we're blocking watch thread
Fix - replace "now()" with "lease.startTime", so that we always find the lease
that we need.
> The Ignite process huge memory overhead for tables creation
> ---
>
> Key: IGNITE-20971
> URL: https://issues.apache.org/jira/browse/IGNITE-20971
> Project: Ignite
> Issue Type: Bug
> Components: general
>Affects Versions: 3.0.0-beta2
>Reporter: Igor
>Assignee: Ivan Bessonov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Attachments: gc.20231128_132812_386649.log
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> *Steps to reproduce*
> # Start process with `-Xms4096m -Xmx4096m`
> # Create tables with 5 columns one by one up to 1000.
> *Expected:*
> 1000 tables are created.
>
> *Actual:*
> After ~219 the process was killed by OOM killer due to process took 64GB of
> memory of available 65GB.
>
> *Additional information:*
> OOM killer output:
> {code:java}
> [Tue Nov 28 18:35:37 2023]
> oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-10002.slice/session-201.scope,task=java,pid=11748,uid=10002
> [Tue Nov 28 18:35:37 2023] Out of memory: Killed process 11748 (java)
> total-vm:97160836kB, anon-rss:64192728kB, file-rss:0kB, shmem-rss:0kB,
> UID:10002 pgtables:127948kB oom_score_adj:0
> [Tue Nov 28 18:35:39 2023] oom_reaper: reaped process 11748 (java), now
> anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
> pubagent:~$ grep MemTotal /proc/meminfo
> MemTotal: 65038556 kB
> pubagent:~$ free
> totalusedfree shared buff/cache available
> Mem:65038556 32184864557116 900 159592
> 64174540
> Swap: 0 0 0
> {code}
> GC log
> [^gc.20231128_132812_386649.log]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21040) Placement driver logging enhancement
Vladislav Pyatkov created IGNITE-21040:
--
Summary: Placement driver logging enhancement
Key: IGNITE-21040
URL: https://issues.apache.org/jira/browse/IGNITE-21040
Project: Ignite
Issue Type: Improvement
Reporter: Vladislav Pyatkov
h3. Motivation
# The lease grant request has a filed {{leaseExpirationTime}}, but the filed
really means the timestamp until which we are waiting for a response to this
message.
# For some reason (external for the placement driver), the metastorage event
can handle slower and slower. Finally, we cannot receive an event about the
lease being prolonged for a particular period when the lease interval is valid.
h3. Definition of done
# Rename {{leaseExpirationTime}} in the lease grant message to
{{offerExpirationTime}}.
# Add a warning when we are handling the metastorage event for longer than 500
milliseconds.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21039) Network performance optimization
Alexander Belyak created IGNITE-21039:
-
Summary: Network performance optimization
Key: IGNITE-21039
URL: https://issues.apache.org/jira/browse/IGNITE-21039
Project: Ignite
Issue Type: Improvement
Components: networking
Affects Versions: 3.0
Reporter: Alexander Belyak
I've run several test to find out the MessagingService performance metrics and
that is what I've found:
{noformat}
TestBoolaMessage 139MB/sec WARD
TestByteaMessage 132MB/sec WARD
TestDoubleaMessage 102MB/sec WARD
TestFloataMessage 132MB/sec WARD
TestDoubleaMessage 130MB/sec WARD
TestLongaMessage 131MB/sec WARD
TestDoubleaMessage 131MB/sec WARD
TestStringaMessage 280MB/sec WARD
TestBoolMessage 11MB/sec WARD WARD
TestByteMessage 12MB/sec WARD
TestDoubleMessage 12MB/sec WARD
TestFloatMessage 13MB/sec WARD
TestIntMessage 12MB/sec WARD
TestLongMessage 11MB/sec WARD
TestShortMessage 12MB/sec WARD
TestStringMessage 18MB/sec WARD
TestBool20Message 15MB/sec WARD
TestByte20Message 12MB/sec WARD
TestDouble20Message 32MB/sec WARD
TestFloat20Message 22MB/sec WARD
TestInt20Message 13MB/sec WARD
TestLong20Message 14MB/sec WARD
TestShort20Message 14MB/sec WARD
TestString20Message 65MB/sec WARD
{noformat}
All messages were sent in the same setup: 2 server nodes, connected with a
*10GBit* interface. *Iperf3* (iperf3 --time 30 --zerocopy --client
192.168.1.126 --omit 3 --interval 1 --length 16384 --window 131072 --parallel 2
--json --version4) shows about *850MB/sec* network throughput. But the *best
AI3* result was only {*}280MB/sec{*}. Upper results use 3 type of messages:
1. {*}TestaMessage{*}: array of 163840 elements (primitive, except
String) of type .
2. {*}TestMessage{*}: single property (primitive, except String) of type
3. {*}Test20Message{*}: 20 property (primitive, except String) of type
All the messages were sent in parallel from the single thread with the window
of 100 messages (right after getting another first ack - the new message were
sent).
It was expected, that network utilization low for the very short messages (like
1 int or 20 int fields), but in comparison with the iperf3 results, the
performance of MessagingService for 163KBytes messages was very low. It became
significantly better only while sending huge array of strings (same string
"{color:#067d17}Test string to check message service performance.{color}").
I've run another butch of tests with 1KB byte[] property in the message in 1
and 8 threads and without send window at all (each thread sends next message
after getting the ack for the previous one):
** 1 thread* and got *37 MBytes/sec*
*** *8 threads* and got *63 MBytes/sec* result.
So I suppose there is pretty much contention.
All messages were sent in the followin manner:
{code:java}
private void send(ClusterNode target, NetworkMessage msg) {
messagingService.send(target, msg).handle((v, t) -> {
if (t != null) {
LOG.info("Error while sending huge message", t);
}
if (time() < timeout) {
send(target, msg);
}
}{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-21029) Update Ignite dependency log4j2
[ https://issues.apache.org/jira/browse/IGNITE-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794270#comment-17794270 ] Aleksandr Nikolaev commented on IGNITE-21029: - [~mpetrov] thanks for the review! > Update Ignite dependency log4j2 > --- > > Key: IGNITE-21029 > URL: https://issues.apache.org/jira/browse/IGNITE-21029 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Nikolaev >Assignee: Aleksandr Nikolaev >Priority: Major > Labels: ise > Fix For: 2.17 > > Time Spent: 20m > Remaining Estimate: 0h > > Update Ignite dependency log4j2 2.20.0 to 2.22.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21038) Ignite logs in tests should reside in the work directory
[
https://issues.apache.org/jira/browse/IGNITE-21038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-21038:
-
Description: Currently, when running tests, "ignite.log" files are created
in the "build" directory. This is not very convenient, because this directory
only gets cleaned when {{gradlew clean}} command is executed. When running many
tests, this can clump up the "build" directory pretty quickly. I propose to
move these logs to the directory created by the {{WorkDirectoryExtension}},
which automatically cleans up this folder and allows us to preserve it, if
needed.
> Ignite logs in tests should reside in the work directory
>
>
> Key: IGNITE-21038
> URL: https://issues.apache.org/jira/browse/IGNITE-21038
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Minor
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Currently, when running tests, "ignite.log" files are created in the "build"
> directory. This is not very convenient, because this directory only gets
> cleaned when {{gradlew clean}} command is executed. When running many tests,
> this can clump up the "build" directory pretty quickly. I propose to move
> these logs to the directory created by the {{WorkDirectoryExtension}}, which
> automatically cleans up this folder and allows us to preserve it, if needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21038) Ignite logs in tests should reside in the work directory
Aleksandr Polovtcev created IGNITE-21038: Summary: Ignite logs in tests should reside in the work directory Key: IGNITE-21038 URL: https://issues.apache.org/jira/browse/IGNITE-21038 Project: Ignite Issue Type: Improvement Reporter: Aleksandr Polovtcev Assignee: Aleksandr Polovtcev -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21038) Ignite logs in tests should reside in the work directory
[ https://issues.apache.org/jira/browse/IGNITE-21038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Polovtcev updated IGNITE-21038: - Fix Version/s: 3.0.0-beta2 > Ignite logs in tests should reside in the work directory > > > Key: IGNITE-21038 > URL: https://issues.apache.org/jira/browse/IGNITE-21038 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Polovtcev >Assignee: Aleksandr Polovtcev >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21029) Update Ignite dependency log4j2
[ https://issues.apache.org/jira/browse/IGNITE-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Petrov updated IGNITE-21029: Ignite Flags: Release Notes Required (was: Docs Required,Release Notes Required) > Update Ignite dependency log4j2 > --- > > Key: IGNITE-21029 > URL: https://issues.apache.org/jira/browse/IGNITE-21029 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Nikolaev >Assignee: Aleksandr Nikolaev >Priority: Major > Labels: ise > Fix For: 2.17 > > Time Spent: 20m > Remaining Estimate: 0h > > Update Ignite dependency log4j2 2.20.0 to 2.22.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21029) Update Ignite dependency log4j2
[ https://issues.apache.org/jira/browse/IGNITE-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Petrov updated IGNITE-21029: Docs Text: Updated log4j2 dependency to 2.22.0 version (was: Update Ignite dependency log4j2) > Update Ignite dependency log4j2 > --- > > Key: IGNITE-21029 > URL: https://issues.apache.org/jira/browse/IGNITE-21029 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Nikolaev >Assignee: Aleksandr Nikolaev >Priority: Major > Labels: ise > Fix For: 2.17 > > Time Spent: 20m > Remaining Estimate: 0h > > Update Ignite dependency log4j2 2.20.0 to 2.22.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21029) Update Ignite dependency log4j2
[ https://issues.apache.org/jira/browse/IGNITE-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Nikolaev updated IGNITE-21029: Description: Update Ignite dependency log4j2 2.20.0 to 2.22.0 (was: Update Ignite dependency log4j 2.20.0 to 2.22.0) > Update Ignite dependency log4j2 > --- > > Key: IGNITE-21029 > URL: https://issues.apache.org/jira/browse/IGNITE-21029 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Nikolaev >Assignee: Aleksandr Nikolaev >Priority: Major > Labels: ise > Fix For: 2.17 > > Time Spent: 10m > Remaining Estimate: 0h > > Update Ignite dependency log4j2 2.20.0 to 2.22.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21037) Sql. Use single plan physical plan queries with dynamic parameters.
[ https://issues.apache.org/jira/browse/IGNITE-21037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-21037: -- Labels: ignite-3 (was: ) > Sql. Use single plan physical plan queries with dynamic parameters. > --- > > Key: IGNITE-21037 > URL: https://issues.apache.org/jira/browse/IGNITE-21037 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > At the moment we SQL engine creates can different physical plans for a > particular SQL statement, since each physical plan depends on types of > dynamic parameters. > Let's modify prepare service to produce a single plan. In order to do that we > need to add runtime conversion dynamic parameters. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-21029) Update Ignite dependency log4j2
[
https://issues.apache.org/jira/browse/IGNITE-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794223#comment-17794223
]
Ignite TC Bot commented on IGNITE-21029:
{panel:title=Branch: [pull/11080/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/11080/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *--> Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7645376&buildTypeId=IgniteTests24Java8_RunAll]
> Update Ignite dependency log4j2
> ---
>
> Key: IGNITE-21029
> URL: https://issues.apache.org/jira/browse/IGNITE-21029
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksandr Nikolaev
>Assignee: Aleksandr Nikolaev
>Priority: Major
> Labels: ise
> Fix For: 2.17
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Update Ignite dependency log4j 2.20.0 to 2.22.0
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-21029) Update Ignite dependency log4j2
[ https://issues.apache.org/jira/browse/IGNITE-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Nikolaev updated IGNITE-21029: Docs Text: Update Ignite dependency log4j > Update Ignite dependency log4j2 > --- > > Key: IGNITE-21029 > URL: https://issues.apache.org/jira/browse/IGNITE-21029 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Nikolaev >Assignee: Aleksandr Nikolaev >Priority: Major > Labels: ise > Fix For: 2.17 > > Time Spent: 10m > Remaining Estimate: 0h > > Update Ignite dependency log4j 2.20.0 to 2.22.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21029) Update Ignite dependency log4j2
[ https://issues.apache.org/jira/browse/IGNITE-21029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Nikolaev updated IGNITE-21029: Docs Text: Update Ignite dependency log4j2 (was: Update Ignite dependency log4j) > Update Ignite dependency log4j2 > --- > > Key: IGNITE-21029 > URL: https://issues.apache.org/jira/browse/IGNITE-21029 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Nikolaev >Assignee: Aleksandr Nikolaev >Priority: Major > Labels: ise > Fix For: 2.17 > > Time Spent: 10m > Remaining Estimate: 0h > > Update Ignite dependency log4j2 2.20.0 to 2.22.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21037) Sql. Use single plan physical plan queries with dynamic parameters.
[ https://issues.apache.org/jira/browse/IGNITE-21037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-21037: -- Fix Version/s: 3.0.0-beta2 > Sql. Use single plan physical plan queries with dynamic parameters. > --- > > Key: IGNITE-21037 > URL: https://issues.apache.org/jira/browse/IGNITE-21037 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Major > Fix For: 3.0.0-beta2 > > > At the moment we SQL engine creates can different physical plans for a > particular SQL statement, since each physical plan depends on types of > dynamic parameters. > Let's modify prepare service to produce a single plan. In order to do that we > need to add runtime conversion dynamic parameters. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21037) Sql. Use single plan physical plan queries with dynamic parameters.
Maksim Zhuravkov created IGNITE-21037: - Summary: Sql. Use single plan physical plan queries with dynamic parameters. Key: IGNITE-21037 URL: https://issues.apache.org/jira/browse/IGNITE-21037 Project: Ignite Issue Type: Improvement Components: sql Reporter: Maksim Zhuravkov At the moment we SQL engine creates can different physical plans for a particular SQL statement, since each physical plan depends on types of dynamic parameters. Let's modify prepare service to produce a single plan. In order to do that we need to add runtime conversion dynamic parameters. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21036) Sql. Add possibility to obtain parameters of already existing zone.
Evgeny Stanilovsky created IGNITE-21036: --- Summary: Sql. Add possibility to obtain parameters of already existing zone. Key: IGNITE-21036 URL: https://issues.apache.org/jira/browse/IGNITE-21036 Project: Ignite Issue Type: Improvement Components: sql Affects Versions: 3.0.0-beta1 Reporter: Evgeny Stanilovsky For now it`s possible to create distributed zone with different configurable parameters, like: "CREATE ZONE name_of_the_zone WITH partitions=7, DATA_NODES_AUTO_ADJUST=100" but there is no way to obtain this zone params using public api (views, jmx?). Seems we need such a functionality. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21035) Sql. Fix type inference for dynamic parameters in operators.
[
https://issues.apache.org/jira/browse/IGNITE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-21035:
--
Description:
Dynamic parameters used in operators require additional type casts to work
properly, since calcite's type checkers use type families that consist of
multiple types.
Several examples:
{code:java}
// val is VARCHAR
INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
// Error:
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
' || '
' || '
{code}
Function call (INTEGER is a type family):
{code:java}
"SELECT substring('asd', ?) FROM t1"
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
line 1, column 8 to line 1, column 26: Ambiguous operator SUBSTRING(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
{code}
Let's fix this behaviour by selecting the most common type in a type family:
* For the first case it means that the validator should select use a type of a
specified argument, when both arguments are not set it should return an error.
* In the second example the validator should for the second one it should
select an INTEGER type, the first argument should still require a VARCHAR.
was:
Dynamic parameters used in operators require additional type casts to work
properly, since calcite's type checkers use type families that consist of
multiple types.
Several examples:
{code:java}
// val is VARCHAR
INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
// Error:
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
' || '
' || '
{code}
Function call:
{code:java}
"SELECT substring('asd', ?) FROM t1"
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
line 1, column 8 to line 1, column 26: Ambiguous operator SUBSTRING(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
{code}
Let's fix this behaviour by selecting the most common type in a type family.
For the first case it means that the validator should select use a type of a
specified argument, when both arguments are not set it should return an error.
Here is INTEGER is a type family - TINYINT, SMALLINT, INT, BIGINT.
For the first argument we can select VARCHAR, for the second an INTEGER.
> Sql. Fix type inference for dynamic parameters in operators.
>
>
> Key: IGNITE-21035
> URL: https://issues.apache.org/jira/browse/IGNITE-21035
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Dynamic parameters used in operators require additional type casts to work
> properly, since calcite's type checkers use type families that consist of
> multiple types.
> Several examples:
> {code:java}
> // val is VARCHAR
> INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
> // Error:
> org.apache.ignite.sql.SqlException: IGN-SQL-6
> TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
> line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
> ). Dynamic parameter requires adding explicit type cast. Supported
> form(s):
> ' || '
> ' || '
> {code}
> Function call (INTEGER is a type family):
> {code:java}
> "SELECT substring('asd', ?) FROM t1"
> org.apache.ignite.sql.SqlException: IGN-SQL-6
> TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
> line 1, column 8 to line 1, column 26: Ambiguous operator
> SUBSTRING(, ). Dynamic parameter requires adding explicit
> type cast. Supported form(s):
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> {code}
> Let's fix this behaviour by selecting the most common type in a type family:
> * For the first case it means that the validator should select use a typ
[jira] [Updated] (IGNITE-21035) Sql. Fix type inference for dynamic parameters in operators.
[
https://issues.apache.org/jira/browse/IGNITE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-21035:
--
Description:
Dynamic parameters used in operators require additional type casts to work
properly, since calcite's type checkers use type families that consist of
multiple types.
Several examples:
{code:java}
// val is VARCHAR
INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
// Error:
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
' || '
' || '
{code}
Function call:
{code:java}
"SELECT substring('asd', ?) FROM t1"
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
line 1, column 8 to line 1, column 26: Ambiguous operator SUBSTRING(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
{code}
Let's fix this behaviour by selecting the most common type in a type family.
For the first case it means that the validator should select use a type of a
specified argument, when both arguments are not set it should return an error.
Here is INTEGER is a type family - TINYINT, SMALLINT, INT, BIGINT.
For the first argument we can select VARCHAR, for the second an INTEGER.
was:
Dynamic parameters used in operators require additional type cast to work
properly, since calcite's type checkers use type families which consist of
multiple types.
Several examples:
{code:java}
// val is VARCHAR
INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
// Error:
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
' || '
' || '
{code}
Function call:
{code:java}
"SELECT substring('asd', ?) FROM t1"
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
line 1, column 8 to line 1, column 26: Ambiguous operator SUBSTRING(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
{code}
Let's fix this behaviour by selecting the most common type in a type family.
For the first case it means that the validator should select use a type of a
specified argument, when both arguments are not set it should return an error.
Here is INTEGER is a type family - TINYINT, SMALLINT, INT, BIGINT.
For the first argument we can select VARCHAR, for the second an INTEGER.
> Sql. Fix type inference for dynamic parameters in operators.
>
>
> Key: IGNITE-21035
> URL: https://issues.apache.org/jira/browse/IGNITE-21035
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Dynamic parameters used in operators require additional type casts to work
> properly, since calcite's type checkers use type families that consist of
> multiple types.
> Several examples:
> {code:java}
> // val is VARCHAR
> INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
> // Error:
> org.apache.ignite.sql.SqlException: IGN-SQL-6
> TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
> line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
> ). Dynamic parameter requires adding explicit type cast. Supported
> form(s):
> ' || '
> ' || '
> {code}
> Function call:
> {code:java}
> "SELECT substring('asd', ?) FROM t1"
> org.apache.ignite.sql.SqlException: IGN-SQL-6
> TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
> line 1, column 8 to line 1, column 26: Ambiguous operator
> SUBSTRING(, ). Dynamic parameter requires adding explicit
> type cast. Supported form(s):
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> {code}
> Let's fix this behaviour by selecting the most common type in a type family.
> For the first case it means that the validator should select use a type of a
> specified argument, when both arguments are not set i
[jira] [Updated] (IGNITE-21035) Sql. Fix type inference for dynamic parameters in operators.
[
https://issues.apache.org/jira/browse/IGNITE-21035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-21035:
--
Labels: ignite-3 (was: )
> Sql. Fix type inference for dynamic parameters in operators.
>
>
> Key: IGNITE-21035
> URL: https://issues.apache.org/jira/browse/IGNITE-21035
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Dynamic parameters used in operators require additional type cast to work
> properly, since calcite's type checkers use type families which consist of
> multiple types.
> Several examples:
> {code:java}
> // val is VARCHAR
> INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
> // Error:
> org.apache.ignite.sql.SqlException: IGN-SQL-6
> TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
> line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
> ). Dynamic parameter requires adding explicit type cast. Supported
> form(s):
> ' || '
> ' || '
> {code}
> Function call:
> {code:java}
> "SELECT substring('asd', ?) FROM t1"
> org.apache.ignite.sql.SqlException: IGN-SQL-6
> TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
> line 1, column 8 to line 1, column 26: Ambiguous operator
> SUBSTRING(, ). Dynamic parameter requires adding explicit
> type cast. Supported form(s):
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> 'SUBSTRING( FROM )'
> 'SUBSTRING( FROM FOR )'
> {code}
> Let's fix this behaviour by selecting the most common type in a type family.
> For the first case it means that the validator should select use a type of a
> specified argument, when both arguments are not set it should return an error.
> Here is INTEGER is a type family - TINYINT, SMALLINT, INT, BIGINT.
> For the first argument we can select VARCHAR, for the second an INTEGER.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21035) Sql. Fix type inference for dynamic parameters in operators.
Maksim Zhuravkov created IGNITE-21035:
-
Summary: Sql. Fix type inference for dynamic parameters in
operators.
Key: IGNITE-21035
URL: https://issues.apache.org/jira/browse/IGNITE-21035
Project: Ignite
Issue Type: Bug
Components: sql
Reporter: Maksim Zhuravkov
Fix For: 3.0.0-beta2
Dynamic parameters used in operators require additional type cast to work
properly, since calcite's type checkers use type families which consist of
multiple types.
Several examples:
{code:java}
// val is VARCHAR
INSERT INTO t1 (val, id) SELECT ? || 'asd', ?
// Error:
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:557e7c5d-6a7a-4e3f-ba1f-e095c662f675 Failed to validate query. From
line 1, column 33 to line 1, column 42: Ambiguous operator ||(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
' || '
' || '
{code}
Function call:
{code:java}
"SELECT substring('asd', ?) FROM t1"
org.apache.ignite.sql.SqlException: IGN-SQL-6
TraceId:a2a35703-34e6-447f-aebd-bbd5f070eaa2 Failed to validate query. From
line 1, column 8 to line 1, column 26: Ambiguous operator SUBSTRING(,
). Dynamic parameter requires adding explicit type cast. Supported
form(s):
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
'SUBSTRING( FROM )'
'SUBSTRING( FROM FOR )'
{code}
Let's fix this behaviour by selecting the most common type in a type family.
For the first case it means that the validator should select use a type of a
specified argument, when both arguments are not set it should return an error.
Here is INTEGER is a type family - TINYINT, SMALLINT, INT, BIGINT.
For the first argument we can select VARCHAR, for the second an INTEGER.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-20971) The Ignite process huge memory overhead for tables creation
[
https://issues.apache.org/jira/browse/IGNITE-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794206#comment-17794206
]
Roman Puchkovskiy commented on IGNITE-20971:
The patch looks good to me
> The Ignite process huge memory overhead for tables creation
> ---
>
> Key: IGNITE-20971
> URL: https://issues.apache.org/jira/browse/IGNITE-20971
> Project: Ignite
> Issue Type: Bug
> Components: general
>Affects Versions: 3.0.0-beta2
>Reporter: Igor
>Assignee: Ivan Bessonov
>Priority: Major
> Labels: ignite-3
> Attachments: gc.20231128_132812_386649.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *Steps to reproduce*
> # Start process with `-Xms4096m -Xmx4096m`
> # Create tables with 5 columns one by one up to 1000.
> *Expected:*
> 1000 tables are created.
>
> *Actual:*
> After ~219 the process was killed by OOM killer due to process took 64GB of
> memory of available 65GB.
>
> *Additional information:*
> OOM killer output:
> {code:java}
> [Tue Nov 28 18:35:37 2023]
> oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-10002.slice/session-201.scope,task=java,pid=11748,uid=10002
> [Tue Nov 28 18:35:37 2023] Out of memory: Killed process 11748 (java)
> total-vm:97160836kB, anon-rss:64192728kB, file-rss:0kB, shmem-rss:0kB,
> UID:10002 pgtables:127948kB oom_score_adj:0
> [Tue Nov 28 18:35:39 2023] oom_reaper: reaped process 11748 (java), now
> anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
> pubagent:~$ grep MemTotal /proc/meminfo
> MemTotal: 65038556 kB
> pubagent:~$ free
> totalusedfree shared buff/cache available
> Mem:65038556 32184864557116 900 159592
> 64174540
> Swap: 0 0 0
> {code}
> GC log
> [^gc.20231128_132812_386649.log]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-21034) Java thin 3.0: testClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken due to notification mechanism
[ https://issues.apache.org/jira/browse/IGNITE-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-21034: Labels: ignite-3 (was: .NET ignite-3) > Java thin 3.0: testClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken > due to notification mechanism > -- > > Key: IGNITE-21034 > URL: https://issues.apache.org/jira/browse/IGNITE-21034 > Project: Ignite > Issue Type: Bug > Components: platforms, thin client >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > *ComputeClusterAwarenessTests.TestClientRetriesComputeJobOnPrimaryAndDefaultNodes* > is broken because of the new notification mechanism introduced by > IGNITE-20909 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21034) Java thin 3.0: testClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken due to notification mechanism
[ https://issues.apache.org/jira/browse/IGNITE-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-21034: Description: *ClientComputeTest.TestClientRetriesComputeJobOnPrimaryAndDefaultNodes* is broken because of the new notification mechanism introduced by IGNITE-20909 (was: *ComputeClusterAwarenessTests.TestClientRetriesComputeJobOnPrimaryAndDefaultNodes* is broken because of the new notification mechanism introduced by IGNITE-20909) > Java thin 3.0: testClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken > due to notification mechanism > -- > > Key: IGNITE-21034 > URL: https://issues.apache.org/jira/browse/IGNITE-21034 > Project: Ignite > Issue Type: Bug > Components: platforms, thin client >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > *ClientComputeTest.TestClientRetriesComputeJobOnPrimaryAndDefaultNodes* is > broken because of the new notification mechanism introduced by IGNITE-20909 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21034) Java thin 3.0: testClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken due to notification mechanism
Pavel Tupitsyn created IGNITE-21034: --- Summary: Java thin 3.0: testClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken due to notification mechanism Key: IGNITE-21034 URL: https://issues.apache.org/jira/browse/IGNITE-21034 Project: Ignite Issue Type: Bug Components: platforms, thin client Affects Versions: 3.0.0-beta1 Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 3.0.0-beta2 *ComputeClusterAwarenessTests.TestClientRetriesComputeJobOnPrimaryAndDefaultNodes* is broken because of the new notification mechanism introduced by IGNITE-20909 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19436) Add "--add-opens=java.base/java.time=ALL-UNNAMED" to jvm args
[
https://issues.apache.org/jira/browse/IGNITE-19436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikita Amelchev updated IGNITE-19436:
-
Fix Version/s: 2.16
> Add "--add-opens=java.base/java.time=ALL-UNNAMED" to jvm args
> -
>
> Key: IGNITE-19436
> URL: https://issues.apache.org/jira/browse/IGNITE-19436
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.15
> Environment: jdk17 env
>Reporter: YuJue Li
>Assignee: YuJue Li
>Priority: Minor
> Fix For: 2.16
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> 1.start a node with calcite-based sql engine is enabled;
> 2.execute the following SQL:
> select /*{*}+ QUERY_ENGINE('calcite') *{*}/ count(*) from T;
> then:
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make
> field private final int java.time.LocalDate.year accessible: module java.base
> does not "opens java.time" to unnamed module @587d1d39
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-20771) Implement tx coordinator liveness check
[ https://issues.apache.org/jira/browse/IGNITE-20771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794162#comment-17794162 ] Vladislav Pyatkov commented on IGNITE-20771: [~Denis Chudov] Thank you for the review and approval. Merged e7010e4cb749aad69399744a703f3529de23e232 > Implement tx coordinator liveness check > --- > > Key: IGNITE-20771 > URL: https://issues.apache.org/jira/browse/IGNITE-20771 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > Time Spent: 1h 20m > Remaining Estimate: 0h > > h3. Motivation > In order to implement tx coordinator recovery, it's definitely required to > understand whether coordinator is dead or not. Every data node has it's own > txn state local volatile map (txId -> > org.apache.ignite.internal.tx.TxStateMeta) where besides other fields we can > find txCoordinatorId. Liveness check assumes that if a node with given id is > available in physical topology then coordinator is alive, otherwise it's > considered as dead. However despite the fact that such local check is fast > enough there's no sense in checking it too often, espesially with subsequent > sends of initialRecoveryRequests. Thus, it seems reasonable to add one more > field to the TxStateMeta that will store last liveness check timestamp. > Because it's always local checks it's valid to use System.currentTimeMillis > or similar instead of HybridTimestamp in order to reduce the contention on > the clock. Please pay attention that triggers that will initiate liveness > checks will be implemented separetly. > h3. Definition of Done > * One more lastLivenessCheck timestamp is added to the TxStateMeta. > * Aforementioned field is updated locally on each tx operation with > currentTimeMillis. > * New cluster-wide tx liveness interval configuration property is introduced. > * Within liveness check > ** if (the lastLivenessCheck >= currentTimeMillis - livenessInterval) - no-op > ** else > *** update lastLivenessCheck > *** do the probe - check whether txCoordinatorId is still available in > physical topology, if it's available no further actions are required if int's > not then > trigger initiateRecovery procedure implemented in IGNITE-20685. > if commit partition is also unavailable (meaning that there's no > primary replica) mark transaction as abandoned. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19955) Fix create zone on restart rewrites existing data nodes because of trigger key inconsistnecy
[
https://issues.apache.org/jira/browse/IGNITE-19955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-19955:
-
Description:
Outdated, see UPD
Currently we have the logic for initialisation of newly created zone that it
writes keys
{noformat}
zoneDataNodesKey(zoneId), zoneScaleUpChangeTriggerKey(zoneId),
zoneScaleDownChangeTriggerKey(zoneId), zonesChangeTriggerKey(zoneId)
{noformat}
to metastorage, and condition is
{noformat}
static CompoundCondition triggerKeyConditionForZonesChanges(long revision,
int zoneId) {
return or(
notExists(zonesChangeTriggerKey(zoneId)),
value(zonesChangeTriggerKey(zoneId)).lt(ByteUtils.longToBytes(revision))
);
{noformat}
Recovery process implies that the create zone event will be processed again,
but with the higher revision, so data nodes will be rewritten.
We need to handle this situation, so data nodes will be consistent after
restart.
Possible solution is to change condition to
{noformat}
static SimpleCondition triggerKeyConditionForZonesCreation(long revision,
int zoneId) {
return notExists(zonesChangeTriggerKey(zoneId));
}
static SimpleCondition triggerKeyConditionForZonesDelete(int zoneId) {
return exists(zonesChangeTriggerKey(zoneId));
}
{noformat}
so we could not rely on revision and check only existence of the key, when we
create or remove zone. The problem in this solution is that reordering of the
create and remove on some node could lead to not consistent state for zones key
in metastorage
*UPD*:
This problem will be resolves once we implement
https://issues.apache.org/jira/browse/IGNITE-20561
In this ticket we need to unmute all tickets that were muted by this ticket
was:
Outdated, see UPD
Currently we have the logic for initialisation of newly created zone that it
writes keys
{noformat}
zoneDataNodesKey(zoneId), zoneScaleUpChangeTriggerKey(zoneId),
zoneScaleDownChangeTriggerKey(zoneId), zonesChangeTriggerKey(zoneId)
{noformat}
to metastorage, and condition is
{noformat}
static CompoundCondition triggerKeyConditionForZonesChanges(long revision,
int zoneId) {
return or(
notExists(zonesChangeTriggerKey(zoneId)),
value(zonesChangeTriggerKey(zoneId)).lt(ByteUtils.longToBytes(revision))
);
{noformat}
Recovery process implies that the create zone event will be processed again,
but with the higher revision, so data nodes will be rewritten.
We need to handle this situation, so data nodes will be consistent after
restart.
Possible solution is to change condition to
{noformat}
static SimpleCondition triggerKeyConditionForZonesCreation(long revision,
int zoneId) {
return notExists(zonesChangeTriggerKey(zoneId));
}
static SimpleCondition triggerKeyConditionForZonesDelete(int zoneId) {
return exists(zonesChangeTriggerKey(zoneId));
}
{noformat}
so we could not rely on revision and check only existence of the key, when we
create or remove zone. The problem in this solution is that reordering of the
create and remove on some node could lead to not consistent state for zones key
in metastorage
*UPD*:
This problem will be resolves once we implement
https://issues.apache.org/jira/browse/IGNITE-20561
In this ticket we need to unmute all tickets that were muted by this ticket
> Fix create zone on restart rewrites existing data nodes because of trigger
> key inconsistnecy
>
>
> Key: IGNITE-19955
> URL: https://issues.apache.org/jira/browse/IGNITE-19955
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Assignee: Mirza Aliev
>Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Outdated, see UPD
> Currently we have the logic for initialisation of newly created zone that it
> writes keys
> {noformat}
> zoneDataNodesKey(zoneId), zoneScaleUpChangeTriggerKey(zoneId),
> zoneScaleDownChangeTriggerKey(zoneId), zonesChangeTriggerKey(zoneId)
> {noformat}
> to metastorage, and condition is
> {noformat}
> static CompoundCondition triggerKeyConditionForZonesChanges(long
> revision, int zoneId) {
> return or(
> notExists(zonesChangeTriggerKey(zoneId)),
>
> value(zonesChangeTriggerKey(zoneId)).lt(ByteUtils.longToBytes(revision))
> );
> {noformat}
> Recovery process implies that the create zone event will be processed again,
> but with the higher revision, so data nodes will be rewritten.
> We need to handle this situation, so data nodes will be consistent after
> restart.
> Possible solution is to change condition to
> {noformat}
> static SimpleCondition triggerKeyConditionForZonesCreat
[jira] [Updated] (IGNITE-19955) Fix create zone on restart rewrites existing data nodes because of trigger key inconsistnecy
[
https://issues.apache.org/jira/browse/IGNITE-19955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-19955:
-
Description:
Outdated, see UPD
Currently we have the logic for initialisation of newly created zone that it
writes keys
{noformat}
zoneDataNodesKey(zoneId), zoneScaleUpChangeTriggerKey(zoneId),
zoneScaleDownChangeTriggerKey(zoneId), zonesChangeTriggerKey(zoneId)
{noformat}
to metastorage, and condition is
{noformat}
static CompoundCondition triggerKeyConditionForZonesChanges(long revision,
int zoneId) {
return or(
notExists(zonesChangeTriggerKey(zoneId)),
value(zonesChangeTriggerKey(zoneId)).lt(ByteUtils.longToBytes(revision))
);
{noformat}
Recovery process implies that the create zone event will be processed again,
but with the higher revision, so data nodes will be rewritten.
We need to handle this situation, so data nodes will be consistent after
restart.
Possible solution is to change condition to
{noformat}
static SimpleCondition triggerKeyConditionForZonesCreation(long revision,
int zoneId) {
return notExists(zonesChangeTriggerKey(zoneId));
}
static SimpleCondition triggerKeyConditionForZonesDelete(int zoneId) {
return exists(zonesChangeTriggerKey(zoneId));
}
{noformat}
so we could not rely on revision and check only existence of the key, when we
create or remove zone. The problem in this solution is that reordering of the
create and remove on some node could lead to not consistent state for zones key
in metastorage
*UPD*:
This problem will be resolves once we implement
https://issues.apache.org/jira/browse/IGNITE-20561
In this ticket we need to unmute all tickets that were muted by this ticket
was:
Currently we have the logic for initialisation of newly created zone that it
writes keys
{noformat}
zoneDataNodesKey(zoneId), zoneScaleUpChangeTriggerKey(zoneId),
zoneScaleDownChangeTriggerKey(zoneId), zonesChangeTriggerKey(zoneId)
{noformat}
to metastorage, and condition is
{noformat}
static CompoundCondition triggerKeyConditionForZonesChanges(long revision,
int zoneId) {
return or(
notExists(zonesChangeTriggerKey(zoneId)),
value(zonesChangeTriggerKey(zoneId)).lt(ByteUtils.longToBytes(revision))
);
{noformat}
Recovery process implies that the create zone event will be processed again,
but with the higher revision, so data nodes will be rewritten.
We need to handle this situation, so data nodes will be consistent after
restart.
Possible solution is to change condition to
{noformat}
static SimpleCondition triggerKeyConditionForZonesCreation(long revision,
int zoneId) {
return notExists(zonesChangeTriggerKey(zoneId));
}
static SimpleCondition triggerKeyConditionForZonesDelete(int zoneId) {
return exists(zonesChangeTriggerKey(zoneId));
}
{noformat}
so we could not rely on revision and check only existence of the key, when we
create or remove zone. The problem in this solution is that reordering of the
create and remove on some node could lead to not consistent state for zones key
in metastorage
*UPD*:
This problem will be resolves once we implement
https://issues.apache.org/jira/browse/IGNITE-20561
In this ticket we need to unmute all tickets that were muted by this ticket
> Fix create zone on restart rewrites existing data nodes because of trigger
> key inconsistnecy
>
>
> Key: IGNITE-19955
> URL: https://issues.apache.org/jira/browse/IGNITE-19955
> Project: Ignite
> Issue Type: Bug
>Reporter: Mirza Aliev
>Assignee: Mirza Aliev
>Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Outdated, see UPD
> Currently we have the logic for initialisation of newly created zone that it
> writes keys
> {noformat}
> zoneDataNodesKey(zoneId), zoneScaleUpChangeTriggerKey(zoneId),
> zoneScaleDownChangeTriggerKey(zoneId), zonesChangeTriggerKey(zoneId)
> {noformat}
> to metastorage, and condition is
> {noformat}
> static CompoundCondition triggerKeyConditionForZonesChanges(long
> revision, int zoneId) {
> return or(
> notExists(zonesChangeTriggerKey(zoneId)),
>
> value(zonesChangeTriggerKey(zoneId)).lt(ByteUtils.longToBytes(revision))
> );
> {noformat}
> Recovery process implies that the create zone event will be processed again,
> but with the higher revision, so data nodes will be rewritten.
> We need to handle this situation, so data nodes will be consistent after
> restart.
> Possible solution is to change condition to
> {noformat}
> static SimpleCondition triggerKeyConditionForZonesCreation(long revision,
[jira] [Commented] (IGNITE-21032) ReadOnlyDynamicMBean.getAttributes may return a list of attribute values instead of Attribute instances
[ https://issues.apache.org/jira/browse/IGNITE-21032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794132#comment-17794132 ] YuJue Li commented on IGNITE-21032: --- Is this issue the same as IGNITE-13342? > ReadOnlyDynamicMBean.getAttributes may return a list of attribute values > instead of Attribute instances > --- > > Key: IGNITE-21032 > URL: https://issues.apache.org/jira/browse/IGNITE-21032 > Project: Ignite > Issue Type: Bug >Reporter: Vyacheslav Koptilin >Priority: Major > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > > When supplying JMX information, the AttributeList class should contain > Attributes, however the existing code returns attribute values. This can > cause ClassCastExceptions in code that attempts to read an AttributeList. > > [GitHub Issue #11045|https://github.com/apache/ignite/issues/11045] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-20976) Sql. Multistatement dynamic parameters adjusting works incorrect for DELETE operator.
[
https://issues.apache.org/jira/browse/IGNITE-20976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794130#comment-17794130
]
Pavel Pereslegin edited comment on IGNITE-20976 at 12/7/23 11:08 AM:
-
[~mzhuravkov], thanks for the contribution!
[Merged|https://github.com/apache/ignite-3/commit/171fa1220c7834b63e5190898c850f9325559106]
to the main branch.
was (Author: xtern):
[~mzhuravkov], thanks for the contribution!
Merged to the main branch.
> Sql. Multistatement dynamic parameters adjusting works incorrect for DELETE
> operator.
> -
>
> Key: IGNITE-20976
> URL: https://issues.apache.org/jira/browse/IGNITE-20976
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Pavel Pereslegin
>Assignee: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Attachments: overwrite_sql_call_create.png
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Normalized statement cannot be obtained for 'DELETE' statement with dynamic
> parameters.
> It seems that after adjusting the dynamic parameters (see
> {{ScriptParseResult.SqlDynamicParamsAdjuster}}), the new tree cannot be
> converted to a string.
> Reproducer:
> {code:java}
> @Test
> public void testScriptDelete() {
> String query1 = "SELECT 1;";
> String query2 = "DELETE FROM TEST WHERE ID=?";
> // Parse separately - ok.
> StatementParseResult res1 = IgniteSqlParser.parse(query1,
> StatementParseResult.MODE);
> StatementParseResult res2 = IgniteSqlParser.parse(query2,
> StatementParseResult.MODE);
> System.out.println(res1.statement().toString());
> System.out.println(res2.statement().toString());
> // Parse script throws "UnsupportedOperationException" for
> `toString()` from `DELETE` statement.
> ScriptParseResult scriptRes = IgniteSqlParser.parse(query1 + query2,
> ScriptParseResult.MODE);
> for (StatementParseResult res : scriptRes.results()) {
> System.out.println(res.statement().toString());
> }
> }
> {code}
> Output
> {noformat}
> SELECT 1
> DELETE FROM `TEST`
> WHERE `ID` = ?
> SELECT 1
> java.lang.UnsupportedOperationException: class
> org.apache.calcite.sql.SqlSyntax$7: SPECIAL
> at org.apache.calcite.util.Util.needToImplement(Util.java:1119)
> at org.apache.calcite.sql.SqlSyntax$7.unparse(SqlSyntax.java:129)
> at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:385)
> at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:466)
> at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:131)
> at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:156)
> at org.apache.calcite.sql.SqlNode.toString(SqlNode.java:131)
> at
> org.apache.ignite.internal.sql.engine.sql.IgniteSqlParserTest.testScriptDelete(IgniteSqlParserTest.java:59)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20360) Implement the set of zone supported storages
[
https://issues.apache.org/jira/browse/IGNITE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirill Gusakov updated IGNITE-20360:
Description:
*Motivation*
According to IGNITE-20357 we need to have an appropriate zone filter, which
filters the nodes based on their available storages.
*Example*
{code}
create zone z1 with storage_profiles="lru_rocks,*clock_aipersist";
{code}
As a result::
* Zone consists of nodes which supports the *BOTH* storage profiles lru_rocks
and clock_aipersist
* Default storage profile for this zone will be clock_aipersist, because it is
marked with *. It means, that (after the IGNITE-20361) if on table creation we
will skip the storage profile for table:
{code}
create table t1 using zone="z1";
{code}
the table t1 will use clock_aipersist storage_profile.
*Definition of done*
* Zone has the filters, which can be unambiguously used to check if table can
be "deployed" in this zone
*Notes*
* Avoid filter altering for now (but add the appropriate event types)
was:
*Motivation*
According to IGNITE-20357 we need to have an appropriate zone filter, which
filters the nodes based on their available storages.
*Example*
{{create zone z1 with storage_profiles="lru_rocks,*clock_aipersist";}}
As a result::
* Zone consists of nodes which supports the *BOTH* storage profiles lru_rocks
and clock_aipersist
* Default storage profile for this zone will be clock_aipersist, because it is
marked with *. It means, that (after the IGNITE-20361) if on table creation we
will skip the storage profile for table:
{{create table t1 using zone="z1";}}
the table t1 will use clock_aipersist storage_profile.
*Definition of done*
* Zone has the filters, which can be unambiguously used to check if table can
be "deployed" in this zone
*Notes*
* Avoid filter altering for now (but add the appropriate event types)
> Implement the set of zone supported storages
>
>
> Key: IGNITE-20360
> URL: https://issues.apache.org/jira/browse/IGNITE-20360
> Project: Ignite
> Issue Type: Task
>Reporter: Kirill Gusakov
>Assignee: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> *Motivation*
> According to IGNITE-20357 we need to have an appropriate zone filter, which
> filters the nodes based on their available storages.
> *Example*
> {code}
> create zone z1 with storage_profiles="lru_rocks,*clock_aipersist";
> {code}
> As a result::
> * Zone consists of nodes which supports the *BOTH* storage profiles
> lru_rocks and clock_aipersist
> * Default storage profile for this zone will be clock_aipersist, because it
> is marked with *. It means, that (after the IGNITE-20361) if on table
> creation we will skip the storage profile for table:
> {code}
> create table t1 using zone="z1";
> {code}
> the table t1 will use clock_aipersist storage_profile.
> *Definition of done*
> * Zone has the filters, which can be unambiguously used to check if table
> can be "deployed" in this zone
> *Notes*
> * Avoid filter altering for now (but add the appropriate event types)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20360) Implement the set of zone supported storages
[
https://issues.apache.org/jira/browse/IGNITE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirill Gusakov updated IGNITE-20360:
Description:
*Motivation*
According to IGNITE-20357 we need to have an appropriate zone filter, which
filters the nodes based on their available storages.
*Example*
{{create zone z1 with storage_profiles="lru_rocks,*clock_aipersist";}}
As a result::
* Zone consists of nodes which supports the *BOTH* storage profiles lru_rocks
and clock_aipersist
* Default storage profile for this zone will be clock_aipersist, because it is
marked with *. It means, that (after the IGNITE-20361) if on table creation we
will skip the storage profile for table:
{{create table t1 using zone="z1";}}
the table t1 will use clock_aipersist storage_profile.
*Definition of done*
* Zone has the filters, which can be unambiguously used to check if table can
be "deployed" in this zone
*Notes*
* Avoid filter altering for now (but add the appropriate event types)
was:
Motivation
According to IGNITE-20357 we need to have an appropriate zone filter, which
filters the nodes based on their available storages.
Example
create zone z1 with storage_profiles="lru_rocks,*clock_aipersist";
As a result::
Zone consists of nodes which supports the BOTH storage profiles lru_rocks and
clock_aipersist
Default storage profile for this zone will be clock_aipersist, because it is
marked with *. It means, that (after the IGNITE-20361) if on table creation we
will skip the storage profile for table:
create table t1 using zone="z1";
the table t1 will use clock_aipersist storage_profile.
Definition of done
Zone has the filters, which can be unambiguously used to check if table can be
"deployed" in this zone
Notes
Avoid filter altering for now (but add the appropriate event types)
> Implement the set of zone supported storages
>
>
> Key: IGNITE-20360
> URL: https://issues.apache.org/jira/browse/IGNITE-20360
> Project: Ignite
> Issue Type: Task
>Reporter: Kirill Gusakov
>Assignee: Mirza Aliev
>Priority: Major
> Labels: ignite-3
>
> *Motivation*
> According to IGNITE-20357 we need to have an appropriate zone filter, which
> filters the nodes based on their available storages.
> *Example*
>
> {{create zone z1 with storage_profiles="lru_rocks,*clock_aipersist";}}
> As a result::
> * Zone consists of nodes which supports the *BOTH* storage profiles
> lru_rocks and clock_aipersist
> * Default storage profile for this zone will be clock_aipersist, because it
> is marked with *. It means, that (after the IGNITE-20361) if on table
> creation we will skip the storage profile for table:
>
> {{create table t1 using zone="z1";}}
> the table t1 will use clock_aipersist storage_profile.
> *Definition of done*
> * Zone has the filters, which can be unambiguously used to check if table
> can be "deployed" in this zone
> *Notes*
> * Avoid filter altering for now (but add the appropriate event types)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20360) Implement the set of zone supported storages
[ https://issues.apache.org/jira/browse/IGNITE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-20360: Description: Motivation According to IGNITE-20357 we need to have an appropriate zone filter, which filters the nodes based on their available storages. Example create zone z1 with storage_profiles="lru_rocks,*clock_aipersist"; As a result:: Zone consists of nodes which supports the BOTH storage profiles lru_rocks and clock_aipersist Default storage profile for this zone will be clock_aipersist, because it is marked with *. It means, that (after the IGNITE-20361) if on table creation we will skip the storage profile for table: create table t1 using zone="z1"; the table t1 will use clock_aipersist storage_profile. Definition of done Zone has the filters, which can be unambiguously used to check if table can be "deployed" in this zone Notes Avoid filter altering for now (but add the appropriate event types) was: *Motivation* According to IGNITE-20357 we need to have an appropriate zone filter, which filters the nodes based on their available storages. *Definition of done* - Zone has the filters, which can be unambiguously used to check if table can be "deployed" in this zone *Notes* - Avoid filter altering for now (but add the appropriate event types) > Implement the set of zone supported storages > > > Key: IGNITE-20360 > URL: https://issues.apache.org/jira/browse/IGNITE-20360 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Assignee: Mirza Aliev >Priority: Major > Labels: ignite-3 > > Motivation > According to IGNITE-20357 we need to have an appropriate zone filter, which > filters the nodes based on their available storages. > Example > create zone z1 with storage_profiles="lru_rocks,*clock_aipersist"; > As a result:: > Zone consists of nodes which supports the BOTH storage profiles lru_rocks and > clock_aipersist > Default storage profile for this zone will be clock_aipersist, because it is > marked with *. It means, that (after the IGNITE-20361) if on table creation > we will skip the storage profile for table: > create table t1 using zone="z1"; > the table t1 will use clock_aipersist storage_profile. > Definition of done > Zone has the filters, which can be unambiguously used to check if table can > be "deployed" in this zone > Notes > Avoid filter altering for now (but add the appropriate event types) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21032) ReadOnlyDynamicMBean.getAttributes may return a list of attribute values instead of Attribute instances
[ https://issues.apache.org/jira/browse/IGNITE-21032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikita Amelchev updated IGNITE-21032: - Fix Version/s: 2.16 > ReadOnlyDynamicMBean.getAttributes may return a list of attribute values > instead of Attribute instances > --- > > Key: IGNITE-21032 > URL: https://issues.apache.org/jira/browse/IGNITE-21032 > Project: Ignite > Issue Type: Bug >Reporter: Vyacheslav Koptilin >Priority: Major > Fix For: 2.16 > > Time Spent: 20m > Remaining Estimate: 0h > > When supplying JMX information, the AttributeList class should contain > Attributes, however the existing code returns attribute values. This can > cause ClassCastExceptions in code that attempts to read an AttributeList. > > [GitHub Issue #11045|https://github.com/apache/ignite/issues/11045] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21033) .NET: Thin 3.0: TestClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken due to notification mechanism
[ https://issues.apache.org/jira/browse/IGNITE-21033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-21033: Description: ComputeClusterAwarenessTests.TestClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken because of the new notification mechanism introduced by IGNITE-20909 > .NET: Thin 3.0: TestClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken > due to notification mechanism > --- > > Key: IGNITE-21033 > URL: https://issues.apache.org/jira/browse/IGNITE-21033 > Project: Ignite > Issue Type: Bug > Components: platforms, thin client >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET, ignite-3 > Fix For: 3.0.0-beta2 > > > ComputeClusterAwarenessTests.TestClientRetriesComputeJobOnPrimaryAndDefaultNodes > is broken because of the new notification mechanism introduced by > IGNITE-20909 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21033) .NET: Thin 3.0: TestClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken due to notification mechanism
[ https://issues.apache.org/jira/browse/IGNITE-21033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-21033: Description: *ComputeClusterAwarenessTests.TestClientRetriesComputeJobOnPrimaryAndDefaultNodes* is broken because of the new notification mechanism introduced by IGNITE-20909 (was: ComputeClusterAwarenessTests.TestClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken because of the new notification mechanism introduced by IGNITE-20909) > .NET: Thin 3.0: TestClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken > due to notification mechanism > --- > > Key: IGNITE-21033 > URL: https://issues.apache.org/jira/browse/IGNITE-21033 > Project: Ignite > Issue Type: Bug > Components: platforms, thin client >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET, ignite-3 > Fix For: 3.0.0-beta2 > > > *ComputeClusterAwarenessTests.TestClientRetriesComputeJobOnPrimaryAndDefaultNodes* > is broken because of the new notification mechanism introduced by > IGNITE-20909 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-21033) .NET: Thin 3.0: TestClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken due to notification mechanism
Pavel Tupitsyn created IGNITE-21033: --- Summary: .NET: Thin 3.0: TestClientRetriesComputeJobOnPrimaryAndDefaultNodes is broken due to notification mechanism Key: IGNITE-21033 URL: https://issues.apache.org/jira/browse/IGNITE-21033 Project: Ignite Issue Type: Bug Components: platforms, thin client Affects Versions: 3.0.0-beta1 Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 3.0.0-beta2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21032) ReadOnlyDynamicMBean.getAttributes may return a list of attribute values instead of Attribute instances
[ https://issues.apache.org/jira/browse/IGNITE-21032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin reassigned IGNITE-21032: Assignee: (was: Vyacheslav Koptilin) > ReadOnlyDynamicMBean.getAttributes may return a list of attribute values > instead of Attribute instances > --- > > Key: IGNITE-21032 > URL: https://issues.apache.org/jira/browse/IGNITE-21032 > Project: Ignite > Issue Type: Bug >Reporter: Vyacheslav Koptilin >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > When supplying JMX information, the AttributeList class should contain > Attributes, however the existing code returns attribute values. This can > cause ClassCastExceptions in code that attempts to read an AttributeList. > > [GitHub Issue #11045|https://github.com/apache/ignite/issues/11045] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-21032) ReadOnlyDynamicMBean.getAttributes may return a list of attribute values instead of Attribute instances
[ https://issues.apache.org/jira/browse/IGNITE-21032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin reassigned IGNITE-21032: Assignee: Vyacheslav Koptilin > ReadOnlyDynamicMBean.getAttributes may return a list of attribute values > instead of Attribute instances > --- > > Key: IGNITE-21032 > URL: https://issues.apache.org/jira/browse/IGNITE-21032 > Project: Ignite > Issue Type: Bug >Reporter: Vyacheslav Koptilin >Assignee: Vyacheslav Koptilin >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > When supplying JMX information, the AttributeList class should contain > Attributes, however the existing code returns attribute values. This can > cause ClassCastExceptions in code that attempts to read an AttributeList. > > [GitHub Issue #11045|https://github.com/apache/ignite/issues/11045] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21031) Calcite engine. Query fails on performance statistics in case of nested scans
[
https://issues.apache.org/jira/browse/IGNITE-21031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Nikolaev updated IGNITE-21031:
Labels: ise (was: )
> Calcite engine. Query fails on performance statistics in case of nested scans
> -
>
> Key: IGNITE-21031
> URL: https://issues.apache.org/jira/browse/IGNITE-21031
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: ise
> Fix For: 2.16
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Nested scan can be performed by Calcite engine, for example, in case of UNION
> ALL, when the first table scan is completed (and {{{}downstream().end(){}}})
> method is invoked and UNION ALL operator proceed to the next table scan.
> Reproducer:
> {code:java}
> public void testPerformanceStatisticsNestedScan() throws Exception {
> sql(grid(0), "CREATE TABLE test_perf_stat_nested (a INT) WITH
> template=REPLICATED");
> sql(grid(0), "INSERT INTO test_perf_stat_nested VALUES (0), (1), (2),
> (3), (4)");
> startCollectStatistics();
> sql(grid(0), "SELECT * FROM test_perf_stat_nested UNION ALL SELECT * FROM
> test_perf_stat_nested");
> }{code}
> Fails on:
> {noformat}
> at
> org.apache.ignite.internal.metric.IoStatisticsQueryHelper.startGatheringQueryStatistics(IoStatisticsQueryHelper.java:35)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.tracker.PerformanceStatisticsIoTracker.startTracking(PerformanceStatisticsIoTracker.java:65)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanStorageNode.processNextBatch(ScanStorageNode.java:68)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.push(ScanNode.java:145)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.request(ScanNode.java:95)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.UnionAllNode.end(UnionAllNode.java:79)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.processNextBatch(ScanNode.java:185)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanStorageNode.processNextBatch(ScanStorageNode.java:70)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.push(ScanNode.java:145)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.request(ScanNode.java:95)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.UnionAllNode.request(UnionAllNode.java:56)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-21032) ReadOnlyDynamicMBean.getAttributes may return a list of attribute values instead of Attribute instances
Vyacheslav Koptilin created IGNITE-21032: Summary: ReadOnlyDynamicMBean.getAttributes may return a list of attribute values instead of Attribute instances Key: IGNITE-21032 URL: https://issues.apache.org/jira/browse/IGNITE-21032 Project: Ignite Issue Type: Bug Reporter: Vyacheslav Koptilin When supplying JMX information, the AttributeList class should contain Attributes, however the existing code returns attribute values. This can cause ClassCastExceptions in code that attempts to read an AttributeList. [GitHub Issue #11045|https://github.com/apache/ignite/issues/11045] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21032) ReadOnlyDynamicMBean.getAttributes may return a list of attribute values instead of Attribute instances
[ https://issues.apache.org/jira/browse/IGNITE-21032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-21032: - Ignite Flags: (was: Docs Required,Release Notes Required) > ReadOnlyDynamicMBean.getAttributes may return a list of attribute values > instead of Attribute instances > --- > > Key: IGNITE-21032 > URL: https://issues.apache.org/jira/browse/IGNITE-21032 > Project: Ignite > Issue Type: Bug >Reporter: Vyacheslav Koptilin >Priority: Major > > When supplying JMX information, the AttributeList class should contain > Attributes, however the existing code returns attribute values. This can > cause ClassCastExceptions in code that attempts to read an AttributeList. > > [GitHub Issue #11045|https://github.com/apache/ignite/issues/11045] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21030) Sql. Dump debug info if the ExecutionService could not be stopped within a timeout
[ https://issues.apache.org/jira/browse/IGNITE-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov updated IGNITE-21030: -- Fix Version/s: 3.0.0-beta2 > Sql. Dump debug info if the ExecutionService could not be stopped within a > timeout > -- > > Key: IGNITE-21030 > URL: https://issues.apache.org/jira/browse/IGNITE-21030 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Pavel Pereslegin >Assignee: Pavel Pereslegin >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > To investigate the issue reported in IGNITE-20750 (about hanging of > ExecutionService during node shutdown) we need to collect more debugging > information when the sql execution service fails to stop. > It is proposed to add a timeout for ExecutionServiceImpl stop procedure and > output some diagnostic information to the log in case when timeout occurs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-21031) Calcite engine. Query fails on performance statistics in case of nested scans
[
https://issues.apache.org/jira/browse/IGNITE-21031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikita Amelchev updated IGNITE-21031:
-
Fix Version/s: 2.16
> Calcite engine. Query fails on performance statistics in case of nested scans
> -
>
> Key: IGNITE-21031
> URL: https://issues.apache.org/jira/browse/IGNITE-21031
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Fix For: 2.16
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Nested scan can be performed by Calcite engine, for example, in case of UNION
> ALL, when the first table scan is completed (and {{{}downstream().end(){}}})
> method is invoked and UNION ALL operator proceed to the next table scan.
> Reproducer:
> {code:java}
> public void testPerformanceStatisticsNestedScan() throws Exception {
> sql(grid(0), "CREATE TABLE test_perf_stat_nested (a INT) WITH
> template=REPLICATED");
> sql(grid(0), "INSERT INTO test_perf_stat_nested VALUES (0), (1), (2),
> (3), (4)");
> startCollectStatistics();
> sql(grid(0), "SELECT * FROM test_perf_stat_nested UNION ALL SELECT * FROM
> test_perf_stat_nested");
> }{code}
> Fails on:
> {noformat}
> at
> org.apache.ignite.internal.metric.IoStatisticsQueryHelper.startGatheringQueryStatistics(IoStatisticsQueryHelper.java:35)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.tracker.PerformanceStatisticsIoTracker.startTracking(PerformanceStatisticsIoTracker.java:65)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanStorageNode.processNextBatch(ScanStorageNode.java:68)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.push(ScanNode.java:145)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.request(ScanNode.java:95)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.UnionAllNode.end(UnionAllNode.java:79)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.processNextBatch(ScanNode.java:185)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanStorageNode.processNextBatch(ScanStorageNode.java:70)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.push(ScanNode.java:145)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.request(ScanNode.java:95)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.UnionAllNode.request(UnionAllNode.java:56)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-21031) Calcite engine. Query fails on performance statistics in case of nested scans
[
https://issues.apache.org/jira/browse/IGNITE-21031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17794108#comment-17794108
]
Ignite TC Bot commented on IGNITE-21031:
{panel:title=Branch: [pull/11081/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/11081/head] Base: [master] : New Tests
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Calcite SQL{color} [[tests
1|https://ci2.ignite.apache.org/viewLog.html?buildId=7645409]]
* {color:#013220}IgniteCalciteTestSuite:
SqlDiagnosticIntegrationTest.testPerformanceStatisticsNestedScan - PASSED{color}
{panel}
[TeamCity *--> Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7645486&buildTypeId=IgniteTests24Java8_RunAll]
> Calcite engine. Query fails on performance statistics in case of nested scans
> -
>
> Key: IGNITE-21031
> URL: https://issues.apache.org/jira/browse/IGNITE-21031
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Nested scan can be performed by Calcite engine, for example, in case of UNION
> ALL, when the first table scan is completed (and {{{}downstream().end(){}}})
> method is invoked and UNION ALL operator proceed to the next table scan.
> Reproducer:
> {code:java}
> public void testPerformanceStatisticsNestedScan() throws Exception {
> sql(grid(0), "CREATE TABLE test_perf_stat_nested (a INT) WITH
> template=REPLICATED");
> sql(grid(0), "INSERT INTO test_perf_stat_nested VALUES (0), (1), (2),
> (3), (4)");
> startCollectStatistics();
> sql(grid(0), "SELECT * FROM test_perf_stat_nested UNION ALL SELECT * FROM
> test_perf_stat_nested");
> }{code}
> Fails on:
> {noformat}
> at
> org.apache.ignite.internal.metric.IoStatisticsQueryHelper.startGatheringQueryStatistics(IoStatisticsQueryHelper.java:35)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.tracker.PerformanceStatisticsIoTracker.startTracking(PerformanceStatisticsIoTracker.java:65)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanStorageNode.processNextBatch(ScanStorageNode.java:68)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.push(ScanNode.java:145)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.request(ScanNode.java:95)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.UnionAllNode.end(UnionAllNode.java:79)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.processNextBatch(ScanNode.java:185)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanStorageNode.processNextBatch(ScanStorageNode.java:70)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.push(ScanNode.java:145)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.ScanNode.request(ScanNode.java:95)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.rel.UnionAllNode.request(UnionAllNode.java:56)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20971) The Ignite process huge memory overhead for tables creation
[ https://issues.apache.org/jira/browse/IGNITE-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor updated IGNITE-20971: -- Attachment: gc.20231128_132812_386649.log > The Ignite process huge memory overhead for tables creation > --- > > Key: IGNITE-20971 > URL: https://issues.apache.org/jira/browse/IGNITE-20971 > Project: Ignite > Issue Type: Bug > Components: general >Affects Versions: 3.0.0-beta2 >Reporter: Igor >Assignee: Ivan Bessonov >Priority: Major > Labels: ignite-3 > Attachments: gc.20231128_132812_386649.log > > > Creating 1000 tables with 5 column each. > *Expected:* > 1000 tables are created. > > *Actual:* > After some tables (in my case after 75 tables) the Ignite runner process is > silently teared down, no any errors in output. GC log doesn't show any > problem. > > *Additional information:* > On more performant (in CPU) servers it can create up to 855 tables on 4GB > HEAP and then tearing down with > `java.lang.OutOfMemoryError: Java heap space` -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20971) The Ignite process huge memory overhead for tables creation
[
https://issues.apache.org/jira/browse/IGNITE-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor updated IGNITE-20971:
--
Description:
*Steps to reproduce*
# Start process with `-Xms4096m -Xmx4096m`
# Create tables with 5 columns one by one up to 1000.
*Expected:*
1000 tables are created.
*Actual:*
After ~219 the process was killed by OOM killer due to process took 64GB of
memory of available 65GB.
*Additional information:*
OOM killer output:
{code:java}
[Tue Nov 28 18:35:37 2023]
oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-10002.slice/session-201.scope,task=java,pid=11748,uid=10002
[Tue Nov 28 18:35:37 2023] Out of memory: Killed process 11748 (java)
total-vm:97160836kB, anon-rss:64192728kB, file-rss:0kB, shmem-rss:0kB,
UID:10002 pgtables:127948kB oom_score_adj:0
[Tue Nov 28 18:35:39 2023] oom_reaper: reaped process 11748 (java), now
anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
pubagent:~$ grep MemTotal /proc/meminfo
MemTotal: 65038556 kB
pubagent:~$ free
totalusedfree shared buff/cache available
Mem:65038556 32184864557116 900 15959264174540
Swap: 0 0 0
{code}
GC log
[^gc.20231128_132812_386649.log]
was:
Creating 1000 tables with 5 column each.
*Expected:*
1000 tables are created.
*Actual:*
After some tables (in my case after 75 tables) the Ignite runner process is
silently teared down, no any errors in output. GC log doesn't show any problem.
*Additional information:*
On more performant (in CPU) servers it can create up to 855 tables on 4GB HEAP
and then tearing down with
`java.lang.OutOfMemoryError: Java heap space`
> The Ignite process huge memory overhead for tables creation
> ---
>
> Key: IGNITE-20971
> URL: https://issues.apache.org/jira/browse/IGNITE-20971
> Project: Ignite
> Issue Type: Bug
> Components: general
>Affects Versions: 3.0.0-beta2
>Reporter: Igor
>Assignee: Ivan Bessonov
>Priority: Major
> Labels: ignite-3
> Attachments: gc.20231128_132812_386649.log
>
>
> *Steps to reproduce*
> # Start process with `-Xms4096m -Xmx4096m`
> # Create tables with 5 columns one by one up to 1000.
> *Expected:*
> 1000 tables are created.
>
> *Actual:*
> After ~219 the process was killed by OOM killer due to process took 64GB of
> memory of available 65GB.
>
> *Additional information:*
> OOM killer output:
> {code:java}
> [Tue Nov 28 18:35:37 2023]
> oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-10002.slice/session-201.scope,task=java,pid=11748,uid=10002
> [Tue Nov 28 18:35:37 2023] Out of memory: Killed process 11748 (java)
> total-vm:97160836kB, anon-rss:64192728kB, file-rss:0kB, shmem-rss:0kB,
> UID:10002 pgtables:127948kB oom_score_adj:0
> [Tue Nov 28 18:35:39 2023] oom_reaper: reaped process 11748 (java), now
> anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
> pubagent:~$ grep MemTotal /proc/meminfo
> MemTotal: 65038556 kB
> pubagent:~$ free
> totalusedfree shared buff/cache available
> Mem:65038556 32184864557116 900 159592
> 64174540
> Swap: 0 0 0
> {code}
> GC log
> [^gc.20231128_132812_386649.log]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20971) The Ignite process huge memory overhead for tables creation
[ https://issues.apache.org/jira/browse/IGNITE-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor updated IGNITE-20971: -- Summary: The Ignite process huge memory overhead for tables creation (was: The Ignite process quietly tear down while creating a lot of tables) > The Ignite process huge memory overhead for tables creation > --- > > Key: IGNITE-20971 > URL: https://issues.apache.org/jira/browse/IGNITE-20971 > Project: Ignite > Issue Type: Bug > Components: general >Affects Versions: 3.0.0-beta2 >Reporter: Igor >Assignee: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > Creating 1000 tables with 5 column each. > *Expected:* > 1000 tables are created. > > *Actual:* > After some tables (in my case after 75 tables) the Ignite runner process is > silently teared down, no any errors in output. GC log doesn't show any > problem. > > *Additional information:* > On more performant (in CPU) servers it can create up to 855 tables on 4GB > HEAP and then tearing down with > `java.lang.OutOfMemoryError: Java heap space` -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20909) Thin 3.0: Compute jobs should use server notification to signal completion to the client
[ https://issues.apache.org/jira/browse/IGNITE-20909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-20909: Description: Compute jobs can be long-lived and even out-live the client connection. New Compute API is coming that will return some "execution" object immediately, which can be used to monitor or cancel the job. Therefore, job startup and completion should be separated - normal request-response approach is not suitable. * Use request-response to initiate the job execution and return an ID to the client * Use server -> client notification to signal about completion This is a tried approach from Ignite 2.x, see linked [IEP-42|https://cwiki.apache.org/confluence/display/IGNITE/IEP-42+Thin+Client+Compute] and related discussion was: Compute jobs can be long-lived and even out-live the client connection. Therefore, normal request-response approach does not work well for them. * Use request-response to initiate the job execution and return an ID to the client * Use server -> client notification to signal about completion This is a tried approach from Ignite 2.x, see linked [IEP-42|https://cwiki.apache.org/confluence/display/IGNITE/IEP-42+Thin+Client+Compute] and related discussion > Thin 3.0: Compute jobs should use server notification to signal completion to > the client > > > Key: IGNITE-20909 > URL: https://issues.apache.org/jira/browse/IGNITE-20909 > Project: Ignite > Issue Type: Improvement > Components: compute, thin client >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: iep-42, ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > Compute jobs can be long-lived and even out-live the client connection. New > Compute API is coming that will return some "execution" object immediately, > which can be used to monitor or cancel the job. Therefore, job startup and > completion should be separated - normal request-response approach is not > suitable. > * Use request-response to initiate the job execution and return an ID to the > client > * Use server -> client notification to signal about completion > This is a tried approach from Ignite 2.x, see linked > [IEP-42|https://cwiki.apache.org/confluence/display/IGNITE/IEP-42+Thin+Client+Compute] > and related discussion -- This message was sent by Atlassian Jira (v8.20.10#820010)
