[jira] [Updated] (IGNITE-17322) [ducktests] Add ability to configure more options for ignite nodes

2022-07-06 Thread Sergey Korotkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Korotkov updated IGNITE-17322:
-
Summary: [ducktests] Add ability to configure more options for ignite nodes 
 (was: [ducktests] Add abilities to configure more options for server and 
client nodes)

> [ducktests] Add ability to configure more options for ignite nodes
> --
>
> Key: IGNITE-17322
> URL: https://issues.apache.org/jira/browse/IGNITE-17322
> Project: Ignite
>  Issue Type: Test
>Reporter: Sergey Korotkov
>Assignee: Sergey Korotkov
>Priority: Minor
>  Labels: ducktests
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-16760) Performance degradation of IgniteTables#tables after configuration changes

2022-07-06 Thread Evgeny Stanilovsky (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563376#comment-17563376
 ] 

Evgeny Stanilovsky edited comment on IGNITE-16760 at 7/7/22 5:03 AM:
-

[~Denis Chudov] tests with annotation 
@Disabled("https://issues.apache.org/jira/browse/IGNITE-16760";) still present 
in main, can you take a look ? Additionally i still observe degradation on 
annotated test, is it another known issue ?


was (Author: zstan):
[~Denis Chudov] tests with annotation 
@Disabled("https://issues.apache.org/jira/browse/IGNITE-16760";) still present 
in main, can you take a look ?

> Performance degradation of IgniteTables#tables after configuration changes
> --
>
> Key: IGNITE-16760
> URL: https://issues.apache.org/jira/browse/IGNITE-16760
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Taras Ledkov
>Assignee: Denis Chudov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Performance degradation of configuration changes:
> Steps to reproduce:
> 1. Start cluster with 3 nodes
> 2. Run in the loop
> {code}
> CREATE TABLE TEST(ID INTEGER PRIMARY KEY, V INTEGER)
> for (Table t : ign.tables().tables()) {
> ;
> }
> {code}
> On begin {{IgniteTables#tables}} takes ~ 0.7 sec.
> The time of the operation  is grown.
> The time after ~100 iteration is about 20 sec.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17322) [ducktests] Add abilities to configure more options for server and client nodes

2022-07-06 Thread Sergey Korotkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Korotkov updated IGNITE-17322:
-
Labels: ducktests  (was: )

> [ducktests] Add abilities to configure more options for server and client 
> nodes
> ---
>
> Key: IGNITE-17322
> URL: https://issues.apache.org/jira/browse/IGNITE-17322
> Project: Ignite
>  Issue Type: Test
>Reporter: Sergey Korotkov
>Priority: Minor
>  Labels: ducktests
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-17322) [ducktests] Add abilities to configure more options for server and client nodes

2022-07-06 Thread Sergey Korotkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Korotkov reassigned IGNITE-17322:


Assignee: Sergey Korotkov

> [ducktests] Add abilities to configure more options for server and client 
> nodes
> ---
>
> Key: IGNITE-17322
> URL: https://issues.apache.org/jira/browse/IGNITE-17322
> Project: Ignite
>  Issue Type: Test
>Reporter: Sergey Korotkov
>Assignee: Sergey Korotkov
>Priority: Minor
>  Labels: ducktests
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-17322) [ducktests] Add abilities to configure more options for server and client nodes

2022-07-06 Thread Sergey Korotkov (Jira)
Sergey Korotkov created IGNITE-17322:


 Summary: [ducktests] Add abilities to configure more options for 
server and client nodes
 Key: IGNITE-17322
 URL: https://issues.apache.org/jira/browse/IGNITE-17322
 Project: Ignite
  Issue Type: Test
Reporter: Sergey Korotkov






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-12852) Comma in field is not supported by COPY command

2022-07-06 Thread YuJue Li (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

YuJue Li reassigned IGNITE-12852:
-

Assignee: Anton Kurbanov  (was: YuJue Li)

> Comma in field is not supported by COPY command
> ---
>
> Key: IGNITE-12852
> URL: https://issues.apache.org/jira/browse/IGNITE-12852
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.8
>Reporter: YuJue Li
>Assignee: Anton Kurbanov
>Priority: Critical
> Fix For: 2.14
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> CREATE TABLE test(a int,b varchar(100),c int,PRIMARY key(a)); 
>  
> a.csv: 
> 1,"a,b",2 
>  
> COPY FROM '/data/a.csv' INTO test (a,b,c) FORMAT CSV; 
>  
> The copy command fails because there is a comma in the second field,but this 
> is a fully legal and compliant CSV format



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-12852) Comma in field is not supported by COPY command

2022-07-06 Thread YuJue Li (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563516#comment-17563516
 ] 

YuJue Li commented on IGNITE-12852:
---

ok.

> Comma in field is not supported by COPY command
> ---
>
> Key: IGNITE-12852
> URL: https://issues.apache.org/jira/browse/IGNITE-12852
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.8
>Reporter: YuJue Li
>Assignee: YuJue Li
>Priority: Critical
> Fix For: 2.14
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> CREATE TABLE test(a int,b varchar(100),c int,PRIMARY key(a)); 
>  
> a.csv: 
> 1,"a,b",2 
>  
> COPY FROM '/data/a.csv' INTO test (a,b,c) FORMAT CSV; 
>  
> The copy command fails because there is a comma in the second field,but this 
> is a fully legal and compliant CSV format



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-14986) Re-work error handling in meta storage component in accordance with error groups

2022-07-06 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-14986:
-
Labels: iep-84 ignite-3  (was: ignite-3)

> Re-work error handling in meta storage component in accordance with error 
> groups
> 
>
> Key: IGNITE-14986
> URL: https://issues.apache.org/jira/browse/IGNITE-14986
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
>  Labels: iep-84, ignite-3
>
> Need to introduce a new error group related to Meta Storage Service and add 
> all needed error codes.
> Also, the implementation should using _IgniteInternalException_ and 
> _IgniteInternalCheckedException_ with specific error codes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17281) Return error groups from REST API

2022-07-06 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-17281:
-
Labels: iep-84 iep-87 ignite-3  (was: iep-87 ignite-3)

> Return error groups from REST API
> -
>
> Key: IGNITE-17281
> URL: https://issues.apache.org/jira/browse/IGNITE-17281
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Aleksandr
>Assignee: Vyacheslav Koptilin
>Priority: Major
>  Labels: iep-84, iep-87, ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> After IGNITE-14931 and IGNITE-17193 are done, traceId and human-readable 
> error code should be returned from any rest component.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-14984) Re-work error handling in table component in accordance with error scopes and prefixes

2022-07-06 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-14984:
-
Labels: iep-84 ignite-3  (was: ignite-3)

> Re-work error handling in table component in accordance with error scopes and 
> prefixes
> --
>
> Key: IGNITE-14984
> URL: https://issues.apache.org/jira/browse/IGNITE-14984
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vyacheslav Koptilin
>Priority: Major
>  Labels: iep-84, ignite-3
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-14985) Re-work error handling in affinity component in accordance with error scopes and prefixes

2022-07-06 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-14985:
-
Labels: iep-84 ignite-3  (was: ignite-3)

> Re-work error handling in affinity component in accordance with error scopes 
> and prefixes 
> --
>
> Key: IGNITE-14985
> URL: https://issues.apache.org/jira/browse/IGNITE-14985
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vyacheslav Koptilin
>Priority: Major
>  Labels: iep-84, ignite-3
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-14930) Exceptions serialization

2022-07-06 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-14930:
-
Labels: iep-84 ignite-3  (was: ignite-3)

> Exceptions serialization
> 
>
> Key: IGNITE-14930
> URL: https://issues.apache.org/jira/browse/IGNITE-14930
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vyacheslav Koptilin
>Priority: Major
>  Labels: iep-84, ignite-3
>
> Since there are other languages than Java, we should have a generic way to 
> convert exceptions between different representations. Current errors 
> serialization design is described here: [IEP-76 Thin Client Protocol for 
> Ignite 
> 3.0|https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0]
> We should expand the collection of required fields. It should be enough to 
> deserialize exception objects in Java client as close to the original object 
> as possible. This includes cause exceptions.
> Not yet sure about suppressed exceptions. So, we would need a list of errors, 
> where the first element is the head and every next element is the cause for 
> the previous one. Element contains:
> Error code (int)
> Error message (String)
> [Optional] Stack trace, for head only (String)
> Full class name (String)
> Open problem:
> What should we do with standard Java exceptions, like _TimeoutException_ or 
> _IllegalArgumentException_ , or even _NullPointerException_? Right now it's 
> better to have a reserved error group for them and assign a specific codes to 
> all "known" types.
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-84%3A+Error+handling



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-13510) Getting status of snapshot execution via command line and jmx

2022-07-06 Thread Sergei Ryzhov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563432#comment-17563432
 ] 

Sergei Ryzhov commented on IGNITE-13510:


[~xtern] Hi, it's good idea! Thanks!
Yes, i want to finish this patch

> Getting status of snapshot execution via command line and jmx
> -
>
> Key: IGNITE-13510
> URL: https://issues.apache.org/jira/browse/IGNITE-13510
> Project: Ignite
>  Issue Type: Task
>Reporter: Sergei Ryzhov
>Assignee: Sergei Ryzhov
>Priority: Major
>  Labels: iep-43, ise, snapshot
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> the control.sh utility immediately relinquishes control
> and without using metricExporter it is impossible to understand whether the 
> snapshot completed or not
> Restoring
> {code:java}
> Control utility [ver. 2.12.0-SNAPSHOT#20211004-sha1:77de60a7]
> 2021 Copyright(C) Apache Software Foundation
> User: sega
> Time: 2021-10-07T14:18:59.523
> Command [SNAPSHOT] started
> Arguments: --snapshot status --yes 
> 
> Status of SNAPSHOT operations:
> gridCommandHandlerTest0 -> Restoring to snapshot with name: snapshot_02052020
> gridCommandHandlerTest1 -> Restoring to snapshot with name: snapshot_02052020
> Command [SNAPSHOT] finished with code: 0
> Control utility has completed execution at: 2021-10-07T14:18:59.546
> Execution time: 23 ms
> {code}
> Creating
> {code:java}
> Control utility [ver. 2.12.0-SNAPSHOT#20211004-sha1:77de60a7]
> 2021 Copyright(C) Apache Software Foundation
> User: sega
> Time: 2021-10-07T14:18:55.368
> Command [SNAPSHOT] started
> Arguments: --snapshot status --yes 
> 
> Status of SNAPSHOT operations:
> gridCommandHandlerTest0 -> Creating the snapshot with name: snapshot_02052020
> gridCommandHandlerTest1 -> Creating the snapshot with name: snapshot_02052020
> Command [SNAPSHOT] finished with code: 0
> Control utility has completed execution at: 2021-10-07T14:18:55.391
> Execution time: 23 ms
> {code}
> No snapshot operation
> {code:java}
> Control utility [ver. 2.12.0-SNAPSHOT#20211004-sha1:77de60a7]
> 2021 Copyright(C) Apache Software Foundation
> User: sega
> Time: 2021-10-07T14:18:58.408
> Command [SNAPSHOT] started
> Arguments: --snapshot status --yes 
> 
> Status of SNAPSHOT operations:
> gridCommandHandlerTest0 -> No snapshot operation.
> gridCommandHandlerTest1 -> No snapshot operation.
> Command [SNAPSHOT] finished with code: 0
> Control utility has completed execution at: 2021-10-07T14:18:58.439
> Execution time: 31 ms
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-15023) ClassNotFoundException when using security and trying to set a remote listener

2022-07-06 Thread Sergei Ryzhov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-15023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563429#comment-17563429
 ] 

Sergei Ryzhov commented on IGNITE-15023:


[~dpavlov] hi, now i have a bit free time and i don't know when i can do it
but, i'm going to do it

> ClassNotFoundException when using security and trying to set a remote listener
> --
>
> Key: IGNITE-15023
> URL: https://issues.apache.org/jira/browse/IGNITE-15023
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergei Ryzhov
>Assignee: Sergei Ryzhov
>Priority: Minor
>  Labels: ise
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> ClassNotFoundException when using security and trying to set a remote listener
> the issue in using SecurityAwarePredicate on a client node
> it makes a wrapper over the remote listener so the class is not passed 
> through the Peer Class Loader
> Caused by: java.lang.ClassNotFoundException: 
> examples.StartClientXml$$Lambda$703/253380088
>   at java.lang.Class.forName0(Native Method)
>   at java.lang.Class.forName(Class.java:348)
>   at 
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:9064)
>   at 
> org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:9002)
>   at 
> org.apache.ignite.internal.MarshallerContextImpl.getClass(MarshallerContextImpl.java:376)
>   at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.descriptorFromCache(OptimizedMarshallerUtils.java:329)
>   at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshallerUtils.classDescriptor(OptimizedMarshallerUtils.java:274)
>   at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readClass(OptimizedObjectInputStream.java:384)
>   at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:329)
>   at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:205)
>   at java.io.ObjectInputStream.readObject(ObjectInputStream.java:367)
>   at 
> org.apache.ignite.internal.SecurityAwarePredicate.readExternal(SecurityAwarePredicate.java:86)
>   at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:560)
>   at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:980)
>   at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:353)
>   at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:205)
>   at java.io.ObjectInputStream.readObject(ObjectInputStream.java:367)
>   at 
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:251)
>   ... 22 more



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-12852) Comma in field is not supported by COPY command

2022-07-06 Thread Anton Kurbanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563409#comment-17563409
 ] 

Anton Kurbanov commented on IGNITE-12852:
-

[~liyuj] would it be ok for me to get this ticket assigned back to me?

> Comma in field is not supported by COPY command
> ---
>
> Key: IGNITE-12852
> URL: https://issues.apache.org/jira/browse/IGNITE-12852
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.8
>Reporter: YuJue Li
>Assignee: YuJue Li
>Priority: Critical
> Fix For: 2.14
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> CREATE TABLE test(a int,b varchar(100),c int,PRIMARY key(a)); 
>  
> a.csv: 
> 1,"a,b",2 
>  
> COPY FROM '/data/a.csv' INTO test (a,b,c) FORMAT CSV; 
>  
> The copy command fails because there is a comma in the second field,but this 
> is a fully legal and compliant CSV format



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-16760) Performance degradation of IgniteTables#tables after configuration changes

2022-07-06 Thread Evgeny Stanilovsky (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563376#comment-17563376
 ] 

Evgeny Stanilovsky commented on IGNITE-16760:
-

[~Denis Chudov] tests with annotation 
@Disabled("https://issues.apache.org/jira/browse/IGNITE-16760";) still present 
in main, can you take a look ?

> Performance degradation of IgniteTables#tables after configuration changes
> --
>
> Key: IGNITE-16760
> URL: https://issues.apache.org/jira/browse/IGNITE-16760
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0.0-alpha4
>Reporter: Taras Ledkov
>Assignee: Denis Chudov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Performance degradation of configuration changes:
> Steps to reproduce:
> 1. Start cluster with 3 nodes
> 2. Run in the loop
> {code}
> CREATE TABLE TEST(ID INTEGER PRIMARY KEY, V INTEGER)
> for (Table t : ign.tables().tables()) {
> ;
> }
> {code}
> On begin {{IgniteTables#tables}} takes ~ 0.7 sec.
> The time of the operation  is grown.
> The time after ~100 iteration is about 20 sec.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-17267) Preparing a checkpoint for a DeltaFilePageStore

2022-07-06 Thread Roman Puchkovskiy (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563352#comment-17563352
 ] 

Roman Puchkovskiy commented on IGNITE-17267:


The patch looks good to me, thank you for the contribution

> Preparing a checkpoint for a DeltaFilePageStore
> ---
>
> Key: IGNITE-17267
> URL: https://issues.apache.org/jira/browse/IGNITE-17267
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> To implement the *DeltaFilePageStore*, we need to prepare a checkpoint for 
> their creation.
> What should be done:
> # Sort dirty pages (*FullPageId*) by *groupId*, *partitionId*, *pageIdx*;
> # In several threads, write dirty pages in batches for each partition and 
> know the number of these pages.
> What can we look at:
> * 
> *org.apache.ignite.internal.pagememory.persistence.checkpoint.CheckpointWorkflow#beginCheckpoint*
> * 
> *org.apache.ignite.internal.pagememory.persistence.checkpoint.CheckpointWorkflow#splitAndSortCheckpointPagesIfNeeded*
> * 
> *org.apache.ignite.internal.pagememory.persistence.checkpoint.Checkpointer#writePages*
> Some implementation notes:
> * We can replace *CheckpointWorkflow#parallelSortThreshold* with a constant 
> at 40k, according to the 
> [paper|https://www.researchgate.net/publication/331742843_Threshold_Analysis_and_Comparison_of_Sequential_and_Parallel_Divide_and_Conquer_Sorting_Algorithms].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-17032) Apache Ignite Docker container does not run correctly if image is run in read only file system mode

2022-07-06 Thread Petar Tonkovic (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563331#comment-17563331
 ] 

Petar Tonkovic commented on IGNITE-17032:
-

Hi [~slukyanov],

Thanks for your comment. Indeed, we have already discovered the workaround with 
mounting the tmp folder in the yaml definition:

 
{code:java}
volumeMounts:       
- name: temp-vol
  mountPath: /tmp
  readOnly: false{code}
 

However, it might be a nice improvement to have everything working out of the 
box as we lost quite some time finding the workaround ourselves and it was not 
quite obvious.

Regarding the writable disk, sure Ignite needs to write on disk, but again 
usually you provide it the mount which is able to be written to do so, which is 
hopefully not in the installation folder. :)

In many cases Kubernetes clusters provided by company infrastructure teams 
enforce that the images are run with this read only file system flag as part of 
their security requirements and it makes total sense:
 # You prepare your installation file system pre-deployment with your Docker 
file by building it in the required image;
 # After said image is ran, there is no room for any kind of malicious software 
doing any kind of injections in your pods.

My company has exactly this security policy enforced and I have struggled a lot 
with various software that tries to write some temp files in the installation 
folder or something similar when trying to deploy it in a container. They 
either forget to let you set the log path somehow (usually via environment 
variable) and then connect it to a volume mount that is read/write.

I can imagine that these security measures might be a common occurrence in the 
industry, so could be a nice improvement. Thank you for looking into it.

P.S. We have noticed the same script being used in the web console agent (we 
used the one from GridGain, not sure if it is the case with the vanilla one as 
well), just FYI.

> Apache Ignite Docker container does not run correctly if image is run in read 
> only file system mode
> ---
>
> Key: IGNITE-17032
> URL: https://issues.apache.org/jira/browse/IGNITE-17032
> Project: Ignite
>  Issue Type: Bug
>  Components: build
>Affects Versions: 2.13
>Reporter: Petar Tonkovic
>Priority: Major
>
> When following the Kubernetes deployment tutorials (online: 
> https://ignite.apache.org/docs/latest/installation/kubernetes/azure-deployment,
>  youtube: [https://youtu.be/38YgdAOs038]), trying to run the official docker 
> image () with the --read-only flag is causing errors:
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 52: cannot create 
> temp file for here-document: Read-only file system
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 85: [: -lt: unary 
> operator expected2022-05-25T14:27:34.504369604+02:00
>  
> Since most managed company Kubernetes clusters enforce this read-only flag as 
> a security requirement, it would be good to look into these errors.
>  
> Later on, we get the following error on starting up:
> class org.apache.ignite.IgniteException: Failed to instantiate Spring XML 
> application context (make sure all classes used in Spring configuration are 
> present at CLASSPATH) [springUrl=file:/ignite/config/node-configuration.xml]
> at 
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1135)
> at org.apache.ignite.Ignition.start(Ignition.java:356)
> at 
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:365)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to 
> instantiate Spring XML application context (make sure all classes used in 
> Spring configuration are present at CLASSPATH) 
> [springUrl=file:/ignite/config/node-configuration.xml]
> at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:364)
> at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:102)
> at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:96)
> at 
> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:729)2022-05-25T14:27:34.916588365+02:00
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:930)
> at 
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:839)2022-05-25T14:27:34.916609431+02:00
> at 
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:709)2022-05-25T14:27:34.916622089+02:00
> at 
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:678)2022-05-25T14:27:34.916636146+02:00
> at 
> org.apache.ignite.Ignition.start(Ignition.java:353)2022-05-25T

[jira] [Created] (IGNITE-17321) Document which api can work with partition awareness

2022-07-06 Thread Luchnikov Alexander (Jira)
Luchnikov Alexander created IGNITE-17321:


 Summary: Document which api can work with partition awareness
 Key: IGNITE-17321
 URL: https://issues.apache.org/jira/browse/IGNITE-17321
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Reporter: Luchnikov Alexander


In javadoc 
org.apache.ignite.configuration.ClientConfiguration#partitionAwarenessEnabled 
and in the description of functionality
https://ignite.apache.org/docs/latest/thin-clients/java-thin-client#partition-awareness,
 it is not described with which api this functionality will work and in what 
cases. For example, will it work with getAll, in a transaction?

Describe in the documentation and in the javadoc in which cases it works and 
with which api.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: 
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check txnState 
along with adjusting HLC timestamp on txnStates group. If transaction occurred 
to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITTED - writeInent returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself, it's required to adjust HLC timestamp on txnState partition 
in order to guarantee RO reads and tx commit linearizability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO 
timestamp that already resolved corresponding writeInent as pending. It's not 
only required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  

There's an open question about following tx design feature:
{code:java}
If the state is NULL, this means txn is not yet started to finish and its 
outcome is not known. Send TxStateReq(txId,txnReadTs) to the txCoord to trigger 
the coordinator path.{code}
Seems that it's not required, let's discuss it explicitly.

  was:
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on 
txnStates group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITTED - writeInent returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself, it's required to adjust HLC timestamp on txnState partition 
in order to guarantee RO reads and tx commit linearizability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO 
timestamp that already resolved corresponding writeInent as pending. It's not 
only required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  


> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
> Attachments: Screenshot from 2022-07-06 17-00-26.png
>
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> txnState along with adjusting HLC timestamp on txnStates group. If 
> transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITTED - writeInent returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself, it's required to adjust HLC timestamp on txnState 
> partition in order to guarantee RO reads and tx commit linearizability, 
> meaning that it's illegal to commit transaction with commitTimestamp less or 
> equal to RO timestamp that already resolved corresponding writeInent as 
> pending. It's not only required to perform timestamp adjustment but it's also 
> important to do it atomically to txnState switch.  
> There's an open question about following tx design feature:
> {code:java}
> If the state is NULL, this means txn is not yet started to finish and its 
> outcome is not known. Send TxStateReq(txId,txnReadTs) to the txCoord to 
> trigger the coordinator path.{code}
> Seems that it's not required, let's discuss it explicitly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Attachment: Screenshot from 2022-07-06 17-00-26.png

> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
> Attachments: Screenshot from 2022-07-06 17-00-26.png
>
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
> [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] 
> on txnStates group. If transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITTED - writeInent returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself, it's required to adjust HLC timestamp on txnState 
> partition in order to guarantee RO reads and tx commit linearizability, 
> meaning that it's illegal to commit transaction with commitTimestamp less or 
> equal to RO timestamp that already resolved corresponding writeInent as 
> pending. It's not only required to perform timestamp adjustment but it's also 
> important to do it atomically to txnState switch.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17263) Implement leader to replica safe time propagation

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17263:
-
Description: 
In order to perform replica reads, it's required either to use read index or 
check the safe time. Let's recall corresponding section from tx design document.

RO transactions can be executed on non-primary replicas. write intent 
resolution doesn’t help because a write intent for a committed transaction may 
not be yet replicated to the replica. To mitigate this issue, it’s enough to 
run readIndex on each mapped partition leader, fetch the commit index and wait 
on a replica until it’s applied. This will guarantee that all required write 
intents are replicated and present locally. After that the normal write intern 
resolution should do the job.

There is a second option, which doesn’t require the network RTT. We can use a 
special low watermark timestamp (safeTs) per replication group, which 
corresponds to the apply index of a replicated entry, so then an apply index is 
advanced during the replication, then the safeTs is monotonically incremented 
too. The HLC used for safeTs advancing is assigned to a replicated entry in an 
ordered way.

Special measures are needed to periodically advance the safeTs if no updates 
are happening. It’s enough to use a special replication command for this 
purpose.

All we need during RO txn is to wait until a safeTs advances past the RO txn 
readTs. 
 !Screenshot from 2022-07-06 16-48-30.png! 
In the picture we have two concurrent transactions mapped to the same 
partition: T1 and T2.
OpReq(w1(x)) and OpReq(w2(x)) are received concurrently. Each write intent is 
assigned a timestamp in a monotonic order consistent with the replication 
order. This can be for example done when replication entries are dequeued for 
processing by replication protocol (we assume entries are replicated 
successively.

It’s not enough only to wait for safeTs - it may never happen due to absence of 
activity in the partition. Consider the next diagram:
 !Screenshot from 2022-07-06 16-48-41.png! 
We need an additional safeTsSync command to propagate a safeTs event in case 
there are no updates in the partition.

Actually, it seems that it's possible to reuse common raft messages such as 
heartbeatRequests, vote/prevoteRequests together with appendEntriesRequests in 
order to propagate safeTime from leader to replicas. As was mentioned in 
[IGNITE-17261|https://issues.apache.org/jira/browse/IGNITE-17261] txnState 
switch should be linearized with all safe-time propagation requests.

  was:
In order to perform replica reads, it's required either to use read index or 
check the safe time. Let's recall corresponding section from tx design document.

RO transactions can be executed on non-primary replicas. write intent 
resolution doesn’t help because a write intent for a committed transaction may 
not be yet replicated to the replica. To mitigate this issue, it’s enough to 
run readIndex on each mapped partition leader, fetch the commit index and wait 
on a replica until it’s applied. This will guarantee that all required write 
intents are replicated and present locally. After that the normal write intern 
resolution should do the job.

There is a second option, which doesn’t require the network RTT. We can use a 
special low watermark timestamp (safeTs) per replication group, which 
corresponds to the apply index of a replicated entry, so then an apply index is 
advanced during the replication, then the safeTs is monotonically incremented 
too. The HLC used for safeTs advancing is assigned to a replicated entry in an 
ordered way.

Special measures are needed to periodically advance the safeTs if no updates 
are happening. It’s enough to use a special replication command for this 
purpose.

All we need during RO txn is to wait until a safeTs advances past the RO txn 
readTs. 
 !Screenshot from 2022-07-06 16-48-30.png! 


> Implement leader to replica safe time propagation
> -
>
> Key: IGNITE-17263
> URL: https://issues.apache.org/jira/browse/IGNITE-17263
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
> Attachments: Screenshot from 2022-07-06 16-48-30.png, Screenshot from 
> 2022-07-06 16-48-41.png
>
>
> In order to perform replica reads, it's required either to use read index or 
> check the safe time. Let's recall corresponding section from tx design 
> document.
> RO transactions can be executed on non-primary replicas. write intent 
> resolution doesn’t help because a write intent for a committed transaction 
> may not be yet replicated to the replica. To mitigate this issue, it’s enough 
> to run readIndex on each mapped partition leader, fetch the c

[jira] [Updated] (IGNITE-17263) Implement leader to replica safe time propagation

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17263:
-
Attachment: Screenshot from 2022-07-06 16-48-41.png

> Implement leader to replica safe time propagation
> -
>
> Key: IGNITE-17263
> URL: https://issues.apache.org/jira/browse/IGNITE-17263
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
> Attachments: Screenshot from 2022-07-06 16-48-30.png, Screenshot from 
> 2022-07-06 16-48-41.png
>
>
> In order to perform replica reads, it's required either to use read index or 
> check the safe time. Let's recall corresponding section from tx design 
> document.
> RO transactions can be executed on non-primary replicas. write intent 
> resolution doesn’t help because a write intent for a committed transaction 
> may not be yet replicated to the replica. To mitigate this issue, it’s enough 
> to run readIndex on each mapped partition leader, fetch the commit index and 
> wait on a replica until it’s applied. This will guarantee that all required 
> write intents are replicated and present locally. After that the normal write 
> intern resolution should do the job.
> There is a second option, which doesn’t require the network RTT. We can use a 
> special low watermark timestamp (safeTs) per replication group, which 
> corresponds to the apply index of a replicated entry, so then an apply index 
> is advanced during the replication, then the safeTs is monotonically 
> incremented too. The HLC used for safeTs advancing is assigned to a 
> replicated entry in an ordered way.
> Special measures are needed to periodically advance the safeTs if no updates 
> are happening. It’s enough to use a special replication command for this 
> purpose.
> All we need during RO txn is to wait until a safeTs advances past the RO txn 
> readTs. 
>  !Screenshot from 2022-07-06 16-48-30.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17263) Implement leader to replica safe time propagation

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17263:
-
Description: 
In order to perform replica reads, it's required either to use read index or 
check the safe time. Let's recall corresponding section from tx design document.

RO transactions can be executed on non-primary replicas. write intent 
resolution doesn’t help because a write intent for a committed transaction may 
not be yet replicated to the replica. To mitigate this issue, it’s enough to 
run readIndex on each mapped partition leader, fetch the commit index and wait 
on a replica until it’s applied. This will guarantee that all required write 
intents are replicated and present locally. After that the normal write intern 
resolution should do the job.

There is a second option, which doesn’t require the network RTT. We can use a 
special low watermark timestamp (safeTs) per replication group, which 
corresponds to the apply index of a replicated entry, so then an apply index is 
advanced during the replication, then the safeTs is monotonically incremented 
too. The HLC used for safeTs advancing is assigned to a replicated entry in an 
ordered way.

Special measures are needed to periodically advance the safeTs if no updates 
are happening. It’s enough to use a special replication command for this 
purpose.

All we need during RO txn is to wait until a safeTs advances past the RO txn 
readTs. 
 !Screenshot from 2022-07-06 16-48-30.png! 

> Implement leader to replica safe time propagation
> -
>
> Key: IGNITE-17263
> URL: https://issues.apache.org/jira/browse/IGNITE-17263
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
> Attachments: Screenshot from 2022-07-06 16-48-30.png, Screenshot from 
> 2022-07-06 16-48-41.png
>
>
> In order to perform replica reads, it's required either to use read index or 
> check the safe time. Let's recall corresponding section from tx design 
> document.
> RO transactions can be executed on non-primary replicas. write intent 
> resolution doesn’t help because a write intent for a committed transaction 
> may not be yet replicated to the replica. To mitigate this issue, it’s enough 
> to run readIndex on each mapped partition leader, fetch the commit index and 
> wait on a replica until it’s applied. This will guarantee that all required 
> write intents are replicated and present locally. After that the normal write 
> intern resolution should do the job.
> There is a second option, which doesn’t require the network RTT. We can use a 
> special low watermark timestamp (safeTs) per replication group, which 
> corresponds to the apply index of a replicated entry, so then an apply index 
> is advanced during the replication, then the safeTs is monotonically 
> incremented too. The HLC used for safeTs advancing is assigned to a 
> replicated entry in an ordered way.
> Special measures are needed to periodically advance the safeTs if no updates 
> are happening. It’s enough to use a special replication command for this 
> purpose.
> All we need during RO txn is to wait until a safeTs advances past the RO txn 
> readTs. 
>  !Screenshot from 2022-07-06 16-48-30.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17263) Implement leader to replica safe time propagation

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17263:
-
Attachment: Screenshot from 2022-07-06 16-48-30.png

> Implement leader to replica safe time propagation
> -
>
> Key: IGNITE-17263
> URL: https://issues.apache.org/jira/browse/IGNITE-17263
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
> Attachments: Screenshot from 2022-07-06 16-48-30.png, Screenshot from 
> 2022-07-06 16-48-41.png
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: 
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on 
txnStates group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITTED - writeInent returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself, it's required to adjust HLC timestamp on txnState partition 
in order to guarantee RO reads and tx commit linearizability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO 
timestamp that already resolved corresponding writeInent as pending. It's not 
only required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  

  was:
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on 
txnStates group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITTED - writeInent returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself, it's required to adjust HLC timestamp on txnState partition 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO 
timestamp that already resolved corresponding writeInent as pending. It's not 
only required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  


> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
> [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] 
> on txnStates group. If transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITTED - writeInent returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself, it's required to adjust HLC timestamp on txnState 
> partition in order to guarantee RO reads and tx commit linearizability, 
> meaning that it's illegal to commit transaction with commitTimestamp less or 
> equal to RO timestamp that already resolved corresponding writeInent as 
> pending. It's not only required to perform timestamp adjustment but it's also 
> important to do it atomically to txnState switch.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: 
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on 
txnStates group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  

  was:
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on 
txnStates group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent  returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  


> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
> [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] 
> on txnStates group. If transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITED - writeInent returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself it's required to adjust HLC timestamp on txnState 
> partitiion in order to guarantee RO reads and tx commit inevitability, 
> meaning that it's illegal to commit transaction with commitTimestamp less or 
> equal to RO timesmat that already resolved corresponding writeInent as 
> pending. It's not only required to perform timestamp adjustment but it's also 
> important to do it atomically to txnState switch.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: 
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on 
txnStates group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITTED - writeInent returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself, it's required to adjust HLC timestamp on txnState partition 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO 
timestamp that already resolved corresponding writeInent as pending. It's not 
only required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  

  was:
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on 
txnStates group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself, it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  


> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
> [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] 
> on txnStates group. If transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITTED - writeInent returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself, it's required to adjust HLC timestamp on txnState 
> partition in order to guarantee RO reads and tx commit inevitability, meaning 
> that it's illegal to commit transaction with commitTimestamp less or equal to 
> RO timestamp that already resolved corresponding writeInent as pending. It's 
> not only required to perform timestamp adjustment but it's also important to 
> do it atomically to txnState switch.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: 
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on 
txnStates group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself, it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  

  was:
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on 
txnStates group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  


> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
> [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] 
> on txnStates group. If transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITED - writeInent returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself, it's required to adjust HLC timestamp on txnState 
> partitiion in order to guarantee RO reads and tx commit inevitability, 
> meaning that it's illegal to commit transaction with commitTimestamp less or 
> equal to RO timesmat that already resolved corresponding writeInent as 
> pending. It's not only required to perform timestamp adjustment but it's also 
> important to do it atomically to txnState switch.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: 
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
[adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on 
txnStates group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent  returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  

  was:
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check txnState 
along with [adjusting HLC 
timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates 
group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent  returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  


> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> [txnState|https://issues.apache.org/jira/browse/IGNITE-16882] along with 
> [adjusting HLC timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] 
> on txnStates group. If transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITED - writeInent  returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself it's required to adjust HLC timestamp on txnState 
> partitiion in order to guarantee RO reads and tx commit inevitability, 
> meaning that it's illegal to commit transaction with commitTimestamp less or 
> equal to RO timesmat that already resolved corresponding writeInent as 
> pending. It's not only required to perform timestamp adjustment but it's also 
> important to do it atomically to txnState switch.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: 
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check txnState 
along with [adjusting HLC 
timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates 
group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent  returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  

  was:
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|http://example.com] along with adjusting HLC timestamp on txnStates 
group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent  returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  


> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> txnState along with [adjusting HLC 
> timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates 
> group. If transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITED - writeInent  returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself it's required to adjust HLC timestamp on txnState 
> partitiion in order to guarantee RO reads and tx commit inevitability, 
> meaning that it's illegal to commit transaction with commitTimestamp less or 
> equal to RO timesmat that already resolved corresponding writeInent as 
> pending. It's not only required to perform timestamp adjustment but it's also 
> important to do it atomically to txnState switch.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: 
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check 
[txnState|http://example.com] along with adjusting HLC timestamp on txnStates 
group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent  returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  

  was:
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check txnState 
along with [adjusting HLC 
timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates 
group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent  returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  


> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> [txnState|http://example.com] along with adjusting HLC timestamp on txnStates 
> group. If transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITED - writeInent  returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself it's required to adjust HLC timestamp on txnState 
> partitiion in order to guarantee RO reads and tx commit inevitability, 
> meaning that it's illegal to commit transaction with commitTimestamp less or 
> equal to RO timesmat that already resolved corresponding writeInent as 
> pending. It's not only required to perform timestamp adjustment but it's also 
> important to do it atomically to txnState switch.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: 
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check txnState 
along with [adjusting HLC 
timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates 
group. If transaction occurred to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent  returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  

  was:
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check txnState 
along with adjusting HLC timestamp on txnStates group. If transaction occurred 
to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent  returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  


> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> txnState along with [adjusting HLC 
> timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates 
> group. If transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITED - writeInent  returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself it's required to adjust HLC timestamp on txnState 
> partitiion in order to guarantee RO reads and tx commit inevitability, 
> meaning that it's illegal to commit transaction with commitTimestamp less or 
> equal to RO timesmat that already resolved corresponding writeInent as 
> pending. It's not only required to perform timestamp adjustment but it's also 
> important to do it atomically to txnState switch.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: 
Because of lock-free nature, RO reads might interact with writeIntents, meaning 
that such intents should be either evaluated as committed, aborted or pending. 
In order to perform writeIntent resolution it's required to check txnState 
along with adjusting HLC timestamp on txnStates group. If transaction occurred 
to be
 * pending (there is no explicit txnState for given txnId specified in 
writeIntent) or txnState is ABORTED - writeIntent ignored.
 * COMMITED - writeInent  returned as RO read response.

Please pay attention that, as was mentioned above, besides writeIntent 
resolution itself it's required to adjust HLC timestamp on txnState partitiion 
in order to guarantee RO reads and tx commit inevitability, meaning that it's 
illegal to commit transaction with commitTimestamp less or equal to RO timesmat 
that already resolved corresponding writeInent as pending. It's not only 
required to perform timestamp adjustment but it's also important to do it 
atomically to txnState switch.  

  was:Because of lock-free nature RO reads might interact with writeIntents, 
meaning that such intents should be either evaluated as committed, aborted or 
pending. In order to perform writeIntent resolution it's required to check 
txnState along with [adjusting HLC 
timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates 
group.


> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> txnState along with adjusting HLC timestamp on txnStates group. If 
> transaction occurred to be
>  * pending (there is no explicit txnState for given txnId specified in 
> writeIntent) or txnState is ABORTED - writeIntent ignored.
>  * COMMITED - writeInent  returned as RO read response.
> Please pay attention that, as was mentioned above, besides writeIntent 
> resolution itself it's required to adjust HLC timestamp on txnState 
> partitiion in order to guarantee RO reads and tx commit inevitability, 
> meaning that it's illegal to commit transaction with commitTimestamp less or 
> equal to RO timesmat that already resolved corresponding writeInent as 
> pending. It's not only required to perform timestamp adjustment but it's also 
> important to do it atomically to txnState switch.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17261) Implement write intent resolution logic for RO reads

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17261:
-
Description: Because of lock-free nature RO reads might interact with 
writeIntents, meaning that such intents should be either evaluated as 
committed, aborted or pending. In order to perform writeIntent resolution it's 
required to check txnState along with [adjusting HLC 
timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates 
group.

> Implement write intent resolution logic for RO reads
> 
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Because of lock-free nature RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to check 
> txnState along with [adjusting HLC 
> timestamp|https://issues.apache.org/jira/browse/IGNITE-17222] on txnStates 
> group.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17311) KafkaToIgniteLoader instantiates several Spring contexts

2022-07-06 Thread Nikolay Izhikov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikolay Izhikov updated IGNITE-17311:
-
Labels: IEP-59  (was: )

> KafkaToIgniteLoader instantiates several Spring contexts
> 
>
> Key: IGNITE-17311
> URL: https://issues.apache.org/jira/browse/IGNITE-17311
> Project: Ignite
>  Issue Type: Improvement
>  Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>  Labels: IEP-59
> Fix For: 2.14
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> KafkaToIgniteLoader instantiates two SpringApplicationContext during app 
> start.
> This can lead to error in case other beans inside context not designed for 
> this case.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-16650) Exclude ignite-log4j, log4j 1.2.17

2022-07-06 Thread Mikhail Petrov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Petrov updated IGNITE-16650:

Description: 
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces

I suggest excluding the ignite-log4j module from ignite

Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571

As a result of the mentioned migration, the following changes will be applied:
1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after the 
refactoring we will get two configuration ignite-log4j.xml and 
ignite-log4j2.xml both in log4j2 format because ignite-log4j2.xml is in use now 
and but provide log formatitng different from ignite-log4j.xml.
2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because 
it is used with compatibility tests.
3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j 
format. The current  version of core/src/test/config/log4j2-test.xml  is moved 
to the log4j2/src/test/config folder.
4. osgi-paxlogging will be removed because it's only meant to provide some 
log4j dependencies. We have no need in them now.
5. Exception logging format will change slightly:

Before:
{code:java}
class org.apache.ignite.IgniteException: Platform error:System.Exception: 
EXCEPTION_TEST_Warn
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at 
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
After:
{code:java}
org.apache.ignite.IgniteException: Platform error:System.Exception: 
EXCEPTION_TEST_Warn
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at 
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
As you can see, only the first word "class" is omitted.
6. All other files containing log4j configuration will be refactored to suite 
log4j2 and will be renamed if previously their name allowed log4j to 
automatically find them in the class path (e.g. log4j.xml -> log4j2.xml and so 
on)




  was:
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces

I suggest excluding the ignite-log4j module from ignite

Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571

As a result of the mentioned migration, the following changes will be applied:
1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after the 
refactoring we will get two configuration ignite-log4j.xml and 
ignite-log4j2.xml both in log4j2 format because the main goal of this 
refactoring is to keep current log formatting that is used with log4j intact. 
Currently ignite-log4j.xml and ignite-log4j2.xml provides different log formats 
for log4j and log4j2 respectively.
2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because 
it is used with compatibility tests.
3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j 
format. The current  version of core/src/test/config/log4j2-test.xml  is moved 
to the log4j2/src/test/config folder.
4. osgi-paxlogging will be removed because it's only meant to provide some 
log4j dependencies. We have no need in them now.
5. Exception logging format will change slightly:

Before:
{code:java}
class org.apache.ignite.IgniteException: Platform error:System.Exception: 
EXCEPTION_TEST_Warn
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at 
or

[jira] [Created] (IGNITE-17320) Implement B+Tree based sorted index storage

2022-07-06 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-17320:
--

 Summary: Implement B+Tree based sorted index storage
 Key: IGNITE-17320
 URL: https://issues.apache.org/jira/browse/IGNITE-17320
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov


Like in IGNITE-17318, binary tuple format is a main goal here, because that's 
what we're gonna pass into API.

Implementing a tree for indexes isn't hard by itself. I expect it to be stored 
in same partition files as raw storage, everything should be colocated in a 
single file.

What's complicated is the inlining of data into trees pages. I see it as the 
following:
 * If tuple has no offset table, we can always store the entire payload in the 
tree. This is the best case scenario, because the size is fixed and known a 
priori, we don't even need to store it before the payload.
 * If tuple has an offset table, the inline size will immediately get bigger. 
In my view, we will have to:
 ** store the size of inlined payload, that's 4 bytes
 ** store null table if it's there, that's a known amount of bytes
 ** store header and offset table:
 *** if there are non-fixed-length columns, then a single entry in offset table 
can be up to 4 bytes
 *** if there are only fixed-length columns, like ints, floats or even bitsets, 
the amount of bytes per single entry can be accurately estimated with the upper 
bound
 ** then store a good amount of the actual columns data. How much? I'd be 
generous, but then we would probably have too much space, so all of this is 
debatable:
 *** for columns with fixed size, allocate room for entire value
 *** for strings and numbers (is there something else?) we have to pre-allocate 
a reasonable amount of bytes. Like, 8, for example. I don't know, there are 
defaults somewhere in the code of Ignite 2.x, we can use them.

So, my point is, there's no new data format for inlined section of the tuple, 
we should reuse it and thus avoid many possible errors. And, if record fits 
into a tree page, there's no need in inserting it into a free list. Good!

And yes, of course, there has to be a room for row id in inlined section as 
well.

Last, meta tree is still a thing. But, we shouldn't identify indexes by their 
names, cause there's UUID id or even integer id (see IGNITE-17318).
h3. Other ideas

I don't like how durable background tasks work in Ignite 2.x, there are always 
some issues. I would prefer having a general-purposed "recycle bin" in 
partition and a background cleaner process that would clean it. Maybe this 
queue should contain other entities in the future.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-17319) Remove entries replicated to all nodes from RAFT log in in-memory scenario

2022-07-06 Thread Roman Puchkovskiy (Jira)
Roman Puchkovskiy created IGNITE-17319:
--

 Summary: Remove entries replicated to all nodes from RAFT log in 
in-memory scenario
 Key: IGNITE-17319
 URL: https://issues.apache.org/jira/browse/IGNITE-17319
 Project: Ignite
  Issue Type: Improvement
  Components: persistence
Reporter: Roman Puchkovskiy
Assignee: Roman Puchkovskiy
 Fix For: 3.0.0-alpha6


TBD



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-16650) Exclude ignite-log4j, log4j 1.2.17

2022-07-06 Thread Mikhail Petrov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Petrov updated IGNITE-16650:

Description: 
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces

I suggest excluding the ignite-log4j module from ignite

Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571

As a result of the mentioned migration, the following changes will be applied:
1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after the 
refactoring we will get two configuration ignite-log4j.xml and 
ignite-log4j2.xml both in log4j2 format because the main goal of this 
refactoring is to keep current log formatting that is used with log4j intact. 
Currently ignite-log4j.xml and ignite-log4j2.xml provides different log formats 
for log4j and log4j2 respectively.
2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because 
it is used with compatibility tests.
3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j 
format. The current  version of core/src/test/config/log4j2-test.xml  is moved 
to the log4j2/src/test/config folder.
4. osgi-paxlogging will be removed because it's only meant to provide some 
log4j dependencies. We have no need in them now.
5. Exception logging format will change slightly:

Before:
{code:java}
class org.apache.ignite.IgniteException: Platform error:System.Exception: 
EXCEPTION_TEST_Warn
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at 
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
After:
{code:java}
org.apache.ignite.IgniteException: Platform error:System.Exception: 
EXCEPTION_TEST_Warn
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at 
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
As you can see, only the first word "class" is omitted.
6. All other files containing log4j configuration will be refactored to suite 
log4j2 and will be renamed if previously their name allowed log4j to 
automatically find them in the class path (e.g. log4j.xml -> log4j2.xml and so 
on)




  was:
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces

I suggest excluding the ignite-log4j module from ignite

Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571

As a result of the mentioned migration, the following changes will be applied:
1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after the 
refactoring we will get two configuration ignite-log4j.xml and 
ignite-log4j2.xml both in log4j2 format because the main goal of this 
refactoring is to keep current log formatting that is used with log4j intact. 
Currently ignite-log4j.xml and ignite-log4j2.xml provides different log formats 
for log4j and log4j2 respectively.
2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because 
it is used in compatibility tests.
3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j 
format. The current  version of core/src/test/config/log4j2-test.xml  is moved 
to the log4j2/src/test/config folder/
4. osgi-paxlogging will be removed because it's only meant to provide some 
log4j dependencies.
5. Exception logging format will change slightly:

Before:
{code:java}
class org.apache.ignite.IgniteException: Platform error:System.Exception: 
EXCEPTION_TEST_Warn
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at 
org.apache.ignite.internal.processors.platform.P

[jira] [Updated] (IGNITE-17318) Implement RocksDB based sorted index storage

2022-07-06 Thread Ivan Bessonov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-17318:
---
Labels: ignite-3  (was: )

> Implement RocksDB based sorted index storage
> 
>
> Key: IGNITE-17318
> URL: https://issues.apache.org/jira/browse/IGNITE-17318
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
>
> Pretty straightforward. Complicated places are:
>  * Binary tuples comparator: should be as fast as possible. Some 
> optimizations might be moved to other issues.
>  * Thorough testing is required. We have both Java and native comparators 
> planned. They should behave identically. This means a specific way of writing 
> tests, to account for this in advance.
>  * Bounds checking on range scan:
> by default, comparator should include the lower bound and exclude the upper 
> bound. This is how prefix search works. This means that exclusion of the 
> lower bound (if needed) and inclusion of the upper bound (if needed) +must be 
> performed manually+ inside of the scan method.
> The question is, do we use separate column families for indexes? At one hand, 
> this increases the number of files and potentially even increases time of the 
> flush, but on the other, it looks easy (or is it).
> Currently, every index has only a UUID id. Just like for tables, we could 
> create an integer identifier, because why not. This way we could store all 
> indexes in a single column family without too much overhead.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-16650) Exclude ignite-log4j, log4j 1.2.17

2022-07-06 Thread Mikhail Petrov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Petrov updated IGNITE-16650:

Description: 
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces

I suggest excluding the ignite-log4j module from ignite

Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571

As a result of the mentioned migration, the following changes will be applied:
1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after the 
refactoring we will get two configuration ignite-log4j.xml and 
ignite-log4j2.xml both in log4j2 format because the main goal of this 
refactoring is to keep current log formatting that is used with log4j intact. 
Currently ignite-log4j.xml and ignite-log4j2.xml provides different log formats 
for log4j and log4j2 respectively.
2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because 
it is used in compatibility tests.
3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j 
format. The current  version of core/src/test/config/log4j2-test.xml  is moved 
to the log4j2/src/test/config folder/
4. osgi-paxlogging will be removed because it's only meant to provide some 
log4j dependencies.
5. Exception logging format will change slightly:

Before:
{code:java}
class org.apache.ignite.IgniteException: Platform error:System.Exception: 
EXCEPTION_TEST_Warn
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at 
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
After:
{code:java}
org.apache.ignite.IgniteException: Platform error:System.Exception: 
EXCEPTION_TEST_Warn
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
at 
org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
at 
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
{code}
As we see - only the first "class" word is omitted.
6. All other files containing log4j configuration will be refactored to suite 
log4j2 and will be renamed if previously their name allowed log4j to 
automatically find them in the class path (e.g. log4j.xml -> log4j2.xml and so 
on)




  was:
log4j 1.2.17 is not supported and contains critical vulnerabilities
https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces

I suggest excluding the ignite-log4j module from ignite

Direct vulnerabilities:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571


> Exclude ignite-log4j, log4j 1.2.17
> --
>
> Key: IGNITE-16650
> URL: https://issues.apache.org/jira/browse/IGNITE-16650
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergei Ryzhov
>Assignee: Mikhail Petrov
>Priority: Major
>  Labels: ise
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> log4j 1.2.17 is not supported and contains critical vulnerabilities
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> I suggest excluding the ignite-log4j module from ignite
> Direct vulnerabilities:
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571
> As a result of the mentioned migration, the following changes will be applied:
> 1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after 
> the refactoring we will get two configuration ignite-log4j.xml and 
> ignite-log4j2.xml both in log4j2 format because the main goal of this 
> refactoring is to keep current log formatting that

[jira] [Assigned] (IGNITE-16650) Exclude ignite-log4j, log4j 1.2.17

2022-07-06 Thread Mikhail Petrov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Petrov reassigned IGNITE-16650:
---

Assignee: Mikhail Petrov  (was: Sergei Ryzhov)

> Exclude ignite-log4j, log4j 1.2.17
> --
>
> Key: IGNITE-16650
> URL: https://issues.apache.org/jira/browse/IGNITE-16650
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergei Ryzhov
>Assignee: Mikhail Petrov
>Priority: Major
>  Labels: ise
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> log4j 1.2.17 is not supported and contains critical vulnerabilities
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> I suggest excluding the ignite-log4j module from ignite
> Direct vulnerabilities:
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17311) KafkaToIgniteLoader instantiates several Spring contexts

2022-07-06 Thread Amelchev Nikita (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amelchev Nikita updated IGNITE-17311:
-
Fix Version/s: 2.14

> KafkaToIgniteLoader instantiates several Spring contexts
> 
>
> Key: IGNITE-17311
> URL: https://issues.apache.org/jira/browse/IGNITE-17311
> Project: Ignite
>  Issue Type: Improvement
>  Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
> Fix For: 2.14
>
>
> KafkaToIgniteLoader instantiates two SpringApplicationContext during app 
> start.
> This can lead to error in case other beans inside context not designed for 
> this case.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-17318) Implement RocksDB based sorted index storage

2022-07-06 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-17318:
--

 Summary: Implement RocksDB based sorted index storage
 Key: IGNITE-17318
 URL: https://issues.apache.org/jira/browse/IGNITE-17318
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov


Pretty straightforward. Complicated places are:
 * Binary tuples comparator: should be as fast as possible. Some optimizations 
might be moved to other issues.
 * Thorough testing is required. We have both Java and native comparators 
planned. They should behave identically. This means a specific way of writing 
tests, to account for this in advance.
 * Bounds checking on range scan:
by default, comparator should include the lower bound and exclude the upper 
bound. This is how prefix search works. This means that exclusion of the lower 
bound (if needed) and inclusion of the upper bound (if needed) +must be 
performed manually+ inside of the scan method.

The question is, do we use separate column families for indexes? At one hand, 
this increases the number of files and potentially even increases time of the 
flush, but on the other, it looks easy (or is it).

Currently, every index has only a UUID id. Just like for tables, we could 
create an integer identifier, because why not. This way we could store all 
indexes in a single column family without too much overhead.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-17317) Test IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata is flaky on TC

2022-07-06 Thread Sergey Chugunov (Jira)
Sergey Chugunov created IGNITE-17317:


 Summary: Test 
IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata
 is flaky on TC
 Key: IGNITE-17317
 URL: https://issues.apache.org/jira/browse/IGNITE-17317
 Project: Ignite
  Issue Type: Bug
Reporter: Sergey Chugunov
 Fix For: 2.14


Test history is available here: [TC 
link|https://ci.ignite.apache.org/test/-1717009344468901983?currentProjectId=IgniteTests24Java8&branch=%3Cdefault%3E]

Test fails on a cache put operation before a condition being verified is 
reached:

{code:java}
 org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys 
(retry update if possible).: [2]
at 
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
at 
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
at 
org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
{code}

There is also a stack trace in logs indicating there is some problem when 
writing to WAL:

{code:java}
org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys 
(retry update if possible).: [2]
at 
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
at 
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
at 
org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2428)
at java.base/java.lang.Thread.run(Thread.java:834)
  Caused by: class 
org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException: 
Failed to update keys (retry update if possible).: [2]
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:401)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:248)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateResponse(GridDhtAtomicCache.java:3213)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$500(GridDhtAtomicCache.java:145)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:305)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:300)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:592)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:393)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:319)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:110)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:309)
at 
org.apach

[jira] [Assigned] (IGNITE-17317) Test IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata is flaky on TC

2022-07-06 Thread Sergey Chugunov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Chugunov reassigned IGNITE-17317:


Assignee: Sergey Chugunov

> Test 
> IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata
>  is flaky on TC
> -
>
> Key: IGNITE-17317
> URL: https://issues.apache.org/jira/browse/IGNITE-17317
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Chugunov
>Assignee: Sergey Chugunov
>Priority: Major
> Fix For: 2.14
>
>
> Test history is available here: [TC 
> link|https://ci.ignite.apache.org/test/-1717009344468901983?currentProjectId=IgniteTests24Java8&branch=%3Cdefault%3E]
> Test fails on a cache put operation before a condition being verified is 
> reached:
> {code:java}
>  org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys 
> (retry update if possible).: [2]
> at 
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at 
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at 
> org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> {code}
> There is also a stack trace in logs indicating there is some problem when 
> writing to WAL:
> {code:java}
> org.apache.ignite.cache.CachePartialUpdateException: Failed to update keys 
> (retry update if possible).: [2]
> at 
> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1260)
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:2111)
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1347)
> at 
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:870)
> at 
> org.apache.ignite.internal.processors.cache.persistence.IgnitePdsBinaryMetadataAsyncWritingTest.testNodeIsStoppedOnExceptionDuringStoringMetadata(IgnitePdsBinaryMetadataAsyncWritingTest.java:257)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
> Method)
> at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at 
> org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2428)
> at java.base/java.lang.Thread.run(Thread.java:834)
>   Caused by: class 
> org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException:
>  Failed to update keys (retry update if possible).: [2]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:401)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:248)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processNearAtomicUpdateResponse(GridDhtAtomicCache.java:3213)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$500(GridDhtAtomicCache.java:145)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:305)
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$6.apply(GridDhtAtomicCache.java:300)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
> at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.jav

[jira] [Updated] (IGNITE-17309) Transactional support for partition scans

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17309:
-
Description: 
It's required to add transactional support to
{code:java}
org.apache.ignite.internal.table.InternalTable#scan {code}
binding it to
{code:java}
org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
 java.util.UUID) {code}
along with acquiring corresponding locks, namely S_commit(table) - if a 
predicate can produce phantom reads, IS_commit(table) - otherwise.

Besides transactional support itself, it worth to introduce direct storage 
reads from within PrimaryReplica instead of going through raft.

  was:
It's required to add transactional support to
{code:java}
org.apache.ignite.internal.table.InternalTable#scan {code}
binding it to
{code:java}
org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
 java.util.UUID) {code}
along with acquiring corresponding locks, namely S_commit(table) - if a 
predicate can produce phantom reads, IS_commit(table) - otherwise.


> Transactional support for partition scans
> -
>
> Key: IGNITE-17309
> URL: https://issues.apache.org/jira/browse/IGNITE-17309
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_rw
>
> It's required to add transactional support to
> {code:java}
> org.apache.ignite.internal.table.InternalTable#scan {code}
> binding it to
> {code:java}
> org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
>  java.util.UUID) {code}
> along with acquiring corresponding locks, namely S_commit(table) - if a 
> predicate can produce phantom reads, IS_commit(table) - otherwise.
> Besides transactional support itself, it worth to introduce direct storage 
> reads from within PrimaryReplica instead of going through raft.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17309) Transactional support for partition scans

2022-07-06 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-17309:
-
Description: 
It's required to add transactional support to
{code:java}
org.apache.ignite.internal.table.InternalTable#scan {code}
binding it to
{code:java}
org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
 java.util.UUID) {code}
along with acquiring corresponding locks, namely S_commit(table) - if a 
predicate can produce phantom reads, IS_commit(table) - otherwise.

  was:
It's required to add transactional support to
{code:java}
org.apache.ignite.internal.table.InternalTable#scan {code}
binding it to
{code:java}
org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
 java.util.UUID) {code}
along with acquiring corresponding locks, namely S_commit(table) - if a 
predicate can produce phantom reads, IS_commit(table) - otherwise.

Besides transactional support itself, it worth to ensure that scans together 
with other read commands don't linearized wihtin raft, meaning that raft in 
lease-based mode shouldn't perform read-index and thus properly perform 
multiple scans concurrently. If given assumption occurred to be incorrect it'll 
be required to re-implement scans in order to use storage directly without any 
raft interaction. It's possible because any RW-reads goes through 
PrimaryReplica that is guaranteed to be up-to-date. The only non-trivial moment 
here, is that according to raft paper storage itself isn't guaranteed to apply 
all commands up to commit index in a synchronous way, meaning that it's 
required to await for applyIndex to be greater or equal than commitIndex. 
IGNITE-16907 will introduce storage specific applied index, so that we should 
only extract commit index to replica somehow.


> Transactional support for partition scans
> -
>
> Key: IGNITE-17309
> URL: https://issues.apache.org/jira/browse/IGNITE-17309
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_rw
>
> It's required to add transactional support to
> {code:java}
> org.apache.ignite.internal.table.InternalTable#scan {code}
> binding it to
> {code:java}
> org.apache.ignite.internal.storage.MvPartitionStorage#scan(java.util.function.Predicate,
>  java.util.UUID) {code}
> along with acquiring corresponding locks, namely S_commit(table) - if a 
> predicate can produce phantom reads, IS_commit(table) - otherwise.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-17220) YCSB benchmark run for ignite2 vs ignite3

2022-07-06 Thread Alexander Belyak (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Belyak reassigned IGNITE-17220:
-

Assignee: Alexander Belyak

> YCSB benchmark run for ignite2 vs ignite3
> -
>
> Key: IGNITE-17220
> URL: https://issues.apache.org/jira/browse/IGNITE-17220
> Project: Ignite
>  Issue Type: Task
>Reporter: Kirill Gusakov
>Assignee: Alexander Belyak
>Priority: Major
>  Labels: ignite-3
>
> For further investigation of ignite3 performance issues, we need to run the 
> following benchmarks to compare ignite2 vs ignite3 performance:
>  * Usual ycsb benchmark with mixed load patterns
>  * Insert-only ycsb benchmark
> For ignite2 and ignite3 in the following configurations:
>  * 3 ignite nodes setup (so, table must have 1 partition and 3 replicas)
>  * 1 ignite node setup (so, table must have 1 partitoin and 1 replica)
> Also, please provide:
>  * Hardware configuration of the environment, where benchmark was executed
>  * JFRs for every node in every run.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (IGNITE-17315) [Extensions] Fix topology-validator-ext compilation

2022-07-06 Thread Amelchev Nikita (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amelchev Nikita resolved IGNITE-17315.
--
Resolution: Fixed

> [Extensions] Fix topology-validator-ext compilation
> ---
>
> Key: IGNITE-17315
> URL: https://issues.apache.org/jira/browse/IGNITE-17315
> Project: Ignite
>  Issue Type: Bug
>  Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> {noformat}
> 05:05:05   [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile 
> (default-testCompile) on project ignite-topology-validator-ext: Compilation 
> failure: Compilation failure:
> 05:05:05   [ERROR] 
> /opt/buildagent/work/9319dd66c384518/modules/topology-validator-ext/src/test/java/org/apache/ignite/plugin/cache/IgniteCacheTopologyValidatorTest.java:[43,53]
>  cannot find symbol
> 05:05:05   [ERROR]   symbol:   class RunnableX
> 05:05:05   [ERROR]   location: class 
> org.apache.ignite.testframework.GridTestUtils
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-17315) [Extensions] Fix topology-validator-ext compilation

2022-07-06 Thread Amelchev Nikita (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563158#comment-17563158
 ] 

Amelchev Nikita commented on IGNITE-17315:
--

Merged into the master.

> [Extensions] Fix topology-validator-ext compilation
> ---
>
> Key: IGNITE-17315
> URL: https://issues.apache.org/jira/browse/IGNITE-17315
> Project: Ignite
>  Issue Type: Bug
>  Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> {noformat}
> 05:05:05   [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile 
> (default-testCompile) on project ignite-topology-validator-ext: Compilation 
> failure: Compilation failure:
> 05:05:05   [ERROR] 
> /opt/buildagent/work/9319dd66c384518/modules/topology-validator-ext/src/test/java/org/apache/ignite/plugin/cache/IgniteCacheTopologyValidatorTest.java:[43,53]
>  cannot find symbol
> 05:05:05   [ERROR]   symbol:   class RunnableX
> 05:05:05   [ERROR]   location: class 
> org.apache.ignite.testframework.GridTestUtils
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-17315) [Extensions] Fix topology-validator-ext compilation

2022-07-06 Thread Amelchev Nikita (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563157#comment-17563157
 ] 

Amelchev Nikita commented on IGNITE-17315:
--

https://ci.ignite.apache.org/buildConfiguration/IgniteExtensions_Tests_TopologyValidator/6669220

> [Extensions] Fix topology-validator-ext compilation
> ---
>
> Key: IGNITE-17315
> URL: https://issues.apache.org/jira/browse/IGNITE-17315
> Project: Ignite
>  Issue Type: Bug
>  Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> {noformat}
> 05:05:05   [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile 
> (default-testCompile) on project ignite-topology-validator-ext: Compilation 
> failure: Compilation failure:
> 05:05:05   [ERROR] 
> /opt/buildagent/work/9319dd66c384518/modules/topology-validator-ext/src/test/java/org/apache/ignite/plugin/cache/IgniteCacheTopologyValidatorTest.java:[43,53]
>  cannot find symbol
> 05:05:05   [ERROR]   symbol:   class RunnableX
> 05:05:05   [ERROR]   location: class 
> org.apache.ignite.testframework.GridTestUtils
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17316) Thin client pluggable affnity function to use partition awareness

2022-07-06 Thread Maxim Muzafarov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maxim Muzafarov updated IGNITE-17316:
-
Description: 
We should provie a pluggable affinity for the thin client, thus using the 
partition awareness features become possible on only for 
{{RendezvousAffinityFunction}}.

Dev-list discussion:
https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh

  was:We should provie a pluggable affinity for the thin client, thus using the 
partition awareness features become possible on only for 
{{RendezvousAffinityFunction}}.


> Thin client pluggable affnity function to use partition awareness
> -
>
> Key: IGNITE-17316
> URL: https://issues.apache.org/jira/browse/IGNITE-17316
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.14
>
>
> We should provie a pluggable affinity for the thin client, thus using the 
> partition awareness features become possible on only for 
> {{RendezvousAffinityFunction}}.
> Dev-list discussion:
> https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17316) Thin client pluggable affnity function to use partition awareness

2022-07-06 Thread Maxim Muzafarov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maxim Muzafarov updated IGNITE-17316:
-
Component/s: thin client

> Thin client pluggable affnity function to use partition awareness
> -
>
> Key: IGNITE-17316
> URL: https://issues.apache.org/jira/browse/IGNITE-17316
> Project: Ignite
>  Issue Type: Improvement
>  Components: thin client
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.14
>
>
> We should provie a pluggable affinity for the thin client, thus using the 
> partition awareness features become possible on only for 
> {{RendezvousAffinityFunction}}.
> Dev-list discussion:
> https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17316) Thin client pluggable affnity function to use partition awareness

2022-07-06 Thread Maxim Muzafarov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maxim Muzafarov updated IGNITE-17316:
-
Affects Version/s: 2.13

> Thin client pluggable affnity function to use partition awareness
> -
>
> Key: IGNITE-17316
> URL: https://issues.apache.org/jira/browse/IGNITE-17316
> Project: Ignite
>  Issue Type: Improvement
>  Components: thin client
>Affects Versions: 2.13
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.14
>
>
> We should provie a pluggable affinity for the thin client, thus using the 
> partition awareness features become possible on only for 
> {{RendezvousAffinityFunction}}.
> Dev-list discussion:
> https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-17316) Thin client pluggable affnity function to use partition awareness

2022-07-06 Thread Maxim Muzafarov (Jira)
Maxim Muzafarov created IGNITE-17316:


 Summary: Thin client pluggable affnity function to use partition 
awareness
 Key: IGNITE-17316
 URL: https://issues.apache.org/jira/browse/IGNITE-17316
 Project: Ignite
  Issue Type: Improvement
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
 Fix For: 2.14


We should provie a pluggable affinity for the thin client, thus using the 
partition awareness features become possible on only for 
{{RendezvousAffinityFunction}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17081) Implement checkpointIndex for RocksDB

2022-07-06 Thread Ivan Bessonov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-17081:
---
Description: 
Please refer to https://issues.apache.org/jira/browse/IGNITE-16907 for 
prerequisites.

Please also familiarize yourself with 
https://issues.apache.org/jira/browse/IGNITE-17077 for better understanding, 
the description is continued from there.

For RocksDB based storage the recovery process is trivial, because RocksDB has 
its own WAL. So, for testing purposes, it would be enough to just store update 
index in meta column family.

Immediately we have a write amplification issue, on top of possible performance 
degradation. Obvious solution is inherently bad and needs to be improved.
h2. General idea & implementation

Obviously, WAL needs to be disabled (WriteOptions#setDisableWAL). This kinda 
breaks RocksDB recovery procedure, we need to take measures to avoid it.

The only feasible way to do so is to use DBOptions#setAtomicFlush in 
conjunction with org.rocksdb.WriteBatchWithIndex. This allows RocksDB to save 
all column families consistently, if you have batches that cover several CFs. 
Basically, {{acquireConsistencyLock()}} would create a thread-local write 
batch, that's applied on locks release. Most of RocksDbMvPartitionStorage will 
be affected by this change.

NOTE: I believe that scans with unapplied batches should be prohibited for now  
(gladly, there's a WriteBatchInterface#count() to check). I don't see any 
practical value and a proper way of implementing it, considering how spread-out 
in time the scan process is.
h2. Callbacks and RAFT snapshots

Simply storing and reading update index is easy. Reading committed index is 
more challenging, I propose caching it and update only from the closure, that 
can also be used by RAFT to truncate the log.

For a closure, there are several things to account for during the 
implementation:
 * DBOptions#setListeners. We need two events - ON_FLUSH_BEGIN and 
ON_FLUSH_COMPLETED. All "completed" events go after all "begin" events in 
atomic flush mode. And, once you have your first "completed" event ,you have a 
guarantee that *all* memtables are already persisted.
This allows easy tracking of RocksDB flushes, monitoring events alteration is 
all that's needed.
 * Unlike PDS implementation, here we will be writing updateIndex value into a 
memtable every time. This makes it harder to find persistedIndex values for 
partitions. Gladly, considering the events that we have, during the time 
between first "completed" and the very next "begin", the state on disk is fully 
consistent. And there's a way to read data from storage avoiding memtable 
completely - ReadOptions#setReadTier(PERSISTED_TIER).

Summarizing everything from the above, we should implement following protocol:

 
{code:java}
During table start: read latest values of update indexes. Store them in an 
in-memory structure.
Set "lastEventType = ON_FLUSH_COMPLETED;".

onFlushBegin:
  if (lastEventType == ON_FLUSH_BEGIN)
return;

  waitForLastAsyncUpdateIndexesRead();

  lastEventType = ON_FLUSH_BEGIN;

onFlushCompleted:
  if (lastEventType == ON_FLUSH_COMPLETED)
return;

  asyncReadUpdateIndexesFromDisk();

  lastEventType = ON_FLUSH_COMPLETED;{code}
Reading values from disk must be performed asynchronously to not stall flushing 
process. We don't control locks that RocksDb holds while calling listener's 
methods.

That asynchronous process would invoke closures that provide presisted 
updateIndex values to other components.

NODE: One might say that we should call "waitForLastAsyncUpdateIndexesRead();" 
as late as possible just in case. But my implementation says calling it during 
the first event. This is fine. I noticed that column families are flushed in 
order of their internal ids. These ids correspond to a sequence number of CFs, 
and the "default" CF is always created first. This is the exact CF that we use 
to store meta. Maybe we're going to change this and create a separate meta CF. 
Only then we could start optimizing this part, and only if we'll have an actual 
proof that there's a stall in this exact place.
h3. Types of storages

RocksDB is used for:
 * tables
 * cluster management
 * meta-storage

All these types should use the same recovery procedure, but code is located in 
different places. I hope that it won't be a big problem and we can do 
everything at once.

  was:
Please refer to https://issues.apache.org/jira/browse/IGNITE-16907 for 
prerequisites.

Please also familiarize yourself with 
https://issues.apache.org/jira/browse/IGNITE-17077 for better understanding, 
the description is continued from there.

For RocksDB based storage the recovery process is trivial, because RocksDB has 
its own WAL. So, for testing purposes, it would be enough to just store update 
index in meta column family.

Immediately we have a write amplification issue, on top 

[jira] [Created] (IGNITE-17315) [Extensions] Fix topology-validator-ext compilation

2022-07-06 Thread Amelchev Nikita (Jira)
Amelchev Nikita created IGNITE-17315:


 Summary: [Extensions] Fix topology-validator-ext compilation
 Key: IGNITE-17315
 URL: https://issues.apache.org/jira/browse/IGNITE-17315
 Project: Ignite
  Issue Type: Bug
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita


{noformat}
05:05:05   [ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile 
(default-testCompile) on project ignite-topology-validator-ext: Compilation 
failure: Compilation failure:
05:05:05   [ERROR] 
/opt/buildagent/work/9319dd66c384518/modules/topology-validator-ext/src/test/java/org/apache/ignite/plugin/cache/IgniteCacheTopologyValidatorTest.java:[43,53]
 cannot find symbol
05:05:05   [ERROR]   symbol:   class RunnableX
05:05:05   [ERROR]   location: class 
org.apache.ignite.testframework.GridTestUtils
{noformat}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17315) [Extensions] Fix topology-validator-ext compilation

2022-07-06 Thread Amelchev Nikita (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amelchev Nikita updated IGNITE-17315:
-
Component/s: extensions

> [Extensions] Fix topology-validator-ext compilation
> ---
>
> Key: IGNITE-17315
> URL: https://issues.apache.org/jira/browse/IGNITE-17315
> Project: Ignite
>  Issue Type: Bug
>  Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> {noformat}
> 05:05:05   [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile 
> (default-testCompile) on project ignite-topology-validator-ext: Compilation 
> failure: Compilation failure:
> 05:05:05   [ERROR] 
> /opt/buildagent/work/9319dd66c384518/modules/topology-validator-ext/src/test/java/org/apache/ignite/plugin/cache/IgniteCacheTopologyValidatorTest.java:[43,53]
>  cannot find symbol
> 05:05:05   [ERROR]   symbol:   class RunnableX
> 05:05:05   [ERROR]   location: class 
> org.apache.ignite.testframework.GridTestUtils
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17314) .NET: Thin 3.0: Unified exception handling

2022-07-06 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-17314:

Description: Implement a set of public exception types according to 
IGNITE-17312.

> .NET: Thin 3.0: Unified exception handling
> --
>
> Key: IGNITE-17314
> URL: https://issues.apache.org/jira/browse/IGNITE-17314
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-alpha6
>
>
> Implement a set of public exception types according to IGNITE-17312.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-17314) .NET: Thin 3.0: Unified exception handling

2022-07-06 Thread Pavel Tupitsyn (Jira)
Pavel Tupitsyn created IGNITE-17314:
---

 Summary: .NET: Thin 3.0: Unified exception handling
 Key: IGNITE-17314
 URL: https://issues.apache.org/jira/browse/IGNITE-17314
 Project: Ignite
  Issue Type: Improvement
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 3.0.0-alpha6






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17312) Thin 3.0: Unified exception handling

2022-07-06 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-17312:

Description: 
Java thin client uses the same public API interfaces as the server side.
Those user-facing APIs should behave in the same way on the server and client 
sides, including exceptions that may be thrown.

# Remove *IgniteClientException*. 
# Keep *IgniteClientConnectionException* and other client-specific types. 
Inherit them from *IgniteException*.
# When sending exception details from server to client, include the following:
## Code (group  + error, see IgniteException.code)
## TraceId
## Message
## Fully-qualified class name
## OPTIONALLY: stack trace (Sensitive info. Send only when enabled in 
server-side ClientConnectorConfig. Disabled by default.)

  was:
Java thin client uses the same public API interfaces as the server side.
Those user-facing APIs should behave in the same way on the server and client 
sides, including exceptions that may be thrown.

# Remove *IgniteClientException*. 
# Keep *IgniteClientConnectionException* and other client-specific types. 
Inherit them from *IgniteException*.
# When sending exception details from server to client, include the following:
## Code (group  + error, see IgniteException.code)
## TraceId
## Message
## Fully-qualified class name
## OPTIONALLY: stack trace (when enabled in server-side ClientConnectorConfig. 
Disabled by default.)


> Thin 3.0: Unified exception handling
> 
>
> Key: IGNITE-17312
> URL: https://issues.apache.org/jira/browse/IGNITE-17312
> Project: Ignite
>  Issue Type: Improvement
>  Components: thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> Java thin client uses the same public API interfaces as the server side.
> Those user-facing APIs should behave in the same way on the server and client 
> sides, including exceptions that may be thrown.
> # Remove *IgniteClientException*. 
> # Keep *IgniteClientConnectionException* and other client-specific types. 
> Inherit them from *IgniteException*.
> # When sending exception details from server to client, include the following:
> ## Code (group  + error, see IgniteException.code)
> ## TraceId
> ## Message
> ## Fully-qualified class name
> ## OPTIONALLY: stack trace (Sensitive info. Send only when enabled in 
> server-side ClientConnectorConfig. Disabled by default.)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17312) Thin 3.0: Unified exception handling

2022-07-06 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-17312:

Description: 
Java thin client uses the same public API interfaces as the server side.
Those user-facing APIs should behave in the same way on the server and client 
sides, including exceptions that may be thrown.

# Remove *IgniteClientException*. 
# Keep *IgniteClientConnectionException* and other client-specific types. 
Inherit them from *IgniteException*.
# When sending exception details from server to client, include the following:
## Code (group  + error, see IgniteException.code)
## TraceId
## Message
## Fully-qualified class name
## OPTIONALLY: stack trace (when enabled in server-side ClientConnectorConfig. 
Disabled by default.)

  was:
Java thin client uses the same public API interfaces as the server side.
Those user-facing APIs should behave in the same way on the server and client 
sides, including exceptions that may be thrown.


> Thin 3.0: Unified exception handling
> 
>
> Key: IGNITE-17312
> URL: https://issues.apache.org/jira/browse/IGNITE-17312
> Project: Ignite
>  Issue Type: Improvement
>  Components: thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> Java thin client uses the same public API interfaces as the server side.
> Those user-facing APIs should behave in the same way on the server and client 
> sides, including exceptions that may be thrown.
> # Remove *IgniteClientException*. 
> # Keep *IgniteClientConnectionException* and other client-specific types. 
> Inherit them from *IgniteException*.
> # When sending exception details from server to client, include the following:
> ## Code (group  + error, see IgniteException.code)
> ## TraceId
> ## Message
> ## Fully-qualified class name
> ## OPTIONALLY: stack trace (when enabled in server-side 
> ClientConnectorConfig. Disabled by default.)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17312) Thin 3.0: Unified exception handling

2022-07-06 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-17312:

Description: 
Java thin client uses the same public API interfaces as the server side.
Those user-facing APIs should behave in the same way on the server and client 
sides, including exceptions that may be thrown.

  was:TODO


> Thin 3.0: Unified exception handling
> 
>
> Key: IGNITE-17312
> URL: https://issues.apache.org/jira/browse/IGNITE-17312
> Project: Ignite
>  Issue Type: Improvement
>  Components: thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> Java thin client uses the same public API interfaces as the server side.
> Those user-facing APIs should behave in the same way on the server and client 
> sides, including exceptions that may be thrown.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Deleted] (IGNITE-17313) Thin 3.0: Unified exception handling

2022-07-06 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn deleted IGNITE-17313:



> Thin 3.0: Unified exception handling
> 
>
> Key: IGNITE-17313
> URL: https://issues.apache.org/jira/browse/IGNITE-17313
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: ignite-3
>
> TODO



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-17313) Thin 3.0: Unified exception handling

2022-07-06 Thread Pavel Tupitsyn (Jira)
Pavel Tupitsyn created IGNITE-17313:
---

 Summary: Thin 3.0: Unified exception handling
 Key: IGNITE-17313
 URL: https://issues.apache.org/jira/browse/IGNITE-17313
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 3.0.0-alpha6


TODO



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-17312) Thin 3.0: Unified exception handling

2022-07-06 Thread Pavel Tupitsyn (Jira)
Pavel Tupitsyn created IGNITE-17312:
---

 Summary: Thin 3.0: Unified exception handling
 Key: IGNITE-17312
 URL: https://issues.apache.org/jira/browse/IGNITE-17312
 Project: Ignite
  Issue Type: Improvement
  Components: thin client
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 3.0.0-alpha6


TODO



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17311) KafkaToIgniteLoader instantiates several Spring contexts

2022-07-06 Thread Amelchev Nikita (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amelchev Nikita updated IGNITE-17311:
-
Component/s: extensions

> KafkaToIgniteLoader instantiates several Spring contexts
> 
>
> Key: IGNITE-17311
> URL: https://issues.apache.org/jira/browse/IGNITE-17311
> Project: Ignite
>  Issue Type: Improvement
>  Components: extensions
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
>
> KafkaToIgniteLoader instantiates two SpringApplicationContext during app 
> start.
> This can lead to error in case other beans inside context not designed for 
> this case.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-17311) KafkaToIgniteLoader instantiates several Spring contexts

2022-07-06 Thread Amelchev Nikita (Jira)
Amelchev Nikita created IGNITE-17311:


 Summary: KafkaToIgniteLoader instantiates several Spring contexts
 Key: IGNITE-17311
 URL: https://issues.apache.org/jira/browse/IGNITE-17311
 Project: Ignite
  Issue Type: Improvement
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita


KafkaToIgniteLoader instantiates two SpringApplicationContext during app start.
This can lead to error in case other beans inside context not designed for this 
case.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-17032) Apache Ignite Docker container does not run correctly if image is run in read only file system mode

2022-07-06 Thread Stanislav Lukyanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563063#comment-17563063
 ] 

Stanislav Lukyanov edited comment on IGNITE-17032 at 7/6/22 8:57 AM:
-

Thanks for reporting [~tonkovic].

TLDR: this is fixable, and I think there is a workaround - see it at the end of 
the comment.

My first read of this was - well, Ignite should normally be used with a 
writable disk, so "Not an issue".

BUT reading the second time I realized that the problem is that we're trying to 
write into the *root* volume. We shouldn't do that.

 

Apparently, the thing that doesn't work is the following line

{code}
version=$(awk -F[\"\.] '{print $2}' <<< ${version})
{code}

In particular, the `<<<` is at fault.

Turns out, in Bash the `<<<` ("here-string") is implemented via `<<` 
("here-document") which is in turn implemented via temp files. The root file 
system is read-only, hence `/tmp` is also read-only, hence `<<` and `<<<` can't 
work. You learn something new everyday, don't you? 
[This](https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash)
 explains it.

I'm quite surprised that basic Bash syntax can be unfriendly to cloud but here 
we are.

Two things we should do (each of them will suffice, but I'd do both):

1. Get rid of the `<<<` and `<<` in the scripts, just in case.

2. Mount `/tmp` as tmpfs. It should always be tmpfs in all environments, for 
many good reasons. I'm a bit surprised it's not tmpfs now, and I'd certainly 
change that. The question is whether we can and should do this right in the 
Dockerfile, or ask everyone to mount `/tmp` as tmpfs when running. The latter 
requires at least docs changes.

As a workaround, I think running the container with an [explicit tmpfs 
mount](https://stackoverflow.com/questions/34698620/docker-and-volatile-volumes-ala-tmp)
 will do the trick:
{code}
docker run --tmpfs /tmp ...
{code}

Disclaimer: I haven't got my hands dirty yet, so all of the above needs to be 
confirmed with tests.


was (Author: slukyanov):
Thanks for reporting [~tonkovic].

TLDR: this is fixable, and I think there is a workaround - see it at the end of 
the comment.

My first read of this was - well, Ignite should normally be used with a 
writable disk, so "Not an issue".

BUT reading the second time I realized that the problem is that we're trying to 
write into the *root* volume. We shouldn't do that.

 

Apparently, the thing that doesn't work is the following line

```
version=$(awk -F[\"\.] '{print $2}' <<< ${version})
```

In particular, the `<<<` is at fault.

Turns out, in Bash the `<<<` ("here-string") is implemented via `<<` 
("here-document") which is in turn implemented via temp files. The root file 
system is read-only, hence `/tmp` is also read-only, hence `<<` and `<<<` can't 
work. You learn something new everyday, don't you? 
[This](https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash)
 explains it.

I'm quite surprised that basic Bash syntax can be unfriendly to cloud but here 
we are.

Two things we should do (each of them will suffice, but I'd do both):

1. Get rid of the `<<<` and `<<` in the scripts, just in case.

2. Mount `/tmp` as tmpfs. It should always be tmpfs in all environments, for 
many good reasons. I'm a bit surprised it's not tmpfs now, and I'd certainly 
change that. The question is whether we can and should do this right in the 
Dockerfile, or ask everyone to mount `/tmp` as tmpfs when running. The latter 
requires at least docs changes.

As a workaround, I think running the container with an [explicit tmpfs 
mount](https://stackoverflow.com/questions/34698620/docker-and-volatile-volumes-ala-tmp)
 will do the trick:
```
docker run --tmpfs /tmp ...
```

Disclaimer: I haven't got my hands dirty yet, so all of the above needs to be 
confirmed with tests.

> Apache Ignite Docker container does not run correctly if image is run in read 
> only file system mode
> ---
>
> Key: IGNITE-17032
> URL: https://issues.apache.org/jira/browse/IGNITE-17032
> Project: Ignite
>  Issue Type: Bug
>  Components: build
>Affects Versions: 2.13
>Reporter: Petar Tonkovic
>Priority: Major
>
> When following the Kubernetes deployment tutorials (online: 
> https://ignite.apache.org/docs/latest/installation/kubernetes/azure-deployment,
>  youtube: [https://youtu.be/38YgdAOs038]), trying to run the official docker 
> image () with the --read-only flag is causing errors:
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 52: cannot create 
> temp file for here-document: Read-only file system
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 85: [: -lt: unary 
> operator expected2022-05-25T14:27:34.50

[jira] [Comment Edited] (IGNITE-17032) Apache Ignite Docker container does not run correctly if image is run in read only file system mode

2022-07-06 Thread Stanislav Lukyanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563063#comment-17563063
 ] 

Stanislav Lukyanov edited comment on IGNITE-17032 at 7/6/22 8:56 AM:
-

Thanks for reporting [~tonkovic].

TLDR: this is fixable, and I think there is a workaround - see it at the end of 
the comment.

My first read of this was - well, Ignite should normally be used with a 
writable disk, so "Not an issue".

BUT reading the second time I realized that the problem is that we're trying to 
write into the *root* volume. We shouldn't do that.

 

Apparently, the thing that doesn't work is the following line

```
version=$(awk -F[\"\.] '{print $2}' <<< ${version})
```

In particular, the `<<<` is at fault.

Turns out, in Bash the `<<<` ("here-string") is implemented via `<<` 
("here-document") which is in turn implemented via temp files. The root file 
system is read-only, hence `/tmp` is also read-only, hence `<<` and `<<<` can't 
work. You learn something new everyday, don't you? 
[This](https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash)
 explains it.

I'm quite surprised that basic Bash syntax can be unfriendly to cloud but here 
we are.

Two things we should do (each of them will suffice, but I'd do both):

1. Get rid of the `<<<` and `<<` in the scripts, just in case.

2. Mount `/tmp` as tmpfs. It should always be tmpfs in all environments, for 
many good reasons. I'm a bit surprised it's not tmpfs now, and I'd certainly 
change that. The question is whether we can and should do this right in the 
Dockerfile, or ask everyone to mount `/tmp` as tmpfs when running. The latter 
requires at least docs changes.

As a workaround, I think running the container with an [explicit tmpfs 
mount](https://stackoverflow.com/questions/34698620/docker-and-volatile-volumes-ala-tmp)
 will do the trick:
```
docker run --tmpfs /tmp ...
```

Disclaimer: I haven't got my hands dirty yet, so all of the above needs to be 
confirmed with tests.


was (Author: slukyanov):
Thanks for reporting [~tonkovic].

TLDR: this is fixable, and I think there is a workaround - see it at the end of 
the comment.

My first read of this was - well, Ignite should normally be used with a 
writable disk, so "Not an issue".

BUT reading the second time I realized that the problem is that we're trying to 
write into the *root* volume. We shouldn't do that.

 

Apparently, the thing that doesn't work is the following line
```
version=$(awk -F[\"\.] '{print $2}' <<< ${version})
```
In particular, the `<<<` is at fault.

Turns out, in Bash the `<<<` ("here-string") is implemented via `<<` 
("here-document") which is in turn implemented via temp files. The root file 
system is read-only, hence `/tmp` is also read-only, hence `<<` and `<<<` can't 
work. You learn something new everyday, don't you? 
[This](https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash)
 explains it.

I'm quite surprised that basic Bash syntax can be unfriendly to cloud but here 
we are.

Two things we should do (each of them will suffice, but I'd do both):

1. Get rid of the `<<<` and `<<` in the scripts, just in case.

2. Mount `/tmp` as tmpfs. It should always be tmpfs in all environments, for 
many good reasons. I'm a bit surprised it's not tmpfs now, and I'd certainly 
change that. The question is whether we can and should do this right in the 
Dockerfile, or ask everyone to mount `/tmp` as tmpfs when running. The latter 
requires at least docs changes.

As a workaround, I think running the container with an [explicit tmpfs 
mount](https://stackoverflow.com/questions/34698620/docker-and-volatile-volumes-ala-tmp)
 will do the trick:
```
docker run --tmpfs /tmp ...
```

Disclaimer: I haven't got my hands dirty yet, so all of the above needs to be 
confirmed with tests.

> Apache Ignite Docker container does not run correctly if image is run in read 
> only file system mode
> ---
>
> Key: IGNITE-17032
> URL: https://issues.apache.org/jira/browse/IGNITE-17032
> Project: Ignite
>  Issue Type: Bug
>  Components: build
>Affects Versions: 2.13
>Reporter: Petar Tonkovic
>Priority: Major
>
> When following the Kubernetes deployment tutorials (online: 
> https://ignite.apache.org/docs/latest/installation/kubernetes/azure-deployment,
>  youtube: [https://youtu.be/38YgdAOs038]), trying to run the official docker 
> image () with the --read-only flag is causing errors:
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 52: cannot create 
> temp file for here-document: Read-only file system
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 85: [: -lt: unary 
> operator expected2022-05-25T14:27:34.504369604+02:00

[jira] [Commented] (IGNITE-17032) Apache Ignite Docker container does not run correctly if image is run in read only file system mode

2022-07-06 Thread Stanislav Lukyanov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563063#comment-17563063
 ] 

Stanislav Lukyanov commented on IGNITE-17032:
-

Thanks for reporting [~tonkovic].

TLDR: this is fixable, and I think there is a workaround - see it at the end of 
the comment.

My first read of this was - well, Ignite should normally be used with a 
writable disk, so "Not an issue".

BUT reading the second time I realized that the problem is that we're trying to 
write into the *root* volume. We shouldn't do that.

 

Apparently, the thing that doesn't work is the following line
```
version=$(awk -F[\"\.] '{print $2}' <<< ${version})
```
In particular, the `<<<` is at fault.

Turns out, in Bash the `<<<` ("here-string") is implemented via `<<` 
("here-document") which is in turn implemented via temp files. The root file 
system is read-only, hence `/tmp` is also read-only, hence `<<` and `<<<` can't 
work. You learn something new everyday, don't you? 
[This](https://stackoverflow.com/questions/2500436/how-does-cat-eof-work-in-bash)
 explains it.

I'm quite surprised that basic Bash syntax can be unfriendly to cloud but here 
we are.

Two things we should do (each of them will suffice, but I'd do both):

1. Get rid of the `<<<` and `<<` in the scripts, just in case.

2. Mount `/tmp` as tmpfs. It should always be tmpfs in all environments, for 
many good reasons. I'm a bit surprised it's not tmpfs now, and I'd certainly 
change that. The question is whether we can and should do this right in the 
Dockerfile, or ask everyone to mount `/tmp` as tmpfs when running. The latter 
requires at least docs changes.

As a workaround, I think running the container with an [explicit tmpfs 
mount](https://stackoverflow.com/questions/34698620/docker-and-volatile-volumes-ala-tmp)
 will do the trick:
```
docker run --tmpfs /tmp ...
```

Disclaimer: I haven't got my hands dirty yet, so all of the above needs to be 
confirmed with tests.

> Apache Ignite Docker container does not run correctly if image is run in read 
> only file system mode
> ---
>
> Key: IGNITE-17032
> URL: https://issues.apache.org/jira/browse/IGNITE-17032
> Project: Ignite
>  Issue Type: Bug
>  Components: build
>Affects Versions: 2.13
>Reporter: Petar Tonkovic
>Priority: Major
>
> When following the Kubernetes deployment tutorials (online: 
> https://ignite.apache.org/docs/latest/installation/kubernetes/azure-deployment,
>  youtube: [https://youtu.be/38YgdAOs038]), trying to run the official docker 
> image () with the --read-only flag is causing errors:
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 52: cannot create 
> temp file for here-document: Read-only file system
> /opt/ignite/apache-ignite/bin/include/functions.sh: line 85: [: -lt: unary 
> operator expected2022-05-25T14:27:34.504369604+02:00
>  
> Since most managed company Kubernetes clusters enforce this read-only flag as 
> a security requirement, it would be good to look into these errors.
>  
> Later on, we get the following error on starting up:
> class org.apache.ignite.IgniteException: Failed to instantiate Spring XML 
> application context (make sure all classes used in Spring configuration are 
> present at CLASSPATH) [springUrl=file:/ignite/config/node-configuration.xml]
> at 
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:1135)
> at org.apache.ignite.Ignition.start(Ignition.java:356)
> at 
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:365)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to 
> instantiate Spring XML application context (make sure all classes used in 
> Spring configuration are present at CLASSPATH) 
> [springUrl=file:/ignite/config/node-configuration.xml]
> at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.applicationContext(IgniteSpringHelperImpl.java:364)
> at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:102)
> at 
> org.apache.ignite.internal.util.spring.IgniteSpringHelperImpl.loadConfigurations(IgniteSpringHelperImpl.java:96)
> at 
> org.apache.ignite.internal.IgnitionEx.loadConfigurations(IgnitionEx.java:729)2022-05-25T14:27:34.916588365+02:00
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:930)
> at 
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:839)2022-05-25T14:27:34.916609431+02:00
> at 
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:709)2022-05-25T14:27:34.916622089+02:00
> at 
> org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:678)2022-05-25T14:27:34.916636146+02:00
> at 
> org.apache.ignite.Ignition.start(Ignition

[jira] [Commented] (IGNITE-17281) Return error groups from REST API

2022-07-06 Thread Aleksandr (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563050#comment-17563050
 ] 

Aleksandr commented on IGNITE-17281:


Hi [~slava.koptilin], looks good to me, thanks.

> Return error groups from REST API
> -
>
> Key: IGNITE-17281
> URL: https://issues.apache.org/jira/browse/IGNITE-17281
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Aleksandr
>Assignee: Vyacheslav Koptilin
>Priority: Major
>  Labels: iep-87, ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> After IGNITE-14931 and IGNITE-17193 are done, traceId and human-readable 
> error code should be returned from any rest component.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17215) Write ClusterSnapshotRecord to WAL

2022-07-06 Thread Maksim Timonin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Timonin updated IGNITE-17215:

Description: 
For PITR [1] process of recovering based on ClusterSnapshot + archived WALs.

It's required to have a point in WAL which splits whole WAL on 2 areas:
 # Before this point all data changes are contained within ClusterSnapshot, and 
no need to recover them from WAL archived files.
 # After this point all data need to be recovered from WAL archived files.

It's proposed to write ClusterSnapshotRecord while the checkpoint is running 
(cp#writeLock has acquired). ClusterSnapshot process guarantees:
 # there is no active transactions (or any data changes) in moment of running 
checkpoint.
 # ClusterSnapshot contains all data pages that will be persisted within this 
checkpoint process.

Then every logical record after begin CheckointRecord doesn't belong to 
ClusterSnapshot. Then it's safe to write ClusterSnapshotRecord within the 
checkpoint process.

[1] [https://cwiki.apache.org/confluence/pages/editpage.action?pageId=211884314]

  was:
For PITR [1] process of recovering based on ClusterSnapshot + archived WALs.

It's required to have a point in WAL which splits whole WAL on 2 areas:
 # Before this point all data changes are contained within ClusterSnapshot, and 
no need to recover them from WAL archived files.
 # After this point all data need to be recovered from WAL archived files.

It's proposed to write ClusterSnapshotRecord at the moment checkpoint process 
starts (cp#writeLock has acquired). ClusterSnapshot process guarantees:
 # there is no active transactions (or any data changes) in moment of writing 
begin CheckpointRecord.
 # ClusterSnapshot consist of data pages that are materialized within this 
checkpoint process.

Then every logical record after begin CheckointRecord doesn't belong to 
ClusterSnapshot. Then it's safe to write ClusterSnapshotRecord align with 
CheckpointRecord.

 

[1] [https://cwiki.apache.org/confluence/pages/editpage.action?pageId=211884314]


> Write ClusterSnapshotRecord to WAL
> --
>
> Key: IGNITE-17215
> URL: https://issues.apache.org/jira/browse/IGNITE-17215
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Maksim Timonin
>Assignee: Maksim Timonin
>Priority: Major
>  Labels: IEP-89
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> For PITR [1] process of recovering based on ClusterSnapshot + archived WALs.
> It's required to have a point in WAL which splits whole WAL on 2 areas:
>  # Before this point all data changes are contained within ClusterSnapshot, 
> and no need to recover them from WAL archived files.
>  # After this point all data need to be recovered from WAL archived files.
> It's proposed to write ClusterSnapshotRecord while the checkpoint is running 
> (cp#writeLock has acquired). ClusterSnapshot process guarantees:
>  # there is no active transactions (or any data changes) in moment of running 
> checkpoint.
>  # ClusterSnapshot contains all data pages that will be persisted within this 
> checkpoint process.
> Then every logical record after begin CheckointRecord doesn't belong to 
> ClusterSnapshot. Then it's safe to write ClusterSnapshotRecord within the 
> checkpoint process.
> [1] 
> [https://cwiki.apache.org/confluence/pages/editpage.action?pageId=211884314]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17203) SQL API: Implement scale and precision metadata

2022-07-06 Thread Konstantin Orlov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Orlov updated IGNITE-17203:
--
Ignite Flags:   (was: Docs Required,Release Notes Required)

> SQL API: Implement scale and precision metadata
> ---
>
> Key: IGNITE-17203
> URL: https://issues.apache.org/jira/browse/IGNITE-17203
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Pavel Tupitsyn
>Assignee: Andrey Mashenkov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> *scale()* and *precision()* in *ColumnMetadata* interface return incorrect 
> values. Implement them properly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-17203) SQL API: Implement scale and precision metadata

2022-07-06 Thread Konstantin Orlov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563035#comment-17563035
 ] 

Konstantin Orlov commented on IGNITE-17203:
---

[~amashenkov], I've left a few comments. Please see PR

> SQL API: Implement scale and precision metadata
> ---
>
> Key: IGNITE-17203
> URL: https://issues.apache.org/jira/browse/IGNITE-17203
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Pavel Tupitsyn
>Assignee: Andrey Mashenkov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> *scale()* and *precision()* in *ColumnMetadata* interface return incorrect 
> values. Implement them properly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-17279) Mapping of partition states to nodes can erroneously skip lost partitions on the coordinator node

2022-07-06 Thread Vladislav Pyatkov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563003#comment-17563003
 ] 

Vladislav Pyatkov commented on IGNITE-17279:


LGTM

> Mapping of partition states to nodes can erroneously skip lost partitions on 
> the coordinator node
> -
>
> Key: IGNITE-17279
> URL: https://issues.apache.org/jira/browse/IGNITE-17279
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Minor
> Fix For: 2.14
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> It seems that a coordinator node does not correctly update node2part mapping 
> for lost partitions. 
> {noformat}
> [test-runner-#1%distributed.CachePartitionLostAfterSupplierHasLeftTest%][root]
>  dump partitions state for :
> preload sync futures
> nodeId=b57ca812-416d-40d7-bb4f-27199490 
> consistentId=distributed.CachePartitionLostAfterSupplierHasLeftTest0 
> isDone=true
> nodeId=20fdfa4a-ddf6-4229-b25e-38cd8d31 
> consistentId=distributed.CachePartitionLostAfterSupplierHasLeftTest1 
> isDone=true
> rebalance futures
> nodeId=b57ca812-416d-40d7-bb4f-27199490 isDone=true res=true topVer=null
> remaining: {}
> nodeId=20fdfa4a-ddf6-4229-b25e-38cd8d31 isDone=true res=false 
> topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0]
> remaining: {}
> partition state
> localNodeId=b57ca812-416d-40d7-bb4f-27199490 
> grid=distributed.CachePartitionLostAfterSupplierHasLeftTest0
> local part=0 counters=Counter [lwm=200, missed=[], maxApplied=200, hwm=200] 
> fullSize=200 *state=LOST* reservations=0 isAffNode=true
>  nodeId=20fdfa4a-ddf6-4229-b25e-38cd8d31 part=0 *state=LOST* 
> isAffNode=true
> ...
> localNodeId=20fdfa4a-ddf6-4229-b25e-38cd8d31 
> grid=distributed.CachePartitionLostAfterSupplierHasLeftTest1
> local part=0 counters=Counter [lwm=0, missed=[], maxApplied=0, hwm=0] 
> fullSize=100 *state=LOST* reservations=0 isAffNode=true
>  nodeId=b57ca812-416d-40d7-bb4f-27199490 part=0 *state=OWNING* 
> isAffNode=true
> ...
> {noformat}
> *Update*:
> The root cause of the issue is that the coordinator node incorrectly 
> update mapping nodes to partition states on PME (see 
> GridDhtPartitionTopologyImpl.node2part). It seems to me, that the coordinator 
> node should set partition state to LOST on all affinity nodes (if this 
> partition is assumed as LOST on the coordinator) before creating and sending 
> a “full map” message.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)