[jira] [Created] (IGNITE-20958) PersistentPageMemoryMvPartitionStorageTest#groupConfigShorteningWorksCorrectly threw an AssertionError

2023-11-24 Thread Aleksandr Polovtcev (Jira)
Aleksandr Polovtcev created IGNITE-20958:


 Summary: 
PersistentPageMemoryMvPartitionStorageTest#groupConfigShorteningWorksCorrectly 
threw an AssertionError
 Key: IGNITE-20958
 URL: https://issues.apache.org/jira/browse/IGNITE-20958
 Project: Ignite
  Issue Type: Bug
Reporter: Aleksandr Polovtcev
Assignee: Kirill Tkalenko


Failed build: 
https://ci.ignite.apache.org/buildConfiguration/ApacheIgnite3xGradle_Test_RunUnitTests/7650826

{noformat}
[2023-11-23T19:08:47,351][WARN ][%test%compaction-thread-425][Compactor] 
Runtime error caught during ignite runnable execution [worker=IgniteWorker 
[name=compaction-thread, igniteInstanceName=test, listener=null, 
finished=false, heartbeatTimestamp=1700766527341, hashCode=1964034487, 
interrupted=false, runner=%test%compaction-thread-425]]
org.apache.ignite.internal.lang.IgniteInternalException: 
java.lang.AssertionError: pageIdx=1, pageCount=0
  at 
org.apache.ignite.internal.pagememory.persistence.compaction.Compactor.body(Compactor.java:140)
 ~[ignite-page-memory-3.0.0-SNAPSHOT.jar:?]
  at 
org.apache.ignite.internal.util.worker.IgniteWorker.run(IgniteWorker.java:108) 
[ignite-core-3.0.0-SNAPSHOT.jar:?]
  at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.util.concurrent.CompletionException: java.lang.AssertionError: 
pageIdx=1, pageCount=0
  at 
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
 ~[?:?]
  at 
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
 ~[?:?]
  at 
java.util.concurrent.CompletableFuture$BiRelay.tryFire(CompletableFuture.java:1423)
 ~[?:?]
  at 
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) 
~[?:?]
  at 
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
 ~[?:?]
  at 
org.apache.ignite.internal.pagememory.persistence.compaction.Compactor.lambda$doCompaction$1(Compactor.java:241)
 ~[ignite-page-memory-3.0.0-SNAPSHOT.jar:?]
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 
~[?:?]
  at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 
~[?:?]
  ... 1 more
Caused by: java.lang.AssertionError: pageIdx=1, pageCount=0
  at 
org.apache.ignite.internal.pagememory.persistence.store.FilePageStore.write(FilePageStore.java:194)
 ~[ignite-page-memory-3.0.0-SNAPSHOT.jar:?]
  at 
org.apache.ignite.internal.pagememory.persistence.compaction.Compactor.mergeDeltaFileToMainFile(Compactor.java:373)
 ~[ignite-page-memory-3.0.0-SNAPSHOT.jar:?]
  at 
org.apache.ignite.internal.pagememory.persistence.compaction.Compactor.lambda$doCompaction$1(Compactor.java:235)
 ~[ignite-page-memory-3.0.0-SNAPSHOT.jar:?]
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 
~[?:?]
  at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 
~[?:?]
  ... 1 more
{noformat}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20867) Server nodes crash after create persistent cache with /" name

2023-11-24 Thread Nikita Amelchev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789378#comment-17789378
 ] 

Nikita Amelchev commented on IGNITE-20867:
--

Fixed a typo in the cache directory name check error message: 
https://github.com/apache/ignite/pull/11068

[~annyakimova], Thank you for the contribution.

Merged into the master and 2.16.

> Server nodes crash after create persistent cache with /" name 
> --
>
> Key: IGNITE-20867
> URL: https://issues.apache.org/jira/browse/IGNITE-20867
> Project: Ignite
>  Issue Type: Bug
>Reporter: Anastasia Iakimova
>Assignee: Anastasia Iakimova
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> ignite.log:
> {code:java}
> 2023-11-14 18:12:34.985 [INFO 
> ][exchange-worker-#75][org.apache.ignite.internal.exchange.time] Started 
> exchange init [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=32], 
> crd=false, evt=DISCOVERY_CUSTOM_EVT, evtNode=d91c8f37
> -dc4f-435e-97dc-8ec02b953b6a, customEvt=DynamicCacheChangeBatch 
> [id=3a05698cb81-782afb33-985f-4efc-a47f-2c178ae29301, reqs=ArrayList 
> [DynamicCacheChangeRequest [cacheName=/", hasCfg=true, 
> nodeId=d91c8f37-dc4f-435e-97dc-8ec02b953b6
> a, clientStartOnly=false, stop=false, destroy=false, 
> disabledAfterStart=false]], exchangeActions=ExchangeActions 
> [startCaches=[/"], stopCaches=null, startGrps=[/"], stopGrps=[], 
> resetParts=null, finalizePartitionCounters=false, stateChangeRequest=null], 
> startCaches=false], allowMerge=false, exchangeFreeSwitch=false]
> 2023-11-14 18:12:34.989 [ERROR][exchange-worker-#75][] Critical system error 
> detected. Will be handled accordingly to configured handler 
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], 
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class 
> o.a.i.IgniteCheckedException: Failed to initialize cache working directory 
> (failed to create, make sure the work folder has correct permissions): 
> /u01/ift/storage/server/data/ise_ift_ignite05/cache-/"]]
> org.apache.ignite.IgniteCheckedException: Failed to initialize cache working 
> directory (failed to create, make sure the work folder has correct 
> permissions): /u01/ift/storage/server/data/ise_ift_ignite05/cache-/"
> at 
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.checkAndInitCacheWorkDir(FilePageStoreManager.java:796)
>  ~[ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CachesRegistry.persistCacheConfigurations(CachesRegistry.java:275)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CachesRegistry.registerAllCachesAndGroups(CachesRegistry.java:262)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CachesRegistry.update(CachesRegistry.java:200)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:877)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1462)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:980)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:3338)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:3172)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125) 
> [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at java.lang.Thread.run(Thread.java:834) [?:?]
> Caused by: java.nio.file.NoSuchFileException: 
> /u01/ift/storage/server/data/ise_ift_ignite05/cache-/"
> at 
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) ~[?:?]
> at 
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
> at 
> sun.nio.fs.UnixException.re

[jira] [Updated] (IGNITE-20781) WAL in fsync mode doesn't write HEADER_RECORD

2023-11-24 Thread Maksim Timonin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Timonin updated IGNITE-20781:

Labels: IEP-104  (was: )

> WAL in fsync mode doesn't write HEADER_RECORD
> -
>
> Key: IGNITE-20781
> URL: https://issues.apache.org/jira/browse/IGNITE-20781
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maksim Timonin
>Priority: Major
>  Labels: IEP-104
>
> subj



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20781) WAL in fsync mode doesn't write HEADER_RECORD

2023-11-24 Thread Maksim Timonin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Timonin updated IGNITE-20781:

Ignite Flags:   (was: Docs Required,Release Notes Required)

> WAL in fsync mode doesn't write HEADER_RECORD
> -
>
> Key: IGNITE-20781
> URL: https://issues.apache.org/jira/browse/IGNITE-20781
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maksim Timonin
>Priority: Major
>  Labels: IEP-104
>
> subj



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20713) WalWriter doesn't poll all read segments

2023-11-24 Thread Maksim Timonin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Timonin updated IGNITE-20713:

Labels: IEP-104  (was: )

> WalWriter doesn't poll all read segments
> 
>
> Key: IGNITE-20713
> URL: https://issues.apache.org/jira/browse/IGNITE-20713
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.15
>Reporter: Maksim Timonin
>Priority: Major
>  Labels: IEP-104
>
> See FileHandleManagerImpl#WalWriter#body()
> For WalWriter the #poll return cut segment, limited with the buffer 
> `capacity` value. Then it's required to run the #poll in the while loop while 
> it returns non-null segment.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20959) CdcMain should not parse whole segment for CdcManager records

2023-11-24 Thread Maksim Timonin (Jira)
Maksim Timonin created IGNITE-20959:
---

 Summary: CdcMain should not parse whole segment for CdcManager 
records
 Key: IGNITE-20959
 URL: https://issues.apache.org/jira/browse/IGNITE-20959
 Project: Ignite
  Issue Type: Improvement
Reporter: Maksim Timonin


To find CdcManager / CdcManagerStopRecord in CdcMain it parse whole segment.

Instead using files in CDC dir should be used. Open question how to monitor 
changes in this files in CdcMain, how to restore basing on this files, how to 
guarantee order between stop/walState files.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20732) WAL doesn't sync in FileWriteHandleImpl

2023-11-24 Thread Maksim Timonin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Timonin updated IGNITE-20732:

Ignite Flags:   (was: Docs Required,Release Notes Required)

> WAL doesn't sync in FileWriteHandleImpl
> ---
>
> Key: IGNITE-20732
> URL: https://issues.apache.org/jira/browse/IGNITE-20732
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maksim Timonin
>Priority: Major
>  Labels: IEP-104
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20732) WAL doesn't sync in FileWriteHandleImpl

2023-11-24 Thread Maksim Timonin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Timonin updated IGNITE-20732:

Labels: IEP-104  (was: )

> WAL doesn't sync in FileWriteHandleImpl
> ---
>
> Key: IGNITE-20732
> URL: https://issues.apache.org/jira/browse/IGNITE-20732
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maksim Timonin
>Priority: Major
>  Labels: IEP-104
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20732) WAL doesn't sync in FileWriteHandleImpl

2023-11-24 Thread Maksim Timonin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Timonin updated IGNITE-20732:

Description: 
fsync() call is missed in FileWriteHandleImpl#close for mmap=true, but 
performed for mode with WalWriter. Try to figure out what is a reason for this 
behavior.

It performs #force and might it is a reason for skipping fsync, but it also 
performs for WalWriter.

 

> WAL doesn't sync in FileWriteHandleImpl
> ---
>
> Key: IGNITE-20732
> URL: https://issues.apache.org/jira/browse/IGNITE-20732
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maksim Timonin
>Priority: Major
>  Labels: IEP-104
>
> fsync() call is missed in FileWriteHandleImpl#close for mmap=true, but 
> performed for mode with WalWriter. Try to figure out what is a reason for 
> this behavior.
> It performs #force and might it is a reason for skipping fsync, but it also 
> performs for WalWriter.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20960) Extract the Placement driver integration tests from runner module to separate one

2023-11-24 Thread Denis Chudov (Jira)
Denis Chudov created IGNITE-20960:
-

 Summary: Extract the Placement driver integration tests from 
runner module to separate one
 Key: IGNITE-20960
 URL: https://issues.apache.org/jira/browse/IGNITE-20960
 Project: Ignite
  Issue Type: Task
 Environment: *Motivation*

Runner module incorporate big amount of tests related to another modules. 
That's lead to long running time for the suite on TC.

Currently, integration tests for the Placement driver are located in 
ignite-runner module. So, we need to extract them to the separate module via 
runner test-fixtures support to decrease the execution time of tests for the 
runner module.

*Definition of done*

All Placement driver related integration tests are moved to the 
placement-driver module.

*Implementation notes*

As reference for such activities could be used IGNITE-20670
Reporter: Denis Chudov






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20960) Extract the Placement driver integration tests from runner module to separate one

2023-11-24 Thread Denis Chudov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Chudov updated IGNITE-20960:
--
Description: 
*Motivation*

Runner module incorporate big amount of tests related to another modules. 
That's lead to long running time for the suite on TC.

Currently, integration tests for the Placement driver are located in 
ignite-runner module. So, we need to extract them to the separate module via 
runner test-fixtures support to decrease the execution time of tests for the 
runner module.

*Definition of done*

All Placement driver related integration tests are moved to the 
placement-driver module.

*Implementation notes*

As reference for such activities could be used IGNITE-20670

> Extract the Placement driver integration tests from runner module to separate 
> one
> -
>
> Key: IGNITE-20960
> URL: https://issues.apache.org/jira/browse/IGNITE-20960
> Project: Ignite
>  Issue Type: Task
>Reporter: Denis Chudov
>Priority: Major
>  Labels: ignite-3
>
> *Motivation*
> Runner module incorporate big amount of tests related to another modules. 
> That's lead to long running time for the suite on TC.
> Currently, integration tests for the Placement driver are located in 
> ignite-runner module. So, we need to extract them to the separate module via 
> runner test-fixtures support to decrease the execution time of tests for the 
> runner module.
> *Definition of done*
> All Placement driver related integration tests are moved to the 
> placement-driver module.
> *Implementation notes*
> As reference for such activities could be used IGNITE-20670



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20960) Extract the Placement driver integration tests from runner module to separate one

2023-11-24 Thread Denis Chudov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Chudov updated IGNITE-20960:
--
Environment: (was: *Motivation*

Runner module incorporate big amount of tests related to another modules. 
That's lead to long running time for the suite on TC.

Currently, integration tests for the Placement driver are located in 
ignite-runner module. So, we need to extract them to the separate module via 
runner test-fixtures support to decrease the execution time of tests for the 
runner module.

*Definition of done*

All Placement driver related integration tests are moved to the 
placement-driver module.

*Implementation notes*

As reference for such activities could be used IGNITE-20670)

> Extract the Placement driver integration tests from runner module to separate 
> one
> -
>
> Key: IGNITE-20960
> URL: https://issues.apache.org/jira/browse/IGNITE-20960
> Project: Ignite
>  Issue Type: Task
>Reporter: Denis Chudov
>Priority: Major
>  Labels: ignite-3
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20960) Extract the Placement driver integration tests from runner module to separate one

2023-11-24 Thread Denis Chudov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Chudov updated IGNITE-20960:
--
Description: 
*Motivation*

Runner module incorporate big amount of tests related to another modules. 
That's lead to long running time for the suite on TC.

Currently, integration tests for the Placement driver are located in 
ignite-runner module (package {{org.apache.ignite.internal.placementdriver}} ). 
So, we need to extract them to the separate module via runner test-fixtures 
support to decrease the execution time of tests for the runner module.

*Definition of done*

All Placement driver related integration tests are moved to the 
placement-driver module.

*Implementation notes*

As reference for such activities could be used IGNITE-20670

  was:
*Motivation*

Runner module incorporate big amount of tests related to another modules. 
That's lead to long running time for the suite on TC.

Currently, integration tests for the Placement driver are located in 
ignite-runner module. So, we need to extract them to the separate module via 
runner test-fixtures support to decrease the execution time of tests for the 
runner module.

*Definition of done*

All Placement driver related integration tests are moved to the 
placement-driver module.

*Implementation notes*

As reference for such activities could be used IGNITE-20670


> Extract the Placement driver integration tests from runner module to separate 
> one
> -
>
> Key: IGNITE-20960
> URL: https://issues.apache.org/jira/browse/IGNITE-20960
> Project: Ignite
>  Issue Type: Task
>Reporter: Denis Chudov
>Priority: Major
>  Labels: ignite-3
>
> *Motivation*
> Runner module incorporate big amount of tests related to another modules. 
> That's lead to long running time for the suite on TC.
> Currently, integration tests for the Placement driver are located in 
> ignite-runner module (package {{org.apache.ignite.internal.placementdriver}} 
> ). So, we need to extract them to the separate module via runner 
> test-fixtures support to decrease the execution time of tests for the runner 
> module.
> *Definition of done*
> All Placement driver related integration tests are moved to the 
> placement-driver module.
> *Implementation notes*
> As reference for such activities could be used IGNITE-20670



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20961) Extract the Read only TX integration tests from runner module to separate one

2023-11-24 Thread Denis Chudov (Jira)
Denis Chudov created IGNITE-20961:
-

 Summary: Extract the Read only TX integration tests from runner 
module to separate one
 Key: IGNITE-20961
 URL: https://issues.apache.org/jira/browse/IGNITE-20961
 Project: Ignite
  Issue Type: Task
Reporter: Denis Chudov


*Motivation*

Runner module incorporate big amount of tests related to another modules. 
That's lead to long running time for the suite on TC.

Currently, integration tests for the read only tx are located in ignite-runner 
module (package {{{}org.apache.ignite.internal.{}}}{{{}readonly{}}} {{{}{}}}). 
So, we need to extract them to the separate module via runner test-fixtures 
support to decrease the execution time of tests for the runner module.

*Definition of done*

All read only tx related integration tests are moved to the ignite-transactions 
module.

*Implementation notes*

As reference for such activities could be used IGNITE-20670



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-20914) Make ScaleCube's metadataTimeout configurable

2023-11-24 Thread Roman Puchkovskiy (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy reassigned IGNITE-20914:
--

Assignee: Roman Puchkovskiy

> Make ScaleCube's metadataTimeout configurable
> -
>
> Key: IGNITE-20914
> URL: https://issues.apache.org/jira/browse/IGNITE-20914
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> ScaleCube's MembershipProtocolImpl fetches node's metadata periodically 
> (using GetMetaDataRequest). If it does not get a response before 
> metadataTimeout expires, it seems to think that the node is not alive anymore 
> and generates a REMOVED event:
> [2023-11-17T00:20:22,153][WARN ][sc-cluster-3345-2][MembershipProtocol] 
> [default:sqllogic1:1ca7b2f5308489d@10.233.107.205:3345][updateMembership][MEMBERSHIP_GOSSIP]
>  Skipping to add/update member: \{m: 
> default:sqllogic0:6a78c57fcd0a496d@10.233.107.205:3344, s: ALIVE, inc: 9}, 
> due to failed fetchMetadata call (cause: 
> java.util.concurrent.TimeoutException: Did not observe any item or terminal 
> signal within 1000ms in 'source(MonoDefer)' (and no fallback has been 
> configured))
> [2023-11-17T00:20:29,189][INFO ][sc-cluster-3345-2][MembershipProtocol] 
> [default:sqllogic1:1ca7b2f5308489d@10.233.107.205:3345] Member left without 
> notification: default:sqllogic0:6a78c57fcd0a496d@10.233.107.205:3344
> [2023-11-17T00:20:29,190][INFO ][sc-cluster-3345-2][MembershipProtocol] 
> [default:sqllogic1:1ca7b2f5308489d@10.233.107.205:3345][publishEvent] 
> MembershipEvent[type=REMOVED, 
> member=default:sqllogic0:6a78c57fcd0a496d@10.233.107.205:3344, 
> oldMetadata=1e61c6c8-154, newMetadata=null, 
> timestamp=2023-11-17T00:20:29.189Z]
> We should avoid this. It seems that 1 second might be too small for a node 
> under load.
> We should make this configurable via Ignite configuration.
> Also, it probably makes sense to set a higher default (like 10 seconds). The 
> reason for the latter is that, if the timeout expires, a node is removed from 
> the physical topology and cannot return there without a restart (this is what 
> our connection establishment protocol requires), so this timeout is critical 
> for stability of Ignite (while it is probably not critical for an average 
> ScaleCube-based application).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20962) .NET: Thin 3.0: TestAuthnOnClientNoAuthnOnServer is flaky

2023-11-24 Thread Pavel Tupitsyn (Jira)
Pavel Tupitsyn created IGNITE-20962:
---

 Summary: .NET: Thin 3.0: TestAuthnOnClientNoAuthnOnServer is flaky
 Key: IGNITE-20962
 URL: https://issues.apache.org/jira/browse/IGNITE-20962
 Project: Ignite
  Issue Type: Bug
  Components: platforms, thin client
Affects Versions: 3.0.0-beta1
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 3.0.0-beta2


*TestAuthnOnClientNoAuthnOnServer* is flaky. Reproduces locally when entire 
*BasicAuthenticatorTests* is executed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20962) .NET: Thin 3.0: TestAuthnOnClientNoAuthnOnServer is flaky

2023-11-24 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-20962:

Labels: .NET ignite-3  (was: )

> .NET: Thin 3.0: TestAuthnOnClientNoAuthnOnServer is flaky
> -
>
> Key: IGNITE-20962
> URL: https://issues.apache.org/jira/browse/IGNITE-20962
> Project: Ignite
>  Issue Type: Bug
>  Components: platforms, thin client
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-beta2
>
>
> *TestAuthnOnClientNoAuthnOnServer* is flaky. Reproduces locally when entire 
> *BasicAuthenticatorTests* is executed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20569) .NET: Thin 3.0: Revise logging

2023-11-24 Thread Igor Sapego (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789431#comment-17789431
 ] 

Igor Sapego commented on IGNITE-20569:
--

Looks good to me

> .NET: Thin 3.0: Revise logging
> --
>
> Key: IGNITE-20569
> URL: https://issues.apache.org/jira/browse/IGNITE-20569
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Blocker
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> .NET client in Ignite 3.x uses custom *IIgniteLogger* interface. This 
> approach is copied from 2.x client; it is outdated and inefficient. 
> Revise it:
> * Should we add a dependency on 
> [Microsoft.Extensions.Logging.Abstractions|https://www.nuget.org/packages/Microsoft.Extensions.Logging.Abstractions/6.0.4#dependencies-body-tab],
>  and use standard *ILogger* instead (accept *ILoggerFactory* in config)?
> * Without that, how do we support structured logging, templates, 
> code-generated loggers?
> * How do we integrate with popular loggers? With *ILogger* it comes out of 
> the box.
> See *Logging guidance for .NET library authors*: 
> https://learn.microsoft.com/en-us/dotnet/core/extensions/logging-library-authors



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20954) [Extensions] cdc-ext: prohibit the same group for event and metadata consumers

2023-11-24 Thread Dmitry Pavlov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitry Pavlov updated IGNITE-20954:
---
Labels: ise  (was: )

> [Extensions] cdc-ext: prohibit the same group for event and metadata consumers
> --
>
> Key: IGNITE-20954
> URL: https://issues.apache.org/jira/browse/IGNITE-20954
> Project: Ignite
>  Issue Type: Task
>Reporter: Nikita Amelchev
>Assignee: Nikita Amelchev
>Priority: Major
>  Labels: ise
>
> The Kafka commit operation throws the exception if consumer group mixes 
> manual and subscription partition assignment (even for different topics). 
> Expected behavior not described: [Kafka Partition 
> Assignment|https://kafka.apache.org/28/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#manualassignment].
> {noformat}
> [async-runnable-runner-1] ERROR 
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer 
> clientId=consumer-my-group-1, groupId=my-group] Offset commit failed on 
> partition my-topic-0 at offset 0: The coordinator is not aware of this member.
> [async-runnable-runner-1] INFO 
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer 
> clientId=consumer-my-group-1, groupId=my-group] OffsetCommit failed with 
> Generation{generationId=-1, memberId='', protocol='null'}: The coordinator is 
> not aware of this member.
> org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be 
> completed since the group has already rebalanced and assigned the partitions 
> to another member. This means that the time between subsequent calls to 
> poll() was longer than the configured max.poll.interval.ms, which typically 
> implies that the poll loop is spending too much time message processing. You 
> can address this either by increasing max.poll.interval.ms or by reducing the 
> maximum size of batches returned in poll() with max.poll.records.
> {noformat}
> Add the consumer group to reproduce:
> {noformat}
> CdcKafkaReplicationTest#kafkaToIgnite
> KafkaToIgniteCdcStreamerConfiguration cfg = new 
> KafkaToIgniteCdcStreamerConfiguration();
> 
> cfg.setMetadataConsumerGroup("kafka-to-ignite-applier");
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20957) Get rid of or rework local partitions versioned value in table manager

2023-11-24 Thread Denis Chudov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Chudov updated IGNITE-20957:
--
Summary: Get rid of or rework local partitions versioned value in table 
manager  (was: Get rid of local partitions versioned value in table manager)

> Get rid of or rework local partitions versioned value in table manager
> --
>
> Key: IGNITE-20957
> URL: https://issues.apache.org/jira/browse/IGNITE-20957
> Project: Ignite
>  Issue Type: Task
>Reporter: Denis Chudov
>Priority: Major
>  Labels: ignite-3
>
> This versioned value ( {{TableManager#localPartsByTableIdVv}} ) is currently 
> used not correctly. It stores the map of local partition storages (table id 
> to partition set), which are created before the partitions start. The 
> partitions start happens on table creation and on assignment change. In first 
> case. this VV is updated via {{VersionedValue#update}} , in the second - the 
> partition set is supposed to be modified within one version (which is 
> incorrect - the value corresponding to a version must be immutable).
> The only point to have this versioned value is to register indexes in the 
> index manager when the partition storages are ready. After IGNITE-19712 there 
> should be no such dependency (indexes may be registered in the storages as 
> they are created), or it should look different: the assignments listener 
> modifying indexes should wait for meta storage revision update on storages 
> versioned value.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20957) Get rid of local partitions versioned value in table manager

2023-11-24 Thread Denis Chudov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Chudov updated IGNITE-20957:
--
Description: 
This versioned value ( {{TableManager#localPartsByTableIdVv}} ) is currently 
used not correctly. It stores the map of local partition storages (table id to 
partition set), which are created before the partitions start. The partitions 
start happens on table creation and on assignment change. In first case. this 
VV is updated via {{VersionedValue#update}} , in the second - the partition set 
is supposed to be modified within one version (which is incorrect - the value 
corresponding to a version must be immutable).

The only point to have this versioned value is to register indexes in the index 
manager when the partition storages are ready. After IGNITE-19712 there should 
be no such dependency (indexes may be registered in the storages as they are 
created), or it should look different: the assignments listener modifying 
indexes should wait for meta storage revision update on storages versioned 
value.

> Get rid of local partitions versioned value in table manager
> 
>
> Key: IGNITE-20957
> URL: https://issues.apache.org/jira/browse/IGNITE-20957
> Project: Ignite
>  Issue Type: Task
>Reporter: Denis Chudov
>Priority: Major
>  Labels: ignite-3
>
> This versioned value ( {{TableManager#localPartsByTableIdVv}} ) is currently 
> used not correctly. It stores the map of local partition storages (table id 
> to partition set), which are created before the partitions start. The 
> partitions start happens on table creation and on assignment change. In first 
> case. this VV is updated via {{VersionedValue#update}} , in the second - the 
> partition set is supposed to be modified within one version (which is 
> incorrect - the value corresponding to a version must be immutable).
> The only point to have this versioned value is to register indexes in the 
> index manager when the partition storages are ready. After IGNITE-19712 there 
> should be no such dependency (indexes may be registered in the storages as 
> they are created), or it should look different: the assignments listener 
> modifying indexes should wait for meta storage revision update on storages 
> versioned value.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (IGNITE-20792) ItRebalanceDistributedTest.testRaftClientsUpdatesAfterRebalance rare fails with "No such partition 0 in table TBL1"

2023-11-24 Thread Kirill Gusakov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kirill Gusakov resolved IGNITE-20792.
-
Resolution: Duplicate

> ItRebalanceDistributedTest.testRaftClientsUpdatesAfterRebalance rare fails 
> with "No such partition 0 in table TBL1"
> ---
>
> Key: IGNITE-20792
> URL: https://issues.apache.org/jira/browse/IGNITE-20792
> Project: Ignite
>  Issue Type: Bug
>Reporter: Kirill Gusakov
>Priority: Major
>  Labels: ignite-3
>
> From time to time test 
> ItRebalanceDistributedTest.testRaftClientsUpdatesAfterRebalance  fails with:
> {code}
> 3:29:03 
>   org.apache.ignite.internal.lang.IgniteInternalException: IGN-CMN-65535 
> TraceId:969a42d9-18ee-4dc9-a18b-d275f893b6b1 No such partition 0 in table TBL1
> 13:29:03 
>   org.apache.ignite.internal.lang.IgniteInternalException: IGN-CMN-65535 
> TraceId:969a42d9-18ee-4dc9-a18b-d275f893b6b1 No such partition 0 in table TBL1
> at 
> app//org.apache.ignite.internal.table.distributed.storage.InternalTableImpl.partitionRaftGroupService(InternalTableImpl.java:1535)
> at 
> app//org.apache.ignite.internal.rebalance.ItRebalanceDistributedTest.lambda$testRaftClientsUpdatesAfterRebalance$23(ItRebalanceDistributedTest.java:637)
> at 
> java.base@11.0.17/java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
> at 
> java.base@11.0.17/java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1632)
> at 
> java.base@11.0.17/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127)
> at 
> java.base@11.0.17/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502)
> at 
> java.base@11.0.17/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488)
> at 
> java.base@11.0.17/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
> at 
> java.base@11.0.17/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
> at 
> java.base@11.0.17/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
> at 
> java.base@11.0.17/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at 
> java.base@11.0.17/java.util.stream.ReferencePipeline.allMatch(ReferencePipeline.java:533)
> at 
> app//org.apache.ignite.internal.rebalance.ItRebalanceDistributedTest.lambda$testRaftClientsUpdatesAfterRebalance$24(ItRebalanceDistributedTest.java:632)
> at 
> app//org.apache.ignite.internal.testframework.IgniteTestUtils.waitForCondition(IgniteTestUtils.java:613)
> at 
> app//org.apache.ignite.internal.testframework.IgniteTestUtils.waitForCondition(IgniteTestUtils.java:596)
> at 
> app//org.apache.ignite.internal.rebalance.ItRebalanceDistributedTest.testRaftClientsUpdatesAfterRebalance(ItRebalanceDistributedTest.java:631)
> at 
> java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
>  Method)
> at 
> java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> java.base@11.0.17/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base@11.0.17/java.lang.reflect.Method.invoke(Method.java:566)
> at 
> app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
> at 
> app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
> at 
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
> at 
> app//org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
> at 
> app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
> at 
> app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
> at 
> app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
> at 
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
> at 
> app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
> at 
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
> at 
> app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(Invo

[jira] [Commented] (IGNITE-20830) Do not retry attempts to subscribe in TopologyAwareRaftGroupService

2023-11-24 Thread Vladislav Pyatkov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789450#comment-17789450
 ] 

Vladislav Pyatkov commented on IGNITE-20830:


We have doubts about the implementation of a topology-aware RAFT client. 
Currently, the subscription to the leader update occurs when the node appears 
in the logical topology. So the questions are:
# Does a node have to appear in a physical topology when it appears in a 
logical one?
# When a node leaves the cluster, is there only one way for a node to return to 
the logical topology: return to the physical one and then appear in the logical 
one?

> Do not retry attempts to subscribe in TopologyAwareRaftGroupService
> ---
>
> Key: IGNITE-20830
> URL: https://issues.apache.org/jira/browse/IGNITE-20830
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> h3. Motivation
> In IGNITE-20828, retries on UNsubscription were removed. It needs to be 
> considered whether we should retry subscriptions or not.
> h3. Implementation notes
> * Replace the TopologyAwareRaftGroupService#sendWithRetry method with the 
> simple netwotk invoke.
> * Move specific exception handling logic to the particular method 
> (TopologyAwareRaftGroupService#unsubscribeLeader).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (IGNITE-20622) OOMe during TPC-C scale10

2023-11-24 Thread Yury Gerzhedovich (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yury Gerzhedovich resolved IGNITE-20622.

Resolution: Cannot Reproduce

> OOMe during TPC-C scale10
> -
>
> Key: IGNITE-20622
> URL: https://issues.apache.org/jira/browse/IGNITE-20622
> Project: Ignite
>  Issue Type: Bug
>  Components: networking
>Affects Versions: 3.0
>Reporter: Alexander Belyak
>Priority: Major
>  Labels: ignite-3
>
> Got OOMe during loading TPC-C database with scale 10 (benchbase). Just 
> starting benchbase with TPC-C benchmark with scale factor is enough to get 
> stable OOMe exception in 10 min.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-20830) Do not retry attempts to subscribe in TopologyAwareRaftGroupService

2023-11-24 Thread Vladislav Pyatkov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789450#comment-17789450
 ] 

Vladislav Pyatkov edited comment on IGNITE-20830 at 11/24/23 1:01 PM:
--

[~rpuch] Could you please explain whether my understanding is correct or not?
We have doubts about the implementation of a topology-aware RAFT client. 
Currently, the subscription to the leader update occurs when the node appears 
in the logical topology. So the questions are:
# Does a node have to appear in a physical topology when it appears in a 
logical one?
# When a node leaves the cluster, is there only one way for a node to return to 
the logical topology: return to the physical one and then appear in the logical 
one?


was (Author: v.pyatkov):
We have doubts about the implementation of a topology-aware RAFT client. 
Currently, the subscription to the leader update occurs when the node appears 
in the logical topology. So the questions are:
# Does a node have to appear in a physical topology when it appears in a 
logical one?
# When a node leaves the cluster, is there only one way for a node to return to 
the logical topology: return to the physical one and then appear in the logical 
one?

> Do not retry attempts to subscribe in TopologyAwareRaftGroupService
> ---
>
> Key: IGNITE-20830
> URL: https://issues.apache.org/jira/browse/IGNITE-20830
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> h3. Motivation
> In IGNITE-20828, retries on UNsubscription were removed. It needs to be 
> considered whether we should retry subscriptions or not.
> h3. Implementation notes
> * Replace the TopologyAwareRaftGroupService#sendWithRetry method with the 
> simple netwotk invoke.
> * Move specific exception handling logic to the particular method 
> (TopologyAwareRaftGroupService#unsubscribeLeader).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20569) .NET: Thin 3.0: Revise logging

2023-11-24 Thread Pavel Tupitsyn (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789458#comment-17789458
 ] 

Pavel Tupitsyn commented on IGNITE-20569:
-

Merged to main: 91323af877c2591d7238fae03c373c0539e75afe

> .NET: Thin 3.0: Revise logging
> --
>
> Key: IGNITE-20569
> URL: https://issues.apache.org/jira/browse/IGNITE-20569
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Blocker
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> .NET client in Ignite 3.x uses custom *IIgniteLogger* interface. This 
> approach is copied from 2.x client; it is outdated and inefficient. 
> Revise it:
> * Should we add a dependency on 
> [Microsoft.Extensions.Logging.Abstractions|https://www.nuget.org/packages/Microsoft.Extensions.Logging.Abstractions/6.0.4#dependencies-body-tab],
>  and use standard *ILogger* instead (accept *ILoggerFactory* in config)?
> * Without that, how do we support structured logging, templates, 
> code-generated loggers?
> * How do we integrate with popular loggers? With *ILogger* it comes out of 
> the box.
> See *Logging guidance for .NET library authors*: 
> https://learn.microsoft.com/en-us/dotnet/core/extensions/logging-library-authors



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20569) .NET: Thin 3.0: Revise logging

2023-11-24 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-20569:

Release Note: .NET: Use Microsoft.Extensions.Logging APIs for logging.  
(was: .NET: Refactor logging to Microsoft.Extensions.Logging)

> .NET: Thin 3.0: Revise logging
> --
>
> Key: IGNITE-20569
> URL: https://issues.apache.org/jira/browse/IGNITE-20569
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Blocker
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> .NET client in Ignite 3.x uses custom *IIgniteLogger* interface. This 
> approach is copied from 2.x client; it is outdated and inefficient. 
> Revise it:
> * Should we add a dependency on 
> [Microsoft.Extensions.Logging.Abstractions|https://www.nuget.org/packages/Microsoft.Extensions.Logging.Abstractions/6.0.4#dependencies-body-tab],
>  and use standard *ILogger* instead (accept *ILoggerFactory* in config)?
> * Without that, how do we support structured logging, templates, 
> code-generated loggers?
> * How do we integrate with popular loggers? With *ILogger* it comes out of 
> the box.
> See *Logging guidance for .NET library authors*: 
> https://learn.microsoft.com/en-us/dotnet/core/extensions/logging-library-authors



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20963) Java thin 3.0: testClientReceivesPartitionAssignmentUpdates is flaky

2023-11-24 Thread Pavel Tupitsyn (Jira)
Pavel Tupitsyn created IGNITE-20963:
---

 Summary: Java thin 3.0:  
testClientReceivesPartitionAssignmentUpdates is flaky
 Key: IGNITE-20963
 URL: https://issues.apache.org/jira/browse/IGNITE-20963
 Project: Ignite
  Issue Type: Bug
  Components: thin client
Affects Versions: 3.0.0-beta1
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 3.0.0-beta2






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20963) Java thin 3.0: testClientReceivesPartitionAssignmentUpdates is flaky

2023-11-24 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-20963:

Description: 
Test history: 
https://ci.ignite.apache.org/test/5411269592198682160?currentProjectId=ApacheIgnite3xGradle_Test&expandTestHistoryChartSection=true

{code}
org.opentest4j.AssertionFailedError: Operation get was not executed on expected 
node ==> expected:  but was: 
  at 
app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
  at 
app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
  at app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
  at app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
  at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1153)
  at 
app//org.apache.ignite.client.PartitionAwarenessTest.assertOpOnNode(PartitionAwarenessTest.java:582)
  at 
app//org.apache.ignite.client.PartitionAwarenessTest.testClientReceivesPartitionAssignmentUpdates(PartitionAwarenessTest.java:189)
{code}

> Java thin 3.0:  testClientReceivesPartitionAssignmentUpdates is flaky
> -
>
> Key: IGNITE-20963
> URL: https://issues.apache.org/jira/browse/IGNITE-20963
> Project: Ignite
>  Issue Type: Bug
>  Components: thin client
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Test history: 
> https://ci.ignite.apache.org/test/5411269592198682160?currentProjectId=ApacheIgnite3xGradle_Test&expandTestHistoryChartSection=true
> {code}
> org.opentest4j.AssertionFailedError: Operation get was not executed on 
> expected node ==> expected:  but was: 
>   at 
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
>   at 
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
>   at 
> app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
>   at 
> app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
>   at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1153)
>   at 
> app//org.apache.ignite.client.PartitionAwarenessTest.assertOpOnNode(PartitionAwarenessTest.java:582)
>   at 
> app//org.apache.ignite.client.PartitionAwarenessTest.testClientReceivesPartitionAssignmentUpdates(PartitionAwarenessTest.java:189)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20963) Java thin 3.0: testClientReceivesPartitionAssignmentUpdates is flaky

2023-11-24 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-20963:

Ignite Flags:   (was: Docs Required,Release Notes Required)

> Java thin 3.0:  testClientReceivesPartitionAssignmentUpdates is flaky
> -
>
> Key: IGNITE-20963
> URL: https://issues.apache.org/jira/browse/IGNITE-20963
> Project: Ignite
>  Issue Type: Bug
>  Components: thin client
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Test history: 
> https://ci.ignite.apache.org/test/5411269592198682160?currentProjectId=ApacheIgnite3xGradle_Test&expandTestHistoryChartSection=true
> {code}
> org.opentest4j.AssertionFailedError: Operation get was not executed on 
> expected node ==> expected:  but was: 
>   at 
> app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
>   at 
> app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
>   at 
> app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
>   at 
> app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
>   at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1153)
>   at 
> app//org.apache.ignite.client.PartitionAwarenessTest.assertOpOnNode(PartitionAwarenessTest.java:582)
>   at 
> app//org.apache.ignite.client.PartitionAwarenessTest.testClientReceivesPartitionAssignmentUpdates(PartitionAwarenessTest.java:189)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-20830) Do not retry attempts to subscribe in TopologyAwareRaftGroupService

2023-11-24 Thread Roman Puchkovskiy (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789463#comment-17789463
 ] 

Roman Puchkovskiy edited comment on IGNITE-20830 at 11/24/23 1:52 PM:
--

[~v.pyatkov] At least theoretically it's possible that node B sees a message 
about a node A being added to the logical topology (by CMG leader C) before it 
sees A in its physical topology.


was (Author: rpuch):
At least theoretically it's possible that node B sees a message about a node A 
being added to the logical topology (by CMG leader C) before it sees A in its 
physical topology.

> Do not retry attempts to subscribe in TopologyAwareRaftGroupService
> ---
>
> Key: IGNITE-20830
> URL: https://issues.apache.org/jira/browse/IGNITE-20830
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> h3. Motivation
> In IGNITE-20828, retries on UNsubscription were removed. It needs to be 
> considered whether we should retry subscriptions or not.
> h3. Implementation notes
> * Replace the TopologyAwareRaftGroupService#sendWithRetry method with the 
> simple netwotk invoke.
> * Move specific exception handling logic to the particular method 
> (TopologyAwareRaftGroupService#unsubscribeLeader).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20964) Add a missing node to the physical topology when it appears in the logical topology

2023-11-24 Thread Roman Puchkovskiy (Jira)
Roman Puchkovskiy created IGNITE-20964:
--

 Summary: Add a missing node to the physical topology when it 
appears in the logical topology
 Key: IGNITE-20964
 URL: https://issues.apache.org/jira/browse/IGNITE-20964
 Project: Ignite
  Issue Type: Improvement
Reporter: Roman Puchkovskiy
 Fix For: 3.0.0-beta2


It might happen that a node is observed to be added to a logical topology (LT) 
before it's added to the physical topology (PT). This might create a strange 
situation when a node is already in LT, but it's not possible to send it a 
message.

A race between removing a node from the PT and addint it to the LT might be 
resolved using the stale detector: if a node ID is stale, we don't add it to 
the PT.

Same protection should be used to prevent old nodes from being added to the PT 
when LT events are replayed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20965) Handling timeouts and errors during shutdown

2023-11-24 Thread Roman Puchkovskiy (Jira)
Roman Puchkovskiy created IGNITE-20965:
--

 Summary: Handling timeouts and errors during shutdown
 Key: IGNITE-20965
 URL: https://issues.apache.org/jira/browse/IGNITE-20965
 Project: Ignite
  Issue Type: Improvement
Reporter: Roman Puchkovskiy






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20965) Handling timeouts and errors during shutdown

2023-11-24 Thread Roman Puchkovskiy (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy updated IGNITE-20965:
---
Description: 
A component might hang forever during shutdown, or it might throw an exception 
during shutdown, effectively interrupting the shutdown routine. This is bad UX.

We should shut a node down as gracefully as possible.
 #  Some components might define their own shutdown timeouts if they are 
well-defined (like time to let a checkpoint finish)
 #  Whole shutdown routine should have a time budget
 #  If shutdown times out, a Failure Handler should be notified
 #  Same happens if an exception is thrown during shutdown

> Handling timeouts and errors during shutdown
> 
>
> Key: IGNITE-20965
> URL: https://issues.apache.org/jira/browse/IGNITE-20965
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
>
> A component might hang forever during shutdown, or it might throw an 
> exception during shutdown, effectively interrupting the shutdown routine. 
> This is bad UX.
> We should shut a node down as gracefully as possible.
>  #  Some components might define their own shutdown timeouts if they are 
> well-defined (like time to let a checkpoint finish)
>  #  Whole shutdown routine should have a time budget
>  #  If shutdown times out, a Failure Handler should be notified
>  #  Same happens if an exception is thrown during shutdown



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20748) Document new flags for JDK 11 and JDK 17

2023-11-24 Thread Igor Gusev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789475#comment-17789475
 ] 

Igor Gusev commented on IGNITE-20748:
-

[~sk0x50] can you help review and merge?

> Document new flags for JDK 11 and JDK 17
> 
>
> Key: IGNITE-20748
> URL: https://issues.apache.org/jira/browse/IGNITE-20748
> Project: Ignite
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Ivan Daschinsky
>Assignee: Igor Gusev
>Priority: Major
>  Labels: documentation
> Fix For: 2.16
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20966) Improve error messages for schema change validation failures

2023-11-24 Thread Roman Puchkovskiy (Jira)
Roman Puchkovskiy created IGNITE-20966:
--

 Summary: Improve error messages for schema change validation 
failures
 Key: IGNITE-20966
 URL: https://issues.apache.org/jira/browse/IGNITE-20966
 Project: Ignite
  Issue Type: Improvement
Reporter: Roman Puchkovskiy
 Fix For: 3.0.0-beta2


At least, 'Commit failed because schema %d is not forward-compatible with %d 
for table %d' should be improved to make it useful for an end-user. We shoiuld 
at least add table name to the message.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-20942) Primary replica events are handled by all replication groups

2023-11-24 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin reassigned IGNITE-20942:


Assignee: Alexander Lapin

> Primary replica events are handled by all replication groups
> 
>
> Key: IGNITE-20942
> URL: https://issues.apache.org/jira/browse/IGNITE-20942
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> h3. Motivation
> PRIMARY_REPLICA_EXPIRED (PRIMARY_REPLICA_ELECTED) event is triggered by the 
> placement driver. The message contains a property that points to the 
> certificate replication group. But the event listener, which is determined in 
> each PartitionReplicaListener, does not handle the property:
> {code}
> if (!localNode.name().equals(evt.leaseholder())) {
> return completedFuture(false);
>  }
> {code}
> The issue is also shown in the log: 
> {noformat}
> [2023-11-22T10:00:17,056][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_19]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_0]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_9]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_10]
> {noformat}
> h3. Implementation notes:
> Checking a replication group ID is enough to fix this issue (in both event 
> handlers).
> {code}
> if (!localNode.name().equals(evt.leaseholder()) || 
> !replicationGroupId.equals(evt.groupId())) {
> return completedFuture(false);
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20942) Primary replica events are handled by all replication groups

2023-11-24 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-20942:
-
Fix Version/s: 3.0.0-beta2

> Primary replica events are handled by all replication groups
> 
>
> Key: IGNITE-20942
> URL: https://issues.apache.org/jira/browse/IGNITE-20942
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> h3. Motivation
> PRIMARY_REPLICA_EXPIRED (PRIMARY_REPLICA_ELECTED) event is triggered by the 
> placement driver. The message contains a property that points to the 
> certificate replication group. But the event listener, which is determined in 
> each PartitionReplicaListener, does not handle the property:
> {code}
> if (!localNode.name().equals(evt.leaseholder())) {
> return completedFuture(false);
>  }
> {code}
> The issue is also shown in the log: 
> {noformat}
> [2023-11-22T10:00:17,056][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_19]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_0]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_9]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_10]
> {noformat}
> h3. Implementation notes:
> Checking a replication group ID is enough to fix this issue (in both event 
> handlers).
> {code}
> if (!localNode.name().equals(evt.leaseholder()) || 
> !replicationGroupId.equals(evt.groupId())) {
> return completedFuture(false);
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20942) Primary replica events are handled by all replication groups

2023-11-24 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20942:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Primary replica events are handled by all replication groups
> 
>
> Key: IGNITE-20942
> URL: https://issues.apache.org/jira/browse/IGNITE-20942
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> h3. Motivation
> PRIMARY_REPLICA_EXPIRED (PRIMARY_REPLICA_ELECTED) event is triggered by the 
> placement driver. The message contains a property that points to the 
> certificate replication group. But the event listener, which is determined in 
> each PartitionReplicaListener, does not handle the property:
> {code}
> if (!localNode.name().equals(evt.leaseholder())) {
> return completedFuture(false);
>  }
> {code}
> The issue is also shown in the log: 
> {noformat}
> [2023-11-22T10:00:17,056][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_19]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_0]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_9]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_10]
> {noformat}
> h3. Implementation notes:
> Checking a replication group ID is enough to fix this issue (in both event 
> handlers).
> {code}
> if (!localNode.name().equals(evt.leaseholder()) || 
> !replicationGroupId.equals(evt.groupId())) {
> return completedFuture(false);
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20942) Primary replica events are handled by all replication groups

2023-11-24 Thread Denis Chudov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789498#comment-17789498
 ] 

Denis Chudov commented on IGNITE-20942:
---

[~alapin] LGTM.

> Primary replica events are handled by all replication groups
> 
>
> Key: IGNITE-20942
> URL: https://issues.apache.org/jira/browse/IGNITE-20942
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> h3. Motivation
> PRIMARY_REPLICA_EXPIRED (PRIMARY_REPLICA_ELECTED) event is triggered by the 
> placement driver. The message contains a property that points to the 
> certificate replication group. But the event listener, which is determined in 
> each PartitionReplicaListener, does not handle the property:
> {code}
> if (!localNode.name().equals(evt.leaseholder())) {
> return completedFuture(false);
>  }
> {code}
> The issue is also shown in the log: 
> {noformat}
> [2023-11-22T10:00:17,056][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_19]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_0]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_9]
> [2023-11-22T10:00:17,057][INFO 
> ][%isnt_tmpar_0%metastorage-watch-executor-3][PartitionReplicaListener] 
> Primary replica expired [grp=5_part_10]
> {noformat}
> h3. Implementation notes:
> Checking a replication group ID is enough to fix this issue (in both event 
> handlers).
> {code}
> if (!localNode.name().equals(evt.leaseholder()) || 
> !replicationGroupId.equals(evt.groupId())) {
> return completedFuture(false);
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-12625) Thin client: Marshaling/unmarshaling of objects performed twice

2023-11-24 Thread Mikhail Petrov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-12625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Petrov reassigned IGNITE-12625:
---

Assignee: Mikhail Petrov

> Thin client: Marshaling/unmarshaling of objects performed twice
> ---
>
> Key: IGNITE-12625
> URL: https://issues.apache.org/jira/browse/IGNITE-12625
> Project: Ignite
>  Issue Type: Bug
>  Components: binary, thin client
>Reporter: Aleksey Plekhanov
>Assignee: Mikhail Petrov
>Priority: Major
>
> For each thin client cache operation marshaling/unmarshaling of objects 
> performed twice. For example, cache "put" operation marshal object on the 
> client-side, then unmarshal object (with keep binaries) on the server-side 
> and marshal it again before putting it to the cache. It causes some 
> undesirable effects. For example, references to the same binary object in a 
> collection ( {{new ArrayList(Arrays.asList(person, person))}} ) deserialized 
> as different objects.
> Reproducer:
> {code:java}
> try (IgniteClient client = startClient()) {
> ClientCache cache = 
> client.getOrCreateCache(DEFAULT_CACHE_NAME);
> Person person = new Person(0, "name");
> cache.put(0, new Object[] {person, person} );
> Object[] res = cache.get(0);
> assertTrue(res[0] == res[1]);
> }{code}
> Also, we need to unwrap binaries recursively since all objects inside 
> collections, arrays and maps become binary objects after 
> marshaling/unmarshalling (see IGNITE-12468)
> Also, we don't know do we really need to deserialize binary objects. If 
> object was originally binary there is no evidence of this after 
> marshaling/unmarshaling on server-side. This leads to problems when a binary 
> object was created for unknown class.
> Reproducer:
> {code:java}
> cache.put(0, client.binary().builder("TestType").setField("test", 
> "val").build());
> cache.get(0);{code}
> Will throw exception:
> {noformat}
> class org.apache.ignite.binary.BinaryInvalidTypeException: TestType
> at 
> org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:762)
> at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1757)
> at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:319)
> at 
> org.apache.ignite.internal.client.thin.ClientBinaryMarshaller.deserialize(ClientBinaryMarshaller.java:74)
> at 
> org.apache.ignite.internal.client.thin.ClientUtils.unwrapBinary(ClientUtils.java:558)
> at 
> org.apache.ignite.internal.client.thin.ClientUtils.readObject(ClientUtils.java:547){noformat}
> To avoid double marshaling we could pass byte array from request content to 
> cache directly (for example using {{CacheObject}}), but we don't have object 
> size in thin client protocol, so in any case, we need to traverse the 
> objects. Also, we don't have the ability to get {{CacheObject}} from the 
> cache now, so such an approach will only work in one way, for "put" 
> operations, but not for "get" operations.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (IGNITE-18995) CDC test failures on BinaryMetadataFileStore.restoreMetadata

2023-11-24 Thread Ilya Shishkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Shishkov resolved IGNITE-18995.

Resolution: Duplicate

Fixed in IGNITE-20572

> CDC test failures on BinaryMetadataFileStore.restoreMetadata
> 
>
> Key: IGNITE-18995
> URL: https://issues.apache.org/jira/browse/IGNITE-18995
> Project: Ignite
>  Issue Type: Test
>  Components: extensions
>Reporter: Ilya Shishkov
>Assignee: Ilya Shishkov
>Priority: Minor
>  Labels: IEP-59, ise
> Attachments: _TESTS_CDC_408.log.zip, _TESTS_CDC_417.log.zip
>
>
> Sometimes tests fails with following stacktraces:
> {noformat}
>   Caused by: java.lang.AssertionError
> at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:299)
> at 
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:120)
> at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:92)
> at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10662)
> at 
> org.apache.ignite.internal.processors.cache.binary.BinaryMetadataFileStore.restoreMetadata(BinaryMetadataFileStore.java:222)
> at 
> org.apache.ignite.internal.processors.cache.binary.BinaryMetadataFileStore.restoreMetadata(BinaryMetadataFileStore.java:207)
> at 
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.start(CacheObjectBinaryProcessorImpl.java:370)
> at 
> org.apache.ignite.internal.cdc.CdcMain.startStandaloneKernal(CdcMain.java:375)
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:301)
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:267)
> ... 6 more
> {noformat}
> or:
> {noformat}
>  Caused by: java.lang.AssertionError
> at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:299)
> at 
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:120)
> at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:92)
> at 
> org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:10662)
> at 
> org.apache.ignite.internal.processors.cache.binary.BinaryMetadataFileStore.restoreMetadata(BinaryMetadataFileStore.java:222)
> at 
> org.apache.ignite.internal.processors.cache.binary.BinaryMetadataFileStore.restoreMetadata(BinaryMetadataFileStore.java:216)
> at 
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.cacheMetadataLocally(CacheObjectBinaryProcessorImpl.java:1076)
> at 
> org.apache.ignite.internal.cdc.CdcMain.lambda$updateTypes$4(CdcMain.java:617)
> at 
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at 
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at 
> java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)
> at 
> java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
> at 
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
> at 
> java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161)
> at 
> java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
> at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
> at org.apache.ignite.internal.cdc.CdcMain.updateTypes(CdcMain.java:628)
> at org.apache.ignite.internal.cdc.CdcMain.updateMetadata(CdcMain.java:589)
> at 
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:464)
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:325)
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:267)
> ... 6 more
> {noformat}
> Reason should be investigated and fixed.
> Build logs with failures: 
> * [^_TESTS_CDC_408.log.zip]  
> * [^_TESTS_CDC_417.log.zip] 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)