[MTCGA]: new failures in builds [5780347, 5780334] needs to be handled

2020-12-09 Thread dpavlov . tasks
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. *New Critical Failure in master Control Utility https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_ControlUtility?branch=%3Cdefault%3E No changes in the build *

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
Hi, Andrey! Users expect DataStorageConfiguration#maxWalArchiveSize to mean that WAL archive will not exceed this value, but it is not. It seems that to reduce the chance of getting into a situation when we exceed WAL archive, it will be lowed when we clean it when switching to a new segment

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
Hi, Vishwas! Speed of uploading data directly associated with the growth of WAL archive. 09.12.2020, 17:05, "vbm" : > Hi Kirill Tkalenko, > > Is there any relation to rate of ingestion of data to ignite ? > > We had seen the issue of WAL growing infinitely recently in our K8s cluster. > We were

Re: 2.9.1 release scope and dates

2020-12-09 Thread Yaroslav Molochkov
Guys, hello! I did some research concerning performance comparison 2.9 and 2.9.1 before moving any further and starting a vote. It seems that there is no degradation of any kind. You can find relevant metrics here

Re: [DISCUSSION] Modules organization in Ignite 3

2020-12-09 Thread Ilya Kasnacheev
Hello! When you do mvn dependencies:copy, you will now end up with "ignite-core-X.Y.Z.jar", "ignite-indexing-X.Y.Z.jar" But if we remove "ignite" from artifact name, user will end up with cryptic "configuration.jar", "raft.jar", etc, etc. Remember that group name is discarded in file names.

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread Andrey Gura
Kiriill, Issue description contains the following: > At the moment, WAL archive is cleared at the end of the checkpoint, which > does not seem correct and needs to be moved Could you please explain why existing behavior is not correct. It seems that it is not enough motivation for change. On

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread vbm
Hi Kirill Tkalenko, Is there any relation to rate of ingestion of data to ignite ? We had seen the issue of WAL growing infinitely recently in our K8s cluster. We were ingesting data at around 2Mbps. In other clusters where we did not have such a fast ingestion of data, this issue was not

Re: Issue with custom security plugin and thin clients

2020-12-09 Thread vbm
Hi Denis, Any thoughts on the approach mentioned above ? Regards, Vishwas -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [DISCUSSION] Modules organization in Ignite 3

2020-12-09 Thread Nikolay Izhikov
> Nikolay, maybe is it better to discuss your question on a separate topic? > Because looks like it is a pretty discussable topic. Sorry for interrupting original discussion. For me it was a overlapping topic in the beginning. Agree that we we should have separate thread. > 9 дек. 2020 г., в

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
And this property IGNITE_THRESHOLD_WAL_ARCHIVE_SIZE_PERCENTAGE 09.12.2020, 16:28, "ткаленко кирилл" : > And this property IGNITE_THRESHOLD_WAL_ARCHIVE_SIZE_PERCENTAGE > > 09.12.2020, 16:25, "ткаленко кирилл" : >> Hi, Shiva! >> >> I am not aware of such tickets yet. Later I think I can deal with

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
Hi, Shiva! I am not aware of such tickets yet. Later I think I can deal with this issue. Now you can try to increase the frequency of checkpoints, the maximum WAL archive size and try to change the system property IGNITE_CHECKPOINT_TRIGGER_ARCHIVE_SIZE_PERCENTAGE. This will not solve the

Re: [DISCUSSION] Modules organization in Ignite 3

2020-12-09 Thread Anton Kalashnikov
Hello, I totally agree that we should start to think about the module organization. I suggest making the new confluence page where we define new rules on how to develop modules. In my opinion, at least, the following topics should be covered there(it makes sense to discuss every topic

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread shm
Hi Kirill Tkalenko, Thanks for your reply! Do you know any known issue related to problem I'm facing ? or any ticket related to infinite WAL growing issue which can be correlated to my issue? Regards, Shiva -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[jira] [Created] (IGNITE-13832) disco-notifier-worker handles IgniteInterruptedCheckedException incorrectly

2020-12-09 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-13832: -- Summary: disco-notifier-worker handles IgniteInterruptedCheckedException incorrectly Key: IGNITE-13832 URL: https://issues.apache.org/jira/browse/IGNITE-13832

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
Hi, Shiva! Yes, this ticket will only be about moving WAL archive cleanup. I think further it is possible to solve the problem of WAL archive overflow, but before that we need to solve several problems and deal with heuristics. 09.12.2020, 15:02, "shm" : > Hi Kirill Tkalenko > > Is this ticket

Re: Pull request for minor fix in index page documentation

2020-12-09 Thread Sumit Deshinge
Great, thanks Denis. On Tue, Dec 8, 2020 at 11:35 PM Denis Magda wrote: > Hi Sumit, > > Thanks for catching the issue and preparing a pull-request! The > pull-request merges changes to your Ignite master branch rather than to the > Ignite master. Not a bid deal. I see where the problem is.

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread shm
Hi Kirill Tkalenko Is this ticket is just for moving WAL archive clean-up logic from "post checkpoint" to FileWriteAheadLogManager#rollOver ? Or is there any plan to change the logic which considers Archived WAL segments for clean-up ? I'm facing a Major issue related to WAL usage and WAL usage

Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
Hello to all! At the moment, the archive is cleared at the end of the checkpoint, it seems it should happen in FileWriteAheadLogManager. I suggest moving it into the FileWriteAheadLogManager#rollOver when the DataStorageConfiguration#maxWalArchiveSize is reached. To do this, I created a

[jira] [Created] (IGNITE-13831) Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread Kirill Tkalenko (Jira)
Kirill Tkalenko created IGNITE-13831: Summary: Move WAL archive cleanup from checkpoint to rollover Key: IGNITE-13831 URL: https://issues.apache.org/jira/browse/IGNITE-13831 Project: Ignite

Re: Removing MVCC public API

2020-12-09 Thread Maxim Muzafarov
+1 Also, I want to mention the list of MVCC related opened issues [1] without any updates for over a year. [1] https://s.apache.org/1r5yk On Wed, 9 Dec 2020 at 10:22, Alexei Scherbakov wrote: > > +1 > > ср, 9 дек. 2020 г. в 10:03, Petr Ivanov : > > > +1 > > > > > > > On 9 Dec 2020, at 09:39,