Re:[ANNOUNCE] Welcome Kirill Tkalenko as a new committer

2022-10-14 Thread ткаленко кирилл
It's an honor for me, thank you all!

Re:[ANNOUNCE] New PMC member: Vyacheslav Koptilin

2022-08-18 Thread ткаленко кирилл
Congratulations!

Re:[VOTE] Release Apache Ignite 3.0.0-alpha5 RC1

2022-06-09 Thread ткаленко кирилл
+1

Re:[ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

2022-06-07 Thread ткаленко кирилл
Hello everyone, I would like to fix the identified bug when destroying the table for persistent PageMemory, I could fix it tomorrow, can we add it to the scope? https://issues.apache.org/jira/browse/IGNITE-17128

Re:[VOTE] Add micronaut dependency to Ignite 3 (reopened)

2022-06-01 Thread ткаленко кирилл
+ 1

Re:Adding a system view of recently completed compute tasks

2022-01-18 Thread ткаленко кирилл
I will proceed with the implementation.

Re:Adding a system view of recently completed compute tasks

2022-01-13 Thread ткаленко кирилл
OK, that seems like a reasonable suggestion.

Re:[VOTE] @Nullable/@NotNull annotation usage in Ignite 3

2022-01-13 Thread ткаленко кирилл
+1 to option 2

Re:Adding a system view of recently completed compute tasks

2022-01-12 Thread ткаленко кирилл
Judging by the code, this is the task session timeout.

Re:Adding a system view of recently completed compute tasks

2022-01-12 Thread ткаленко кирилл
Thanks Zhenya, but here we can see only the current working tasks, after they are completed it is not possible to get the actual time of work, for example, for statistics.

Adding a system view of recently completed compute tasks

2022-01-11 Thread ткаленко кирилл
Hello everyone! I want to add a new system view to get the last N (configurable by system property, default 100) completed compute tasks, are there any objections?

Re:[DISCUSSION] @Nullable/@NotNull annotation usage in Ignite 3

2021-12-16 Thread ткаленко кирилл
I'm for the second option.

Re: [VOTE] Create separate Jira project and Confluence space for Ignite 3

2021-10-05 Thread ткаленко кирилл
+1 05.10.2021, 02:52, "Valentin Kulichenko" : > Hello Community, > > As discussed in [1], I would like to propose the creation of a separate > Jira project and Confluence space for Ignite 3. > > Ignite 2 and Ignite 3 are developed in parallel in separate repos, so we > need a clear separation in o

Re: IGNITE-15256 request for review

2021-08-11 Thread ткаленко кирилл
Hi, in the beginning it is worth getting a green visa. 11.08.2021, 15:21, "Ivan Fedorenkov" : > Dear Ignite Devs, > > I would like to contribute a fix that addresses a bug associated with > registration of user types by Ignite Java Thin Client. Could someone please > review the code? https://githu

Fwd: [ANNOUNCE] Welcome Zhenya Stanilovsky as a new committer

2021-07-30 Thread ткаленко кирилл
Zhenya, congratulations! Пересылаемое сообщение 30.07.2021, 09:50, "Ivan Daschinsky" : Zhenya, congrats, well deserved! пт, 30 июл. 2021 г. в 00:44, Andrey Mashenkov : >  Congratulations Zhenya! > >  On Fri, Jul 30, 2021 at 12:06 AM Maxim Muzafarov >  wrote: > >  > The Proje

Re: [DISCUSS] Confuse default inspections.

2021-07-21 Thread ткаленко кирилл
+1 20.07.2021, 23:06, "Юрий" : > I totally agree. > Let's get rid of them. > > вт, 20 июл. 2021 г. в 18:11, Konstantin Orlov : > >>  + for both >> >>  -- >>  Regards, >>  Konstantin Orlov >> >>  > On 20 Jul 2021, at 16:32, Pavel Tupitsyn wrote: >>  > >>  > Agree with for both points >>  > >>  > O

Re: Problem with dropping the index

2021-07-07 Thread ткаленко кирилл
gnite/pull/9223 07.07.2021, 10:28, "Alexei Scherbakov" : > вт, 6 июл. 2021 г. в 15:57, ткаленко кирилл : > >>  >> Can you clarify what it means to rename root index trees ? >>  Replacing >>   >> org.apache.ignite.internal.processors.cache.persistence

Re: Problem with dropping the index

2021-07-06 Thread ткаленко кирилл
ation - on reading it create an index > during logical recovery > 2) write logical record on index deletion - on reading it delete an index > during logical recovery and start background clearing task with real root > pages. > > Will it work for you ? > > вт, 6 июл. 2021 г. в 12

Problem with dropping the index

2021-07-06 Thread ткаленко кирилл
Hello everyone! Currently, dropping indexes consists of the following steps (based on SchemaAbstractDiscoveryMessage's): Step 1: Removing the index from the SQL engine and starting DurableBackgroundCleanupIndexTreeTask, which removes the index trees in the background; Step 1.1: DurableBack

Re: Exceeding the DataStorageConfiguration#getMaxWalArchiveSize due to historical rebalance

2021-06-21 Thread ткаленко кирилл
Created the second task by this discussion, IGNITE-14952. 17.06.2021, 14:26, "ткаленко кирилл" : > Created the first task by this discussion IGNITE-14923. > > 13.05.2021, 18:37, "Stanislav Lukyanov" : >>  What I mean by degradation when archive size < mi

Re: Exceeding the DataStorageConfiguration#getMaxWalArchiveSize due to historical rebalance

2021-06-17 Thread ткаленко кирилл
me getWalArchiveSize - I think it's a bit confusing >>>>>> (is it the current size? the minimal size? the target size?) >>>>>>  I suggest to name the property geMintWalArchiveSize. I think that this >>>>>> is exactly what it is - the mini

Re: Exceeding the DataStorageConfiguration#getMaxWalArchiveSize due to historical rebalance

2021-05-07 Thread ткаленко кирилл
do this in a >> different ticket. >> >>  Thanks, >>  Stan >> >>>  On 5 May 2021, at 19:25, Maxim Muzafarov wrote: >>> >>>  Hello, Kirill >>> >>>  +1 for this change, however, there are too many configuration settings &

Re: Exceeding the DataStorageConfiguration#getMaxWalArchiveSize due to historical rebalance

2021-05-04 Thread ткаленко кирилл
ing? > > So we will be throttling for both checkpoint page buffer and WAL limit. > > Regards, > -- > Ilya Kasnacheev > > вт, 4 мая 2021 г. в 11:29, ткаленко кирилл : > >>  Hello everybody! >> >>  At the moment, if there are partitions for the rebalance f

Exceeding the DataStorageConfiguration#getMaxWalArchiveSize due to historical rebalance

2021-05-04 Thread ткаленко кирилл
Hello everybody! At the moment, if there are partitions for the rebalance for which the historical rebalance will be used, then we reserve segments in the WAL archive (we do not allow cleaning the WAL archive) until the rebalance for all cache groups is over. If a cluster is under load during

Re: Fix force rebuild indexes

2021-03-24 Thread ткаленко кирилл
tion plan? > > I think the [1] issue may be interesting for you. > > [1] https://issues.apache.org/jira/browse/IGNITE-13056 > > On Tue, 23 Mar 2021 at 21:04, ткаленко кирилл wrote: >>  Hello everyone! >> >>  I found that a forced rebuild of indexes does not

Fix force rebuild indexes

2021-03-23 Thread ткаленко кирилл
Hello everyone! I found that a forced rebuild of indexes does not work correctly. If the indexes were rebuilt once, then nothing will happen each time a forced rebuild is attempted. Also, if during the first rebuild of indexes (before the checkpoint) we call a forced rebuild of indexes, then we

(Без темы)

2021-03-23 Thread ткаленко кирилл
Hello everyone! I found that a forced rebuild of indexes does not work correctly. If the indexes were rebuilt once, then nothing will happen each time a forced rebuild is attempted. Also, if during the first rebuild of indexes (before the checkpoint) we call a forced rebuild of indexes, then we

Re: Adding metrics of using WAL archive

2021-03-17 Thread ткаленко кирилл
Hi Nikolay! Can you do an additional review or can we merge? 15.03.2021, 08:48, "ткаленко кирилл" : > Hi Nikolay! Can you do an additional review or can we merge? > > 05.03.2021, 16:33, "Nikolay Izhikov" : >>  Yes, definitely. >> >>>   5 марта

Re: Adding metrics of using WAL archive

2021-03-14 Thread ткаленко кирилл
Hi Nikolay! Can you do an additional review or can we merge? 05.03.2021, 16:33, "Nikolay Izhikov" : > Yes, definitely. > >>  5 марта 2021 г., в 16:31, ткаленко кирилл написал(а): >> >>  Hi Nikolay, can you do a review? >> >>  02.03.2021, 18:59, &quo

Re: Adding metrics of using WAL archive

2021-03-05 Thread ткаленко кирилл
Hi Nikolay, can you do a review? 02.03.2021, 18:59, "Nikolay Izhikov" : > +1 For this. > >>  2 марта 2021 г., в 18:32, ткаленко кирилл написал(а): >> >>  Hi, Nikolay! >> >>  I thought about your proposal and offer the following two metrics: >>

Adding metrics of using WAL archive

2021-03-02 Thread ткаленко кирилл
Hi, Nikolay! I thought about your proposal and offer the following two metrics: 1)The number of bytes logged to the WAL; 2)The number of compressed bytes in the WAL. Monotonically increasing long. WDYT?

Long stop of node due to diagnostics of CorruptedTreeException

2021-02-19 Thread ткаленко кирилл
Hi everyone! I found that if an CorruptedTreeException occurs, the node can fall for a long time (more than an hour), due to the fact that the DiagnosticProcessor#dumpPageHistory can use many WAL segments, which does not seem very good. I propose to move the diagnostics to a IgniteWalConverter

Re: Adding metrics of using WAL archive

2021-02-18 Thread ткаленко кирилл
Ignite? > >>  18 февр. 2021 г., в 14:11, ткаленко кирилл >> написал(а): >> >>  Hi, Nikolay! >> >>  Have we reached a consensus? >> >>  16.02.2021, 17:09, "ткаленко кирилл" : >>>  Hi, Zhenya! >>> >>>  Users c

Re: Adding metrics of using WAL archive

2021-02-18 Thread ткаленко кирилл
Hi, Nikolay! Have we reached a consensus? 16.02.2021, 17:09, "ткаленко кирилл" : > Hi, Zhenya! > > Users can also use it, I see nothing wrong with the presence of two metrics. > > 16.02.2021, 16:50, "Zhenya Stanilovsky" : >>  Kirill, is it good practice

Re: Adding metrics of using WAL archive

2021-02-16 Thread ткаленко кирилл
independent on internal WAL >>> implementation. >>> >>>  So, I think exact number is always better to have then some approximation. >>> >>>  What do you think? >>> >>>>   15 февр. 2021 г., в 20:45, ткаленко кирилл < tkalkir...@yand

Re: Adding metrics of using WAL archive

2021-02-16 Thread ткаленко кирилл
have exact number without any suggestions or calculations. > > Moreover, «Count of bytes written in WAL» is independent on internal WAL > implementation. > > So, I think exact number is always better to have then some approximation. > > What do you think? > >>  15 ф

Re: Adding metrics of using WAL archive

2021-02-15 Thread ткаленко кирилл
hikov" : > My suggestion that «count of files» is meaningless number. > And «count of bytes written to the files» is useful number to know and use > for capacity planning.. > >>  15 февр. 2021 г., в 15:59, ткаленко кирилл >> написал(а): >> >>  Hi, Nikolay! &

Re: Adding metrics of using WAL archive

2021-02-15 Thread ткаленко кирилл
gt; >>  t also seems that it will be more natural to operate not just bytes but >> multiples of a segment. > > Can’t agree here. > From my point of view - it’s better to know exact number, not just «count of > segments». > >>  15 февр. 2021 г., в 13:00, ткаленко

Re: Adding metrics of using WAL archive

2021-02-15 Thread ткаленко кирилл
an exact number of space we need for WAL. > > How user should estimate capacity for a page memory and indexes? > > [1] https://issues.apache.org/jira/browse/IGNITE-13582 > >>  14 февр. 2021 г., в 09:48, ткаленко кирилл >> написал(а): >> >>  Hi, Nikolay! &

Re: Adding metrics of using WAL archive

2021-02-13 Thread ткаленко кирилл
gt; the archive so as not to overflow it under its load > > It still not clear for me why do we need those metrics. > Can you please, write down specific scenario - how user will use these > metrics to estimate required WAL volume? > >>  12 февр. 2021 г., в 19:35, ткаленко кирилл

Re: Adding metrics of using WAL archive

2021-02-12 Thread ткаленко кирилл
ou, please, clarify - What question about WAL user have in mind? > And what answers he(or she) gets with these new metrics? > >>  12 февр. 2021 г., в 14:26, ткаленко кирилл >> написал(а): >> >>  Hi everyone! >>  At the moment, I have not found an opportunity to e

Adding metrics of using WAL archive

2021-02-12 Thread ткаленко кирилл
Hi everyone! At the moment, I have not found an opportunity to estimate how many WAL segments fall into the archive, say per day. So I created a ticket https://issues.apache.org/jira/browse/IGNITE-14170 to add a couple of new metrics.

Re: Hard limit WAL archive size

2021-01-26 Thread ткаленко кирилл
ith the fix provided as part of IGNITE-13912 > > Regards, > Vishwas > > On Tue, 26 Jan, 2021, 21:18 ткаленко кирилл, wrote: > >>  Hello, everyone! >> >>  Currently, property DataStorageConfiguration#maxWalArchiveSize is not >>  working as expected by users. We c

Hard limit WAL archive size

2021-01-26 Thread ткаленко кирилл
Hello, everyone! Currently, property DataStorageConfiguration#maxWalArchiveSize is not working as expected by users. We can easily go beyond this limit and overflow the disk, which will lead to errors and a crash of the node. I propose to fix this behavior and not let WAL archive overflow. It

Re: [ANNOUNCE] Welcome Ivan Bessonov as a new committer

2021-01-12 Thread ткаленко кирилл
Ivan, congratulations! 12.01.2021, 14:36, "Вячеслав Коптилин" : > Hello, > > My congratulations, Ivan! May the Force be with you! > > Thanks, > S. > > вт, 12 янв. 2021 г. в 14:27, Ivan Pavlukhin : > >>  Ivan, congratulations! >> >>  2021-01-12 13:36 GMT+03:00, Kseniya Romanova : >>  > Ivan, my co

Error restarting the node with switching from disabled WAL archiving to enabled

2020-12-20 Thread ткаленко кирилл
Hello everyone! I came across the fact that if we started a node with WAL archiving disabled, and then poured data there and there were more than DataStorageConfiguration#walSegments, then when we try to restart the node with WAL archiving enabled, there will be an error. Workaround for this er

Re: Metastorage limitations

2020-12-18 Thread ткаленко кирилл
Hello everybody! If you look at the stackTrace, the error is that deserialized objects are being sent to listeners. It may be more correct to send a raw arrays of bytes, and each plugin will be able to process it if needed. 18.12.2020, 12:18, "Nikolay Izhikov" : > Hello, Denis. > > It’s a know

Re: Make GridEncryptionManager#onWalSegmentRemoved async

2020-12-13 Thread ткаленко кирилл
Hi, Pavel! I created a IGNITE-13847, but I was unable to create a PR from your branch. 13.12.2020, 18:35, "ткаленко кирилл" : > Hi, Pavel! > > I think we can use the system pool as well. I think it would be more correct > to do it in a separate ticket. Thanks! > &g

Re: Make GridEncryptionManager#onWalSegmentRemoved async

2020-12-13 Thread ткаленко кирилл
GNITE-13831 [1] on the implementation of this change. > > [1] https://issues.apache.org/jira/browse/IGNITE-13831 > > пт, 11 дек. 2020 г. в 13:23, ткаленко кирилл : >>  Hello to all! >> >>  When implementing IGNITE-13831 I was faced with deadlock. >> >>  Wh

Make GridEncryptionManager#onWalSegmentRemoved async

2020-12-11 Thread ткаленко кирилл
Hello to all! When implementing IGNITE-13831 I was faced with deadlock. When execute FileWriteAheadLogManager#rollOver, begin to clean WAL archive since we have reached the DataStorageConfiguration#maxWalArchiveSize, after deleting a segment, execute the GridEncryptionManager#onWalSegmentRemove

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
Hi, Andrey! Users expect DataStorageConfiguration#maxWalArchiveSize to mean that WAL archive will not exceed this value, but it is not. It seems that to reduce the chance of getting into a situation when we exceed WAL archive, it will be lowed when we clean it when switching to a new segment t

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
Hi, Vishwas! Speed of uploading data directly associated with the growth of WAL archive. 09.12.2020, 17:05, "vbm" : > Hi Kirill Tkalenko, > > Is there any relation to rate of ingestion of data to ignite ? > > We had seen the issue of WAL growing infinitely recently in our K8s cluster. > We were i

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
And this property IGNITE_THRESHOLD_WAL_ARCHIVE_SIZE_PERCENTAGE 09.12.2020, 16:28, "ткаленко кирилл" : > And this property IGNITE_THRESHOLD_WAL_ARCHIVE_SIZE_PERCENTAGE > > 09.12.2020, 16:25, "ткаленко кирилл" : >> Hi, Shiva! >> >> I am not aware of

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
Hi, Shiva! I am not aware of such tickets yet. Later I think I can deal with this issue. Now you can try to increase the frequency of checkpoints, the maximum WAL archive size and try to change the system property IGNITE_CHECKPOINT_TRIGGER_ARCHIVE_SIZE_PERCENTAGE. This will not solve the proble

Re: Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
Hi, Shiva! Yes, this ticket will only be about moving WAL archive cleanup. I think further it is possible to solve the problem of WAL archive overflow, but before that we need to solve several problems and deal with heuristics. 09.12.2020, 15:02, "shm" : > Hi Kirill Tkalenko > > Is this ticket i

Move WAL archive cleanup from checkpoint to rollover

2020-12-09 Thread ткаленко кирилл
Hello to all! At the moment, the archive is cleared at the end of the checkpoint, it seems it should happen in FileWriteAheadLogManager. I suggest moving it into the FileWriteAheadLogManager#rollOver when the DataStorageConfiguration#maxWalArchiveSize is reached. To do this, I created a ticket

Re: Removing MVCC public API

2020-12-08 Thread ткаленко кирилл
+1 08.12.2020, 23:47, "Andrey Mashenkov" : > +1 > > On Tue, Dec 8, 2020 at 11:22 PM Igor Seliverstov > wrote: > >>  +1 >> >>  08.12.2020 22:38, Andrey Gura пишет: >>  > +1 >>  > >>  > On Tue, Dec 8, 2020 at 10:02 PM Nikolay Izhikov >>  wrote: >>  >> +1 >>  >> >>  >>> 8 дек. 2020 г., в 21:54, Va

Re: [DISCUSSION] Modules organization in Ignite 3

2020-12-08 Thread ткаленко кирилл
+1 for modules 08.12.2020, 16:02, "Andrey Gura" : > Definitely agree with Alexey. Separating API declaration from > implementation could really improve system design and avoid coupling. > > About extending @IgniteExperimental annotation. It doesn't look good > to me. We should consider any API eit

Re: Replace Future.get with Future.get(int timeout) in tests

2020-12-08 Thread ткаленко кирилл
It seems to be a good topic, but it seems to be left to the reviewer's discretion. 08.12.2020, 18:36, "Nikolay Izhikov" : > Hello, Igniters. > > Currently, we have a lot of usages `Future.get` without a timeout in tests. > In case the test that uses `Future.get` is flaky it can lead to the whole

Re: [DISCUSSION] Java 11 for Ignite 3.0 development

2020-12-08 Thread ткаленко кирилл
+1 08.12.2020, 12:48, "Philipp Masharov" : > Hello! > > Andrey's arguments are solid. > > On Tue, Dec 8, 2020 at 12:23 PM Pavel Tupitsyn wrote: > >>  +1, Java 11 seems to be the only right choice at the moment. >> >>  On Tue, Dec 8, 2020 at 12:08 PM Alexey Zinoviev >>  wrote: >> >>  > I totally

Remove ability to delete segments from the middle of WAL archive

2020-12-04 Thread ткаленко кирилл
Hello to all! I found that we have the option to remove segments from the middle of WAL archive and thus make it invalid due to gaps. I'll fix it in https://issues.apache.org/jira/browse/IGNITE-13815.

Re: Disable MVCC test suites

2020-12-02 Thread ткаленко кирилл
+1 02.12.2020, 16:47, "Nikolay Izhikov" : >>  Why should we waste every contributor's time? IMHO, MVCC suites are useless >> and everyone just pushes "re-run possible blockers" button and doesn't care >> about MVCC tests at all. > > +1. > >>  2 дек. 2020 г., в 16:39, Вячеслав Коптилин >> напис

Re: [VOTE][EXTENSION] Release Apache Ignite Streaming extensions 1.0.0 RC1

2020-11-30 Thread ткаленко кирилл
+1 30.11.2020, 16:15, "Andrey Gura" : > +1 > > On Tue, Nov 24, 2020 at 3:24 AM Saikat Maitra wrote: >>  +1 >> >>  Denis, Alex >> >>  I have raised PR for the docs for Maven artifactIds and versions on the >>  documentation pages of the integrations. >> >>  PR : https://github.com/apache/ignite/pu

Re: Why WAL archives enabled by default?

2020-11-09 Thread ткаленко кирилл
e it can cause >>  > some performance problems. But as an option, >>  > it can be used and should be switchable. >>  > >>  > пт, 6 нояб. 2020 г. в 12:36, Ivan Daschinsky : >>  > >>  > > Kirill, how your approach will help if user tuned a clus

Re: Why WAL archives enabled by default?

2020-11-06 Thread ткаленко кирилл
nd tuning. > There are a lot of similar topics for almost every DB. [3] > > [1] -- https://man7.org/linux/man-pages/man2/ftruncate.2.html > [2] -- https://man7.org/linux/man-pages/man2/mmap.2.html > [3] -- > https://www.google.com/search?q=pg_wal%2Fxlogtemp+no+space+left+on+devi

Re: Why WAL archives enabled by default?

2020-11-05 Thread ткаленко кирилл
segment if > and only if it is not required for recovery. If last checkpoint marker > points to segment > with lower index, we cannot delete any segment with higher index. So the > only moment where we can remove truncate segments is a finish of checkpoint. > > пт, 6 нояб. 202

Re: Why WAL archives enabled by default?

2020-11-05 Thread ткаленко кирилл
Hello, everybody! As far as I know, WAL archive is used for PITP(GridGain feature) and historical rebalancing. Facundo seems to have a problem with running out of directory (/opt/work/walarchive) space. Currently, WAL archive is cleared at the end of checkpoint. Potentially long transaction ma

Re: [VOTE] Release Apache Ignite 2.9.0 RC3

2020-10-14 Thread ткаленко кирилл
+1 14.10.2020, 17:07, "Pavel Tupitsyn" : > +1 (binding) > > - Checked .NET binaries, examples, documentation > - Started a mixed cluster of .NET and Java nodes > - Compiled Ignite from sources > > On Tue, Oct 13, 2020 at 10:48 PM Denis Magda wrote: > >>  +1 (binding) >> >>  Started a two-node Ign

Re: WalPointer interface - do we need it?

2020-10-02 Thread ткаленко кирилл
Kolya, hi! This seems to sound good! 02.10.2020, 11:43, "Nikolay Izhikov" : > Hello, Igniters. > > Crawled through the WAL code and wonder - do we really need WALPointer > interface? > As far as I can see it has only one implementation FileWALPointer. > And all usages of WALPointer expect FileWAL

Re: [DISCUSSION] Add autocompletion for commands in control.sh

2020-09-08 Thread ткаленко кирилл
hint a shell what our executable can do so > that it can discover and auto-complete our control.sh > > Regards, > -- > Ilya Kasnacheev > > пн, 7 сент. 2020 г. в 17:47, ткаленко кирилл : > >>  Hello, folks! >> >>  I spent time to analyze the possibility of

[DISCUSSION] Add autocompletion for commands in control.sh

2020-09-07 Thread ткаленко кирилл
Hello, folks! I spent time to analyze the possibility of adding auto completion for the "control.sh" with the [1]. To do this, at the beginning, we need to adapt the "control.sh" code to [1], then we can automatically create a "bash completion script" via [2], and then install it, for example,

Re: [DISCUSSION] control.sh argument to enable experimental commands

2020-09-07 Thread ткаленко кирилл
07.09.2020, 13:26, "ткаленко кирилл" : > Adding option "—enable-experimental". > > 07.09.2020, 13:22, "ткаленко кирилл" : >> Hi, Nikolay! >> >> It seems that you shouldn't just open experimental commands, that's why they >&g

Re: [DISCUSSION] control.sh argument to enable experimental commands

2020-09-07 Thread ткаленко кирилл
Hi, Nikolay! It seems that you shouldn't just open experimental commands, that's why they are experimental. 07.09.2020, 13:03, "Philipp Masharov" : > I will try it. Including information about experimental commands into > documentation sounds like a good idea. Am I need to create a Jira ticket?

Re: [DISCUSSION] Add index rebuild time metrics

2020-08-10 Thread ткаленко кирилл
divide to sum of all cache > sizes. > On the other hand, % of index rebuild progress is self-descriptive. I don't > understand why we tend to make user's life harder. > > -- > Best regards, > Ivan > > On Mon, Aug 10, 2020 at 8:53 PM ткаленко кирилл > wrote:

Re: [DISCUSSION] Add index rebuild time metrics

2020-08-10 Thread ткаленко кирилл
t;  > or updates a set of all indexes that need to catch up (see >>  > IndexRebuildPartialClosure). GIven that, I do not see any need for >>  > per-index rebuild status as this status will be updated for all outdated >>  > indexes simultaneously. >>  >

Re: [DISCUSSION] Cache warmup

2020-08-10 Thread ткаленко кирилл
Great! Then I proceed to ticket https://issues.apache.org/jira/browse/IGNITE-13345. 10.08.2020, 16:30, "Stanislav Lukyanov" : > All of this looks awesome, covers the use cases I know about. > Thanks! > > Stan > >>  On 10 Aug 2020, at 15:39, ткаленко кирил

Re: [DISCUSSION] Cache warmup

2020-08-10 Thread ткаленко кирилл
, when starting warm-up, region configuration is taken at beginning, and if it is not present, it is taken from default. And if we don't want to warm up region, we set NoOp. 10.08.2020, 10:20, "ткаленко кирилл" : > Hi, Stan! > > As a result of personal correspondence I rea

Re: [DISCUSSION] Cache warmup

2020-08-10 Thread ткаленко кирилл
. 04.08.2020, 13:29, "ткаленко кирилл" : > Hi, Slava! > > Thank you for looking at the offer and making fair comments. > > I personally discussed with Anton and Alexey because they are author and > sponsor of "IEP-40" and we found out that point 2 in it i

Re: [DISCUSSION] Cache warmup

2020-08-07 Thread ткаленко кирилл
then I should be putting them in different regions. > Do you see a good way to have distinct warmup configuration for different > regions while the config is on IgniteConfiguration level? > > Thanks, > Stan > >>  On 6 Aug 2020, at 15:39, ткаленко кирилл wrote: >> >>

Re: [DISCUSSION] Cache warmup

2020-08-06 Thread ткаленко кирилл
Hello, Alexey! Your comments are fair. 05.08.2020, 15:51, "Alexey Goncharuk" : > Kirill, > > Thank you for driving this discussion and implementation. > > A few points from my side: > * Agree that it will be best to keep the strategy interface private because > it will be very dependent on the pe

Re: [DISCUSSION] Cache warmup

2020-08-04 Thread ткаленко кирилл
gt; > How hard would it be to implement a "load all indexes, metapages and > freelists" strategy in addition to the "load everything"? > I think it would be an MVP for environments with a datasets larger than RAM. > A "load everything" strategy will not wor

Re: [DISCUSSION] Cache warmup

2020-08-04 Thread ткаленко кирилл
gt; When loading of all cache data is required, it can be done by running a > local scan query. It will iterate through all data pages and result in > their allocation in memory. > > So, I don't really see a scenario when the suggested API will help. Do you > have a suitable use-case that

Re: [DISCUSSION] Cache warmup

2020-08-04 Thread ткаленко кирилл
gt;  >import org.apache.ignite.IgniteCheckedException; >>  >import org.apache.ignite.lang.IgniteFuture; >>  > >>  >public interface CacheWarmupper { >>  > /** >>  > * Warmup cache. >>  > * >>  > * @param cachename Cache name. >>

Re: [DISCUSSION] Cache warmup

2020-08-04 Thread ткаленко кирилл
* Warmup cache. >>    * >>    * @param cachename Cache name. >>    * @return Future cache warmup. >>    * @throws IgniteCheckedException If failed. >>    */ >>   IgniteFuture warmup(String cachename) throws >> IgniteCheckedException

Re: [DISCUSSION] Cache warmup

2020-08-04 Thread ткаленко кирилл
; > > public interface CacheWarmupper { > /** >  * Warmup cache. >  * >  * @param cachename Cache name. >  * @return Future cache warmup. >  * @throws IgniteCheckedException If failed. >  */ > IgniteFuture warmup(String cachename) throws Ignit

[DISCUSSION] Cache warmup

2020-07-27 Thread ткаленко кирилл
Now, after restarting node, we have only cold caches, which at first requests to them will gradually load data from disks, which can slow down first calls to them. If node has more RAM than data on disk, then they can be loaded at start "warmup", thereby solving the issue of slowdowns during fir

Re: Extended logging for rebalance performance analysis

2020-07-27 Thread ткаленко кирилл
received in B,KB,MB,GB by historical mode fullPartitions=1 - total number of partitions received by full mode fullEntries=30 - total number of entries received by full mode fullBytesRcvd=3 KB - total number of bytes received in B,KB,MB,GB by full mode 27.07.2020, 11:50, "ткаленко к

Re: Extended logging for rebalance performance analysis

2020-07-03 Thread ткаленко кирилл
Sorry, forget. [1] - org.apache.ignite.internal.processors.cache.CacheGroupsMetricsRebalanceTest#testCacheGroupRebalance 03.07.2020, 17:20, "ткаленко кирилл" : > Hi, Stan! > > I don't understand you yet. > > Now you can use metrics as it was done in the test [1].

Re: Extended logging for rebalance performance analysis

2020-07-03 Thread ткаленко кирилл
s=400, duration=41ms, bytesRcvd=40,4 KB] These messages have a common parameter rebalanceId=1. 03.07.2020, 16:48, "Stanislav Lukyanov" : >>  On 3 Jul 2020, at 09:51, ткаленко кирилл wrote: >> >>  To calculate the average value, you can use the existing

Re: [DISCUSSION] Add index rebuild time metrics

2020-07-03 Thread ткаленко кирилл
e > easiest to understand; we may add multiple metrics though. > - For "time per each index" I'd add detailed log messages stating how long > did it take to build a particular index > > Thanks, > Stan > >>  On 26 Jun 2020, at 12:49, ткаленко кирилл wro

Re: Extended logging for rebalance performance analysis

2020-07-02 Thread ткаленко кирилл
s slow due to uneven suppliers >>  distribution or network issues. >>  Option to disable the feature in runtime shouldn't be used often, but it >>  will keep us on the safe side in case something goes wrong. >>  The format described in >>  https://issues.apache.org/

[DISCUSSION] Add index rebuild time metrics

2020-06-26 Thread ткаленко кирилл
Hi, Igniters. I would like to know if it is possible to estimate how much the index rebuild will take? At the moment, I have found the following metrics [1] and [2] and since the rebuild is based on caches, I think it would be useful to know how many records are processed in indexing. This way

Re: Extended logging for rebalance performance analysis

2020-06-23 Thread ткаленко кирилл
Hello, Alexey! Currently there is no way to disable / enable it, but it seems that the logs will not be overloaded, since Alexei Scherbakov offer seems reasonable and compact. Of course, you can add disabling / enabling statistics collection via jmx for example. 23.06.2020, 18:47, "Alexey Gonc

Re: Extended logging for rebalance performance analysis

2020-06-23 Thread ткаленко кирилл
eted rebalance chain: [rebalanceId=2, partitions=10, entries=200, duration=50ms, bytesRcvd=10M] Any comments or suggestions? [1] - https://issues.apache.org/jira/browse/IGNITE-12080 20.05.2020, 23:08, "ткаленко кирилл" : > Hello, Alexey! Unfortunately, my response was delayed. > >

Re: [DISCUSSION] Add autocompletion for commands in control.sh

2020-06-19 Thread ткаленко кирилл
Hi Alexey! I will take a short pause before the implementation and try to answer all your questions. 18.06.2020, 18:05, "Alexey Goncharuk" : > Definitely a +1 from me for moving the CLI tooling to a separate module. > > As for the autocompletion - can you elaborate how it works? Will it require

Re: [DISCUSSION] Add autocompletion for commands in control.sh

2020-06-19 Thread ткаленко кирилл
> Evgenii > > пт, 5 июн. 2020 г. в 01:59, ткаленко кирилл : > >>  Folks have created a ticket [1]. >> >>  1 - https://issues.apache.org/jira/browse/IGNITE-13120 >> >>  02.06.2020, 16:48, "ткаленко кирилл" : >>  > Maxim

Re: [DISCUSSION] Add autocompletion for commands in control.sh

2020-06-05 Thread ткаленко кирилл
Folks have created a ticket [1]. 1 - https://issues.apache.org/jira/browse/IGNITE-13120 02.06.2020, 16:48, "ткаленко кирилл" : > Maxim I suggested moving control.sh in a separate module, are we talking > about the same thing? > > 02.06.2020, 16:15, "Maxim Muzafa

Re: [DISCUSS] Ignite process exit code on node stop by failure handler

2020-06-04 Thread ткаленко кирилл
- org.apache.ignite.failure.StopNodeOrHaltFailureHandler 02.06.2020, 16:59, "ткаленко кирилл" : > Hello, Alexey! > > I didn't quite understand about merge. > > If we use StopNodeOrHaltFailureHandler with tryStop=true, then if we don't > stop node by timeout, we will terminate jv

Re: [DISCUSS] Ignite process exit code on node stop by failure handler

2020-06-02 Thread ткаленко кирилл
Hello, Alexey! I didn't quite understand about merge. If we use StopNodeOrHaltFailureHandler with tryStop=true, then if we don't stop node by timeout, we will terminate jvm. Or do you suggest only stopping the node in StopNodeFailureHandler and terminate jvm in StopNodeOrHaltFailureHandler? o

Re: [DISCUSSION] Add autocompletion for commands in control.sh

2020-06-02 Thread ткаленко кирилл
Maxim I suggested moving control.sh in a separate module, are we talking about the same thing? 02.06.2020, 16:15, "Maxim Muzafarov" : > Folks, > > +1 > > However, AFAIK control.sh is the part of the ignite-core module with > zero dependencies from external resources. > Would it be better going th

  1   2   >