Hello, Kirill Your conclusions still not clear for me.
> It is not possible for us to estimate how much space a user will need in the > archive so as not to overflow it under its load > We take the maximum 44 and multiply it by a > DataStorageConfiguration#getWalSegmentSize Why you take a single day (24h) for a standard period? Is there any rationale behind this? 1. We have `walAutoArchiveAfterInactivity` property. So WAL segment can have a size less than the maximum. 2. For CDC feature I want to introduce «WAL force rollover timeout» to make data available for a consumer in a guaranteed period [1]. Why does the user want to estimate those numbers in the first place? Are we talking about some kind of capacity planning? If yes, then maybe it will be better to have a metric for a count of bytes written in the WAL? With it, we will have an exact number of space we need for WAL. How user should estimate capacity for a page memory and indexes? [1] https://issues.apache.org/jira/browse/IGNITE-13582 > 14 февр. 2021 г., в 09:48, ткаленко кирилл <tkalkir...@yandex.ru> написал(а): > > Hi, Nikolay! > > The user will be able to take the getLastArchivedSegmentIndex every day and > remember it and do it, say, for several days. > > For example, when starting the application, the getLastArchivedSegmentIndex > is 0, then at the end of the first day the value will be 30 at the end of the > second 55 and at the end of the third 99. > It turns out that 30 segments were used for the first day, 25 for the second > and 44 for the third. We take the maximum 44 and multiply it by a > DataStorageConfiguration#getWalSegmentSize, and we get the possible maximum > that the archive overflow was the least likely. If the user uses compression, > then it can be subtracted from the result (result * > getMaxSizeCompressedArchivedSegment). > > 13.02.2021, 10:47, "Nikolay Izhikov" <nizhi...@apache.org>: >> Hello, Kirill. >> >>> It is not possible for us to estimate how much space a user will need in >>> the archive so as not to overflow it under its load >> >> It still not clear for me why do we need those metrics. >> Can you please, write down specific scenario - how user will use these >> metrics to estimate required WAL volume? >> >>> 12 февр. 2021 г., в 19:35, ткаленко кирилл <tkalkir...@yandex.ru> >>> написал(а): >>> >>> Hi, Nikolay! >>> >>> It is not possible for us to estimate how much space a user will need in >>> the archive so as not to overflow it under its load. And the proposed >>> metrics will allow you to make a rough estimate. >>> >>> 12.02.2021, 17:23, "Nikolay Izhikov" <nizhi...@apache.org>: >>>> Hello, Kirill. >>>> >>>> Can you, please, clarify - What question about WAL user have in mind? >>>> And what answers he(or she) gets with these new metrics? >>>> >>>>> 12 февр. 2021 г., в 14:26, ткаленко кирилл <tkalkir...@yandex.ru> >>>>> написал(а): >>>>> >>>>> Hi everyone! >>>>> At the moment, I have not found an opportunity to estimate how many WAL >>>>> segments fall into the archive, say per day. >>>>> So I created a ticket >>>>> https://issues.apache.org/jira/browse/IGNITE-14170 to add a couple of new >>>>> metrics.
