Hi Justin, thanks for your reply. There is just a latest point I would need to clarify. Do the "max-disk-usage"/"min-disk-free" settings apply also to the persisted messages, and not only to the paging behavior, right? I will try to explain my test scenario a little more:
* we are working with the broker in the k8s scenario (broker + operator), where we enabled the persistence * this resulted into a "max-disk-usage" set by default to 90 on the broker.xml file and a persistence volume of 2Gi reserved by default for the persistence * we are simulating some MQTT messages sent with qos 2, while some subscribers (that subscribed to the corresponding addresses with qos2) are disconnected; * since the clients are disconnected, we observed that the messages sent were immediately stored in the "/opt/amq-broker/data/journal" folder, as expected (waiting for the subscribers to reconnect again) * so, we would like to understand if this persistence behavior, even without passing for paging, can be controlled with the "max-disk-usage" that you mentioned before, to make sure we don't saturate the 2Gi of the persistence volume Many thanks for your help ________________________________ Da: Justin Bertram <jbert...@apache.org> Inviato: venerdì 1 marzo 2024 15:36 A: users@activemq.apache.org <users@activemq.apache.org> Oggetto: Re: Persistence: is there way to limit the maximum storage size? The journal exists in order to restore the persistent messages in the broker's memory when it restarts. Therefore, the amount of data in the journal is directly proportional to the amount of data in memory. Therefore, if you want to control the amount of data in the journal you can do so by controlling the amount of data in memory and to do that you configure the size of your addresses (e.g. per-address via max-size-bytes [1] or max-size-messages [1] or globally via global-max-size [2] or global-max-messages [3]). When the address gets full the address-full-policy is enforced which might also result in writing data to disk (i.e. if using PAGE). In order to prevent the disk from filling up you can use max-disk-usage [4] or min-disk-free [5]. Justin [1] https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fpaging.html%23configuration-2&data=05%7C02%7C%7C2de9c226f3f447fd6c8408dc3a05b599%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638449043388777235%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=MMc1VrmKvNJTRptznK3RbB%2FOiHZ1dKvj1i2tY30PaaE%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/paging.html#configuration-2> [2] https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fpaging.html%23global-max-size&data=05%7C02%7C%7C2de9c226f3f447fd6c8408dc3a05b599%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638449043388785214%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=fYEseAt4%2B7zNBRFHw%2BcSxRhN%2B2a4OQzbqNKL3%2FEAZio%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/paging.html#global-max-size> [3] https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fpaging.html%23global-max-messages&data=05%7C02%7C%7C2de9c226f3f447fd6c8408dc3a05b599%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638449043388790822%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=%2B76rrOHgiHLFDTVnEJgxKHTAcxNJcK0YuKPNZ23DZtg%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/paging.html#global-max-messages> [4] https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fpaging.html%23max-disk-usage&data=05%7C02%7C%7C2de9c226f3f447fd6c8408dc3a05b599%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638449043388795441%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=nrO53iiWjCUH9i2Lm83kKLh8Q4zpUdXmydSX9oCR5P0%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/paging.html#max-disk-usage> [5] https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fpaging.html%23minimum-disk-free&data=05%7C02%7C%7C2de9c226f3f447fd6c8408dc3a05b599%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638449043388799925%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=rPySymSBCgyAW4%2Fb%2FTwriUGEqxbxCdBCGCcitiayzRo%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/paging.html#minimum-disk-free> On Fri, Mar 1, 2024 at 5:05 AM andrea bisogno <bisoma...@hotmail.it> wrote: > Hi support, > I would need some insights about the Artemis Broker Persistence feature. > Looking into the documentation at > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fpersistence.html&data=05%7C02%7C%7C2de9c226f3f447fd6c8408dc3a05b599%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638449043388804379%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=GxzAhBRvY2pm4cnSHZAwX81lLGIiZyc5WPvAC3r1Y0M%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/persistence.html>, > I was not able to find anything about possible limits to disk size usage > once te File Journal persistence option is enabled. > We would like to understand if (and how) it is possible to limit the > number of journal files produced, and so be able to limit the disk size > used by the persistence. > Is it possible to configure, for example, a maximum numer of Journal > files, and make the broker reuse the first one (i.e. the oldest file) once > the max number of files is reached, overwriting the oldest data? > Basically, it would be important for us being able to avoid disk > saturation, to avoid dangerous situation where the persistence can cause > unexpected issues due to too much storage space used. > > Many thanks in advance for your support, > > Andrea >
