If there is a limit to oldreplica files, we could determine the size of disk we should use.
Em seg., 25 de out. de 2021 às 09:49, Saulo Carvalho <saul...@gmail.com> escreveu: > "When you say, "we had something like 110MB ~ 300MB of messages," how are > you measuring this? Are you counting *all* the data in the data directory > (e.g. large messages, paging, etc.)?" > > Yes. I'm counting all the data cause i check all the disc. When Artemis > restart, it's cleaning oldreplica files and turns into 110MB~300MB. > > "Also, when you, "it generated 20 GB or 80 GB on oldreplica," how are you > measuring this? Can you provide a listing of the files?" > > Cause it's filling all the disc. We improve to 80GB and happened again. > > "Can you provide steps to reproduce the behavior you're observing?" > > Sure! We start main and replica and start producing. When we have > something like 30k messages and without stopping the producer, we kill the > replica. When we do it, Artemis fill all the disc with oldreplica files. > When we changed the max-replica-files to 0 as you said, it's removing when > restart but with the default value the cleaning has to be manual. > > Thanks for your help! > > > > > Em sáb., 23 de out. de 2021 às 09:12, Justin Bertram <jbert...@apache.org> > escreveu: > >> When you say, "we had something like 110MB ~ 300MB of messages," how are >> you measuring this? Are you counting *all* the data in the data directory >> (e.g. large messages, paging, etc.)? >> >> Also, when you, "it generated 20 GB or 80 GB on oldreplica," how are you >> measuring this? Can you provide a listing of the files? >> >> Can you provide steps to reproduce the behavior you're observing? >> >> >> Justin >> >> On Fri, Oct 22, 2021 at 8:08 AM Saulo Carvalho <saul...@gmail.com> wrote: >> >> > Hi! Is there some explanation for this size of backup? Maybe some limit. >> > >> > Em ter., 19 de out. de 2021 às 11:31, Saulo Carvalho <saul...@gmail.com >> > >> > escreveu: >> > >> > > Hi Gary! >> > > >> > > Thanks for the explanation. But why is so bigger than the journal >> size? >> > > Sometimes we had something like 110MB ~ 300MB of messages and it >> > generated >> > > 20 GB or 80 GB on oldreplica. Is it normal? >> > > We decided to rollback to 2.17 when we hadn't this problem and test >> 2.19. >> > > >> > > >> > > Em ter., 19 de out. de 2021 às 06:20, Gary Tully < >> gary.tu...@gmail.com> >> > > escreveu: >> > > >> > >> use max-saved-replicated-journals-size=0 may be your best option. >> > >> >> > >> first: understand the value in max-saved-replicated-journals-size > 0 >> > >> >> > >> I can be used to limit data loss in the case of catastrophic failure. >> > >> One scenario where it can be useful is as follows: >> > >> A primary is active, a backup begins to replicate and fails during >> > >> initial synchronisation at say, 90% complete. >> > >> The backup restart and saves the partial replica, then starts to >> > >> replicate again (from the beginning), it gets to 10% and the primary >> > >> goes on fire, all data from the primary machine is lost! >> > >> Now you have a backup with a 10% replica and an old replica with 90% >> > >> of the data. There is less data loss if the backup is activated with >> > >> the saved old 90% replica rather than the current 10% replica. >> > >> >> > >> it is a trade off of disk usage against the possibility of recovering >> > >> from a total failure of the primary during replication. >> > >> >> > >> /gary >> > >> >> > >> On Mon, 18 Oct 2021 at 14:17, Saulo Carvalho <saul...@gmail.com> >> wrote: >> > >> > >> > >> > Hi Justin and folks. >> > >> > >> > >> > I tried this configuration and the Artemis seems to be more >> resilient >> > >> (when >> > >> > restart, it delete these files), but stil creates oldreplica files >> and >> > >> fill >> > >> > all the disc. >> > >> > Is there something more for it does not create thes files or limit >> > these >> > >> > creation? Maybe in the new version?! >> > >> > >> > >> > Thanks for your help! >> > >> > >> > >> > >> > >> > Em qui., 14 de out. de 2021 às 18:34, Saulo Carvalho < >> > saul...@gmail.com >> > >> > >> > >> > escreveu: >> > >> > >> > >> > > Thanks Justin! I will try this! >> > >> > > >> > >> > > Em qui., 14 de out. de 2021 às 17:55, Justin Bertram < >> > >> jbert...@apache.org> >> > >> > > escreveu: >> > >> > > >> > >> > >> I referred you to the max-saved-replicated-journals-size >> > >> configuration >> > >> > >> parameter on the Jira you opened for this [1]. Have you adjusted >> > this >> > >> > >> parameter according to your use-case? >> > >> > >> >> > >> > >> For what it's worth, I wouldn't expect 20GB to matter that much. >> > Disk >> > >> > >> space >> > >> > >> these days is super cheap. Perhaps you're running in some kind >> of >> > >> > >> especially constrained environment? >> > >> > >> >> > >> > >> >> > >> > >> Justin >> > >> > >> >> > >> > >> [1] https://issues.apache.org/jira/browse/ARTEMIS-3527 >> > >> > >> >> > >> > >> On Thu, Oct 14, 2021 at 12:44 PM Saulo Carvalho < >> saul...@gmail.com >> > > >> > >> > >> wrote: >> > >> > >> >> > >> > >> > Hi! >> > >> > >> > Can anyone help me with this problem? >> > >> > >> > >> > >> > >> > Artemis in version 2.18.0 is creating oldreplica files and >> it's >> > >> filling >> > >> > >> all >> > >> > >> > the disc. Is there some flag to turn it off or some way to >> limit >> > >> these >> > >> > >> > files? We have 1 master and 1 replica in our cluster. >> > >> > >> > Thanks! >> > >> > >> > -- >> > >> > >> > Saulo de Carvalho Neto >> > >> > >> > Software Engineer >> > >> > >> > >> > >> > >> >> > >> > > >> > >> > > >> > >> > > -- >> > >> > > Saulo de Carvalho Neto >> > >> > > Software Engineer >> > >> > > >> > >> > >> > >> > >> > >> > -- >> > >> > Saulo de Carvalho Neto >> > >> > Software Engineer >> > >> >> > > >> > > >> > > -- >> > > Saulo de Carvalho Neto >> > > Software Engineer >> > > >> > >> > >> > -- >> > Saulo de Carvalho Neto >> > Software Engineer >> > >> > > > -- > Saulo de Carvalho Neto > Software Engineer > -- Saulo de Carvalho Neto Software Engineer
