To be clear, my previous explanation assumes that nothing outside the broker is removing the persistent data. If you are operating in a containerized/virtual/cloud environment where the persistent data is being deleted then clearly no data will be available when the broker restarts. If this is the case then you need to use a persistent volume to store the file-based journal as mentioned by another user on this thread.
Justin On Wed, Sep 13, 2023 at 12:29 PM Justin Bertram <jbert...@apache.org> wrote: > If you are sending one or more durable messages in your transaction then > the broker will attempt to write them to storage when they arrive. If the > broker fails to write the messages, for whatever reason, then the sender > will receive an exception when it attempts to commit the transaction. If > the broker succeeds in writing the messages to storage but the broker stops > for whatever reason before the messages are consumed then when the broker > restarts those messages will be loaded from storage and made available to > consumers. > > This is the same whether you're using the file-based journal or a database > to store the messages. > > > Justin > > On Wed, Sep 13, 2023 at 11:57 AM Shivang Modi <sm...@provenir.com.invalid> > wrote: > >> Hi Justin, >> >> We are using Artemis docker image and start kubernetes pods with it. We >> have >> one sender which will write messages on queue and one receiver which will >> read messages queue. >> Now due to any reason, kubernetes queue pod gets restarted so before >> restarts whatever transactions gets enqueued by sender but not read by >> receiver, will that persisted with file storage and If yes, in any >> scenario >> file storage, chances of losing transactions? >> >> Thanks, >> Shivang. >> >> -----Original Message----- >> From: Justin Bertram <jbert...@apache.org> >> Sent: Wednesday, September 13, 2023 10:23 PM >> To: users@activemq.apache.org >> Subject: Re: Artemis File Storage Persistence vs JDBC Persistence >> >> I'm not really sure what you're asking. Are you asking whether you should >> use the file-based journal or a database if you have 100k transactions? >> >> To be clear, what is "best" in one situation is often not "best" in >> another. >> Everything depends on the specifics of your particular use-case. >> >> >> Justin >> >> On Wed, Sep 13, 2023 at 11:47 AM Shivang Modi <sm...@provenir.com.invalid >> > >> wrote: >> >> > If scenario is no loss transactions 100% if queue goes down whatever >> > transactions gets enqueued, should get dequeued once queue comes up, >> > we have 100k transactions or more need to flow up via queue. What >> > would be best in such scenarios? >> > >> > Thanks, >> > Shivang >> > >> > -----Original Message----- >> > From: Justin Bertram <jbert...@apache.org> >> > Sent: Wednesday, September 13, 2023 8:38 PM >> > To: users@activemq.apache.org >> > Subject: Re: Artemis File Storage Persistence vs JDBC Persistence >> > >> > When deciding between the file-based journal on local storage versus a >> > remote database I think the three main considerations are: >> > >> > - Performance >> > - Infrastructure >> > - Reliability >> > >> > The file-based journal on local storage will be faster than a database >> > for a few reasons: >> > - The storage is local so there's no network latency to deal with. >> > - The file-based journal was specifically written and heavily >> > optimized for the message broker use-case. >> > >> > The file-based journal on local storage requires less infrastructure >> > than a database since most servers already come with local storage. >> > Using a database requires provisioning additional hardware as well as >> > installing and maintaining a distinct piece of software. This can be >> > costly both in terms of money and man-power. >> > >> > Generally speaking, local storage is always going to be more reliable >> > than a remote database simply because it's much simpler (i.e. no >> > network, no database with its own maintenance requirements, etc.). >> > This simplicity tends to reduce downtime. >> > >> > In my experience the only folks who choose to use a database are those >> > in an environment where there's already been a substantial investment >> > in an enterprise database and stuff like automated backups, redundant >> > networking, data replications, etc. are available. >> > >> > No matter which option you choose, the broker is written so that you >> > should >> > *never* lose messages. >> > >> > >> > Justin >> > >> > >> > >> > On Wed, Sep 13, 2023 at 7:14 AM Shivang Modi >> > <sm...@provenir.com.invalid> >> > wrote: >> > >> > > Hi Team, >> > > >> > > >> > > >> > > Can anyone share pros and cons in depth between both. I see only >> > > file storage is faster than JDBC storage. Is there any disadvantage >> > > of File Storage like losing the enqueued data or anything? >> > > >> > > >> > > >> > > Thanks, >> > > >> > > Shivang. >> > > >> > > -- >> > > *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION >> > > intended solely for the use of the addressee(s). If you are not the >> > > intended recipient, please notify the sender by e-mail and delete >> > > the original message. Further, you are not to copy, disclose, or >> > > distribute this e-mail or its contents to any other person and any >> > > such actions maybe unlawful*. >> > > This e-mail may contain viruses. Provenir has taken every reasonable >> > > precaution to minimize this risk, but is not liable for any damage >> > > you may sustain as a result of any virus in this e-mail. You should >> > > carry out your own virus checks before opening the e-mail or >> attachment. >> > > Provenir reserves the right to monitor and review the content of all >> > > messages sent to or from this e-mail address. Messages sent to or >> > > from this e-mail address may be stored on the Provenir e-mail system. >> > > >> > >> > -- >> > *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended >> > solely for the use of the addressee(s). If you are not the intended >> > recipient, please notify the sender by e-mail and delete the original >> > message. Further, you are not to copy, disclose, or distribute this >> > e-mail or its contents to any other person and any such actions maybe >> > unlawful*. >> > This e-mail may contain viruses. Provenir has taken every reasonable >> > precaution to minimize this risk, but is not liable for any damage you >> > may sustain as a result of any virus in this e-mail. You should carry >> > out your own virus checks before opening the e-mail or attachment. >> > Provenir reserves the right to monitor and review the content of all >> > messages sent to or from this e-mail address. Messages sent to or from >> > this e-mail address may be stored on the Provenir e-mail system. >> > >> > >> >> -- >> *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended >> solely for the use of the addressee(s). If you are not the intended >> recipient, please notify the sender by e-mail and delete the original >> message. Further, you are not to copy, disclose, or distribute this >> e-mail >> or its contents to any other person and any such actions maybe unlawful*. >> This e-mail may contain viruses. Provenir has taken every reasonable >> precaution to minimize this risk, but is not liable for any damage you >> may >> sustain as a result of any virus in this e-mail. You should carry out >> your >> own virus checks before opening the e-mail or attachment. Provenir >> reserves >> the right to monitor and review the content of all messages sent to or >> from >> this e-mail address. Messages sent to or from this e-mail address may be >> stored on the Provenir e-mail system. >> >>
