You might be hitting the ephemeral nature of Kubernetes on-disk files. See Kubernetes documentation.
https://kubernetes.io/docs/concepts/storage/volumes/ Youyu Information Classification *General -----Original Message----- From: Justin Bertram <jbert...@apache.org> Sent: Wednesday, September 13, 2023 1:29 PM To: users@activemq.apache.org Subject: [EXTERNAL] Re: Artemis File Storage Persistence vs JDBC Persistence If you are sending one or more durable messages in your transaction then the broker will attempt to write them to storage when they arrive. If the broker fails to write the messages, for whatever reason, then the sender will receive an exception when it attempts to commit the transaction. If the broker succeeds in writing the messages to storage but the broker stops for whatever reason before the messages are consumed then when the broker restarts those messages will be loaded from storage and made available to consumers. This is the same whether you're using the file-based journal or a database to store the messages. Justin On Wed, Sep 13, 2023 at 11:57 AM Shivang Modi <sm...@provenir.com.invalid> wrote: > Hi Justin, > > We are using Artemis docker image and start kubernetes pods with it. > We have one sender which will write messages on queue and one receiver > which will read messages queue. > Now due to any reason, kubernetes queue pod gets restarted so before > restarts whatever transactions gets enqueued by sender but not read by > receiver, will that persisted with file storage and If yes, in any > scenario file storage, chances of losing transactions? > > Thanks, > Shivang. > > -----Original Message----- > From: Justin Bertram <jbert...@apache.org> > Sent: Wednesday, September 13, 2023 10:23 PM > To: users@activemq.apache.org > Subject: Re: Artemis File Storage Persistence vs JDBC Persistence > > I'm not really sure what you're asking. Are you asking whether you > should use the file-based journal or a database if you have 100k transactions? > > To be clear, what is "best" in one situation is often not "best" in > another. > Everything depends on the specifics of your particular use-case. > > > Justin > > On Wed, Sep 13, 2023 at 11:47 AM Shivang Modi > <sm...@provenir.com.invalid> > wrote: > > > If scenario is no loss transactions 100% if queue goes down whatever > > transactions gets enqueued, should get dequeued once queue comes up, > > we have 100k transactions or more need to flow up via queue. What > > would be best in such scenarios? > > > > Thanks, > > Shivang > > > > -----Original Message----- > > From: Justin Bertram <jbert...@apache.org> > > Sent: Wednesday, September 13, 2023 8:38 PM > > To: users@activemq.apache.org > > Subject: Re: Artemis File Storage Persistence vs JDBC Persistence > > > > When deciding between the file-based journal on local storage versus > > a remote database I think the three main considerations are: > > > > - Performance > > - Infrastructure > > - Reliability > > > > The file-based journal on local storage will be faster than a > > database for a few reasons: > > - The storage is local so there's no network latency to deal with. > > - The file-based journal was specifically written and heavily > > optimized for the message broker use-case. > > > > The file-based journal on local storage requires less infrastructure > > than a database since most servers already come with local storage. > > Using a database requires provisioning additional hardware as well > > as installing and maintaining a distinct piece of software. This can > > be costly both in terms of money and man-power. > > > > Generally speaking, local storage is always going to be more > > reliable than a remote database simply because it's much simpler > > (i.e. no network, no database with its own maintenance requirements, etc.). > > This simplicity tends to reduce downtime. > > > > In my experience the only folks who choose to use a database are > > those in an environment where there's already been a substantial > > investment in an enterprise database and stuff like automated > > backups, redundant networking, data replications, etc. are available. > > > > No matter which option you choose, the broker is written so that you > > should > > *never* lose messages. > > > > > > Justin > > > > > > > > On Wed, Sep 13, 2023 at 7:14 AM Shivang Modi > > <sm...@provenir.com.invalid> > > wrote: > > > > > Hi Team, > > > > > > > > > > > > Can anyone share pros and cons in depth between both. I see only > > > file storage is faster than JDBC storage. Is there any > > > disadvantage of File Storage like losing the enqueued data or anything? > > > > > > > > > > > > Thanks, > > > > > > Shivang. > > > > > > -- > > > *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION > > > intended solely for the use of the addressee(s). If you are not > > > the intended recipient, please notify the sender by e-mail and > > > delete the original message. Further, you are not to copy, > > > disclose, or distribute this e-mail or its contents to any other > > > person and any such actions maybe unlawful*. > > > This e-mail may contain viruses. Provenir has taken every > > > reasonable precaution to minimize this risk, but is not liable for > > > any damage you may sustain as a result of any virus in this > > > e-mail. You should carry out your own virus checks before opening > > > the e-mail or > attachment. > > > Provenir reserves the right to monitor and review the content of > > > all messages sent to or from this e-mail address. Messages sent to > > > or from this e-mail address may be stored on the Provenir e-mail system. > > > > > > > -- > > *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION > > intended solely for the use of the addressee(s). If you are not the > > intended recipient, please notify the sender by e-mail and delete > > the original message. Further, you are not to copy, disclose, or > > distribute this e-mail or its contents to any other person and any > > such actions maybe unlawful*. > > This e-mail may contain viruses. Provenir has taken every reasonable > > precaution to minimize this risk, but is not liable for any damage > > you may sustain as a result of any virus in this e-mail. You should > > carry out your own virus checks before opening the e-mail or attachment. > > Provenir reserves the right to monitor and review the content of all > > messages sent to or from this e-mail address. Messages sent to or > > from this e-mail address may be stored on the Provenir e-mail system. > > > > > > -- > *This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended > solely for the use of the addressee(s). If you are not the intended > recipient, please notify the sender by e-mail and delete the original > message. Further, you are not to copy, disclose, or distribute this > e-mail or its contents to any other person and any such actions maybe unlawful*. > This e-mail may contain viruses. Provenir has taken every reasonable > precaution to minimize this risk, but is not liable for any damage you > may sustain as a result of any virus in this e-mail. You should carry > out your own virus checks before opening the e-mail or attachment. > Provenir reserves the right to monitor and review the content of all > messages sent to or from this e-mail address. Messages sent to or from > this e-mail address may be stored on the Provenir e-mail system. > > The information in this e-mail communication together with any attachments is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you are not the intended recipient of this e-mail communication, please notify us immediately. Any views expressed in this e-mail communication are those of the individual sender, unless otherwise specifically stated. Charles River Development does not represent, warrant or guarantee that the integrity of this communication has been maintained or that the communication is free of errors, virus or interference.
