Hi On Mon, May 10, 2010 at 7:01 AM, Vomityn <carlruth.h...@gmail.com> wrote: > > I have a rather unique problem, which I was hoping the Camel community could > assist me with. > > Take the following scenario: > A queue is being populated with messages regularly every few milliseconds. > The data within the message is only relevant for 5 seconds, at which point > it expires. > > The messages enter a volatile container in heap, which will be processed > every second. > > Any messages that have expired i.e older than 5 seconds will be removed from > the container. > > Once the process completes, a database will be updated with the latest > information. > > Given that roughly 600 messages will be processed per second, I'd like to > keep the container of relevant messages in heap. > </scenario> >
So to be clear you want to hold up messages in a memory queue, and if a message grows older than 5 seconds it should be discarded? And a process will once every second process messages from the queue? Will it drain the queue? Or how many messages does it try to process? What happens if there are more messages to process than it can do within that 1 second time window? > I've tried to implement an AggregatorStrategy to process the messages as > they come in, but storing messages for 5 seconds makes this difficult. > You could also simply just add a timestamp on the message before you put it on the in memory queue. And then use a Filter EIP to only process messages which are < 5 seconds old. http://camel.apache.org/message-filter.html > I believe I can use Quartz to schedule the process each second, and along > with stateful=true, I can use the RAMJobStore. But I'm confused about how to > consume messages off the queue using a quartz endpoint. Is this where > pollEnrich comes in? Why do you want to use Quartz? There is a Timer which is simpler and can trigger once every second? http://camel.apache.org/timer.html And if using Quartz why should it be stateful? And if you want to poll on demand, then take a look at Polling Consumer EIP http://camel.apache.org/polling-consumer.html > -- > View this message in context: > http://old.nabble.com/Question-about-processing-and-persistence-tp28507720p28507720.html > Sent from the Camel - Users mailing list archive at Nabble.com. > > -- Claus Ibsen Apache Camel Committer Author of Camel in Action: http://www.manning.com/ibsen/ Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus
