On 9/18/06, Robert Greig <[EMAIL PROTECTED]> wrote:
On 18/09/06, Hiram Chirino <[EMAIL PROTECTED]> wrote:
> Feel free to peek at the ActiveMQ message store implementations. this
> may be an area where our projects can cooperate.
Cooperation on this sounds like an excellent idea. Also very timely
since I am about to look again at our persistence implementation to
implement flow-to-disk and reduce our memory footprint (which is nasty
at the moment - I will raise a jira describing it in detail).
Does ActiveMQ do flow to disk?
I guess I will need your definition of "flow to disk".. sounds like
what we have been terming spooling to disk. Where a producer is
streaming to the tail of a file and consumers are consuming from the
head. Allowing your producer to be totally decoupled from the
consumer (since a memory buffer would be need otherwise which could
fill and halt the producer). In short, we have a hacky implementation
that is no where close to what we actually want. Basically we need to
add cursoring capabilities to our interfaces.
> We have implemented
> JDBC based message store implementations that do batch operations. We
> have also implemented a message store filter that uses a journal like
> HOWL to do fast/concurrent syncs to a log and then do async
> checkpoints in batches to a journal.
um. I meant checkpoint in batches to the jdbc database (or any other
message store).
I presume that you did the journal because the JDBC performance on its
own was not good enough?
correct. the sync nature of jdbc operations causes imposes undesired latencies.
> We have also started a BDB like
> file based implementation optimized for the messaging use cases that
> is still in development.
>
> Some of the things we still need to do to our implementations is add
> better cursoring interfaces.. perhaps that some work we can do
> together?
Yes, I will take a look at the ActiveMQ implementation in the next
couple of days - you may be more sophisticated than us already though!
We may need to tear our implementations down a bit to implement good
cursoring interfaces, a second eye ball will would help. Here is a
link to the amq source:
http://svn.apache.org/viewvc/incubator/activemq/trunk/
and the package that is of interest is:
http://svn.apache.org/viewvc/incubator/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/store/
Have you benchmarked your different implementations? We have got the
IBM JMS Performance harness running against Qpid to help us analyse
the impact of our changes.
No. I have not had a chance to do that yet. Is there much setup
involved? I might look into it if someone could send me some
pointers. :)
RG
--
Regards,
Hiram
Blog: http://hiramchirino.com
