I can't give you any really exact measurements, but I can tell you
that we found that getting Fedora and GSearch attached via a separate
instance of ActiveMQ was trivial. It took perhaps five minutes. Your
question reminds that perhaps we ought to do some of those JMeter test
we've been putting off. {grin}
We've successfully run batch ingests of a few thousand objects through
that system without any difficulty. It was necessary to spend some
time (perhaps half a "man-day") tuning ActiveMQ's datastores and
memory usage and the like to get reliably good performance, but that
was expected and normal.
Have you decided on a particular piece of broker software? Most can be
scaled both vertically and horizontally. We'll be clustering ActiveMQ
at some point in the future.
---
A. Soroka
Digital Research and Scholarship R & D
the University of Virginia Library
On Oct 27, 2009, at 9:09 AM, Scott Prater wrote:
> We'll soon be activating JMS in our Fedora installation, in order to
> update our Solr index and memory cache on purge/ingest/update events.
> Has anyone had any other experience using a third-party message
> broker?
> In particular, does anyone have any experience using a message broker
> in situations of high traffic or with large datasets or heavy batch
> operations? Any recommendations or gotchas to look out for?
>
> TIA,
>
> -- Scott
>
> [email protected] wrote:
>> I may have missed part of the conversation where you describe this,
>> but are you using the JMS gear that comes with Fedora or a separate
>> JMS broker (e.g. ActiveMQ)?
>>
>> ---
>> A. Soroka
>> Digital Research and Scholarship R & D
>> the University of Virginia Library
>>
>>
>>
>> On Oct 26, 2009, at 6:56 PM, <[email protected]> wrote:
>>
>>> · Using the message queue with fedoragsearch to update SOLR
>>> seems not to be a good idea for larger datasets. I suspect using the
>>> message queue for anything with a large datasets is not a good idea.
>>> It seems to consume a lot of resources when running, does not seem
>>> to shut down reliably (resulting in very long rebuild time next time
>>> you start tomcat) and seems to scale very badly (performance goes
>>> down significantly the more objects we add). There are a lot of
>>> “seems” in this statement because we didn’t actually measure this
>>> and it is only based on our subjective observation.
>>
>>
>> ------------------------------------------------------------------------------
>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
>> is the only developer event you need to attend this year. Jumpstart
>> your
>> developing skills, take BlackBerry mobile applications to market
>> and stay
>> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
>> http://p.sf.net/sfu/devconference
>> _______________________________________________
>> Fedora-commons-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
>
>
> --
> Scott Prater
> Library, Instructional, and Research Applications (LIRA)
> Division of Information Technology (DoIT)
> University of Wisconsin - Madison
> [email protected]
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart
> your
> developing skills, take BlackBerry mobile applications to market and
> stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Fedora-commons-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Fedora-commons-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
