I'm liking a) and allowing c) (a single mbean for all endpoints of a certain type) make sense to me.
Using an LRU cache for non-singleton endpoints makes sense too; though I wonder if an endpoint might want to mark itself as not cachable if it knows its not really worth the trouble (as it has no state?). 2009/6/30 Willem Jiang <willem.ji...@gmail.com>: > Hi Claus, > > I think we need to implements a method for get unique name from the > endpoint. It will base on the endpoint's parameter not the customer > parameter. With this method we could avoid to register to much temp endpoint > with different customer parameter. > > In this case we need find a way to trim the customer parameter from the > endpoint URI. If we could do that, it will save us lots room for the store > the fake temp http endpoints (we could share the endpoints in the > ProducerCache/ConsumerCache with different customer parameter) and make the > LRUCache more effective. > > Just my two cents. > > Willem > > Claus Ibsen wrote: >> >> Hi >> >> Tickets >> ====== >> I am looking into two issues related to Camel eating memory >> https://issues.apache.org/activemq/browse/CAMEL-1771 >> https://issues.apache.org/activemq/browse/CAMEL-1738 >> >> >> The problem >> ========= >> It all boils down to using a lot of http endpoints with unique urls. >> So over time Camel accumulate a lot of created endpoints in its >> internal endpoint registry and as well as JMX Beans >> That consumes memory, and for people with millions unique endpoints >> over time that consumes to much memory >> >> >> Solutions >> ======= >> To remedy this I have addressed in two areas >> >> a) failsafe with a limited size of 1000 entries >> - Using the LRUCache for ProducerCache/ConsumerCache (verified by end >> user that its much better) >> - Using the LRUCache in CamelContext for its endpoint reigstry >> >> I assume having more than 1000 unique endpoints, producers or >> consumers is not within normal usage for a given CamelContext? >> And it does not bring harm as Camel will just recreate the object if >> not already cached. >> >> >> b) >> - Not registering endpoints with lenient properties in JMX or camel >> context. That is if only they use properties that Camel does not know >> about (= lenient properties) >> as these endpoints is highly not reusable and short lived. And its >> only a few endpoints that support lenient properties (http, restlet, >> cxf, atom, rss) >> >> So if you setup an endpoint with custom parameters (not Camel options) >> then its a lenient property and Camel will not cache/register it. >> For example: >> >> http://myserver/mypath?id=1 >> http://myserver/mypath?id=2 >> http://myserver/mypath?id=3 >> ... >> >> where id is a lenient property that is not a Camel option. >> So these will not be registered. >> >> >> But if you use >> http://myserver/mypath >> then it will be registered as its does not contain any lenient >> properties at all. >> >> >> c) >> - Only registering a single JMX bean. >> Otherwise the JMX registry will be cluttered with millions endpoints. >> So what we do not is to register the same parent endpoint for the >> endpoints. >> We use the new getEndpointKey() on Endpoint to provide the key to use >> for JMX registration. This allows us to provide the same key for all >> the http endpoints. >> >> >> Questions >> ======== >> 1) >> Option a and c is already nearly done. I am running further unit tests >> and do a bit of code polish. >> >> 2) >> As option b is a bit controversial, I am wondering if that is >> feasible. Should we just go ahead with the LRUCache being able to >> filter out the eldest endpoints? >> And I wonder if people just create a few endpoints with lenient >> properties then we still have "room" to store them in the cache so I >> wonder if we should do this at all? >> I do think we could keep this as a thought for the future, and just >> let the LRUCache handle it, so when people use millions of unqiue http >> endpoints we let the chace >> filter out the not used ones. >> >> >> >> > > -- James ------- http://macstrac.blogspot.com/ Open Source Integration http://fusesource.com/
