Consumer distribituion over connections
I am looking for a bit of advice. I have been trying to get a better balance of consumer per connection but have run into a bit of a sticky situation. So while using the AMQ 5.8 and the AMQ PooledConnectionFactory[1], I was setting my camel routes with Cache_None to enable more connections from the pool to be used. In addition to that I am using DMLC session transactions to ensure messages are properly consumed.[2] Unfortunately this has the side effect of spamming the Consumer advisory topics because consumers are only used after consuming a single message. Is there an easier way to get a better consumer distribution on connections without this side effect? Thanks, Josh 1. PooledConnectionFactory config: (0 to deal with AMQ-4366) 2. Camel config: cacheLevelName=CACHE_NONE transacted=true lazyCreateTransactionManager=false replyToDeliveryPersistent=false acknowledgementModeName=CLIENT_ACKNOWLEDGE
Re: Consumer distribituion over connections
No one have any ideas on this? I was thinking that I could up the max connections to something higher and then setting the maximum number of active sessions per connection on the PooledConnectionFactory. I don't really like this solution though as it leads to an issue when you hit that max. On 30/01/2014 23:06, "Joshua Watkins" wrote: > >I am looking for a bit of advice. I have been trying to get a better >balance of consumer per connection but have run into a bit of a sticky >situation. > >So while using the AMQ 5.8 and the AMQ PooledConnectionFactory[1], I was >setting my camel routes with Cache_None to enable more connections from >the pool to be used. In addition to that I am using DMLC session >transactions to ensure messages are properly consumed.[2] Unfortunately >this has the side effect of spamming the Consumer advisory topics because >consumers are only used after consuming a single message. Is there an >easier way to get a better consumer distribution on connections without >this side effect? > > >Thanks, >Josh > > > >1. PooledConnectionFactory config: >class="org.apache.activemq.pool.PooledConnectionFactory" >init-method="start" destroy-method="stop"> > > >(0 to deal with > AMQ-4366) > > > > >2. Camel config: >cacheLevelName=CACHE_NONE >transacted=true >lazyCreateTransactionManager=false >replyToDeliveryPersistent=false >acknowledgementModeName=CLIENT_ACKNOWLEDGE > > >
Re: Resequencing on many individual topics (1000's)
I have figure out a way to limit the number of threads per JMS consumer. I needed to use the JMSConfiguration pojo to configure my ActiveMQComponent. The not-so-apparent change was changing the consumerType to simple instead of default. Hopefully this helps anyone else with the same issue. -josh On 31/03/2011 11:48, "Joshua Watkins" wrote: >I have run into a bit of a pickle. I am trying to use camel and Activemq >to resequence messages off of thousands of individual topics. A >particular topic would have a sequence number in messages delivered to it >for that topic. Additionally, I don't know the specific topics to listen >to ahead of time. So, at run time I will dynamically need to listen to >NEWS.US.IDAHO.BOISE, or NEWS.EU.SPAIN.MADRID. > >Unfortunately, since the resequencer can't work with multiple streams of >messages I can't simply subscribe to NEWS.> or a subtopic. (This is a bit >undesirable anyway as I would potentially be getting messages that I am >not interested in.) > >I have also tried simply creating a route dynamically with the specific >topic and its resequencer. This however led to thousands of threads being >created (1 thread for the JMS consumer and 2 threads for each >resequencer). > >Then I thought it would be better to use mildly more specific routes >(NEWS.EU.SPAIN.> ) and use a filter to funnel a particular stream of >messages to the resequencer. This ran into the problem of missing out on >messages to the topic while I was altering the route. Retroactive >consumer support doesn't seem to be supported in the Activemq component. >(And I am guessing that I was right to believe that routes must be >removed and re-added to the CamelContext to take effect.) > >So, am I missing an easy solution for this? >Is there a way for me to pool JMS topic consumer threads somehow? >Can live camel routes be altered at runtime without them missing topic >messages? > >Thanks, >Josh
Using CamelJmsDestinationName to publish to topics in Xml
I am trying to use the CamelJmsDestinationName header to send to a topic. However prepending topic: or activemq:topic: to my topic name simply sends reroutes the message to a queue with the name "activemq:topic:dynamicdestinationName". Ex: test dynamicDeliveryAddress topic:dynamicdestinationName Is there a way for me to publish to a topic using the CamelJmsDestinationName in xml? Thanks, -josh
Re: Using CamelJmsDestinationName to publish to topics in Xml
That worked. Thanks! On 07/04/2011 13:36, "Björn Bength" wrote: >Hi, > > > test dynamicDeliveryAddress > > > dynamicdestinationName > > > > >worked for me once. but i was using java dsl at the time. > >/björn > > >On Thu, Apr 7, 2011 at 2:25 PM, Joshua Watkins > wrote: >> I am trying to use the CamelJmsDestinationName header to send to a >>topic. >> However prepending topic: or activemq:topic: to my topic name simply >>sends >> reroutes the message to a queue with the name >> "activemq:topic:dynamicdestinationName". >> >> Ex: >> >>test dynamicDeliveryAddress >> >> >>topic:dynamicdestinationName >> >> >> >> >> Is there a way for me to publish to a topic using the >> CamelJmsDestinationName in xml? >> >> Thanks, >> -josh >> >>
Re: FYI, JmsEndpoint.consumerType removed but still in docs
Hadrian, I don't see that the SimpleMessageListenerContainer is deprecated in 3.0.5_RELEASE (http://static.springsource.org/spring/docs/3.0.5.RELEASE/javadoc-api/org/s pringframework/jms/listener/SimpleMessageListenerContainer.html). The reason I bring this up is that when using the DefaultMessageListenerContainer (which JmsMessageListenerContainer extends) a new listener thread is created for every JMS consumer route in camel. However utilizing the SimpleMessageListenerContainer and a thread pool bounds the number of threads created. -josh On 08/04/2011 18:23, "Hadrian Zbarcea" wrote: >Hi Barry, > >Claus made that change part of the upgrade to spring 3 because the >SimpleMessageListenerContainer was deprecated. Therefore the consumerType >made no sense anymore as you only have one option, the >JmsMessageListenerContainer. I'll update the docs. Thanks for reporting >this! > >Hadrian > >On Apr 8, 2011, at 1:00 PM, Barry Kaplan wrote: > >> I see it was removed in revision 1067441. Yet the web site docs still >>how it >> as an option. Also I could not find any indication in the release notes >>that >> it was removed and why. >> >> -- >> View this message in context: >>http://camel.465427.n5.nabble.com/FYI-JmsEndpoint-consumerType-removed-bu >>t-still-in-docs-tp4291400p4291400.html >> Sent from the Camel - Users mailing list archive at Nabble.com. >
Re: Cometd component
Have you tried using the Jackson library instead of Xstream? There is Jackson support in camel greater than 2.0 (http://camel.apache.org/json.html). I haven't used the Camel Jackson component but if it works anything like the regular jackson libraries it should be straight forward with no annotations/config for most POJOs. Alternatively, you could just serialize your POJOs to JSON using the Jackson Library (http://jackson.codehaus.org/) and a custom processor bean. And while you are looking at serialization you might want to check out how the different serializers perform: https://github.com/eishay/jvm-serializers/wiki/. -josh On 12/04/2011 22:25, "Gert Villemos" wrote: >I have problems understanding how the cometd component publish objects. > >I have my own class called 'Parameter'. I want to publish instances of >'Parameter' through cometd to a website. I would expect the data arriving >through cometd to have a structure similar to my Parameter definition. > >However the only thing that arrives is a single field, holding the a value >like >"org.hbird.business.simpleparametersimulator.ConstantParameter@3da03ade", >i.e. it seems the object is transformed to a string using a simple >toString >call. All field information is lost. Not very useful. > >Is there a way to transform any POJO to a format (Map?) that can be >transfered using cometd, keeping the information in the POJO? > >My route looks like; > > > > > > > > >I have tried inserting a Xstream data transformer in the route. Indeed I >receive a different set of data, but its an array of 160 integers, not a >structure reflecting the structure of my 'Parameter' class as I had >expected. > >I know that I can create bean that converts the 'Parameter' instance into >a >Map and put this into the body prior to sending the data over cometd. >However I then have to create a 'formatter' per data type. I had hoped to >have a generic formatter. > >What am I doing wrong? > >-- >View this message in context: >http://camel.465427.n5.nabble.com/Cometd-component-tp4299174p4299174.html >Sent from the Camel - Users mailing list archive at Nabble.com.
Reinstating JMS SimpleMessageListenerContainer
A while ago there was a discussion topic that the SimpleMessageListenerContainer/ConsumerType.Simple was removed from camel-jms as it was deprecated. However from looking at the latest Spring docs ( http://static.springsource.org/spring/docs/3.1.0.M1/javadoc-api/org/springf ramework/jms/listener/SimpleMessageListenerContainer.html ) it is alive and kicking. Is there another reason that this feature was removed? The reason I am so keen on the SimpleMessageListenerContainer is that the DefaultMessageListenerContainer does not allow you to constrain the number of threads used in a useful way. If you give it a task executor with a maximum 5 threads and want to listen on 10 routes, only 5 of those routes would be consumed from. There is even a note about this fact in the docs ( http://static.springsource.org/spring/docs/3.1.0.M1/javadoc-api/org/springf ramework/jms/listener/DefaultMessageListenerContainer.html#setTaskExecutor( java.util.concurrent.Executor) ). This makes the SimpleMessageListenerContainer invaluable for scaling out an application using camel-jms. -josh
Re: Adding Routes dynamically
You are missing using the routeId. When you create your route you need to:
from(uri + "?splitEntries=false&consumer.initialDelay=0").
marshal().rss().
to("mock:result").routeId("routeId");
Then when you remove the route you use the same routeId string.
I hope this helps.
-josh.
On 26/04/2011 15:13, "Jeff Segal" wrote:
>Hi all,
>
>I have written a service bean which can add and remove Routes dynamically
>and I'd like to know if my approach was "kosher" in terms of best
>practices.
>It's not exactly pretty, but it does work. Here are the two classes:
>
>@Component
>public class RssRouter implements CamelContextAware {
>
>protected CamelContext camelContext;
>RssRouteBuilder rssRouteBuilder;
>
>final Logger log = Logger.getLogger(this.getClass());
>
>public void addFeed(String uri) throws Exception {
>log.info("Adding feed '" + uri + "'");
>uri = "rss://" + uri;
>
> camelContext.addRouteDefinition(rssRouteBuilder.getRouteDefinition(uri));
>}
>
>public void removeFeed(String uri) throws Exception {
>log.info("Removing feed '" + uri + "'");
>uri = "rss://" + uri;
>camelContext.removeRoute(uri);
>}
>
>public CamelContext getCamelContext() {
>return camelContext;
>}
>
>public void setCamelContext(CamelContext camelContext) {
>this.camelContext = camelContext;
>}
>
>public void setRssRouteBuilder(RssRouteBuilder rssRouteBuilder) {
>this.rssRouteBuilder = rssRouteBuilder;
>}
>
>}
>
>public class RssRouteBuilder extends RouteBuilder {
>
>@Override
>public void configure() throws Exception {
>// Do nothing here since we are creating routes dynamically
>}
>
>RouteDefinition getRouteDefinition(String uri) {
>return from(uri + "?splitEntries=false&consumer.initialDelay=0").
>marshal().rss().
>to("mock:result");
>}
>
>}
>
>Thanks!
>Jeff
Rejecting JMS messages on camel route update.
I was using ActiveMQ 5.5 with Camel 2.7.0 and during a update of the route
lost some messages and found this error in the log
2011-06-13 13:25:42,950 | INFO | Starting to graceful shutdown 1 routes
(timeout 300 seconds) | org.apache.camel.impl.DefaultShutdownStrategy |
qtp788458698-45
2011-06-13 13:25:42,971 | WARN | Rejecting received message because of
the listener container having been stopped in the meantime:
ActiveMQTextMessage
My route is something like:
${in.headers.JMSType} == 'bob'
I did a search and found a bug corrected back in Camel 2.2 but nothing
since then. Is there anyway to prevent accepting/rejecting messages when
restarting the route?
Re: Rejecting JMS messages on camel route update.
The configuration I am using is the built in camel that comes with
ActiveMQ. (So stand alone broker with the Camel enabled from enabling
Jetty.) Additionally, I got the warning when starting and stopping the
route via the camel web console. With that said, I don't think there is
much spring work I can do in this situation. Also from the documentation
the acceptMessagesWhileStopping = false is the default behaviour. On our
route we do not set this to being true.
Any other ideas?
Thanks,
-josh
On 14/06/2011 12:51, "Claus Ibsen" wrote:
>Hi
>
>There is an option on JMS endpoint, acceptMessagesWhileStopping
>http://camel.apache.org/jms
>
>
>If you have ActiveMQ embedded in the same JVM and its all Spring XML
>that set it up, then you need to mess with Spring to tell it to stop
>Camel before ActiveMQ etc. There is unfortunately not so many options
>for you with spring. You usually only have the depends-on attribute.
>
>See for example the camel management example in the Apache
>distribution. It has an embedded AMQ + Camel in the same JVM.
>
>
>On Tue, Jun 14, 2011 at 12:48 PM, Joshua Watkins
> wrote:
>> I was using ActiveMQ 5.5 with Camel 2.7.0 and during a update of the
>>route
>> lost some messages and found this error in the log
>>
>> 2011-06-13 13:25:42,950 | INFO | Starting to graceful shutdown 1 routes
>> (timeout 300 seconds) | org.apache.camel.impl.DefaultShutdownStrategy |
>> qtp788458698-45
>> 2011-06-13 13:25:42,971 | WARN | Rejecting received message because of
>> the listener container having been stopped in the meantime:
>> ActiveMQTextMessage
>>
>>
>> My route is something like:
>>
>>
>>
>>
>>${in.headers.JMSType} == 'bob'
>>
>>>id="to11"/>
>>
>>
>>
>>
>>
>>
>> I did a search and found a bug corrected back in Camel 2.2 but nothing
>> since then. Is there anyway to prevent accepting/rejecting messages when
>> restarting the route?
>>
>>
>>
>>
>
>
>
>--
>Claus Ibsen
>-
>FuseSource
>Email: cib...@fusesource.com
>Web: http://fusesource.com
>Twitter: davsclaus, fusenews
>Blog: http://davsclaus.blogspot.com/
>Author of Camel in Action: http://www.manning.com/ibsen/
Serving static pages with JettyComponent
I am using the camel jetty component to route to activemq to access other
backend services.
For instance:
from("jetty:http://0.0.0.0:8083/people";)
.setHeader(JMS_TYPE, constant("AddPeople"))
.to("activemq:proxy.out").routeId("addPeople");
I would also like to serve up some static web pages that are part of the
classpath. Of course, I want to be able to leverage the jetty server that
is already running within the camel jetty component. Unfortunately I am at
a bit of a loss at how do do this.
I have tried creating a context handler in spring:
And then added it in the list of handlers on the uri:
"jetty:http://0.0.0.0:8083/people??handlers=staticPageHandler";
However that didn't work either.
Any ideas? Am I missing something simple?
Re: Serving static pages with JettyComponent
Thanks Craig. That solution worked.
I will take a look at the deeper cause. It seems odd that you can specify
handlers but they don't really work. I am however only using 2.6, but I
can't see any jira tickets for something like this being fixed. If I get
time I will create a ticket and maybe a patch for the jetty component.
-josh
On 18/08/2011 18:10, "Craig Taylor" wrote:
>For Camel 2.2 I ended up implementing a FileDumper() which dumps a file
>based upon the path directly to the exchange body. Eg:
>
>route.choice()..when(header(Exchange.HTTP_PATH).startsWith("/js")).process
>(fileDumper)
>
>and fileDumper psuedo-code is essentially :
>
>mimetype = decode filename (Exchange.HTTP_PATH based)
>exchange.setHeader(Exchange.CONTENT_TYPE, mimeType)
>exchange.setBody( Resource from baseDirectory + filename . readFile() )
>
>I'd love to find a way to configure the underlying jetty component to do
>this magic for me but it doesn't appear to be accessible.
>
>On Thu, Aug 18, 2011 at 8:06 AM, Joshua Watkins <
>joshua.watk...@gamesys.co.uk> wrote:
>
>> I am using the camel jetty component to route to activemq to access
>>other
>> backend services.
>>
>> For instance:
>>
>> from("jetty:http://0.0.0.0:8083/people";)
>>.setHeader(JMS_TYPE, constant("AddPeople"))
>>.to("activemq:proxy.out").routeId("addPeople");
>>
>> I would also like to serve up some static web pages that are part of the
>> classpath. Of course, I want to be able to leverage the jetty server
>>that
>> is already running within the camel jetty component. Unfortunately I am
>>at
>> a bit of a loss at how do do this.
>>
>>
>> I have tried creating a context handler in spring:
>>
>> > class="org.eclipse.jetty.server.handler.ContextHandler">
>>
>>
>>
>>
>> And then added it in the list of handlers on the uri:
>> "jetty:http://0.0.0.0:8083/people??handlers=staticPageHandler";
>> However that didn't work either.
>>
>> Any ideas? Am I missing something simple?
>>
>>
>>
>>
>
>
>--
>---
>Craig Taylor
>ctalk...@ctalkobt.net
Re: How to get full control over threads used by JMS/ActiveMQ component?
I was able to limit the number of threads by setting the consumerType to simple and the taskExecutor to a spring ThreadPoolTaskExecutor bean that I created. I didn't use the ThreadPoolProfile class. I hope this helps. On 24/11/2011 22:19, "Jason Dillon" wrote: >rType(ConsumerType.Simple); >ActiveMQComponent jms = new ActiveMQComponent(jmsConfig); >
