Re: Question on Camel WireTap EIP custom thread pool creation
2012/7/20 Claus Ibsen : > Can you try with 2.10, I think there was a bug in wire tap custom > thread pool, but it may only be when using XML DSL. > But testing with latest release would be good. Just for your information: I'm using the old Camel 2.6.0 with XML DSL, and when I don't specify a custom threadpool reference, each wireTap results into a separate threadpool. That might be the bug you are referring to. Explicitly linking wireTap elements to a custom threadpool works as advertised. Have not tested the Java DSL variant.
Re: Global Interceptor for all incoming Messages
2012/6/21 sushil_vsk5 : > Unfortunately, the interceptFrom() only seems to intercept messages for > those routes which are defined in the same RouteBuilder class as the > interceptFrom() itself. Is there a way to intercept messages for all > incoming messages for all routes that the camel context holds? We used a custom route policy applied to a list of specific routes, which runs the same set of checks on those routes, at the start of the route. Or something else: http://java.dzone.com/articles/aspect-oriented-programming-camel Maybe that could help you? Tung
Re: keep track of JMSMessageID with InOnly exchange
On Wed, Jan 18, 2012 at 4:53 PM, Claus Ibsen wrote: > A possible issue is that the JMSMessageID is only available when the > JMS Client have really sent the message, and some clients can be in an > async mode, where they may take a little while before sending. So in > that case I would assume Camel will have to wait for that to happen. Sorry to respond so late, lost track of it in the huge pile of camel mailing .. ;-) I did not take into account that the sending could run asynchronously for some JMS clients. You are right that retrieving the JMSMessageID in that case could block the process a little. > Unless the Spring Sent callback happens as a sort of pre construct > step, and the JMS client is able to determine the JMSMessageID before > actually sending it. I'm afraid this is not clearly specified in the JMS spec, and we don't have any guarantee. :-( > Anyway its probably different a bit from JMS vendor to vendor. So a > one shoe solution is maybe not possible. Indeed. > Also a JMS InOnly message, you most likely do not want to have the > Message on the Camel Exchange changed into something else such as a > javax.jms.Message, as you did a fire and forget. So maybe the patch > should just enrich the message by adding a header with the > JMSMessageID value. Sounds reasonable. Code is written a while ago, I'm not sure, but IIRC, I copied the full message to the output because it was easier for logging both the JMSMessageID and the text message body. But the body could be captured earlier of course, so having only the JMSMessageID as header value should probably work fine as well. When I have more time to revisit the code, I will check if I can refactor both the patch and the route definition according to your suggestion. > And I think there should be an option so people can turn this on|off, based > on their needs. Probably not hard to implement, based on some configuration option or exchange property. If you have any preference, just let me know. Will try to integrate into patch as well, but probably not very soon. > BTW: What is your use case. You need to use the actual JMSMessageID > for track and tracing or some sort? Yes, merely for track and tracing. Our application fires messages to a remote system via IBM MQ, and for audit and troubleshooting reasons, we log the message IDs of our all the outgoing messages. We were actually surprised Camel did not provide this out of the box, but maybe we just have strange requirements .. ;-)
Re: keep track of JMSMessageID with InOnly exchange
On Thu, Jan 5, 2012 at 3:21 PM, wing-tung Leung
wrote:
> But there is one more thing we would like to do afterwards, and that's
> capture and log the JMS message ID in some audit table.
I finally patched the Camel JmsProducer class, with a minor
modification of the "InOnly" behavior. In this case, the actually sent
JMS message, containing the filled in "JMSMessageID", is copied to the
"out" from the exchange.
This enables processors further down the line to capture the message
ID for logging.
Attached the patch. Any comments/ideas to improve this further? Any
idea if such idea could end up in the trunk? I applied it to version
2.6.0, but it can be applied to the current trunk without conflicts as
well.
Regards,
Tung
diff --git
a/components/camel-jms/src/main/java/org/apache/camel/component/jms/JmsProducer.java
b/components/camel-jms/src/main/java/org/apache/camel/component/jms/JmsProducer.java
index 4434252..2dc4caf 100644
---
a/components/camel-jms/src/main/java/org/apache/camel/component/jms/JmsProducer.java
+++
b/components/camel-jms/src/main/java/org/apache/camel/component/jms/JmsProducer.java
@@ -1,3 +1,4 @@
+
/**
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
@@ -306,8 +307,11 @@ public class JmsProducer extends DefaultAsyncProducer {
}
};
-doSend(false, destinationName, destination, messageCreator, null);
-
+MessageCaptor messageCaptor = new MessageCaptor();
+doSend(false, destinationName, destination, messageCreator,
messageCaptor);
+Message sentMessage = messageCaptor.message;
+exchange.setOut(new JmsMessage(sentMessage, null));
+
// after sending then set the OUT message id to the JMSMessageID so
its identical
setMessageId(exchange);
@@ -315,6 +319,13 @@ public class JmsProducer extends DefaultAsyncProducer {
callback.done(true);
return true;
}
+
+private class MessageCaptor implements MessageSentCallback {
+ protected Message message;
+ public void sent(Session session, Message message, Destination
destination) {
+ this.message = message;
+ }
+}
/**
* Sends the message using the JmsTemplate.
@@ -336,24 +347,12 @@ public class JmsProducer extends DefaultAsyncProducer {
// destination should be preferred
if (destination != null) {
-if (inOut) {
-if (template != null) {
-template.send(destination, messageCreator, callback);
-}
-} else {
-if (template != null) {
-template.send(destination, messageCreator);
-}
+if (template != null) {
+template.send(destination, messageCreator, callback);
}
} else if (destinationName != null) {
-if (inOut) {
-if (template != null) {
-template.send(destinationName, messageCreator, callback);
-}
-} else {
-if (template != null) {
-template.send(destinationName, messageCreator);
-}
+if (template != null) {
+template.send(destinationName, messageCreator, callback);
}
} else {
throw new IllegalArgumentException("Neither destination nor
destinationName is specified on this endpoint: " + endpoint);
keep track of JMSMessageID with InOnly exchange
We have quite some routes which end with a simple fire and forget towards a JMS queue. In general, this works fine. But there is one more thing we would like to do afterwards, and that's capture and log the JMS message ID in some audit table. For the logging, we can use some custom bean, but the problem is: HOW can we get grip on the message ID, which is assigned by the JMS provider (in this case WebSphere MQ) ? When looking into the code from JmsProducer.setMessageId(), the MQ message ID is only available in request-reply configuration, and not with the "InOnly" case. Is there some "easy" way to capture the ID from the external JMS provider? Or is the only alternative using self generated message ID's ? (Camel 2.6.0 on WebSphere AS 6.1 and WebSphere MQ) Many thanks in advance! Tung
Re: Camel JMS Request/Reply with Websphere
On Thu, Dec 22, 2011 at 11:46 PM, Mark Borner wrote: > > I'm trying to use Camel (v 2.6.0) JMS Request/Reply with Websphere 7 using > Websphere MQ. I'm successfully able to put the request message onto the > queue using the following URI: > > > jms:queue:inboundQueue?connectionFactory=#connectionFactory&taskExecutor=#taskExecutor&transactionManager=#transactionManager&cacheLevelName=CACHE_NONE&replyTo=outboundQueue&requestTimeout=12 > > Note: I have to use cacheLevelName=CACHE_NONE in order for this to work on > Websphere. > We work on WebSphere AS 6.1 in combination with WebSphere MQ as well, and I also had trouble with the cache settings. Lowering cache level to CACHE_NONE, seemed to solved the issue at first sight, but this leads to intermediate JMS session resets, and in our case, possible lost messages because we did not use any durable subscriptions. A colleague of mine warned me for some problems he had with durable subscriptions in WebSphere MQ, so I did not even try that particular setup. Bringing the cache level to CACHE_CONSUMER solves the problem with lost messages, but this is not compatible with the default WAS work manager. The latter uses some kind of asynchronous internal beans, which REQUIRE closing JMS connections at the end of each call. http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fasyncbns%2Fconcepts%2Fcasb_asbover.html The trick to solve this: don't use the WAS work manager, but use some simple Spring thread pool based executor instead. So, maybe you can give this a try for you as well .. Tung
Re: Starting routes in StartupListener
On Tue, Nov 8, 2011 at 2:13 PM, wing-tung Leung wrote: > my camel context contains a few routes which should not be started > automatically, but it should run a business check first for those routes. > My initial idea is to disable "autoStartup" for those routes, and add a > custom StartupListener which performs the check and simply calls > "startRoute()" on the context if the check passes. > For your information, I found an easy workaround: I added an extra Spring bean which implements the "ApplicationListener" interface, which fires up the relevant routes AFTER the complete initialization of the Spring context. Whether the StartupListener documentation/behavior is accurate, is another discussion .. :-)
Starting routes in StartupListener
Hello, my camel context contains a few routes which should not be started automatically, but it should run a business check first for those routes. My initial idea is to disable "autoStartup" for those routes, and add a custom StartupListener which performs the check and simply calls "startRoute()" on the context if the check passes. But, this fails to work because "DefaultCamelContext.startRouteService()" checks the camel context is already started or not, and the listeners are already invoked BEFORE setting the "started" flag, just AFTER starting all relevant routes. Listeners are called in the "doStart()" method, BEFORE setting the "started" flag: https://github.com/apache/camel/blob/trunk/camel-core/src/main/java/org/apache/camel/support/ServiceSupport.java#L51 I have the feeling that the documentation of the StartupListener can be improved here, or should the listeners be invoked AFTER the complete start of the context? http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/StartupListener.html If this is intended behavior: other ideas to achieve my goal? Starting routes in "suspended" mode? Use separate Spring bean and "depends-on" which starts the routes AFTER the complete startup of the initial context? (running on camel 2.6.0, but the related code looks similar in current trunk) Any advice is greatly appreciated, Tung
Re: JMS request-reply and transactions
On Tue, Oct 25, 2011 at 1:24 AM, Mathieu Lalonde wrote: > Try removing the from your route. > Normally, for InOut, you should not send the response back explicitly. > Camel takes care of that for you. :) > This solved my problem, thanks Mathieu! I assume that the fixed echo route, detects the JMSReplyTo header in the message and automatically forwards the result of the route to that reply destination. This is some kind of test setup, the final echo implementation would not be using Camel, but written in some external mainframe application. I guess that program needs to behave accordingly and put the reply in the correct destination. But I don't understand what went wrong in the previous setup, where I send the reply explicitly the the reply destination. The initial request route sees an valid response, so I assume the echo route is finished and has committed the response. Why does the request route still logging some request timeout after a "successful" receive? And secondly, why do I find a reply message in queue after the timeout? Is it because the echo route posts TWO (!) messages? Anyway, my initial problem is solved, and having a better understanding of Camel JMS would only be a bonus .. :-) Thanks! Tung
JMS request-reply and transactions
Hello,
I want to setup 2 separate routes in my camel context to create a
synchronous spring bean method, which triggers a JMS based request-reply
service.
The front (synchronous) bean is invoked via a simple Spring web MVC
controller, and dumps the response back to the browser. The second route, in
a separate transaction, pulls the message from the request queue, performs a
simple transformation and puts the response back into (shared) response
queue (both queues from external IBM MQ server).
At first, things were looking fine, because my browser page renders the
response as expected. But in a second look, I discovered that the log still
mentions a timeout (default 20 secs) waiting for the reply, although the
reply has been received already! And secondly, after the timeout, the
response message (re-) occurs on the response queue, and remains there.
I assume the reply handler is missing some transaction commit, causing some
automatic rollback, so the reply bounces back to the reply queue. Is that
correct?
How should/can I avoid this? I have read something about the
"transactedInOut" option, but it is currently deprecated, and setting it on
the request queue doesn't seem to help.
This is the proxy and surrounding request-reply route:
And the dummy echo service on the other side:
Hello ${body} (id ${id})
${in.header.JMSMessageID}
Last, the 2 external IBM MQ queues referred by JNDI, including some camel
options:
jms.queue.test.request=jms:queue:jms/queue.mud_rt_snd.01?replyTo=jms/queue.mud_rt_rec.01&useMessageIDAsCorrelationID=true&jmsMessageType=Text&messageConverter=#simpleBeanInvocationConverter&transactedInOut=true
jms.queue.test.reply=jms:queue:jms/queue.mud_rt_rec.01
Running on camel-2.6.0, inside IBM WebSphere 6.1 and IBM MQ, most resources
referred by JNDI.
Any help pointing me into the good direction is greatly appreciated!
Thanks,
Tung
Query part of HTTP_URI header ignored
We use a HttpProducer (http4) which receives a URL via the "CamelHttpUri" header, and fetches the data from that URL. The http4 endpoint configuration does not contain any URL at all, because this is all determined based on dynamic message content. Problem is that I initially expected that the query part of the URL would be taken into account, but the HTTP producer IGNORES the query parameter segment when the "CamelHttpQuery" header is not set. It seems to fallback to the query part of the endpoint, but that does not work for us since it's empty. So, setting URL directly on endpoint should respect the query part, but only setting it via the header, does NOT use the query part. I did not expect this behavior, and find it a little confusing. Wouldn't it be better to include the query part in BOTH cases? For the moment, I implemented a workaround by setting the CamelHttpQuery header as well, but this feels like a small hack. http://camel.apache.org/http4.html Using camel 2.5, but code seems to be unchanged in latest trunk, see line 331 in HttpProducer.createMethod() : https://github.com/apache/camel/blob/trunk/components/camel-http4/src/main/java/org/apache/camel/component/http4/HttpProducer.java#L331 Regards, Tung
Re: From http4 via md5checksum to FTP: file cache or streaming ?
2011/8/12 Magnus Palmér : > I would make your md5bean return a file. > Not sure if the current Jira issue of not cleaning (deleting) the file until > JVM stops will apply for you then or not. Well, the MD5 processor would probably need to dive into the InputStream implementation (Camel specific) which holds the reference to the original file, which I don't like very much. Avoiding knowledge of the specific implementation is possibly by creating a new file during the calculation of the MD5 checksum, but then I have the file content twice on the local filesystem: once from the http4 cache, and a second copy to pass to the rest of the route. But I will try to dump the result from http4 into a file myself, I assume I can pass along that reference around easily. Thanks for the tip. Tung
From http4 via md5checksum to FTP: file cache or streaming ?
Hello,
one of our routes pulls binary data from from a HTTP service using
http4, and then uploads the binary to a remote FTP directory. It also
uses temporary files on the local filesystem behind the scenes , which
makes sense because the binary image data can grow up till 10MB.
Now I want to add one extra step: MD5 checksum verification just after
the download. Because of the size, I prefer not to load all the data
into a byte array to calculate a simple checksum, and use a
InputStream instead. At first sight this seems to work. In the
debugger, I can see this input stream is a wrapper around the locally
cached file, and at the end of the function I return the original
input stream.
This is how the route currently looks like:
ftp://{{attachment.ftp.location}}/?username={{attachment.ftp.user}}&password={{attachment.ftp.password}}&binary=true"/>
The processing bean method's signature:
InputStream process(InputStream buffer,
@Header("mgws_file_md5sum") String expectedChecksum)
But now the route seems to hang. I assume returning the "used" input
stream is wrong, since the FTP component can't do anything useful with
this anymore. I basically see options to fix this:
1 - use fancy stream interception with a custom HttpBinder for http4,
integrating MD5 checksum calculation on the fly
2 - redirect to temporary file explicitly, start a new route for the
checksum, reuse same file for FTP upload and cleanup manually
Any other recommendations? I think this is a quite common use case, so
I guess more experienced Camel user may want to give some useful
advice to a novice like me ..
Some pages I have been looking at:
http://camel.apache.org/http4.html
http://camel.apache.org/file2.html
http://camel.apache.org/stream-caching.html
(using Camel 2.6)
Thanks!
Tung
Re: Enter multiple onException clauses in a single route?
On Mon, Aug 8, 2011 at 3:27 PM, Claus Ibsen wrote: > On Mon, Aug 8, 2011 at 3:13 PM, wing-tung Leung > wrote: >> I have a route where I perform some XSD validation first, and after >> that a HTTP download. The XSD validation is "optional", meaning that >> failure is simply logged but should not block the rest of the flow, >> while HTTP download exceptions should abort. [snip] >> Initially, I assumed that after the "silent" handling of >> ValidationException, the second exception (HTTP) handler would kick >> in, but it's never called. Is this because only 1 "onException" >> handler can be called per route? [snip] > What version of Camel are you using? Well, the not so new version 2.6.0, because we are currently still stuck to JDK 1.5. :-( I'm willing to try with a more recent version, but this involves a big bunch of local changes, and prefer to avoid it if not necessary ..
Enter multiple onException clauses in a single route?
I have a route where I perform some XSD validation first, and after that a HTTP download. The XSD validation is "optional", meaning that failure is simply logged but should not block the rest of the flow, while HTTP download exceptions should abort. I have defined two separate "onException" blocks, one for the "ValidationException" with "continued" enabled, and another for the "HttpOperationFailedException". They both seem to work fine independently, but for some reason, capturing both exceptions is not possible. Initially, I assumed that after the "silent" handling of ValidationException, the second exception (HTTP) handler would kick in, but it's never called. Is this because only 1 "onException" handler can be called per route? Wrapping the first XSD check into a explicit doTry/doCatch does work like I expected .. Someone some suggestion how to implement this best? Many thanks! Tung
