Modifying route using adviceWith showing inconsistent behaviour
Hello,
We have a system where we want to monitor all the exchanges flowing
throw camel pipeline (track completed, inprogress, failed exchanges).
*Camel context are added at runtime*. We want enrich pipeline to trigger
start and end of work flow, also add error handler to pipeline. Start
interceptor marks exchange as inprogress, end interceptor marks exchange
as completed and error handler marks exchange as failed in the database.
We are loading camel context from spring xml, then in java using
RouteDefination's adviceWith method modifying route to add start, end
intercepters and error handler.
Camel context spring xml:
Adding error handler using
/route.adviceWith(camelContext, new RouteBuilder() {
public void configure() throws Exception {
onException(Exception.class).handled(true).process(ERROR_HANDLER);
}
});
/Adding start intercepter using
/routeDef.adviceWith(camelContext, new RouteBuilder() {
public void configure() throws Exception {
interceptFrom().process(START_INTERCEPTER);
}
});
/Adding end intercepter using
/routeDef.adviceWith(camelContext, new RouteBuilder() {
public void configure() throws Exception {
onCompletion().process(END_INTERCEPTER);
}
});
/
Now when we try to send exchange on input channel using
/Exchange exchange1 = new DefaultExchange(context);
exchange1.getIn().setBody("Input1");
context.createProducerTemplate().send("direct://input", exchange1);
/
it throws NPE and* *our error handler is called.
java.lang.NullPointerException
at
org.apache.camel.model.InterceptFromDefinition$1.evaluate(InterceptFromDefinition.java:77)
at
org.apache.camel.builder.ProcessorBuilder$4.process(ProcessorBuilder.java:94)
at
org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:50)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
When we try to send exchange using /
Exchange exchange2 = new DefaultExchange(context);
exchange2.getIn().setBody("Input2");
Endpoint endpoint = context.getEndpoint("direct://input");
exchange2.setFromEndpoint(endpoint);
context.createProducerTemplate().send(endpoint, exchange2);/
it pass all the way to pipeline very well. *But in case of exception in
pipeline default error handler gets called and not the error handler
that we added*.
Are we doing in a wrong way or there is any problem with camel when
modifying routes at runtime.
Any help appreciated in advance.
P.S.: maven project attached.
Regards,
Sumit Teke.
camel-sample.tar.gz
Description: GNU Zip compressed data
BindyFixedLengthDataFormat trimming lines
Hi,
We are running Camel 2.4 and are to unmarshal FixedLength lines to objects
using Bindy.
The FLV record is defined with:
@FixedLengthRecord(length = 128, paddingChar = ' ')
I.e. 128 character long with spaces as default paddingchar.
The last "field" in the line is defined as:
@DataField(pos = 62, length = 67, align = "L")
I.e. it takes up the last 67 characters, left aligned using the default
paddingchar.
This implies, that if this last field does not contain any data, the last 67
characters of the FixedLength line will be spaces.
When feeding a line with no data in the last field into the
BindyFixedLengthDataFormat it will fail, because the codes does this when
reading the line:
...
while (scanner.hasNextLine()) {
// Read the line
String line = scanner.nextLine().trim();
LOG.debug("Handling line : " + line);
...
The trim() will remove the last 67 characters in the line (as they are
spaces) and then this part of the code:
...
// Check if the record length corresponds to the parameter
// provided in the @FixedLengthRecord
if ((line.length() < factory.recordLength()) || (line.length() >
factory.recordLength())) {
throw new java.lang.IllegalArgumentException("Size of the record : " +
line.length() + " is not equal to the value provided in the model : " +
factory.recordLength() + " !");
}
...
will throw an exception telling that the size of the record length is not
equal
The same problem will arise in the case of leading spaces in the line.
How come the lines are trimmed?
When working with FixedLength records it must be the resposibility of the
user to ensure the records are of proper length - so the code should, in my
view, not perform any modifications (like trimming) of the records.
Regards
Mikael
--
View this message in context:
http://camel.465427.n5.nabble.com/BindyFixedLengthDataFormat-trimming-lines-tp3374142p3374142.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: BindyFixedLengthDataFormat trimming lines
Hi, Hmm - it looks like the trim() is removed in the 2.6 release so I'll see if I can give it a try. /Mikael -- View this message in context: http://camel.465427.n5.nabble.com/BindyFixedLengthDataFormat-trimming-lines-tp3374142p3374157.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Modifying route using adviceWith showing inconsistent behaviour
Hi, I took a look at your code and I see that when you add interceptors you do not stop the running camel context or the routes and then re-start it the context or routes after adding the interceptors. Please update your code and try again. Cheers, Ashwin... - - Ashwin Karpe Apache Camel Committer & Sr Principal Consultant FUSESource (a Progress Software Corporation subsidiary) http://fusesource.com http://fusesource.com Blog: http://opensourceknowledge.blogspot.com http://opensourceknowledge.blogspot.com - -- View this message in context: http://camel.465427.n5.nabble.com/Modifying-route-using-adviceWith-showing-inconsistent-behaviour-tp3374110p3374232.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Modifying route using adviceWith showing inconsistent behaviour
Hi, I took a look at your code and I see that when you add interceptors you do not stop the running camel context or the routes and then re-start it the context or routes after adding the interceptors. Please update your code and try again. Cheers, Ashwin... - - Ashwin Karpe Apache Camel Committer & Sr Principal Consultant FUSESource (a Progress Software Corporation subsidiary) http://fusesource.com http://fusesource.com Blog: http://opensourceknowledge.blogspot.com http://opensourceknowledge.blogspot.com - -- View this message in context: http://camel.465427.n5.nabble.com/Modifying-route-using-adviceWith-showing-inconsistent-behaviour-tp3374110p3374231.html Sent from the Camel - Users mailing list archive at Nabble.com.
Http component doesn't send Content-Type
Hi,
I've a problem with this simple route (Camel 2.6.0):
from("servlet:///myServlet?matchOnUriPrefix=true").to("http://localhost:8080/service?bridgeEndpoint=true";)
If I send an HTTP packet to the servlet, the Content-Type header is removed.
The same behavior if I force the Content-Type between from and to.
I saw that this bug was resolved in an older version:
https://issues.apache.org/jira/browse/CAMEL-1609
It is quite strange, but in the last version it seems to be the same problem.
Where am I wrong?
Thanks.
Re: Http component doesn't send Content-Type
On Mon, Feb 7, 2011 at 2:31 PM, Cappa Roberto
wrote:
> Hi,
>
> I've a problem with this simple route (Camel 2.6.0):
>
> from("servlet:///myServlet?matchOnUriPrefix=true").to("http://localhost:8080/service?bridgeEndpoint=true";)
>
> If I send an HTTP packet to the servlet, the Content-Type header is removed.
> The same behavior if I force the Content-Type between from and to.
>
> I saw that this bug was resolved in an older version:
> https://issues.apache.org/jira/browse/CAMEL-1609
>
> It is quite strange, but in the last version it seems to be the same problem.
> Where am I wrong?
>
Its fixed on trunk.
There is a fairly new ticket about the problem.
> Thanks.
>
--
Claus Ibsen
-
FuseSource
Email: cib...@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Modifying route using adviceWith showing inconsistent behaviour
Hi Ashwin, Thanks for your reply. adviceRoute javadocs mentioned "Will stop and remove the old route from camel context and add and start this new advised route." When tried with restarting camel context, got NPE while advice route for start interception Exception in thread "main" java.lang.NullPointerException at org.apache.camel.impl.DefaultCamelContext.stopRouteService(DefaultCamelContext.java:1564) at org.apache.camel.impl.DefaultCamelContext.stopRoute(DefaultCamelContext.java:651) at org.apache.camel.impl.DefaultCamelContext.removeRouteDefinition(DefaultCamelContext.java:596) at org.apache.camel.model.RouteDefinition.adviceWith(RouteDefinition.java:221) at CamelContextLoader.addStartIntercepter(CamelContextLoader.java:54) at CamelContextLoader.load(CamelContextLoader.java:21) at CamelContextLoader.main(CamelContextLoader.java:105) When tried with restarting route which is getting modified and got exception Exception in thread "main" java.lang.IllegalArgumentException: Cannot add a 2nd consumer to the same endpoint. Endpoint Endpoint[direct://input] only allows one consumer. at org.apache.camel.component.direct.DirectConsumer.doStart(DirectConsumer.java:45) as route is already started when we do adviceRoute. Attaching modified code. Thanks, Sumit Teke Omniscient Software Pvt Ltd T. +91-20-26680814, Ext. 217 F. +91-20-26680815, Ext. 212 M. +91-9975709032 E. sumit_t...@omniscient.co.in On Monday 07 February 2011 06:50 PM, Ashwin Karpe wrote: Hi, I took a look at your code and I see that when you add interceptors you do not stop the running camel context or the routes and then re-start it the context or routes after adding the interceptors. Please update your code and try again. Cheers, Ashwin... - - Ashwin Karpe Apache Camel Committer& Sr Principal Consultant FUSESource (a Progress Software Corporation subsidiary) http://fusesource.com http://fusesource.com Blog: http://opensourceknowledge.blogspot.com http://opensourceknowledge.blogspot.com - camel-sample.tar.gz Description: GNU Zip compressed data
Re: Memory leak in camel mail component
I've just used jConsole to analyze our application after running it through the weekend, many endpoints URI have been served and all of them have been cached. It appears it is not caching 1000 endpoints at most. -- View this message in context: http://camel.465427.n5.nabble.com/Memory-leak-in-camel-mail-component-tp3364793p3374332.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Memory leak in camel mail component
On Mon, Feb 7, 2011 at 3:38 PM, Marco Crivellaro wrote: > > I've just used jConsole to analyze our application after running it through > the weekend, many endpoints URI have been served and all of them have been > cached. It appears it is not caching 1000 endpoints at most. Using jconsole you are then viewing JMX ? And as said the endpoints in JMX is not restricted to 1000. All endpoints is registered in JMX. We are debating to alter this at the dev mailing list. And also as said provide information about mail recipients as headers to the message and not in the uri, to re-use same endpoint. > -- > View this message in context: > http://camel.465427.n5.nabble.com/Memory-leak-in-camel-mail-component-tp3364793p3374332.html > Sent from the Camel - Users mailing list archive at Nabble.com. > -- Claus Ibsen - FuseSource Email: cib...@fusesource.com Web: http://fusesource.com Twitter: davsclaus Blog: http://davsclaus.blogspot.com/ Author of Camel in Action: http://www.manning.com/ibsen/
Re: Memory leak in camel mail component
Yes I am viewing JMX, sorry. but mail recipients or ftp parameters in URI won't allow us to make use of the recipientList which is very handy and it is on the basis of our application (we are serving a very high number of endpoints dynamically using the recipientList). -- View this message in context: http://camel.465427.n5.nabble.com/Memory-leak-in-camel-mail-component-tp3364793p3374535.html Sent from the Camel - Users mailing list archive at Nabble.com.
ETL: how to monitor the end of processing ?
Hello, I'm currently evaluating Camel for Data Integration purpose. So far, it works very well, but I'm facing the following issue in my test case : when dealing with a (not so) large file (5k lines), the Camel Context is shutdown before all lines are processed, and some assertions (MockEndpoint.expectedMessageCount()) fail. Still, I can by-pass this issue by adding a Thread.sleep() in my test and get the expected result. (Unless there 's some better way to proceed ?) Now, in a context of ETL, the first step would be loading data into a staging database with Camel, and *then* (2nd step) perform some business controls before publishing in production (3rd step). My question is : how can I detect that the initial/input file was fully treated (end of 1st step) before starting the business controls ? Would you recommend using JMX or BAM ? Or can a listener be notified when some queue was idle for a given time ? Thank you in advance Regards, Xavier -- View this message in context: http://camel.465427.n5.nabble.com/ETL-how-to-monitor-the-end-of-processing-tp3374562p3374562.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Memory leak in camel mail component
Is disabling JMX an option for you? This could be a workaround/solution for you. Christian
Re: ETL: how to monitor the end of processing ?
Hi Xavier, I suspect camel did the right thing. Camel waits for in-flight messages to complete [1]. I don't know how you set up your routes, but I suspect that you stop the consumer in one of the steps while there are still messages to consume. Since the consumer didn't grab them yet, and hence they are not in flight camel continues and shuts down normally. Since this scenario is about correlation of activities, I believe BAM is your best bet. JMX would work too, would take more effort to implement I think. For your first question, how can you know that the input was fully treated, that's applications specific. Camel cannot know what your input is and when it finishes, it only consumes what's available. Therefore you'll need to check some condition before you stop camel. In essence it's what you do with your Thread.sleep in an indirect way, you are checking if enough time passed since you started to accept input. Since the check is not explicit, but somewhat implicit, you ensure the condition is met by using the sleep. Cheers, Hadrian [1] http://camel.apache.org/graceful-shutdown.html On Feb 7, 2011, at 11:56 AM, Xavier Coulon wrote: > > Hello, > > I'm currently evaluating Camel for Data Integration purpose. So far, it > works very well, but I'm facing the following issue in my test case : when > dealing with a (not so) large file (5k lines), the Camel Context is shutdown > before all lines are processed, and some assertions > (MockEndpoint.expectedMessageCount()) fail. Still, I can by-pass this issue > by adding a Thread.sleep() in my test and get the expected result. (Unless > there 's some better way to proceed ?) > > Now, in a context of ETL, the first step would be loading data into a > staging database with Camel, and *then* (2nd step) perform some business > controls before publishing in production (3rd step). > My question is : how can I detect that the initial/input file was fully > treated (end of 1st step) before starting the business controls ? > Would you recommend using JMX or BAM ? Or can a listener be notified when > some queue was idle for a given time ? > > Thank you in advance > Regards, > Xavier > -- > View this message in context: > http://camel.465427.n5.nabble.com/ETL-how-to-monitor-the-end-of-processing-tp3374562p3374562.html > Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Modifying route using adviceWith showing inconsistent behaviour
Hi, If you stop the CamelContext all the routes are automatically stopped. Alternatively, you can stop the routes and make changes without stopping the context and then start the routes individually. As for the second error, it is quite clear... Looks like you are starting two routes with the same from(...) endpoint or you have two routes with the same consumer. Cheers, Ashwin... - - Ashwin Karpe Apache Camel Committer & Sr Principal Consultant FUSESource (a Progress Software Corporation subsidiary) http://fusesource.com http://fusesource.com Blog: http://opensourceknowledge.blogspot.com http://opensourceknowledge.blogspot.com - -- View this message in context: http://camel.465427.n5.nabble.com/Modifying-route-using-adviceWith-showing-inconsistent-behaviour-tp3374110p3374870.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: ETL: how to monitor the end of processing ?
Hello Hadrian, Thank you for those answers. I'll have a look at the BAM facilities provided by Camel. I agree that there needs to be some application-level specifics to implement (such as the total number of treated messages I should look at, including errors...) I actually agree with you that Camel just did what it had to do, except it seems that it shut down the file consumer before the file was actually fully consumed. I'll have a look at the context graceful shutdown, but I thought it was the default behavior in Camel 2.4 (the Fuse version I use) Regards, Xavier On Mon, Feb 7, 2011 at 7:47 PM, Hadrian Zbarcea wrote: > Hi Xavier, > > I suspect camel did the right thing. Camel waits for in-flight messages to > complete [1]. > I don't know how you set up your routes, but I suspect that you stop the > consumer in one of the steps while there are still messages to consume. > Since the consumer didn't grab them yet, and hence they are not in flight > camel continues and shuts down normally. > > Since this scenario is about correlation of activities, I believe BAM is > your best bet. JMX would work too, would take more effort to implement I > think. > For your first question, how can you know that the input was fully treated, > that's applications specific. Camel cannot know what your input is and when > it finishes, it only consumes what's available. Therefore you'll need to > check some condition before you stop camel. In essence it's what you do with > your Thread.sleep in an indirect way, you are checking if enough time passed > since you started to accept input. Since the check is not explicit, but > somewhat implicit, you ensure the condition is met by using the sleep. > > Cheers, > Hadrian > > > > [1] http://camel.apache.org/graceful-shutdown.html > > > On Feb 7, 2011, at 11:56 AM, Xavier Coulon wrote: > > > > > Hello, > > > > I'm currently evaluating Camel for Data Integration purpose. So far, it > > works very well, but I'm facing the following issue in my test case : > when > > dealing with a (not so) large file (5k lines), the Camel Context is > shutdown > > before all lines are processed, and some assertions > > (MockEndpoint.expectedMessageCount()) fail. Still, I can by-pass this > issue > > by adding a Thread.sleep() in my test and get the expected result. > (Unless > > there 's some better way to proceed ?) > > > > Now, in a context of ETL, the first step would be loading data into a > > staging database with Camel, and *then* (2nd step) perform some business > > controls before publishing in production (3rd step). > > My question is : how can I detect that the initial/input file was fully > > treated (end of 1st step) before starting the business controls ? > > Would you recommend using JMX or BAM ? Or can a listener be notified when > > some queue was idle for a given time ? > > > > Thank you in advance > > Regards, > > Xavier > > -- > > View this message in context: > http://camel.465427.n5.nabble.com/ETL-how-to-monitor-the-end-of-processing-tp3374562p3374562.html > > Sent from the Camel - Users mailing list archive at Nabble.com. > > -- Xavier
JMSConfiguration - Camel vs CXF
I have a CXF configuration (in applicationContext.xml) that talks to IBM MQ via JNDI resources. I'd like to use this cxf-configured message queue as a Camel destination. I'm stuck on the difference between org.apache.cxf.transport.jms.JMSConfiguration and org.apache.camel.component.jms.JmsConfiguration, and whether I can convince Camel to talk to my cxf-configured message queue.
Re: Modifying route using adviceWith showing inconsistent behaviour
Hi Ashwin,
Sorry i think, i was not clear in last post.
I understand that stopping camel context will stop all the routes. What
i did was
*Test 1: *Tried stopping camel context and then modified routes and
again started the context
/CamelContext camelContext = (CamelContext)
context.getBean(CamelContext.class);
camelContext.stop();
addErrorHandler(camelContext);
addStartIntercepter(camelContext);
addEndIntercepter(camelContext);
camelContext.start();
/but when i run this code it throws NPE while advicing route for start
intercepter
Exception in thread "main" java.lang.NullPointerException
at
org.apache.camel.impl.DefaultCamelContext.stopRouteService(DefaultCamelContext.java:1564)
at
org.apache.camel.impl.DefaultCamelContext.stopRoute(DefaultCamelContext.java:651)
at
org.apache.camel.impl.DefaultCamelContext.removeRouteDefinition(DefaultCamelContext.java:596)
at
org.apache.camel.model.RouteDefinition.adviceWith(RouteDefinition.java:221)
at CamelContextLoader.addStartIntercepter(CamelContextLoader.java:54)
at CamelContextLoader.load(CamelContextLoader.java:21)
at CamelContextLoader.main(CamelContextLoader.java:105)
*Test 2: *Tried stopping routes before advicing and starting the new
modified route, but as advice adds new modified route to context and
starts it, so it throws the multiple consumer exception.
camelContext.stopRoute(route);
route = route.adviceWith(camelContext, new RouteBuilder() {
public void configure() throws Exception {
System.out.println(">> adding error handler on route "
+ routeId);
onException(Exception.class).handled(true).process(ERROR_HANDLER);
}
});
camelContext.startRoute(route);
/
/
Thanks,
Sumit Teke
Omniscient Software Pvt Ltd
T. +91-20-26680814, Ext. 217
F. +91-20-26680815, Ext. 212
M. +91-9975709032
E. sumit_t...@omniscient.co.in
On Tuesday 08 February 2011 01:57 AM, Ashwin Karpe wrote:
Hi,
If you stop the CamelContext all the routes are automatically stopped.
Alternatively, you can stop the routes and make changes without stopping the
context and then start the routes individually.
As for the second error, it is quite clear... Looks like you are starting
two routes with the same from(...) endpoint or you have two routes with the
same consumer.
Cheers,
Ashwin...
-
-
Ashwin Karpe
Apache Camel Committer& Sr Principal Consultant
FUSESource (a Progress Software Corporation subsidiary)
http://fusesource.com http://fusesource.com
Blog: http://opensourceknowledge.blogspot.com
http://opensourceknowledge.blogspot.com
-
Re: Memory leak in camel mail component
On Mon, Feb 7, 2011 at 5:38 PM, Marco Crivellaro wrote: > > Yes I am viewing JMX, sorry. > > > but mail recipients or ftp parameters in URI won't allow us to make use of > the recipientList which is very handy and it is on the basis of our > application (we are serving a very high number of endpoints dynamically > using the recipientList). Yes it does you can just use a processor / bean or whatever to set the information as a header instead of uri parameters. The recipient list can still be used as well. btw the talk on the dev list is: http://camel.465427.n5.nabble.com/DISCUSS-JMX-Enlisting-MBeans-only-during-starting-routes-tp3371184p3371184.html In terms of MBean listed. What types of MBeans do you see a lot in JMX? And what parameters are you using on those? There is a getEndpointKey method on Endpoint we could adjust to resolve a coarse grained key for endpoints which has recipient information in the uri. We do this for a few components. This allows us to avoid registering too many endpoints in JMX and thus it could be a resolution to your issue. But having some samples would help implement a solution quicker. > -- > View this message in context: > http://camel.465427.n5.nabble.com/Memory-leak-in-camel-mail-component-tp3364793p3374535.html > Sent from the Camel - Users mailing list archive at Nabble.com. > -- Claus Ibsen - FuseSource Email: cib...@fusesource.com Web: http://fusesource.com Twitter: davsclaus Blog: http://davsclaus.blogspot.com/ Author of Camel in Action: http://www.manning.com/ibsen/
Re: ETL: how to monitor the end of processing ?
On Mon, Feb 7, 2011 at 10:00 PM, Xavier Coulon wrote: > Hello Hadrian, > > Thank you for those answers. I'll have a look at the BAM facilities provided > by Camel. I agree that there needs to be some application-level specifics to > implement (such as the total number of treated messages I should look at, > including errors...) > > I actually agree with you that Camel just did what it had to do, except it > seems that it shut down the file consumer before the file was actually fully > consumed. I'll have a look at the context graceful shutdown, but I thought > it was the default behavior in Camel 2.4 (the Fuse version I use) > If you use mock endpoints then they are used for testing purposes. When the assertItSatisified() method is invoked it will timeout after 10 seconds. So you would have to increase the timeout value using the setResultWaitTime method. > Regards, > Xavier > > On Mon, Feb 7, 2011 at 7:47 PM, Hadrian Zbarcea wrote: > >> Hi Xavier, >> >> I suspect camel did the right thing. Camel waits for in-flight messages to >> complete [1]. >> I don't know how you set up your routes, but I suspect that you stop the >> consumer in one of the steps while there are still messages to consume. >> Since the consumer didn't grab them yet, and hence they are not in flight >> camel continues and shuts down normally. >> >> Since this scenario is about correlation of activities, I believe BAM is >> your best bet. JMX would work too, would take more effort to implement I >> think. >> For your first question, how can you know that the input was fully treated, >> that's applications specific. Camel cannot know what your input is and when >> it finishes, it only consumes what's available. Therefore you'll need to >> check some condition before you stop camel. In essence it's what you do with >> your Thread.sleep in an indirect way, you are checking if enough time passed >> since you started to accept input. Since the check is not explicit, but >> somewhat implicit, you ensure the condition is met by using the sleep. >> >> Cheers, >> Hadrian >> >> >> >> [1] http://camel.apache.org/graceful-shutdown.html >> >> >> On Feb 7, 2011, at 11:56 AM, Xavier Coulon wrote: >> >> > >> > Hello, >> > >> > I'm currently evaluating Camel for Data Integration purpose. So far, it >> > works very well, but I'm facing the following issue in my test case : >> when >> > dealing with a (not so) large file (5k lines), the Camel Context is >> shutdown >> > before all lines are processed, and some assertions >> > (MockEndpoint.expectedMessageCount()) fail. Still, I can by-pass this >> issue >> > by adding a Thread.sleep() in my test and get the expected result. >> (Unless >> > there 's some better way to proceed ?) >> > >> > Now, in a context of ETL, the first step would be loading data into a >> > staging database with Camel, and *then* (2nd step) perform some business >> > controls before publishing in production (3rd step). >> > My question is : how can I detect that the initial/input file was fully >> > treated (end of 1st step) before starting the business controls ? >> > Would you recommend using JMX or BAM ? Or can a listener be notified when >> > some queue was idle for a given time ? >> > >> > Thank you in advance >> > Regards, >> > Xavier >> > -- >> > View this message in context: >> http://camel.465427.n5.nabble.com/ETL-how-to-monitor-the-end-of-processing-tp3374562p3374562.html >> > Sent from the Camel - Users mailing list archive at Nabble.com. >> >> > > > -- > Xavier > -- Claus Ibsen - FuseSource Email: cib...@fusesource.com Web: http://fusesource.com Twitter: davsclaus Blog: http://davsclaus.blogspot.com/ Author of Camel in Action: http://www.manning.com/ibsen/
Re: ETL: how to monitor the end of processing ?
On Mon, Feb 7, 2011 at 5:56 PM, Xavier Coulon wrote: > > Hello, > > I'm currently evaluating Camel for Data Integration purpose. So far, it > works very well, but I'm facing the following issue in my test case : when > dealing with a (not so) large file (5k lines), the Camel Context is shutdown > before all lines are processed, and some assertions > (MockEndpoint.expectedMessageCount()) fail. Still, I can by-pass this issue > by adding a Thread.sleep() in my test and get the expected result. (Unless > there 's some better way to proceed ?) > > Now, in a context of ETL, the first step would be loading data into a > staging database with Camel, and *then* (2nd step) perform some business > controls before publishing in production (3rd step). > My question is : how can I detect that the initial/input file was fully > treated (end of 1st step) before starting the business controls ? You can also look at on completion http://camel.apache.org/oncompletion.html That allows you to do additional route after the exchange has been completed. For example to signal some internal state that you are done or whatever. > Would you recommend using JMX or BAM ? Or can a listener be notified when > some queue was idle for a given time ? > Camel has an EventNotifier which acts as a global listener. http://camel.apache.org/eventnotifier-to-log-details-about-all-sent-exchanges.html And as Hadrian also pointed out there is BAM as well http://camel.apache.org/bam > Thank you in advance > Regards, > Xavier > -- > View this message in context: > http://camel.465427.n5.nabble.com/ETL-how-to-monitor-the-end-of-processing-tp3374562p3374562.html > Sent from the Camel - Users mailing list archive at Nabble.com. > -- Claus Ibsen - FuseSource Email: cib...@fusesource.com Web: http://fusesource.com Twitter: davsclaus Blog: http://davsclaus.blogspot.com/ Author of Camel in Action: http://www.manning.com/ibsen/
