Re: Return Value from Python Script Not In Message Body
I'd like to follow up on this as I've just hit the same issue. I know this thread dates from a year and a half ago, but it seems unanswered. The issue is that if you are using a Python script using the language component, it works fine if the script is a one line piece of text. e.g. the whole script is this: "Hello World!" In that case the return value of the script, and so what get's set to the message body is "Hello World!". But if its anything more complex like: val = 1 + 2 "Hello World!" then the return value from the script is null. I did find a workaround - set the return value to the request.body property in the Python script, and use the transform=false attribute for the language component URL. But was wondering what the way to get a multi-line script working correctly was. Tim -- View this message in context: http://camel.465427.n5.nabble.com/Return-Value-from-Python-Script-Not-In-Message-Body-tp5724056p5753009.html Sent from the Camel - Users mailing list archive at Nabble.com.
How to stream to file
I'm having problems working out how to write my output to file.
I'm wanting something that can work with the File component so that I
get the benefit of its flexibility (and can use FTP etc.).
But the problem is that I need to write the data using a class that uses
a File or OutputStream. Its something like this:
public class MyDataWriter {
public MyDataWriter(File file) { }
public MyDataWriter(OutputStream out) { }
public void write(MyData bar) { ... }
}
The workflow is something like this:
1. read data from file using File consumer
2. split it up into parts (e.g. lines) using a Splitter (streaming)
3. Process each part
4. Write modified parts to new file
The data is potentially large, so needs to support streaming.
I've hacked something up using AggregationStrategy for step 4 which sort
of works, but I feel there should be a better way.
I could write my own Component/Endpoint, but its feels like I'd be
re-writing the File component.
Somehow it feels like I should be able to use the File component (send
some type of OutputStream to it?) but I can't figure out how.
Any advice would be very welcome.
Thanks
Tim
Re: lazyLoad with CSV blows up on last line?
I don't think that's the problem. I added long sleep before shutting
down the camel context and still get the error (my test file is very
small so gets processed almost instantly).
So think it must be something else closing the stream prematurely.
Tim
On 05/05/2014 02:50, Willem Jiang wrote:
The Stream is closed by other thread.
I think you need to let the main thread sleep for few minutes to let the camel
route finish the processing of the message.
--
Willem Jiang
Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On May 4, 2014 at 11:26:27 PM, Tim Dudgeon (tdudgeon...@gmail.com) wrote:
I'm trying to use the CSV data format to handle a large file.
Thinking I should use CsvDataFormat.setLazyLoad(true)
I sort of got it working but it blows up on the last line of the file
with a java.io.IOException: Stream closed exception.
Is this a bug, or am I dong it wrong? This is with Camel 2.13.0
Here is example of what I'm doing:
CsvDataFormat csv = new CsvDataFormat()
csv.setDelimiter(' ')
csv.setSkipFirstLine(false)
csv.setLazyLoad(true)
CamelContext camelContext = new DefaultCamelContext()
camelContext.addRoutes(new RouteBuilder() {
def void configure() {
from('direct:start')
.unmarshal(csv)
.split(body()).streaming()
.log('row: ${body}')
}
})
camelContext.start()
ProducerTemplate t = camelContext.createProducerTemplate()
t.sendBody('direct:start', new File('/Users/timbo/data/test.txt'))
camelContext.stop()
And here is exception.
java.lang.IllegalStateException: java.io.IOException: Stream closed
at
org.apache.camel.dataformat.csv.CsvIterator.next(CsvIterator.java:61)
at
org.apache.camel.processor.Splitter$SplitterIterable$1.next(Splitter.java:170)
at
org.apache.camel.processor.Splitter$SplitterIterable$1.next(Splitter.java:146)
at
org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:502)
at
org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:215)
at org.apache.camel.processor.Splitter.process(Splitter.java:98)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at
org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:51)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at
org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
at
org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
at
org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:346)
at
org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
at
org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
at
org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
at
org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
at
org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
at org.apache.camel.ProducerTemplate$sendBody.call(Unknown Source)
Thanks
Tim
lazyLoad with CSV blows up on last line?
I'm trying to use the CSV data format to handle a large file.
Thinking I should use CsvDataFormat.setLazyLoad(true)
I sort of got it working but it blows up on the last line of the file
with a java.io.IOException: Stream closed exception.
Is this a bug, or am I dong it wrong? This is with Camel 2.13.0
Here is example of what I'm doing:
CsvDataFormat csv = new CsvDataFormat()
csv.setDelimiter(' ')
csv.setSkipFirstLine(false)
csv.setLazyLoad(true)
CamelContext camelContext = new DefaultCamelContext()
camelContext.addRoutes(new RouteBuilder() {
def void configure() {
from('direct:start')
.unmarshal(csv)
.split(body()).streaming()
.log('row: ${body}')
}
})
camelContext.start()
ProducerTemplate t = camelContext.createProducerTemplate()
t.sendBody('direct:start', new File('/Users/timbo/data/test.txt'))
camelContext.stop()
And here is exception.
java.lang.IllegalStateException: java.io.IOException: Stream closed
at
org.apache.camel.dataformat.csv.CsvIterator.next(CsvIterator.java:61)
at
org.apache.camel.processor.Splitter$SplitterIterable$1.next(Splitter.java:170)
at
org.apache.camel.processor.Splitter$SplitterIterable$1.next(Splitter.java:146)
at
org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:502)
at
org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:215)
at org.apache.camel.processor.Splitter.process(Splitter.java:98)
at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at
org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:51)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at
org.apache.camel.processor.UnitOfWorkProducer.process(UnitOfWorkProducer.java:73)
at
org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:378)
at
org.apache.camel.impl.ProducerCache$2.doInProducer(ProducerCache.java:346)
at
org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:242)
at
org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:346)
at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:184)
at
org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:124)
at
org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:137)
at
org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:144)
at org.apache.camel.ProducerTemplate$sendBody.call(Unknown Source)
Thanks
Tim
Re: MalformedObjectNameException when starting camel context
Yes, that seems to explain it. The hostname was indeed the cause of the
problem.
Thanks
Tim
On 31/12/2012 02:40, Liam Clarke-Hutchinson wrote:
Hi Tim,
It's trying to get an mbean or similar with a name derived from (I
guess) your machine's IP - which is in IPV6, hence lots of colons.
However, if you look at the documentation for ObjectName
(http://docs.oracle.com/javase/1.5.0/docs/api/javax/management/ObjectName.html)
you'll see it states:
An ObjectName can be written as a String with the following elements in order:
The domain.
A colon (:).
A key property list as defined below.
And also
An unquoted value is a possibly empty string of characters which may not
contain any of the characters comma, equals, colon, quote, asterisk, or
question mark.
So it's hitting the first colon, starts interpreting the rest of the
IP as a property, hits the next colon while parsing it as a property
value, and blows up because colons aren't allowed.
Looking at what DefaultManagementNamingStrategy.createObjectName does,
it ultimately calls java.net.InetAddress#getLocalHost() to get that
IP. Maybe you need to alias localhost in your /etc/hosts?
Regards,
Liam Clarke
On Mon, Dec 31, 2012 at 2:38 PM, Willem Jiang wrote:
Hi,
It looks something is wrong with you host name. Can you just change it?
Willem
在 2012-12-30，下午11:45，Tim Dudgeon 写道：
I'm seeing a strange error when starting a camel context. Its dependent on the
machine I'm running on. On one machine (Windows) the same context and simple
route runsfine, but on a different machine (Mac) the context fails to start.
The key part seems to be this:
javax.management.MalformedObjectNameException: Could not create ObjectName from:
org.apache.camel:context=unknown-00:25:4b:c4:94:f4.home/camel-1,type=context,name="camel-1".
Reason: javax.management.MalformedObjectNameException: Invalid character ':' in value
part of property
The context and route is very simple:
def camelContext = new DefaultCamelContext()
camelContext.addRoutes(new RouteBuilder() {
def void configure() {
from("timer://jdkTimer?period=3000")
.to("log://camelLogger?level=INFO")
}
})
This is using Camel 2.10.3.
Any ideas what might be going wrong?
The full error is here:
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java
-Dtools.jar=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/lib/tools.jar
-Dgroovy.home=/Users/timbo/etc/groovy/groovy-1.8.8
-Dgroovy.starter.conf=/Users/timbo/etc/groovy/groovy-1.8.8/conf/groovy-starter.conf
-Didea.launcher.port=7532 "-Didea.launcher.bin.path=/Applications/IntelliJ IDEA 11
CE.app/bin" -Dfile.encoding=UTF-8 -classpath
"/Users/timbo/etc/groovy/groovy-1.8.8/lib/groovy-1.8.8.jar:/Applications/IntelliJ IDEA 11
CE.app/lib/idea_rt.jar" com.intellij.rt.execution.application.AppMain
org.codehaus.groovy.tools.GroovyStarter --conf
/Users/timbo/etc/groovy/groovy-1.8.8/conf/groovy-starter.conf --main groovy.ui.GroovyMain
--classpath
/Users/timbo/tmp/TestingCamel/out/production/TestingCamel:/Users/timbo/etc/groovy/groovy-1.8.8/lib/ant-1.8.3.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/ant-antlr-1.8.3.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/ant-junit-1.8.3.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/ant-launcher-1.8.3.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/antlr-2.7.7.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/asm-3.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/asm-analysis-3.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/asm-commons-3.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/asm-tree-3.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/asm-util-3.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/bsf-2.4.0.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/commons-cli-1.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/commons-logging-1.1.1.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/gpars-1.0-beta-3.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/groovy-1.8.8.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/hamcrest-core-1.1.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/ivy-2.2.0.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/jansi-1.6.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/jline-1.0.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/jsp-api-2.0.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/jsr166y-1.7.0.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/junit-4.10.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/servlet-api-2.4.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/xmlpull-1.1.3.1.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/xstream-1.4.1.jar:/Users/timbo/tmp/TestingCamel/lib/camel-core-2.10.3.jar:/Users/timbo/tmp/TestingCamel/lib/slf4j-api-1.6.6.jar:/Users/timbo/tmp/TestingCamel/lib/slf4j-simple-1.6.6.jar
--encoding=UTF-8 /Users/timbo/tmp/TestingCamel/src/Simple.groovy
2347 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel
2.10.3 (CamelContext: camel-1) is starting
38
MalformedObjectNameException when starting camel context
I'm seeing a strange error when starting a camel context. Its dependent
on the machine I'm running on. On one machine (Windows) the same
context and simple route runsfine, but on a different machine (Mac) the
context fails to start. The key part seems to be this:
javax.management.MalformedObjectNameException: Could not create
ObjectName from:
org.apache.camel:context=unknown-00:25:4b:c4:94:f4.home/camel-1,type=context,name="camel-1".
Reason: javax.management.MalformedObjectNameException: Invalid character
':' in value part of property
The context and route is very simple:
def camelContext = new DefaultCamelContext()
camelContext.addRoutes(new RouteBuilder() {
def void configure() {
from("timer://jdkTimer?period=3000")
.to("log://camelLogger?level=INFO")
}
})
This is using Camel 2.10.3.
Any ideas what might be going wrong?
The full error is here:
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java -Dtools.jar=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/lib/tools.jar
-Dgroovy.home=/Users/timbo/etc/groovy/groovy-1.8.8
-Dgroovy.starter.conf=/Users/timbo/etc/groovy/groovy-1.8.8/conf/groovy-starter.conf
-Didea.launcher.port=7532
"-Didea.launcher.bin.path=/Applications/IntelliJ IDEA 11 CE.app/bin"
-Dfile.encoding=UTF-8 -classpath
"/Users/timbo/etc/groovy/groovy-1.8.8/lib/groovy-1.8.8.jar:/Applications/IntelliJ
IDEA 11 CE.app/lib/idea_rt.jar"
com.intellij.rt.execution.application.AppMain
org.codehaus.groovy.tools.GroovyStarter --conf
/Users/timbo/etc/groovy/groovy-1.8.8/conf/groovy-starter.conf --main
groovy.ui.GroovyMain --classpath
/Users/timbo/tmp/TestingCamel/out/production/TestingCamel:/Users/timbo/etc/groovy/groovy-1.8.8/lib/ant-1.8.3.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/ant-antlr-1.8.3.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/ant-junit-1.8.3.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/ant-launcher-1.8.3.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/antlr-2.7.7.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/asm-3.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/asm-analysis-3.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/asm-commons-3.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/asm-tree-3.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/asm-util-3.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/bsf-2.4.0.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/commons-cli-1.2.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/commons-logging-1.1.1.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/gpars-1.0-beta-3.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/groovy-1.8.8.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/hamcrest-core-1.1.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/ivy-2.2.0.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/jansi-1.6.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/jline-1.0.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/jsp-api-2.0.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/jsr166y-1.7.0.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/junit-4.10.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/servlet-api-2.4.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/xmlpull-1.1.3.1.jar:/Users/timbo/etc/groovy/groovy-1.8.8/lib/xstream-1.4.1.jar:/Users/timbo/tmp/TestingCamel/lib/camel-core-2.10.3.jar:/Users/timbo/tmp/TestingCamel/lib/slf4j-api-1.6.6.jar:/Users/timbo/tmp/TestingCamel/lib/slf4j-simple-1.6.6.jar
--encoding=UTF-8 /Users/timbo/tmp/TestingCamel/src/Simple.groovy
2347 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache
Camel 2.10.3 (CamelContext: camel-1) is starting
3832 [main] INFO org.apache.camel.management.ManagementStrategyFactory -
JMX enabled.
3996 [main] WARN org.apache.camel.impl.DefaultCamelContext - Lifecycle
strategy
org.apache.camel.management.DefaultManagementLifecycleStrategy@410d3f0d
failed starting CamelContext (camel-1) due
javax.management.MalformedObjectNameException: Could not create
ObjectName from:
org.apache.camel:context=unknown-00:25:4b:c4:94:f4.home/camel-1,type=context,name="camel-1".
Reason: javax.management.MalformedObjectNameException: Invalid character
':' in value part of property
4004 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache
Camel 2.10.3 (CamelContext: camel-1) is shutting down
4051 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache
Camel 2.10.3 (CamelContext: camel-1) is shutdown in 0.012 seconds.
Uptime 1.706 seconds.
Caught: org.apache.camel.RuntimeCamelException:
javax.management.MalformedObjectNameException: Could not create
ObjectName from:
org.apache.camel:context=unknown-00:25:4b:c4:94:f4.home/camel-1,type=context,name="camel-1".
Reason: javax.management.MalformedObjectNameException: Invalid character
':' in value part of property
org.apache.camel.RuntimeCamelException:
javax.management.MalformedObjectNameException: Could not create
ObjectName from:
org.apache.camel:context=unknown-00:25:4b:c4:94:f4.home/camel-1,type=context,name="camel-1".
Reason: javax.management.MalformedObjectNameException: Invalid c
Re: lifecycle of components
Thanks, Got it working with a Processor. start() seems to be called twice. Is this expected? Is it possible to use this for beans? It seems not? e.g. if a route has this .bean(new MyBean(), 'foo') and MyBean implements Service then the start() method is not called. So using a Processor is the only option? Tim On 02/10/2012 15:03, Claus Ibsen wrote: On Tue, Oct 2, 2012 at 3:38 PM, Tim Dudgeon wrote: Are there any tricks or patterns to use when it comes to needing to manage the lifecyle of components used in Camel routes. For instance if I write a custom Processor that creates something like a PreparedStatement how can I make sure its is close()'d when the CamelContext shuts down. See http://camel.apache.org/lifecycle.html You can implement the Service interface, or extend ServiceSupport. Then you have callbacks when its started/stopped etc. Thanks Tim
Re: Reading CSV as Map not List
On 02/10/2012 19:07, Christian Müller wrote: That would be a incompatible change. And what if the user don't want to specify the keys? I was more asking if there was an option to allow this, not asking for a change in behaviour. Would be useful e.g. if the data was to be inserted into a database using SQL, where the keys were the column names. I think it could somehow be done with a custom Aggregator, but hoping there was already a solution. Do you have an issue with the current implementation? Not at all. Just doesn't support my use case ;-) Tim Best, Christian Sent from a mobile device Am 02.10.2012 17:05 schrieb "Tim Dudgeon" : Is it possible to use the CSV component to read a CSV file and return a List>? The keys being the values in the first line of the file. I can only see a way to get it to generate a List>. Thanks Tim
Reading CSV as Map not List
Is it possible to use the CSV component to read a CSV file and return a List>? The keys being the values in the first line of the file. I can only see a way to get it to generate a List>. Thanks Tim
lifecycle of components
Are there any tricks or patterns to use when it comes to needing to manage the lifecyle of components used in Camel routes. For instance if I write a custom Processor that creates something like a PreparedStatement how can I make sure its is close()'d when the CamelContext shuts down. Thanks Tim
Must File component write the body?
When using the File component to write to a file is it always the message body that is written, or is it possible to use an expression (e.g. simple)? Of course body can be replaced with new content using an expression, but is that the only way to do this? Tim
Re: SEDA and concurrentConsumers
OK, thanks. So I think I get it now. Some params need defining on the
'from' side and some on the 'to' side. And the order the queue appears
in the routes is important. So this seems to work for me. Is this correct?
from('seda:insert?concurrentConsumers=2&size=3')
.delay(1000)
.log('seda ${body}')
from('direct:start')
.split(body())
.log('direct ${body}')
.to('seda:insert?blockWhenFull=true')
For instance, if the two routes are defined in the other order its
doesn't work as wanted.
Tim
On 12/09/2012 07:54, Claus Ibsen wrote:
On Wed, Sep 12, 2012 at 8:52 AM, Tim Dudgeon wrote:
Great. That works. Thanks.
BTW, the example in the Camel in Action book (p 322) seems wrong then as it
describes it the way I originally had it.
The book is not wrong. The block parameter is on the *producer* side,
so you should have it in the "to".
Tim
On 12/09/2012 03:34, Willem jiang wrote:
You need set the seda endpoint parameter on the first one endpoint,
because Camel will pickup the created queue in the next endpoint.
Please change the first route like this
…
.log('direct ${body}')
.to('seda:insert?blockWhenFull=true&size=5')
Re: SEDA and concurrentConsumers
Great. That works. Thanks.
BTW, the example in the Camel in Action book (p 322) seems wrong then as
it describes it the way I originally had it.
Tim
On 12/09/2012 03:34, Willem jiang wrote:
You need set the seda endpoint parameter on the first one endpoint, because
Camel will pickup the created queue in the next endpoint.
Please change the first route like this
…
.log('direct ${body}')
.to('seda:insert?blockWhenFull=true&size=5')
SEDA and concurrentConsumers
I'm trying to use the SEDA component with the concurrentConsumers options.
I'm setting a queue size and expecting the calling thread to block, but
its not working for me.
I have groovy code like this:
def vals = 1..10
println vals
CamelContext camelContext = new DefaultCamelContext()
camelContext.addRoutes(new RouteBuilder() {
def void configure() {
from('direct:start')
.split(body())
.log('direct ${body}')
.to('seda:insert')
from('seda:insert?concurrentConsumers=3&blockWhenFull=true&size=5')
.delay(100)
.log('seda ${body}')
}
})
camelContext.start()
def template = camelContext.createProducerTemplate()
println 'starting'
template.sendBody('direct:start', vals)
println 'done'
and its outputting this:
915 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel
2.10.1 (CamelContext: camel-1) started in 0.348 seconds
starting
941 [main] INFO route1 - direct 1
944 [main] INFO route1 - direct 2
944 [main] INFO route1 - direct 3
945 [main] INFO route1 - direct 4
945 [main] INFO route1 - direct 5
945 [main] INFO route1 - direct 6
946 [main] INFO route1 - direct 7
946 [main] INFO route1 - direct 8
946 [main] INFO route1 - direct 9
946 [main] INFO route1 - direct 10
done
2009 [Camel (camel-1) thread #2 - seda://insert] INFO route2 - seda 3
2009 [Camel (camel-1) thread #1 - seda://insert] INFO route2 - seda 2
2009 [Camel (camel-1) thread #0 - seda://insert] INFO route2 - seda 1
2110 [Camel (camel-1) thread #0 - seda://insert] INFO route2 - seda 6
2110 [Camel (camel-1) thread #1 - seda://insert] INFO route2 - seda 4
2110 [Camel (camel-1) thread #2 - seda://insert] INFO route2 - seda 5
2210 [Camel (camel-1) thread #0 - seda://insert] INFO route2 - seda 7
2210 [Camel (camel-1) thread #1 - seda://insert] INFO route2 - seda 8
2210 [Camel (camel-1) thread #2 - seda://insert] INFO route2 - seda 9
2310 [Camel (camel-1) thread #0 - seda://insert] INFO route2 - seda 10
Clearly the calling route is not blocking.
What am I doing wrong?
This is with 2.10.
Tim
Re: Surviving exceptions during splitting
On 10/09/2012 07:04, Claus Ibsen wrote: Where from is the IOException being thrown From within the next() method of the Iterator that the splitter generates. Tim
Re: Surviving exceptions during splitting
Hi,
I'm using 2.10 version.
Yes, its a custom splitter so I could handle the exception. But not sure
what I would do with it, or whethr that is really a good thing to do.
stopOnException seems to relate to exceptions that occur downstream of
the splitter, not from the splitting itself?
And I'm not needing to aggregate the results, so don't need an
AggregationStrategy.
Thanks
Tim
On 09/09/2012 16:58, Christian Müller wrote:
Which version of Camel do you use [1]?
Checkout [2] -> stopOnException how to continue if an exception occurs (use
a custom AggregationStrategy).
And because it's a custom splitter, you have the full control how to handle
the error...
[1] http://camel.apache.org/how-can-i-get-help.html
[2] http://camel.apache.org/splitter.html
Best,
Christian
On Sun, Sep 9, 2012 at 4:09 PM, Tim Dudgeon wrote:
I've got a route that splits a large file into records, using a splitter.
Its basically working but I can't get it to tolerate errors during the
splitting. The route terminates on the first error, but I want that error
to be caught so that the bad record can be logged and processing continue
with the next record.
I'm trying something like this in Groovy, but its not working.
camelContext.addRoutes(new RouteBuilder() {
def void configure() {
onException(IOException).**handled(true).log('ERROR')
from("file://C:/Users/**username/tmp/examples/in?**fileName=letters.txt")
.log('Processing file ${header.CamelFileName}')
.split(bean('splitter', 'split')).streaming()
.log('Found ${body}')
}
})
My simple example splits the text in the file into individual characters,
but throws IOException when a 'Z' is encountered.
The route halts at the first 'Z'.
What am I doing wrong?
Thanks
Tim
--
Surviving exceptions during splitting
I've got a route that splits a large file into records, using a splitter.
Its basically working but I can't get it to tolerate errors during the
splitting. The route terminates on the first error, but I want that
error to be caught so that the bad record can be logged and processing
continue with the next record.
I'm trying something like this in Groovy, but its not working.
camelContext.addRoutes(new RouteBuilder() {
def void configure() {
onException(IOException).handled(true).log('ERROR')
from("file://C:/Users/username/tmp/examples/in?fileName=letters.txt")
.log('Processing file ${header.CamelFileName}')
.split(bean('splitter', 'split')).streaming()
.log('Found ${body}')
}
})
My simple example splits the text in the file into individual
characters, but throws IOException when a 'Z' is encountered.
The route halts at the first 'Z'.
What am I doing wrong?
Thanks
Tim
Re: Advice on mirroring using FTP2 component
On 03/09/2012 19:54, Vincent Nonnenmacher wrote: one way could be to not use idempotent and implement a filter that return true or false on the modification date of the underlying file and handle a water mark so you know that any file with such date after the mark is selected. Yes, maybe, but that means you need to stat the remote file first, not download it. AFAIK its completely different process to what I'm currently trying to do. Maybe it can be made to. On other could be to implement your own org.apache.camel.spi.IdempotentRepository object that would delegate to one of the current Memory or File based one, but overrides the contains(key) to return false for any given entry that is already present (processed) but which 'modification date' is after your 'water mark'. Sounds better. The IdempotentRepository would need to handle modification time and file size as well to do this properly, and the interface doesn't support this: http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/spi/IdempotentRepository.html as it only takes a single param, and that's the file path. How does file size and modification time get passed? It seems I'm being really stupid here, but it feels like I'm trying to re-invent a wheel that's been invented before plenty of times. Tim On Mon, Sep 3, 2012 at 6:31 PM, Tim Dudgeon wrote: Hi, I'm new to Camel, so please excuse my ignorance. I'm wanting to set up mirroring of files on a FTP server, so that whenever new files appear, or old files get changed, they get copied to the local mirror, and maybe some processing performed on them. I've worked out the basics of how to use the FTP and File components, and am now wondering how to actually set this up to do what I want. I got the idempotentRepository working (using a FileIdempotentRepository) to make sure that files are only processed once, but then realised that this isn't going to allow updated files to be re-fetched. Is there an already baked approach to doing this, or do I need to cook my own? Tim -- View this message in context: http://camel.465427.n5.nabble.com/Advice-on-mirroring-using-FTP2-component-tp5718557.html Sent from the Camel - Users mailing list archive at Nabble.com.
Advice on mirroring using FTP2 component
Hi, I'm new to Camel, so please excuse my ignorance. I'm wanting to set up mirroring of files on a FTP server, so that whenever new files appear, or old files get changed, they get copied to the local mirror, and maybe some processing performed on them. I've worked out the basics of how to use the FTP and File components, and am now wondering how to actually set this up to do what I want. I got the idempotentRepository working (using a FileIdempotentRepository) to make sure that files are only processed once, but then realised that this isn't going to allow updated files to be re-fetched. Is there an already baked approach to doing this, or do I need to cook my own? Tim -- View this message in context: http://camel.465427.n5.nabble.com/Advice-on-mirroring-using-FTP2-component-tp5718557.html Sent from the Camel - Users mailing list archive at Nabble.com.
