Re: camel:netty udpConnectionlessSending option and "Too many open files"

[Willem Jiang]

How did you use the camel:netty component?
Camel always try to reuse the cached channel object if it is possible, can you 
share some detail of your camel route.

--  
Willem Jiang


Blog: http://willemjiang.blogspot.com (English)  
http://jnn.iteye.com (Chinese)
Twitter: willemjiang  
Weibo: 姜宁willem



On August 17, 2015 at 9:51:02 PM, SteveR (srichard...@vonage.com) wrote:
> I'm using Camel 2.15.2 and the *camel-netty* component and I have a Camel
> route that mirrors UDP datagrams to a remote server via *netty:udp*.
>  
> I saw that *camel-netty *supports the *udpConnectionlessSending *option, and
> I thought it would be a good idea as it will not result in failures when the
> remote server is not listening.
>  
> However, when I set *udpConnectionlessSendin=true* and then send lots of UDP
> packets into the route, my Linux box quickly exhausts file descriptors and I
> get the below java exception.
>  
> Any thoughts appreciated,
> Thanks, Steve
>  
>  
> *org.jboss.netty.channel.ChannelException: Failed to open a
> DatagramChannel.*
> at
> org.jboss.netty.channel.socket.nio.NioDatagramChannel.openNonBlockingChannel(NioDatagramChannel.java:94)
>   
> ~[netty-3.9.6.Final.jar:?]
> at
> org.jboss.netty.channel.socket.nio.NioDatagramChannel.(NioDatagramChannel.java:58)
>   
> ~[netty-3.9.6.Final.jar:?]
> at
> org.jboss.netty.channel.socket.nio.NioDatagramChannelFactory.newChannel(NioDatagramChannelFactory.java:207)
>   
> ~[netty-3.9.6.Final.jar:?]
> at
> org.jboss.netty.channel.socket.nio.NioDatagramChannelFactory.newChannel(NioDatagramChannelFactory.java:79)
>   
> ~[netty-3.9.6.Final.jar:?]
> at
> org.jboss.netty.bootstrap.ConnectionlessBootstrap.bind(ConnectionlessBootstrap.java:184)
>   
> ~[netty-3.9.6.Final.jar:?]
> at
> org.apache.camel.component.netty.NettyProducer.openConnection(NettyProducer.java:433)
>   
> ~[camel-netty-2.15.2.jar:2.15.2]
> at
> org.apache.camel.component.netty.NettyProducer$NettyProducerPoolableObjectFactory.makeObject(NettyProducer.java:543)
>   
> ~[camel-netty-2.15.2.jar:2.15.2]
> at
> org.apache.camel.component.netty.NettyProducer$NettyProducerPoolableObjectFactory.makeObject(NettyProducer.java:539)
>   
> ~[camel-netty-2.15.2.jar:2.15.2]
> at
> org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1188)
>   
> ~[commons-pool-1.6.jar:1.6]
> at
> org.apache.camel.component.netty.NettyProducer.process(NettyProducer.java:232)
>   
> ~[camel-netty-2.15.2.jar:2.15.2]
> at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)
>   
> [camel-core-2.15.2.jar:2.15.2]
> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
>   
> [camel-core-2.15.2.jar:2.15.2]
> at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109)
>   
> [camel-core-2.15.2.jar:2.15.2]
> at
> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:87)
>   
> [camel-core-2.15.2.jar:2.15.2]
> at
> org.apache.camel.processor.WireTapProcessor$1.call(WireTapProcessor.java:119) 
>  
> [camel-core-2.15.2.jar:2.15.2]
> at
> org.apache.camel.processor.WireTapProcessor$1.call(WireTapProcessor.java:113) 
>  
> [camel-core-2.15.2.jar:2.15.2]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_75]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   
> [?:1.7.0_75]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   
> [?:1.7.0_75]
> at java.lang.Thread.run(Thread.java:745) [?:1.7.0_75]
> *Caused by: java.net.SocketException: Too many open files*
> at sun.nio.ch.Net.socket0(Native Method) ~[?:1.7.0_75]
> at sun.nio.ch.Net.socket(Net.java:423) ~[?:1.7.0_75]
> at sun.nio.ch.DatagramChannelImpl.(DatagramChannelImpl.java:115)
> ~[?:1.7.0_75]
> at
> sun.nio.ch.SelectorProviderImpl.openDatagramChannel(SelectorProviderImpl.java:42)
>   
> ~[?:1.7.0_75]
> at java.nio.channels.DatagramChannel.open(DatagramChannel.java:146)
> ~[?:1.7.0_75]
> at
> org.jboss.netty.channel.socket.nio.NioDatagramChannel.openNonBlockingChannel(NioDatagramChannel.java:70)
>   
> ~[netty-3.9.6.Final.jar:?]
> ... 19 more
>  
>  
>  
> --
> View this message in context: 
> http://camel.465427.n5.nabble.com/camel-netty-udpConnectionlessSending-option-and-Too-many-open-files-tp5770768.html
>   
> Sent from the Camel - Users mailing list archive at Nabble.com.
>  

Re: Getting JMSConsumer to look for messages every N seconds

[Christian Müller]

Or use the timer component to send a message each 10 seconds to a
processor. This processor could use a consumer template to poll the queue
in a loop and send the message to another camel route with a producer
template (using the direct component ).
The consumer template also has a timeout parameter. You can use it to break
out from the loop, if there is no message in the queue.

I think this is an easier - less sophisticated  - solution.

Best,
Christian
Am 27.08.2015 00:59 schrieb "Raul Kripalani" :

> Currently I think that we only support fixed schedules. So with the cron
> scheduler you could achieve something like: start/resume the route on every
> minute at 00 and 30 seconds, stop/suspend every minute at 10 and 40 seconds
> time marks.
>
> This would effectively keep your route running for 10 seconds every 30
> seconds.
>
> If you want something more sophisticated, like stopping it once the queue
> is empty, you would need to extend what we offer OOTB with your own logic.
>
> Maybe you want to stop the route when t be consumer has been inactive for 1
> second or 500ms, or when queue size is 0 at the broker (checking via JMX).
>
> Raúl.
> On 26 Aug 2015 23:19, "gilboy"  wrote:
>
> > Thanks for the feedback. So just to confirm if I have the following route
> > and
> > I want to consume a message every 30 seconds would I inject in to the
> route
> > a RoutePolicy(e.g. SimpleScheduledRoutePolicy) where I specify the
> > routeStartRepeatInterval(30 seconds). I feel like I am missing something.
> > What shuts down the route etc
> >
> > from("jms:endpointDetails")
> >.routeId("test")
> >.routePolicy(policy)
> >.to("mock:success");
> >
> > Thanks
> > Joe
> >
> >
> >
> > --
> > View this message in context:
> >
> http://camel.465427.n5.nabble.com/Getting-JMSConsumer-to-look-for-messages-every-N-seconds-tp5770986p5770997.html
> > Sent from the Camel - Users mailing list archive at Nabble.com.
> >
>

Camel features using karaf feature xml

[Ravi Nallappan]

Hi,

How do I get camel features automatically installed by a feature xml.

In my scenario, I am creating my own Karaf feature xml to install all dependent 
bundles before my bundles can be started. From what I observe, few camel 
features such as camel-core, camel-http4, camel-jackson are installed while 
camel-blueprint does not get installed, this eventually cause an issue starting 
my bundle.

Here is a simplified example.

1. Generate maven project
mvn archetype:generate \
-DarchetypeGroupId=org.apache.karaf.archetypes \
-DarchetypeArtifactId=karaf-feature-archetype \
-DarchetypeVersion=4.0.1 \
-DgroupId=com.acme.mediation \
-DartifactId=acme-mediation \
-Dversion=1.3.0-SNAPSHOT \
-Dpackage=com.acme.mediation \
-DinteractiveMode=false

2. Create feature xml as follow
[build@localhost EIP]$ cat acme-mediation/src/main/feature/feature.xml 

http://karaf.apache.org/xmlns/features/v1.0.0;>

mvn:org.apache.camel.karaf/apache-camel/2.15.2/xml/features

${project.description}
camel
camel-blueprint
camel-jdbc
camel-http4
camel-jackson



3. Add following to pom.xml (I am using Nexus as MRM)


snapshots

http://localhost:8081/nexus/content/repositories/snapshots



4. Deploy the feature
mvn deploy

5. Install the feature in karaf (4.0.1)
dev@ubuntu:~/apache-karaf-4.0.1$ bin/karaf
karaf@root()> feature:repo-add 
mvn:com.acme.mediation/acme-mediation/1.3.0-SNAPSHOT/xml/features
karaf@root()> feature:install acme-mediation-common
karaf@root()> feature:list -i
Name| Version   | 
Required | State   | Repository| Description
-
acme-mediation-common   | 1.3.0.SNAPSHOT| x| Started | 
acme-mediation-1.3.0-SNAPSHOT | acme-mediation-feature
aries-proxy | 4.0.1 |  | 
Started | standard-4.0.1| Aries Proxy
aries-blueprint | 4.0.1 | x| 
Started | standard-4.0.1| Aries Blueprint
feature | 4.0.1 | x 
   | Started | standard-4.0.1| Features Support
shell   | 4.0.1 | x 
   | Started | standard-4.0.1| Karaf Shell
shell-compat| 4.0.1 | x| 
Started | standard-4.0.1| Karaf Shell Compatibility
deployer| 4.0.1 | x| 
Started | standard-4.0.1| Karaf Deployer
bundle  | 4.0.1 | x 
   | Started | standard-4.0.1| Provide Bundle support
config  | 4.0.1 | x 
   | Started | standard-4.0.1| Provide OSGi ConfigAdmin support
diagnostic  | 4.0.1 | x| 
Started | standard-4.0.1| Provide Diagnostic support
instance| 4.0.1 | x 
   | Started | standard-4.0.1| Provide Instance support
jaas| 4.0.1 | x 
   | Started | standard-4.0.1| Provide JAAS support
log | 4.0.1 | x 
   | Started | standard-4.0.1| Provide Log support
package | 4.0.1 | x 
   | Started | standard-4.0.1| Package commands and mbeans
service | 4.0.1 | x 
   | Started | standard-4.0.1| Provide Service support
system  | 4.0.1 | x 
   | Started | standard-4.0.1| Provide System support
kar | 4.0.1 | x 
   | Started | standard-4.0.1| Provide KAR (KARaf archive) 
support
ssh | 4.0.1 | x 
   | Started | standard-4.0.1| Provide a SSHd server on Karaf
management  | 4.0.1 | x| 
Started | standard-4.0.1| Provide a JMX MBeanServer and a set 
of MBeans in
wrap| 0.0.0 | x 
   | Started | standard-4.0.1| Wrap URL handler
spring-dm   | 1.2.1 |  | 
Started | spring-4.0.1  | Spring DM support
spring

Polling messages from topics ActiveMQ

[S-camel]

Dear fellow camelriders,

At the moment I want to poll messages from a specific topic in ActiveMQ
using camel in eclipse.
 from("activemq:topic:test").log("transfered to file")
.to("file:src/data");  

Getting the following error:
Exception in thread "main" java.lang.NoClassDefFoundError:
org/springframework/beans/factory/Aware

Polling messages from a queue works fine but topic doesn't work.





--
View this message in context: 
http://camel.465427.n5.nabble.com/Polling-messages-from-topics-ActiveMQ-tp5771510.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: How can I manually redelivery the message to source endpoint while an exception throws?

[zhichao....@hpe.com]

Hi Greg,

I appreciate your quickly response. 
I know Camel has OnException to catch the exception, as well as the auto
re-delivery mechanism. But in some case, user hope they can manually to
resubmit the message. 
For example, if the exception throws, we need serialize Message/Exchange
object as String and save it to database. Then user can correct the
message/exchange then deserialize the String as Message/Exchange object and
resubmit it.

Do you have any good solution for this scenario? If you can provide some
code example, I really appreciate.

Zhi-Chao Fei



--
View this message in context: 
http://camel.465427.n5.nabble.com/How-can-I-manually-redelivery-the-message-to-source-endpoint-while-an-exception-throws-tp5771486p5771509.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Processing N files at a time - Threads

[pkx]

I have seen the various posts for processing files concurrently, and I have
seen the various types of errors that result from it. As others have done I
managed to get around the issue by somehow limiting the poll from the file
consumer, or using an idempotentConsumer before the threads, in my case a
idempotentConsumer seems like a better option.

In my opinion this is a workaround, and I think there is an issue in the
file component's creation of exchanges and tracking them with .threads(N)
added.
It is certainly creating duplicate exchanges for the same file, regardless
of the various locking methods I tried, or the post processing (move |
delete)

The problem starts the with file component when you add .threads(N) in the
route, even if it is a .threads(1)

from("file://spool?delete=true" +
otherOptions).routeId("ConsumeByThreads").threads(N).process(someAsyncProcessor);

The logged error is seen when you attempt do getBody, which triggers reading
the file...

If I track the exchange details in my "someAsyncProcessor" using the
"CamelFileName" header, I can confirm that the exchange is for a file which
was already processed, hence the FileNotFoundException by the time the file
content needs to be read.  There is NO data loss, as the files are processed
already, but there is a lot of noise in error messages produced.

I have written a test case that confirms the issue, I will see how to get it
submitted. 

This is on Camel 2.15.3, and I have seen it on 2.15.2 as well, on Windows
and OSX

I would love to hear if anyone has a solution I have not considered, or if a
bug needs to be tracked for the file component.

Edited error log:
-
Exchange[
Id  ID-
ExchangePattern InOnly
Headers {breadcrumbId=ID-, CamelFileAbsolute=false,
CamelFileAbsolutePath=C:\poc\spool\ID-, CamelFileContentType=null,
CamelFileLength=0, CamelFileName=ID-, CamelFileNameConsumed=ID-,
CamelFileNameOnly=ID-, CamelFileParent=spool,
CamelFilePath=spool\ID-, CamelFileRelativePath=ID-,
CamelRedelivered=false, CamelRedeliveryCounter=0}
BodyTypeorg.apache.camel.component.file.GenericFile
Body[Body is file based: GenericFile[ID-]]
]

Stacktrace
---

org.apache.camel.TypeConversionException: Error during type conversion from
type: java.lang.String to the required type: byte[] with value [Body is file
based: GenericFile[ID-]] due java.io.FileNotFoundException:
C:\...\poc\spool\ID- (The system cannot find the file specified)
...
Caused by: java.io.FileNotFoundException: ...\spool\ID-X (The system
cannot find the file specified)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(FileInputStream.java:131)
at
org.apache.camel.converter.IOConverter.toInputStream(IOConverter.java:78)
at 
org.apache.camel.converter.IOConverter.toByteArray(IOConverter.java:266)
at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at 
org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:1220)
... 25 common frames omitted






--
View this message in context: 
http://camel.465427.n5.nabble.com/Processing-N-files-at-a-time-Threads-tp5771511.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Workaround with REST DSL to avoid HTTP method not allowed - 405

[Henryk Konsek]

Hi Charles,

You have to enable the CORS on the route DSL configuration, not only on the
routes level. For example here is the REST+Netty combination for Camel
2.15.1 that works properly with the AngularJS REST client:

RestPropertyDefinition corsAllowedHeaders = new RestPropertyDefinition();
corsAllowedHeaders.setKey("Access-Control-Allow-Headers");
corsAllowedHeaders.setValue("Origin, Accept, X-Requested-With,
Content-Type, Access-Control-Request-Method,
Access-Control-Request-Headers, Authorization");

RestConfigurationDefinition restConfiguration =
restConfiguration().component("netty4-http").
  host("0.0.0.0").port(restPort).bindingMode(json).
  enableCORS(enableCors).endpointProperty("matchOnUriPrefix", "true");

restConfiguration.setCorsHeaders(singletonList(corsAllowedHeaders));

rest("*/api/*")
  .verb("OPTIONS", "/").route()
  .setHeader("Access-Control-Allow-Origin", constant("*"))
  .setHeader("Access-Control-Allow-Methods", constant("GET, HEAD, POST,
PUT, DELETE, TRACE, OPTIONS, CONNECT, PATCH"))
.setHeader("Access-Control-Allow-Headers",
constant("Origin, Accept, X-Requested-With, Content-Type,
Access-Control-Request-Method, Access-Control-Request-Headers,
Authorization"))
.setHeader("Allow", constant("GET, OPTIONS, POST, PATCH"));

In general, Camel REST DSL tries to provide the OPTIONS routes for you, so
you don't have to define those, but sometimes it fails to do it properly. I
fixed some of such corner cases for REST+Netty combination in Camel 2.15.1,
but I think that there are some corner cases still to cover.

The configuration above is the mix of my fixes included in Camel 2.15.1 and
workarounds to make Netty REST API properly consume the requests from
Angular clients. Probably the latest Camel works better in this regards.

Cheers!

pon., 14.09.2015 o 14:42 użytkownik Charles Moulliard 
napisał:

> Hi,
>
> I try to see if there is a workaround to avoid this issue (HTTP method not
> allowed - 405) or If I have to dig into the code to provide a fix
>
> 1) HTTP Method 405 - returned
>
> This REST DSL definition generates a HTTP 405 Method not allowed when this
> request is issued (http -v OPTIONS http://localhost:9191/blog/article/)
>
>
>
> rest("/blog/").id("rest-blog-service").produces("application/json").consumes("application/json")
>
>
>  
> .get("/article/search/user/{user}").id("rest-searchbyuser").outTypeList(Blog.class)
>.to("direct:searchByUser")
>
>.put("/article/").id("rest-put-article").type(Blog.class)
>.to("direct:add")
>
>rest("/blog/article/").id("rest-options")
>  .verb("options")
>  .route()
>  .setHeader("Access-Control-Allow-Origin", constant("*"))
>  .setHeader("Access-Control-Allow-Methods", constant("GET,
> HEAD, POST, PUT, DELETE, OPTIONS"))
>  .setHeader("Access-Control-Allow-Headers", constant("Origin,
> Accept, X-Requested-With, Content-Type, Access-Control-Request-Method,
> Access-Control-Request-Headers"))
>  .setHeader("Allow", constant("GET, HEAD, POST, PUT, DELETE,
> OPTIONS"));
>
>
> http -v options http://localhost:9191/blog/article/
> OPTIONS /blog/article/ HTTP/1.1
> Accept: */*
> Accept-Encoding: gzip, deflate
> Connection: keep-alive
> Content-Length: 0
> Host: localhost:9191
> User-Agent: HTTPie/0.9.2
>
> -->
>
> HTTP/1.1 405 Method Not Allowed
>
> endpoints registered
>
>  INFO  Route: rest-searchbyid started and consuming from: Endpoint[
> http://0.0.0.0:9191/blog/article/search/id/%7Bid%7D?httpMethodRestrict=GET
> ]
> INFO  Route: rest-searchbyuser started and consuming from: Endpoint[
>
> http://0.0.0.0:9191/blog/article/search/user/%7Buser%7D?httpMethodRestrict=GET
> ]
> INFO  Route: rest-put-article started and consuming from: Endpoint[
> http://0.0.0.0:9191/blog/article?httpMethodRestrict=PUT]
> INFO  Route: rest-deletearticle started and consuming from: Endpoint[
> http://0.0.0.0:9191/blog/article/%7Bid%7D?httpMethodRestrict=DELETE]
> INFO  Route: route1 started and consuming from: Endpoint[
> http://0.0.0.0:9191/blog/article?httpMethodRestrict=OPTIONS]
>
>
> while this is not the case If I comment the line with the put as you can
> see hereafter
>
>
> 2) HTTP Method - 200 OK
>
>
>
> rest("/blog/").id("rest-blog-service").produces("application/json").consumes("application/json")
>
>
>
>  
> .get("/article/search/user/{user}").id("rest-searchbyuser").outTypeList(Blog.class)
>.to("direct:searchByUser")
>
>/* .put("/article/").id("rest-put-article").type(Blog.class)
>.to("direct:add") */
>
>rest("/blog/article/").id("rest-options")
>  .verb("options")
>  .route()
>  .setHeader("Access-Control-Allow-Origin", constant("*"))
>  .setHeader("Access-Control-Allow-Methods", constant("GET,
> HEAD, POST, PUT, DELETE, OPTIONS"))
>  

Executing PERL script in Camel route

[anish.somadevan]

Am trying to execute a PERL script from my route XML. I use the following
syntax,



But i don't see the script being executed. This is the log i see,

23:28:44,243 INFO  [org.apache.camel.component.exec.ExecProducer] (Camel
(camelAgent) thread #5 - JmsConsumer[ISEEOutboundQueue]) Executing
ExecCommand [args=[/home/scripts/test.pl], executable=perl,
timeout=9223372036854775807, outFile=null, workingDir=null,
useStderrOnEmptyStdout=false]
23:28:44,327 INFO  [org.apache.camel.component.exec.ExecProducer] (Camel
(camelAgent) thread #5 - JmsConsumer[ISEEOutboundQueue]) The command
ExecCommand [args=[/home/scripts/test.pl], executable=perl,
timeout=9223372036854775807, outFile=null, workingDir=null,
useStderrOnEmptyStdout=false] had exit value 0

This is the content of my PERL script,


#!/usr/bin/perl
print "Hello World\n";


Can anyone please let me know what am missing here?



--
View this message in context: 
http://camel.465427.n5.nabble.com/Executing-PERL-script-in-Camel-route-tp5771521.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Configure a camel route to start and stop at certain time of day

[briane80]

Hi,

I have a requirement where my camel route should only pick up new messages
between 8am and 10pm due to database backups and other nightly processes.

I have looked at the SimpleScheduledRoutePolicy class but i'm not sure how
to use it to start and stop routes at predefined times every day. Cann
someone provide an example of how to configure this (i'm using camel 2.12
and mainly xml based config).

Thanks,

Brian



--
View this message in context: 
http://camel.465427.n5.nabble.com/Configure-a-camel-route-to-start-and-stop-at-certain-time-of-day-tp5771520.html
Sent from the Camel - Users mailing list archive at Nabble.com.

High Volume ETL using camel

[contactreji]

Hello Geeks

We are a huge retail organization with over 1500 stores as well as online
shopping websites. Volumes of messages at the middleware run probably over a
billion daily. Currently we have a commercial product by another vendor
taking up the responsibility.

We plan to move out systems soon into complete open source stack. That will
include products like Camel, Fuse , Jenkins, Gerrit etc. 

We have 2 scenarios where we are trying to fit in Camel

1) For real time traffic. That will be the typical busy Fuse ESB layer which
takes care of messaging / transformation / reliable delivery etc between
enterprise applications. We are clear on that and going ahead with Camel and
OSGi containers.

2) Scheduled Batch/ETL jobs. Now these will be really heavy jobs and may
contain payloads running to few GB's in size. We want to fit in Camel for
these and alternative in case we hit a road block will be Talend ETL.

My queries are below

1) Is camel capable of  doing heavy batch ETL's reliably? If so what would
be the programmer's best practices & strategy for dealing heavy csv files. I
wish not to see OOM issues.

2) What would be the best suitable container for running batch ETL jobs?
Should I use Servicemix / Jboss Fuse or consider other Java based
containers?

Any insights from you all would help evaluate on options.

Cheers
Reji



-
Reji Mathews
Sr. Developer - Middleware Integration / SOA ( Open Source - Apache Camel & 
Jboss Fuse ESB | Mule ESB )
LinkedIn - http://in.linkedin.com/pub/reji-mathews/31/9a2/40a
Twitter - reji_mathews
--
View this message in context: 
http://camel.465427.n5.nabble.com/High-Volume-ETL-using-camel-tp5771522.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Dynamic DataSource

[arnaud]

Chistian, yes the datasource is configured in the camel context, when the
dataSource is hard coded it works :

from("direct:create")
.to("sql:insert into customer(firstname, lastname) values(:#firstName,   
#lastName)?dataSource=#myDynamicSource"));

Arnaud



--
View this message in context: 
http://camel.465427.n5.nabble.com/Dynamic-DataSource-tp5771402p5771487.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: kyro data format in camel

[fxthomas]

here is the class implementataion also.

public final class KyroDataFormat implements DataFormat {

@Override
public void marshal(Exchange exchange, Object graph, OutputStream 
stream)
throws Exception {
stream.write(new KryoSerializationUtil().serialize(graph));

}

@Override
public Object unmarshal(Exchange exchange, InputStream stream)
throws Exception {
final byte[] bytes = exchange.getContext().getTypeConverter()
.mandatoryConvertTo(byte[].class, stream);
return new KryoSerializationUtil().deserialize(bytes, 
.class);
}

}



--
View this message in context: 
http://camel.465427.n5.nabble.com/kyro-data-format-in-camel-tp5771093p5771489.html
Sent from the Camel - Users mailing list archive at Nabble.com.

BeanIO - No Stream mapping configured

[Sashika]

Hi,
I’m trying to configure beanIO for one of my routes as follows Route 
builderDataFormatformat=newBeanIODataFormat("mapping.xml","csvFile");from("direct:welcome").unmarshal(format).to("akka://mmb/user/sample").wireTap("file://log");

Not using Mixmax yet?
But I get the following 
error.atakka.camel.ProducerSupport$$anonfun$produce$1.applyOrElse(Producer.scala:73)atscala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)atscala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)atscala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)atakka.camel.javaapi.UntypedProducerActor.onReceive(UntypedProducerActor.scala:51)atakka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)atakka.actor.Actor$class.aroundReceive(Actor.scala:467)atakka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97)atakka.actor.ActorCell.receiveMessage(ActorCell.scala:516)atakka.actor.ActorCell.invoke(ActorCell.scala:487)atakka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)atakka.dispatch.Mailbox.run(Mailbox.scala:220)atakka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)atscala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)atscala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)atscala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)atscala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)Causedby:
 java.lang.IllegalArgumentException: 
Nostreammappingconfiguredforname'csvFile'atorg.beanio.internal.DefaultStreamFactory.getStream(DefaultStreamFactory.java:129)atorg.beanio.internal.DefaultStreamFactory.createReader(DefaultStreamFactory.java:70)atorg.beanio.StreamFactory.createReader(StreamFactory.java:98)atorg.apache.camel.dataformat.beanio.BeanIODataFormat.readModels(BeanIODataFormat.java:148)atorg.apache.camel.dataformat.beanio.BeanIODataFormat.unmarshal(BeanIODataFormat.java:116)atorg.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:67)atorg.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)atorg.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)atorg.apache.camel.processor.Pipeline.process(Pipeline.java:118)atorg.apache.camel.processor.Pipeline.process(Pipeline.java:80)atorg.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)atorg.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:51)atorg.apache.camel.processor.SendProcessor.process(SendProcessor.java:129)atakka.camel.ProducerSupport$ProducerChild.produce(Producer.scala:137)atakka.camel.ProducerSupport$ProducerChild$$anonfun$receive$1.applyOrElse(Producer.scala:111)atakka.actor.Actor$class.aroundReceive(Actor.scala:467)atakka.camel.ProducerSupport$ProducerChild.aroundReceive(Producer.scala:108)
... 9more

Not using Mixmax yet?
Do I need to configure an explicit stream. I use akka-camel

How can I manually redelivery the message to source endpoint while an exception throws?

[zhichao....@hpe.com]

Hi All,

As we known, Camel has auto redelivery machanism, but sometimes, we need
re-delivery the message to source endpoint while delivery faild. Nomally, we
need to process below steps.

1. While an exception throws, we will save the Exchange/Message to the
database.
2. Provide a page to modify the Exchange/Message then click a button to
re-delivery the message.

How can I implement this scenario.

I appreciate if someone can provide good idea.


Zhi-Chao Fei





--
View this message in context: 
http://camel.465427.n5.nabble.com/How-can-I-manually-redelivery-the-message-to-source-endpoint-while-an-exception-throws-tp5771486.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: kyro data format in camel

[fxthomas]

ohh Sorry for that, below is the part where it is used.




 
  
 





  
 








--
View this message in context: 
http://camel.465427.n5.nabble.com/kyro-data-format-in-camel-tp5771093p5771488.html
Sent from the Camel - Users mailing list archive at Nabble.com.

salesforce dto generator: Unrecognized field "is_readonly" in LoginToken

[rpcat]

perhaps a new salesforce field "is_readonly" is needed in LoginToken to fix
the DTO generator?


[INFO] --- camel-salesforce-maven-plugin:2.15.3:generate (default-cli) @
salesforceIntegration ---
[DEBUG] Dependency collection stats: {ConflictMarker.analyzeTime=0,
ConflictMarker.markTime=0, ConflictMarker.nodeCount=39,
ConflictIdSorter.graphTim   
e=0, ConflictIdSorter.topsortTime=0, ConflictIdSorter.conflictIdCount=25,
ConflictIdSorter.conflictIdCycleCount=0, ConflictResolver.totalTime=10, Con 
  
flictResolver.conflictItemCount=38,
DefaultDependencyCollector.collectTime=250,
DefaultDependencyCollector.transformTime=10}
[DEBUG] org.apache.camel.maven:camel-salesforce-maven-plugin:jar:2.15.3:
[DEBUG]org.apache.maven:maven-plugin-api:jar:2.0:compile
[DEBUG]org.codehaus.plexus:plexus-utils:jar:3.0.8:compile
[DEBUG]org.apache.camel:camel-salesforce:jar:2.15.3:compile
[DEBUG]   org.apache.camel:camel-core:jar:2.15.3:compile
[DEBUG]   org.eclipse.jetty:jetty-client:jar:8.1.17.v20150415:compile
[DEBUG]  org.eclipse.jetty:jetty-http:jar:8.1.17.v20150415:compile
[DEBUG]   org.eclipse.jetty:jetty-util:jar:8.1.17.v20150415:compile
[DEBUG]   org.eclipse.jetty:jetty-io:jar:8.1.17.v20150415:compile
[DEBUG]   org.codehaus.jackson:jackson-mapper-asl:jar:1.9.12:compile
[DEBUG]  org.codehaus.jackson:jackson-core-asl:jar:1.9.12:compile
[DEBUG]   com.thoughtworks.xstream:xstream:jar:1.4.8:compile
[DEBUG]  xmlpull:xmlpull:jar:1.1.3.1:compile
[DEBUG]  xpp3:xpp3_min:jar:1.1.4c:compile
[DEBUG]   org.cometd.java:cometd-java-client:jar:2.4.3:compile
[DEBUG]  org.cometd.java:bayeux-api:jar:2.4.3:compile
[DEBUG]  org.cometd.java:cometd-java-common:jar:2.4.3:compile
[DEBUG]   joda-time:joda-time:jar:2.5:compile
[DEBUG]org.apache.velocity:velocity:jar:1.7:compile
[DEBUG]   commons-collections:commons-collections:jar:3.2.1:compile
[DEBUG]   commons-lang:commons-lang:jar:2.4:compile
[DEBUG]org.slf4j:slf4j-api:jar:1.6.6:compile
[DEBUG]log4j:log4j:jar:1.2.17:compile
[DEBUG]com.sun.xml.bind:jaxb-core:jar:2.2.11:compile
[DEBUG]com.sun.xml.bind:jaxb-impl:jar:2.2.11:compile
[DEBUG] Created new class realm
plugin>org.apache.camel.maven:camel-salesforce-maven-plugin:2.15.3
[DEBUG] Importing foreign packages into class realm
plugin>org.apache.camel.maven:camel-salesforce-maven-plugin:2.15.3
[DEBUG]   Imported:  < maven.api
[DEBUG] Populating class realm
plugin>org.apache.camel.maven:camel-salesforce-maven-plugin:2.15.3
[DEBUG]   Included:
org.apache.camel.maven:camel-salesforce-maven-plugin:jar:2.15.3
[DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:3.0.8
[DEBUG]   Included: org.apache.camel:camel-salesforce:jar:2.15.3
[DEBUG]   Included: org.apache.camel:camel-core:jar:2.15.3
[DEBUG]   Included: org.eclipse.jetty:jetty-client:jar:8.1.17.v20150415
[DEBUG]   Included: org.eclipse.jetty:jetty-http:jar:8.1.17.v20150415
[DEBUG]   Included: org.eclipse.jetty:jetty-util:jar:8.1.17.v20150415
[DEBUG]   Included: org.eclipse.jetty:jetty-io:jar:8.1.17.v20150415
[DEBUG]   Included: org.codehaus.jackson:jackson-mapper-asl:jar:1.9.12
[DEBUG]   Included: org.codehaus.jackson:jackson-core-asl:jar:1.9.12
[DEBUG]   Included: com.thoughtworks.xstream:xstream:jar:1.4.8
[DEBUG]   Included: xmlpull:xmlpull:jar:1.1.3.1
[DEBUG]   Included: xpp3:xpp3_min:jar:1.1.4c
[DEBUG]   Included: org.cometd.java:cometd-java-client:jar:2.4.3
[DEBUG]   Included: org.cometd.java:bayeux-api:jar:2.4.3
[DEBUG]   Included: org.cometd.java:cometd-java-common:jar:2.4.3
[DEBUG]   Included: joda-time:joda-time:jar:2.5
[DEBUG]   Included: org.apache.velocity:velocity:jar:1.7
[DEBUG]   Included: commons-collections:commons-collections:jar:3.2.1
[DEBUG]   Included: commons-lang:commons-lang:jar:2.4
[DEBUG]   Included: log4j:log4j:jar:1.2.17
[DEBUG]   Included: com.sun.xml.bind:jaxb-core:jar:2.2.11
[DEBUG]   Included: com.sun.xml.bind:jaxb-impl:jar:2.2.11
[DEBUG] Configuring mojo
org.apache.camel.maven:camel-salesforce-maven-plugin:2.15.3:generate from
plugin realm ClassRealm[plugin>org.apache.camel.ma  
 
ven:camel-salesforce-maven-plugin:2.15.3, parent:
sun.misc.Launcher$AppClassLoader@4e25154f]
[DEBUG] Configuring mojo
'org.apache.camel.maven:camel-salesforce-maven-plugin:2.15.3:generate' with
basic configurator -->

...


[ERROR] Failed to execute goal
org.apache.camel.maven:camel-salesforce-maven-plugin:2.15.3:generate
(default-cli) on project salesforceIntegration: S   

alesforce login error Login error: unexpected exception Unrecognized field
"is_readonly" (Class org.apache.camel.component.salesforce.internal.dto.Lo  
 
ginToken), not marked as ignorable
[ERROR] at [Source: [B@3703bf3c; line: 1, column: 

Re: Workaround with REST DSL to avoid HTTP method not allowed - 405

[Charles Moulliard]

Hi Henryk,

Some additional infos : I'm using jetty component & Camel
2.15.1.redhat-620133.

CORS has been enabled at the level of the REST Configuration (
https://github.com/FuseByExample/rest-dsl-in-action/blob/master/routing/src/main/java/org/jboss/fuse/route/RestToServicesRoute.java)
as you can see here.

I see from the netty4-http component code that Allow header is returned for
OPTIONS request -
https://github.com/apache/camel/blob/master/components/camel-netty4-http/src/main/java/org/apache/camel/component/netty4/http/handlers/HttpServerChannelHandler.java#L103-L107


Nevertheless, there is a problem with the logic implemented as we only get
a response (= Allow Header) when the path of the URL is created without
httpMethodRestrict :
https://github.com/apache/camel/blob/master/components/camel-netty4-http/src/test/java/org/apache/camel/component/netty4/http/NettyHttpRestOptionsAllowTest.java#L54


If you use the url path like this -->
from("netty4-http:http://0.0.0.0:{{port}}/myapp?httpMethodRestrict=OPTIONS;),
we don't receive the Allow Header

FYI : I have created the verb options to avoid to use verb("") syntax -
camel-9129

On Mon, Sep 14, 2015 at 5:46 PM, Henryk Konsek  wrote:

> Hi Charles,
>
> You have to enable the CORS on the route DSL configuration, not only on the
> routes level. For example here is the REST+Netty combination for Camel
> 2.15.1 that works properly with the AngularJS REST client:
>
> RestPropertyDefinition corsAllowedHeaders = new RestPropertyDefinition();
> corsAllowedHeaders.setKey("Access-Control-Allow-Headers");
> corsAllowedHeaders.setValue("Origin, Accept, X-Requested-With,
> Content-Type, Access-Control-Request-Method,
> Access-Control-Request-Headers, Authorization");
>
> RestConfigurationDefinition restConfiguration =
> restConfiguration().component("netty4-http").
>   host("0.0.0.0").port(restPort).bindingMode(json).
>   enableCORS(enableCors).endpointProperty("matchOnUriPrefix", "true");
>
> restConfiguration.setCorsHeaders(singletonList(corsAllowedHeaders));
>
> rest("*/api/*")
>   .verb("OPTIONS", "/").route()
>   .setHeader("Access-Control-Allow-Origin", constant("*"))
>   .setHeader("Access-Control-Allow-Methods", constant("GET, HEAD, POST,
> PUT, DELETE, TRACE, OPTIONS, CONNECT, PATCH"))
> .setHeader("Access-Control-Allow-Headers",
> constant("Origin, Accept, X-Requested-With, Content-Type,
> Access-Control-Request-Method, Access-Control-Request-Headers,
> Authorization"))
> .setHeader("Allow", constant("GET, OPTIONS, POST, PATCH"));
>
> In general, Camel REST DSL tries to provide the OPTIONS routes for you, so
> you don't have to define those, but sometimes it fails to do it properly. I
> fixed some of such corner cases for REST+Netty combination in Camel 2.15.1,
> but I think that there are some corner cases still to cover.
>
> The configuration above is the mix of my fixes included in Camel 2.15.1 and
> workarounds to make Netty REST API properly consume the requests from
> Angular clients. Probably the latest Camel works better in this regards.
>
> Cheers!
>
> pon., 14.09.2015 o 14:42 użytkownik Charles Moulliard 
> napisał:
>
> > Hi,
> >
> > I try to see if there is a workaround to avoid this issue (HTTP method
> not
> > allowed - 405) or If I have to dig into the code to provide a fix
> >
> > 1) HTTP Method 405 - returned
> >
> > This REST DSL definition generates a HTTP 405 Method not allowed when
> this
> > request is issued (http -v OPTIONS http://localhost:9191/blog/article/)
> >
> >
> >
> >
> rest("/blog/").id("rest-blog-service").produces("application/json").consumes("application/json")
> >
> >
> >
> .get("/article/search/user/{user}").id("rest-searchbyuser").outTypeList(Blog.class)
> >.to("direct:searchByUser")
> >
> >.put("/article/").id("rest-put-article").type(Blog.class)
> >.to("direct:add")
> >
> >rest("/blog/article/").id("rest-options")
> >  .verb("options")
> >  .route()
> >  .setHeader("Access-Control-Allow-Origin", constant("*"))
> >  .setHeader("Access-Control-Allow-Methods", constant("GET,
> > HEAD, POST, PUT, DELETE, OPTIONS"))
> >  .setHeader("Access-Control-Allow-Headers", constant("Origin,
> > Accept, X-Requested-With, Content-Type, Access-Control-Request-Method,
> > Access-Control-Request-Headers"))
> >  .setHeader("Allow", constant("GET, HEAD, POST, PUT, DELETE,
> > OPTIONS"));
> >
> >
> > http -v options http://localhost:9191/blog/article/
> > OPTIONS /blog/article/ HTTP/1.1
> > Accept: */*
> > Accept-Encoding: gzip, deflate
> > Connection: keep-alive
> > Content-Length: 0
> > Host: localhost:9191
> > User-Agent: HTTPie/0.9.2
> >
> > -->
> >
> > HTTP/1.1 405 Method Not Allowed
> >
> > endpoints registered
> >
> >  INFO  Route: rest-searchbyid started and consuming from: Endpoint[
> >
> 

RE: redeliveryPolicy onException based on HTTP statusCode

[pholder]

Thank you for this anwser Vanshul.Chawla,

that was exactly what I was looking for.
Amazing how simple it can be.



--
View this message in context: 
http://camel.465427.n5.nabble.com/redeliveryPolicy-onException-based-on-HTTP-statusCode-tp5771448p5771485.html
Sent from the Camel - Users mailing list archive at Nabble.com.