Adding type awareness in Camel route
Hi Camel developers, I'd like to propose an enhancement on handling data types of Camel message contents. To start a smooth discussion I implemented the idea first: https://github.com/igarashitm/camel/tree/contract-based-type-awareness And these testcases demonstrates the declarative transformer usage according to the declared data types: [Java DSL] https://github.com/igarashitm/camel/commit/498c27d2ba99b04bbe7b90a93329d42b7c718a29#diff-c14a7e8e88a6e41492946e1537bfb1cf [Spring DSL] https://github.com/igarashitm/camel/commit/498c27d2ba99b04bbe7b90a93329d42b7c718a29#diff-b2506c84ddde91438fc6374039e21534 This adds input/output content type declaration on from and all other processors. It also introduces well-known Exchange properties, INPUT_TYPE and OUTPUT_TYPE which are used to specify the current message content type. The data type is URN like string starts with scheme, like java:org.example.ItemA or xml:{org.example.xml}ItemA. If the content type is declared via inputType/outputType/contract, the ContractProcessor wraps the actual processor and process transformation/validation according to the type, say if INPUT_TYPE exchange property has xml:{org.example.xml}ItemA and the declared inputType is xml:{org.example.xml}ItemB, then it transforms xml:{org.example.xml}ItemA content into xml:{org.example.xml}ItemB. The element which is introduced right under the is the one to declare the mappings between transformer implementation and those from/to data type. I implemented only transformer first, but validator would be brought in in a same way. This way allows users to make data types visible in the route definition and keep the transformation/validation apart from route definition itself. The most important thing is that the ContractProcessor is involved only when content type is explicitly declared in a route definition, so that it never breaks existing camel routes. Ofcourse programatic transformations/validations we're doing today are still fully available. It's purely an addition to the existing camel features. Any thoughts? Does it sound acceptable to get merged into camel? ANY feedback would be really appreciated! Thanks, Tomo
Camel.trunk.notest - Build # 2875 - Still Failing
The Apache Jenkins build system has built Camel.trunk.notest (build #2875) Status: Still Failing Check console output at https://builds.apache.org/job/Camel.trunk.notest/2875/ to view the results.
Re: [GitHub] camel pull request #1167: CAMEL-10308
Hi Yeah that is a better name On Thu, Sep 15, 2016 at 6:08 PM, Vitalii Tymchyshynwrote: > How about seThreadedAsyncMode(boolean), defailt true? > > I will also do checkstyle till EOW. > > Best regards, Vitalii Tymchyshyn > > Чт, 15 вер. 2016 05:08 користувач Claus Ibsen пише: > >> Hi >> >> I do wonder if we can come up with a better name for the setter/getter >> of synchronous on ProducerTemplate. >> >> Otherwise great work. The Java 8 API makes the code easier when you >> can compose the futures with the thenApply. >> >> Just a note that we must/should ensure that existing Camel users can >> take their code as-is and compile with Java 8 and Camel 2.18 >> dependencies without having to change their source code. >> >> Down the road we should also look at if some of this makes sense to >> add to the newer and simpler FluentProducerTemplate. But lets first >> get this new stuff working well on ProducerTemplate. >> >> >> >> >> On Mon, Sep 12, 2016 at 1:33 AM, Vitalii Tymchyshyn wrote: >> > Hi, all >> > >> > I've finished first draft of async support for ProducerTemplate. Please >> > review and comment. >> > Primary features: >> > * All async* methods of ProducerTemplate now return CompletableFuture >> > instead of regular Future. It makes a bridge from Camel pipelines to >> plain >> > Java 8 CompletableFuture engine. >> > * Async engine is employed. Previously async* methods were just calling >> > regular sync engine from a thread pool. Thus there was no clean way to >> > employ async engine when calling camel routes from Java code. >> > * New parameter was added to DefaultProducerTemplate allowing to skip >> > thread pool while still using async engine. It should allow very >> efficient >> > reactive non-blocking programming with camel. >> > >> > P.S. Honestly I like using camel in reactive programming much. It >> provides >> > a lot of non-blocking endpoints and make debug much easier with route >> > statistics & in-flight repository. As soon as this features will be in >> core >> > I am going to write an blog post highlighting all value added over >> regular >> > CompletableFuture. >> > >> > Best regards, Vitalii Tymchyshyn >> > >> > Нд, 11 вер. 2016 о 19:27 tivv пише: >> > >> >> GitHub user tivv opened a pull request: >> >> >> >> https://github.com/apache/camel/pull/1167 >> >> >> >> CAMEL-10308 >> >> >> >> Provide a way to use async engine from ProducerTemplate >> >> >> >> You can merge this pull request into a Git repository by running: >> >> >> >> $ git pull https://github.com/tivv/camel j8-producer-template >> >> >> >> Alternatively you can review and apply these changes as the patch at: >> >> >> >> https://github.com/apache/camel/pull/1167.patch >> >> >> >> To close this pull request, make a commit to your master/trunk branch >> >> with (at least) the following in the commit message: >> >> >> >> This closes #1167 >> >> >> >> >> >> commit 5daf4fa79a8cfa74e0d3253d07e8548c844de3a9 >> >> Author: Vitalii Tymchyshyn >> >> Date: 2016-09-11T23:25:42Z >> >> >> >> CAMEL-10308 >> >> Provide a way to use async engine from ProducerTemplate >> >> >> >> >> >> >> >> >> >> --- >> >> If your project is set up for it, you can reply to this email and have >> your >> >> reply appear on GitHub as well. If your project does not have this >> feature >> >> enabled and wishes so, or if the feature is enabled but not working, >> please >> >> contact infrastructure at infrastruct...@apache.org or file a JIRA >> ticket >> >> with INFRA. >> >> --- >> >> >> >> >> >> -- >> Claus Ibsen >> - >> http://davsclaus.com @davsclaus >> Camel in Action 2: https://www.manning.com/ibsen2 >> -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: [GitHub] camel pull request #1167: CAMEL-10308
How about seThreadedAsyncMode(boolean), defailt true? I will also do checkstyle till EOW. Best regards, Vitalii Tymchyshyn Чт, 15 вер. 2016 05:08 користувач Claus Ibsenпише: > Hi > > I do wonder if we can come up with a better name for the setter/getter > of synchronous on ProducerTemplate. > > Otherwise great work. The Java 8 API makes the code easier when you > can compose the futures with the thenApply. > > Just a note that we must/should ensure that existing Camel users can > take their code as-is and compile with Java 8 and Camel 2.18 > dependencies without having to change their source code. > > Down the road we should also look at if some of this makes sense to > add to the newer and simpler FluentProducerTemplate. But lets first > get this new stuff working well on ProducerTemplate. > > > > > On Mon, Sep 12, 2016 at 1:33 AM, Vitalii Tymchyshyn wrote: > > Hi, all > > > > I've finished first draft of async support for ProducerTemplate. Please > > review and comment. > > Primary features: > > * All async* methods of ProducerTemplate now return CompletableFuture > > instead of regular Future. It makes a bridge from Camel pipelines to > plain > > Java 8 CompletableFuture engine. > > * Async engine is employed. Previously async* methods were just calling > > regular sync engine from a thread pool. Thus there was no clean way to > > employ async engine when calling camel routes from Java code. > > * New parameter was added to DefaultProducerTemplate allowing to skip > > thread pool while still using async engine. It should allow very > efficient > > reactive non-blocking programming with camel. > > > > P.S. Honestly I like using camel in reactive programming much. It > provides > > a lot of non-blocking endpoints and make debug much easier with route > > statistics & in-flight repository. As soon as this features will be in > core > > I am going to write an blog post highlighting all value added over > regular > > CompletableFuture. > > > > Best regards, Vitalii Tymchyshyn > > > > Нд, 11 вер. 2016 о 19:27 tivv пише: > > > >> GitHub user tivv opened a pull request: > >> > >> https://github.com/apache/camel/pull/1167 > >> > >> CAMEL-10308 > >> > >> Provide a way to use async engine from ProducerTemplate > >> > >> You can merge this pull request into a Git repository by running: > >> > >> $ git pull https://github.com/tivv/camel j8-producer-template > >> > >> Alternatively you can review and apply these changes as the patch at: > >> > >> https://github.com/apache/camel/pull/1167.patch > >> > >> To close this pull request, make a commit to your master/trunk branch > >> with (at least) the following in the commit message: > >> > >> This closes #1167 > >> > >> > >> commit 5daf4fa79a8cfa74e0d3253d07e8548c844de3a9 > >> Author: Vitalii Tymchyshyn > >> Date: 2016-09-11T23:25:42Z > >> > >> CAMEL-10308 > >> Provide a way to use async engine from ProducerTemplate > >> > >> > >> > >> > >> --- > >> If your project is set up for it, you can reply to this email and have > your > >> reply appear on GitHub as well. If your project does not have this > feature > >> enabled and wishes so, or if the feature is enabled but not working, > please > >> contact infrastructure at infrastruct...@apache.org or file a JIRA > ticket > >> with INFRA. > >> --- > >> > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 >
[GitHub] camel pull request #1172: CAMEL-10303 - added ability to detect when the con...
Github user hqstevenson closed the pull request at: https://github.com/apache/camel/pull/1172 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: [GitHub] camel pull request #1167: CAMEL-10308
Hi I do wonder if we can come up with a better name for the setter/getter of synchronous on ProducerTemplate. Otherwise great work. The Java 8 API makes the code easier when you can compose the futures with the thenApply. Just a note that we must/should ensure that existing Camel users can take their code as-is and compile with Java 8 and Camel 2.18 dependencies without having to change their source code. Down the road we should also look at if some of this makes sense to add to the newer and simpler FluentProducerTemplate. But lets first get this new stuff working well on ProducerTemplate. On Mon, Sep 12, 2016 at 1:33 AM, Vitalii Tymchyshynwrote: > Hi, all > > I've finished first draft of async support for ProducerTemplate. Please > review and comment. > Primary features: > * All async* methods of ProducerTemplate now return CompletableFuture > instead of regular Future. It makes a bridge from Camel pipelines to plain > Java 8 CompletableFuture engine. > * Async engine is employed. Previously async* methods were just calling > regular sync engine from a thread pool. Thus there was no clean way to > employ async engine when calling camel routes from Java code. > * New parameter was added to DefaultProducerTemplate allowing to skip > thread pool while still using async engine. It should allow very efficient > reactive non-blocking programming with camel. > > P.S. Honestly I like using camel in reactive programming much. It provides > a lot of non-blocking endpoints and make debug much easier with route > statistics & in-flight repository. As soon as this features will be in core > I am going to write an blog post highlighting all value added over regular > CompletableFuture. > > Best regards, Vitalii Tymchyshyn > > Нд, 11 вер. 2016 о 19:27 tivv пише: > >> GitHub user tivv opened a pull request: >> >> https://github.com/apache/camel/pull/1167 >> >> CAMEL-10308 >> >> Provide a way to use async engine from ProducerTemplate >> >> You can merge this pull request into a Git repository by running: >> >> $ git pull https://github.com/tivv/camel j8-producer-template >> >> Alternatively you can review and apply these changes as the patch at: >> >> https://github.com/apache/camel/pull/1167.patch >> >> To close this pull request, make a commit to your master/trunk branch >> with (at least) the following in the commit message: >> >> This closes #1167 >> >> >> commit 5daf4fa79a8cfa74e0d3253d07e8548c844de3a9 >> Author: Vitalii Tymchyshyn >> Date: 2016-09-11T23:25:42Z >> >> CAMEL-10308 >> Provide a way to use async engine from ProducerTemplate >> >> >> >> >> --- >> If your project is set up for it, you can reply to this email and have your >> reply appear on GitHub as well. If your project does not have this feature >> enabled and wishes so, or if the feature is enabled but not working, please >> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket >> with INFRA. >> --- >> -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
[GitHub] camel pull request #1177: Camel 10319: SNMP Producer
Github user Fabryprog closed the pull request at: https://github.com/apache/camel/pull/1177 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: Camel 3.0 ideas : LeaderPolicy
Yep I'd like something like an open LeaderElection API a user can implement from scratch (i.e. one may want to gahve multipel eladers) or you can define just the leadership part and the default implementation does everything else for you. --- Luca Burgazzoli On Thu, Sep 15, 2016 at 8:34 AM, Andrea Cosentinowrote: > Maybe we can also give the possibility to end user to implement their own > leader election policy. > -- > Andrea Cosentino > -- > Apache Camel PMC Member > Apache Karaf Committer > Apache Servicemix Committer > Email: ancosen1...@yahoo.com > Twitter: @oscerd2 > Github: oscerd > > > > On Thursday, September 15, 2016 8:33 AM, Claus Ibsen > wrote: > On Thu, Sep 15, 2016 at 8:14 AM, Luca Burgazzoli > wrote: >> Hello everyone, >> >> I've been working on some master/slave RoutePolicy and I'm wondering >> if we can have a proper LeaderPolicy with a standardized >> implementation in Camel 3.0 so one has only to notify when a >> leadership is taken >> >> In addition it may be nice to have: >> - a support for Leader election from the CmelContext so the routes are >> started when the context become leader. >> - an option to warm-up routes or to keep them stopped while not leader >> >> >> Make sense ? >> > > Yes it sure does. There may be an older JIRA ticket about something > like this already. But it does not hurt to log a new JIRA either and > mark it for 3.0 so its not forgotten. > > We got a bunch of components now with the route policy for this > master/slave stuff, but IMHO could really benefit from a proper > cluster API. > > Then we can also make it exposed in JMX so tooling are able to detect > which are current master and slaves, and whatnot. > > >> --- >> Luca Burgazzoli > > > > -- > Claus Ibsen > - > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2
Re: Camel 3.0 ideas : LeaderPolicy
Maybe we can also give the possibility to end user to implement their own leader election policy. -- Andrea Cosentino -- Apache Camel PMC Member Apache Karaf Committer Apache Servicemix Committer Email: ancosen1...@yahoo.com Twitter: @oscerd2 Github: oscerd On Thursday, September 15, 2016 8:33 AM, Claus Ibsenwrote: On Thu, Sep 15, 2016 at 8:14 AM, Luca Burgazzoli wrote: > Hello everyone, > > I've been working on some master/slave RoutePolicy and I'm wondering > if we can have a proper LeaderPolicy with a standardized > implementation in Camel 3.0 so one has only to notify when a > leadership is taken > > In addition it may be nice to have: > - a support for Leader election from the CmelContext so the routes are > started when the context become leader. > - an option to warm-up routes or to keep them stopped while not leader > > > Make sense ? > Yes it sure does. There may be an older JIRA ticket about something like this already. But it does not hurt to log a new JIRA either and mark it for 3.0 so its not forgotten. We got a bunch of components now with the route policy for this master/slave stuff, but IMHO could really benefit from a proper cluster API. Then we can also make it exposed in JMX so tooling are able to detect which are current master and slaves, and whatnot. > --- > Luca Burgazzoli -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: Camel 3.0 ideas : LeaderPolicy
On Thu, Sep 15, 2016 at 8:14 AM, Luca Burgazzoliwrote: > Hello everyone, > > I've been working on some master/slave RoutePolicy and I'm wondering > if we can have a proper LeaderPolicy with a standardized > implementation in Camel 3.0 so one has only to notify when a > leadership is taken > > In addition it may be nice to have: > - a support for Leader election from the CmelContext so the routes are > started when the context become leader. > - an option to warm-up routes or to keep them stopped while not leader > > > Make sense ? > Yes it sure does. There may be an older JIRA ticket about something like this already. But it does not hurt to log a new JIRA either and mark it for 3.0 so its not forgotten. We got a bunch of components now with the route policy for this master/slave stuff, but IMHO could really benefit from a proper cluster API. Then we can also make it exposed in JMX so tooling are able to detect which are current master and slaves, and whatnot. > --- > Luca Burgazzoli -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: Getting ready for Apache Camel 2.18 Release
On Wed, Sep 14, 2016 at 3:32 PM, Nicola Ferrarowrote: > Well, it was one of the drawbacks of the approach. Forcing users to include > *only* the camel BOM allows us to completely control the dependencies, but > it's probably a too strict requirement for users. > > We can also provide a option 1+2: i.e. a auto-generated Camel BOM without > any conflict with the spring-boot one (conflicts verified by eg. a maven > plugin). > Users will be able to import it in any order but, of course, some > components will not work because we cannot override what's in the > spring-boot BOM (unless the users force a different version in their pom, > but it's up to them). > > It makes more sense.. > What do you think about it? > That is a really good idea. I think we should try to make Spring Boot the driver here, and have its BOM being imported by default. This is also how start.spring.io website generates projects. Then you "just" add the dependencies (eg typically named -starter). If we have a number of Camel components that don't yet work this way, we can then have them in some exclude list, and then do not generate a -starter component. Then over time we can make them work with SB. Also we may find out that if we change a version in parent/pom.xml to align with SB then it could be a potential thing to do. > > > On Wed, Sep 14, 2016 at 9:46 AM, Claus Ibsen wrote: > >> Hi Nicola >> >> Great work on all this Spring Boot starter stuff. >> >> I would like to discuss/hear more about the #1 option you listed on >> https://github.com/apache/camel/pull/1164 >> >> I think that end users would really prefer their Spring Boot >> applications to be "pure" spring boot by having the Spring Boot BOM >> first and then possible the Camel BOM imported as 2nd. >> >> I am okay if there is some Camel components that would not work with >> Spring Boot such as Cassandra or others. For ActiveMQ then Camel only >> uses that for testing camel-jms component and do not have a strong >> dependency on the version. So end users should likely use the Spring >> Boot ActiveMQ starter. >> >> >> >> On Mon, Sep 12, 2016 at 11:10 AM, Nicola Ferraro >> wrote: >> > I've worked on the spring-boot starters and BOM topic and opened a PR >> > recently. You can find a summary here [ >> > https://issues.apache.org/jira/browse/CAMEL-10222] and this is latest >> PR: >> > https://github.com/apache/camel/pull/1164. >> > >> > Basically, the aim is allowing users to add camel components to their >> > application by just adding the corresponding "xx-starter" project to >> their >> > POM. This can be useful also for initializer tools like >> > https://start.spring.io/ and the likes, to create skeleton of >> applications >> > that just work, without having to worry about wrong transitive >> dependencies. >> > Starter projects take care of, eg. excluding unwanted logging libraries >> and >> > including eg. libraries that are provided in other contexts. >> > >> > The new BOM part is a semi-automated way to generate a BOM for the users >> > that fixes incompatibilities between the camel-parent BOM and the >> > spring-boot-dependencies BOM. They currently differ for the minor (and >> > sometimes major) version of many libraries, including eg. Jetty, >> ActiveMQ, >> > Hibernate Validator, Cassandra driver, etc. Both BOMs also include >> specific >> > versions of common libraries like guava, guice and gson that take >> > precedence over the transitive versions required by the starters, >> resulting >> > in camel components not working correctly. >> > The new BOM (partly generated) should be used in place of the two >> > Camel+Spring-boot BOMs to avoid such issues. >> > >> > I understand that this is a major change, so I ask your feedback about >> the >> > problem (do we want to have this feature to solve these problems for >> > users?) and the solution. >> > >> > Thanks >> > >> > On Fri, Sep 9, 2016 at 6:08 PM, Quinn Stevenson < >> qu...@pronoia-solutions.com >> >> wrote: >> > >> >> Thanks for taking a look at the PR Thomas - I really appreciate the >> >> feedback. >> >> >> >> 1) The parent pom was wrong because I created this PR before the change >> >> from 2.18-SNAPSHOT to 2.18.0-SNAPSHOT took place - it’s been out there a >> >> while >> >> 2) My bad on the READMEmd - you can probably tell where I copied the >> >> example from to get started :-). I’ll get working on that to clean it >> up >> >> 3) I really struggled with this - what example to use. I thought a >> >> little about replacing JMS with something else, but I wasn’t quite sure >> >> what to use. It gets a little more complicated because of the two JVMs >> >> (one for Karaf and one for the bootstrap code). Anyway, if you have a >> >> “good” test route and what you’d like to see happen for testing, I’d >> really >> >> like to see it and I’ll try and use that instead. >> >> 4) I’m not sure where I came up with the name of the example - but
Re: Apache Camel 2.16.4 release
+1 --- Luca Burgazzoli On Thu, Sep 15, 2016 at 8:03 AM, Andrea Cosentinowrote: > +1, Thanks a lot Gregor! > -- > Andrea Cosentino > -- > Apache Camel PMC Member > Apache Karaf Committer > Apache Servicemix Committer > Email: ancosen1...@yahoo.com > Twitter: @oscerd2 > Github: oscerd > > > > On Wednesday, September 14, 2016 8:36 PM, Gregor Zurowski > wrote: > Hi Everyone: > > As previously discussed on the list, we should get a final Camel > 2.16.4 release out the door. If everyone agrees, I will create a > release candidate this week. What do you think? > > Thanks, > Gregor
Camel 3.0 ideas : LeaderPolicy
Hello everyone, I've been working on some master/slave RoutePolicy and I'm wondering if we can have a proper LeaderPolicy with a standardized implementation in Camel 3.0 so one has only to notify when a leadership is taken In addition it may be nice to have: - a support for Leader election from the CmelContext so the routes are started when the context become leader. - an option to warm-up routes or to keep them stopped while not leader Make sense ? --- Luca Burgazzoli
Re: Apache Camel 2.16.4 release
+1, Thanks a lot Gregor! -- Andrea Cosentino -- Apache Camel PMC Member Apache Karaf Committer Apache Servicemix Committer Email: ancosen1...@yahoo.com Twitter: @oscerd2 Github: oscerd On Wednesday, September 14, 2016 8:36 PM, Gregor Zurowskiwrote: Hi Everyone: As previously discussed on the list, we should get a final Camel 2.16.4 release out the door. If everyone agrees, I will create a release candidate this week. What do you think? Thanks, Gregor
Re: Wiki access
Hi I have granted you karma to be able to edit. Thanks for helping the project. Mind that the documentation is being moved to adoc so you may also consider updating the docs in the adoc files if its related to Camel components / languages / data formats etc. http://camel.apache.org/contributing.html On Wed, Sep 14, 2016 at 10:09 PM, Alvin Kwekelwrote: > LS, > > Can you please provide me access to the Camel wiki to I can help improve > the docs. > > I have an approved ASF as alvinkwe...@gmail.com. > > Thanks in advance. > > Kind regards, > Alvin Kwekel -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
Re: Apache Camel 2.16.4 release
+1 On Wed, Sep 14, 2016 at 8:35 PM, Gregor Zurowskiwrote: > Hi Everyone: > > As previously discussed on the list, we should get a final Camel > 2.16.4 release out the door. If everyone agrees, I will create a > release candidate this week. What do you think? > > Thanks, > Gregor -- Claus Ibsen - http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2