Re: Can't test Camel processor

[Larry Shields]

This might be a little simplistic but I normally set up test routes for my
processors and then send exchanges directly to them with the expected
Exchange structure for them to process.

So for example, I have a processor named CsvProcessor.  It expects to
receive a Map from the Marshaller of a CSV File: (Note, I
normally register my processors as Beans in a Spring Configuration class,
you can use component annotations to get the same effect)

public class CsvProcessor implements Processor {

  @SuppressWarnings("unchecked")
  @Override
  public void process(Exchange exchange) {

Map row = exchange.getMessage().getBody(Map.class);
exchange.getMessage().setHeader("RowId", row.getOrDefault("Id", "0"));
exchange.getMessage().setBody(row.getOrDefault("Name", "None"));
  }
}

To Test is, I will setup a Test Route for my unit tests like the following:

@Component
public class CsvProcessorTestRoute extends RouteBuilder {
  @Override
  public void configure() {

from("direct:csv-input")
.routeId("csv-processor-test-route")
.bean(ProcessorConstants.CSV_PROCESSOR)
.to("mock:csv-output")
.end();

  }
}


Then I can use this route directly in my Unit Tests to send an Exchange to:

@ExtendWith(SpringExtension.class)
@SpringBootTest
class CsvProcessorTest {

  @EndpointInject("direct:csv-input")
  ProducerTemplate template;

  @EndpointInject("mock:csv-output")
  MockEndpoint output;

  @Test
  @DirtiesContext
  void testCsvProcessor() {
Map row = new HashMap<>();
row.put("Id", "1");
row.put("Name", "Jim Smith");

var exchange = ExchangeBuilder.anExchange(template.getCamelContext())
.withBody(row)
.build();
template.send(exchange);

assertThat(output.getExchanges()).as("Output should not be
empty").isNotEmpty();

var msg = output.getExchanges().get(0);

assertThat(msg.getMessage().getHeader("RowId"))
.as("Row Id should be in the Header")
.isEqualTo("1");
assertThat(msg.getMessage().getBody())
.as("Body should be the name")
.hasToString("Jim Smith");
  }
}

I'm sure there are a number of ways to accomplish this problem, but this is
how I've tackled this with pretty positive success.  I've placed this
sample project on GitHub at the link below.
https://github.com/openbatch/camel-unit-testing

Hope this helps!

On Fri, Mar 4, 2022 at 6:58 AM Alaeddine HAOUAS <4codynam...@gmail.com>
wrote:

> Hi,
>
> I have an issue in a unit test, to verify a Processor.
> Context is a Spring-boot application used to control a Camel route, that
> triggers eventually a Camel processor.
>
> In a JUnit test, I achieved to have the RouteBuilder started and tested
> successfully until the instruction setting up the Processor. Processor is
> not starting, so I can't verify it.
>
> How can I successfully trigger/test the processor ?
>
> The code :
>
> @Component
> public class CsvRouteBuilder extends RouteBuilder {
> //
>
> private void process(.) {
> // .
>
> String fromRoute = String.join("", "file:",
> orgParPath.toAbsolutePath().toString(),
> "?moveFailed=", ERROR, "&move=", ARCHIVE, "&delay=",
> delayIntervalMilliSec);
>
> // initializing the context
>
> final Processor processor =
> CsvProcessor.builder().csvExecutionContext(context)
> .csvHdConnectProxyRestTemplate(hdConnectProxy)
> .executorService(executorService).build();
>
> from(fromRoute).unmarshal().csv().process(processor);
> }
>
> // another class
> @Builder
> public class CsvProcessor implements Processor {
> // ...
> // The custom processor has nothing exotic, it is just implementing the
> process method.
> }
>
> // another class
> @ExtendWith(SpringExtension.class)
> @CamelSpringBootTest
> @UseAdviceWith
> @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
> classes = Application.class)
> @EnableAutoConfiguration(exclude = {DataSourceAutoConfiguration.class,
> HibernateJpaAutoConfiguration.class})
> @DisableJmx
> @ContextConfiguration(classes = {TestConfig.class})
> @DirtiesContext(classMode = ClassMode.AFTER_EACH_TEST_METHOD)
> public class CsvRouteBuilderTest {
>
> @Autowired
> ProducerTemplate producerTemplate;
>
> @Autowired
> private TestRestTemplate restTemplate;
>
> @Autowired
> private CsvRouteBuilder route;
>
> // 
> }
>
> Regards
> Alaeddine
>

Re: Camel AWS SQS and concurrentConsumers

[Larry Shields]

Hi Arnaud,

I think what may be happening is that you first consumer is grabbing all of
the messages from the queue that are available.  The default message poll
size is unlimited so your other 4 consumers are polling for nothing on the
queue.  What you might want to try is setting the maxMessagesPerPoll to a
value.  This will reduce the number of messages a consumer retrieves for
each poll.

The other thing I have done is set Greedy=true when using concurrent
consumers if you're looking to increase throughput.  This will cause the
consumer to make another poll immediately without waiting for the next
delay if the previous poll returned at least 1 message.

Example:

aws-sqs://my-queue?greedy=true&maxMessagesPerPoll=10&concurrentConsumers=5

So in this example, each consumer will pull no more than 10 messages per
poll to the SQS Queue.  So if there are less than 10 messages on your
queue, the other consumers aren't going to really get an messages.

On Mon, Jan 31, 2022 at 7:15 AM Arnaud Level  wrote:

> Hi Narsi,
> It's a fifo but I actually tried the simple code I've posted on a standard
> one too and observed the same behavior.
>
> On Mon, Jan 31, 2022 at 1:11 PM Narsi Reddy Nallamilli <
> narsi.nallami...@gmail.com> wrote:
>
> > Hi Arnaud,
> >
> > Is your AWS queue type fifo or standard?
> >
> > On Mon, 31 Jan, 2022, 17:31 Arnaud Level,  wrote:
> >
> > > Hi,
> > >
> > > (Camel version: camel-aws2-sqs-starter: 3.12.0)
> > >
> > > I am trying to use and understand concurentConsumers with a SQS queue:
> > >
> > >
> >
> from("aws2-sqs://queuexxx?concurrentConsumers=5&amazonSQSClient=#sqsClient&
> > > waitTimeSeconds=20")
> > > .process(exchange -> {
> > > System.out.println("Message received...");
> > > })
> > > .process(exchange -> {
> > > try {
> > > Thread.sleep(5000);
> > > } catch (InterruptedException e) {
> > > e.printStackTrace();
> > > }});
> > >
> > > With the above queue, if I send 3 messages at the same time, I have to
> > wait
> > > 5 seconds to see the second message ("Message received...") and 5 more
> > > seconds to see the third one. My understanding of concurentConsumers is
> > > that with a value of 5 I would see the 3 messages at the same time
> since
> > 3
> > > threads will consume them in parallel. If I add the Thread.sleep in a
> > seda
> > > route, I'm having this behavior (= The 3 messages are read at the same
> > > time).
> > >
> > > Turning on the Camel logs it seems that the next polling is done only
> > after
> > > the Delete for the previous message is sent (which is with a delay of
> > 5s).
> > >
> > > I would understand the above behavior with concurentConsumers=1 but I
> > don't
> > > with concurentConsumers=5. Could someone tell me what I've
> misunderstood
> > ?
> > >
> > > Thank you in advance!
> > > Arnaud
> > >
> >
>

Re: camel restart context

[Larry Shields]

If you need a simple example for restarting a route from another route, I
have an example on Github below:
https://github.com/runstache/camel-restart-routes

There are 2 routes, the Maintenance route will restart the other routes
based on a file drop.

On Tue, Jan 14, 2020 at 6:38 AM Claus Ibsen  wrote:

> Hi
>
> Yeah likely, see the FAQ
>
> https://camel.apache.org/manual/latest/faq/how-can-i-stop-a-route-from-a-route.html
>
> On Tue, Jan 14, 2020 at 11:45 AM Damien Nicolas 
> wrote:
> >
> > Hello,
> >
> > I need to restart the camel context with a quartz at fixed interval.
> > But when I do context.stop() and then context.start() inside quartz
> > processing; the context stop but doesnt start again...
> > Also, when I remotely invoke the JMX camel restart, it works.
> > Is this a normal behaviour?
> >
> > Thanks.
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Re: Help with using application.properties entries with Camel and Spring Boot

[Larry Shields]

Looking at the stack trace it looks like the problem is with what it being
returned from your call to the Web Service.   Reading from the properties
file, I normally like to use a configuration class in Spring to hold these
values.  Then just Autowire it into the Route Builder class.  Here's an
example:

https://github.com/runstache/java-came-metrics

The AppSettings Class is populated with the values in the
application.properties file.  Also going this route, you can swap in
different properties files based on a spring profile or inject the class
for things like unit testing different routes and processors.  Once you
have them in a config class, it's really just string building then in the
Camel DSL for the route.

Note: The application.properties file in this project is utilizing
environment variables for it's values.

On Mon, Aug 5, 2019 at 3:44 AM Viki Keaton  wrote:

> Hello-
> I'm new to Camel and excited about it's use.  However, I find myself very
> frustrated with what seems to be the simplest of things.  I have several
> questions, but will try and keep them in separate posts.
>
> I am using Eclipse, Camel 2.24.0, Spring Boot 2.2.0, Java 1.8, and I can
> not use any xml.
>
> I would like to dynamically build my URIs based on properties from an
> application.properties file that is on the classpath, as well as use
> exchange header parameters.
> I thought I could use toD() to create dynamic URIs?  This seems to be
> blank.  What am I doing wrong?  Also, please note that I tried to print my
> application.properties value for the key "data.trans.vm", and I get nothing.
>
> My goal is to be able to use the applications.properties file as well as
> header parameters.  Thanks in advance for your help.
>
>
> My ~/src/main/resources/applications.properties file:
> data.trans.greeting=Hello Viki
>
> @Component
> public class DataTransRouteBuilder extends RouteBuilder {
> @Override
>public void configure() throws Exception {
>
> from("jetty:http://localhost:8081/foobar";).tracing()
>   .log("data mart id is: ${header.dataMartId}")
>   .to("direct:getDataMart");
>
>
>from("direct:getDataMart").tracing()
>.log("at direct:getDataMart")
>.log("properties: DATA_TRANS_VM is
> properties:{{data.trans.greeting}}")
>.setBody().constant(null)
>.setHeader(Exchange.HTTP_METHOD, simple("GET"))
>.setHeader("Content-Type", constant("application/json"))
>.setHeader("Accept", constant("application/json"))
>.setHeader("Accept-Encoding", constant("identity"))
>.setHeader("Authorization", constant(BASIC_AUTH))
>.log("call data mart url for id=${header.dataMartId}")
>.toD("http:// SERIVE>:/${header.dataMartId}/build?bridgeEndpoint=true")
>.choice()
>
> .when().simple("${header.CamelHttpResponseCode} != 200")
>.log("Failure to call: " + DATA_TRANS_DATAMART_URL)
>.to("direct:ERROR")
>  .end()
>
>.unmarshal().json(JsonLibrary.Jackson, JobInfo.class)
>.bean(dataTransJobInfoProcessor, "process")
> ...
> From my console:
> ...
> 2019-08-03 23:39:32.195  INFO 7612 --- [   main]
> o.a.camel.spring.SpringCamelContext  : Apache Camel 2.24.0
> (CamelContext: camel-1) started in 0.918 seconds
> 2019-08-03 23:39:32.254  INFO 7612 --- [   main]
> o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080
> (http) with context path ''
> 2019-08-03 23:39:32.258  INFO 7612 --- [   main]
> a.r.l.a.ReckonLineageAdaptersApplication : Started
> ReckonLineageAdaptersApplication in 16.714 seconds (JVM running for 17.413)
> 2019-08-03 23:39:41.859  INFO 7612 --- [tp1955991197-25] route1
>: data mart id is: 12345
> 2019-08-03 23:39:41.860  INFO 7612 --- [tp1955991197-25] route2
>: at direct:getDataMart
> 2019-08-03 23:39:41.860  INFO 7612 --- [tp1955991197-25] route2
>: properties: DATA_TRANS_VM is
> properties:{{data.trans.vm}}
> 2019-08-03 23:39:41.861  INFO 7612 --- [tp1955991197-25] route2
>: call data mart url for id=12345
> 2019-08-03 23:39:42.197 ERROR 7612 --- [tp1955991197-25]
> o.a.camel.processor.DefaultErrorHandler  : Failed delivery for (MessageId:
> ID-vkeat01116-1564893569848-0-3 on ExchangeId:
> ID-vkeat01116-1564893569848-0-1). Exhausted after delivery attempt: 1
> caught: com.fasterxml.jackson.databind.exc.MismatchedInputException: No
> content to map due to end-of-input
> at [Source:
> (org.apache.camel.converter.stream.CachedOutputStream$WrappedInputStream);
> line: 1, column: 0]
> ...
>
> Message History
>
> --

Re: from camel.main.Main to camel.spring.Main: how to replace registry bind method without XML?

[Larry Shields]

Is there a reason you want to use camel.spring.Main and register the
beans/routes yourself? The beans added to the Spring bean registry should
be available in the Camel Bean registry.  You should be able to use the
Component scan for your Routes and also a Configuration Class to define any
specific beans to be part of the registry like the CloudBlockBlob client.
No Spring XML needed.

Here's an example of using the annotations for Spring/Camel in this manner
which you don't need to manually add your routes and beans.  I didn't test
that the Azure component actually works with this example as i don't have
an Azure account at the ready to verify, but I've been using the same
pattern in other projects with success.

https://github.com/runstache/camel-sample-springdsl



On Mon, May 13, 2019 at 6:54 AM Marx, Peter 
wrote:

> I want to move my use case from standalone JavaDSL to Spring, later to
> Docker/k8s, but with a minimum of XML.
>
> When replacing org.apache.camel.main.Main with
> org.apache.camel.spring.Main, adding a bean to registry no longer works:
>
>import org.apache.camel.spring.Main;
>
> Main myapp = new Main();
>
> myapp.bind("credsreg", new CredsBean().creds()); //add bean to
> default camel registry
>
> ...
>
>
>
> static class CredsBean
> {
>   xy creds()
>
> {
>xy creds = new ... ;
> return creds;
>   }
> }
>
>
>
>   (later I look up the bean in a route like this:
> ...?credentials=#credsreg )
>
>
>
>
> This is because camel.spring.Main has no bind method in contrast to
> camel.main.Main.
>
> What alternatives do I have to get my bean stored in a registry without
> using SpringXML, which would mean e.g.
>  ?
>
> Can I use the default ApplicationContextRegistry then?
>
> Peter
>
>
> From: Marx, Peter
> Sent: Friday, April 19, 2019 10:56 AM
> To: 'users@camel.apache.org' 
> Subject: azure-blob with JavaDSL: can't set URI options with bean or any
> other method
>
> Hi,
>
> I want to add a route for the camel-azure component 2.23.0 in pure JavaDSL
> with no Spring. Documentation
> https://github.com/apache/camel/blob/master/components/camel-azure/src/main/docs/azure-blob-component.adoc
> only recommends to set the credentials and client options with a bean, but
> I can't  get that to work. How to set the options directly in the URI is
> not described.
> Without bean I get an exception like this:
>
> [ERROR] Failed to execute goal
> org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project
> testjava: An exception occured while executing the Java class. Failed to
> create route route1:
> Route(route1)[[From[azure-blob:/shared1/mxtest/blabla?azureB... because of
> Failed to resolve endpoint:
> azure-blob:///shared1/mxtest/blabla?azureBlobClient=client due to: Could
> not find a suitable setter for property: azureBlobClient as there isn't a
> setter method with same type: java.lang.String nor type conversion
> possible: No type converter available to convert from type:
> java.lang.String to the required type:
> com.microsoft.azure.storage.blob.CloudBlob with value client
>
> With bean it is the same error "no type converter...with value
> #clientreg"  - I assume the bean lookup does not work -  would be my 2.
> question how to fix that..
>
> The azureBlobClient option must be of type CloudBlockBlob, credentials of
> type StorageCredentials.
>
> How can I set the required credentials or client options in the URI ?
>
>
> Peter
>
>
> package testjava;
>
> import org.apache.camel.*;
> import org.apache.camel.impl.SimpleRegistry;
> import org.apache.camel.language.Bean;
> import org.apache.camel.main.Main;
> import org.apache.camel.CamelContext;
> import org.apache.camel.impl.DefaultCamelContext;
> import org.apache.camel.builder.RouteBuilder;
> import com.microsoft.azure.storage.*;
> import com.microsoft.azure.storage.blob.*;
>
> public class MainApp
> {
> public static class ClientBean
> {
> public CloudBlockBlob client()
> {
>try
>{
>StorageCredentials creds =
> new StorageCredentialsAccountAndKey("shared1", "here gooes my key
> jG/zyK");
>StorageUri storageURI = new
> StorageUri(new java.net.URI("https://shared1.blob.core.windows.net/mxtest
> "));
>CloudBlockBlob client = new
> CloudBlockBlob(storageURI, creds);
>return client;
>}
>catch(Exception e)
>{
>return null;
>}
> }
> }
>
> public static void main(String... args) throws Exception
> {
>   

Re: using jsonpath for choice

[Larry Shields]

Rajith,

I created an example using Alex's suggestion and it appears to be working
as expected.

Here's the sample on Github:
https://github.com/runstache/java-camel-jsonpath

hth,

Larry

On Wed, Mar 27, 2019 at 2:33 PM Alex Dettinger 
wrote:

> Hi Rajith,
>
>   Not sure we are in the same context but I think it's worth trying this
> expression: $[?(@.approved == true)].
>
> hth,
> Alex
>
> On Tue, Mar 26, 2019 at 9:41 PM Rajith Muditha Attapattu <
> rajit...@gmail.com>
> wrote:
>
> > Thanks for the response Larry.
> > The camel version is 2.20.1
> > And I do have the camel-jsonpath dependency
> >
> > The when clause expects a predicate, and I think it's bcos that
> expression
> > doesn't evaluate to a boolean (I stand to be correct though).
> >
> > On Mon, Mar 25, 2019 at 4:45 PM Larry Shields 
> > wrote:
> >
> > > I'm using the Jsonpath functionality in a split() with the syntax you
> > have
> > > without any issue.  What version of camel and do you have the
> > > camel-jsonpath dependency?
> > >
> > > On Mon, Mar 25, 2019 at 4:29 PM Rajith Muditha Attapattu <
> > > rajit...@gmail.com>
> > > wrote:
> > >
> > > > Is there a better way to do the following ?
> > > >
> > > >
> > >
> >
> .when(PredicateBuilder.isEqualTo(ExpressionBuilder.languageExpression("jsonpath","$.approved"),ExpressionBuilder.constantExpression(true)))
> > > >
> > > >
> > > > as in
> > > > .when().jsonpath(...)
> > > >
> > > > Already tried .when().jsonpath("$.approved[?(@ == true)]") and it
> > doesn't
> > > > work.
> > > >
> > > > --
> > > > Regards,
> > > >
> > > > Rajith Muditha Attapattu <http://rajith.2rlabs.com/>
> > > >
> > >
> >
> >
> > --
> > Regards,
> >
> > Rajith Muditha Attapattu <http://rajith.2rlabs.com/>
> >
>

Re: using jsonpath for choice

[Larry Shields]

I'm using the Jsonpath functionality in a split() with the syntax you have
without any issue.  What version of camel and do you have the
camel-jsonpath dependency?

On Mon, Mar 25, 2019 at 4:29 PM Rajith Muditha Attapattu 
wrote:

> Is there a better way to do the following ?
>
> .when(PredicateBuilder.isEqualTo(ExpressionBuilder.languageExpression("jsonpath","$.approved"),ExpressionBuilder.constantExpression(true)))
>
>
> as in
> .when().jsonpath(...)
>
> Already tried .when().jsonpath("$.approved[?(@ == true)]") and it doesn't
> work.
>
> --
> Regards,
>
> Rajith Muditha Attapattu 
>

Camel Metrics Component v 2.23.0

[Larry Shields]

Hello,

I've run into an issue with the Camel Metrics component in v2.23.0 which
appears to be due to the change in the io.dropwizard.metrics metrics-core
libraries from v3.x to 4.x.  When enabling Jmx support for the
MetricsRoutePolicyFactory, this causes an exception loading the JmxReporter
class.  It appears this is because the JMX support was migrated out of the
metrics-core libraries in v4.x which has a different path to the
JmxReporter class.

Has anyone solved this in v2.23 of the Component?  I've reverted back to
2.22 which appears to be working but would like to move up to v2.23 for the
new FHIR component and updated Spring version support.

Thanks,

Larry Shields