Re: Swagger integration shows "no content" (Camel 2.16.3)

[jpeschke]

Hello,
Well sometimes, the devil is in the detail.

As I found out, Swagger worked from the beginning. The only problem was that
while serving the Swagger docs under 

"http://://api-docs/" 

in Camel 2.14 (with the old Scala-based Swagger servlet)

it seems that the swagger servlet puts it here:

"http://://api-docs"

(without the trailing slash).
Even if you configure the "api.path" to something like "api-docs/", it still
uses the URL without(!) the trailing slash.

So the whole time, I tested with a wrong url :)


Note:
Unlinke the parameter "api.path", the parameter "base.path" needs a trailing
slash, not like the one I configured above, so it's actually






--
View this message in context: 
http://camel.465427.n5.nabble.com/Swagger-integration-shows-no-content-Camel-2-16-3-tp5787962p5787974.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel in Action Book

[jpeschke]

Hello,
I started learning Camel from scratch about two years ago and read "Camel as
action" at the beginning. I can confirm that "Camel in Action" offers still
a very good point to start. I knew nothing about Camel or EIPs in general
and just bought the book to see if Camel could provide some nice features
for the application I was going to build (and yes, it could indeed :)). 

The book has a very clear didactic & straight-forward approach. Most of the
described topics stayed the same, more or less - if you develop a Camel
application, you may want to use the Camel website as your main reference,
because you often need very special endpoints/properties/features that
aren't documented in the book or where replaced by more flexible ones. But
to understand the basic idea behind camel, to understand what is good
practice (and what isn't), the book's  perfect.

Joerg



--
View this message in context: 
http://camel.465427.n5.nabble.com/Camel-in-Action-Book-tp5787674p5787965.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Need input on accessing http servlet

[jpeschke]

Hi,
You cannot use the Servlet component to connect to a third party servlet on
your server context.
The "servlet:" endpoint is only to offer a servlet interface to your camel
routes. See documentation at http://camel.apache.org/servlet.html:


> You can consume only from endpoints generated by the Servlet component.
> Therefore, it should be used only as input into your Camel routes. To
> issue HTTP requests against other HTTP endpoints, use the HTTP Component

So the HTTP-Component (http://camel.apache.org/http.html) or the
Jetty-Component (http://camel.apache.org/jetty.html) should do the trick.

Best regards,
Joerg



--
View this message in context: 
http://camel.465427.n5.nabble.com/Need-input-on-accessing-http-servlet-tp5787954p5787964.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Swagger integration shows "no content" (Camel 2.16.3)

[jpeschke]

Hi,
I just migrated my application to Camel 2.16.3 (previously 2.14.3).

After some adjustments, everyhing works finde except the Swagger
Integration.
Whenever I call the swagger URL, I get HTTP-Response 204 (No Content). I
enabled both the CORS filter in the REST API AND the Swagger-Servlet...but
that doesn't help.
The REST service itself works.

My configuration in web.xml:


And in the REST routes, I did something like this:



What am I missing?

Joerg




--
View this message in context: 
http://camel.465427.n5.nabble.com/Swagger-integration-shows-no-content-Camel-2-16-3-tp5787962.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Mongodb Persistent tail tracking: How to chose an increasing field for tracking

[jpeschke]

Hello,
Perhaps this is more a MongoDB issue, but maybe somebody has an idea:

The Camel MongoDB endpoint supports the persistent tail tracking feature
(which means that it stores the value of an arbitrary increasing field in a
document to reset the tailable cursor to this document when
restarting/restting the cursor). The field can be for example a timestamp.

Now my problem is:
How do I get such an "constantly increasing field" if I haven't any? MongoDB
hasn't something like an "auto increment" data type, so currently, I use a
counters collection which generates sequences of increasing ids (as
suggested in
https://docs.mongodb.com/v3.0/tutorial/create-an-auto-incrementing-field/#auto-increment-counters-collection)

However, this leads to some race conditions in a
multi-threaded/multi-machine environment, as the process of "Sequence ID
generation" and "insetion" is not atomic:

- Consider two servers (A and B), every one inserts an update. A gets the
first sequence number (e.g. 1), but B inserts it's document (with sequence
number 2) earlier, so the natural (= insertion) order in the capped
collection will now be "2 - 1".
- Now, as the tailable cursor consumes the capped collections in natural
order, it will start consuming the document with id 2. If the cursor is
reset/regenerated at exactly this point, the persistent tail tracker will
store "2" as the last processed document, although "1" was never processed,
so 1 will be lost :(.

Sounds academic, I know, but leads to some annoying errors in out
application :(

Thank you for any ideas.

Best regards,
Joerg





--
View this message in context: 
http://camel.465427.n5.nabble.com/Mongodb-Persistent-tail-tracking-How-to-chose-an-increasing-field-for-tracking-tp5787124.html
Sent from the Camel - Users mailing list archive at Nabble.com.

RESOILVED: Durability with dynamically created Virtual Topic consumers

[jpeschke]

Hello,
Ok, sorry - my fault.
The VirtualTopic consumption works like a charm & exactly as expected. I
just made a mistake in my route setup what caused the recipientList for the
consuming system to be empty. So the whole thing worked just fine from the
beginning, only the camel route that consumed the virtual topic didn't
dispatch the message any further *facepalm* :(.

Analyzing the "Enqueue" und "Dequeue" values within the ActiveMQ broker (via
jConsole) finally got me on the right track.

Thanks anyway!

Joerg




--
View this message in context: 
http://camel.465427.n5.nabble.com/ActiveMQ-Durability-with-dynamically-created-Virtual-Topic-consumers-tp5780446p5780484.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: ActiveMQ: Durability with dynamically created Virtual Topic consumers

[jpeschke]

Update:  A similar problem is described in this thread, but is seems
unsolved.

  



--
View this message in context: 
http://camel.465427.n5.nabble.com/ActiveMQ-Durability-with-dynamically-created-Virtual-Topic-consumers-tp5780446p5780451.html
Sent from the Camel - Users mailing list archive at Nabble.com.

ActiveMQ: Durability with dynamically created Virtual Topic consumers

[jpeschke]

Hello,
I have a little problem when using ActiveMQ's "Virtual Topic" feature
together with dynamically created/stopped Camel routes.

My scenario:
- My camel application offers a web service which other systems within the
company can use to subscribe/unsubscribe for messages that will be then sent
to them over a REST API.
- Camel manages all processed messages internally with an embedded ActiveMQ
broker.
- The other systems may disconnect from time to time (due to maintenance
work or failures)
- When a previously subscribed system reconnects after a shutdown, it should
get all messages that were processed during it's offline time.

My setup:
- The processed messages are sent to a  Active MQ Virtual Topic
  
- A self-implemented Manager bean dynamically creates/removes a camel route
for every system that subscribes/unsubscribes.
- These dynamically created routes listen on the virtual topics and send the
data coming in from the virtual topic to the REST interface of the different
systems (or mark the systems as ""offline if an error occurs)

My Test:
1.) Start up the camel application
2.) Subscribe a (dummy) system for updates via the Webservice - Camel
dynamically creates a route for this system that consumes the virtual topic
3.) Let camel process a message (M1) - it is sent to the subscribed system,
as designed
4.) Set the dummy system offline, Camel stops & removes the route for this
system (again, as expeced)
5.) Let Camel process another message (M2)
6.) Re-Subscribe the dummy system (again, a Camel route is created for the
system, receiving messages from the virtual topic)

I would expect (due to the "queue character" of the virtual topic) that the
dummy system now receives the message that was sent during it'soffline time.
But this is not the case. Nothing happens. New Messages (M3, M4) I put into
Camel are forwarded correctlly, but the message M2 is lost.

Am I missing a point here? Is it a mistake to remove the consuming route
from the Camel context when set to offline? Do I understand the use of
virtual topics correctly or are they the wrong stuff for my usecase?

Any ideas & sugesstions are welcome.

Thanks a lot,

Joerg






--
View this message in context: 
http://camel.465427.n5.nabble.com/ActiveMQ-Durability-with-dynamically-created-Virtual-Topic-consumers-tp5780446.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Issues when suspeding JMSConsumers with "receiveTimeout"

[jpeschke]

Hello,
I followed Claus' suggestion about implementing an own route policy for
throttling the reception of incoming JMS messages if there is too much work
to be done (see  Limit number of JMS messages when processing asynchronously

 
).
Actually, this works like a charm (Again, thank you Claus!) :).

The only issue I still have is with the "receiveTimeout" setting of the JMS
Consumer:
When I suspend the consumer via the "stopConsumer" method of the 
RoutePolicySupport

 
, it seems that Camel waits for the blocking  receive(long timeout)-Method

  
before it completely suspends the consumer.
So if I choose a high value for the "receiveTimeout" setting to prevent too
many reconnects, the suspension can take a lot of time.


Is there any way of working around this issue? We are still using Camel
2.13.1 (switching to a later version is planned), maybe there are
workarounds in a later Camel version?

Thank you in advance for any thoughts, ideas and suggestions.

Best regards,
Joerg






--
View this message in context: 
http://camel.465427.n5.nabble.com/Issues-when-suspeding-JMSConsumers-with-receiveTimeout-tp5776663.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Limit number of JMS messages when processing asynchronously

[jpeschke]

Hello all,
We have a system consuming messages at a high rate from a JMS queue
asynchronously and parallel. For Performance reasons, we have a very high
idle timeout, so normally our camel-based software never reconnects to the
broker and stays connected.

The problem is that it seems that the asynchronous behaviour lets Camel take
as many JMS messages as it can get, so sometimes we have thousands of
inflight messages waiting to be processed. This is not such a big problem
during normal operation, but when we shut down the system for maintenance,
sometimes there are so many messages that they cannot be processed during a
graceful shutdown.

My question is: How can I specify how many messages camel takes out of the
queue when using an asynchonrous consumer?

I already defined
- idleConsumerLimit
- concurrentConsumers
- maxConcurrentConsumers
- maxMessagesPerTask

but that doesn't seem to work (as far as I understand because these settings
influence how many threads are used to get messages out of the queue but not
how many messages may be "pumped" into the camel route)

Any ides?

Thank you & best regards,
Joerg






--
View this message in context: 
http://camel.465427.n5.nabble.com/Limit-number-of-JMS-messages-when-processing-asynchronously-tp5774822.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Limit number of JMS messages when processing asynchronously

[jpeschke]

Thank you four your ideas, Claus.
We need the "asyncConsumer=true" setting as it dramatically boosts our
performance (the broker already delivers the messages unsorted, so we have
to re-order them anyway). 

But a route policy should do the trick - I have already thought about
implementing a RoutePolicy to turn off the JMS consumer in some cases (The
processed JMS messages are buffered in a database, another camel route is
used to fetch & aggregate them. If there are many messages in the JMS queue,
we sometimes experience the problem that database reads are blocked by the
write operations of the JMS consumer until the traffic decreases - so for
periods of high traffic, it could be useful to disable the JMS consumer from
time to time to give the database consumer the chance to kick in)

So - RoutePolicy, here we go!


Thank you,
Joerg




--
View this message in context: 
http://camel.465427.n5.nabble.com/Limit-number-of-JMS-messages-when-processing-asynchronously-tp5774822p5774828.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Lazy init for JMS-Component

[jpeschke]

Hi,
@Claus: Thanks a lot for your help.

Found a configuration that works, for the record, here it is: 

The JmsComponent must not have lazy-init=true, because Camel expects the
bean to be there to initialize the route. But with lookupOnStartup =false
in the JndiObjectFactoryBean, the JMSConnection is looked up not before it
is actually needed by a route, and therefore camel doesn't freeze (or wait
unnessecary long) on test systems where there is no connection to JMS but a
file consumer endpoint with testdata instead.

Note that you need to set the expectedType property on the JndiObjectfactory
bean if you use lookupOnStartup=false so that spring can pass a correct
interface to the JmsComponent:

  bean class=org.apache.camel.component.jms.JmsComponent id=myqueue
property name=connectionFactory ref=jmsQueueConnectionFactory
/
/bean

  bean class=org.springframework.jndi.JndiObjectFactoryBean
id=jmsQueueConnectionFactory
 property name=jndiTemplate ref=jndiTemplate /
* property name=lookupOnStartup value=false /
 property name=expectedType value=javax.jms.ConnectionFactory
/*
 property name=jndiName value=${connectionFactoryName} /
/bean

Joerg




--
View this message in context: 
http://camel.465427.n5.nabble.com/Lazy-init-for-JMS-Component-tp5769586p5769823.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Lazy init for JMS-Component

[jpeschke]

Hi Claus,
I'm using two routes, one for receiving, the one for sending (for
acknowledging received messages).

The problem is, as I understand it, that the camel route processor needs a
JmsComponent Bean with the id someQueueBean to resolve an URL like
someQueueBean:queue:..., but which isn't there when lazy-init=true.

But I will try to fiddle around with the parameters and see what the result
is. Maybe a lazy-init=false on the JmsComponent, but a
lookupOnStartup=false on the JndiObjectFactoryBean does the trick.

Thanks,
Joerg




--
View this message in context: 
http://camel.465427.n5.nabble.com/Lazy-init-for-JMS-Component-tp5769586p5769674.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Lazy init for JMS-Component

[jpeschke]

Hi all,
I have a problem with the lazy initialisation of a JMS component:

I have the following Spring configuration (JMS connection factory will be
looked up via JNDI):

bean class=org.apache.camel.component.jms.JmsComponent id=queue 
property name=connectionFactory ref=jmsQueueConnectionFactory 
/
/bean
 
bean class=org.springframework.jndi.JndiObjectFactoryBean
id=jmsQueueConnectionFactory 
 property name=jndiTemplate ref=jndiTemplate / 
 property name=jndiName value=${connectionFactoryName} /
/bean
...

This works fine so far. The only problem is that this will try to setup the
JMS connection right at startup. What I want is a solution that starts the
whole JMS thing at the moment a camel endpoint is created that needs it but
not before this moment (I need this for integration tests where the
JMS-endpoints are replaced by file endpoints because the test systems do not
nessecarily have access to the JMS broker)

So I configured all the beans to lazy-init=true and told the
JNDIObjectFatcory bean to be looked up NOT at startup:

bean class=org.springframework.jndi.JndiObjectFactoryBean
id=jmsQueueConnectionFactory lazy-init=true
 property name=jndiTemplate ref=jndiTemplate /
 property name=lookupOnStartup value=false /
 property name=jndiName value=${connectionFactoryName} /
/bean

This works, but as soon as I setup a Camel route using my queue bean, I
get errors because the camel JMS endpoint cannot find it, so it does not set
up the JMS connection on demand, as I would have expected.

Any ideas how to do this right?

Thanks for any ideas.

Best regards,
Joerg



--
View this message in context: 
http://camel.465427.n5.nabble.com/Lazy-init-for-JMS-Component-tp5769586.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel SFTP not working in Linux-Linux, working in Windows to Linux.

[jpeschke]

Hi,
Just an idea:
Is your file consumer url configured correctly for UNIX paths?
For absolute paths in UNIX, like /absolute/path/to/your/pdf, remember that
you have three slashes at the beginning of the file consumer URL, e.g.:

file:///absolute/path/to/your/pdf?fileName=output.pdf

Best regards,
Joerg



--
View this message in context: 
http://camel.465427.n5.nabble.com/Camel-SFTP-not-working-in-Linux-Linux-working-in-Windows-to-Linux-tp5769587p5769594.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: MongoDB Consumer Endpoint: ReadPreference cannot be set?

[jpeschke]

I created  a JIRA ticket https://issues.apache.org/jira/browse/CAMEL-8483  
and  a Pull Request on GitHub https://github.com/apache/camel/pull/434  
for this. 

First time contributingto Camel, exciting :). I hope, I didn't break
anything.





--
View this message in context: 
http://camel.465427.n5.nabble.com/MongoDB-Consumer-Endpoint-ReadPreference-cannot-be-set-tp5764015p5764100.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: MongoDB Consumer Endpoint: ReadPreference cannot be set?

[jpeschke]

Hi Raul,
Thank you for your reply. I will see what I can do, if I find the time, I
will patch it myself, otherwise I will at least open a ticket in JIRA.

Thanks a lot,

Joerg



--
View this message in context: 
http://camel.465427.n5.nabble.com/MongoDB-Consumer-Endpoint-ReadPreference-cannot-be-set-tp5764015p5764032.html
Sent from the Camel - Users mailing list archive at Nabble.com.

MongoDB Consumer Endpoint: ReadPreference cannot be set?

[jpeschke]

Hello,
When trying to use the readPreference parameter in a mongodb consumer
endpoint, an illegal argument exception will be thrown during
initialisation:

/operation, writeConcern, writeConcernRef, readPreference, dynamicity,
invokeGetLastError options cannot appear on a consumer endpoint/

I do not quiet understand this behaviour:
In my case, I use a tailable cursor consumer that should listen for
updates on a capped collection. These updates are immutable (they don't get
changed once they have been inserted), so for performance optimization, it
would make perfectly sense to perform those read operations on the secondary
hosts in the replica set.

The only reason I see for forbidding the definition of a different read
preference here is that we need a write operation if persistent tail
tracking is enabled and this should be performed on the primary. But as the
MongoDB Java API would allow you to define a readPrefercne=primary only on
the persistent tail tracking collection, this isn't a conflict, is it?

Best regards,
Joerg



--
View this message in context: 
http://camel.465427.n5.nabble.com/MongoDB-Consumer-Endpoint-ReadPreference-cannot-be-set-tp5764015.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: MongoDB: Persistent tail tracking with concurrent tailable consumers

[jpeschke]

Hi,

Claus Ibsen-2 wrote
 Well spotted the bug. Feel free to log a JIRA ticket and provide a patch
 http://camel.apache.org/contributing.html

Thanks.  I submitted a Bug in JIRA.
https://issues.apache.org/jira/browse/CAMEL-8063  

Thanks for this great software, by the way (and exspecially @Claus: Thanks
for Camel in Action, helped me a LOT doing my first steps in camel riding
:))!





--
View this message in context: 
http://camel.465427.n5.nabble.com/MongoDB-Persistent-tail-tracking-with-concurrent-tailable-consumers-tp5759131p5759327.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: MongoDB: Persistent tail tracking with concurrent tailable consumers

[jpeschke]

Hi,
It's me again.
I think I found the problem:

It's in the method initialize in MongoDbTailTrackingManager.java:



If no tail tracking object exists in the database, a new one will be
inserted (that's okay), but the query dbCol.findOne() fetches ANY tail
tracking object from the database (and not the one we've just inserted).

In my oppinition, this is a bug and should be corrected like this:


For now, I work around this by putting every persistent tail tracker in it's
own collection so I make sure that even a findOne() always gives me the
right tracker.

Best regards,
Joerg



--
View this message in context: 
http://camel.465427.n5.nabble.com/MongoDB-Persistent-tail-tracking-with-concurrent-tailable-consumers-tp5759131p5759202.html
Sent from the Camel - Users mailing list archive at Nabble.com.

MongoDB: Persistent tail tracking with concurrent tailable consumers

[jpeschke]

Hello,
I observe a very strange behaviour with concurrent tailable consumers and
persistent tail tracking. Perhaps you have an idea:

Scenario:
- A MongoDB Replicaset of 3 servers, running MongoDB 2.4.11
- 2 camel servers (tomcat7 with Sun JDK, Camel 2.13.1)
- On each server, 5 tailable cursor consumer endpoints, listening on 5
different collections with persistent tail tracking enabled
- The persistent id for each consumer endpoint is defined as:
  hostname_CollectionName
- So the collection storing the persistent tail tracking information
(consumedUpdates) looks something like this:



The endpoint configuration looks something like this:

mongodb://mongoDBInstance?collection=COLLECTIONdatabase=...persistentId=HOST_COLLECTIONpersistentTailTracking=truetailTrackCollection=consumedUpdatestailTrackIncreasingField=updateId

The problem: Only one document in the consumedUpdates collection seems to
be updated, no matter which collection or host is affected. It seems like
this is always the first document in the collection. So all consumers write
into the same document, overwriting information coming from other consumers
:(

The problem does not occur on our test environment (1 camel server, 1
MongoDB, both on the same machine).

Any ideas?

Thanks for your help!

Best regards,
Joerg





--
View this message in context: 
http://camel.465427.n5.nabble.com/MongoDB-Persistent-tail-tracking-with-concurrent-tailable-consumers-tp5759131.html
Sent from the Camel - Users mailing list archive at Nabble.com.