Hi Grzegorz-
What you are experiencing is not uncommon for folks that are used to
100% Java-based coding. Its not really a technical issue, as much as it
is a conceptual one. I think one of the most important concepts to
think about is that the Camel approach encourages "configuring", vs
"coding". If you have a lot of existing Java code, it can be difficult
to make the decision to scrap it, but ultimately, I believe that you
will find you'll have far less project artifacts to maintain when using
Camel vs writing your own. Also, consider the upgrade cycle. When you
write your own transport connectors, you have to maintain and re-build
that code when upgrades for the underlying technology changes. With
Camel, you just have to keep up with any changes to the Camel API
(usually, very minor). As Camel upgrades, it brings the updated
dependencies with it-- at little or no impact to your Camel routes.
A lot of this comes down to maintainability and speed at which you can
implement an integration solution-- less project artifacts equals faster
delivery, easier maintenance.
1. If you are not using the components, you are missing out on one of
the most important benefit Camel brings. All that connection setup and
tear down code is redundant. If you are using JMS, HTTP, and databases,
your Java code is going to have a lot of redundancy to what Camel
provides for you. If you have some custom protocol that Camel doesn't
support yet, consider writing a small wrapper "entry service" that uses
a protocol that Camel does support. A Web Service, or a JMS queue. At
the end of the day, less code is always easier to support.
2. If you are making a lot of direct calls, you may consider the
benefits of having *more* JMS in the solution. JMS provides an easy and
reliable way to exchange information, and the guaranteed delivery, error
handling, HA and load balancing capabilites are provided for you, so you
don't have to worry about coding up process to account for those
scenarios. Using JMS would also provide you some statistics and metrics
"for free". Design your integrations around queues, and you can know
exactly how many requests were handled by process X.
3. Perhaps, replacing the "direct" calls with JMS Queues would help to
some degree. Also, look into the Camel tracer and log component. The
?showAll=true option on the log component is gold. It shows you all the
Camel exchange headers, properties and message payload. Great for
debugging.
4. I haven't done any specific load tests against the direct endpoint,
but I haven't seen anything that leads me to believe its horribly slow
or inefficient. A stack of method calls doesn't always necessarily mean
its inefficient.
5. See #3. One of the concepts I try to convey is that instead of
thinking of an integration from a big picture standpoint, think of it in
terms of discreet synchronous and asynchronous steps. These "steps" are
your routes. Leveraging queues and topics is a good way to provide
separation of steps and routes. Ultimately, there will be a few giant
routes that are just a pain. For those, leveraging the tracer, and log
steps makes debugging those a snap.
Hope this helps,
Matt Pavlovich
On 2/10/12 5:34 AM, Grzegorz Borkowski wrote:
Hi All,
In our project we've been using Camel for a year, but with very mixed
feelings. I'd like to ask you if the way we use it is correct - maybe our
problems come from the wrong usage of the tool. Camel experts, please share
your opinions on this. Below you'll find description of the project.
Out project is an integration tool, integrating 3 other systems, call them
A, B and C. We receive the message from system A, query for some data from
system B and send the message to system C. Camel seemed to be ideal tool
for such scenario.
In practice, there are couple of problems though.
1. It turned out that we don't use any typical communication channels to
integrate with the systems. All those systems provided us with a library
(jar) which is simply a stub, which presents us with an API to call remote
systems synchronously. Even if those stubs use JMS or other channels
underneath, we don't interact with them directly. The only thing we do, is
just calling simple synchronous Java method on the stub. Something similar
to the old Java RMI. Thus, the whole power of Camel components is of no
value for us.
2. There is one place where we use Camel JMS component - we have a test
input, for receiving test data, which uses JMS communication. So currently
this is the only place in the code we use Camel components for real
integration with external systems. We're also now developing additional
interface, and this will be the second entry point based on JMS. So those
are only places when we use Camel capabilities.
3. Inside the application, we have a workflow build on the Camel routers,
predicates, and processors. All of them are glued by the "direct"
components. So when XML message comes, we pass it to a router, which passes
it to a proper processor, then to next router etc. There are around 20
processors and 10 routers in the flow. In practice however, we see only one
advantage of using Camel for this: ability to generate visual diagram of
the flow (using camel:dot feature). There are many downsides though. First,
there are many couplings between the processors, so we end up passing tens
of headers in the messages between processors. We already call it
"header-oriented programming" or simply "header hell". It's a nightmare.
Then, it's horror to debug the code when for example the router passed the
message to the wrong path. If it was a normal java code written by us, with
"if then else", we could just put a breakpoint there and debug it. But in
camel, if you configure routers using the built-in predicates, like:
from(...).choide().when(header(..).isEqualsTo(...)).to(...) - then you're
not able to debug it easily, because the if-then-else logic at runtime is
performed by camel engine, not our code.
4. Using the "direct" component connections everywhere seems to be
extremely inefficient. Say I have a flow: message goes through 2 routers
and two processors, linked by direct components. If you generate stack
trace at the end of this path, you will see literally hundreds of camel
calls. Where do they come from? It's especially frustrating when you try to
debug it - it's infeasible. I hate debugging Spring proxies because they
insert about 5-10 framework method calls between my classes. Camel does the
same, but it uses hundreds of framework calls instead. The stack trace is
littered with DelegateAsyncProcessor and AsyncProcessorHelper calls. I
wouldn't really be surprised if I saw StackOverflowError somewhere in the
flow.
5. Testing the Camel flow is not a nice experience as well. For every test,
you have to set up those dummy endpoints, and record expected number of
incoming messages. Then, when there is an exception thrown in the flow,
instead of just breaking the test where it was thrown, we see the cryptic
message at the end "expected receiving 1 message, received 0". Then you
have to analyze the log to find what was the real reason. Also, Camel
waiting for the messages, make the tests much slower, because time is lost
on this waiting.
Thoughts? Recommendations? What do we do wrong?
