Hi,
Having read your message, there are several things you mention and in the
absence of specific architectural details, deployment and route details it
is hard to answer them unambiguously.
In any case, I will try to address some things you bring up with some of the
same generalities.
1. It turned out that we don't use any typical communication channels to
integrate with the systems. All those systems provided us with a library
(jar) which is simply a stub, which presents us with an API to call remote
systems synchronously. Even if those stubs use JMS or other channels
underneath, we don't interact with them directly. The only thing we do, is
just calling simple synchronous Java method on the stub. Something similar
to the old Java RMI. Thus, the whole power of Camel components is of no
value for us.
>>> If System A is a JMS Queue/Topic, unless the jar does something over and
>>> above a simple consume/subscribe, the Camel JMS component should be able
>>> to consume just as well. If however, the message sent by System A is
>>> sent to a non-JMS/Customized JMS Listener, there are 2 choices
a> Write a bean that sets up a listener and gets a message from System
A and pushes it downstream via a camel route (with a Direct, SEDA/VM
endpoint or any other endpoint).
b> Develop a custom Camel component that wraps your jar and acts as a
Consumer/Producer to Systems A, B and C
2. There is one place where we use Camel JMS component - we have a test
input, for receiving test data, which uses JMS communication. So currently
this is the only place in the code we use Camel components for real
integration with external systems. We're also now developing additional
interface, and this will be the second entry point based on JMS. So those
are only places when we use Camel capabilities.
>>>> At the end of the day, you know your system best. Whether to use or not
>>>> to use Camel is your call to make. This forum can help address your
>>>> technical difficulties but not give you a sales pitch on whether/why
>>>> you should/not use Camel ;). The more context and specific detail you
>>>> provide, the better the quality of answers will be...
3. Inside the application, we have a workflow build on the Camel routers,
predicates, and processors. All of them are glued by the "direct"
components. So when XML message comes, we pass it to a router, which passes
it to a proper processor, then to next router etc. There are around 20
processors and 10 routers in the flow. In practice however, we see only one
advantage of using Camel for this: ability to generate visual diagram of
the flow (using camel:dot feature). There are many downsides though. First,
there are many couplings between the processors, so we end up passing tens
of headers in the messages between processors. We already call it
"header-oriented programming" or simply "header hell". It's a nightmare.
Then, it's horror to debug the code when for example the router passed the
message to the wrong path. If it was a normal java code written by us, with
"if then else", we could just put a breakpoint there and debug it. But in
camel, if you configure routers using the built-in predicates, like:
from(...).choide().when(header(..).isEqualsTo(...)).to(...) - then you're
not able to debug it easily, because the if-then-else logic at runtime is
performed by camel engine, not our code.
>>> Please find my observations below...
a> You do not need to do header oriented programming in Camel at all.
You can get all the info you need from data itself if it is available and/or
infer/compute the value of headers if they are not available.
b> You also do not need to use camel DSL code to make your choices.
You can make these choices in Processors(Java code) as well. Please check
out the Camel Processors to do all your choice related work in Java... This
will simplify routes and allow you all the fine grained logging support you
need. It will also give you all the control you desire...
c> It is possible to write very elegant routes and processors and it
is equally possible to write atrocious and ugly routes with poorly crafted
processors and deployment methodologies. This is where you may need to
re-visit your application to see if this is indeed the case.
4. Using the "direct" component connections everywhere seems to be
extremely inefficient. Say I have a flow: message goes through 2 routers
and two processors, linked by direct components. If you generate stack
trace at the end of this path, you will see literally hundreds of camel
calls. Where do they come from? It's especially frustrating when you try to
debug it - it's infeasible. I hate debugging Spring proxies because they
insert about 5-10 framework method calls between my classes. Camel does the
same, but it uses hundreds of framework calls instead. The stack trace is
littered with DelegateAsyncProcessor and AsyncProcessorHelper calls. I
wouldn't really be surprised if I saw StackOverflowError somewhere in the
flow.
>>>> Now that is a very, let us say, "direct" statement ;). If you look at
>>>> the direct component source code, you will find it to be a simple
>>>> in-memory invocation (procedure call) without using queues or any
>>>> intermediary elements. This is why it works only in the same JVM. Now I
>>>> could write a route in several ways (2 given below for e.g.
1>from(a).to(b).to(c);
2>from(a).to(direct:x);
from(direct:x).to(b).to(direct:y);
from(direct:y).to(c);
Note that in the second example, the direct is a simple indirection to
another route segment and its only reason to be is that there may be many
other routes that push into direct:x or direct:y... Otherwise it is
processing overhead...
As for your question about stack traces and DelegateAsyncProcessors, this is
necessary for optimal thread performance and efficient processing along
every element of the route without locking up threads as a message flows
along the route. If you do see a StackOverflowError please do log an JIRA
entry for the same. If not, then you need check the lowest stack trace
message to get the exact nature of the error. This is a good thing from a
traceability standpoint. Since the throwable is being picked up and
re-thrown from several entities in the code, you know exactly how to
reproduce the problem and where to look for issues.
5. Testing the Camel flow is not a nice experience as well. For every test,
you have to set up those dummy endpoints, and record expected number of
incoming messages. Then, when there is an exception thrown in the flow,
instead of just breaking the test where it was thrown, we see the cryptic
message at the end "expected receiving 1 message, received 0". Then you
have to analyze the log to find what was the real reason. Also, Camel
waiting for the messages, make the tests much slower, because time is lost
on this waiting.
>>> I am assuming you use JUnit or Camel Test (based on JUnit). I also
>>> assume you are using the camel mock component based on (EasyMock). In
>>> this case, the mock component is expected to assert whether it received
>>> the message and whether the contents are as you expect them to be. If
>>> the mock component did not get an expected message, it is obvious you
>>> need to check a log to see what happened.
If you need to find out where in the route the message failed, you might
want to check out the Tracer component that will provide a trace of elements
of the route touched prior to failure.
http://camel.apache.org/tracer.html http://camel.apache.org/tracer.html
That said, you will still need to look at your logs... I doubt there is any
technology, product set, stack or other wise where you can figure out why a
message did not arrive by not looking at a log ;)
Hope this helps. While I am in no position to tell you how you should
proceed, this should hopefully address some of your technical misgivings
about Camel...
Cheers,
Ashwin...
-----
---------------------------------------------------------
Ashwin Karpe
Apache Camel Committer & Sr Principal Consultant
FUSESource (a Progress Software Corporation subsidiary)
http://fusesource.com
Blog: http://opensourceknowledge.blogspot.com
---------------------------------------------------------
--
View this message in context:
http://camel.465427.n5.nabble.com/Question-on-the-proper-usage-of-Camel-tp5472181p5472699.html
Sent from the Camel - Users mailing list archive at Nabble.com.
