Hi Hadrian,
Thanks for explaining, I understand it now what you mean.
About terminology. May be we'd better stick with "transient" and
"persistent" because with "faults" and "exceptions" I'll have to look it
up email each time to remember which one is it.
I totally agree with you on having this distinguish between two types of
faults (I hope I got it right this time :). Many reties of inserting
large batch into SQL when there is foreign key violation can overload
sql server.
As I understand it, components will be responsible for deciding what is
transient and what is not. For example jdbc component can interpret the
same SQL exception class differently. If it is a timeout or a deadlock,
there is sense to try it again. But if it is a key violation or wrong
login then it is persistent.
So what are (will be) the rules for well behaved component? Perhaps we
want to keep it backward compatible and in case if exception is not
handled by component it is interpreted as transient error?
So if component decides that error is persistent then what? Set "fault"
of exchange to an exception? Or there will be a special property in
runtime exception?
Vadim.
Hadrian Zbarcea wrote:
I reluctantly pick the gauntlet. I feel your passion, and it looks to
me more like violent agreement than disagreement (at least of the fact
that there is some cleanup to be done). So let's clarify a few things.
I am not sure how carefully you read
https://issues.apache.org/activemq/browse/CAMEL-316 and the thread
referenced there. I think the consensus is to combine faults and
exceptions, we just have to nail out the details.
So distinguishing fault and exception in Camel is senseless. There is
no difference between two.
I certainly understand this (your) point of view. I wish things were as
clear cut as you suggest, but I am not convinced that's the case, on the
contrary.
I'll move away from the terms fault and exception for a bit. There are
two kinds of errors: transient and persistent. Transient errors
indicate some environmental problem that prevented carrying out the
request. Persistent errors indicate something wrong with the request.
In the former case, if the caller were to issue the request again at a
later time, it may get it processed successfully. In the latter case,
no amount of retries would help.
Some protocols/technologies distinguish between the two kinds of errors,
some do not and some are in between (think http:, on a 500 you might
want to retry, on a 404, 403, 401 one shouldn't!). Wsdl for instance
provides the faults. Some (architects) use them to model the persistent
errors a system may return, some do not. What is the intended audience
for faults? The caller. From this point of view yes, camel, which is
part of the infrastructure should not care about faults, they are an
output like any other, which happens to be an exception, which should be
sent back to the caller to take appropriate action. You might say that
from a Camel perspective a Fault is *not* an Exception, it's a valid
reply to be sent back to the caller.
You ask "What "not in the way intended" mean?", well it means that on a
financial transfer for instance, you may get an "insufficient funds"
fault that indicates that the server got the request and for the reason
provided decided not to carry out the request in the intended way, i.e.
to actually complete the transfer and no amount of retries would change
that until somebody opens the purse and deposits more funds in the said
account. More precisely for me "the way intended" means such as to get
a valid response/output, not a fault. That is more often than not, when
requests are sent, it is done with the intention of receiving a valid
output, not a fault (which may not be true for a test application, but I
think you got the point, sorry for not being more clear in my past post).
Good infrastructure relieves the caller as much as possible of dealing
with transient errors taking the appropriate measures (retries,
whatever), we have the ErrorHandler for that. Great infrastructure
allows one to deal with (at least some of the) persistent errors. Why?
Because in many cases they way to handle them is the same, so that could
be automated. So in that case you could see the Fault as *being* an
Exception and the infrastructure could use predefined rules to handle them.
The issue is not if a Fault is of a java.lang.Exception type (I think
there was a comment stating that: of course Faults are Exceptions,
meaning of java.lang.Exception type). It may very well be. From a web
services framework point of view, they can be modeled as of
java.lang.Exception type. But Camel is a router, the question is if and
how should Camel handle Faults. My point is that I would prefer to not
lose the semantical distinction between fault and exception, always
react to exceptions and optionally react to fault (and we do some of that).
As far as implementation goes, I could see faults going as outputs (and
being of java.lang.Exception type) so one can have rules to handle them
*if* desired. Or we can a header saying that this output is actually a
fault, so handling could be based on that. (That means that in my mind
a Fault is more of an output than an Exception in Camel Exchange sense).
To your second point about In/Out. The In is always read-only. One
never modifies the in, this is always the message that was sent by the
producer. It's history, doesn't change. Out is always the response.
And there is the exchange pattern: InOnly, the Out is always null,
InOut, then well the out is null. There is that ambiguity by convention
that the next in is either last out of *if null* the last in. This
should be clarified indeed and produced confusion more than once.
In your filter example, yeah, of course the result is a message, but
think of the exchange as having the whole context, the message before
the filter, the one after, the exception (if any) and some
context/properties/headers (maybe). I am not sure if a null output
should mean "stop" the pipeline. For an InOut exchange patter, maybe,
although I doubt this happens a lot in practice. For an InOnly pattern
(such as log, wiretap, etc, that don't have an output, they're not an
'echo') I see nothing wrong with doing what we already do, which is
sending the same In to the next processor. Keep in mind that some
technologies do not have a way to formalize the exchange pattern (such
as wsdl does for instance) so the only way to distinguish between an
InOnly and InOut is based on convention, such as the Out being null.
I wholeheartedly agree that this (more or less) edge cases should be
properly clarified and documented, and I hope you'll keep the ball
rolling, so we could hopefully get it done in 1.5. I don't think it's
too hard to do, but one, we need to agree on how to do it and two,
minimize the impact of the api change. (I think it's reasonable tough
to keep the api, deprecate and never use the fault for 1.5).
Thoughts?
Hadrian
On Sep 14, 2008, at 5:23 PM, Vadim Chekan wrote:
I even didn't know about FAULT :(
Does it mean that all my processors are buggy because I do not check
fault?
My understanding of Fault in SOAP is that *it is* an exception. WS
frameworks generate proxy classes which check fault and throw
exception. The reason Soap use fault is that it is the only way you
can serialize an exception.
The same approach on server side: any business exception is caught by
WS handler, wrapped into fault code and sent back.
So distinguishing fault and exception in Camel is senseless. There is
no difference between two.
Fault code is nothing else but serialized exception.
I'm afraid I can't agree with Hadrian:
> Outputs provide a response after
> successful processing. Faults are responses that indicate to the
> client that the expected goal was not achieved, and they usually
> provide a reason. Faults are messages that tell a client that the
> server did receive the request and successfully processed it, but not
> in the way intended and the client may need to take further action
What "not in the way intended" mean? What should Camel do if something
went not in the way intended? The answer is: nothing. Camel knows
nothing about application business stuff. From Camel perspective
message is either processed or not. Period. It is application which
should know that certain component can return two types of responses
and act accordingly. From Camel perspective it is a perfectly valid
response which is indistinguishable from Out.
Now, lets talk use cases. Lets say we have a batch to process. Like
any programmer I'm all about ACID so I design my system to rollback
and fail the whole batch if any item failed.
But my users are unhappy. They come to me and say that if I just
create a list of errors and keep batch executing they would rather fix
errors manually instead of cleaning source data and re-running the batch.
So now I'm facing dilemma. If before knew that if no exception happen
then batch succeeded but now I have a "failed" response with list of
failed items. So I can not throw exception anymore. I need to process
this "failed" list by publishing it.
But it is very important to understand that from service design point
of view there was no errors! I got a response with a list of items
which need to be published. Sounds like typical workflow. It is error
only from customer perspective, not from the Camel or even application
perspective!
I provided this long example because I suspect that this "fault"
conception is interpretation of application-level faults. I want to
show that there exist no "fault" message as opposite to exception. If
you have a use case which demonstrates "fault" message, lets discuss.
In/Out
==============
I must admit I had some confusion grasping "exchange" conception. The
word exchange assumes that there is 2-directional flow between 2
entities. But Filter has only one direction: from one component to
another. So exchange between what and what is happening?
In/Out is perspective of component. The component is given In message
and must provide the Out message. But it is not "exchange" it is
rather replacing In with Out.
Now, on practical side, there is no need to provide both In and Out at
the same time. We can assume that component will return its answer in
the same object. But I'm failing to see any practical benefits. IMHO
it is the same, just matter of variable allocation.
I think Roman's (and mine) frustration comes not from the fact that
there are in/out but from the fact that they are too relaxed and it is
not clear when In is Out :) So stricter rules should address this
concern.
I have implemented a small message routing system for internal needs
at works and I used Soap idea of Envelope with properties and payload
- body. So I have no Exchange entity as such. All service information,
like number of faults, etc is in the envelope header. The processor
function itself is an exchange:
"public MessageEnvelope Process(MessageEnvelope env);"
Later I faced a need to implement a splitter which produces more then
one message so I modified interface to return array of messages:
"public MessageEnvelope[] Process(MessageEnvelope env);"
In Camel there is a special type of processor to handle this. In my
design any component can do broadcast by returning multiple messages.
You can do all the things you do in camel in/out exchange.
You can return null which would mean "no outputs, stop processing";
you can return exact the same object you received for logging for
example; you can create a copy of object, modify it and return; you
can create a bunch of new messages and return then.
In/Out notation is pretty much the same as described above but ability
to modify In message and leave Out null brings uncertainty. Should I
modify In or create Out? Why the heck In work as Out? Does componentX
use In or Out? etc.
I would think about making rules more strict: result is always in Out.
Providing many ways of doing the same thing without benefits is a bad
idea. You'll spend infinite amount of time explaining newbies what
in/out thing is. Principle of the least surprise :)
Another note about HTTP request/response.
I think it has nothing to do with Camel. Camel In/Out implements
conception of filter. You have a message asking to convert X $USD to
EUR. Your component use http to do it. So what the output of your
component is, http response? I doubt it. It is a message with EUR
amount. The whole http request-response is internal business of
component. Http request-response in an Exchange, when Camel Filter is
not.
So my opinion is:
1. Faults must be removed and exceptions used instead like any WS
framework does.
2. "In" must be In and Out must be Out, meaning In can not be modified
and Out *must* be set; no tricks "if out is null then interpret In as
Out". We should agree what to do with "null" out. Is it an error or it
means "no messages to process, stop the pipe"?
Vadim.
P.S. Great discussion btw.
Claus Ibsen wrote:
Hi
Just a quick mail to get the discussion rolling.
I think we should @deprecated exchange.fault in Camel 1.5 and remove
it in Camel 2.0.
It is a confusing concept that isn't used much, and many components
doesn't handle the FAULT message. For instance it was missing in
camel-jms a very central component.
Med venlig hilsen
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk
