Hi Alvaro,
Beam is big data processing whereas Camel is messaging/integration
framework.
One of key difference is in Camel you have InOut message pattern. It
means that, on completion of a Camel Exchange, you can return (using the
out message) something to the client (like a REST client).
The Camel Exchange only contains a In (and optionally Out) message
containing the message body and headers.
This Exchange is processed on a unique route, local to one JVM, each
Exchange is "atomic".
In Beam, the PCollection contains bunch of data (elements), distributed
on the executed (that's why the elements in the PCollection have to be
serializable). The processing of the elements (with ParDo) is also
spread of the cluster. Beam is really made for large scale, lot of data
processing. Camel is more for message (element) routing (with EIP),
preparation and storage to a backend.
So, Camel is a great way to get your data in your Kafka topic, or your
data repository (with optionally some transformation/preparation of the
data, especially data format transformation).
Then, once your data repository or Kafka topic has been populated by
Camel, on large scale, Beam can be used to analyze the data (lile fraud
detection, user sentiment analysis, ...).
Let me know if you need some more details !
Regards
JB
On 01/26/2017 09:37 AM, Alvaro Cabrerizo wrote:
Hello,
I have been searching information about the difference between Apache
Beam and Apache Camel
<http://www.jesse-anderson.com/2016/07/question-and-answers-with-the-apache-beam-team/>,
but haven't found much of it. Moreover I would like to know when I
should use one or the other. If the decision is a matter of data volume
or there are other elements that should be taken into account.
Regards.
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
