Matt/Hadrian, to your question about the distributed lock being "safe" when
things crash...it depends on the implementation, but
http://www.hazelcast.com/documentation.jsp#Lock Hazelcast claims that it is
designed to handle this scenario...
"Locks are fail-safe. If a member holds a lock and some of the members go
down, cluster will keep your locks safe and available. Moreover, when a
member leaves the cluster, all the locks acquired by this dead member will
be removed so that these locks can be available for live members
immediately. "
Overall, I don't think the concept of synchronizing/locking blocks of
code/routes via the DSL would be that confusing to any Java developer (since
they are core Java concepts). Though it may be fairly simple to do this in
a POJO processor/bean, so are loops and other EIPs that we provide APIs for.
Adding camel-hazelcast locking support seems to be an easier sell given that
we already support the other Hazelcast features and it doesn't affect the
DSL directly...
That being said, I agree that Camel shouldn't try to do everything. Seems
like a fine line between adding value vs. clutter given how much it already
does...
Matt Pavlovich wrote:
>
> Hadrian-
>
> I agree. This is the same conceptual problem NFS/GFS and other similar
> systems have. Anytime you implement a centralized locking, you always
> have to account for the clients that requested locks disappearing, or the
> thing that is centralizing the locks goes away. This is usually solved
> with some sort of timeout, but that can be problematic if the only thing
> lost was the heart beat link. What happens if data you thought wasn't
> being processed all of a sudden shows up? Do you cluster your centralized
> locking? At some point the complexity kills you more than anything.
>
> I'd say I'm generally against adding the locking to the DSL, since I think
> one of the goals should be to keep things easy for users by being
> defensive with really complicated concepts, like locking. I think the
> number of users that will mis-use locking will greatly out number the
> users that will use it correctly-- read: make Camel "look" broken and/or
> too hard to use. Also, I think that the benefit of making it "easy" for
> the users that know they need locking is very small, since they will
> probably know how to properly implement locking anyway. (aka the Ben
> O'Days of the world). I see this today with transactions.
>
> My $0.02
>
> Matt Pavlovich
>
> On Sep 7, 2011, at 11:20 AM, Hadrian Zbarcea wrote:
>
>> Ben, no issue with the term lock(), I get the semantics. What I don't get
>> is this: what happens in your scenario if one of the boxes crashes (say
>> physically, burnt memory chip, power supply explodes) with the lock
>> acquired. What do the other camel processes do?
>>
>> Why not just queue the requests?
>>
>> Hadrian
>>
>>
>> On 09/07/2011 12:10 PM, boday wrote:
>>> Christian, my use case is that I have multiple apps that update the same
>>> data
>>> (PurchaseOrders) and we periodically see some stale writes. We have a
>>> combination of Camel/CXF web services (load balanced across 2 machines)
>>> and
>>> batch processing in Camel routes (in a separate app) and are evaluating
>>> using Hazelcast distributed locking to make sure only one thread works
>>> on a
>>> PO at any given time.
>>>
>>> Its simple enough to just manually add the Hazelcast locking in as
>>> pre/post
>>> processors in routes, it just got me thinking about how it could fit
>>> into
>>> Camel natively (since we already have a Hazelcast component, etc). As
>>> suggested by others, maybe the term "lock" is confusing and we should
>>> term
>>> it "synchronized" or "exclusive" or something...
>>>
>>>
>>> Christian Schneider wrote:
>>>>
>>>> I am not sure if this is the same use case that I had some years ago.
>>>>
>>>> We wanted to have several instances of an integration running but only
>>>> one of them should be active at any time.
>>>> At the time there was no easy solution for this problem.
>>>>
>>>> So in that use case you would have several instances of camel each on
>>>> its own machine.
>>>> They could have the same route but only one should be active.
>>>>
>>>> In the dsl this could also be expressed with a lock or perhaps
>>>> exclusive
>>>> element.
>>>>
>>>> from("some source ").exclusive("some id")....
>>>>
>>>> So the question in this case is: What lib or backend would be suitable
>>>> to impement this and what parameters would we need?
>>>> Does Hazelcast support this scenario?
>>>>
>>>> Would we start and stop the route when it gets or looses the lock?
>>>>
>>>> Christian
>>>>
>>>> Am 06.09.2011 22:16, schrieb boday:
>>>>> hey guys, while working on a solution for a client, I ran into the
>>>>> need
>>>>> to
>>>>> "synchronize" multiple routes that modify the same data (users,
>>>>> orders,
>>>>> etc)
>>>>> to prevent collisions (stale writes, etc). Is there an existing way
>>>>> to
>>>>> do
>>>>> this that I've overlooked?
>>>>>
>>>>> In the past, I've relied on
>>>>> http://activemq.apache.org/message-groups.html
>>>>> AMQ message groups for JMS based routes and Hazelcast to span
>>>>> routes/containers. I feel this would be a good fit for the Camel DSL.
>>>>> Has
>>>>> anyone looked into this in the past?
>>>>>
>>>>> After doing some quick research, it seems like Java
>>>>> http://download.oracle.com/javase/6/docs/api/java/util/concurrent/locks/Lock.html
>>>>> Lock seems to be the way to go. Also, Hazelcast can extend this to
>>>>> be
>>>>> distributed across VMs.
>>>>>
>>>>> So, I've been playing around with adding distributed locking support
>>>>> with
>>>>> Hazelcast (see https://issues.apache.org/jira/browse/CAMEL-4397
>>>>> CAMEL-4397
>>>>> ) and wanted to get some feedback on doing this. In particular, I'm
>>>>> using a
>>>>> Lock instance stored in a ThreadLocal variable (to ensure that the
>>>>> same
>>>>> thread releases the lock). Does anyone see any issue with this
>>>>> approach?
>>>>>
>>>>> If all goes well with this, I was planning on also adding a DSL API to
>>>>> provide a basic support for (non distributed)
>>>>> http://download.oracle.com/javase/6/docs/api/java/util/concurrent/locks/ReentrantLock.html
>>>>> ReentrantLocks .
>>>>>
>>>>> something like this...
>>>>>
>>>>> from(route1).lock(userId).process(processor1).unlock();
>>>>> from(route2).lock(userId).process(processor2).unlock();
>>>>>
>>>>> Any advice/comments are welcome...thanks
>>>>>
>>>>>
>>>>> -----
>>>>> Ben O'Day
>>>>> IT Consultant -http://consulting-notes.com
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://camel.465427.n5.nabble.com/discuss-implementing-Locks-in-Camel-tp4775904p4775904.html
>>>>> Sent from the Camel Development mailing list archive at Nabble.com.
>>>>>
>>>>
>>>> --
>>>> Christian Schneider
>>>> http://www.liquid-reality.de
>>>>
>>>> Open Source Architect
>>>> http://www.talend.com
>>>>
>>>
>>>
>>> -----
>>> Ben O'Day
>>> IT Consultant -http://consulting-notes.com
>>>
>>> --
>>> View this message in context:
>>> http://camel.465427.n5.nabble.com/discuss-implementing-Locks-in-Camel-tp4775904p4779134.html
>>> Sent from the Camel Development mailing list archive at Nabble.com.
>
-----
Ben O'Day
IT Consultant -http://consulting-notes.com
--
View this message in context:
http://camel.465427.n5.nabble.com/discuss-implementing-Locks-in-Camel-tp4775904p4779876.html
Sent from the Camel Development mailing list archive at Nabble.com.
