Hadrian-
I agree. This is the same conceptual problem NFS/GFS and other similar systems
have. Anytime you implement a centralized locking, you always have to account
for the clients that requested locks disappearing, or the thing that is
centralizing the locks goes away. This is usually solved with some sort of
timeout, but that can be problematic if the only thing lost was the heart beat
link. What happens if data you thought wasn't being processed all of a sudden
shows up? Do you cluster your centralized locking? At some point the
complexity kills you more than anything.
I'd say I'm generally against adding the locking to the DSL, since I think one
of the goals should be to keep things easy for users by being defensive with
really complicated concepts, like locking. I think the number of users that
will mis-use locking will greatly out number the users that will use it
correctly-- read: make Camel "look" broken and/or too hard to use. Also, I
think that the benefit of making it "easy" for the users that know they need
locking is very small, since they will probably know how to properly implement
locking anyway. (aka the Ben O'Days of the world). I see this today with
transactions.
My $0.02
Matt Pavlovich
On Sep 7, 2011, at 11:20 AM, Hadrian Zbarcea wrote:
> Ben, no issue with the term lock(), I get the semantics. What I don't get is
> this: what happens in your scenario if one of the boxes crashes (say
> physically, burnt memory chip, power supply explodes) with the lock acquired.
> What do the other camel processes do?
>
> Why not just queue the requests?
>
> Hadrian
>
>
> On 09/07/2011 12:10 PM, boday wrote:
>> Christian, my use case is that I have multiple apps that update the same data
>> (PurchaseOrders) and we periodically see some stale writes. We have a
>> combination of Camel/CXF web services (load balanced across 2 machines) and
>> batch processing in Camel routes (in a separate app) and are evaluating
>> using Hazelcast distributed locking to make sure only one thread works on a
>> PO at any given time.
>>
>> Its simple enough to just manually add the Hazelcast locking in as pre/post
>> processors in routes, it just got me thinking about how it could fit into
>> Camel natively (since we already have a Hazelcast component, etc). As
>> suggested by others, maybe the term "lock" is confusing and we should term
>> it "synchronized" or "exclusive" or something...
>>
>>
>> Christian Schneider wrote:
>>>
>>> I am not sure if this is the same use case that I had some years ago.
>>>
>>> We wanted to have several instances of an integration running but only
>>> one of them should be active at any time.
>>> At the time there was no easy solution for this problem.
>>>
>>> So in that use case you would have several instances of camel each on
>>> its own machine.
>>> They could have the same route but only one should be active.
>>>
>>> In the dsl this could also be expressed with a lock or perhaps exclusive
>>> element.
>>>
>>> from("some source ").exclusive("some id")....
>>>
>>> So the question in this case is: What lib or backend would be suitable
>>> to impement this and what parameters would we need?
>>> Does Hazelcast support this scenario?
>>>
>>> Would we start and stop the route when it gets or looses the lock?
>>>
>>> Christian
>>>
>>> Am 06.09.2011 22:16, schrieb boday:
>>>> hey guys, while working on a solution for a client, I ran into the need
>>>> to
>>>> "synchronize" multiple routes that modify the same data (users, orders,
>>>> etc)
>>>> to prevent collisions (stale writes, etc). Is there an existing way to
>>>> do
>>>> this that I've overlooked?
>>>>
>>>> In the past, I've relied on
>>>> http://activemq.apache.org/message-groups.html
>>>> AMQ message groups for JMS based routes and Hazelcast to span
>>>> routes/containers. I feel this would be a good fit for the Camel DSL.
>>>> Has
>>>> anyone looked into this in the past?
>>>>
>>>> After doing some quick research, it seems like Java
>>>> http://download.oracle.com/javase/6/docs/api/java/util/concurrent/locks/Lock.html
>>>> Lock seems to be the way to go. Also, Hazelcast can extend this to be
>>>> distributed across VMs.
>>>>
>>>> So, I've been playing around with adding distributed locking support with
>>>> Hazelcast (see https://issues.apache.org/jira/browse/CAMEL-4397
>>>> CAMEL-4397
>>>> ) and wanted to get some feedback on doing this. In particular, I'm
>>>> using a
>>>> Lock instance stored in a ThreadLocal variable (to ensure that the same
>>>> thread releases the lock). Does anyone see any issue with this approach?
>>>>
>>>> If all goes well with this, I was planning on also adding a DSL API to
>>>> provide a basic support for (non distributed)
>>>> http://download.oracle.com/javase/6/docs/api/java/util/concurrent/locks/ReentrantLock.html
>>>> ReentrantLocks .
>>>>
>>>> something like this...
>>>>
>>>> from(route1).lock(userId).process(processor1).unlock();
>>>> from(route2).lock(userId).process(processor2).unlock();
>>>>
>>>> Any advice/comments are welcome...thanks
>>>>
>>>>
>>>> -----
>>>> Ben O'Day
>>>> IT Consultant -http://consulting-notes.com
>>>>
>>>> --
>>>> View this message in context:
>>>> http://camel.465427.n5.nabble.com/discuss-implementing-Locks-in-Camel-tp4775904p4775904.html
>>>> Sent from the Camel Development mailing list archive at Nabble.com.
>>>>
>>>
>>> --
>>> Christian Schneider
>>> http://www.liquid-reality.de
>>>
>>> Open Source Architect
>>> http://www.talend.com
>>>
>>
>>
>> -----
>> Ben O'Day
>> IT Consultant -http://consulting-notes.com
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/discuss-implementing-Locks-in-Camel-tp4775904p4779134.html
>> Sent from the Camel Development mailing list archive at Nabble.com.
