Re: Releasing latches when waiting for locks. When and why?

2006-12-12 Thread Knut Anders Hatlen
Olav Sandstaa <[EMAIL PROTECTED]> writes: > Knut Anders Hatlen wrote: [...] > This is impressive, and much higher than what I would have expected by > just moving the latching out of the lock manager. > >> I would guess that the improvement is mainly caused by >> >> a) Less contention on the loc

Re: Releasing latches when waiting for locks. When and why?

2006-12-12 Thread Knut Anders Hatlen
Suresh Thalamati <[EMAIL PROTECTED]> writes: > If we are removing Latching out of LockManager for perfromance > reasons, my preference also would be to handle at the page level, > instead of having yet another indirection. > > As Mike mentioned , it would be nice to have a VTI that will show the

Re: Releasing latches when waiting for locks. When and why?

2006-12-12 Thread Knut Anders Hatlen
Anders Morken <[EMAIL PROTECTED]> writes: > While I'm not an expert on the subject matter, I can't help myself but > chime in and share my thoughts on this... =) > > During the course of our project on analyzing Derby's SMP scalability > properties, we've investigated Derby's locking and latching

Re: Releasing latches when waiting for locks. When and why?

2006-12-11 Thread Olav Sandstaa
Knut Anders Hatlen wrote: To see the effect of this change, I tested the patch on a dual-CPU machine with the test client from DERBY-1961 running single-record select operations. Derby was running in embedded mode, and the entire database was in the page cache. The results for 1 to 100 concurrent

Re: Releasing latches when waiting for locks. When and why?

2006-12-08 Thread Suresh Thalamati
Mike Matrigali wrote: Daniel John Debrunner wrote: Knut Anders Hatlen wrote: Mike Matrigali <[EMAIL PROTECTED]> writes: < snip .? I'm not so sure. If it's simpler to keep page latching at the page level, then it would be better to remove the added complexity from the lock manager. Wha

Re: Releasing latches when waiting for locks. When and why?

2006-12-08 Thread Anders Morken
While I'm not an expert on the subject matter, I can't help myself but chime in and share my thoughts on this... =) During the course of our project on analyzing Derby's SMP scalability properties, we've investigated Derby's locking and latching pretty thoroughly. We've followed this thread with

Re: Releasing latches when waiting for locks. When and why?

2006-12-07 Thread Knut Anders Hatlen
Olav Sandstaa <[EMAIL PROTECTED]> writes: > Knut Anders Hatlen wrote: >> Mike Matrigali <[EMAIL PROTECTED]> writes: >> >> >>> One approach would be to continue to support the latching in the >>> lockmanager, but to provide an alternate implementation using the >>> same interfaces (or new interf

Re: Releasing latches when waiting for locks. When and why?

2006-12-06 Thread Olav Sandstaa
Knut Anders Hatlen wrote: Mike Matrigali <[EMAIL PROTECTED]> writes: One approach would be to continue to support the latching in the lockmanager, but to provide an alternate implementation using the same interfaces (or new interfaces if necessary). Then make the default module implementati

Re: Releasing latches when waiting for locks. When and why?

2006-11-27 Thread Mike Matrigali
Daniel John Debrunner wrote: Knut Anders Hatlen wrote: Mike Matrigali <[EMAIL PROTECTED]> writes: One approach would be to continue to support the latching in the lockmanager, but to provide an alternate implementation using the same interfaces (or new interfaces if necessary). Then make t

Re: Releasing latches when waiting for locks. When and why?

2006-11-27 Thread Daniel John Debrunner
Knut Anders Hatlen wrote: Mike Matrigali <[EMAIL PROTECTED]> writes: One approach would be to continue to support the latching in the lockmanager, but to provide an alternate implementation using the same interfaces (or new interfaces if necessary). Then make the default module implementation

Re: Releasing latches when waiting for locks. When and why?

2006-11-27 Thread Knut Anders Hatlen
Mike Matrigali <[EMAIL PROTECTED]> writes: > One approach would be to continue to support the latching in the > lockmanager, but to provide an alternate implementation using the > same interfaces (or new interfaces if necessary). Then make the > default module implementation be the new improved o

Re: Releasing latches when waiting for locks. When and why?

2006-11-27 Thread Mike Matrigali
One approach would be to continue to support the latching in the lockmanager, but to provide an alternate implementation using the same interfaces (or new interfaces if necessary). Then make the default module implementation be the new improved one, but if a deadlock is suspected run under the ex

Re: Releasing latches when waiting for locks. When and why?

2006-11-22 Thread Dyre . Tjeldvoll
Knut Anders Hatlen <[EMAIL PROTECTED]> writes: > Daniel John Debrunner <[EMAIL PROTECTED]> writes: > >> Knut Anders Hatlen wrote: >>> Mike Matrigali <[EMAIL PROTECTED]> writes: >>> Having said that it would be interesting if someone had time to implement a higher performance latch implem

Re: Releasing latches when waiting for locks. When and why?

2006-11-22 Thread Knut Anders Hatlen
Daniel John Debrunner <[EMAIL PROTECTED]> writes: > Knut Anders Hatlen wrote: >> Mike Matrigali <[EMAIL PROTECTED]> writes: >> >>> Having said that it would be interesting if someone had time to >>> implement a higher performance latch implementation and plug it in >>> and see how much it helps.

Re: Releasing latches when waiting for locks. When and why?

2006-11-21 Thread Dyre . Tjeldvoll
Anders Morken <[EMAIL PROTECTED]> writes: > We've taken some long hard looks at the lock manager, and we're > convinced that there's a lot of potential for improvement in Derby lock > management. Not that there's anything wrong with the approach taken in > the current implementation - correctness

Re: Releasing latches when waiting for locks. When and why?

2006-11-21 Thread Knut Anders Hatlen
[EMAIL PROTECTED] writes: > Anders Morken <[EMAIL PROTECTED]> writes: > >> We've taken some long hard looks at the lock manager, and we're >> convinced that there's a lot of potential for improvement in Derby lock >> management. Not that there's anything wrong with the approach taken in >> the cur

Re: Releasing latches when waiting for locks. When and why?

2006-11-21 Thread Daniel John Debrunner
Knut Anders Hatlen wrote: Mike Matrigali <[EMAIL PROTECTED]> writes: Having said that it would be interesting if someone had time to implement a higher performance latch implementation and plug it in and see how much it helps. It would decrease the total time spent in lock manager. Ok, a new

Re: Releasing latches when waiting for locks. When and why?

2006-11-21 Thread Knut Anders Hatlen
Mike Matrigali <[EMAIL PROTECTED]> writes: > Having said that it would be interesting if someone had time to > implement a higher performance latch implementation and plug it in > and see how much it helps. It would decrease the total time spent > in lock manager. Ok, a new experiment: I removed

Re: Releasing latches when waiting for locks. When and why?

2006-11-21 Thread Mike Matrigali
Knut Anders Hatlen wrote: Thanks Bryan, that could be the case. I have done some more investigation and it seems like the method is only called when a B-tree container is opened with a lock policy. It also seems like the B-tree containers always are opened with a null/NoLocking lock policy (r

Re: Releasing latches when waiting for locks. When and why?

2006-11-21 Thread Knut Anders Hatlen
Bryan Pendleton <[EMAIL PROTECTED]> writes: >> When this method is used, the specified latch will be >> released if the lock cannot be obtained immediately, and that page >> will be re-latched when the lock has been obtained. > > I'm just guessing here ... > > I think that there are some fairly ad

Re: Releasing latches when waiting for locks. When and why?

2006-11-21 Thread Anders Morken
Mike Matrigali: > by single row update do you mean (obviously not exact syntax): > update x = y where key = z This one, yeah. ~100-byte rows, INTEGER primary keys, we update the filler only. UPDATE thetable SET filler = WHERE key = ; > In the cursor case derby maintains an "internal" lock on the

Re: Releasing latches when waiting for locks. When and why?

2006-11-21 Thread Dyre . Tjeldvoll
Knut Anders Hatlen <[EMAIL PROTECTED]> writes: > For one client, there was not much gained, but for two clients, the > throughput increased 20% compared to trunk. For three clients, the > increase was 40%, and it was 145% for 30 clients. This was a lot more > than I expected! I also ran a TPC-B li

Re: Releasing latches when waiting for locks. When and why?

2006-11-21 Thread Dyre . Tjeldvoll
Anders Morken <[EMAIL PROTECTED]> writes: > The major benefit of reworking the lock manager is scalability - right > now it's pretty much single threaded, which is fine for a lot of the > scenarios Derby was written to perform in. However, for "benchmark > compliance" in a world of multicore deskt

Re: Releasing latches when waiting for locks. When and why?

2006-11-17 Thread Bryan Pendleton
When this method is used, the specified latch will be released if the lock cannot be obtained immediately, and that page will be re-latched when the lock has been obtained. I'm just guessing here ... I think that there are some fairly advanced Btree traversal algorithms, in particular situation

Releasing latches when waiting for locks. When and why?

2006-11-17 Thread Knut Anders Hatlen
Hi, I have come across some use of locks and latches which I don't understand. One of the lockObject() methods in the lock manager (SinglePool.lockObject(Object, Lockable, Object, int, Latch)) takes a Latch parameter. When this method is used, the specified latch will be released if the lock canno