[
https://issues.apache.org/jira/browse/DERBY-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12522495
]
Knut Anders Hatlen commented on DERBY-3024:
-------------------------------------------
Thanks for investigating this, Manish!
I agree that the test is not representative of a real-world application, but
that wasn't my aim when I wrote it. I just wanted to see if there were any
basic part of the SQL execution layer that would be a bottleneck on a multi-CPU
machine. VALUES 1 seemed to be a good choice since it avoids accesses to the
buffer manager, which is a known multi-CPU bottleneck. I think of it more like
looking at a small part of Derby through a magnifying glass or a microscope. :)
When I run the test, I only see three calls to GPS.upToDate(), one call to
BA.checkStatementValidity(), and none to GPS.finish() and GPS.getActivation().
You didn't by any chance use a Statement instead of a PreparedStatement?
I'm not sure I quite understand how the interaction with upToDate() works. If
upToDate() returns true, we know (because of the synchronization) that at some
point after we called upToDate() and before it returned, the compiled plan was
up to date. However, the synchronization doesn't guarantee that the plan is up
to date the moment after the method has returned, does it? How do we know the
plan is still valid then? Is it because of the uncertainty we keep calling
upToDate() multiple times during execution?
> Validation of shared plans hurts scalability
> --------------------------------------------
>
> Key: DERBY-3024
> URL: https://issues.apache.org/jira/browse/DERBY-3024
> Project: Derby
> Issue Type: Improvement
> Components: Performance, SQL
> Affects Versions: 10.4.0.0
> Environment: Sun Java SE 6, Solaris 10, Sun Fire V880 (8 CPUs)
> Reporter: Knut Anders Hatlen
> Priority: Minor
> Attachments: Values.java, values1.png
>
>
> To investigate whether there was anything in the SQL execution layer that
> prevented scaling on a multi-CPU machine, I wrote a multi-threaded test which
> continuously executed "VALUES 1" using a PreparedStatement. I ran the test on
> a machine with 8 CPUs and expected the throughput to be proportional to the
> number of concurrent clients up to 8 clients (the same as the number of
> CPUs). However, the throughput only had a small increase from 1 to 2 clients,
> and adding more clients did not increase the throughput. Looking at the test
> in a profiler, it seems like the threads are spending a lot of time waiting
> to enter synchronization blocks in GenericPreparedStatement.upToDate() and
> BaseActivation.checkStatementValidity() (both of which are synchronized on
> the a GenericPreparedStatement object).
> I then changed the test slightly, appending a comment with a unique thread id
> to the "VALUES 1" statement. That means the threads still did the same work,
> but each thread got its own plan (GenericPreparedStatement object) since the
> statement cache didn't regard the SQL text strings as identical. When I made
> that change, the test scaled more or less perfectly up to 8 concurrent
> threads.
> We should try to find a way to make the scalability the same regardless of
> whether or not the threads share the same plan.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
