[
https://issues.apache.org/jira/browse/PHOENIX-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271095#comment-16271095
]
Josh Elser commented on PHOENIX-4413:
-------------------------------------
Nope, I'd need some more information from [~alexaraujo] to better characterize
the performance and/or memory impact. Some general suggestions:
* Investigate tweaking PQS JVM properties. You may be hitting issues simply due
to the impact of the GC.
* Use jmap or similar to inspect the contents of the JVM heap to understand if
the number of live objects (especially connections and statements) are growing
unboundedly
* Turn on DEBUG logging in PQS (I think). It might be at the TRACE level, but I
know that there is logging when the server evicts connections/statements
automatically
* Use your profiler of choice (e.g. yourkit, java flight recorder) to better
quantify what changes in terms of performance.
Alternatively, if you are a C# felllow and not a Java person, encapsulating
your scenario into something that we can reproduce easily is helpful. I've done
a little bit of work in Avatica around self-contained docker images which might
help isolate the problem: https://calcite.apache.org/avatica/docs/docker.html.
If there is a problem lurking, it's unlikely to be only seen with Phoenix.
Even in the case of misbehaving clients, PQS will clean up idle connections and
statements. see https://phoenix.apache.org/server.html, "Configurations
relating to the server connection cache." and "Configurations relating to the
server statement cache.".
> Possible queryserver memory leak when reusing connections and statements
> ------------------------------------------------------------------------
>
> Key: PHOENIX-4413
> URL: https://issues.apache.org/jira/browse/PHOENIX-4413
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.12.0
> Reporter: Alex Araujo
>
> While testing client-side connection pooling using the [C# client from
> Microsoft|https://github.com/Azure/hdinsight-phoenix-sharp], we attempted to
> avoid creating new connections and statements for every Phoenix statement
> execution (essentially just a simple SELECT for single key). The results
> were very positive from a performance perspective. However, after a certain
> amount of statements executed in this manner, memory on the PQS appears to
> spike, and performance degrades significantly.
> Steps to Recreate
> Setup
> 1) Create the table: "CREATE TABLE <TableName> (TestKey varchar(255) PRIMARY
> KEY, TestValue varchar(10000))".
> 2) Populate the table with 100 random TestKey and TestValue records.
> Execution (if done with one thread, this can take up to 24 hours, so we
> multithreaded it)
> 1) Create connection using OpenConnectionRequestAsync.
> 2) Create statement using CreateStatementRequestAsync.
> 3) Loop n times, selecting a record with a single random key: "SELECT
> TestKey, TestValue FROM <TableName> WHERE TestKey = <TestKey>'' issued using
> PrepareAndExecuteRequestAsync.
> 4) Close statement.
> 5) Close connection.
> Teardown
> 1) Drop the table.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
