[
https://issues.apache.org/jira/browse/PHOENIX-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16273323#comment-16273323
]
Josh Elser commented on PHOENIX-4413:
-------------------------------------
bq. Basic GC tuning was performed; GC does not appear to be a factor
Good :) Covering all of the bases here. Not everyone realizes this ;)
bq. Could reuse of statementIDs cause this? Pooling connections and statements
is required for the use case in question as it is very sensitive to latency.
I see no reason. Using the same StatementID and ConnectionID are analogous to a
JDBC application which would use the same Statement and Connection instance
(it's a loose mapping in PQS from the ID to the Java Object).
bq. Unbounded memory growth is not observed when the number of statements
executed using the same statementID is kept below a threshold (500 in their
test environment)
This one makes me turn my head again in confusion. How are you (or your team)
determining the difference between "require memory usage" and "unbounded
growth"? {{jmap}} (with either {{-histo}} or {{-dump}}) would be great tools to
actually understand what the memory usage is.
> Possible queryserver memory leak when reusing connections and statements
> ------------------------------------------------------------------------
>
> Key: PHOENIX-4413
> URL: https://issues.apache.org/jira/browse/PHOENIX-4413
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.12.0, 4.13.0
> Reporter: Alex Araujo
>
> While testing client-side connection pooling using the [C# client from
> Microsoft|https://github.com/Azure/hdinsight-phoenix-sharp], we attempted to
> avoid creating new connections and statements for every Phoenix statement
> execution (essentially just a simple SELECT for single key). The results
> were very positive from a performance perspective. However, after a certain
> amount of statements executed in this manner, memory on the PQS appears to
> spike, and performance degrades significantly.
> Steps to Recreate
> Setup
> 1) Create the table: "CREATE TABLE <TableName> (TestKey varchar(255) PRIMARY
> KEY, TestValue varchar(10000))".
> 2) Populate the table with 100 random TestKey and TestValue records.
> Execution (if done with one thread, this can take up to 24 hours, so we
> multithreaded it)
> 1) Create connection using OpenConnectionRequestAsync.
> 2) Create statement using CreateStatementRequestAsync.
> 3) Loop n times, selecting a record with a single random key: "SELECT
> TestKey, TestValue FROM <TableName> WHERE TestKey = <TestKey>'' issued using
> PrepareAndExecuteRequestAsync.
> 4) Close statement.
> 5) Close connection.
> Teardown
> 1) Drop the table.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
