tolbertam commented on code in PR #3917:
URL: https://github.com/apache/cassandra/pull/3917#discussion_r2045643799
##########
src/java/org/apache/cassandra/db/SystemKeyspace.java:
##########
@@ -1863,16 +1864,38 @@ public static void resetPreparedStatements()
}
public static int loadPreparedStatements(TriFunction<MD5Digest, String,
String, Boolean> onLoaded)
+ {
+ return loadPreparedStatements(onLoaded,
QueryProcessor.PRELOAD_PREPARED_STATEMENTS_FETCH_SIZE);
+ }
+
+ public static int loadPreparedStatements(TriFunction<MD5Digest, String,
String, Boolean> onLoaded, int pageSize)
{
String query = String.format("SELECT prepared_id, logged_keyspace,
query_string FROM %s.%s", SchemaConstants.SYSTEM_KEYSPACE_NAME,
PREPARED_STATEMENTS);
- UntypedResultSet resultSet = executeOnceInternal(query);
+ UntypedResultSet resultSet = executeOnceInternalWithPaging(query,
pageSize);
int counter = 0;
+ long initialStatementsEvicted =
QueryProcessor.metrics.preparedStatementsEvicted.getCount();
for (UntypedResultSet.Row row : resultSet)
{
if
(onLoaded.accept(MD5Digest.wrap(row.getByteArray("prepared_id")),
row.getString("query_string"),
row.has("logged_keyspace") ?
row.getString("logged_keyspace") : null))
counter++;
+
+ if (counter % pageSize == 0
+ && QueryProcessor.metrics.preparedStatementsEvicted.getCount()
- initialStatementsEvicted > 0)
Review Comment:
Pushed something that I think this should be sufficient, but do have a small
inkling of doubt.
We could consider adding a 5% or so buffer as you suggested, but by
under-compensating (e.g. `currentSize >= maximumSize * 0.95`) so we can handle
the case of the cache being evicted right below it's capacity.
I suspect if we page through leaked statements we will eventually hit a case
where the cache is briefly above its capacity and it will cut off.
If we do add an under-compensation, it's possible that we will erroneously
stop and log a warning which may not be correct. On the other hand we could
always adjust that warning to indicate that it's possible we stopped because
the cache was close to full.
In the usual case, the prepared statement cache should not really approach
close to getting full unless non-fully qualified statements are being prepared
anyways, so it might be useful to warn in that case as well ("Detected that the
prepared statement cache was close to full on preload, please investigate
whether your statements being prepared are fully qualified"). Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
