sarankk commented on code in PR #238:
URL:
https://github.com/apache/cassandra-analytics/pull/238#discussion_r4107771952
##########
cassandra-six-zero-types/src/main/java/org/apache/cassandra/bridge/SchemaUpdater.java:
##########
@@ -72,6 +72,27 @@ public static void openKeyspaceInstances()
ClusterMetadata.current().schema.initializeKeyspaceInstances(DistributedSchema.empty(),
false);
}
+ /**
+ * Creates the keyspace instance for one keyspace, leaving every other
keyspace's instance untouched.
+ *
+ * <p>Prefer this to {@link #openKeyspaceInstances()} whenever the caller
knows which keyspace is missing its
+ * instance. That method diffs against {@link DistributedSchema#empty()},
so every keyspace in the process looks
+ * newly created and all of them are rebuilt. Rebuilding a keyspace that
already has an instance constructs its
+ * {@code KeyspaceMetrics} again, and each metric it re-registers throws
{@code IllegalArgumentException} out of
+ * Dropwizard's {@code MetricRegistry.register}. Filling in those stack
traces dominates the cost, and the cost
+ * grows with the number of keyspaces the process has seen, so a
long-lived JVM that builds schemas repeatedly -
+ * a Spark executor, or a single test class - degrades quadratically.
+ *
+ * <p>Diffing against the current keyspaces minus this one leaves exactly
one keyspace in
+ * {@code Keyspaces.diff().created}, so one instance is built and no
metric is registered twice.
Review Comment:
Because a bulk job currently deals with only 1 keyspace, this issue need not
show up in prod, could we update the comment to make it clear? Also shall we
remove `openKeyspaceInstances()` if possible
```suggestion
* Creates the keyspace instance for one keyspace, leaving every other
keyspace's instance untouched.
*
* <p>Prefer this to {@link #openKeyspaceInstances()} whenever the
caller knows which keyspace is missing its
* instance. That method diffs against {@link DistributedSchema#empty()}
causing every keyspace to be rebuilt again,
* which in turn rebuilds {@code KeyspaceMetrics} and registers each
metric, throwing
* {@code IllegalArgumentException}. A bulk job builds schema only once
per task, stack trace waste from
* exceptions thrown grows in integration tests which may open many
distinct keyspaces.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
