splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r521856417
########## File path: core/src/main/scala/kafka/network/SocketServer.scala ########## @@ -1324,7 +1401,59 @@ class ConnectionQuotas(config: KafkaConfig, time: Time, metrics: Metrics) extend private[network] def updateBrokerMaxConnectionRate(maxConnectionRate: Int): Unit = { // if there is a connection waiting on the rate throttle delay, we will let it wait the original delay even if // the rate limit increases, because it is just one connection per listener and the code is simpler that way - updateConnectionRateQuota(maxConnectionRate) + updateConnectionRateQuota(maxConnectionRate, BrokerQuotaEntity) + } + + /** + * Update the connection rate quota for a given IP and updates quota configs for updated IPs. + * If an IP is given, metric config will be updated only for the given IP, otherwise + * all metric configs will be checked and updated if required. + * + * @param ip ip to update or default if None + * @param maxConnectionRate new connection rate, or resets entity to default if None + */ + def updateIpConnectionRateQuota(ip: Option[String], maxConnectionRate: Option[Int]): Unit = { + def isIpConnectionRateMetric(metricName: MetricName) = { + metricName.name == ConnectionRateMetricName && + metricName.group == MetricsGroup && + metricName.tags.containsKey(IpMetricTag) + } + + def shouldUpdateQuota(metric: KafkaMetric, quotaLimit: Int) = { + quotaLimit != metric.config.quota.bound + } + counts.synchronized { Review comment: After thinking on this a bit more, I think that only locking on updates to `defaultConnectionRatePerIp` should be sufficient for correctness. ZK dynamic config changes are processed within one thread, so we will only have one thread executing in `updateIpConnectionRateQuota`. If we want to be really careful about this, we can have `updateIpConnectionRateQuota` synchronized on `ConnectionQuotas`. The case we want to avoid by synchronizing on `counts` is that another thread `defaultConnectionRatePerIp` as connection rate limit `A` while calling `inc()`, then we update connection rate and quota metrics to `B`, then the other thread creates a sensor/metric with quota limit `A` => inconsistency. If we synchronize on `counts` for only updates to `connectionRateForIp/defaultConnectionRate`, we know that any thread that has read a connection rate quota as `A` will finish creating quota metrics with limit `A` before we acquire the `counts` lock. After acquiring the `counts` lock and updating `connectionRateForIp/defaultConnectionRate` to `B`, any subsequent threads calling `inc()` will read the quota as `B` and create a metric as `B`. We will then be able to update any quota metrics from `A` to `B`, knowing that there are no operations that could have read the connection rate limit as `A` without already having created the quota metric. The only issue remaining is that we can get concurrent reads of `connectionRatePerIp` while updating quota metrics, but we can just replace `mutable.Map` with `ConcurrentHashMap` which is preferable to coarsely locking on `counts`. Let me know if I'm missing something here with respect to thread safety. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org