deepthi912 commented on code in PR #20118:
URL: https://github.com/apache/pinot/pull/20118#discussion_r4531067494


##########
pinot-controller/src/main/java/org/apache/pinot/controller/validation/RealtimeSegmentValidationManager.java:
##########
@@ -195,32 +198,79 @@ boolean shouldEnsureConsuming(String tableNameWithType) {
     }
     _controllerMetrics.setOrUpdateTableGauge(tableNameWithType, 
ControllerGauge.RESOURCE_UTILIZATION_LIMIT_EXCEEDED,
         0L);
-    // Perform storage quota checks.
+    // Perform storage and primary key quota checks. Both pause realtime 
consumption on breach but share a single
+    // IdealState pause flag, so the logic below is pause-reason-aware: it 
never resumes a table paused for one reason
+    // just because the other reason is clear, and it resumes only when ALL 
quota reasons are within limits.
     TableConfig tableConfig = 
_pinotHelixResourceManager.getTableConfig(tableNameWithType);
-    boolean isQuotaExceeded = 
_storageQuotaChecker.isTableStorageQuotaExceeded(tableConfig);
-    QuotaEnforcementMode enforcementMode = 
_storageQuotaChecker.getEnforcementMode(tableConfig);
-    if (isQuotaExceeded && enforcementMode == QuotaEnforcementMode.SHADOW) {
+
+    // Only an enforced breach pauses consumption; a breach in SHADOW or 
DISABLED mode is treated as within quota.
+    boolean isStorageQuotaBreached = 
_storageQuotaChecker.isTableStorageQuotaExceeded(tableConfig);
+    QuotaEnforcementMode storageMode = 
_storageQuotaChecker.getEnforcementMode(tableConfig);
+    if (isStorageQuotaBreached && storageMode == QuotaEnforcementMode.SHADOW) {
       // The breach meter is already emitted by StorageQuotaChecker
       LOGGER.warn("Storage quota exceeded for table: {}, but enforcement mode 
is SHADOW, not pausing consumption",
           tableNameWithType);
     }
-    // Only an enforced breach pauses consumption; a breach in SHADOW or 
DISABLED mode is treated as within quota
-    boolean shouldPauseForQuota = isQuotaExceeded && enforcementMode == 
QuotaEnforcementMode.ENFORCED;
-    if (shouldPauseForQuota == isTablePaused) {
-      return !isTablePaused;
+    boolean storageExceeded = isStorageQuotaBreached && storageMode == 
QuotaEnforcementMode.ENFORCED;
+
+    // Primary key quota: same gating. UNDETERMINED (counts could not be read) 
is handled below.
+    PrimaryKeyQuotaChecker.Status pkStatus = 
_primaryKeyQuotaChecker.checkPrimaryKeyQuota(tableConfig);
+    QuotaEnforcementMode pkMode = 
_primaryKeyQuotaChecker.getEnforcementMode(tableConfig);
+    boolean isPkQuotaBreached = pkStatus == 
PrimaryKeyQuotaChecker.Status.EXCEEDED;
+    if (isPkQuotaBreached && pkMode == QuotaEnforcementMode.SHADOW) {
+      // The breach meter is already emitted by PrimaryKeyQuotaChecker
+      LOGGER.warn("Primary key quota exceeded for table: {}, but enforcement 
mode is SHADOW, not pausing consumption",
+          tableNameWithType);
     }
-    // if quota breach and pause flag is not in sync, update the IS
-    if (shouldPauseForQuota) {
-      String storageQuota = tableConfig.getQuotaConfig() != null ? 
tableConfig.getQuotaConfig().getStorage() : "NA";
-      // as quota is breached pause the consumption right away
-      _llcRealtimeSegmentManager.pauseConsumption(tableNameWithType, 
PauseState.ReasonCode.STORAGE_QUOTA_EXCEEDED,
-          "Storage quota of " + storageQuota + " exceeded.");
-    } else {
-      // as quota limit is being honored, unset the pause state and allow 
consuming segment recreation.
-      
_llcRealtimeSegmentManager.updatePauseStateInIdealState(tableNameWithType, 
false,
-          PauseState.ReasonCode.STORAGE_QUOTA_EXCEEDED, "Table storage within 
quota limits");
+    boolean pkExceeded = isPkQuotaBreached && pkMode == 
QuotaEnforcementMode.ENFORCED;
+
+    // By this point the table is either not paused, or paused for a quota 
reason (ADMINISTRATIVE returned early and
+    // RESOURCE_UTILIZATION was fully resolved above).
+    PauseState.ReasonCode currentReason = isTablePaused ? 
pauseStatus.getReasonCode() : null;
+    boolean pausedByQuota = currentReason == 
PauseState.ReasonCode.STORAGE_QUOTA_EXCEEDED
+        || currentReason == PauseState.ReasonCode.PRIMARY_KEY_QUOTA_EXCEEDED;
+    // Storage takes precedence over primary key for the single stored pause 
reason when both are exceeded.
+    PauseState.ReasonCode desiredReason = storageExceeded ? 
PauseState.ReasonCode.STORAGE_QUOTA_EXCEEDED
+        : pkExceeded ? PauseState.ReasonCode.PRIMARY_KEY_QUOTA_EXCEEDED : null;
+
+    if (desiredReason == null) {
+      // No quota is effectively exceeded. Don't resume a PK-paused table when 
the PK status couldn't be determined.
+      if (currentReason == PauseState.ReasonCode.PRIMARY_KEY_QUOTA_EXCEEDED
+          && pkStatus == PrimaryKeyQuotaChecker.Status.UNDETERMINED) {

Review Comment:
   This guard only covers PK-labelled pauses. If a table is paused for storage 
while also over the PK cap, and storage clears while PK counts are unreadable, 
the table resumes. The next run re-pauses it and force-commits again.
   
   Suggest holding whenever an enforced PK quota is undetermined:
   
   ```java
   if (pausedByQuota && pkMode == QuotaEnforcementMode.ENFORCED
       && pkStatus == PrimaryKeyQuotaChecker.Status.UNDETERMINED) {
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to