noob-se7en commented on code in PR #15405:
URL: https://github.com/apache/pinot/pull/15405#discussion_r2020090409
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/ConsumerCoordinator.java:
##########
@@ -71,39 +70,34 @@ public ConsumerCoordinator(boolean
enforceConsumptionInOrder, RealtimeTableDataM
_serverMetrics = ServerMetrics.get();
}
+ /**
+ * Acquires the consumer semaphore for the given LLC segment. When
consumption order is enforced, it waits for the
+ * previous segment to be registered.
+ *
+ * TODO: Revisit if we want to handle the following corner case:
+ * - Seg 100 (OFFLINE -> CONSUMING pending)
+ * - Seg 101 (OFFLINE -> CONSUMING returned because of status change)
+ * - Seg 101 (CONSUMING -> ONLINE processed)
+ * - Seg 102 (OFFLINE -> CONSUMING started consuming while 100 is not
registered)
+ * It should be extremely rare because there has to be multiple CONSUMING
state transitions pending, which means
+ * state transition for Seg 100 has been delayed for at least the
consumption time of Seg 101. Since we are not
+ * blocking the state transition of OFFLINE -> CONSUMING, the chance of
this happening is very low.
Review Comment:
Another case it can also happen is when no previous `offline -> consuming`
is pending.
That is quite rare when helix threads pick multiple `consuming -> online`
transitions.
I think we should set Semaphore with true fairness policy in that case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
