Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/132#discussion_r48548217
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java
---
@@ -392,9 +412,20 @@ public void emitTuples()
if (maxTuplesPerWindow > 0) {
count = Math.min(count, maxTuplesPerWindow - emitCount);
}
- for (int i = 0; i < count; i++) {
+ if (count > 0) {
+ // If the total size transmitted in the window will be exceeded
don't transmit anymore messages in this window
+ // Make an exception for the case when no message has been
transmitted in the window and transmit at least one
+ // message even if the message size is greater than max message size
so that the processing doesn't get stuck
+ if ((emitCount > 0) && ((maxTotalMsgSizePerWindow -
emitTotalMsgSize) < consumer.peekMessage().msg.size())) {
--- End diff --
I had thought about that but then you can have more than queue size number
of outstanding messages, you would have queue size + 1 which would violate the
limit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
