[jira] [Comment Edited] (BEAM-8121) Messages are not distributed per machines when consuming from Kafka topic with 1 partition

[ https://issues.apache.org/jira/browse/BEAM-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921555#comment-16921555 ] Raghu Angadi edited comment on BEAM-8121 at 9/3/19 5:41 PM: Th

[jira] [Commented] (BEAM-8121) Messages are not distributed per machines when consuming from Kafka topic with 1 partition

[ https://issues.apache.org/jira/browse/BEAM-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921555#comment-16921555 ] Raghu Angadi commented on BEAM-8121: Thanks for filing this [~TauJan].  {quote}With si

[jira] [Commented] (BEAM-7143) adding withConsumerConfigUpdates

[ https://issues.apache.org/jira/browse/BEAM-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834077#comment-16834077 ] Raghu Angadi commented on BEAM-7143: Yes, it will be good to go with 'with' prefix to

[jira] [Resolved] (BEAM-6991) EOS: Streaming job fails on job restart with withEOS specified

[ https://issues.apache.org/jira/browse/BEAM-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved BEAM-6991. Resolution: Not A Problem Fix Version/s: Not applicable I am closing this as 'Working as Inte

[jira] [Commented] (BEAM-6991) EOS: Streaming job fails on job restart with withEOS specified

[ https://issues.apache.org/jira/browse/BEAM-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813819#comment-16813819 ] Raghu Angadi commented on BEAM-6991: [~aromanenko], you are right, the error is approp

[jira] [Commented] (BEAM-2185) KafkaIO bounded source

[ https://issues.apache.org/jira/browse/BEAM-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767411#comment-16767411 ] Raghu Angadi commented on BEAM-2185: Yes. The whole batch use case should document and

[jira] [Closed] (BEAM-5375) KafkaIO reader should handle runtime exceptions kafka client

[ https://issues.apache.org/jira/browse/BEAM-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi closed BEAM-5375. -- Resolution: Fixed > KafkaIO reader should handle runtime exceptions kafka client > -

[jira] [Updated] (BEAM-5375) KafkaIO reader should handle runtime exceptions kafka client

[ https://issues.apache.org/jira/browse/BEAM-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated BEAM-5375: --- Fix Version/s: 2.7.0 > KafkaIO reader should handle runtime exceptions kafka client >

[jira] [Commented] (BEAM-2185) KafkaIO bounded source

[ https://issues.apache.org/jira/browse/BEAM-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16756354#comment-16756354 ] Raghu Angadi commented on BEAM-2185: [~JozoVilcek], actually, I don't think 'commitOff

[jira] [Commented] (BEAM-2185) KafkaIO bounded source

[ https://issues.apache.org/jira/browse/BEAM-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755490#comment-16755490 ] Raghu Angadi commented on BEAM-2185: You are correct about issue about 'BoundedReadFro

[jira] [Commented] (BEAM-6466) KafkaIO doesn't commit offsets while being used as bounded source

[ https://issues.apache.org/jira/browse/BEAM-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16747715#comment-16747715 ] Raghu Angadi commented on BEAM-6466: BoundedReadFromUnboundedSource does not seem to b

[jira] [Commented] (BEAM-6466) KafkaIO doesn't commit offsets while being used as bounded source

[ https://issues.apache.org/jira/browse/BEAM-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746519#comment-16746519 ] Raghu Angadi commented on BEAM-6466: Added this comment to the above thread: Yeah, t

[jira] [Resolved] (BEAM-6333) allow disabling/configuring automatic watermark generation for idle kafka partitions

[ https://issues.apache.org/jira/browse/BEAM-6333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi resolved BEAM-6333. Resolution: Won't Fix Fix Version/s: Not applicable > allow disabling/configuring automatic w

[jira] [Commented] (BEAM-6333) allow disabling/configuring automatic watermark generation for idle kafka partitions

[ https://issues.apache.org/jira/browse/BEAM-6333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733539#comment-16733539 ] Raghu Angadi commented on BEAM-6333: Thanks Jan. I agree with everything in the above

[jira] [Commented] (BEAM-6333) allow disabling/configuring automatic watermark generation for idle kafka partitions

[ https://issues.apache.org/jira/browse/BEAM-6333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733426#comment-16733426 ] Raghu Angadi commented on BEAM-6333: > Would it be correct to say that the mechanism d

[jira] [Commented] (BEAM-6333) allow disabling/configuring automatic watermark generation for idle kafka partitions

[ https://issues.apache.org/jira/browse/BEAM-6333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732540#comment-16732540 ] Raghu Angadi commented on BEAM-6333: Trying to understand what would break with curren

[jira] [Commented] (BEAM-6333) allow disabling/configuring automatic watermark generation for idle kafka partitions

[ https://issues.apache.org/jira/browse/BEAM-6333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732539#comment-16732539 ] Raghu Angadi commented on BEAM-6333: > If there is some automatic timeout then you can

[jira] [Comment Edited] (BEAM-6333) allow disabling/configuring automatic watermark generation for idle kafka partitions

[ https://issues.apache.org/jira/browse/BEAM-6333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732539#comment-16732539 ] Raghu Angadi edited comment on BEAM-6333 at 1/3/19 12:14 AM: -

[jira] [Commented] (BEAM-6333) allow disabling/configuring automatic watermark generation for idle kafka partitions

[ https://issues.apache.org/jira/browse/BEAM-6333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732298#comment-16732298 ] Raghu Angadi commented on BEAM-6333: [~domsj], could you explain the specific motivati

[jira] [Commented] (BEAM-6244) updateProducerProperties is missing in KafkaIO.Write

[ https://issues.apache.org/jira/browse/BEAM-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16726491#comment-16726491 ] Raghu Angadi commented on BEAM-6244: Also add [~aromanenko] to reviewers. I am on leav

[jira] [Commented] (BEAM-6244) updateProducerProperties is missing in KafkaIO.Write

[ https://issues.apache.org/jira/browse/BEAM-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16726488#comment-16726488 ] Raghu Angadi commented on BEAM-6244: [~morokosi], please go ahead with the fix. Please

[jira] [Commented] (BEAM-6285) option to disable offsetConsumer in KafkaIO

[ https://issues.apache.org/jira/browse/BEAM-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16726486#comment-16726486 ] Raghu Angadi commented on BEAM-6285: Could you expand a bit more on the warning? Is it

[jira] [Commented] (BEAM-6063) KafkaIO: add writing support for ProducerRecord

[ https://issues.apache.org/jira/browse/BEAM-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16686998#comment-16686998 ] Raghu Angadi commented on BEAM-6063: I was thinking the second option. If you are neut

[jira] [Commented] (BEAM-6063) KafkaIO: add writing support for ProducerRecord

[ https://issues.apache.org/jira/browse/BEAM-6063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16686953#comment-16686953 ] Raghu Angadi commented on BEAM-6063: Thanks for filing this. We might keep interface b

[jira] [Commented] (BEAM-6010) Deprecate KafkaIO withTimestampFn()

[ https://issues.apache.org/jira/browse/BEAM-6010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16678588#comment-16678588 ] Raghu Angadi commented on BEAM-6010: Thank you for filing this. > Deprecate KafkaIO w

[jira] [Commented] (BEAM-5798) Add support for dynamic destinations when writing to Kafka

[ https://issues.apache.org/jira/browse/BEAM-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16678582#comment-16678582 ] Raghu Angadi commented on BEAM-5798: [~aromanenko], yeah, that's what I had in mind. B

[jira] [Commented] (BEAM-5798) Add support for dynamic destinations when writing to Kafka

[ https://issues.apache.org/jira/browse/BEAM-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677246#comment-16677246 ] Raghu Angadi commented on BEAM-5798: Outline sounds great. Thanks [~aromanenko]. Whil

[jira] [Comment Edited] (BEAM-5514) BigQueryIO doesn't handle quotaExceeded errors properly

[ https://issues.apache.org/jira/browse/BEAM-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665747#comment-16665747 ] Raghu Angadi edited comment on BEAM-5514 at 10/26/18 11:14 PM: -

[jira] [Commented] (BEAM-5514) BigQueryIO doesn't handle quotaExceeded errors properly

[ https://issues.apache.org/jira/browse/BEAM-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665747#comment-16665747 ] Raghu Angadi commented on BEAM-5514: > 1. Handle quotaExceeded within the client with

[jira] [Commented] (BEAM-5786) KafkaIO should support topic patterns

[ https://issues.apache.org/jira/browse/BEAM-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657085#comment-16657085 ] Raghu Angadi commented on BEAM-5786: It will cause reader to fail. Streaming runners t

[jira] [Commented] (BEAM-5786) KafkaIO should support topic patterns

[ https://issues.apache.org/jira/browse/BEAM-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655883#comment-16655883 ] Raghu Angadi commented on BEAM-5786: It does not. The root reason is that Beam runners

[jira] [Comment Edited] (BEAM-5786) KafkaIO should support topic patterns

[ https://issues.apache.org/jira/browse/BEAM-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655604#comment-16655604 ] Raghu Angadi edited comment on BEAM-5786 at 10/18/18 5:16 PM: --

[jira] [Comment Edited] (BEAM-5786) KafkaIO should support topic patterns

[ https://issues.apache.org/jira/browse/BEAM-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655604#comment-16655604 ] Raghu Angadi edited comment on BEAM-5786 at 10/18/18 5:15 PM: --

[jira] [Comment Edited] (BEAM-5786) KafkaIO should support topic patterns

[ https://issues.apache.org/jira/browse/BEAM-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655604#comment-16655604 ] Raghu Angadi edited comment on BEAM-5786 at 10/18/18 5:14 PM: --

[jira] [Commented] (BEAM-5786) KafkaIO should support topic patterns

[ https://issues.apache.org/jira/browse/BEAM-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655604#comment-16655604 ] Raghu Angadi commented on BEAM-5786: Sounds good. More I think about completely dynam

[jira] [Commented] (BEAM-5786) KafkaIO should support topic patterns

[ https://issues.apache.org/jira/browse/BEAM-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655471#comment-16655471 ] Raghu Angadi commented on BEAM-5786: [~aromanenko], if we need to resolve the topics o

[jira] [Commented] (BEAM-5786) KafkaIO should support topic patterns

[ https://issues.apache.org/jira/browse/BEAM-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655443#comment-16655443 ] Raghu Angadi commented on BEAM-5786: Oh, if we don't need to detect changes to topic l

[jira] [Commented] (BEAM-5786) KafkaIO should support topic patterns

[ https://issues.apache.org/jira/browse/BEAM-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655377#comment-16655377 ] Raghu Angadi commented on BEAM-5786: I don't think it is minor change :). We essentia