[ https://issues.apache.org/jira/browse/BEAM-6285?focusedWorklogId=190245&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-190245 ]
ASF GitHub Bot logged work on BEAM-6285: ---------------------------------------- Author: ASF GitHub Bot Created on: 25/Jan/19 23:15 Start Date: 25/Jan/19 23:15 Worklog Time Spent: 10m Work Description: rangadi commented on pull request #7598: [BEAM-6285] add parameters for offsetConsumer in KafkaIO.read() URL: https://github.com/apache/beam/pull/7598#discussion_r251165749 ########## File path: sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java ########## @@ -656,6 +661,28 @@ return toBuilder().setCommitOffsetsInFinalizeEnabled(true).build(); } + /** + * Set additional parameters for KafkaUnboundedReader.offsetConsumer, added by BEAM-6285. Review comment: nit. It is better to write JavaDoc that explains functionality to the end user who should be assumed to not know the code at all. It is better not to refer to specific bug reports since they may think the description here may be incomplete. E.g. here the first line is pretty opaque to most users. It could start with something like: Set additional configuration for secondary 'offset consumer' used inside the reader. Each KafkaIO reader has two consumers, ... Remaining JavaDoc looks fairly good, But please avoid referring internal variables like KafkaUnboundedReader.offsetConsumer . ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 190245) Time Spent: 2.5h (was: 2h 20m) > add parameters for offsetConsumer in KafkaIO.read() > --------------------------------------------------- > > Key: BEAM-6285 > URL: https://issues.apache.org/jira/browse/BEAM-6285 > Project: Beam > Issue Type: Improvement > Components: io-java-kafka > Reporter: Xu Mingmin > Assignee: Xu Mingmin > Priority: Major > Time Spent: 2.5h > Remaining Estimate: 0h > > Add an option in KafkaIO.read() to avoid using `offsetConsumer`. > *What's the problem:* > When security is enabled, function > `KafkaUnboundedReader.updateLatestOffsets()` always fails with lots of WARN > log. > *What's the cause:* > `offsetConsumer` uses a mocked `ConsumerConfig.GROUP_ID_CONFIG`, and it > doesn't work as expected when security is enabled. In our case, > `ConsumerConfig.GROUP_ID_CONFIG` is the key to validate whether this consumer > is approved. > *What's the solution:* > Add an option to disable `offsetConsumer` by developers; > *What's the side effect:* > With `offsetConsumer` disabled, backlog is unknown. > > [~rangadi] any comments? > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)