[GitHub] [hudi] li36909 commented on a change in pull request #2754: [HUDI-1751] DeltaStreamer print many unnecessary warn log

[GitBox]

li36909 commented on a change in pull request #2754:
URL: https://github.com/apache/hudi/pull/2754#discussion_r606207765



##
File path: 
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java
##
@@ -173,9 +173,11 @@ public KafkaOffsetGen(TypedProperties props) {
 this.props = props;
 
 kafkaParams = new HashMap<>();
-for (Object prop : props.keySet()) {
+props.keySet().stream().filter(prop -> {

Review comment:
   BTW, I find a UT fail cause by concurrent write to a hudi table, I will 
try to analyze it later




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hudi] li36909 commented on a change in pull request #2754: [HUDI-1751] DeltaStreamer print many unnecessary warn log

[GitBox]

li36909 commented on a change in pull request #2754:
URL: https://github.com/apache/hudi/pull/2754#discussion_r606207354



##
File path: 
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KafkaOffsetGen.java
##
@@ -173,9 +173,11 @@ public KafkaOffsetGen(TypedProperties props) {
 this.props = props;
 
 kafkaParams = new HashMap<>();
-for (Object prop : props.keySet()) {
+props.keySet().stream().filter(prop -> {

Review comment:
   how about change to this: "DeltaStream print many unnecessary warn log 
because of passing hoodie config to kafka consumer". the warn log is printed by 
kafkaconsumer. when hudi new the kafka consumer, hudi pass some non-kafka 
parameter to the kafka comsumer, then lead to these warn log, to solve this 
problem we just need to filter the hoodie config.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org