[jira] [Commented] (BEAM-4420) Add KafkaIO Integration Tests
[ https://issues.apache.org/jira/browse/BEAM-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17027701#comment-17027701 ] Alexey Romanenko commented on BEAM-4420: [~mwalenia] Could we close this issue since KafkaIOIT jenkins job has been already added? > Add KafkaIO Integration Tests > - > > Key: BEAM-4420 > URL: https://issues.apache.org/jira/browse/BEAM-4420 > Project: Beam > Issue Type: Test > Components: io-java-kafka, testing >Reporter: Ismaël Mejía >Priority: Minor > Time Spent: 8h 10m > Remaining Estimate: 0h > > It is a good idea to have ITs for KafkaIO. > There are two possible issues: > 1. The tests should probably invert the pattern to be readThenWrite given > that Unbounded IOs block on Read and ... > 2. Until we have a way to do PAsserts on Unbounded sources we can rely on > withMaxNumRecords to ensure this test ends. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-4420) Add KafkaIO Integration Tests
[ https://issues.apache.org/jira/browse/BEAM-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858533#comment-16858533 ] Alexey Romanenko commented on BEAM-4420: [~ŁukaszG] Thanks for details, I'll take a look on this. > Add KafkaIO Integration Tests > - > > Key: BEAM-4420 > URL: https://issues.apache.org/jira/browse/BEAM-4420 > Project: Beam > Issue Type: Test > Components: io-java-kafka, testing >Reporter: Ismaël Mejía >Assignee: Lukasz Gajowy >Priority: Minor > > It is a good idea to have ITs for KafkaIO. > There are two possible issues: > 1. The tests should probably invert the pattern to be readThenWrite given > that Unbounded IOs block on Read and ... > 2. Until we have a way to do PAsserts on Unbounded sources we can rely on > withMaxNumRecords to ensure this test ends. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-4420) Add KafkaIO Integration Tests
[ https://issues.apache.org/jira/browse/BEAM-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858471#comment-16858471 ] Lukasz Gajowy commented on BEAM-4420: - [~mwalenia] has provided a Kubernetes scripts for setting up the Kafka cluster for testing but after that we stopped working on this (other things got into way). Feel free to steal that issue from me if you want to. I'll gladly help/review. [https://github.com/apache/beam/tree/master/.test-infra/kubernetes/kafka-cluster] > Add KafkaIO Integration Tests > - > > Key: BEAM-4420 > URL: https://issues.apache.org/jira/browse/BEAM-4420 > Project: Beam > Issue Type: Test > Components: io-java-kafka, testing >Reporter: Ismaël Mejía >Assignee: Lukasz Gajowy >Priority: Minor > > It is a good idea to have ITs for KafkaIO. > There are two possible issues: > 1. The tests should probably invert the pattern to be readThenWrite given > that Unbounded IOs block on Read and ... > 2. Until we have a way to do PAsserts on Unbounded sources we can rely on > withMaxNumRecords to ensure this test ends. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-4420) Add KafkaIO Integration Tests
[ https://issues.apache.org/jira/browse/BEAM-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857867#comment-16857867 ] Alexey Romanenko commented on BEAM-4420: [~ŁukaszG] Do you know if someone is working on this? Also, it would be great to test KafkaIO against different major kafka-client version, > Add KafkaIO Integration Tests > - > > Key: BEAM-4420 > URL: https://issues.apache.org/jira/browse/BEAM-4420 > Project: Beam > Issue Type: Test > Components: io-java-kafka, testing >Reporter: Ismaël Mejía >Assignee: Lukasz Gajowy >Priority: Minor > > It is a good idea to have ITs for KafkaIO. > There are two possible issues: > 1. The tests should probably invert the pattern to be readThenWrite given > that Unbounded IOs block on Read and ... > 2. Until we have a way to do PAsserts on Unbounded sources we can rely on > withMaxNumRecords to ensure this test ends. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
