aromanenko-dev commented on pull request #13282: URL: https://github.com/apache/beam/pull/13282#issuecomment-724678000
> Thanks! I'm curious is it intent that the original KafkaIO performance test runs in batch mode? That seems wired to me since the major usage for Kafka is in streaming and Dataflow has different execution path for batch and streaming. Afaict, `KafkaIOIT` was initially mostly an integration test and it was done similar to other ITs, developed for bounded IOs. The main idea of such tests are to write a limited number of pre-defined records (for which we already calculated the right hash) and then read it with IO's `Read` transform and compare the hashes. We can probably do similar for unbounded source just with significant amount of waiting time for read. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
