[
https://issues.apache.org/jira/browse/BEAM-11172?focusedWorklogId=509668&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-509668
]
ASF GitHub Bot logged work on BEAM-11172:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Nov/20 12:42
Start Date: 10/Nov/20 12:42
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on pull request #13282:
URL: https://github.com/apache/beam/pull/13282#issuecomment-724678000
> Thanks! I'm curious is it intent that the original KafkaIO performance
test runs in batch mode? That seems wired to me since the major usage for Kafka
is in streaming and Dataflow has different execution path for batch and
streaming.
Afaict, `KafkaIOIT` was initially mostly an integration test and it was done
similar to other ITs, developed for bounded IOs. The main idea of such tests
are to write a limited number of pre-defined records (for which we already
calculated the right hash) and then read it with IO's `Read` transform and
compare the hashes. We can probably do similar for unbounded source just with
significant amount of waiting time for read.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 509668)
Time Spent: 2h 20m (was: 2h 10m)
> Set up Java Kafka performance test with runner v2
> -------------------------------------------------
>
> Key: BEAM-11172
> URL: https://issues.apache.org/jira/browse/BEAM-11172
> Project: Beam
> Issue Type: Test
> Components: io-java-kafka, testing
> Reporter: Boyuan Zhang
> Assignee: Boyuan Zhang
> Priority: P2
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Currently we have KafkaIO performance test with dataflow batch java
> production worker. We want to test it with runner v2 + SDF implementation in
> streaming as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
