[
https://issues.apache.org/jira/browse/SAMZA-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cameron Lee reassigned SAMZA-2307:
----------------------------------
Assignee: (was: Cameron Lee)
> Migrate tests in samza-test to use in-memory system instead of Kafka/Zookeeper
> ------------------------------------------------------------------------------
>
> Key: SAMZA-2307
> URL: https://issues.apache.org/jira/browse/SAMZA-2307
> Project: Samza
> Issue Type: Improvement
> Reporter: Cameron Lee
> Priority: Major
>
> Many tests in samza-test use the IntegrationTestHarness which sets up local
> instances of Kafka and Zookeeper. However, these systems take time to start
> up. Also, using Kafka as a system is slower since there are inter-process
> calls.
> If we can replace the Kafka usage with an in-memory system, then we don't
> have to start up Kafka/Zookeeper, and test runs should be significantly
> faster.
> Some tests that could possibly be updated (not a complete list). I have added
> some notes about the current state of some of the tests. For example, the
> tests which directly use Kafka would need changes to produce to the in-memory
> system instead.
> - FaultInjectionTest: directly produces to kafka
> - WatermarkIntegrationTest: need special runner to inspect processors
> - EndOfStreamIntegrationTest
> - TestStreamProcessor: directly depends on kafka
> - TestZkLocalApplicationRunner: tied to publishing kafka events
> - TestRepartitionJoinWindowApp: tied to KafkaSystemFactory for checking
> delete messages
> - StreamApplicationIntegrationTestHarness: need to create input topics with
> correct partition counts
> - TestLocalTable*: already uses some system that is in-memory-like; uses
> kafka for default systems
> - update IntegrationTestHarness.createTopic to use in-memory system instead
> of kafka?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
