Re: Samza Unit Test Instrucations

2015-04-15 Thread Yan Fang
Hi Yuanchi,

There is no out-of-box unit tests provided by Samza. But there are some
ways:

1) If you only want to test the logic in the Task class, normal unit tests
will work. You can create a unit test that tests init(), process(), etc.

2) Create mock systems by implementing SystemAdmin, SystemConsumer, etc.
You can refer to this one TestSystemStreamPartitionIterator
https://github.com/apache/samza/blob/master/samza-api/src/test/java/org/apache/samza/system/TestSystemStreamPartitionIterator.java

3) bring up zookeeper, kafka in the unit test. Here is the reference
TestStatefulTask
https://github.com/apache/samza/blob/master/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala

4) run the job locally
http://samza.apache.org/learn/documentation/0.9/jobs/job-runner.html by
setting the job.factory.class
http://samza.apache.org/learn/documentation/0.9/jobs/configuration-table.html
.

Usually, I think 1) and 4) are sufficient. Hope this helps.

Thanks,

Fang, Yan
yanfang...@gmail.com

On Wed, Apr 15, 2015 at 1:47 PM, Yuanchi Ning yuan...@uber.com wrote:

 Hello Samza Team,

 This is Yuanchi Ning from Uber Data Engineering, Realtime Metrics,
 Streaming Platform team.

 We are planning to use Samza to process the realtime data we have, and
 thanks for developing such an awesome open source project.

 While I am building our streaming service using Samza, I am wondering is
 there anyway to do unit tests for each Samza application Task instead of
 integration test? Say, set up the mocker environment for feeding data into
 MessageEnvelope or something similar?

 Thanks for your assistance.

 Best,
 Yuanchi



Samza Unit Test Instrucations

2015-04-15 Thread Yuanchi Ning
Hello Samza Team,

This is Yuanchi Ning from Uber Data Engineering, Realtime Metrics,
Streaming Platform team.

We are planning to use Samza to process the realtime data we have, and
thanks for developing such an awesome open source project.

While I am building our streaming service using Samza, I am wondering is
there anyway to do unit tests for each Samza application Task instead of
integration test? Say, set up the mocker environment for feeding data into
MessageEnvelope or something similar?

Thanks for your assistance.

Best,
Yuanchi