Review Request 35515: SAMZA-449 Expose RocksDB statistic

2015-06-16 Thread Gustavo Anatoly F . V . Solís
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35515/ --- Review request for samza. Repository: samza Description --- RocksDB

Re: 3 processed message per incoming message

2015-06-16 Thread Shekar Tippur
This is what I see on Yarn monitoring page: As we can see, there are 9998 apps pending. There is some 10k limit we are hitting. I see only 1 app running. Apps SubmittedApps PendingApps RunningApps CompletedContainers RunningMemory UsedMemory TotalMemory ReservedVCores UsedVCores TotalVCores

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Yi Pan
Hi, Shekar, This 0.9.1 is a bug-fix only release. No features added yet. New features are expected in 0.10.0. Thanks! On Tue, Jun 16, 2015 at 10:59 AM, Shekar Tippur ctip...@gmail.com wrote: Wang, I have not caught up but can you please highlight if there are any feature additions as well?

Re: 3 processed message per incoming message

2015-06-16 Thread Yan Fang
Hi Shekar, Ok. If there is only one application is running, if you kill this one, will you still be able to see the processed messages coming? If not, I think the code in your application maybe the cause of the problem. We can have a further look at your code to see where the problem is. Thanks,

improving hello-samza / testing

2015-06-16 Thread Tim Williams
I'm learning samza by the hello-samza project and notice the lack of tests. Where's a good place to learn how folks are properly testing things written with samza? Thanks, --tim

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Gustavo Anatoly
+1 Thanks. 2015-06-16 14:15 GMT-03:00 Yan Fang yanfang...@gmail.com: Agreed on this. Thanks, Fang, Yan yanfang...@gmail.com On Tue, Jun 16, 2015 at 10:14 AM, Guozhang Wang wangg...@gmail.com wrote: Hi all, We have been running a couple of our jobs against `0.9.1` branch last

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Chinmay Soman
+1 On Tue, Jun 16, 2015 at 11:17 AM, Navina Ramesh nram...@linkedin.com.invalid wrote: +1 for the release! On 6/16/15, 11:03 AM, Yi Pan nickpa...@gmail.com wrote: +1 Agreed. Thanks! On Tue, Jun 16, 2015 at 10:15 AM, Yan Fang yanfang...@gmail.com wrote: Agreed on this. Thanks,

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Guozhang Wang
Cool. I will start a voting process soon. On Tue, Jun 16, 2015 at 11:55 AM, Chinmay Soman chinmay.cere...@gmail.com wrote: +1 On Tue, Jun 16, 2015 at 11:17 AM, Navina Ramesh nram...@linkedin.com.invalid wrote: +1 for the release! On 6/16/15, 11:03 AM, Yi Pan nickpa...@gmail.com

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Yi Pan
+1 Agreed. Thanks! On Tue, Jun 16, 2015 at 10:15 AM, Yan Fang yanfang...@gmail.com wrote: Agreed on this. Thanks, Fang, Yan yanfang...@gmail.com On Tue, Jun 16, 2015 at 10:14 AM, Guozhang Wang wangg...@gmail.com wrote: Hi all, We have been running a couple of our jobs against

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Yan Fang
Agreed on this. Thanks, Fang, Yan yanfang...@gmail.com On Tue, Jun 16, 2015 at 10:14 AM, Guozhang Wang wangg...@gmail.com wrote: Hi all, We have been running a couple of our jobs against `0.9.1` branch last week at LinkedIn with some critical bug fixes back-ported, including: SAMZA-608

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Shekar Tippur
Wang, I have not caught up but can you please highlight if there are any feature additions as well? - Shekar On Tue, Jun 16, 2015 at 10:14 AM, Guozhang Wang wangg...@gmail.com wrote: Hi all, We have been running a couple of our jobs against `0.9.1` branch last week at LinkedIn with some

Re: Review Request 34974: SAMZA-676: implement broadcast stream

2015-06-16 Thread Yan Fang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34974/ --- (Updated June 16, 2015, 9:06 p.m.) Review request for samza. Changes ---

[DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Guozhang Wang
Hi all, We have been running a couple of our jobs against `0.9.1` branch last week at LinkedIn with some critical bug fixes back-ported, including: SAMZA-608 Deserialization error causes SystemConsumers to hang SAMZA-616 Shutdown hook does not wait for container to finish SAMZA-658

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Navina Ramesh
+1 for the release! On 6/16/15, 11:03 AM, Yi Pan nickpa...@gmail.com wrote: +1 Agreed. Thanks! On Tue, Jun 16, 2015 at 10:15 AM, Yan Fang yanfang...@gmail.com wrote: Agreed on this. Thanks, Fang, Yan yanfang...@gmail.com On Tue, Jun 16, 2015 at 10:14 AM, Guozhang Wang

Re: Review Request 34974: SAMZA-676: implement broadcast stream

2015-06-16 Thread Yan Fang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34974/ --- (Updated June 16, 2015, 9:23 p.m.) Review request for samza. Changes ---

Re: improving hello-samza / testing

2015-06-16 Thread Chinmay Soman
We've built a driver program which kinda falls along approach (1) listed in your email. The driver program accepts a custom task object and has a way to inject data - which in turn invokes the process method. For now we're assuming logical time and use the frequency of process() invocations to

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Percy Wegmann
Thank you! Sent using CloudMagichttps://cloudmagic.com/k/d/mailapp?ct=picv=6.0.64pv=8.2 On Tue, Jun 16, 2015 at 8:11 PM, Chris Riccomini criccom...@apache.org wrote: +1 Here. On Tue, Jun 16, 2015 at 12:01 PM, Guozhang Wang wangg...@gmail.com wrote: Cool. I will start a voting process soon.

Measuring Samza Job Throughput

2015-06-16 Thread Milinda Pathirage
Hi Devs, I was looking for a way to measure Samza job throughput and found that its possible to do it via Samza's metrics reporter. But there several types of metrics reported via this method. For example, TaskInstanceMetrics reports number of messages sent. But if I wanted to get a measurement

Re: improving hello-samza / testing

2015-06-16 Thread Luis Fernando De Pombo
https://issues.apache.org/jira/browse/SAMZA-681 tracks the first effort towards the driver program or unit test harness for samza tasks that Chinmay is referring to. ᐧ On Tue, Jun 16, 2015 at 6:11 PM, Chinmay Soman chinmay.cere...@gmail.com wrote: We've built a driver program which kinda falls

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Shekar Tippur
+1 On Jun 16, 2015 6:39 PM, Percy Wegmann percy.wegm...@evariant.com wrote: Thank you! Sent using CloudMagic https://cloudmagic.com/k/d/mailapp?ct=picv=6.0.64pv=8.2 On Tue, Jun 16, 2015 at 8:11 PM, Chris Riccomini criccom...@apache.org wrote: +1 Here. On Tue, Jun 16, 2015 at 12:01 PM,

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread sriram
+1 On Tue, Jun 16, 2015 at 7:41 PM, Shekar Tippur ctip...@gmail.com wrote: +1 On Jun 16, 2015 6:39 PM, Percy Wegmann percy.wegm...@evariant.com wrote: Thank you! Sent using CloudMagic https://cloudmagic.com/k/d/mailapp?ct=picv=6.0.64pv=8.2 On Tue, Jun 16, 2015 at 8:11 PM, Chris

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Naveen Somasundaram
+1 On Jun 16, 2015, at 7:41 PM, Shekar Tippur ctip...@gmail.com wrote: +1 On Jun 16, 2015 6:39 PM, Percy Wegmann percy.wegm...@evariant.com wrote: Thank you! Sent using CloudMagic https://cloudmagic.com/k/d/mailapp?ct=picv=6.0.64pv=8.2 On Tue, Jun 16, 2015 at 8:11 PM, Chris

Re: Review Request 35241: refactoring the code for coordinator stream writer

2015-06-16 Thread Navina Ramesh
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35241/#review88174 ---

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread Chris Riccomini
+1 Here. On Tue, Jun 16, 2015 at 12:01 PM, Guozhang Wang wangg...@gmail.com wrote: Cool. I will start a voting process soon. On Tue, Jun 16, 2015 at 11:55 AM, Chinmay Soman chinmay.cere...@gmail.com wrote: +1 On Tue, Jun 16, 2015 at 11:17 AM, Navina Ramesh

Re: improving hello-samza / testing

2015-06-16 Thread Chris Riccomini
Hey Tim, This is a really good discussion to have. The testing that I've seen with Samza falls into two categories: 1. Instantiate your StreamTask, and mock all params in the process()/init() methods. 2. A mini-ontegration test that starts ZooKeeper, and Kafka, and feeds messages into a topic,

Powered by page update

2015-06-16 Thread Chris Riccomini
Hey all, I'm seeing a lot of new faces on the mailing list, which is really awesome. I want to invite you all to add yourselves to our Powered by page: https://cwiki.apache.org/confluence/display/SAMZA/Powered+By The Apache wiki is pretty locked down due to spam. If you'd like to send me a link

Re: Review Request 35492: SAMZA-701 : Hello Samza - Port docker setup from hadoop-common

2015-06-16 Thread Darrell Taylor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35492/ --- (Updated June 16, 2015, 7:36 a.m.) Review request for samza. Summary

Review Request 35492: Port docker setup from hadoop-common

2015-06-16 Thread Darrell Taylor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35492/ --- Review request for samza. Repository: samza-hello-samza Description ---