Review Request 36903: SAMZA-744: WIP patch

2015-07-28 Thread Yi Pan (Data Infrastructure)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36903/ --- Review request for samza, Yan Fang, Chinmay Soman, Chris Riccomini, and Navina

Re: Review Request 36815: SAMZA-741 Support for versioning with Elasticsearch Producer

2015-07-28 Thread Roger Hoover
On July 28, 2015, 7:36 a.m., Dan Harvey wrote: samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/ElasticsearchSystemProducer.java, line 115 https://reviews.apache.org/r/36815/diff/3/?file=1023400#file1023400line115 could switch these around so you've got

Re: [DISCUSS] Release 0.10.0

2015-07-28 Thread Roger Hoover
Thanks, Yi. I propose that we also include SAMZA-741 for Elasticsearch versioning support with the new ES producer. I think it's very close to being merged. Roger On Tue, Jul 28, 2015 at 10:08 PM, Yi Pan nickpa...@gmail.com wrote: Hi, all, I want to start the discussion on the release

Re: Review Request 36815: SAMZA-741 Support for versioning with Elasticsearch Producer

2015-07-28 Thread Roger Hoover
On July 29, 2015, 5:47 a.m., Yi Pan (Data Infrastructure) wrote: samza-elasticsearch/src/main/java/org/apache/samza/system/elasticsearch/ElasticsearchSystemProducer.java, line 149 https://reviews.apache.org/r/36815/diff/4/?file=1024086#file1024086line149 Quick question: Is it

[DISCUSS] Release 0.10.0

2015-07-28 Thread Yi Pan
Hi, all, I want to start the discussion on the release schedule for 0.10.0. There are a few important features that we plan to release in 0.10.0 and I want to start this thread s.t. we can agree on what to include in 0.10.0 release. There are the following main features added in 0.10.0: -

Re: no new topic created on Kafka

2015-07-28 Thread Job-Selina Wu
Hi, Yan I like to correct my previous comment, when I comment out systems.kafka.streams.http-demo.samza.offset.default=oldest systems.kafka.streams.http-demo.samza.reset.offset=true *the logger is not show at *at samza-container-0.log, but it make sense. Sincerely, Seina On Tue, Jul 28, 2015

Re: no new topic created on Kafka

2015-07-28 Thread Yan Fang
task.class=samza.http.demo.task.HttpDemoParserStreamTask ... you are not using the StateStream class... Fang, Yan yanfang...@gmail.com On Tue, Jul 28, 2015 at 11:48 AM, Job-Selina Wu swucaree...@gmail.com wrote: Hi, Yan I like to correct my previous comment, when I comment out

Re: Review Request 36815: SAMZA-741 Support for versioning with Elasticsearch Producer

2015-07-28 Thread Roger Hoover
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36815/ --- (Updated July 28, 2015, 6:13 a.m.) Review request for samza. Changes ---

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

2015-07-28 Thread Shadi A. Noghabi
On July 21, 2015, 9:53 p.m., Navina Ramesh wrote: Ideally, the job coordinator should act as the config manager. Since the chain of control is still from AppMaster to JobCoordinator (instead of the other way around), the only reasonable way to get this working is to have it as a

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

2015-07-28 Thread Shadi A. Noghabi
On July 22, 2015, 6:25 p.m., Yi Pan (Data Infrastructure) wrote: samza-yarn/src/main/scala/org/apache/samza/job/yarn/SamzaAppMaster.scala, line 130 https://reviews.apache.org/r/36006/diff/2/?file=1015156#file1015156line130 Question: can we move this code into the

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

2015-07-28 Thread Shadi A. Noghabi
On July 22, 2015, 7:08 p.m., Yi Pan (Data Infrastructure) wrote: samza-core/src/main/java/org/apache/samza/autoScaling/deployer/ConfigManager.java, line 188 https://reviews.apache.org/r/36006/diff/2/?file=1015151#file1015151line188 Any reason the boostrap, skipUnreadMessages,

Re: Review Request 36006: Writing a tool to read from the coordinator stream and react to config changes accordingly.

2015-07-28 Thread Navina Ramesh
On July 21, 2015, 9:53 p.m., Navina Ramesh wrote: Ideally, the job coordinator should act as the config manager. Since the chain of control is still from AppMaster to JobCoordinator (instead of the other way around), the only reasonable way to get this working is to have it as a