Review Request 36390: SAMZA-729 git url problem in documentation

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36390/ --- Review request for samza. Repository: samza Description --- Changed git

Re: Review Request 33297: [SAMZA-654] Added ElasticsearchSystemProducer and Factory to output messages into Elasticseach indexes.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33297/ --- (Updated July 10, 2015, 2:48 p.m.) Review request for samza. Repository: samz

Re: Samza job on YARN stuck Unassigned

Hi Krzysztof, I haven't seen that error before. It does sound like it could be a connection issue. Did you check that the YARN node has access to hdfs:///user/samza/deploy/event-log-etl-nested-0.1.0-dist.tar.gz? One way to set the AM and containers to debug is to include a log4j.xml file in you

Re: Thoughts and obesrvations on Samza

That would be great to let Kafka do as much heavy lifting as possible and make it easier for other languages to implement Samza apis. One thing to watch out for is the interplay between Kafka's group management and the external scheduler/process manager's fault tolerance. If a container dies, the

Re: Thoughts and obesrvations on Samza

Yeah totally agree. I think you have this issue even today, right? I.e. if you need to make a simple config change and you're running in YARN today you end up bouncing the job which then rebuilds state. I think the fix is exactly what you described which is to have a long timeout on partition movem

Re: Thoughts and obesrvations on Samza

That's great. Thanks, Jay. On Fri, Jul 10, 2015 at 8:46 AM, Jay Kreps wrote: > Yeah totally agree. I think you have this issue even today, right? I.e. if > you need to make a simple config change and you're running in YARN today > you end up bouncing the job which then rebuilds state. I think t

Re: Review Request 35397: Fix SAMZA-697

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35397/ --- (Updated July 10, 2015, 4:54 p.m.) Review request for samza. Summary (updated

Re: Review Request 35397: Fix SAMZA-697

> On July 7, 2015, 12:42 a.m., Boris Shkolnik wrote: > > The latest diff seems incomplete. Missing some file(s). Ahh yes. Added the missing files. - Guozhang --- This is an automatically generated e-mail. To reply, visit: https://review

Re: Samza job on YARN stuck Unassigned

Hi Krzysztof, I had connectivity errors, but in my case was the /etc/hosts misconfigured. Cheers. 2015-07-10 12:11 GMT-03:00 Roger Hoover : > Hi Krzysztof, > > I haven't seen that error before. It does sound like it could be a > connection issue. Did you check that the YARN node has access >

Review Request 36398: SAMZA-714: update release doc in 0.9.1

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36398/ --- Review request for samza, Yan Fang, Chinmay Soman, Chris Riccomini, and Navina R

Re: Thoughts and obesrvations on Samza

> This leads me to thinking that merging projects and communities might be a > good idea: with the union of experience from both communities, we will > probably build a better system that is better for users. Is this what's being proposed though? Merging the projects seems like a consequence of

Question on newBlockingQueue in BlockingEnvelopeMap

Hi Samza devs and users I wrote customized Samza S3 consumer which downloads files from S3 and put messages in BlockedEnvelopeMap. It was straightforward because there's a nice example, filereader. I tried to a little optimize with newBlockingQueue() method because I guess that single queue shared

Re: Thoughts and obesrvations on Samza

Yeah I agree with this summary. I think there are kind of two questions here: 1. Technically does alignment/reliance on Kafka make sense 2. Branding wise (naming, website, concepts, etc) does alignment with Kafka make sense Personally I do think both of these things would be really valuable, and w

Re: Thoughts and obesrvations on Samza

{quote} maintained in a separate repository and retaining the existing committership but sharing as much else as possible (website, etc) {quote} Overall, I agree on this idea. Now the question is more about "how to do it". On the other hand, one thing I want to point out is that, if we decide to

Re: Thoughts and obesrvations on Samza

I broadly support it, with one big proviso. One of the attractive things about Kafka has been its minimalism -- the fact that it solves one part of the problem, simply, and very well. It is very important that it continues to do that, and that people continue to perceive it that way. Make Kafka in

Re: Thoughts and obesrvations on Samza

Julian, does this mean you AREN'T in favor of our new Kafka-based email reader? How else will Kafka users read email :-) Seriously, could not agree more. -Jay On Fri, Jul 10, 2015 at 2:03 PM, Julian Hyde wrote: > I broadly support it, with one big proviso. > > One of the attractive things abou

Re: Question on newBlockingQueue in BlockingEnvelopeMap

Hi Jae, I think the messages are not "lost", instead, they all go to one partition, in your "shared queue" implementation. If you check the code in BlockingEnvelopeMap line 123

Re: Thoughts and obesrvations on Samza

Hey Yan, Yeah philosophically I think the argument is that you should capture the stream in Kafka independent of the transformation. This is obviously a Kafka-centric view point. Advantages of this: - In practice I think this is what e.g. Storm people often end up doing anyway. You usually need t

Re: Thoughts and obesrvations on Samza

Thanks, Jay. This argument persuaded me actually. :) Fang, Yan yanfang...@gmail.com On Fri, Jul 10, 2015 at 2:33 PM, Jay Kreps wrote: > Hey Yan, > > Yeah philosophically I think the argument is that you should capture the > stream in Kafka independent of the transformation. This is obviously a

Review Request 36405: SAMZA-714: Doc publish for 0.9.1 release

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36405/ --- Review request for samza, Yan Fang, Chinmay Soman, Chris Riccomini, Navina Rames

Re: Question on newBlockingQueue in BlockingEnvelopeMap

I expected it should have not lost messages but it did. After I fixed overridden method, it was fixed. Anyway, thanks a lot for responding. On Fri, Jul 10, 2015 at 2:11 PM, Yan Fang wrote: > Hi Jae, > > I think the messages are not "lost", instead, they all go to one partition, > in your "share