Re: Testing comm...@samza.apache.org

2015-01-23 Thread sriram
yes On Fri, Jan 23, 2015 at 9:49 AM, Chris Riccomini wrote: > Hey all, > > Could you please confirm that you're seeing this? I'm trying to verify the > TLP migration for: > > https://issues.apache.org/jira/browse/INFRA-9055 > > Cheers, > Chris >

Re: Versioning beyond 0.9.0

2015-01-30 Thread sriram
+1 On Fri, Jan 30, 2015 at 10:40 AM, Yi Pan wrote: > +1. > > On Fri, Jan 30, 2015 at 9:22 AM, Chris Riccomini > wrote: > > > Hey all, > > > > I'm planning on opening a post-0.9.0 ticket. I was going to go with > 0.10.0. > > Is everyone good with that? Some people are kind of finicky about > > d

Re: [DISCUSS] SQL workflow

2015-02-04 Thread sriram
Can we have updated RBs for all the three sub tasks before we commit? This would help us to review even after we commit. On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini wrote: > Hey all, > > Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets related > to SQL operators. We're get

Re: [DISCUSS] Samza 0.9.1 release

2015-06-16 Thread sriram
+1 On Tue, Jun 16, 2015 at 7:41 PM, Shekar Tippur wrote: > +1 > On Jun 16, 2015 6:39 PM, "Percy Wegmann" > wrote: > > > Thank you! > > > > Sent using CloudMagic< > > https://cloudmagic.com/k/d/mailapp?ct=pi&cv=6.0.64&pv=8.2> > > > > On Tue, Jun 16, 2015 at 8:11 PM, Chris Riccomini > > wrote: >

Re: Thoughts and obesrvations on Samza

2015-07-02 Thread Sriram
One thing that is worth exploring is to have a transformation and ingestion library in Kafka but use the same framework for fault tolerance, resource isolation and management. The biggest difference I see in these two use cases is the API and data model. > On Jul 2, 2015, at 8:59 AM, Jay Kreps

Samza Mesos

2016-08-30 Thread Sriram Ramachandrasekaran
;explicit" dependency on Kafka. Some clarity on this would really help us. Sriram -- It's just about how deep your longing is!

Re: Samza Mesos

2016-08-31 Thread Sriram Ramachandrasekaran
Yi, That's a good amount of history to know. I will take a look at 680 and then see if I can implement something as well. If there's some stuff that's already done, would be glad to re-use it too. Thanks again On Wed, Aug 31, 2016 at 10:58 PM, Yi Pan wrote: > Hi, Sriram, &g

Re: Samza Mesos

2016-08-31 Thread Sriram Ramachandrasekaran
Thanks Jagadish. So, in essence, I should be looking at samza-11 branch for the final API against which I would have to write the Mesos integration pieces? On Thu, Sep 1, 2016 at 12:56 AM, Jagadish Venkatraman < jagadish1...@gmail.com> wrote: > Hi Sriram, > > I had started prototy

Collocating Samza(YARN) and Kafka/ZK clusters

2015-02-09 Thread Karthik Sriram
Hey all, I'm trying to run samza on a 5 node (YARN/Kafka/ZK) cluster with each box running all 3 processes on AWS. I have been facing very weird performance issues with Kafka when run this way. Kafka seems to get unbalanced very often with replicas going out of sync every so often. This results in

Re: Collocating Samza(YARN) and Kafka/ZK clusters

2015-02-13 Thread Karthik Sriram
replica out > of > > > sync if it falls more than N messages behind. Can you try tuning this > > > setting as described here: > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ# > > > FAQ-HowtoreducechurnsinISR?WhendoesabrokerleavetheISR > > > ? &g

Re: Windowing Guarantees in samza

2015-02-15 Thread Karthik Sriram
Hi Benjamin, I'm trying to employ samza for a similar use case and this is what I did to mitigate this: 1> I have a notion of timestamp in the messages itself that I listen to. This way, as I get messages, I can maintain state by time period of aggregation by attaching the time period to the key

Re: Dinamically asigning output topic

2015-08-03 Thread Karthik Sriram
I believe you only need to declare the system stream you are writing to before hand . ie your Kafka broker list if you are using Kafka. If auto create topics is enabled, you can dynamically write to any topic within a predefined system stream. I'd suggest simply use a hash map of topic to OutputStr

Samza / Kafka Streams Future

2016-05-23 Thread Sriram Ramachandrasekaran
Hello Yi, et all, I've been following Samza and Kafka (and, Kafka Streams). Given the state where Kafka Streams is, it provides a nice high level API for consuming stuff from Kafka + support for localized state. If thrown into an environment like mesos(fronted by marathon), we should get distribut