[GitHub] samza pull request #208: SAMZA-1264: Make Operator Functions Closable

Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/208 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[VOTE] Apache Samza 0.13.0 RC2

This is a call for a vote on a release of Apache Samza 0.13.0. Thanks to everyone who has contributed to this release. We are very glad to see some new contributors and features in this release. The release candidate can be downloaded from here: http://home.apache.org/~jagadish/samza-0.13.0-rc2/

Re: [VOTE][RESULT] Apache Samza 0.12.0 RC2

I've run the build and all the regular tests and they passed. But when I ran integration tests I got the following results: python 2.6 doesn't work - version too low. python 3.7 doesn't work - version too high ;-). It fails with a syntax error in zopkio. python 2.7 seems to work. Integration tes

[GitHub] samza pull request #204: SAMZA-1234: Documentation for 0.13.0 release

Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/204 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

Re: [DISCUSS] SEP-6: Support Watermark Across Intermediate Streams for Batch Processing

Hi, Xinyu, Thanks for the update. So I have two suggestions: - It seems to me that EndOfStream can be implemented as a special type of Watermark as well. a) we can use MAX_INT in the watermark value to indicate the end-of-stream; b) the streamId are simply the key to the Map in the source ingestio

Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

Dong, Thanks for your prompt responses. > And usually the underlying system allows user to select arbitrary partition number if it supports partition expansion. Do you know any system that does not meet these two requirement? I am not aware of a system that won't meet the modulo requirement. I

Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

Hi, Don, Thanks for the detailed design doc for a long-waited feature in Samza! Really appreciate it! I did a quick pass and have the following comments: - minor: "limit the maximum size of partition" ==> "limit the maximum size of each partition" - "However, Samza currently is not able to handle

Re: Recommended Kafka version for Samza 0.12

Hi Louis, Are you referring to the Kafka broker version or the Kafka client version? If it's the broker version, I believe you should be fine as older minor versions of clients are _usually_ compatible with newer broker versions. Samza 0.12 supports Kafka 0.10.0.1. We have not done testing on hig

Recommended Kafka version for Samza 0.12

We are finally upgrading our Samza / Kafka environments. Some non-Samza users of the cluster requested we upgrade to Kafka 0.10.2. Is Kafka 0.10.2 supported by Samza 0.12? Should we continue to use the Kafka-clients 0.10.0.1 or upgrade the client to match Kafka 0.10.2? Thanks, Louis Calisi Prin

Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

Hey Navina, Thanks much for the comment. Please see my response below. Regarding your biggest gripe with the SEP, I personally think the operational requirement proposed in the KIP are pretty general and could be easily enforced by other systems. The reason is that the module operation is pretty

Re: [DISCUSS] SEP-5: Enable partition expansion of input streams

Hey Dong, > I have updated the motivation section to clarify this. Thanks for updating the motivation. Couple of notes here: 1. > "The motivation of increasing partition number of Kafka topic includes 1) limit the maximum size of a partition in order to improve broker performance and 2) increas