Re: [VOTE] SEP-32: Elasticity for Samza

Thank you Manasa. I reviewed it and it looks good to me. +1 (approve) this SEP. On Tue, Feb 7, 2023 at 2:14 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > +1 (binding) > > Cheers, > Bharath > > On Tue, Feb 7, 2023 at 12:56 PM Lakshmi Manasa > wrote: > > > Hi folks, > > > >

Re: [DISCUSS] SEP-32: Elasticity for Samza

Thank you Manasa for the proposal. I reviewed it and it looks good to me. nice work! +1 (approve) from my end. On Mon, Feb 6, 2023 at 11:41 PM Yi Pan wrote: > Hi, Manasa, > > Sorry for the late reply. The revision lgtm. Thanks for the great work! > > Best, > > -Yi > > On Mon, Jan 30, 2023 at

Re: [VOTE] SEP-31: Pipeline Drain- Support the ability to drain pipelines to allow incompatible intermediate schema changes

Hi Ajo, I reviewed your work and the overall design. Looks great! Thank you for contributing this much needed feature. [+1] I approve. On Thu, Dec 8, 2022 at 4:45 PM Yi Pan wrote: > +1. Long awaited feature! Thanks! > > -Yi > > On Tue, Nov 29, 2022 at 11:46 AM Xinyu Liu wrote: > > > +1. >

Re: [ANNOUNCE] Welcome Daniel Chen as Samza Committer

Congrats Daniel on this well deserved recognition. Look forward to more contributions! On Fri, Sep 17, 2021 at 11:25 AM Yi Pan wrote: > Congrats, Daniel, well deserved!!! > > -Yi > > On Fri, Sep 17, 2021 at 11:23 AM Xinyu Liu wrote: > > > Hi, all, > > > > I am glad to announce that Daniel Che

Re: How are samza.container.id generated in yarn?

Hi Debraj, Clarifying a bit on Yi’s response, since it was referring to the physical Yarn container id.. If there are N Yarn containers, samza.container.ids are generated sequentially from 0 to N-1. This ID is meant to be durable - ie., if a particular container fails, the Samza AM will restart i

Re: Welcome Sanil Jain as Apache Samza committer

Congrats Sanil! Looking forward to more contributions! On Tue, Feb 2, 2021 at 7:31 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > Congratulations Sanil! > > On Tue, Feb 2, 2021 at 3:43 PM Miguel Sanchez Schwarz > wrote: > > > Congratulations Sanil!! > > __

Re: [DISCUSS] Samza 1.5.1 release

+1. On Saturday, August 22, 2020, Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > It has been more than 72+ hours and the response seems positive. > Let me proceed with a voting thread. > > Thanks, > Bharath > > On Wed, Aug 19, 2020 at 10:31 AM Xinyu Liu wrote: > > > +1. Hopefully

Re: [VOTE] SEP-22: Container Placements in Samza

+1 (binding) Thanks Sanil for driving this proposal! On Fri, May 29, 2020 at 5:24 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > +1. > Looking forward to use the feature. > > -- > Bharath > > On Fri, May 29, 2020 at 12:16 PM Prateek Maheshwari > wrote: > > > +1 from me too. T

Re: [VOTE] SEP-24: Cluster-based Job Coordinator Dependency Isolation

+1 binding. Thanks Cameron. I look forward to this feature taking our "Stream Processing as a service" offering to the next level. Cheers On Tuesday, March 3, 2020, Prateek Maheshwari wrote: > +1 (binding) from me. Thanks for contributing this feature. Looking forward > to having dependency is

Re: Samza - New Relic integration

91-9505020924 > > vaibhavgar...@gmail.com > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/> > > > > > > On Sun, Mar 1, 2020 at 12:31 PM Jagadish Venkatraman < > > jagadish1...@gmail.com> wrote: > > > >> Hi Vaibhav, > >> > >&

Re: Samza - New Relic integration

Hi Vaibhav, Can you try ./__package/newrelic/newrelic.jar? If that doesn't work, I'd recommend doing the following: 1. Print the current working directory of your Samza container JVM 2. Use (1) to determine a relative reference to the directory path within your package Cheers Jagadish On Saturd

Re: [VOTE] Apache Samza 1.3.1 RC0

+1 (binding) On Wednesday, February 19, 2020, Prateek Maheshwari wrote: > Integration tests and check-all passed successfully. +1 (binding) from me. > > Thanks, > Prateek > > On Tue, Feb 18, 2020 at 12:47 PM Bharath Kumara Subramanian < > codin.mart...@gmail.com> wrote: > > > Ran check-all and i

Re: [ANNOUNCE] Please welcome Bharath Kumarasubramanian to the Samza PMC

Congrats Bharath. Great work! Looking forward to continued contributions! On Thursday, February 13, 2020, Yang Zhang wrote: > Congratulations, Bharath! Nice work and thanks for the contributions! > > Best, > Yang > > On Thu, Feb 13, 2020 at 4:27 PM Xinyu Liu wrote: > > > Hi all, > > > > I'm ver

Re: [DISCUSS] Samza 1.3.1 release

+1 thank you Hai for shepherding this release. On Thu, Feb 13, 2020 at 3:53 PM Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > +1. > > Cheers, > Bharath > > On Thu, Feb 13, 2020 at 12:54 PM Yang Zhang wrote: > > > +1. Thanks! > > > > Best, > > Yang > > > > On Thu, Feb 13, 2020 at

Re: [Draft] Samza quarterly report

lgtm, thanks Yi. On Thu, Jan 9, 2020 at 6:46 PM Prateek Maheshwari wrote: > Looks good to me, thanks! > > - Prateek > > On Thu, Jan 9, 2020 at 1:43 PM Yi Pan wrote: > > > ## Description: > > - Apache Samza is a distributed stream processing engine that are highly > > configurable to process e

Re: [VOTE] SEP-26: Add SystemProducer for Azure Blob Storage

+1 (binding), looking forward to Samza's integration with Azure blobs On Wednesday, January 8, 2020, Lakshmi Manasa wrote: > Hi, > > This is a call for a vote on SEP-26: Add SystemProducer for Azure Blob > Storage. > Thanks for taking a look and giving feedback. > > I have addressed the comments

Re: [VOTE] SEP 25: PR Title and Description Guidelines

+1 (binding) On Thursday, December 19, 2019, Daniel Nishimura wrote: > +1 non-binding > > > On Dec 18, 2019, at 7:09 PM, Yi Pan wrote: > > > > +1 (binding) > > > > On Wed, Dec 18, 2019 at 10:49 AM Bharath Kumara Subramanian < > > codin.mart...@gmail.com> wrote: > > > >> +1 (non-binding). > >> >

[DISCUSS] SEP 25: PR Title and Description Guidelines

+1, thanks for the write-up Prateek. Let's also update the contributor's guidelines at: https://samza.apache.org/contribute/contributors-corner.html On Friday, December 13, 2019, Prateek Maheshwari wrote: > Hi folks, > > In order to make Samza PR descriptions and commit messages more consisten

Re: [VOTE] SEP-23: Simplify Job Runner

+1 (binding) thanks Ke for driving this proposal! On Wed, Dec 11, 2019 at 2:45 PM Yi Pan wrote: > +1(binding). > > Thanks! > > -Yi > > On Wed, Dec 11, 2019 at 12:13 PM Prateek Maheshwari > wrote: > > > +1 (binding). > > > > Thanks, > > Prateek > > > > On Wed, Dec 11, 2019 at 11:50 AM Xinyu Liu

Re: [DISCUSS] SEP-23: Simplify Job Runner

+1 Lgtm, thanks Ke for the proposal. On Tuesday, December 10, 2019, Prateek Maheshwari wrote: > Looks good to me as well. +1 for the overall proposal, and thanks for > putting it together. > > - Prateek > > On Tue, Dec 10, 2019 at 1:26 PM Xinyu Liu wrote: > > > Thanks for updating the SEP wiki.

Re: [VOTE] Apache Samza 1.3.0 RC2

Hai, Do you mind sending a VOTE RESULT email to the mailing list to officially conclude the vote? Thanks Jagadish On Wednesday, December 4, 2019, Hai Lu wrote: > Thanks everyone. The vote passed. > > > On Wed, Dec 4, 2019 at 3:25 PM Prateek Maheshwari > wrote: > > > + 1 (binding) > > > > Ver

Re: [VOTE] SEP-20: Samza on Kubernetes

+1 binding. Thanks Weiqing for driving this! On Thursday, November 7, 2019, Xinyu Liu wrote: > +1 (binding). > > Thanks, > Xinyu > > On Thu, Nov 7, 2019 at 10:50 AM Weiqing Yang > wrote: > > > Hi All, > > > > The feedback from the discussion thread: > > http://mail-archives.apache.org/mod_mbox

Re: [DISCUSS] SEP-20: Samza on Kubernetes

+1, look forward to Samza K8s integration :) On Monday, November 4, 2019, Xinyu Liu wrote: > +1 on the design. This is a great feature to allow Samza to expand its > deployment to Kubernetes clusters. Nice job! > > Thanks, > Xinyu > > On Mon, Nov 4, 2019 at 10:10 AM Weiqing Yang > wrote: > > >

Re: [VOTE] SEP-18: Startpoints - Manipulating Starting Offsets for Input Streams

+1 (binding) Excellent work Dan! LGTM; On Fri, Sep 6, 2019 at 8:02 AM Daniel Nishimura wrote: > Please vote for SEP-18 > < > https://cwiki.apache.org/confluence/display/SAMZA/SEP-18%3A+Startpoints+-+Manipulating+Starting+Offsets+for+Input+Streams > >. > Thanks to the committers and contributors

Re: At-least once processing guarantee

Is the rejected alternatives document meant to be private (link at the > bottom of the document)? > > On Wed, Jul 24, 2019 at 11:46 AM Jagadish Venkatraman > > wrote: > > > Dear Samza users, > > > > We recently discovered an issue with the way we handle state in Samza

At-least once processing guarantee

Dear Samza users, We recently discovered an issue with the way we handle state in Samza Beam and Samza High-Level API Window operators. Under certain situations, at least once processing guarantee is violated. *Details on the issue* The Samza high-level API includes operators such as windows wh

[ANNOUNCE] Welcoming Rayman and Bharath as new Samza committers!

I'm pleased to announce that the Samza PMC has voted to invite Rayman & Bharath as committers, and they have accepted. Please join us in congratulating them on this recognition! A quick summary of their accomplishments.. *Rayman * Ray has been driving multiple improvements to Samza for stateful

Re: [ANNOUNCE] Please welcome Boris Shkolnik to the Samza PMC

Congratulations Boris! On Fri, Jun 7, 2019 at 3:15 PM Xinyu Liu wrote: > Congrats, Boris! > > Xinyu > > On Fri, Jun 7, 2019 at 3:13 PM Jakob Homan wrote: > > > Howdy all- > >I'm very pleased to announce that the Samza PMC has voted Boris > > Shkolnik to be a Project Management Committee (PM

Re: Auto-sizing subsystem for Samza

Thanks Ray. This proposal that takes Samza's capabilities to the next level. On Thu, Jun 6, 2019 at 11:38 AM rayman preet wrote: > I'm trying to put together a Samza subsystem that is able to auto-size a > Samza job > by looking at metrics. Here are some initial thoughts: > > https://docs.google

Re: REMINDER. [VOTE] Apache Samza 1.2.0 RC4

+1 (binding) Verified signatures, built the RC successfully. On Wed, Jun 5, 2019 at 5:17 PM Prateek Maheshwari wrote: > +1 (binding) > > Verified build + check-all + integration tests + signatures. > Thanks for help with the release, Boris and Pawas. > > - Prateek > > On Wed, Jun 5, 2019 at

Re: AM resource needs

ersonal and confidential use of the > recipient(s) named above. If you have received this message in error, > please notify the sender immediately and delete the original message. > > > On Thu, May 23, 2019 at 10:39 PM Jagadish Venkatraman < > jagadish1...@gmail.com> wrote: &

Re: AM resource needs

No, the default setup should be sufficient - the number of tasks should have no significant impact on AM memory/resources. If you run out-of-memory, you can of course increase yarn.am.memory.mb. On Thu, May 23, 2019 at 10:45 AM Malcolm McFarland wrote: > Hey folks, > > Are there any guidelines

Re: Samza tasks aren't starting in YARN containers

va.net.ConnectException. Retrying... > > There's no other information in the log about what is going on. Does > anybody have ideas on this? > > Btw, how do I pull a thread-dump of the stuck container? > > Cheers, > Malcolm > > > On Tue, May 7, 2019 at 10:48 PM Jaga

Re: Samza tasks aren't starting in YARN containers

Malcolm, Did the AM-process come up? If so, can you attach its entire log-file? "> everything will launch fine one time, and then it will do this RUNNING-but-no-Samza thing the next." IIUC, you believe your container is not making progress. If the issue is recurs, can you attach a thread-dump &

Re: Samza for text processing

do you think would be more appropriate? > > Also for files over 1mb would you increase the default kafka limit? Break > the document into chunks or pass a reference in the message? > > Thanks again > > > > On Sun, 28 Apr 2019, 16:20 Jagadish Venkatraman, > wrote: &

Re: Running Apache Samza on Kubernetes cluster with Zookeeper

+Weiqing Yang, who gave a community talk on this at KubeCon this year. Hi Stefano, Running Samza-standalone + Zk on Kubernetes should be no different than running any other application on Kubernetes. At a high-level, you would: 1. Package your application as a container image - just as you

Re: Samza for text processing

Hi Rob, Yes, your use-case is a good fit. You can use Samza for fault-tolerant stream processing. We have document (eg: member profiles, articles/blogs) standardization use-cases at LinkedIn powered by Samza. Please let us know should you have further questions! On Sun, Apr 28, 2019 at 7:09 AM

Re: [ANNOUNCE] New committer announcement: Cameron Lee

Awesome addition! Congrats Cameron. well deserved. On Tuesday, April 16, 2019, Xinyu Liu wrote: > Hi, all, > > Please join me and the rest of the Samza PMC in welcoming a new committer: > Cameron Lee. > > Cameron has been contributing to Samza since early 2018. He worked on > multiple areas: the

Re: [DISCUSS] Change Apache Samza git comments/merge email recipient to commits@samza

+1 to the proposal Xinyu. Thanks, On Fri, Apr 5, 2019 at 5:01 PM Prateek Maheshwari wrote: > Thanks Xinyu. Separating discussions and commit messages sounds good to me. > I'm +1, but happy to keep it as-is if others find the commit emails useful. > > - Prateek > > On Thu, Apr 4, 2019 at 3:14 PM

Re: [VOTE] SEP-21: Samza Async API for High Level

+1 (binding); thanks Bharath for the proposal and the implementation. LGTM; On Monday, March 25, 2019, Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > Hi all, > > > This is a call for a vote for SEP-21: Samza Async API for High Level > > > SEP-21 has been discussed and implemente

Re: Empty Kafka topic partition Warning

nput topic partitions empty. One of our upstream jobs is likely configured > incorrectly. > > > > > > Jeremiah Adams > Software Engineer > www.helixeducation.com > Blog | Twitter | Facebook | LinkedIn > > ____ > From: Jagadi

Re: SSL with Samza 0.14.1?

Hi Matt, >> We’re trying to connect to a cluster that has SSL enabled, and we’re getting some errors that are indicative of SSL connectivity failing. Can you share some more details on the errors - eg: log files, stack-traces? Do you get the error when Samza attempts to create the topic? >> Is

Re: Empty Kafka topic partition Warning

che.samza.storage.kv.RocksDbKeyValueStorageEngineFactory > stores.redelivery-store.changelog=kafka.delivery-changelog > stores.default.changelog.replication.factor=3 > > > > > > Jeremiah Adams > Software Engineer > www.helixeducation.com > Blog | Twitter | Facebook | Li

Re: Empty Kafka topic partition Warning

Hi Jeremiah, >> why is the offset 0? This likely means that the change-log is empty and does not have any messages. Can you try consuming from partition-number: 0 using a KafkaConsumer? Best, Jagadish On Fri, Mar 22, 2019 at 11:45 AM Jeremiah Adams wrote: > I'm seeing these in our log pe

Re: [VOTE] Apache Samza 1.1.0 RC2

1. Verified check-sum and signatures for the release binaries. 2. Ran ./check-all.sh successfully 3. Ran YARN integration tests successfully 4. Encountered an error on the standalone integration test, but it succeeded after setting Kafka's replication factor config to 1. +1(binding) from my side.

Announcing Samza 101: Video tutorials on Stream processing

Hello Samza users, We are really excited to share with you a short video course for getting started with Stream Processing and Samza: https://www.youtube.com/playlist?list=PLZDyxA22zzGyNgtBMUIXAgIaO5Ok3PR-x A few topics c

Re: Looking for YARN Alternative

The standalone mode was introduced for this exact reason for customers who don’t want to run YARN. Have you considered running Samza in stand-alone mode? In this mode, Samza is an embedded library - very similar to Kafka Streams. https://samza.apache.org/learn/documentation/latest/deployment/stan

Re: [VOTE] Migration of Samza git repo to gitbox.apache.org

+1 (binding). Thank you Pawas for driving this! On Wed, Jan 23, 2019 at 2:40 PM Xinyu Liu wrote: > +1 (binding). > > On Wed, Jan 23, 2019 at 2:39 PM Prateek Maheshwari > wrote: > > > +1 (binding) again > > > > - Prateek > > > > On Wed, Jan 23, 2019 at 11:50 AM Pawas Chhokra > > wrote: > > > >

Re: Question about too much log on resource manager and node manager

Hi Qi, >> I want to know is this normal and how to avoid such log. >From Samza's perspective, I would treat these messages as red-herrings as long as your containers are up and running. The INFO messages in the ResourceManager log correspond to resources requested by your application. On

Re: Alerting and Monitoring Samza Checkpointing?

Hi Jeremiah, +1 to what Tom said. Samza currently does not rely on Kafka consumer's checkpointing behavior and exposes its own notion of a "lag". This is reported as a per-partition metric under KafkaSystemConsumerMetrics#messagesBehindHighWatermark

Re: [VOTE] [RESULT] Apache Samza 1.0.0 RC4

dy. > > Cheers! > > -Yi > > On Tue, Nov 6, 2018 at 9:33 PM Jagadish Venkatraman < > jagadish1...@gmail.com> > wrote: > > > The vote of 1.0.0 has been more than 72 hours. We got +1 (binding)x 3 and > > +1 (non-binding) x 3 and no vetos. > > > > *

Release artifacts published for Samza 1.0.0

Hey everyone, Now that the vote has concluded, binaries for Samza 1.0.0 have been published to Maven.. I'll follow up with a blogpost on the release early next week. Note that Samza 1.0.0 has a dependency on Kafka 0.11.2, which does not support

[VOTE] [RESULT] Apache Samza 1.0.0 RC4

llo-samza (High level and low level jobs). > > +1 > > > > On Wed, Oct 31, 2018 at 7:15 PM Jagadish Venkatraman < > jagad...@apache.org> > > wrote: > > > > > Hi all, > > > > > > This is a call for a vote on a release of Apache Samza

Re: [VOTE] Apache Samza 1.0.0 RC4

; On Fri, Nov 2, 2018 at 2:39 PM Boris S wrote: > > > > ran check-all and integration tests. All passed. > > verified signatures. > > +1 > > > > On Wed, Oct 31, 2018 at 7:15 PM Jagadish Venkatraman < > jagad...@apache.org> > > wrote: > >

[VOTE] Apache Samza 1.0.0 RC4

Hi all, This is a call for a vote on a release of Apache Samza 1.0.0. Thanks to everyone who has contributed to this release. The release candidate can be downloaded from here: http://home.apache.org/~jagadish/samza-1.0.0-rc4/ The release candidate is signed with pgp key AF81FFBF, which can be f

Re: [VOTE] Apache Samza 1.0.0 RC2

Boris, Do users have the option to switch to use the "old" Kafka consumer if they encounter any issue with the "new" consumer?. If not, should we pull in https://github.com/apache/samza/pull/755? It is my understanding that PR-755 adds support for this. Thanks, Jagadish On Tue, Oct 23, 2018 at 2

Re: [VOTE] Apache Samza 1.0.0 RC0

I ran the integration test and encountered this failure: 2018-10-22 15:58:43,687 zopkio.test_runner [INFO] test_samza_jobfailed 2018-10-22 15:58:43,688 zopkio.test_runner [INFO] ['AssertionError: Job (negate_number) appears not to have started. Expected to see a log line matching regex: .*Subm

Re: [VOTE] SEP-15: New Runtime Context API

+1 (binding) from my side. LGTM On Mon, Oct 15, 2018 at 10:44 AM Prateek Maheshwari wrote: > +1 (non-binding) for these changes. > > (Resending from a non-LI email due to email delivery issues) > > - Prateek > On Fri, Oct 12, 2018 at 3:28 PM Cameron Lee wrote: > > > > Hi all, > > > > SEP-15 ha

Re: [VOTE] SEP-14: System and Stream Descriptors

+1 binding from my side, thanks! This is a great addition to Samza-1.0 On Fri, Oct 12, 2018 at 12:30 PM Prateek Maheshwari wrote: > Hi folks, > > Now that SAMZA-1804 has been implemented and reviewed, we've updated > SEP-14 with the latest APIs and design decisions. > > Please vote for accepting

Re: [VOTE] SEP-13: unified ApplicationDescriptor and ApplicationRunner APIs for high and low- level APIs in YARN and standalone deployment

+1(binding) from my side. This is a great overhaul of our APIs. Thanks On Monday, October 15, 2018, Xinyu Liu wrote: > +1 (binding). Thanks for the effort to making our API consistent. > > Thanks, > Xinyu > > On Fri, Oct 12, 2018 at 12:32 PM Prateek Maheshwari > wrote: > > > +1 (non-binding) f

Re: Path to Java 11

/java-platform-group/update-and-faq- > on-the-java-se-release-cadence > > > > > Jeremiah Adams > Software Engineer > www.helixeducation.com > Blog | Twitter | Facebook | LinkedIn > > > From: Jagadish Venkatraman > Sent:

Re: Path to Java 11

Hi Jeremiah, Most of the users in the Samza community currently use Java 8. We have not investigated/tested Java-11 support just yet since it's fairly bleeding edge. Are you aware of any known incompatibilities / pitfalls? Thanks, Jagadish On Mon, Sep 10, 2018 at 2:52 PM, Jeremiah Adams wrote

Re: Samza/Yarn cluster having issue with OffsetOutOfRangeException

Hi Will, Is the topic in question your change-log topic or the checkpoint-topic or one of your inputs? (My understanding from reading this is its your checkpoint) Can you please attach some more surrounding logs? Thanks, Jagadish On Mon, Aug 20, 2018 at 6:16 AM, Will Schneider wrote: > Hell

Re: Using dynamodb as source stream

Hi Anoop, Currently, Samza has built-in support for consuming data from AWS Kinesis. As a short-term solution, if you want to consume from "Dynamo DB Streams", you can certainly pipe it to Kinesis and write a Samza job that consume from it. Please see: Consuming from Kinesis

Re: Question regarding Samza's Kafka consumer

Hi Thomas, >> Has Samza been tested against newer broker versions using the new message format, and if so does it have a significant performance impact? We have not benchmarked Kafka broker performance with the new message format. Any benchmarking may not be reliably reproducible since there are

Re: MessageCollector and WindowableTask

; Software Engineer > www.helixeducation.com > Blog | Twitter | Facebook | LinkedIn > > > From: Jagadish Venkatraman > Sent: Tuesday, June 5, 2018 10:22 AM > To: dev@samza.apache.org > Subject: Re: MessageCollector and WindowableTask > > Hey Jeremiah, > > >&g

Re: MessageCollector and WindowableTask

Hey Jeremiah, >> Please only use that MessageCollector object for sending messages, and don't use it outside of the call to window(). Why MessageCollector should not be used outside the call to window() in a WindowableTask? The collector and coordinator instances are expected to be used within a

Re: [VOTE] Apache Samza 0.14.1 RC2

- Verified the signature in the release tag. - Ran check-all.sh successfully - Ran integration-tests.sh successfully - Executed all SQL queries successfully in the tutorial at http://samza.apache.org/learn/tutorials/0.14/samza-sql.html

Re: [VOTE] SEP-12: Integration Test Framework

Thanks Sanil for the proposal. This will go a long way in simplifying testing of Samza applications. +1 (binding) On Thu, May 17, 2018 at 9:45 AM, Daniel Nishimura wrote: > +1 > > Looks great! > > On Thu, May 17, 2018 at 9:08 AM, Xinyu Liu wrote: > > > +1 > > > > The proposal looks great to

Re: [Discuss] Samza 0.14.1 release

Xinyu, This comes up with an exciting lineup of features. +1 (binding) from my side. Tom, >> These all look like excellent changes to me, I'd be happy to have more stability in Standalone! We are actively investing in standalone and have fixed some edge-cases that surfaced when running it at sca

Re: [VOTE] SEP-11: Host affinity in standalone.

, Jagadish Venkatraman wrote: > Let's extend the vote to Monday 11:59PM PST. > > On Thu, Apr 5, 2018 at 5:05 PM, xinyu liu wrote: > >> +1 (binding). Look forward to the implementation. >> >> Xinyu >> >> On Wed, Apr 4, 2018 at 2:43 PM, Yi Pan wrote:

Re: [VOTE] SEP-11: Host affinity in standalone.

Let's extend the vote to Monday 11:59PM PST. On Thu, Apr 5, 2018 at 5:05 PM, xinyu liu wrote: > +1 (binding). Look forward to the implementation. > > Xinyu > > On Wed, Apr 4, 2018 at 2:43 PM, Yi Pan wrote: > > > +1 (binding). Thanks for the revisions! > > > > -Yi > > > > On Wed, Apr 4, 2018 at

Re: Old style "low level" Tasks with alternative deployment model(s)

and renders in our web-page: https://samza.apache.org/learn/documentation/0.14/aws/kinesis.html Best, Jagdish On Tue, Mar 20, 2018 at 11:24 AM, Tom Davis wrote: > What format would that be best in? Happy to put something together this > weekend as well. > > Jagadish Venkatraman writ

Re: Old style "low level" Tasks with alternative deployment model(s)

;> >> >> -Original Message- >> From: Thunder Stumpges >> Sent: Monday, March 19, 2018 20:58 >> To: dev@samza.apache.org >> Cc: Jagadish Venkatraman ; t...@recursivedream.com; >> yi...@linkedin.com; Yi Pan >> Subject: RE: Old style "low le

Re: Stream Processing Meetup @LinkedIn tomorrow (3/21)

Hey Tom, No sign-ups are necessary to watch the live-stream. The sign-ups are for in-person attendance. Hoping you'll join us virtually :-) Thanks, Jagdish On Tue, Mar 20, 2018 at 10:20 AM, Tom Davis wrote: > To access the live stream do we need to sign up? I don't want to take a > spot from

Re: Old style "low level" Tasks with alternative deployment model(s)

pass, (or pull from an env variable and > add to the config) ? I am assuming it is my responsibility to ensure that > each instance is started with a unique container ID..? > > > > I am getting started on the above (Approach 2.), and looking closer at the > code so I may have my o

Re: Streams Meetup at LinkedIn Sunnyvale 3/21/2018

Hey folks, We have an exciting line-up of tech-talks on use-cases for Apache Samza and Kafka! A friendly reminder to RSVP at https://www.meetup.com/Stream- Processing-Meetup-LinkedIn/events/248309045/ Best, Jagdish On Thu, Mar 1, 2018 at 3:16 PM, Jake Maes wrote: > Hi Folks, > > We're going

Re: Old style "low level" Tasks with alternative deployment model(s)

le "low level" Tasks with alternative deployment > model(s) > > Hey there! > > You are correct that this is focused on the higher-level API but doesn't > preclude using the lower-level API. I was at the same point you were not > long ago, in fact, and had a very prod

Re: Question about custom StreamJob/Factory

guration on my part or if it's just not designed to be used > inside a Docker container. > > I'm sure I'll have more as time goes on. Thanks for all the help so far! > > [1]: https://gist.github.com/tdavis/3398dc66fecb948eca53af0142a0c331 > > Jagadish Ve

Re: Question about custom StreamJob/Factory

Factory`, `TaskFactoryUtil#createTaskFactory` only returns > `StreamTaskFactory` when passed a `StreamApplication`. The crux being: I > don't see a great story for operators that need to, e.g., make database > calls for each message or perform another blocking operation. > > An

Re: Question about custom StreamJob/Factory

s > in my cluster? > > Okay, enough questions for now! I hope to publish the repository > tomorrow and would love to get some more experienced eyes on it to learn > all the ways I screwed up. I'll post to this thread again with a link. > > > Thanks, > > Tom > &g

Re: Question about custom StreamJob/Factory

+Yi Hi Tom, Thank you for your feedback on Samza's architecture. Pluggability has been a differentiator that has enabled us to support a wide range of use-cases - from stand-alone deployments to managed services, from streaming to batch inputs and integrations with various systems from Kafka, Kin

Re: No Metrics Emitted in new High-Level API

Can you turn on DEBUG logging for the MetricsSnapshotReporter class and re-attach the logs? I cannot observe any obvious red-flag from your logs. On Fri, Jan 26, 2018 at 7:13 PM, Srujun Thanmay Gupta wrote: > Hello Yi, > > The logs are less than 100KB each, I'm not sure if that's too large for t

Re: No Metrics Emitted in new High-Level API

Hi Srujun, Metrics reporting should be supported independent of the deployment model. Can you please attach your container log file? Are you seeing log lines from MetricsSnapshotReporter: https://github.com/apache/samza/blob/master/samza-core/src/main/scala/org/apache/samza/metrics/reporter/Metri

Re: Welcome Xinyu as new Samza PMC!

Big Congrats Xinyu. Thanks for your continued contributions to all aspects of the project! On Wed, Jan 17, 2018 at 10:36 AM, Wei Song wrote: > Congrats, Xinyu! > > -- > Thanks > -Wei > > > On 1/17/18, 10:35 AM, "Navina Ramesh" wrote: > > Congratulations, Xinyu! > Thanks for all your con

Re: Samza Freestyle Build

Thanks Chris for making these changes for us. Appreciate it. Best, Jagdish On Thu, Jan 11, 2018 at 5:05 PM, Chris Thistlethwaite wrote: > Greetings! > > I was working on cleaning up some builds on the Jenkins Windows nodes > and noticed your build failing. When I inspected the output, I noticed

[RESULT][VOTE] Apache Samza 0.14 RC5

The vote of 0.14.0 has been more than 72 hours and we got +1 (binding) x 3 and +1 (non-binding) x 4. Samza 0.14 officially passed the VOTE! Thanks all for helping validate this release! -- Jagadish V, Graduate Student, Department of Computer Science, Stanford University

Re: [VOTE] Apache Samza 0.14.0 RC5

+1 (binding) Verified the RC. Ran *check-all.sh* and integration tests successfully on OS X. Thanks Xinyu, and everyone for driving Samza-0.14! On Thu, Dec 28, 2017 at 3:22 AM, Yi Pan wrote: > +1 (binding). > > Verified the signature and MD5 > Ran ./bin/check-all.sh on OSX > Ran integration tes

Re: [RESULT][VOTE] Apache Samza 0.14.0 RC3

Thank you everyone for the effort, and to Xinyu for shepherding Samza 0.14! This includes an exciting line-up of features including - SQL support, Kinesis integration and EventHub support. On Thu, Dec 21, 2017 at 11:03 AM, xinyu liu wrote: > The vote of 0.14.0 RC3 has been more than 72 hours an

Re: [VOTE] Apache Samza 0.14.0 RC3

LGTM. I verified signatures and ran all tests on OsX. Additionally, *check-all.sh* succeeded. +1 (binding) On Thu, Dec 21, 2017 at 1:07 AM, Yi Pan wrote: > +1 binding > > Verified git tag and source signatures > > Ran check-all.sh on OSX > Ran integration tests from OSX > > Thanks for push

Re: [DISCUSS] Samza 0.14.0 release

+1 from my side. Thank you Bharath for driving the release! On Mon, Nov 27, 2017 at 9:50 AM, Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > Hi all, > > > > We have added couple of major features to master since 0.13.1 that warrants > a major release. > > Within LinkedIn, some o

Re: Comparison between Samza and Kafka Streams

Thanks for the feedback Giridhar! We'll add a comparison with KStreams there as well. Roughly, the two are similar - The design of Samza certainly influenced what went into Kafka Streams. However, here are some key differences: - Support for non-Kafka source and sink natively: Samza has native c

Re: Exit code 248 from YARN and InterruptedException

Hi Xiaochuan, Is there any exception in your custom SystemConsumer? Or, are you saying that your consumer is being interrupted externally? The entire log will be helpful. Thanks, Jagdish On Mon, Oct 30, 2017 at 9:14 AM, XiaoChuan Yu wrote: > Hi, > > I'm trying to debug some problem with a Sam

Re: Per task/topic checkpoint?

In Samza, the logical unit of processing (and hence, checkpointing) is a task. Hence, you cannot selectively checkpoint SSPs within a task. However, you can configure how you group your SSPs into tasks by choosing a Grouper. If you want to control checkpointing at the granularity of an SSP, then y

Re: permission to create a SEP page

I provisioned edit permissions on the wiki. Thanks, Jagdish On Tue, Oct 10, 2017 at 1:24 PM Aditya wrote: > Sounds good. Thanks Jagadish. I do not seem to have edit permissions. > > On Tue, Oct 10, 2017 at 12:30 PM, Jagadish Venkatraman < > jagadish1...@gmail.com> wrot

Re: permission to create a SEP page

Hi Aditya, We usually create a SEP page first, and allow contributors to edit it later (instead of provisioning create permissions). I just created https://cwiki.apache.org/confluence/display/SAMZA/SEP-9+Add+a+Kinesis+SystemConsumer+and+SystemProducer Can you please let me know if you are able t

Re: [VOTE] SEP-8: Add in-memory system consumer & producer

LGTM , +1 on the overall design. This will drastically improve testing of Samza applications! -- Jagdish On Thu, Sep 14, 2017 at 1:23 AM, Yi Pan wrote: > Hi, Bharath, > > Overall looks good! I have the following comments: > > i) Question on the Type of IME + data partition: > > How do we enforc

Re: Deploying Samza Jobs Using S3 and YARN on AWS

, due to a dependency conflict on the AWS SDK between our code and > "hadoop-aws", we can't actually include it. > We are now planning to make use of HTTP FS instead. > > On Fri, Sep 15, 2017 at 2:45 PM Jagadish Venkatraman < > jagadish1...@gmail.com> > wrote

Re: Connection timed out error while installing "Hello Samza"

> Waiting for zookeeper to start... > ^C > [root@csadatlak-wc-a1t hello-samza]# bin/grid start yarn > EXECUTING: start yarn > starting resourcemanager, logging to /home/sanant003c/hello-samza/ > deploy/yarn/logs/yarn-root-resourcemanager-csadatlak-wc- > a1t.sys.comcast.net.out >

Re: Samza Job Slow to Restart

Hi Xiaochuan, >> What does that loop do exactly? Most of what the run-loop does is documented in https://samza.apache.org/learn/documentation/0.9/container/event-loop.html >> We are running into a problem where it seems to take a very long time to restart a Samza job. Some follow-up questions,

  1   2   3   4   5   >