ilable in the Samza 1.2 release:
> https://samza.apache.org/releases/1.2.0
>
> Thanks,
> Prateek
>
> On Tue, Mar 12, 2019 at 9:11 AM Tom Davis wrote:
> >
> > Bummer! Yes, that works. The phase from "pending" to "running" is a
> > nice-to-have at
this.
- Prateek
On Mon, Mar 25, 2019 at 7:51 AM Tom Davis wrote:
I am using Samza 1.0, yes. The stacktrace is:
> 19:24:49.326 [Samza StreamProcessor Container Thread-0] ERROR
> org.apache.samza.processor.StreamProcessor - Container:
> org.apache.samza.container.SamzaContainer
r has shut down. We fixed a similar standalone bug
recently, are you already using Samza 1.0.
If this is reproducible / happens again, a thread dump + logs would also be
very helpful for debugging and verifying if the issue is already fixed.
Thanks,
Prateek
On Fri, Mar 22, 2019 at 7:23 AM Tom Da
state: FAILED
This doesn't cause the Pod to be killed, though, so we just silently
stop processing events. How do you determine the number of expected
containers? Or are you speaking of containers in terms of YARN and not
Samza processors?
- Prateek
On Fri, Mar 15, 2019 at 7:26 AM Tom Davis
I'm using the LocalApplicationRunner and had added a liveness check
around the `status` method. The app is running in Kubernetes so, in
theory, it could be restarted if exceptions happened during processing.
However, it seems that "container failure" is divorced from "app
failure" because the app
be able to include SAMZA-2116 in the upcoming 1.1
release due to time constraints. It'll have to go in to the 1.2 release,
which will tentatively be in June. Does that still work for you?
Thanks,
Prateek
On Thu, Feb 28, 2019 at 2:16 PM Tom Davis wrote:
Thanks, Prateek! Yes, the workaro
this in the next Samza release, which should be
mid to late next month.
In the mean time, the workaround would be to keep all of this functionality
in a sink function. Does this work for you?
Thanks,
Prateek
On Wed, Feb 27, 2019 at 2:54 PM Tom Davis wrote:
Prateek Maheshwari writes:
> H
ing multiple chained operators more heavily.
Since both sendTo and sink are terminal operators (void return type), I
don't think you'll be able to easily get around this. Let me discuss this
with the team and get back to you with a workaround / fix.
Thanks a lot! <3
Thanks,
Pr
s for customizing system components?
Cheers,
Tom
Tom Davis writes:
Hey folks!
We have noticed some inconsistencies in message ordering when running a
StreamApplication that calls two separate `map` functions over an input
and sends results to the same output. I have attached my Execution Plan,
b
Hey folks!
We have noticed some inconsistencies in message ordering when running a
StreamApplication that calls two separate `map` functions over an input
and sends results to the same output. I have attached my Execution Plan,
but the gist is that the first `map` function marks a thing as "pendi
I'll chime in to say I am running the standalone mode successfully in
Kubernetes. The ZK coordinator is very useful in this context as you can
partition a topic for max *desired* parallelism without continually
running that many nodes. You could also use the "operator" pattern in
Kube to create a
is> feature and I create the ticket
> https://issues.apache.org/jira/browse/SAMZA-2075 to track this. We
> will see> whether we can get it there in the next release.
>
> Thanks,
> Xinyu
>
>
>
>
>
> On Thu, Jan 17, 2019 at 6:11 AM Tom Davis
> wrote
understand the problem better?
Thanks
Sanil
On Tue, 8 Jan 2019 at 17:28, Tom Davis wrote:
I am in the process of updating a project to 1.0 and spent today debugging
a
rather odd test failure. When using input/output streams with IntegerSerde,
things worked fine -- however, using LongSerde
I am in the process of updating a project to 1.0 and spent today debugging a
rather odd test failure. When using input/output streams with IntegerSerde,
things worked fine -- however, using LongSerde, every message value was 0! I
eventually found that InMemorySystemDescriptor#getInputDescriptor ig
Have you looked into KafkaSystemConsumerMetrics? Is the meaning of "lag"
there different from what you mean?
Jeremiah Adams writes:
We are replacing a node.js app that consumed topics on a Kafka cluster with
Samza jobs. We use kafka-offsets to trigger alerts based on message lag. e.g.,
messa
Hey Rayman,
I can't speak for anybody else but I'd love a Samza 1.0 release!
Ideally
it would coincide with Kafka 1.0 support but I suppose major
features
and a ton of bug fixes will have to do ;)
rayman preet writes:
Hi all,
We have added a number of major features and changes to mast
I am having one hell of a head-scratching issue here. I am trying to
write a test that uses LocalApplicationRunner to run an app configured
to use the InMemorySystemFactory. But when I run my test,
org.apache.samza.system.inmemory.InMemorySystemFactory is not found! Theweird
thing is, I see the `.
>>
>>>>> Any timeline on getting Kafka 0.11 upgrade in?
>>>
>>> We have historically been doing Kafka upgrades on major releases
>>> (instead>>> of
>>> minor ones). So, it will likely be in 0.15.0. I'll provide an
&
These all look like excellent changes to me, I'd be happy to have more
stability in Standalone!
Any timeline on getting Kafka 0.11 upgrade in? It seems to have been
done a while, but waiting for 0.15 I presume.
Best,
Tom
On Mon, Apr 30, 2018, at 3:10 PM, Xinyu Liu wrote:
> Hi, All,
>
> We have
ssage-
From: Jagadish Venkatraman [mailto:jagadish1...@gmail.com]
Sent: Tuesday, March 20, 2018 11:36
To: Tom Davis
Cc: Prateek Maheshwari ; dev@samza.apache.org;
yi...@linkedin.com; Yi Pan
Subject: Re: Old style "low level" Tasks with alternative deployment model(s)
Hi Tom,
H
s with other nice tools for
> >> monitoring offsets, lag, etc
>
> Samza exposes<https://github.com/apache/samza/blob/master/
> samza-kafka/src/main/scala/org/apache/samza/system/kafka/
> KafkaSystemConsumerMetrics.scala> the below metrics for lag-monitoring:
> - Th
To access the live stream do we need to sign up? I don't want to take a
spot from someone able to be physically present.
Yi Pan writes:
Hi, all,
Just a reminder that we will host another stream processing meetup
@LinkedIn tomorrow at 6pm:
https://www.meetup.com/Stream-Processing-Meetup-Linked
Hey there!
You are correct that this is focused on the higher-level API but doesn't
preclude using the lower-level API. I was at the same point you were not
long ago, in fact, and had a very productive conversation on the list:
you should look for "Question about custom StreamJob/Factory" in the
a tutorial on how to use high-level API and
Kubernetes. I'd be
happy to help with design/code-reviews.
On Sun, Jan 28, 2018 at 12:17 PM, Tom Davis wrote:
As promised, here's the link to the repository:
https://github.com/sonian/samza-kubernetes
The section "Your Job Image"
know should you need any further help. Here's an example to
get you started:
https://github.com/apache/samza-hello-samza/tree/master/src/main/java/samza/examples/wikipedia/application
Best,
Jagdish
On Sat, Jan 27, 2018 at 8:54 AM, Tom Davis wrote:
Hi there! First off, thanks for the conti
/tree/master/src/main/java/samza/examples/wikipedia/application
Best,
Jagdish
On Sat, Jan 27, 2018 at 8:54 AM, Tom Davis wrote:
Hi there! First off, thanks for the continued work on Samza -- I looked
into many DC/stream processors and Samza was a real standout with its
smart architecture
Hi there! First off, thanks for the continued work on Samza -- I looked
into many DC/stream processors and Samza was a real standout with its
smart architecture and pluggable design.
I'm working on a custom StreamJob/Factory for running Samza jobs on
Kubernetes. Those two classes are pretty strai
27 matches
Mail list logo
