RE: Restructure data in GlobakKTable

obalKTable (other than the join which is what it's there for in the first place). (This only seems practical because the amount of data I'm handling here is quite small.) But now the requirements have changed and I'm struggling with handling deletions in the new requirements ... such is life ...

RE: How do I tell Kafka Streams not to repartition?

Thanks. Tim Ward -Original Message- From: Matthias J. Sax Sent: 13 August 2019 08:23 To: users@kafka.apache.org Subject: Re: How do I tell Kafka Streams not to repartition? Atm, it's not possible to tell Kafka Streams that repartitioning is not necessary after a key-changing operation

RE: KSTREAM-AGGREGATE-STATE-STORE persistence?

Thanks! Tim Ward -Original Message- From: Bruno Cadonna Sent: 12 August 2019 14:18 To: users@kafka.apache.org Subject: Re: KSTREAM-AGGREGATE-STATE-STORE persistence? Hi Tim, Kafka Streams guarantees at-least-once processing semantics by default. That means, a record is processed (e.g

KSTREAM-AGGREGATE-STATE-STORE persistence?

the aggregation process idempotent? I don't see this explained in the JavaDoc for either Aggregator (or Reducer, where presumably it also applies). Tim Ward This email is from Origami Energy Limited. The contents of this email and any attachment are confidential to the intended recipient(s). If you

RE: How do I tell Kafka Streams not to repartition?

a problem I've discovered with persistence following a crash in the middle of aggregation, which I'll post separately. Tim Ward -Original Message- From: Boyang Chen Sent: 09 August 2019 23:31 To: users@kafka.apache.org Subject: Re: How do I tell Kafka Streams not to repartition? In case

RE: How to start a stream from only new records?

of what offset it committed last time it ran. Tim Ward -Original Message- From: Boyang Chen Sent: 09 August 2019 17:23 To: users@kafka.apache.org Subject: Re: How to start a stream from only new records? Hey Tim, if you are talking about avoid re-processing data and start consumption from

How do I tell Kafka Streams not to repartition?

no need to repartition in this case, because all the data that should remain together in the same instance of the application will do so without repartitioning? (I appreciate that Streams can't know about the parent-child relationship unless I *do* tell it in some way.) Tim Ward This email is from

How to start a stream from only new records?

any backlog? Tim Ward This email is from Origami Energy Limited. The contents of this email and any attachment are confidential to the intended recipient(s). If you are not an intended recipient: (i) do not use, disclose, distribute, copy or publish this email or its contents; (ii) please contact

Restructure data in GlobakKTable

don't know how to do this if using StreamsBuilder.stream().) What am I missing please? Tim Ward This email is from Origami Energy Limited. The contents of this email and any attachment are confidential to the intended recipient(s). If you are not an intended recipient: (i) do not use, disclose

Java consumer error handling on DNS lookup failure

) ~[?:1.8.0_212] at org.apache.kafka.common.network.Selector.doConnect(Selector.java:233) ~[kafka-clients-2.0.0.jar:?] ... 19 more Tim Ward This email is from Origami Energy Limited. The contents of this email and any attachment are confidential to the intended

RE: Prioritized Topics for Kafka

bly have a look at it when they get bored"), of which there could be a long queue. Urgent alerts (a phone call telling someone "you need to do this now") should be processed before non-urgent alerts (a phone call telling someone "FYI, such and such is going to happen in a couple o

RE: Kafka running on AWS - how to retain broker.id on new instance spun-up in-place of instance/broker failed

Isn't that what stateful sets are for, giving you a predictable pod name from which you can derive the ID? Tim Ward -Original Message- From: Srinivas Rapolu Sent: 15 November 2018 17:42 To: users@kafka.apache.org Subject: Re: Kafka running on AWS - how to retain broker.id on new

RE: Terminate Streams application from within Transformer?

That works, thanks. Tim Ward -Original Message- From: Bill Bejeck Sent: 21 September 2018 15:06 To: users@kafka.apache.org Subject: Re: Terminate Streams application from within Transformer? Hi Tim, I wouldn't recommend System.exit(), as it won't give streams a chance to go through

Terminate Streams application from within Transformer?

rently being processed, so that on application restart it re-processes this message and everything works fine. How should I do this? System.exit() or something more sophisticated? Tim Ward The contents of this email and any attachment are confidential to the intended recipient(s).

RE: Understanding default.deserialization.exception.handler

Thanks. No, it's not that big a deal now that I understand it, but as I'd had to spend a fair amount of time working out what was going on I thought I'd flag it up in case it wasn't known about. Tim Ward -Original Message- From: Matthias J. Sax Sent: 14 September 2018 19:57 To: users

RE: Timing state changes?

ng on there, in relation to how I can crash the application and restart it and all the state is preserved with my having to write 0 characters of code to achieve this - I think I'm going to like Kafka Streams. Thanks. Tim Ward The contents of this email and any attachment are confidential to the inte

Understanding default.deserialization.exception.handler

t any good messages having been read after the bad ones. Have I understood this correctly? If so, is this correct behaviour as designed? Is it documented to that level of detail? Tim Ward The contents of this email and any attachment are confidential to the intended recipient(s). If you are not a

RE: Timing state changes?

to worry about it? (I'll be deploying to Kubernetes, and applications going away and restarting at random seems to be a fact of life there.) Tim Ward The contents of this email and any attachment are confidential to the intended recipient(s). If you are not an intended recipient: (i) do not u

RE: Timing state changes?

Thanks - I've been diverted to other things but will come back to this shortly. Tim Ward -Original Message- From: John Roesler Sent: 06 September 2018 19:38 To: users@kafka.apache.org Subject: Re: Timing state changes? Hi Tim, From your spec, I think that Kafka Streams has several

Timing state changes?

s involving updating a database table and then polling it on a cron job are really the sort of thing we're trying to get away from here.) Tim Ward The contents of this email and any attachment are confidential to the intended recipient(s). If you are not an intended recipient: (i) do not use, d