Re: "send to" ordering is inconsistent

Oh, excellent, thank you! I changed jobs and no longer have the great joy of programming in Clojure, but I have seen my co-worker posting here so I'm sure it'll get sorted all the same. On Sun, Jun 23, 2019, at 8:10 PM, Prateek Maheshwari wrote: > Just FYI Tom, this fix is available in the Samza

Re: "send to" ordering is inconsistent

Just FYI Tom, this fix is available in the Samza 1.2 release: https://samza.apache.org/releases/1.2.0 Thanks, Prateek On Tue, Mar 12, 2019 at 9:11 AM Tom Davis wrote: > > Bummer! Yes, that works. The phase from "pending" to "running" is a > nice-to-have at the moment (operations don't take long

Re: "send to" ordering is inconsistent

Bummer! Yes, that works. The phase from "pending" to "running" is a nice-to-have at the moment (operations don't take long enough yet to warrant the extra state) so we just removed it for the time being. Prateek Maheshwari writes: Hi Tom, It looks like we won't be able to include SAMZA-2116 i

Re: "send to" ordering is inconsistent

Hi Tom, It looks like we won't be able to include SAMZA-2116 in the upcoming 1.1 release due to time constraints. It'll have to go in to the 1.2 release, which will tentatively be in June. Does that still work for you? Thanks, Prateek On Thu, Feb 28, 2019 at 2:16 PM Tom Davis wrote: > Thanks,

Re: "send to" ordering is inconsistent

Thanks, Prateek! Yes, the workaround will be fine for the time being. Thank you again! Prateek Maheshwari writes: Hi Tom, Thanks for reporting this. I created a ticket (SAMZA-2116 ) to make the required API changes. We'll include this in the n

Re: "send to" ordering is inconsistent

Hi Tom, Thanks for reporting this. I created a ticket (SAMZA-2116 ) to make the required API changes. We'll include this in the next Samza release, which should be mid to late next month. In the mean time, the workaround would be to keep all of th

Re: "send to" ordering is inconsistent

Prateek Maheshwari writes: Hi Tom, I'm assuming that the two sub-DAGs you're talking about are the two Map -> Send To chains acting on the "audit-report-requests" input and sending their results to the "audit-report-status" output. Yes, that's correct. Although processing within each T

Re: "send to" ordering is inconsistent

I was just looking to resolve this by using KafkaProducer's transaction API, assuming the underlying issue is just the async, unordered nature of actual producer sends. I turned on idempotent mode via producer config, which (as expected) reduced the failure rate but didn't eliminate it. Actual se

Re: "send to" ordering is inconsistent

Hi Tom, I'm assuming that the two sub-DAGs you're talking about are the two Map -> Send To chains acting on the "audit-report-requests" input and sending their results to the "audit-report-status" output. Although processing within each Task is in-order, the framework does not guarantee the order

"send to" ordering is inconsistent

Hey folks! We have noticed some inconsistencies in message ordering when running a StreamApplication that calls two separate `map` functions over an input and sends results to the same output. I have attached my Execution Plan, but the gist is that the first `map` function marks a thing as "pendi