Re: Consumer service that supports retry with exponential backoff

2017-10-10 Thread Michal Michalski
Hi John, It doesn't seem like you care for the ordering (since you're using multiple "fallback" topics that are processed in parallel if I understood you correctly), but the alternative would be to implement the backoff using the same topic and consumer. We're using the "pausing" feature of the

Re: Consumer service that supports retry with exponential backoff

2017-10-09 Thread Stephen Powis
Pretty much the implementation that we're using, exception using partitions within a single topic because we have a low number of failures passing through. Granted the first partition will take the brunt of the load vs the higher order partitions. On Tue, Oct 10, 2017 at 7:30 AM, Steven

Re: Consumer service that supports retry with exponential backoff

2017-10-09 Thread Steven Schlansker
> On Oct 9, 2017, at 2:41 PM, John Walker wrote: > > I have a pair of services. One dispatches commands to the other for > processing. > > My consumer sometimes fails to execute commands as a result of transient > errors. To deal with this, commands are retried after

Fwd: Consumer service that supports retry with exponential backoff

2017-10-09 Thread John Walker
I have a pair of services. One dispatches commands to the other for processing. My consumer sometimes fails to execute commands as a result of transient errors. To deal with this, commands are retried after an exponentially increasing delay up to a maximum of 4 times. (Delays: 1hr, 2hr, 4hr,