This solution will block receiving polling thread for 15 minutes. Not good.
What should we do if a topic has messages that should be processed immediately and delayed messages at the same time? *Pavel Molchanov* On Wed, May 22, 2019 at 2:41 PM Peter Bukowinski <pmb...@gmail.com> wrote: > There is no out-of-the-box way to tell a consumer to not consume an offset > until it is x minutes old. Your best bet is encode the creation time into > the message themselves and add some processing logic into your consumer. > Let’s assume your topic has a single partition or your partitions are keyed > to guarantee message order. Your consumer could work like this in > pseudo-code: > > consumer loop: > consume message > if (current time - message.timestamp) >= 15 minutes > process message > else > sleep 15 minutes - (current time - message.timestamp) > process message > > Since the messages enter the topic in the order they were published, > pausing on the current offset should never cause a bottleneck on the later > messages. If you fall behind, the greater than or equal to logic will > prevent your consumer from pausing until it has caught up to your desired > delay. > > This is a simplified scenario that may or may not map to your production > use case, though. > > — > Peter > > > > On May 22, 2019, at 11:12 AM, Pavel Molchanov < > pavel.molcha...@infodesk.com> wrote: > > > > Andrien, > > > > Thank you for asking this question! I have the same problem and wanted to > > ask the same question. I hope that someone will answer soon. > > > > *Pavel Molchanov* > > > > > > > > On Wed, May 22, 2019 at 9:54 AM Adrien Ruffie <adrye...@gmail.com> > wrote: > > > >> Hello all, > >> > >> I have a specific need and I don't know if a generic solution exist ... > >> maybe you can enlighten me > >> > >> I need to delay each sended message about 15 mins. > >> Example > >> Message with offset 1 created at 2:41PM by the producer and received by > the > >> consumer at 2:56PM > >> Message with offset 2 created at 2:46PM by the producer and received by > the > >> consumer at 3:01PM > >> Message with offset 3 created at 2:46PM by the producer and received by > the > >> consumer at 3:01PM > >> Message with offset 4 created at 3:01PM by the producer and received by > the > >> consumer at 3:16PM > >> and so forth ... > >> > >> any option, mechanism, producer/consumer implementations already exist ? > >> > >> Thank a lot and best regards > >> > >> Adrian > >> > >
