+1 to Dan's suggestion. In addition we should think about the following scenario: In certain cases, cleanQueue would cause stop to block if Receivers are not up or causing exception while applying the events. In that case events will not be deletd from queue and stop will block. Currently, it is controlled by a timeout, however we can give an option to the user it he/she still wants to force stop the queue in case queue is not drained in the a given time.
On Fri, Nov 15, 2019 at 6:18 AM Michael Oleske <mole...@pivotal.io> wrote: > > > > vsd stat in the gfs file > > > > Just here to say consider using a meter instead of stat as documented in > https://cwiki.apache.org/confluence/display/GEODE/How+to+add+a+Meter if > more than a log message is warrented. > > -michael > > On Thu, Nov 14, 2019 at 11:29 AM Nabarun Nag <n...@apache.org> wrote: > > > +1 to Dan's suggestion > > > > What about a log and a vsd stat in the gfs file which tells us if any > > cleanQueue commands were executed. > > > > > > Regards > > Nabarun Nag > > > > On Thu, Nov 14, 2019 at 10:27 AM Udo Kohlmeyer <u...@apache.com> wrote: > > > > > In addition... making is default has bigger consequences that we have > > > not thought about.. > > > > > > e.g if you purge an existing queue on start up.. is this the start up > of > > > the server node / GS Queue? Given that we have shared-nothing > > > architecture, purging *should* only be local and not cluster-wide... > I'd > > > be interested and see a proposal on this feature. > > > > > > --Udo > > > > > > On 11/14/19 10:24 AM, Jason Huynh wrote: > > > > +1 to Dan's suggestion > > > > > > > > On Thu, Nov 14, 2019 at 9:52 AM Dan Smith <dsm...@pivotal.io> wrote: > > > > > > > >> I'm ok with adding a --cleanQueue option. > > > >> > > > >> However, I don't think it should default to be true, since that > would > > be > > > >> changing the behavior of the existing command. It should default to > > > false. > > > >> > > > >> -Dan > > > >> > > > >> On Thu, Nov 14, 2019 at 9:28 AM Xiaojian Zhou <gz...@pivotal.io> > > wrote: > > > >> > > > >>> The --cleanQueue option is a similar idea as Barry's "DeadLetter" > > > spike. > > > >> I > > > >>> remembered that we decided not to do it. > > > >>> > > > >>> > > > >>> On Wed, Nov 13, 2019 at 11:41 PM Mario Ivanac > <mario.iva...@est.tech > > > > > > >>> wrote: > > > >>> > > > >>>> Hi, > > > >>>> > > > >>>> just to remind you on last question: > > > >>>> > > > >>>> what is your opinion on adding additional option in gfsh command > > > >> "start > > > >>>> gateway sender" > > > >>>> to control clearing of existing queues --cleanQueues. > > > >>>> > > > >>>> This option will indicate, when gateway sender is started, should > we > > > >>>> discard/clean existing queue, or should we use existing queue. > > > >>>> By default it will be to discard/clean existing queue. > > > >>>> > > > >>>> Best Regards, > > > >>>> Mario > > > >>>> ________________________________ > > > >>>> Šalje: Mario Ivanac <mario.iva...@est.tech> > > > >>>> Poslano: 8. studenog 2019. 13:00 > > > >>>> Prima: dev@geode.apache.org <dev@geode.apache.org> > > > >>>> Predmet: Odg: gateway sender queue > > > >>>> > > > >>>> Hi all, > > > >>>> > > > >>>> one more clarification regarding 3rd question: > > > >>>> > > > >>>> "* Could we add extra option in gfsh command "start gateway > > sender" > > > >>>> that allows to control queues reset (for instance > > > --cleanQueues)" > > > >>>> > > > >>>> This option will indicate, when gateway sender is started, should > we > > > >>>> discard/clean existing queue, or should we use existing queue. > > > >>>> By default it will be to discard/clean existing queue. > > > >>>> > > > >>>> Best Regards, > > > >>>> Mario > > > >>>> ________________________________ > > > >>>> Šalje: Mario Ivanac <mario.iva...@est.tech> > > > >>>> Poslano: 7. studenog 2019. 9:01 > > > >>>> Prima: Dan Smith <dsm...@pivotal.io>; dev@geode.apache.org < > > > >>>> dev@geode.apache.org> > > > >>>> Predmet: Odg: gateway sender queue > > > >>>> > > > >>>> Hi, > > > >>>> > > > >>>> thanks for answers. > > > >>>> > > > >>>> Some more details regarding 1st question. > > > >>>> > > > >>>> Is this behavior same (for serial and parallel gateway sender) in > > case > > > >>>> queue is persistent? > > > >>>> Meaning, should queue (persistent) be purged if we restart gateway > > > >>> sender? > > > >>>> > > > >>>> Thanks, > > > >>>> Mario > > > >>>> > > > >>>> ________________________________ > > > >>>> Šalje: Dan Smith <dsm...@pivotal.io> > > > >>>> Poslano: 5. studenog 2019. 18:52 > > > >>>> Prima: dev@geode.apache.org <dev@geode.apache.org> > > > >>>> Predmet: Re: gateway sender queue > > > >>>> > > > >>>> Some replies, inline: > > > >>>> > > > >>>> * During testing we have observed, different behavior in > > parallel > > > >> and > > > >>>>> serial gateway senders. In case we manually stop, than start > > gateway > > > >>>>> senders, for parallel gateway senders, queue is purged, but for > > > >> serial > > > >>>>> gateway senders this is not the case. Is this normal behavior or > > bug? > > > >>>>> > > > >>>> Hmm, I also think stop is supposed to clear the queue. I think if > > you > > > >> are > > > >>>> seeing that it doesn't clear the queue, that might be a bug. > > > >>>> > > > >>>> > > > >>>> > > > >>>>> * What happens with the queues when whole cluster is stopped > > and > > > >>>> later > > > >>>>> started (In our tests with persistent queues, the events are > kept)? > > > >>>>> > > > >>>> Persistent queues will keep all of the events when you restart. > > > >>>> > > > >>>> > > > >>>>> * Could we add extra option in gfsh command "start gateway > > > >> sender" > > > >>>>> that allows to control queues reset (for instance --cleanQueues)? > > > >>>>> > > > >>>> If stop does clear the queue, would this be needed? It might still > > be > > > >>>> reasonable - I've heard folks request a way to clear running > queues > > as > > > >>>> well. > > > >>>> > > > >>>> -Dan > > > >>>> > > > > > >
