Denis, +1 to the idea to get rid of priority.
Discovery should handle only messages used to update cluster state and topology. They are all small and have high priority. There is no real reason to use discovery for metrics and similar "readonly" features. Maybe, better case it to have special "discovery like" channel (with ring or analog) for metrics like messages which will not affect priority features like PME? Anyway, Why are fighting with duplicates inside the queue instead of fighting with new message initial creation while previous not yet processed on the cluster? On Fri, Jan 11, 2019 at 4:30 PM Denis Mekhanikov <dmekhani...@gmail.com> wrote: > I like the idea to make all messages be processed with equal priority. > It will make nodes with overgrown discovery message queues die more often, > though. But maybe, this is how it's supposed to work. > > Denis > > пт, 11 янв. 2019 г. в 16:26, Denis Mekhanikov <dmekhani...@gmail.com>: > > > Yakov, > > > > Sounds good. But there is a flaw in the procedure, that you described. > > If we have a TcpDiscoveryMetricsUpdateMessage in a queue, and a newer one > > arrives, then we will consider the existing one obsolete and won't > process > > it. The newest metrics update message will be moved to the queue's tail, > > thus delaying the moment, when it will be processed. > > So, we may never come to a point, when an actual > > TcpDiscoveryMetricsUpdateMessage is processed. > > But if we replace an existing message with a newer one, and save the old > > position in a queue, then this approach will work. It will require a more > > complex synchronization, though, so it will still lead to some overhead. > > > > How big the message worker's queue may grow until it becomes a problem? > If > > it's 20 elements max, then linear time check is not that bad. > > BTW, RingMessageWorker#addMessage method checks for duplicating messages > > in the queue for some message types, including all custom discovery > > messages, which is done in linear time. > > > > Denis > > > > пт, 11 янв. 2019 г. в 00:54, Yakov Zhdanov <yzhda...@apache.org>: > > > >> Denis, what if we remove priority difference for messages and always add > >> new to the end of the queue? > >> > >> As far as traversing the queue - I don't like O(n) approaches =). So, > with > >> adding all messages to the end of the queue (removing prio difference) I > >> would suggest that we save latest 1st lap message and latest 2nd lap > >> message and process metrics message in message worker thread in queue > >> order > >> if they are latest and skip the otherwise. > >> > >> Does this make sense? > >> > >> --Yakov > >> > > >
