@Becket, I've asked for review by Jun and Joel in the vote thread. Regarding the separate thread and port, I did talk about it in the rejected alternative design 1. Please let me know if you'd like more elaboration or moving it to the motivation, etc.
Thanks, Lucas On Wed, Aug 8, 2018 at 3:59 PM, Becket Qin <becket....@gmail.com> wrote: > Hi Lucas, > > Yes, a separate Jira is OK. > > Since the proposal has significantly changed since the initial vote > started. We probably should let the others who have already voted know and > ensure they are happy with the updated proposal. > Also, it seems the motivation part of the KIP wiki is still just talking > about the separate queue and not fully cover the changes we make now, e.g. > separate thread, port, etc. We might want to explain a bit more so for > people who did not follow the discussion mail thread also understand the > whole proposal. > > Thanks, > > Jiangjie (Becket) Qin > > On Wed, Aug 8, 2018 at 12:44 PM, Lucas Wang <lucasatu...@gmail.com> wrote: > > > Hi Becket, > > > > Thanks for the review. The current write up in the KIP won’t change the > > ordering behavior. Are you ok with addressing that as a separate > > independent issue (I’ll create a separate ticket for it)? > > If so, can you please give me a +1 on the vote thread? > > > > Thanks, > > Lucas > > > > On Tue, Aug 7, 2018 at 7:34 PM Becket Qin <becket....@gmail.com> wrote: > > > > > Thanks for the updated KIP wiki, Lucas. Looks good to me overall. > > > > > > It might be an implementation detail, but do we still plan to use the > > > correlation id to ensure the request processing order? > > > > > > Thanks, > > > > > > Jiangjie (Becket) Qin > > > > > > On Tue, Jul 31, 2018 at 3:39 AM, Lucas Wang <lucasatu...@gmail.com> > > wrote: > > > > > > > Thanks for your review, Dong. > > > > Ack that these configs will have a bigger impact for users. > > > > > > > > On the other hand, I would argue that the request queue becoming full > > > > may or may not be a rare scenario. > > > > How often the request queue gets full depends on the request incoming > > > rate, > > > > the request processing rate, and the size of the request queue. > > > > When that happens, the dedicated endpoints design can better handle > > > > it than any of the previously discussed options. > > > > > > > > Another reason I made the change was that I have the same taste > > > > as Becket that it's a better separation of the control plane from the > > > data > > > > plane. > > > > > > > > Finally, I want to clarify that this change is NOT motivated by the > > > > out-of-order > > > > processing discussion. The latter problem is orthogonal to this KIP, > > and > > > it > > > > can happen in any of the design options we discussed for this KIP so > > far. > > > > So I'd like to address out-of-order processing separately in another > > > > thread, > > > > and avoid mentioning it in this KIP. > > > > > > > > Thanks, > > > > Lucas > > > > > > > > On Fri, Jul 27, 2018 at 7:51 PM, Dong Lin <lindon...@gmail.com> > wrote: > > > > > > > > > Hey Lucas, > > > > > > > > > > Thanks for the update. > > > > > > > > > > The current KIP propose new broker configs > "listeners.for.controller" > > > and > > > > > "advertised.listeners.for.controller". This is going to be a big > > change > > > > > since listeners are among the most important configs that every > user > > > > needs > > > > > to change. According to the rejected alternative section, it seems > > that > > > > the > > > > > reason to add these two configs is to improve performance when the > > data > > > > > request queue is full rather than for correctness. It should be a > > very > > > > rare > > > > > scenario and I am not sure we should add configs for all users just > > to > > > > > improve the performance in such rare scenario. > > > > > > > > > > Also, if the new design is based on the issues which are discovered > > in > > > > the > > > > > recent discussion, e.g. out of order processing if we don't use a > > > > dedicated > > > > > thread for controller request, it may be useful to explain the > > problem > > > in > > > > > the motivation section. > > > > > > > > > > Thanks, > > > > > Dong > > > > > > > > > > On Fri, Jul 27, 2018 at 1:28 PM, Lucas Wang <lucasatu...@gmail.com > > > > > > wrote: > > > > > > > > > > > A kind reminder for review of this KIP. > > > > > > > > > > > > Thank you very much! > > > > > > Lucas > > > > > > > > > > > > On Wed, Jul 25, 2018 at 10:23 PM, Lucas Wang < > > lucasatu...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > I've updated the KIP by adding the dedicated endpoints for > > > controller > > > > > > > connections, > > > > > > > and pinning threads for controller requests. > > > > > > > Also I've updated the title of this KIP. Please take a look and > > let > > > > me > > > > > > > know your feedback. > > > > > > > > > > > > > > Thanks a lot for your time! > > > > > > > Lucas > > > > > > > > > > > > > > On Tue, Jul 24, 2018 at 10:19 AM, Mayuresh Gharat < > > > > > > > gharatmayures...@gmail.com> wrote: > > > > > > > > > > > > > >> Hi Lucas, > > > > > > >> I agree, if we want to go forward with a separate controller > > plane > > > > and > > > > > > >> data > > > > > > >> plane and completely isolate them, having a separate port for > > > > > controller > > > > > > >> with a separate Acceptor and a Processor sounds ideal to me. > > > > > > >> > > > > > > >> Thanks, > > > > > > >> > > > > > > >> Mayuresh > > > > > > >> > > > > > > >> > > > > > > >> On Mon, Jul 23, 2018 at 11:04 PM Becket Qin < > > becket....@gmail.com > > > > > > > > > > wrote: > > > > > > >> > > > > > > >> > Hi Lucas, > > > > > > >> > > > > > > > >> > Yes, I agree that a dedicated end to end control flow would > be > > > > > ideal. > > > > > > >> > > > > > > > >> > Thanks, > > > > > > >> > > > > > > > >> > Jiangjie (Becket) Qin > > > > > > >> > > > > > > > >> > On Tue, Jul 24, 2018 at 1:05 PM, Lucas Wang < > > > > lucasatu...@gmail.com> > > > > > > >> wrote: > > > > > > >> > > > > > > > >> > > Thanks for the comment, Becket. > > > > > > >> > > So far, we've been trying to avoid making any request > > handler > > > > > thread > > > > > > >> > > special. > > > > > > >> > > But if we were to follow that path in order to make the > two > > > > planes > > > > > > >> more > > > > > > >> > > isolated, > > > > > > >> > > what do you think about also having a dedicated processor > > > > thread, > > > > > > >> > > and dedicated port for the controller? > > > > > > >> > > > > > > > > >> > > Today one processor thread can handle multiple > connections, > > > > let's > > > > > > say > > > > > > >> 100 > > > > > > >> > > connections > > > > > > >> > > > > > > > > >> > > represented by connection0, ... connection99, among which > > > > > > >> connection0-98 > > > > > > >> > > are from clients, while connection99 is from > > > > > > >> > > > > > > > > >> > > the controller. Further let's say after one selector > > polling, > > > > > there > > > > > > >> are > > > > > > >> > > incoming requests on all connections. > > > > > > >> > > > > > > > > >> > > When the request queue is full, (either the data request > > being > > > > > full > > > > > > in > > > > > > >> > the > > > > > > >> > > two queue design, or > > > > > > >> > > > > > > > > >> > > the one single queue being full in the deque design), the > > > > > processor > > > > > > >> > thread > > > > > > >> > > will be blocked first > > > > > > >> > > > > > > > > >> > > when trying to enqueue the data request from connection0, > > then > > > > > > >> possibly > > > > > > >> > > blocked for the data request > > > > > > >> > > > > > > > > >> > > from connection1, ... etc even though the controller > request > > > is > > > > > > ready > > > > > > >> to > > > > > > >> > be > > > > > > >> > > enqueued. > > > > > > >> > > > > > > > > >> > > To solve this problem, it seems we would need to have a > > > separate > > > > > > port > > > > > > >> > > dedicated to > > > > > > >> > > > > > > > > >> > > the controller, a dedicated processor thread, a dedicated > > > > > controller > > > > > > >> > > request queue, > > > > > > >> > > > > > > > > >> > > and pinning of one request handler thread for controller > > > > requests. > > > > > > >> > > > > > > > > >> > > Thanks, > > > > > > >> > > Lucas > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > On Mon, Jul 23, 2018 at 6:00 PM, Becket Qin < > > > > becket....@gmail.com > > > > > > > > > > > > >> > wrote: > > > > > > >> > > > > > > > > >> > > > Personally I am not fond of the dequeue approach simply > > > > because > > > > > it > > > > > > >> is > > > > > > >> > > > against the basic idea of isolating the controller plane > > and > > > > > data > > > > > > >> > plane. > > > > > > >> > > > With a single dequeue, theoretically speaking the > > controller > > > > > > >> requests > > > > > > >> > can > > > > > > >> > > > starve the clients requests. I would prefer the approach > > > with > > > > a > > > > > > >> > separate > > > > > > >> > > > controller request queue and a dedicated controller > > request > > > > > > handler > > > > > > >> > > thread. > > > > > > >> > > > > > > > > > >> > > > Thanks, > > > > > > >> > > > > > > > > > >> > > > Jiangjie (Becket) Qin > > > > > > >> > > > > > > > > > >> > > > On Tue, Jul 24, 2018 at 8:16 AM, Lucas Wang < > > > > > > lucasatu...@gmail.com> > > > > > > >> > > wrote: > > > > > > >> > > > > > > > > > >> > > > > Sure, I can summarize the usage of correlation id. But > > > > before > > > > > I > > > > > > do > > > > > > >> > > that, > > > > > > >> > > > it > > > > > > >> > > > > seems > > > > > > >> > > > > the same out-of-order processing can also happen to > > > Produce > > > > > > >> requests > > > > > > >> > > sent > > > > > > >> > > > > by producers, > > > > > > >> > > > > following the same example you described earlier. > > > > > > >> > > > > If that's the case, I think this probably deserves a > > > > separate > > > > > > doc > > > > > > >> and > > > > > > >> > > > > design independent of this KIP. > > > > > > >> > > > > > > > > > > >> > > > > Lucas > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > On Mon, Jul 23, 2018 at 12:39 PM, Dong Lin < > > > > > lindon...@gmail.com > > > > > > > > > > > > > >> > > wrote: > > > > > > >> > > > > > > > > > > >> > > > > > Hey Lucas, > > > > > > >> > > > > > > > > > > > >> > > > > > Could you update the KIP if you are confident with > the > > > > > > approach > > > > > > >> > which > > > > > > >> > > > > uses > > > > > > >> > > > > > correlation id? The idea around correlation id is > kind > > > of > > > > > > >> scattered > > > > > > >> > > > > across > > > > > > >> > > > > > multiple emails. It will be useful if other reviews > > can > > > > read > > > > > > the > > > > > > >> > KIP > > > > > > >> > > to > > > > > > >> > > > > > understand the latest proposal. > > > > > > >> > > > > > > > > > > > >> > > > > > Thanks, > > > > > > >> > > > > > Dong > > > > > > >> > > > > > > > > > > > >> > > > > > On Mon, Jul 23, 2018 at 12:32 PM, Mayuresh Gharat < > > > > > > >> > > > > > gharatmayures...@gmail.com> wrote: > > > > > > >> > > > > > > > > > > > >> > > > > > > I like the idea of the dequeue implementation by > > > Lucas. > > > > > This > > > > > > >> will > > > > > > >> > > > help > > > > > > >> > > > > us > > > > > > >> > > > > > > avoid additional queue for controller and > additional > > > > > configs > > > > > > >> in > > > > > > >> > > > Kafka. > > > > > > >> > > > > > > > > > > > > >> > > > > > > Thanks, > > > > > > >> > > > > > > > > > > > > >> > > > > > > Mayuresh > > > > > > >> > > > > > > > > > > > > >> > > > > > > On Sun, Jul 22, 2018 at 2:58 AM Becket Qin < > > > > > > >> becket....@gmail.com > > > > > > >> > > > > > > > > >> > > > > wrote: > > > > > > >> > > > > > > > > > > > > >> > > > > > > > Hi Jun, > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > The usage of correlation ID might still be > useful > > to > > > > > > address > > > > > > >> > the > > > > > > >> > > > > cases > > > > > > >> > > > > > > > that the controller epoch and leader epoch check > > are > > > > not > > > > > > >> > > sufficient > > > > > > >> > > > > to > > > > > > >> > > > > > > > guarantee correct behavior. For example, if the > > > > > controller > > > > > > >> > sends > > > > > > >> > > a > > > > > > >> > > > > > > > LeaderAndIsrRequest followed by a > > > StopReplicaRequest, > > > > > and > > > > > > >> the > > > > > > >> > > > broker > > > > > > >> > > > > > > > processes it in the reverse order, the replica > may > > > > still > > > > > > be > > > > > > >> > > wrongly > > > > > > >> > > > > > > > recreated, right? > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > Thanks, > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > Jiangjie (Becket) Qin > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > On Jul 22, 2018, at 11:47 AM, Jun Rao < > > > > > j...@confluent.io > > > > > > > > > > > > > >> > > wrote: > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > Hmm, since we already use controller epoch and > > > > leader > > > > > > >> epoch > > > > > > >> > for > > > > > > >> > > > > > > properly > > > > > > >> > > > > > > > > caching the latest partition state, do we > really > > > > need > > > > > > >> > > correlation > > > > > > >> > > > > id > > > > > > >> > > > > > > for > > > > > > >> > > > > > > > > ordering the controller requests? > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > Thanks, > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > Jun > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > On Fri, Jul 20, 2018 at 2:18 PM, Becket Qin < > > > > > > >> > > > becket....@gmail.com> > > > > > > >> > > > > > > > wrote: > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > >> Lucas and Mayuresh, > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> Good idea. The correlation id should work. > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> In the ControllerChannelManager, a request > will > > > be > > > > > > resent > > > > > > >> > > until > > > > > > >> > > > a > > > > > > >> > > > > > > > response > > > > > > >> > > > > > > > >> is received. So if the controller to broker > > > > > connection > > > > > > >> > > > disconnects > > > > > > >> > > > > > > after > > > > > > >> > > > > > > > >> controller sends R1_a, but before the > response > > of > > > > > R1_a > > > > > > is > > > > > > >> > > > > received, > > > > > > >> > > > > > a > > > > > > >> > > > > > > > >> disconnection may cause the controller to > > resend > > > > > R1_b. > > > > > > >> i.e. > > > > > > >> > > > until > > > > > > >> > > > > R1 > > > > > > >> > > > > > > is > > > > > > >> > > > > > > > >> acked, R2 won't be sent by the controller. > > > > > > >> > > > > > > > >> This gives two guarantees: > > > > > > >> > > > > > > > >> 1. Correlation id wise: R1_a < R1_b < R2. > > > > > > >> > > > > > > > >> 2. On the broker side, when R2 is seen, R1 > must > > > > have > > > > > > been > > > > > > >> > > > > processed > > > > > > >> > > > > > at > > > > > > >> > > > > > > > >> least once. > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> So on the broker side, with a single thread > > > > > controller > > > > > > >> > request > > > > > > >> > > > > > > handler, > > > > > > >> > > > > > > > the > > > > > > >> > > > > > > > >> logic should be: > > > > > > >> > > > > > > > >> 1. Process what ever request seen in the > > > controller > > > > > > >> request > > > > > > >> > > > queue > > > > > > >> > > > > > > > >> 2. For the given epoch, drop request if its > > > > > correlation > > > > > > >> id > > > > > > >> > is > > > > > > >> > > > > > smaller > > > > > > >> > > > > > > > than > > > > > > >> > > > > > > > >> that of the last processed request. > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> Thanks, > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> Jiangjie (Becket) Qin > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> On Fri, Jul 20, 2018 at 8:07 AM, Jun Rao < > > > > > > >> j...@confluent.io> > > > > > > >> > > > > wrote: > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >>> I agree that there is no strong ordering > when > > > > there > > > > > > are > > > > > > >> > more > > > > > > >> > > > than > > > > > > >> > > > > > one > > > > > > >> > > > > > > > >>> socket connections. Currently, we rely on > > > > > > >> controllerEpoch > > > > > > >> > and > > > > > > >> > > > > > > > leaderEpoch > > > > > > >> > > > > > > > >>> to ensure that the receiving broker picks up > > the > > > > > > latest > > > > > > >> > state > > > > > > >> > > > for > > > > > > >> > > > > > > each > > > > > > >> > > > > > > > >>> partition. > > > > > > >> > > > > > > > >>> > > > > > > >> > > > > > > > >>> One potential issue with the dequeue > approach > > is > > > > > that > > > > > > if > > > > > > >> > the > > > > > > >> > > > > queue > > > > > > >> > > > > > is > > > > > > >> > > > > > > > >> full, > > > > > > >> > > > > > > > >>> there is no guarantee that the controller > > > requests > > > > > > will > > > > > > >> be > > > > > > >> > > > > enqueued > > > > > > >> > > > > > > > >>> quickly. > > > > > > >> > > > > > > > >>> > > > > > > >> > > > > > > > >>> Thanks, > > > > > > >> > > > > > > > >>> > > > > > > >> > > > > > > > >>> Jun > > > > > > >> > > > > > > > >>> > > > > > > >> > > > > > > > >>> On Fri, Jul 20, 2018 at 5:25 AM, Mayuresh > > > Gharat < > > > > > > >> > > > > > > > >>> gharatmayures...@gmail.com > > > > > > >> > > > > > > > >>>> wrote: > > > > > > >> > > > > > > > >>> > > > > > > >> > > > > > > > >>>> Yea, the correlationId is only set to 0 in > > the > > > > > > >> > NetworkClient > > > > > > >> > > > > > > > >> constructor. > > > > > > >> > > > > > > > >>>> Since we reuse the same NetworkClient > between > > > > > > >> Controller > > > > > > >> > and > > > > > > >> > > > the > > > > > > >> > > > > > > > >> broker, > > > > > > >> > > > > > > > >>> a > > > > > > >> > > > > > > > >>>> disconnection should not cause it to reset > to > > > 0, > > > > in > > > > > > >> which > > > > > > >> > > case > > > > > > >> > > > > it > > > > > > >> > > > > > > can > > > > > > >> > > > > > > > >> be > > > > > > >> > > > > > > > >>>> used to reject obsolete requests. > > > > > > >> > > > > > > > >>>> > > > > > > >> > > > > > > > >>>> Thanks, > > > > > > >> > > > > > > > >>>> > > > > > > >> > > > > > > > >>>> Mayuresh > > > > > > >> > > > > > > > >>>> > > > > > > >> > > > > > > > >>>> On Thu, Jul 19, 2018 at 1:52 PM Lucas Wang > < > > > > > > >> > > > > lucasatu...@gmail.com > > > > > > >> > > > > > > > > > > > > >> > > > > > > > >>> wrote: > > > > > > >> > > > > > > > >>>> > > > > > > >> > > > > > > > >>>>> @Dong, > > > > > > >> > > > > > > > >>>>> Great example and explanation, thanks! > > > > > > >> > > > > > > > >>>>> > > > > > > >> > > > > > > > >>>>> @All > > > > > > >> > > > > > > > >>>>> Regarding the example given by Dong, it > > seems > > > > even > > > > > > if > > > > > > >> we > > > > > > >> > > use > > > > > > >> > > > a > > > > > > >> > > > > > > queue, > > > > > > >> > > > > > > > >>>> and a > > > > > > >> > > > > > > > >>>>> dedicated controller request handling > > thread, > > > > > > >> > > > > > > > >>>>> the same result can still happen because > > R1_a > > > > will > > > > > > be > > > > > > >> > sent > > > > > > >> > > on > > > > > > >> > > > > one > > > > > > >> > > > > > > > >>>>> connection, and R1_b & R2 will be sent on > a > > > > > > different > > > > > > >> > > > > connection, > > > > > > >> > > > > > > > >>>>> and there is no ordering between different > > > > > > >> connections on > > > > > > >> > > the > > > > > > >> > > > > > > broker > > > > > > >> > > > > > > > >>>> side. > > > > > > >> > > > > > > > >>>>> I was discussing with Mayuresh offline, > and > > it > > > > > seems > > > > > > >> > > > > correlation > > > > > > >> > > > > > id > > > > > > >> > > > > > > > >>>> within > > > > > > >> > > > > > > > >>>>> the same NetworkClient object is > > monotonically > > > > > > >> increasing > > > > > > >> > > and > > > > > > >> > > > > > never > > > > > > >> > > > > > > > >>>> reset, > > > > > > >> > > > > > > > >>>>> hence a broker can leverage that to > properly > > > > > reject > > > > > > >> > > obsolete > > > > > > >> > > > > > > > >> requests. > > > > > > >> > > > > > > > >>>>> Thoughts? > > > > > > >> > > > > > > > >>>>> > > > > > > >> > > > > > > > >>>>> Thanks, > > > > > > >> > > > > > > > >>>>> Lucas > > > > > > >> > > > > > > > >>>>> > > > > > > >> > > > > > > > >>>>> On Thu, Jul 19, 2018 at 12:11 PM, Mayuresh > > > > Gharat > > > > > < > > > > > > >> > > > > > > > >>>>> gharatmayures...@gmail.com> wrote: > > > > > > >> > > > > > > > >>>>> > > > > > > >> > > > > > > > >>>>>> Actually nvm, correlationId is reset in > > case > > > of > > > > > > >> > connection > > > > > > >> > > > > > loss, I > > > > > > >> > > > > > > > >>>> think. > > > > > > >> > > > > > > > >>>>>> > > > > > > >> > > > > > > > >>>>>> Thanks, > > > > > > >> > > > > > > > >>>>>> > > > > > > >> > > > > > > > >>>>>> Mayuresh > > > > > > >> > > > > > > > >>>>>> > > > > > > >> > > > > > > > >>>>>> On Thu, Jul 19, 2018 at 11:11 AM Mayuresh > > > > Gharat > > > > > < > > > > > > >> > > > > > > > >>>>>> gharatmayures...@gmail.com> > > > > > > >> > > > > > > > >>>>>> wrote: > > > > > > >> > > > > > > > >>>>>> > > > > > > >> > > > > > > > >>>>>>> I agree with Dong that out-of-order > > > processing > > > > > can > > > > > > >> > happen > > > > > > >> > > > > with > > > > > > >> > > > > > > > >>>> having 2 > > > > > > >> > > > > > > > >>>>>>> separate queues as well and it can even > > > happen > > > > > > >> today. > > > > > > >> > > > > > > > >>>>>>> Can we use the correlationId in the > > request > > > > from > > > > > > the > > > > > > >> > > > > controller > > > > > > >> > > > > > > > >> to > > > > > > >> > > > > > > > >>>> the > > > > > > >> > > > > > > > >>>>>>> broker to handle ordering ? > > > > > > >> > > > > > > > >>>>>>> > > > > > > >> > > > > > > > >>>>>>> Thanks, > > > > > > >> > > > > > > > >>>>>>> > > > > > > >> > > > > > > > >>>>>>> Mayuresh > > > > > > >> > > > > > > > >>>>>>> > > > > > > >> > > > > > > > >>>>>>> > > > > > > >> > > > > > > > >>>>>>> On Thu, Jul 19, 2018 at 6:41 AM Becket > > Qin < > > > > > > >> > > > > > becket....@gmail.com > > > > > > >> > > > > > > > >>> > > > > > > >> > > > > > > > >>>>> wrote: > > > > > > >> > > > > > > > >>>>>>> > > > > > > >> > > > > > > > >>>>>>>> Good point, Joel. I agree that a > > dedicated > > > > > > >> controller > > > > > > >> > > > > request > > > > > > >> > > > > > > > >>>> handling > > > > > > >> > > > > > > > >>>>>>>> thread would be a better isolation. It > > also > > > > > > solves > > > > > > >> the > > > > > > >> > > > > > > > >> reordering > > > > > > >> > > > > > > > >>>>> issue. > > > > > > >> > > > > > > > >>>>>>>> > > > > > > >> > > > > > > > >>>>>>>> On Thu, Jul 19, 2018 at 2:23 PM, Joel > > > Koshy < > > > > > > >> > > > > > > > >> jjkosh...@gmail.com> > > > > > > >> > > > > > > > >>>>>> wrote: > > > > > > >> > > > > > > > >>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>> Good example. I think this scenario > can > > > > occur > > > > > in > > > > > > >> the > > > > > > >> > > > > current > > > > > > >> > > > > > > > >>> code > > > > > > >> > > > > > > > >>>> as > > > > > > >> > > > > > > > >>>>>>>> well > > > > > > >> > > > > > > > >>>>>>>>> but with even lower probability given > > that > > > > > there > > > > > > >> are > > > > > > >> > > > other > > > > > > >> > > > > > > > >>>>>>>> non-controller > > > > > > >> > > > > > > > >>>>>>>>> requests interleaved. It is still > > sketchy > > > > > though > > > > > > >> and > > > > > > >> > I > > > > > > >> > > > > think > > > > > > >> > > > > > a > > > > > > >> > > > > > > > >>>> safer > > > > > > >> > > > > > > > >>>>>>>>> approach would be separate queues and > > > > pinning > > > > > > >> > > controller > > > > > > >> > > > > > > > >> request > > > > > > >> > > > > > > > >>>>>>>> handling > > > > > > >> > > > > > > > >>>>>>>>> to one handler thread. > > > > > > >> > > > > > > > >>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>> On Wed, Jul 18, 2018 at 11:12 PM, Dong > > > Lin < > > > > > > >> > > > > > > > >> lindon...@gmail.com > > > > > > >> > > > > > > > >>>> > > > > > > >> > > > > > > > >>>>>> wrote: > > > > > > >> > > > > > > > >>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>> Hey Becket, > > > > > > >> > > > > > > > >>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>> I think you are right that there may > be > > > > > > >> out-of-order > > > > > > >> > > > > > > > >>> processing. > > > > > > >> > > > > > > > >>>>>>>> However, > > > > > > >> > > > > > > > >>>>>>>>>> it seems that out-of-order processing > > may > > > > > also > > > > > > >> > happen > > > > > > >> > > > even > > > > > > >> > > > > > > > >> if > > > > > > >> > > > > > > > >>> we > > > > > > >> > > > > > > > >>>>>> use a > > > > > > >> > > > > > > > >>>>>>>>>> separate queue. > > > > > > >> > > > > > > > >>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>> Here is the example: > > > > > > >> > > > > > > > >>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>> - Controller sends R1 and got > > > disconnected > > > > > > before > > > > > > >> > > > > receiving > > > > > > >> > > > > > > > >>>>>> response. > > > > > > >> > > > > > > > >>>>>>>>> Then > > > > > > >> > > > > > > > >>>>>>>>>> it reconnects and sends R2. Both > > requests > > > > now > > > > > > >> stay > > > > > > >> > in > > > > > > >> > > > the > > > > > > >> > > > > > > > >>>>> controller > > > > > > >> > > > > > > > >>>>>>>>>> request queue in the order they are > > sent. > > > > > > >> > > > > > > > >>>>>>>>>> - thread1 takes R1_a from the request > > > queue > > > > > and > > > > > > >> then > > > > > > >> > > > > thread2 > > > > > > >> > > > > > > > >>>> takes > > > > > > >> > > > > > > > >>>>>> R2 > > > > > > >> > > > > > > > >>>>>>>>> from > > > > > > >> > > > > > > > >>>>>>>>>> the request queue almost at the same > > > time. > > > > > > >> > > > > > > > >>>>>>>>>> - So R1_a and R2 are processed in > > > parallel. > > > > > > >> There is > > > > > > >> > > > > chance > > > > > > >> > > > > > > > >>> that > > > > > > >> > > > > > > > >>>>>> R2's > > > > > > >> > > > > > > > >>>>>>>>>> processing is completed before R1. > > > > > > >> > > > > > > > >>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>> If out-of-order processing can happen > > for > > > > > both > > > > > > >> > > > approaches > > > > > > >> > > > > > > > >> with > > > > > > >> > > > > > > > >>>>> very > > > > > > >> > > > > > > > >>>>>>>> low > > > > > > >> > > > > > > > >>>>>>>>>> probability, it may not be worthwhile > > to > > > > add > > > > > > the > > > > > > >> > extra > > > > > > >> > > > > > > > >> queue. > > > > > > >> > > > > > > > >>>> What > > > > > > >> > > > > > > > >>>>>> do > > > > > > >> > > > > > > > >>>>>>>> you > > > > > > >> > > > > > > > >>>>>>>>>> think? > > > > > > >> > > > > > > > >>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>> Thanks, > > > > > > >> > > > > > > > >>>>>>>>>> Dong > > > > > > >> > > > > > > > >>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>> On Wed, Jul 18, 2018 at 6:17 PM, > Becket > > > > Qin < > > > > > > >> > > > > > > > >>>> becket....@gmail.com > > > > > > >> > > > > > > > >>>>>> > > > > > > >> > > > > > > > >>>>>>>>> wrote: > > > > > > >> > > > > > > > >>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>> Hi Mayuresh/Joel, > > > > > > >> > > > > > > > >>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>> Using the request channel as a > dequeue > > > was > > > > > > >> bright > > > > > > >> > up > > > > > > >> > > > some > > > > > > >> > > > > > > > >>> time > > > > > > >> > > > > > > > >>>>> ago > > > > > > >> > > > > > > > >>>>>>>> when > > > > > > >> > > > > > > > >>>>>>>>>> we > > > > > > >> > > > > > > > >>>>>>>>>>> initially thinking of prioritizing > the > > > > > > request. > > > > > > >> The > > > > > > >> > > > > > > > >> concern > > > > > > >> > > > > > > > >>>> was > > > > > > >> > > > > > > > >>>>>> that > > > > > > >> > > > > > > > >>>>>>>>> the > > > > > > >> > > > > > > > >>>>>>>>>>> controller requests are supposed to > be > > > > > > >> processed in > > > > > > >> > > > > order. > > > > > > >> > > > > > > > >>> If > > > > > > >> > > > > > > > >>>> we > > > > > > >> > > > > > > > >>>>>> can > > > > > > >> > > > > > > > >>>>>>>>>> ensure > > > > > > >> > > > > > > > >>>>>>>>>>> that there is one controller request > > in > > > > the > > > > > > >> request > > > > > > >> > > > > > > > >> channel, > > > > > > >> > > > > > > > >>>> the > > > > > > >> > > > > > > > >>>>>>>> order > > > > > > >> > > > > > > > >>>>>>>>> is > > > > > > >> > > > > > > > >>>>>>>>>>> not a concern. But in cases that > there > > > are > > > > > > more > > > > > > >> > than > > > > > > >> > > > one > > > > > > >> > > > > > > > >>>>>> controller > > > > > > >> > > > > > > > >>>>>>>>>> request > > > > > > >> > > > > > > > >>>>>>>>>>> inserted into the queue, the > > controller > > > > > > request > > > > > > >> > order > > > > > > >> > > > may > > > > > > >> > > > > > > > >>>> change > > > > > > >> > > > > > > > >>>>>> and > > > > > > >> > > > > > > > >>>>>>>>>> cause > > > > > > >> > > > > > > > >>>>>>>>>>> problem. For example, think about > the > > > > > > following > > > > > > >> > > > sequence: > > > > > > >> > > > > > > > >>>>>>>>>>> 1. Controller successfully sent a > > > request > > > > R1 > > > > > > to > > > > > > >> > > broker > > > > > > >> > > > > > > > >>>>>>>>>>> 2. Broker receives R1 and put the > > > request > > > > to > > > > > > the > > > > > > >> > head > > > > > > >> > > > of > > > > > > >> > > > > > > > >> the > > > > > > >> > > > > > > > >>>>>> request > > > > > > >> > > > > > > > >>>>>>>>>> queue. > > > > > > >> > > > > > > > >>>>>>>>>>> 3. Controller to broker connection > > > failed > > > > > and > > > > > > >> the > > > > > > >> > > > > > > > >> controller > > > > > > >> > > > > > > > >>>>>>>>> reconnected > > > > > > >> > > > > > > > >>>>>>>>>> to > > > > > > >> > > > > > > > >>>>>>>>>>> the broker. > > > > > > >> > > > > > > > >>>>>>>>>>> 4. Controller sends a request R2 to > > the > > > > > broker > > > > > > >> > > > > > > > >>>>>>>>>>> 5. Broker receives R2 and add it to > > the > > > > head > > > > > > of > > > > > > >> the > > > > > > >> > > > > > > > >> request > > > > > > >> > > > > > > > >>>>> queue. > > > > > > >> > > > > > > > >>>>>>>>>>> Now on the broker side, R2 will be > > > > processed > > > > > > >> before > > > > > > >> > > R1 > > > > > > >> > > > is > > > > > > >> > > > > > > > >>>>>> processed, > > > > > > >> > > > > > > > >>>>>>>>>> which > > > > > > >> > > > > > > > >>>>>>>>>>> may cause problem. > > > > > > >> > > > > > > > >>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>> Thanks, > > > > > > >> > > > > > > > >>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>> Jiangjie (Becket) Qin > > > > > > >> > > > > > > > >>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>> On Thu, Jul 19, 2018 at 3:23 AM, > Joel > > > > Koshy > > > > > < > > > > > > >> > > > > > > > >>>>> jjkosh...@gmail.com> > > > > > > >> > > > > > > > >>>>>>>>> wrote: > > > > > > >> > > > > > > > >>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>> @Mayuresh - I like your idea. It > > > appears > > > > to > > > > > > be > > > > > > >> a > > > > > > >> > > > simpler > > > > > > >> > > > > > > > >>>> less > > > > > > >> > > > > > > > >>>>>>>>> invasive > > > > > > >> > > > > > > > >>>>>>>>>>>> alternative and it should work. > > > > > > >> Jun/Becket/others, > > > > > > >> > > do > > > > > > >> > > > > > > > >> you > > > > > > >> > > > > > > > >>>> see > > > > > > >> > > > > > > > >>>>>> any > > > > > > >> > > > > > > > >>>>>>>>>>> pitfalls > > > > > > >> > > > > > > > >>>>>>>>>>>> with this approach? > > > > > > >> > > > > > > > >>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>> On Wed, Jul 18, 2018 at 12:03 PM, > > Lucas > > > > > Wang > > > > > > < > > > > > > >> > > > > > > > >>>>>>>> lucasatu...@gmail.com> > > > > > > >> > > > > > > > >>>>>>>>>>>> wrote: > > > > > > >> > > > > > > > >>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> @Mayuresh, > > > > > > >> > > > > > > > >>>>>>>>>>>>> That's a very interesting idea > that > > I > > > > > > haven't > > > > > > >> > > thought > > > > > > >> > > > > > > > >>>>> before. > > > > > > >> > > > > > > > >>>>>>>>>>>>> It seems to solve our problem at > > hand > > > > > pretty > > > > > > >> > well, > > > > > > >> > > > and > > > > > > >> > > > > > > > >>>> also > > > > > > >> > > > > > > > >>>>>>>>>>>>> avoids the need to have a new size > > > > metric > > > > > > and > > > > > > >> > > > capacity > > > > > > >> > > > > > > > >>>>> config > > > > > > >> > > > > > > > >>>>>>>>>>>>> for the controller request queue. > In > > > > fact, > > > > > > if > > > > > > >> we > > > > > > >> > > were > > > > > > >> > > > > > > > >> to > > > > > > >> > > > > > > > >>>>> adopt > > > > > > >> > > > > > > > >>>>>>>>>>>>> this design, there is no public > > > > interface > > > > > > >> change, > > > > > > >> > > and > > > > > > >> > > > > > > > >> we > > > > > > >> > > > > > > > >>>>>>>>>>>>> probably don't need a KIP. > > > > > > >> > > > > > > > >>>>>>>>>>>>> Also implementation wise, it seems > > > > > > >> > > > > > > > >>>>>>>>>>>>> the java class LinkedBlockingQueue > > can > > > > > > readily > > > > > > >> > > > satisfy > > > > > > >> > > > > > > > >>> the > > > > > > >> > > > > > > > >>>>>>>>>> requirement > > > > > > >> > > > > > > > >>>>>>>>>>>>> by supporting a capacity, and also > > > > > allowing > > > > > > >> > > inserting > > > > > > >> > > > > > > > >> at > > > > > > >> > > > > > > > >>>>> both > > > > > > >> > > > > > > > >>>>>>>> ends. > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> My only concern is that this > design > > is > > > > > tied > > > > > > to > > > > > > >> > the > > > > > > >> > > > > > > > >>>>> coincidence > > > > > > >> > > > > > > > >>>>>>>> that > > > > > > >> > > > > > > > >>>>>>>>>>>>> we have two request priorities and > > > there > > > > > are > > > > > > >> two > > > > > > >> > > ends > > > > > > >> > > > > > > > >>> to a > > > > > > >> > > > > > > > >>>>>>>> deque. > > > > > > >> > > > > > > > >>>>>>>>>>>>> Hence by using the proposed > design, > > it > > > > > seems > > > > > > >> the > > > > > > >> > > > > > > > >> network > > > > > > >> > > > > > > > >>>>> layer > > > > > > >> > > > > > > > >>>>>>>> is > > > > > > >> > > > > > > > >>>>>>>>>>>>> more tightly coupled with upper > > layer > > > > > logic, > > > > > > >> e.g. > > > > > > >> > > if > > > > > > >> > > > > > > > >> we > > > > > > >> > > > > > > > >>>> were > > > > > > >> > > > > > > > >>>>>> to > > > > > > >> > > > > > > > >>>>>>>> add > > > > > > >> > > > > > > > >>>>>>>>>>>>> an extra priority level in the > > future > > > > for > > > > > > some > > > > > > >> > > > reason, > > > > > > >> > > > > > > > >>> we > > > > > > >> > > > > > > > >>>>>> would > > > > > > >> > > > > > > > >>>>>>>>>>> probably > > > > > > >> > > > > > > > >>>>>>>>>>>>> need to go back to the design of > > > > separate > > > > > > >> queues, > > > > > > >> > > one > > > > > > >> > > > > > > > >>> for > > > > > > >> > > > > > > > >>>>> each > > > > > > >> > > > > > > > >>>>>>>>>> priority > > > > > > >> > > > > > > > >>>>>>>>>>>>> level. > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> In summary, I'm ok with both > designs > > > and > > > > > > lean > > > > > > >> > > toward > > > > > > >> > > > > > > > >>> your > > > > > > >> > > > > > > > >>>>>>>> suggested > > > > > > >> > > > > > > > >>>>>>>>>>>>> approach. > > > > > > >> > > > > > > > >>>>>>>>>>>>> Let's hear what others think. > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> @Becket, > > > > > > >> > > > > > > > >>>>>>>>>>>>> In light of Mayuresh's suggested > new > > > > > design, > > > > > > >> I'm > > > > > > >> > > > > > > > >>> answering > > > > > > >> > > > > > > > >>>>>> your > > > > > > >> > > > > > > > >>>>>>>>>>> question > > > > > > >> > > > > > > > >>>>>>>>>>>>> only in the context > > > > > > >> > > > > > > > >>>>>>>>>>>>> of the current KIP design: I think > > > your > > > > > > >> > suggestion > > > > > > >> > > > > > > > >> makes > > > > > > >> > > > > > > > >>>>>> sense, > > > > > > >> > > > > > > > >>>>>>>> and > > > > > > >> > > > > > > > >>>>>>>>>> I'm > > > > > > >> > > > > > > > >>>>>>>>>>>> ok > > > > > > >> > > > > > > > >>>>>>>>>>>>> with removing the capacity config > > and > > > > > > >> > > > > > > > >>>>>>>>>>>>> just relying on the default value > of > > > 20 > > > > > > being > > > > > > >> > > > > > > > >> sufficient > > > > > > >> > > > > > > > >>>>>> enough. > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> Thanks, > > > > > > >> > > > > > > > >>>>>>>>>>>>> Lucas > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>> On Wed, Jul 18, 2018 at 9:57 AM, > > > > Mayuresh > > > > > > >> Gharat > > > > > > >> > < > > > > > > >> > > > > > > > >>>>>>>>>>>>> gharatmayures...@gmail.com > > > > > > >> > > > > > > > >>>>>>>>>>>>>> wrote: > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>> Hi Lucas, > > > > > > >> > > > > > > > >>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>> Seems like the main intent here > is > > to > > > > > > >> prioritize > > > > > > >> > > the > > > > > > >> > > > > > > > >>>>>>>> controller > > > > > > >> > > > > > > > >>>>>>>>>>> request > > > > > > >> > > > > > > > >>>>>>>>>>>>>> over any other requests. > > > > > > >> > > > > > > > >>>>>>>>>>>>>> In that case, we can change the > > > request > > > > > > queue > > > > > > >> > to a > > > > > > >> > > > > > > > >>>>> dequeue, > > > > > > >> > > > > > > > >>>>>>>> where > > > > > > >> > > > > > > > >>>>>>>>>> you > > > > > > >> > > > > > > > >>>>>>>>>>>>>> always insert the normal requests > > > > > (produce, > > > > > > >> > > > > > > > >>>> consume,..etc) > > > > > > >> > > > > > > > >>>>>> to > > > > > > >> > > > > > > > >>>>>>>> the > > > > > > >> > > > > > > > >>>>>>>>>> end > > > > > > >> > > > > > > > >>>>>>>>>>>> of > > > > > > >> > > > > > > > >>>>>>>>>>>>>> the dequeue, but if its a > > controller > > > > > > request, > > > > > > >> > you > > > > > > >> > > > > > > > >>> insert > > > > > > >> > > > > > > > >>>>> it > > > > > > >> > > > > > > > >>>>>> to > > > > > > >> > > > > > > > >>>>>>>>> the > > > > > > >> > > > > > > > >>>>>>>>>>> head > > > > > > >> > > > > > > > >>>>>>>>>>>>> of > > > > > > >> > > > > > > > >>>>>>>>>>>>>> the queue. This ensures that the > > > > > controller > > > > > > >> > > request > > > > > > >> > > > > > > > >>> will > > > > > > >> > > > > > > > >>>>> be > > > > > > >> > > > > > > > >>>>>>>> given > > > > > > >> > > > > > > > >>>>>>>>>>>> higher > > > > > > >> > > > > > > > >>>>>>>>>>>>>> priority over other requests. > > > > > > >> > > > > > > > >>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>> Also since we only read one > request > > > > from > > > > > > the > > > > > > >> > > socket > > > > > > >> > > > > > > > >>> and > > > > > > >> > > > > > > > >>>>> mute > > > > > > >> > > > > > > > >>>>>>>> it > > > > > > >> > > > > > > > >>>>>>>>> and > > > > > > >> > > > > > > > >>>>>>>>>>>> only > > > > > > >> > > > > > > > >>>>>>>>>>>>>> unmute it after handling the > > request, > > > > > this > > > > > > >> would > > > > > > >> > > > > > > > >>> ensure > > > > > > >> > > > > > > > >>>>> that > > > > > > >> > > > > > > > >>>>>>>> we > > > > > > >> > > > > > > > >>>>>>>>>> don't > > > > > > >> > > > > > > > >>>>>>>>>>>>>> handle controller requests out of > > > > order. > > > > > > >> > > > > > > > >>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>> With this approach we can avoid > the > > > > > second > > > > > > >> queue > > > > > > >> > > and > > > > > > >> > > > > > > > >>> the > > > > > > >> > > > > > > > >>>>>>>>> additional > > > > > > >> > > > > > > > >>>>>>>>>>>>> config > > > > > > >> > > > > > > > >>>>>>>>>>>>>> for the size of the queue. > > > > > > >> > > > > > > > >>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>> What do you think ? > > > > > > >> > > > > > > > >>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>> Thanks, > > > > > > >> > > > > > > > >>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>> Mayuresh > > > > > > >> > > > > > > > >>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>> On Wed, Jul 18, 2018 at 3:05 AM > > > Becket > > > > > Qin > > > > > > < > > > > > > >> > > > > > > > >>>>>>>> becket....@gmail.com > > > > > > >> > > > > > > > >>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>> wrote: > > > > > > >> > > > > > > > >>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> Hey Joel, > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> Thank for the detail > explanation. > > I > > > > > agree > > > > > > >> the > > > > > > >> > > > > > > > >>> current > > > > > > >> > > > > > > > >>>>>> design > > > > > > >> > > > > > > > >>>>>>>>>> makes > > > > > > >> > > > > > > > >>>>>>>>>>>>> sense. > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> My confusion is about whether > the > > > new > > > > > > config > > > > > > >> > for > > > > > > >> > > > > > > > >> the > > > > > > >> > > > > > > > >>>>>>>> controller > > > > > > >> > > > > > > > >>>>>>>>>>> queue > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> capacity is necessary. I cannot > > > think > > > > > of a > > > > > > >> case > > > > > > >> > > in > > > > > > >> > > > > > > > >>>> which > > > > > > >> > > > > > > > >>>>>>>> users > > > > > > >> > > > > > > > >>>>>>>>>>> would > > > > > > >> > > > > > > > >>>>>>>>>>>>>> change > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> it. > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> Thanks, > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> Jiangjie (Becket) Qin > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> On Wed, Jul 18, 2018 at 6:00 PM, > > > > Becket > > > > > > Qin > > > > > > >> < > > > > > > >> > > > > > > > >>>>>>>>>> becket....@gmail.com> > > > > > > >> > > > > > > > >>>>>>>>>>>>>> wrote: > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> Hi Lucas, > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> I guess my question can be > > > rephrased > > > > to > > > > > > >> "do we > > > > > > >> > > > > > > > >>>> expect > > > > > > >> > > > > > > > >>>>>>>> user to > > > > > > >> > > > > > > > >>>>>>>>>>> ever > > > > > > >> > > > > > > > >>>>>>>>>>>>>> change > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> the controller request queue > > > > capacity"? > > > > > > If > > > > > > >> we > > > > > > >> > > > > > > > >>> agree > > > > > > >> > > > > > > > >>>>> that > > > > > > >> > > > > > > > >>>>>>>> 20 > > > > > > >> > > > > > > > >>>>>>>>> is > > > > > > >> > > > > > > > >>>>>>>>>>>>> already > > > > > > >> > > > > > > > >>>>>>>>>>>>>> a > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> very generous default number > and > > we > > > > do > > > > > > not > > > > > > >> > > > > > > > >> expect > > > > > > >> > > > > > > > >>>> user > > > > > > >> > > > > > > > >>>>>> to > > > > > > >> > > > > > > > >>>>>>>>>> change > > > > > > >> > > > > > > > >>>>>>>>>>>> it, > > > > > > >> > > > > > > > >>>>>>>>>>>>> is > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> it > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> still necessary to expose this > > as a > > > > > > config? > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> Thanks, > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> Jiangjie (Becket) Qin > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> On Wed, Jul 18, 2018 at 2:29 > AM, > > > > Lucas > > > > > > >> Wang < > > > > > > >> > > > > > > > >>>>>>>>>>> lucasatu...@gmail.com > > > > > > >> > > > > > > > >>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> wrote: > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> @Becket > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> 1. Thanks for the comment. You > > are > > > > > right > > > > > > >> that > > > > > > >> > > > > > > > >>>>> normally > > > > > > >> > > > > > > > >>>>>>>> there > > > > > > >> > > > > > > > >>>>>>>>>>>> should > > > > > > >> > > > > > > > >>>>>>>>>>>>> be > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> just > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> one controller request because > > of > > > > > > muting, > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> and I had NOT intended to say > > > there > > > > > > would > > > > > > >> be > > > > > > >> > > > > > > > >> many > > > > > > >> > > > > > > > >>>>>>>> enqueued > > > > > > >> > > > > > > > >>>>>>>>>>>>> controller > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> requests. > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> I went through the KIP again, > > and > > > > I'm > > > > > > not > > > > > > >> > sure > > > > > > >> > > > > > > > >>>> which > > > > > > >> > > > > > > > >>>>>> part > > > > > > >> > > > > > > > >>>>>>>>>>> conveys > > > > > > >> > > > > > > > >>>>>>>>>>>>> that > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> info. > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> I'd be happy to revise if you > > > point > > > > it > > > > > > out > > > > > > >> > the > > > > > > >> > > > > > > > >>>>> section. > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> 2. Though it should not happen > > in > > > > > normal > > > > > > >> > > > > > > > >>>> conditions, > > > > > > >> > > > > > > > >>>>>> the > > > > > > >> > > > > > > > >>>>>>>>>> current > > > > > > >> > > > > > > > >>>>>>>>>>>>>> design > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> does not preclude multiple > > > > controllers > > > > > > >> > running > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> at the same time, hence if we > > > don't > > > > > have > > > > > > >> the > > > > > > >> > > > > > > > >>>>> controller > > > > > > >> > > > > > > > >>>>>>>>> queue > > > > > > >> > > > > > > > >>>>>>>>>>>>> capacity > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> config and simply make its > > > capacity > > > > to > > > > > > be > > > > > > >> 1, > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> network threads handling > > requests > > > > from > > > > > > >> > > > > > > > >> different > > > > > > >> > > > > > > > >>>>>>>> controllers > > > > > > >> > > > > > > > >>>>>>>>>>> will > > > > > > >> > > > > > > > >>>>>>>>>>>> be > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> blocked during those > troublesome > > > > > times, > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> which is probably not what we > > > want. > > > > On > > > > > > the > > > > > > >> > > > > > > > >> other > > > > > > >> > > > > > > > >>>>> hand, > > > > > > >> > > > > > > > >>>>>>>>> adding > > > > > > >> > > > > > > > >>>>>>>>>>> the > > > > > > >> > > > > > > > >>>>>>>>>>>>>> extra > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> config with a default value, > say > > > 20, > > > > > > >> guards > > > > > > >> > us > > > > > > >> > > > > > > > >>> from > > > > > > >> > > > > > > > >>>>>>>> issues > > > > > > >> > > > > > > > >>>>>>>>> in > > > > > > >> > > > > > > > >>>>>>>>>>>> those > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> troublesome times, and IMO > there > > > > isn't > > > > > > >> much > > > > > > >> > > > > > > > >>>> downside > > > > > > >> > > > > > > > >>>>> of > > > > > > >> > > > > > > > >>>>>>>>> adding > > > > > > >> > > > > > > > >>>>>>>>>>> the > > > > > > >> > > > > > > > >>>>>>>>>>>>>> extra > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> config. > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> @Mayuresh > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> Good catch, this sentence is > an > > > > > obsolete > > > > > > >> > > > > > > > >>> statement > > > > > > >> > > > > > > > >>>>>> based > > > > > > >> > > > > > > > >>>>>>>> on > > > > > > >> > > > > > > > >>>>>>>>> a > > > > > > >> > > > > > > > >>>>>>>>>>>>> previous > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> design. I've revised the > wording > > > in > > > > > the > > > > > > >> KIP. > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> Thanks, > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> Lucas > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> On Tue, Jul 17, 2018 at 10:33 > > AM, > > > > > > Mayuresh > > > > > > >> > > > > > > > >>> Gharat < > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> gharatmayures...@gmail.com> > > > wrote: > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> Hi Lucas, > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> Thanks for the KIP. > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> I am trying to understand why > > you > > > > > think > > > > > > >> "The > > > > > > >> > > > > > > > >>>> memory > > > > > > >> > > > > > > > >>>>>>>>>>> consumption > > > > > > >> > > > > > > > >>>>>>>>>>>>> can > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> rise > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> given the total number of > > queued > > > > > > requests > > > > > > >> > can > > > > > > >> > > > > > > > >>> go > > > > > > >> > > > > > > > >>>> up > > > > > > >> > > > > > > > >>>>>> to > > > > > > >> > > > > > > > >>>>>>>> 2x" > > > > > > >> > > > > > > > >>>>>>>>>> in > > > > > > >> > > > > > > > >>>>>>>>>>>> the > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> impact > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> section. Normally the > requests > > > from > > > > > > >> > > > > > > > >> controller > > > > > > >> > > > > > > > >>>> to a > > > > > > >> > > > > > > > >>>>>>>> Broker > > > > > > >> > > > > > > > >>>>>>>>>> are > > > > > > >> > > > > > > > >>>>>>>>>>>> not > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> high > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> volume, right ? > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> Thanks, > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> Mayuresh > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> On Tue, Jul 17, 2018 at 5:06 > AM > > > > > Becket > > > > > > >> Qin < > > > > > > >> > > > > > > > >>>>>>>>>>>> becket....@gmail.com> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> wrote: > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>>> Thanks for the KIP, Lucas. > > > > > Separating > > > > > > >> the > > > > > > >> > > > > > > > >>>> control > > > > > > >> > > > > > > > >>>>>>>> plane > > > > > > >> > > > > > > > >>>>>>>>>> from > > > > > > >> > > > > > > > >>>>>>>>>>>> the > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> data > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> plane > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>>> makes a lot of sense. > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>>> > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>>> In the KIP you mentioned > that > > > the > > > > > > >> > > > > > > > >> controller > > > > > > >> > > > > > > > >>>>>> request > > > > > > >> > > > > > > > >>>>>>>>> queue > > > > > > >> > > > > > > > >>>>>>>>>>> may > > > > > > >> > > > > > > > >>>>>>>>>>>>>> have > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> many > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>>> requests in it. Will this > be a > > > > > common > > > > > > >> case? > > > > > > >> > > > > > > > >>> The > > > > > > >> > > > > > > > >>>>>>>>> controller > > > > > > >> > > > > > > > >>>>>>>>>>>>>> requests > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> still > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>>> goes through the > SocketServer. > > > The > > > > > > >> > > > > > > > >>> SocketServer > > > > > > >> > > > > > > > >>>>>> will > > > > > > >> > > > > > > > >>>>>>>>> mute > > > > > > >> > > > > > > > >>>>>>>>>>> the > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> channel > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>> once > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>>> a request is read and put > into > > > the > > > > > > >> request > > > > > > >> > > > > > > > >>>>> channel. > > > > > > >> > > > > > > > >>>>>>>> So > > > > > > >> > > > > > > > >>>>>>>>>>>> assuming > > > > > > >> > > > > > > > >>>>>>>>>>>>>>> there > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>> is > > > > > > >> > > > > > > > >>>>>>>>>>>>>>>>>>> only one connection between > > > > > controller > > > > > > >> and > > > > > > >> > > > > > > > >>> each > > > > > > >> > > > > > > > >>>>>>>> broker, > > > > > > >> > > > > > > > >>>>>>>>> on > > > > > > >> > > > > > > > >>>>>>>>>>> the > > > > > > >> > > > > > > > >
