Thank Shengkai for driving this work! +1 to start a VOTE. Best, Jark
On Tue, 10 May 2022 at 12:13, Shengkai Fang <fskm...@gmail.com> wrote: > Hi, everyone. > > Thanks for all the inputs. Hope my feedback can address most of questions. > > After the long discussion, I think we have reached the consensus about the > design. If the discussion doesn't get more response, I think we can start > the vote tomorrow. > > Best, > Shengkai > > Shengkai Fang <fskm...@gmail.com> 于2022年5月9日周一 14:18写道： > > > Hi JingSong. > > > > Thanks for your feedback. > > > > > reorganize the FLIP, what Pluggable Endpoint Discovery is, and how > users > > to add new Endpoint, before introducing SQLGatewayService. > > > > I update the FLIP: reorganize the order and add more details. > > > > > Then I have some doubts about the name SQLGatewayService, I think > > > Service is seriously misleading, it is just an implementation class > > > and not a Service. > > > > After discuss with Jingsong offline, we agree to still use the > > SQLGatewayService. According to the defination of the wiki[1], service > > refers to a software functionality or a set of software functionalities > > with a purpose that different clients can reuse for different purposes. > > Here the GatewayService plays the same role. > > > > Best, > > Shengkai > > > > [1] https://en.wikipedia.org/wiki/Service_(systems_architecture) > > > > Jingsong Li <jingsongl...@gmail.com> 于2022年5月7日周六 16:55写道： > > > >> Hi Shengkai, thanks for your reply. > >> > >> > REST API is the user interface. The REST API transforms the request to > >> the > >> invocation of the SQLGatewayService that is the one doing the work. We > >> split the Gateway into the SQLGatewayService and Endpoint(REST API) and > >> its > >> benefit is that all the Endpoints share the same SQLGatewayService. > >> > >> Finally, I got the point of SQLGatewayService, It is just for > >> `Pluggable Endpoint Discovery`. > >> I suggest you reorganize the FLIP, what Pluggable Endpoint Discovery > >> is, and how users to add new Endpoint, before introducing > >> SQLGatewayService. > >> > >> Then I have some doubts about the name SQLGatewayService, I think > >> Service is seriously misleading, it is just an implementation class > >> and not a Service. > >> > >> What about just `SQLGateway`? > >> > >> Best, > >> Jingsong > >> > >> On Sat, May 7, 2022 at 4:03 PM Shengkai Fang <fskm...@gmail.com> wrote: > >> > > >> > Hi Martijn. > >> > > >> > It seems we have reached consensus to support the Gateway inside the > >> Flink > >> > code base. > >> > > >> > Hi, Echo. > >> > > >> > Thanks for your interest. > >> > > >> > > whether flink-sql-gateway should be retained in the Flink project. > >> > > >> > I think the discussion above is clear. It is the essential tool to > >> provide > >> > out-of-box experience for users. > >> > > >> > > For stream processing, what's the point of getting the result? Is it > >> just > >> > for debugging and how to unify with batch processing > >> > > >> > At the client side, when the OperationStaus is FINISHED, the client is > >> able > >> > to fetch the results from the Gateway. It is unified with the batch > >> > processing now. > >> > > >> > > For batch processing, does the gateway need to cache all fetch > >> results? > >> > > >> > No. In the Gateway, we will only buffer partial data and wait for the > >> > client to consume. If the client takes away the buffered data, the > >> Gateway > >> > will clear the buffer and notify the fetcher thread starts to work > until > >> > the buffer is full again. The mechanism behind is much like the > >> > producer-consumer model[1]. > >> > > >> > [1] https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem > >> > > >> > > Whether executing query and fetch results should be synchronous or > >> > asynchronous? > >> > > >> > Do you mean the executeStatement response should also contain the > >> result? I > >> > don't think using the asynchronous API will cause performance > >> regression. > >> > In most cases, the network latency is about 100ms or lower. You can > ping > >> > www.baidu.com or www.google.com to test the latency. > >> > > >> > > When executing a query in flink-sql-client, I often find error logs > of > >> > FlinkJobNotFoundException. Should this be optimized? > >> > > >> > It's related to the current client implementation. I think you can > open > >> a > >> > jira ticket, add more details and we can discuss the problem in the > >> ticket. > >> > In the FLIP-91, the Gateway can store the log per Operation. It may > >> solve > >> > your problems. > >> > > >> > > >> > Best, > >> > Shengkai > >> > > >
