Hi Martijn. It seems we have reached consensus to support the Gateway inside the Flink code base.
Hi, Echo. Thanks for your interest. > whether flink-sql-gateway should be retained in the Flink project. I think the discussion above is clear. It is the essential tool to provide out-of-box experience for users. > For stream processing, what's the point of getting the result? Is it just for debugging and how to unify with batch processing At the client side, when the OperationStaus is FINISHED, the client is able to fetch the results from the Gateway. It is unified with the batch processing now. > For batch processing, does the gateway need to cache all fetch results? No. In the Gateway, we will only buffer partial data and wait for the client to consume. If the client takes away the buffered data, the Gateway will clear the buffer and notify the fetcher thread starts to work until the buffer is full again. The mechanism behind is much like the producer-consumer model[1]. [1] https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem > Whether executing query and fetch results should be synchronous or asynchronous? Do you mean the executeStatement response should also contain the result? I don't think using the asynchronous API will cause performance regression. In most cases, the network latency is about 100ms or lower. You can ping www.baidu.com or www.google.com to test the latency. > When executing a query in flink-sql-client, I often find error logs of FlinkJobNotFoundException. Should this be optimized? It's related to the current client implementation. I think you can open a jira ticket, add more details and we can discuss the problem in the ticket. In the FLIP-91, the Gateway can store the log per Operation. It may solve your problems. Best, Shengkai
