[jira] [Commented] (SPARK-41628) Support async query execution
[ https://issues.apache.org/jira/browse/SPARK-41628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17708024#comment-17708024 ] ci-cassandra.apache.org commented on SPARK-41628: - User 'Hisoka-X' has created a pull request for this issue: https://github.com/apache/spark/pull/40649 > Support async query execution > - > > Key: SPARK-41628 > URL: https://issues.apache.org/jira/browse/SPARK-41628 > Project: Spark > Issue Type: Sub-task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Martin Grund >Priority: Major > > Today the query execution is completely synchronous, add an additional > asynchronous API that allows to submit and polll for the result. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41628) Support async query execution
[ https://issues.apache.org/jira/browse/SPARK-41628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707580#comment-17707580 ] Martin Grund commented on SPARK-41628: -- I think this needs a bit more discussion. Generally, I think it would be possible to model the asynchronous execution using the existing model. A couple of things would be needed up front: 1. `ExecutePlanRequest` needs an execution mode to indicate if the client wants a blocking request or non-blocking, this should probably be an enum 2. `ExecutePlanResponse` needs to indicate the query ID to be able to resume the query to block and fetch results 3. We need a way to check the status, but this could probably be modeled as a `Command` (e.g. QueryStatusCommand) Once this API is specced out, the next step is to identify how to perform the query execution in the background so that the results can be fetched when available. My suggestion would be to prepare a small doc on what exactly you're doing so that we can have a discussion. Feel free to do the design for this in a README in a pull request if this is preferred. > Support async query execution > - > > Key: SPARK-41628 > URL: https://issues.apache.org/jira/browse/SPARK-41628 > Project: Spark > Issue Type: Sub-task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Martin Grund >Priority: Major > > Today the query execution is completely synchronous, add an additional > asynchronous API that allows to submit and polll for the result. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41628) Support async query execution
[ https://issues.apache.org/jira/browse/SPARK-41628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707513#comment-17707513 ] Jia Fan commented on SPARK-41628: - Hi, [~grundprinzip-db] . I have a question, Should I use grpc futureStub to implement asynchronous API? It just should add some connect client api. Or I should add new grpc method named `AsyncExecutePlanRequest` and return requestId. Then User can use another grpc method named `WaitResponse` to get `ExecutePlanResponse`? > Support async query execution > - > > Key: SPARK-41628 > URL: https://issues.apache.org/jira/browse/SPARK-41628 > Project: Spark > Issue Type: Sub-task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Martin Grund >Priority: Major > > Today the query execution is completely synchronous, add an additional > asynchronous API that allows to submit and polll for the result. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41628) Support async query execution
[ https://issues.apache.org/jira/browse/SPARK-41628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707501#comment-17707501 ] Jia Fan commented on SPARK-41628: - I'm working on it > Support async query execution > - > > Key: SPARK-41628 > URL: https://issues.apache.org/jira/browse/SPARK-41628 > Project: Spark > Issue Type: Sub-task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Martin Grund >Priority: Major > > Today the query execution is completely synchronous, add an additional > asynchronous API that allows to submit and polll for the result. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
