[
https://issues.apache.org/jira/browse/SPARK-41628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707580#comment-17707580
]
Martin Grund commented on SPARK-41628:
--------------------------------------
I think this needs a bit more discussion. Generally, I think it would be
possible to model the asynchronous execution using the existing model. A couple
of things would be needed up front:
1. `ExecutePlanRequest` needs an execution mode to indicate if the client wants
a blocking request or non-blocking, this should probably be an enum
2. `ExecutePlanResponse` needs to indicate the query ID to be able to resume
the query to block and fetch results
3. We need a way to check the status, but this could probably be modeled as a
`Command` (e.g. QueryStatusCommand)
Once this API is specced out, the next step is to identify how to perform the
query execution in the background so that the results can be fetched when
available.
My suggestion would be to prepare a small doc on what exactly you're doing so
that we can have a discussion. Feel free to do the design for this in a README
in a pull request if this is preferred.
> Support async query execution
> -----------------------------
>
> Key: SPARK-41628
> URL: https://issues.apache.org/jira/browse/SPARK-41628
> Project: Spark
> Issue Type: Sub-task
> Components: Connect
> Affects Versions: 3.4.0
> Reporter: Martin Grund
> Priority: Major
>
> Today the query execution is completely synchronous, add an additional
> asynchronous API that allows to submit and polll for the result.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
