Ted-Jiang commented on issue #1780:
URL:
https://github.com/apache/arrow-datafusion/issues/1780#issuecomment-1039826718
> > > preemptively
> > > @houqp
> > > Sorry for my confusion , You mean if a job has 3 stage, when stage3 is
running, we can delete stage 1 first?
>
> IMO, I think when a SQL is finished, all the immediate shuffle data can be
cleared except for the result data.
@houqp @mingmwang It sounds very reasonable , i thinks this will handles
some error cases for robustness.
IMHO, keep both of them and create a separate issue to capture for future
improvement (maybe after separate shuffle data and result data).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
