maropu commented on pull request #29067: URL: https://github.com/apache/spark/pull/29067#issuecomment-658506540
> We want to produce a transparent replacement for .cache, .persist and the SQL CACHE operator using GPUs for acceleration. Caching data right now is slow on the CPU. .cache is a common enough operation that we would like to support it in our plugin and feel that it is something that we can really accelerate. In theory, we could reuse the datasource v2 API, but it would require a lot more refactoring to make it fit into the cache operator. Possibly refactoring of the data source V2 API as well. If that is what you think we need to do I can work on it, but it will be a much bigger change. hm, I think its worth documenting the proposal as SPIP. Then, we need to discuss it in the dev mailing list first (I'm currently not sure that other developers want this interface) WDYT? @HyukjinKwon @cloud-fan @dongjoon-hyun ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
