Hi, As far as I know, exporting data from a SQL database to a CSV file or other external file format is typically not considered part of the logical plan for executing a SQL query.
At present, I am developing a table sink feature in Datafusion, where I have successfully added new APIs (insert_into and copy_to) to the TableProvider trait. Although I have not yet submitted the PR, the new APIs are functioning well. Listing table INSERT INTO support (WAITING ARROW / OBJECT STORE UPDATE) by metesynnada · Pull Request #62 · synnada-ai/arrow-datafusion (github.com) <https://github.com/synnada-ai/arrow-datafusion/pull/62> Tt should be noted that the object_store crate is primarily responsible for the main functionality of the table sink feature. It provides ample support for file sinking related to listing tables. If you are looking for support beyond this, I'd like to hear the use for more help. Mete. On Sat, Apr 1, 2023 at 11:07 PM Jaroslaw Nowosad <yare...@gmail.com> wrote: > Hi, > > Looking for advice: > I'm looking into creating a writer part for ballista. > There is a data source but not a sink. > I started looking into object store -> put/put_multipart. > But looks like simple context extension is not enough - do I need to > extend logical/physical plan? > > If you have any pointers... > > Best Regards, > Jaro >
