xinlifoobar commented on PR #10792: URL: https://github.com/apache/datafusion/pull/10792#issuecomment-2241423909
> Hi @xinlifoobar > > I am sorry for the delay in responding to this PR. This is an amazing piece of software engineering. Very nice 🎩 👌 > > As you have noted, the challege here is that the `hs_store` is non trivial and yet somewhat specialized for HuggingFace. It is a really neat feature but somewhat hard to justify adding to the datafusion-cli in the DataFusion repo. > > I feel like there is a tension between making datafusion-cli easy to use with many built in integrations (e.g. hugging face, delta-rs, etc) and keeping the dependencies manageable > > What would you think about somehow moving this hugging face integration into another repo and making some version of datafusion-cli that had a bunch of pre-defined integrations? > > For example, maybe put it in https://github.com/datafusion-contrib/datafusion-cli-plus or something > > That could be like the power user version of datafusion-cli that we could use all the fun table providers (like what is in the connector libraries, etc), delta-rust, iceberg-rust, etc > > If you think this is a reasonable idea, I will file a ticket for larger discussion Hey @alamb, thanks for taking the time to review this PR. Great honor to me on this. I am glad to have a repo like `datafusion-cli-plus` but think of a broader project for a new repo. As you may have already observed, the datafusion-cli currently supports the Datafusion SQL interface. Are you considering expanding its capabilities to encompass protocols such as MySQL client, Arrow Flight SQL, and others? This expansion would entail naming it act to a full-fledged server. I've observed similar approaches being implemented in downstream projects like InfluxDB. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
