houqp commented on issue #1273: URL: https://github.com/apache/arrow-datafusion/issues/1273#issuecomment-964848613
I agree with @alamb that we could really benefit from having someone to actively drive ballista. Even though I mentioned in my dev list email that ballista introduces extra overhead, I think it acted as a good forcing function for us to design datafusion changes with serailization in mind as you mentioned. For example, It triggered a lot of good discussions around object store serialization in @rdettai 's recent refactoring. If we decided that the overhead is not worth it later, we can just make ballista depend on datafusion by published crate version instead of path so it's less decoupled from datafusion code change. I think this should address all the overhead concerns. I don't think we need to do this right now though. I think it's best for us to come back to the discuss of moving ballista out of datafusion when it gets significant momentum. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
