gaogaotiantian commented on PR #58892: URL: https://github.com/apache/spark/pull/58892#issuecomment-5768537100
I think a major problem here is that, the more explicit we make the modules, the more manual work is required to make the graph. We basically have to declare the dependencies between modules by ourselves. It's not trivial for pyspark (not sure about java side). A lot of the connect tests actually rely on the classic tests. It's kind of complicated to build the dependency graph between modules - that's why I used an automated solution. However, splitting source with tests still makes sense. We just need to consider the extra effort for managing dependencies. Oh and python source files can be tests (doctest). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
