[ https://issues.apache.org/jira/browse/SPARK-8422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-8422: ----------------------------------- Assignee: Josh Rosen (was: Apache Spark) > Introduce a module abstraction inside of dev/run-tests > ------------------------------------------------------ > > Key: SPARK-8422 > URL: https://issues.apache.org/jira/browse/SPARK-8422 > Project: Spark > Issue Type: Sub-task > Components: Build, Project Infra > Reporter: Josh Rosen > Assignee: Josh Rosen > > At a high level, we have Spark modules / components which > 1. are affected / impacted by file changes (e.g. a module is associated with > a set of source files, so changes to those files change the module), > 2. contain a set of tests to run, which are triggered via Maven, SBT, or via > Python / R scripts. > 3. depend on other modules and have dependent modules: if we change core, > then every downstream test should be run. If we change only MLlib, then we > can skip the SQL tests but should probably run the Python MLlib tests, etc. > Right now, the per-module logic is spread across a few different places > inside of the {{dev/run-tests}} script: we have one function that describes > how to detect changes for all modules, another function that (implicitly) > deals with module dependencies, etc. > Instead, I propose that we introduce a class for describing a module, use > instances of this class to build up a dependency graph, then phrase the "find > which tests to run" operations in terms of that graph. I think that this > will be easier to understand / maintain. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org