gaogaotiantian commented on PR #58274:
URL: https://github.com/apache/spark/pull/58274#issuecomment-5419683392
> This way, we can move them between scheduled/periodic CI, pre-merge CI,
and post-merge CI without having to rename the groups.
We don't really have a way to move tests between these CIs. Whether the CI
runs relies on the module name (at least for now). The CI knows it should only
run `pyspark-install` on coverage, so it only runs on coverage and pre-merge.
I'm not against the idea to have a new mechanism to label where these tests
should run, but that's a much bigger change.
For our current infra, asking the CI to run `pyspark-periodic` for scheduled
CI makes sense. The alternative would be asking the CI to run
`pyspark-upstream` and `pyspark-install` on scheduled CI.
Feel free to take a look at the `pyspark` section of `build_and_test` and
propose your idea. Do you want something like
```yaml
- modules: ${{
fromJson(needs.precondition.outputs.required).pyspark-periodic != 'true' &&
'pyspark-upstream, pyspark-install' }}
```
that could be another option - having a switch and a list of modules under
it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
