zhengruifeng commented on PR #55726: URL: https://github.com/apache/spark/pull/55726#issuecomment-4402226975
@gaogaotiantian > All workflows MUST have a job concurrency level less than or equal to 20. This means a workflow cannot have more than 20 jobs running at the same time across all matrices. I am not sure whether it takes a slot, since now the pyspark task depend on the new `precompile` take and they are not executed at the same time. @dongjoon-hyun @HyukjinKwon @LuciferYang one the compression, I tried `zstd` at first, but `zstd` is not available in images, we have to install it in all images, to minimize the code change, I changed it to `xz` https://github.com/apache/spark/pull/55726/changes/00c3fc9db774c4bf0c75bb8926ac8d394a826670 , but `xz` is too slow, so finally I chose `bzip2`. I think we can optimize this part later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
