(cc @Yikun Jiang <yikunk...@gmail.com> @Gengliang Wang <gengliang.w...@databricks.com> @Maxim Gekk <maxim.g...@databricks.com> @Yang,Jie(INF) <yangji...@baidu.com> FYI)
On Wed, 22 Jun 2022 at 19:34, Hyukjin Kwon <gurwls...@gmail.com> wrote: > Couple of updates: > > - > > All builds passed now with all combinations we defined in the GitHub > Actions (e.g., branch-3.2, branch-3.3, JDK 11, > JDK 17 and Scala 2.13), see https://github.com/apache/spark/actions cc @Tom > Graves <tgraves...@yahoo.com> @Dongjoon Hyun <dongjoon.h...@gmail.com> > FYI > - > > except one test that is being failed due to OOM. That’s being fixed at > https://github.com/apache/spark/pull/36954, see > also https://github.com/apache/spark/pull/36787#discussion_r901190636 > - > > I am now adding PySpark, SparkR jobs to the scheduled builds at > https://github.com/apache/spark/pull/36940 > and see if they pass. We might need a couple of more fixes there. > - > > There’s one last task to simply caching the Docker image ( > https://issues.apache.org/jira/browse/SPARK-39522). > I will have to be less active for this week and next week because of > the Spark Summit. Would appreciate if somebody > finds some time to take a stab. > > About a quick hallway meetup, I will be there after Holden’s talk at least > to say hello to her :-). > Let’s have a quick chat about our CI. We still have some general problems > to cope with like the lack of resources in > GitHub Actions. > > > > On Tue, 21 Jun 2022 at 11:49, Hyukjin Kwon <gurwls...@gmail.com> wrote: > >> Just chatted offline - both I and Holden have multiple sessions :-). >> Probably let's meet up for a quick chat after your talk >> https://databricks.com/dataaisummit/session/what-do-when-your-job-goes-oom-night-flowcharts >> ? >> >> >> On Mon, 20 Jun 2022 at 22:23, Holden Karau <hol...@pigscanfly.ca> wrote: >> >>> How about a hallway meet up at Data AI summit to talk about build CI if >>> folks are >>> Interested? >>> >>> On Sun, Jun 19, 2022 at 7:50 PM Hyukjin Kwon <gurwls...@gmail.com> >>> wrote: >>> >>>> Increased the priority to a blocker - I don't think we can release with >>>> these build failures and poor CI >>>> >>>> On Mon, 20 Jun 2022 at 10:39, Hyukjin Kwon <gurwls...@gmail.com> wrote: >>>> >>>>> There are too many test failures here. I pinged in some PRs I could >>>>> identify from a cursory look but would be great for you guys to take a >>>>> look >>>>> if you guys haven't tested your change against other environments like JDK >>>>> 11, Scala 2.13. >>>>> >>>>> On Mon, 20 Jun 2022 at 10:04, Hyukjin Kwon <gurwls...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I am trying to rework GitHub Actions CI at >>>>>> https://issues.apache.org/jira/browse/SPARK-39515. Any help would be >>>>>> very appreciated. >>>>>> >>>>>> >>>>>> -- >>> Twitter: https://twitter.com/holdenkarau >>> Books (Learning Spark, High Performance Spark, etc.): >>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>> >>
