panbingkun commented on PR #47369: URL: https://github.com/apache/spark/pull/47369#issuecomment-2235681312
> from the log we can see > > ``` > Warning: [1888.608s][warning][gc,alloc] broadcast-exchange-907: Retried waiting for GCLocker too often allocating 33554434 words > ``` > > If this is the reason for the test failures, I think we should first investigate the root cause of the GCLocker's activity to avoid covering up any unknown performance bottlenecks. I have only encountered this `once` so far, and this is just a record. Additionally, I believe it originated from `TPCDSQueryTestSuite`. I compared it slightly with `TPCDSQueryTestSuite` and found that there are two major differences between it and `TPCDSQueryTestSuite`: `joinConfs` and `System.gc() // Workaround for GitHub Actions memory limitation, see also SPARK-37368` https://github.com/apache/spark/blob/3b4c423fec08b3740c20257e130c7b016a538e79/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala#L223-L224 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
