[
https://issues.apache.org/jira/browse/SPARK-21693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121866#comment-16121866
]
Felix Cheung commented on SPARK-21693:
--------------------------------------
splitting test matrix is also possible, I worry though since caching is
disabled, then isn't Spark jar being built multiple times? My main concerns are
how long tests will run and whether that will lengthen queuing of test runs
(which could get quite long already and people are ignoring pending appveyor
runs sometimes)
> AppVeyor tests reach the time limit, 1.5 hours, sometimes in SparkR tests
> -------------------------------------------------------------------------
>
> Key: SPARK-21693
> URL: https://issues.apache.org/jira/browse/SPARK-21693
> Project: Spark
> Issue Type: Test
> Components: Build, SparkR
> Affects Versions: 2.3.0
> Reporter: Hyukjin Kwon
>
> We finally sometimes reach the time limit, 1.5 hours,
> https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/build/1676-master
> I requested to increase this from an hour to 1.5 hours before but it looks we
> should fix this in AppVeyor. I asked this for my account few times before but
> it looks we can't increase this time limit again and again.
> I could identify two things that look taking a quite a bit of time:
> 1. Disabled cache feature in pull request builder, which ends up downloading
> Maven dependencies (10-20ish mins)
> https://www.appveyor.com/docs/build-cache/
> {quote}
> Note: Saving cache is disabled in Pull Request builds.
> {quote}
> and also see
> http://help.appveyor.com/discussions/problems/4159-cache-doesnt-seem-to-be-working
> This seems difficult to fix within Spark.
> 2. "MLlib classification algorithms" tests (30-35ish mins)
> This test below looks taking 30-35ish mins.
> {code}
> MLlib classification algorithms, except for tree-based algorithms: Spark
> package found in SPARK_HOME: C:\projects\spark\bin\..
> ......................................................................
> {code}
> As a (I think) last resort, we could make a matrix for this test alone, so
> that we run the other tests after a build and then run this test after
> another build, for example, I run Scala tests by this workaround -
> https://ci.appveyor.com/project/spark-test/spark/build/757-20170716 (a matrix
> with 7 build and test each).
> I am also checking and testing other ways.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
