Thanks for being this up and much appreciate with keeping on top of this at all times.
Can upgrading R able to fix the issue. Is this perhaps not necessarily malform but some new format for new versions perhaps? Anyway we should consider upgrading R version if that fixes the problem. As an option we could also disable the repo check in Jenkins but I can see that could also be problematic. On Thu, Nov 1, 2018 at 7:35 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: > Hi all, > > I want to raise the CRAN failure issue because it started to block Spark > PRs time to time. Since the number > of PRs grows hugely in Spark community, this is critical to not block > other PRs. > > There has been a problem at CRAN (See > https://github.com/apache/spark/pull/20005 for analysis). > To cut it short, the root cause is malformed package info from > https://cran.r-project.org/src/contrib/PACKAGES > from server side, and this had to be fixed by requesting it to CRAN > sysaadmin's help. > > https://issues.apache.org/jira/browse/SPARK-24152 <- newly open. I am > pretty sure it's the same issue > https://issues.apache.org/jira/browse/SPARK-25923 <- reopen/resolved 2 > times > https://issues.apache.org/jira/browse/SPARK-22812 > > This happened 5 times for roughly about 10 months, causing blocking almost > all PRs in Apache Spark. > Historically, it blocked whole PRs for few days once, and whole Spark > community had to stop working. > > I assume this has been not a super big big issue so far for other projects > or other people because apparently > higher version of R has some logics to handle this malformed documents (at > least I verified R 3.4.0 works fine). > > For our side, Jenkins has low R version (R 3.1.1 if that's not updated > from what I have seen before), > which is unable to parse the malformed server's response. > > So, I want to talk about how we are going to handle this. Possible > solutions are: > > 1. We should start a talk with CRAN sysadmin to permanently prevent this > issue > 2. We upgrade R to 3.4.0 in Jenkins (however we will not be able to test > low R versions) > 3. ... > > If if we fine, I would like to suggest to forward this email to CRAN > sysadmin to discuss further about this. > > Adding Liang-Chi Felix and Shivaram who I already talked about this few > times before. > > Thanks all. > > > >
