Sorry for late. There is a malformed record at CRAN package page again. I've already asked CRAN admin for help. It should be fixed soon according to past experience.
Related discussion will be in https://issues.apache.org/jira/browse/SPARK-24152. I will post here if I get reply from CRAN admin. Thanks. Liang-Chi Hsieh wrote > Thanks for letting me know! I will look into it and ask CRAN admin for > help. > > > Hyukjin Kwon wrote >> Looks it's happening again. Liang-Chi, do you mind if I ask it again? >> >> FYI, R 3.4 is officially deprecated as of >> https://github.com/apache/spark/pull/23012 >> We could upgrade R version to 3.4.x in Jenkins, which deals with the >> malformed(?) responses after 3.0 release. >> Then, we could get rid of this problem..! >> >> 2018년 11월 12일 (월) 오후 1:47, Hyukjin Kwon < > >> gurwls223@ > >> >님이 작성: >> >>> I made a PR to officially drop R prior to version 3.4 ( >>> https://github.com/apache/spark/pull/23012). >>> The tests will probably fail for now since it produces warnings for >>> using >>> R 3.1.x. >>> >>> 2018년 11월 11일 (일) 오전 3:00, Felix Cheung < > >> felixcheung_m@ > >> >님이 작성: >>> >>>> It’s a great point about min R version. From what I see, mostly because >>>> of fixes and packages support, most users of R are fairly up to date? >>>> So >>>> perhaps 3.4 as min version is reasonable esp. for Spark 3. >>>> >>>> Are we getting traction with CRAN sysadmin? It seems like this has been >>>> broken a few times. >>>> >>>> >>>> ------------------------------ >>>> *From:* Liang-Chi Hsieh < > >> viirya@ > >> > >>>> *Sent:* Saturday, November 10, 2018 2:32 AM >>>> *To:* > >> dev@.apache > >>>> *Subject:* Re: [discuss] SparkR CRAN feasibility check server problem >>>> >>>> >>>> Yeah, thanks Hyukjin Kwon for bringing this up for discussion. >>>> >>>> I don't know how higher versions of R are widely used across R >>>> community. >>>> If >>>> R version 3.1.x was not very commonly used, I think we can discuss to >>>> upgrade minimum R version in next Spark version. >>>> >>>> If we ended up with not upgrading, we can discuss with CRAN sysadmin to >>>> fix >>>> it by the service side automatically that prevents malformed R packages >>>> info. So we don't need to fix it manually every time. >>>> >>>> >>>> >>>> Hyukjin Kwon wrote >>>> >> Can upgrading R able to fix the issue. Is this perhaps not >>>> necessarily >>>> > malform but some new format for new versions perhaps? >>>> > That's my guess. I am not totally sure about it tho. >>>> > >>>> >> Anyway we should consider upgrading R version if that fixes the >>>> problem. >>>> > Yea, we should. If we should, it should be more them R 3.4. Maybe >>>> it's >>>> > good >>>> > time to start to talk about minimum R version. 3.1.x is too old. It's >>>> > released 4.5 years ago. >>>> > R 3.4.0 is released 1.5 years ago. Considering the timing for Spark >>>> 3.0, >>>> > deprecating lower versions, bumping up R to 3.4 might be reasonable >>>> > option. >>>> > >>>> > Adding Shane as well. >>>> > >>>> > If we ended up with not upgrading it, I will forward this email to >>>> CRAN >>>> > sysadmin to discuss further anyway. >>>> > >>>> > >>>> > >>>> > 2018년 11월 2일 (금) 오후 12:51, Felix Cheung < >>>> >>>> > felixcheung@ >>>> >>>> > >님이 작성: >>>> > >>>> >> Thanks for being this up and much appreciate with keeping on top of >>>> this >>>> >> at all times. >>>> >> >>>> >> Can upgrading R able to fix the issue. Is this perhaps not >>>> necessarily >>>> >> malform but some new format for new versions perhaps? Anyway we >>>> should >>>> >> consider upgrading R version if that fixes the problem. >>>> >> >>>> >> As an option we could also disable the repo check in Jenkins but I >>>> can >>>> >> see >>>> >> that could also be problematic. >>>> >> >>>> >> >>>> >> On Thu, Nov 1, 2018 at 7:35 PM Hyukjin Kwon < >>>> >>>> > gurwls223@ >>>> >>>> > > wrote: >>>> >> >>>> >>> Hi all, >>>> >>> >>>> >>> I want to raise the CRAN failure issue because it started to block >>>> Spark >>>> >>> PRs time to time. Since the number >>>> >>> of PRs grows hugely in Spark community, this is critical to not >>>> block >>>> >>> other PRs. >>>> >>> >>>> >>> There has been a problem at CRAN (See >>>> >>> https://github.com/apache/spark/pull/20005 for analysis). >>>> >>> To cut it short, the root cause is malformed package info from >>>> >>> https://cran.r-project.org/src/contrib/PACKAGES >>>> >>> from server side, and this had to be fixed by requesting it to CRAN >>>> >>> sysaadmin's help. >>>> >>> >>>> >>> https://issues.apache.org/jira/browse/SPARK-24152 <- newly open. I >>>> am >>>> >>> pretty sure it's the same issue >>>> >>> https://issues.apache.org/jira/browse/SPARK-25923 <- >>>> reopen/resolved >>>> 2 >>>> >>> times >>>> >>> https://issues.apache.org/jira/browse/SPARK-22812 >>>> >>> >>>> >>> This happened 5 times for roughly about 10 months, causing blocking >>>> >>> almost all PRs in Apache Spark. >>>> >>> Historically, it blocked whole PRs for few days once, and whole >>>> Spark >>>> >>> community had to stop working. >>>> >>> >>>> >>> I assume this has been not a super big big issue so far for other >>>> >>> projects or other people because apparently >>>> >>> higher version of R has some logics to handle this malformed >>>> documents >>>> >>> (at least I verified R 3.4.0 works fine). >>>> >>> >>>> >>> For our side, Jenkins has low R version (R 3.1.1 if that's not >>>> updated >>>> >>> from what I have seen before), >>>> >>> which is unable to parse the malformed server's response. >>>> >>> >>>> >>> So, I want to talk about how we are going to handle this. Possible >>>> >>> solutions are: >>>> >>> >>>> >>> 1. We should start a talk with CRAN sysadmin to permanently prevent >>>> this >>>> >>> issue >>>> >>> 2. We upgrade R to 3.4.0 in Jenkins (however we will not be able to >>>> test >>>> >>> low R versions) >>>> >>> 3. ... >>>> >>> >>>> >>> If if we fine, I would like to suggest to forward this email to >>>> CRAN >>>> >>> sysadmin to discuss further about this. >>>> >>> >>>> >>> Adding Liang-Chi Felix and Shivaram who I already talked about this >>>> few >>>> >>> times before. >>>> >>> >>>> >>> Thanks all. >>>> >>> >>>> >>> >>>> >>> >>>> >>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe e-mail: > >> dev-unsubscribe@.apache > >>>> >>>> > > > > > > -- > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: > dev-unsubscribe@.apache -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
