Just got reply from CRAN admin. It should be fixed now.
Hyukjin Kwon wrote > Thanks, Liang-chi. > > On Thu, 13 Dec 2018, 8:29 am Liang-Chi Hsieh < > viirya@ > wrote: > > > >> Sorry for late. There is a malformed record at CRAN package page again. >> I've >> already asked CRAN admin for help. It should be fixed soon according to >> past >> experience. >> >> Related discussion will be in >> https://issues.apache.org/jira/browse/SPARK-24152. I will post here if I >> get >> reply from CRAN admin. >> >> Thanks. >> >> >> Liang-Chi Hsieh wrote >> > Thanks for letting me know! I will look into it and ask CRAN admin for >> > help. >> > >> > >> > Hyukjin Kwon wrote >> >> Looks it's happening again. Liang-Chi, do you mind if I ask it again? >> >> >> >> FYI, R 3.4 is officially deprecated as of >> >> https://github.com/apache/spark/pull/23012 >> >> We could upgrade R version to 3.4.x in Jenkins, which deals with the >> >> malformed(?) responses after 3.0 release. >> >> Then, we could get rid of this problem..! >> >> >> >> 2018년 11월 12일 (월) 오후 1:47, Hyukjin Kwon < >> > >> >> gurwls223@ >> > >> >> >님이 작성: >> >> >> >>> I made a PR to officially drop R prior to version 3.4 ( >> >>> https://github.com/apache/spark/pull/23012). >> >>> The tests will probably fail for now since it produces warnings for >> >>> using >> >>> R 3.1.x. >> >>> >> >>> 2018년 11월 11일 (일) 오전 3:00, Felix Cheung < >> > >> >> felixcheung_m@ >> > >> >> >님이 작성: >> >>> >> >>>> It’s a great point about min R version. From what I see, mostly >> because >> >>>> of fixes and packages support, most users of R are fairly up to >> date? >> >>>> So >> >>>> perhaps 3.4 as min version is reasonable esp. for Spark 3. >> >>>> >> >>>> Are we getting traction with CRAN sysadmin? It seems like this has >> been >> >>>> broken a few times. >> >>>> >> >>>> >> >>>> ------------------------------ >> >>>> *From:* Liang-Chi Hsieh < >> > >> >> viirya@ >> > >> >> > >> >>>> *Sent:* Saturday, November 10, 2018 2:32 AM >> >>>> *To:* >> > >> >> dev@.apache >> > >> >>>> *Subject:* Re: [discuss] SparkR CRAN feasibility check server >> problem >> >>>> >> >>>> >> >>>> Yeah, thanks Hyukjin Kwon for bringing this up for discussion. >> >>>> >> >>>> I don't know how higher versions of R are widely used across R >> >>>> community. >> >>>> If >> >>>> R version 3.1.x was not very commonly used, I think we can discuss >> to >> >>>> upgrade minimum R version in next Spark version. >> >>>> >> >>>> If we ended up with not upgrading, we can discuss with CRAN sysadmin >> to >> >>>> fix >> >>>> it by the service side automatically that prevents malformed R >> packages >> >>>> info. So we don't need to fix it manually every time. >> >>>> >> >>>> >> >>>> >> >>>> Hyukjin Kwon wrote >> >>>> >> Can upgrading R able to fix the issue. Is this perhaps not >> >>>> necessarily >> >>>> > malform but some new format for new versions perhaps? >> >>>> > That's my guess. I am not totally sure about it tho. >> >>>> > >> >>>> >> Anyway we should consider upgrading R version if that fixes the >> >>>> problem. >> >>>> > Yea, we should. If we should, it should be more them R 3.4. Maybe >> >>>> it's >> >>>> > good >> >>>> > time to start to talk about minimum R version. 3.1.x is too old. >> It's >> >>>> > released 4.5 years ago. >> >>>> > R 3.4.0 is released 1.5 years ago. Considering the timing for >> Spark >> >>>> 3.0, >> >>>> > deprecating lower versions, bumping up R to 3.4 might be >> reasonable >> >>>> > option. >> >>>> > >> >>>> > Adding Shane as well. >> >>>> > >> >>>> > If we ended up with not upgrading it, I will forward this email to >> >>>> CRAN >> >>>> > sysadmin to discuss further anyway. >> >>>> > >> >>>> > >> >>>> > >> >>>> > 2018년 11월 2일 (금) 오후 12:51, Felix Cheung < >> >>>> >> >>>> > felixcheung@ >> >>>> >> >>>> > >님이 작성: >> >>>> > >> >>>> >> Thanks for being this up and much appreciate with keeping on top >> of >> >>>> this >> >>>> >> at all times. >> >>>> >> >> >>>> >> Can upgrading R able to fix the issue. Is this perhaps not >> >>>> necessarily >> >>>> >> malform but some new format for new versions perhaps? Anyway we >> >>>> should >> >>>> >> consider upgrading R version if that fixes the problem. >> >>>> >> >> >>>> >> As an option we could also disable the repo check in Jenkins but >> I >> >>>> can >> >>>> >> see >> >>>> >> that could also be problematic. >> >>>> >> >> >>>> >> >> >>>> >> On Thu, Nov 1, 2018 at 7:35 PM Hyukjin Kwon < >> >>>> >> >>>> > gurwls223@ >> >>>> >> >>>> > > wrote: >> >>>> >> >> >>>> >>> Hi all, >> >>>> >>> >> >>>> >>> I want to raise the CRAN failure issue because it started to >> block >> >>>> Spark >> >>>> >>> PRs time to time. Since the number >> >>>> >>> of PRs grows hugely in Spark community, this is critical to not >> >>>> block >> >>>> >>> other PRs. >> >>>> >>> >> >>>> >>> There has been a problem at CRAN (See >> >>>> >>> https://github.com/apache/spark/pull/20005 for analysis). >> >>>> >>> To cut it short, the root cause is malformed package info from >> >>>> >>> https://cran.r-project.org/src/contrib/PACKAGES >> >>>> >>> from server side, and this had to be fixed by requesting it to >> CRAN >> >>>> >>> sysaadmin's help. >> >>>> >>> >> >>>> >>> https://issues.apache.org/jira/browse/SPARK-24152 <- newly open. >> I >> >>>> am >> >>>> >>> pretty sure it's the same issue >> >>>> >>> https://issues.apache.org/jira/browse/SPARK-25923 <- >> >>>> reopen/resolved >> >>>> 2 >> >>>> >>> times >> >>>> >>> https://issues.apache.org/jira/browse/SPARK-22812 >> >>>> >>> >> >>>> >>> This happened 5 times for roughly about 10 months, causing >> blocking >> >>>> >>> almost all PRs in Apache Spark. >> >>>> >>> Historically, it blocked whole PRs for few days once, and whole >> >>>> Spark >> >>>> >>> community had to stop working. >> >>>> >>> >> >>>> >>> I assume this has been not a super big big issue so far for >> other >> >>>> >>> projects or other people because apparently >> >>>> >>> higher version of R has some logics to handle this malformed >> >>>> documents >> >>>> >>> (at least I verified R 3.4.0 works fine). >> >>>> >>> >> >>>> >>> For our side, Jenkins has low R version (R 3.1.1 if that's not >> >>>> updated >> >>>> >>> from what I have seen before), >> >>>> >>> which is unable to parse the malformed server's response. >> >>>> >>> >> >>>> >>> So, I want to talk about how we are going to handle this. >> Possible >> >>>> >>> solutions are: >> >>>> >>> >> >>>> >>> 1. We should start a talk with CRAN sysadmin to permanently >> prevent >> >>>> this >> >>>> >>> issue >> >>>> >>> 2. We upgrade R to 3.4.0 in Jenkins (however we will not be able >> to >> >>>> test >> >>>> >>> low R versions) >> >>>> >>> 3. ... >> >>>> >>> >> >>>> >>> If if we fine, I would like to suggest to forward this email to >> >>>> CRAN >> >>>> >>> sysadmin to discuss further about this. >> >>>> >>> >> >>>> >>> Adding Liang-Chi Felix and Shivaram who I already talked about >> this >> >>>> few >> >>>> >>> times before. >> >>>> >>> >> >>>> >>> Thanks all. >> >>>> >>> >> >>>> >>> >> >>>> >>> >> >>>> >>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Sent from: >> http://apache-spark-developers-list.1001551.n3.nabble.com/ >> >>>> >> >>>> >> --------------------------------------------------------------------- >> >>>> To unsubscribe e-mail: >> > >> >> dev-unsubscribe@.apache >> > >> >>>> >> >>>> >> > >> > >> > >> > >> > >> > -- >> > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ >> > >> > --------------------------------------------------------------------- >> > To unsubscribe e-mail: >> >> > dev-unsubscribe@.apache >> >> >> >> >> >> -- >> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: > dev-unsubscribe@.apache >> >> -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
