I made a PR to officially drop R prior to version 3.4 ( https://github.com/apache/spark/pull/23012). The tests will probably fail for now since it produces warnings for using R 3.1.x.
2018년 11월 11일 (일) 오전 3:00, Felix Cheung <felixcheun...@hotmail.com>님이 작성: > It’s a great point about min R version. From what I see, mostly because of > fixes and packages support, most users of R are fairly up to date? So > perhaps 3.4 as min version is reasonable esp. for Spark 3. > > Are we getting traction with CRAN sysadmin? It seems like this has been > broken a few times. > > > ------------------------------ > *From:* Liang-Chi Hsieh <vii...@gmail.com> > *Sent:* Saturday, November 10, 2018 2:32 AM > *To:* dev@spark.apache.org > *Subject:* Re: [discuss] SparkR CRAN feasibility check server problem > > > Yeah, thanks Hyukjin Kwon for bringing this up for discussion. > > I don't know how higher versions of R are widely used across R community. > If > R version 3.1.x was not very commonly used, I think we can discuss to > upgrade minimum R version in next Spark version. > > If we ended up with not upgrading, we can discuss with CRAN sysadmin to fix > it by the service side automatically that prevents malformed R packages > info. So we don't need to fix it manually every time. > > > > Hyukjin Kwon wrote > >> Can upgrading R able to fix the issue. Is this perhaps not necessarily > > malform but some new format for new versions perhaps? > > That's my guess. I am not totally sure about it tho. > > > >> Anyway we should consider upgrading R version if that fixes the problem. > > Yea, we should. If we should, it should be more them R 3.4. Maybe it's > > good > > time to start to talk about minimum R version. 3.1.x is too old. It's > > released 4.5 years ago. > > R 3.4.0 is released 1.5 years ago. Considering the timing for Spark 3.0, > > deprecating lower versions, bumping up R to 3.4 might be reasonable > > option. > > > > Adding Shane as well. > > > > If we ended up with not upgrading it, I will forward this email to CRAN > > sysadmin to discuss further anyway. > > > > > > > > 2018년 11월 2일 (금) 오후 12:51, Felix Cheung < > > > felixcheung@ > > > >님이 작성: > > > >> Thanks for being this up and much appreciate with keeping on top of this > >> at all times. > >> > >> Can upgrading R able to fix the issue. Is this perhaps not necessarily > >> malform but some new format for new versions perhaps? Anyway we should > >> consider upgrading R version if that fixes the problem. > >> > >> As an option we could also disable the repo check in Jenkins but I can > >> see > >> that could also be problematic. > >> > >> > >> On Thu, Nov 1, 2018 at 7:35 PM Hyukjin Kwon < > > > gurwls223@ > > > > wrote: > >> > >>> Hi all, > >>> > >>> I want to raise the CRAN failure issue because it started to block > Spark > >>> PRs time to time. Since the number > >>> of PRs grows hugely in Spark community, this is critical to not block > >>> other PRs. > >>> > >>> There has been a problem at CRAN (See > >>> https://github.com/apache/spark/pull/20005 for analysis). > >>> To cut it short, the root cause is malformed package info from > >>> https://cran.r-project.org/src/contrib/PACKAGES > >>> from server side, and this had to be fixed by requesting it to CRAN > >>> sysaadmin's help. > >>> > >>> https://issues.apache.org/jira/browse/SPARK-24152 <- newly open. I am > >>> pretty sure it's the same issue > >>> https://issues.apache.org/jira/browse/SPARK-25923 <- reopen/resolved 2 > >>> times > >>> https://issues.apache.org/jira/browse/SPARK-22812 > >>> > >>> This happened 5 times for roughly about 10 months, causing blocking > >>> almost all PRs in Apache Spark. > >>> Historically, it blocked whole PRs for few days once, and whole Spark > >>> community had to stop working. > >>> > >>> I assume this has been not a super big big issue so far for other > >>> projects or other people because apparently > >>> higher version of R has some logics to handle this malformed documents > >>> (at least I verified R 3.4.0 works fine). > >>> > >>> For our side, Jenkins has low R version (R 3.1.1 if that's not updated > >>> from what I have seen before), > >>> which is unable to parse the malformed server's response. > >>> > >>> So, I want to talk about how we are going to handle this. Possible > >>> solutions are: > >>> > >>> 1. We should start a talk with CRAN sysadmin to permanently prevent > this > >>> issue > >>> 2. We upgrade R to 3.4.0 in Jenkins (however we will not be able to > test > >>> low R versions) > >>> 3. ... > >>> > >>> If if we fine, I would like to suggest to forward this email to CRAN > >>> sysadmin to discuss further about this. > >>> > >>> Adding Liang-Chi Felix and Shivaram who I already talked about this few > >>> times before. > >>> > >>> Thanks all. > >>> > >>> > >>> > >>> > > > > > > -- > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
