RE: SparkR read.df Option type doesn't match

Yes - please see the code example on the SparkR API doc: http://spark.apache.org/docs/latest/api/R/read.df.html Suggestion or contribution to improve the doc is welcome! > Date: Thu, 26 Nov 2015 15:08:31 -0700 > From: s...@phemi.com > To: dev@spark.apache.org > Subject: Re: SparkR read.df

RE: Are we running SparkR tests in Jenkins?

I think that breaks sparkR, the commandline script, and Jenkins, in which run-test.sh is calling sparkR. I'll work on this - since this also affects my PR #10652... Date: Fri, 15 Jan 2016 15:33:13 -0800 Subject: Re: Are we running SparkR tests in Jenkins? From: zjf...@gmail.com To:

RE: Fwd: Writing to jdbc database from SparkR (1.5.2)

I mean not exposed from the SparkR API. Calling it from R without a SparkR API would require either a serializer change or a JVM wrapper function. On Sun, Feb 7, 2016 at 4:47 AM -0800, "Felix Cheung" <felixcheun...@hotmail.com> wrote: That does but it's a bit hard to c

RE: Fwd: Writing to jdbc database from SparkR (1.5.2)

Correct :) _ From: Sun, Rui <rui@intel.com> Sent: Sunday, February 7, 2016 5:19 AM Subject: RE: Fwd: Writing to jdbc database from SparkR (1.5.2) To: <dev@spark.apache.org>, Felix Cheung <felixcheun...@hotmail.com>, Andrew

Re: [VOTE] Release Apache Spark 2.0.0 (RC5)

+1 Tested on Ubuntu, ran a bunch of SparkR tests, found a broken link in doc but not a blocker. _ From: Michael Armbrust > Sent: Friday, July 22, 2016 3:18 PM Subject: Re: [VOTE] Release Apache Spark 2.0.0 (RC5)

Re: PSA: Java 8 unidoc build

+1 for all the great work going in for this, HyukjinKwon, and +1 on what Sean says about "Jenkins builds with Java 8" and we should catch these nasty javadoc8 issue quickly. I think that would be the great first step to move away from java 7 _ From: Reynold Xin

Re: welcoming Burak and Holden as committers

Congrats and welcome!! From: Reynold Xin Sent: Tuesday, January 24, 2017 10:13:16 AM To: dev@spark.apache.org Cc: Burak Yavuz; Holden Karau Subject: welcoming Burak and Holden as committers Hi all, Burak and Holden have recently been

Re: welcoming Takuya Ueshin as a new Apache Spark committer

Congratulations! From: Xuefu Zhang Sent: Monday, February 13, 2017 11:29:12 AM To: Xiao Li Cc: Holden Karau; Reynold Xin; dev@spark.apache.org Subject: Re: welcoming Takuya Ueshin as a new Apache Spark committer Congratulations, Takuya! --Xuefu

Re: Feedback on MLlib roadmap process proposal

Hi Seth Re: "The most important thing we can do, given that MLlib currently has a very limited committer review bandwidth, is to make clear issues that, if worked on, will definitely get reviewed. " We are adopting a Shepherd model, as described in the JIRA Joseph has, in which, when

Re: PSA: JIRA resolutions and meanings

+1 on this proposal and everyone can contribute to updates and discussions on JIRAs Will be great if this could be put on the Spark wiki. On Sat, Oct 8, 2016 at 9:05 AM -0700, "Ted Yu" > wrote: Makes sense. I trust Hyukjin, Holden and

Re: Suggestion in README.md for guiding pull requests/JIRAs (probably about linking CONTRIBUTING.md or wiki)

Should we just link to https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark On Sun, Oct 9, 2016 at 10:09 AM -0700, "Hyukjin Kwon" > wrote: Thanks for confirming this, Sean. I filed this in

Re: [discuss] Spark 2.x release cadence

+1 on longer release cycle at schedule and more maintenance releases. _ From: Mark Hamstra > Sent: Tuesday, September 27, 2016 2:01 PM Subject: Re: [discuss] Spark 2.x release cadence To: Reynold Xin

Re: [VOTE] Release Apache Spark 2.0.1 (RC4)

+1 Tested and didn't find any blocker - found a few minor R doc issues to follow up. _ From: Reynold Xin > Sent: Wednesday, September 28, 2016 7:15 PM Subject: [VOTE] Release Apache Spark 2.0.1 (RC4) To:

Re: welcoming Xiao Li as a committer

Congrats and welcome, Xiao! _ From: Reynold Xin > Sent: Monday, October 3, 2016 10:47 PM Subject: welcoming Xiao Li as a committer To: Xiao Li >,

Re: SparkR issue with array types in gapply()

This is a R native data.frame behavior. While arr is a character vector of length = 2, > arr [1] "rows= 50" "cols= 2" > length(arr) [1] 2 when it is set as R data.frame the character vector is splitted into 2 rows > data.frame(key, strings = arr, stringsAsFactors = F) key strings 1 a rows=

Re: Question about SPARK-11374 (skip.header.line.count)

+1 I think it's useful to always have a pure SQL way and skip header for plain text / csv that lots of companies have. From: Dongjoon Hyun Sent: Friday, December 9, 2016 9:42:58 AM To: Dongjin Lee; dev@spark.apache.org Subject: Re: Question

Re: Spark GraphFrame ConnectedComponents

. From: Ankur Srivastava <ankur.srivast...@gmail.com> Sent: Thursday, January 5, 2017 3:45:59 PM To: Felix Cheung; dev@spark.apache.org Cc: u...@spark.apache.org Subject: Re: Spark GraphFrame ConnectedComponents Adding DEV mailing list to see if this is a defect with ConnectedComponent or if th

Re: ml word2vec finSynonyms return type

Could you link to the JIRA here? What you suggest makes sense to me. Though we might want to maintain compatibility and add a new method instead of changing the return type of the existing one. _ From: Asher Krim > Sent:

Re: [ML] [GraphFrames] : Bayesian Network framework

GraphFrames has a Belief Propagation example Have you checked it out? graphframes.github.io/api/scala/index.html#org.graphframes.examples.BeliefPropagation$ From:

Spark checkpointing

From: Steve Loughran <ste...@hortonworks.com> Sent: Friday, January 6, 2017 9:57:05 AM To: Ankur Srivastava Cc: Felix Cheung; u...@spark.apache.org Subject: Re: Spark GraphFrame ConnectedComponents On 5 Jan 2017, at 21:10, Ankur Srivastava <ankur.srivast...@gmail.com<m

Re: [VOTE] Apache Spark 2.1.0 (RC5)

nnot fit in memory. On Sat, Dec 17, 2016 at 10:23 AM, Joseph Bradley <jos...@databricks.com<mailto:jos...@databricks.com>> wrote: +1 On Fri, Dec 16, 2016 at 3:21 PM, Herman van Hövell tot Westerflier <hvanhov...@databricks.com<mailto:hvanhov...@databricks.com>> wrote: +1 O

Re: Outstanding Spark 2.1.1 issues

I've been scrubbing R and think we are tracking 2 issues https://issues.apache.org/jira/browse/SPARK-19237 https://issues.apache.org/jira/browse/SPARK-19925 From: holden.ka...@gmail.com on behalf of Holden Karau

Re: [VOTE] Apache Spark 2.1.1 (RC2)

-1 sorry, found an issue with SparkR CRAN check. Opened SPARK-20197 and working on fix. From: holden.ka...@gmail.com on behalf of Holden Karau Sent: Friday, March 31, 2017 6:25:20 PM To: Xiao Li Cc: Michael

Re: [SparkR] - options around setting up SparkSession / SparkContext

? _ From: Vin J <winjos...@gmail.com<mailto:winjos...@gmail.com>> Sent: Saturday, April 22, 2017 12:33 AM Subject: Re: [SparkR] - options around setting up SparkSession / SparkContext To: Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> C

Re: Should we consider a Spark 2.1.1 release?

+1 there are a lot of good fixes in overall and we need a release for Python and R packages. From: Holden Karau <hol...@pigscanfly.ca> Sent: Monday, March 13, 2017 12:06:47 PM To: Felix Cheung; Shivaram Venkataraman; dev@spark.apache.org Subject: Sho

Re: [VOTE] Apache Spark 2.2.0 (RC6)

+1 (non binding) Tested R, R package on Ubuntu and Windows, CRAN checks, manual tests with steaming & udf. _ From: Denny Lee > Sent: Monday, July 3, 2017 9:30 PM Subject: Re: [VOTE] Apache Spark 2.2.0 (RC6) To:

Re: [SparkR] - options around setting up SparkSession / SparkContext

How would you handle this in Scala? If you are adding a wrapper func like getSparkSession for Scala, and have your users call it, can't you do that same in SparkR? After all, while true you don't need a SparkSession object to call the R API, someone still needs to call sparkR.session() to

Re: [VOTE] Apache Spark 2.1.1 (RC4)

+1 Tested R on linux and windows Previous issue with building vignettes on windows with stackoverflow in ALS still reproduce but as confirmed the issue was in 2.1.0 so this isn't a regression (and hope for the best on CRAN..) https://issues.apache.org/jira/browse/SPARK-20402

Re: Welcoming Hyukjin Kwon and Sameer Agarwal as committers

Congrats!! From: Kevin Kim (Sangwoo) Sent: Monday, August 7, 2017 7:30:01 PM To: Hyukjin Kwon; dev Cc: Bryan Cutler; Mridul Muralidharan; Matei Zaharia; Holden Karau Subject: Re: Welcoming Hyukjin Kwon and Sameer Agarwal as committers Thanks

Re: SBT / PR builder builds failing on "include an external JAR in SparkR"

Facepalm I broken them - I was making changes to test files and of course Jenkins was only running only R tests since I was only changing R files, and everything passed there. Fix is Seq(sparkHome, "R", "pkg", "inst", "tests", To Seq(sparkHome, "R", "pkg", "tests", "fulltests", And 2

Re: [build system] rolling back R to working version

Thanks Shane! From: shane knapp Sent: Tuesday, June 20, 2017 9:23:57 PM To: dev Subject: Re: [build system] rolling back R to working version this is done... i backported R to 3.1.1 and reinstalled all the R packages so we're starting w/a

Re: [VOTE] Apache Spark 2.2.0 (RC4)

All tasks on the R QA umbrella are completed SPARK-20512 We can close this. _ From: Sean Owen > Sent: Tuesday, June 6, 2017 1:16 AM Subject: Re: [VOTE] Apache Spark 2.2.0 (RC4) To: Michael Armbrust

Re: [VOTE] Apache Spark 2.2.0 (RC4)

Hmm, that's odd. This test would be in Jenkins too - let me double check _ From: Nick Pentreath > Sent: Friday, June 9, 2017 1:12 AM Subject: Re: [VOTE] Apache Spark 2.2.0 (RC4) To: dev

Re: [VOTE] Apache Spark 2.2.0 (RC4)

<so...@cloudera.com<mailto:so...@cloudera.com>>, Nick Pentreath <nick.pentre...@gmail.com<mailto:nick.pentre...@gmail.com>>, Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> For the test failure on R, I checked: Per https://githu

Re: [VOTE] Apache Spark 2.2.0 (RC4)

Thanks! Will try to setup RHEL/CentOS to test it out _ From: Nick Pentreath <nick.pentre...@gmail.com<mailto:nick.pentre...@gmail.com>> Sent: Tuesday, June 13, 2017 11:38 PM Subject: Re: [VOTE] Apache Spark 2.2.0 (RC4) To: Felix Cheung <felixcheun

Re: [VOTE] Apache Spark 2.2.0 (RC4)

Sounds good. Think we checked and should be good to go. Appreciated. From: Michael Armbrust <mich...@databricks.com> Sent: Wednesday, June 14, 2017 4:51:48 PM To: Hyukjin Kwon Cc: Felix Cheung; Nick Pentreath; dev; Sean Owen Subject: Re: [VOTE] Apache Spark

Re: Spark 2.2.0 or Spark 2.3.0?

Yes 2.2.0 From: kant kodali Sent: Monday, May 1, 2017 10:43:44 PM To: dev Subject: Spark 2.2.0 or Spark 2.3.0? Hi All, If I understand the Spark standard release process correctly. It looks like the official release is going to be sometime

Re: [VOTE] Spark 2.1.2 (RC1)

+1 tested SparkR package on Windows, r-hub, Ubuntu. _ From: Sean Owen > Sent: Thursday, September 14, 2017 3:12 PM Subject: Re: [VOTE] Spark 2.1.2 (RC1) To: Holden Karau >,

Re: [VOTE] Spark 2.1.2 (RC1)

Yes ;) From: Xiao Li <gatorsm...@gmail.com> Sent: Friday, September 15, 2017 2:22:03 PM To: Holden Karau Cc: Ryan Blue; Denny Lee; Felix Cheung; Sean Owen; dev@spark.apache.org Subject: Re: [VOTE] Spark 2.1.2 (RC1) Sorry, this release candidate is

Re: Nightly builds for master branch failed

Hmm, sounds like some sort of corruption of the maven directory on the Jenkins box... From: Liwei Lin Sent: Wednesday, October 4, 2017 6:52:54 PM To: Spark dev list Subject: Nightly builds for master branch failed

Re: Disabling Closed -> Reopened transition for non-committers

To be sure, this is only for JIRA and not for github PR, right? If then +1 but I think the access control on JIRA does not necessarily match the committer list, and is manually maintained, last I hear. From: Sean Owen Sent: Wednesday,

Re: [VOTE] Spark 2.1.2 (RC4)

+1 Tested SparkR package manually on multiple platforms and checked different Hadoop release jar. And previously tested the last RC on different R releases (see the last RC vote thread) I found some differences in bin release jars created by the different options when running the

Re: [VOTE] Spark 2.1.2 (RC4)

Thanks Nick, Hyukjin. Yes this seems to be a longer standing issue on RHEL with respect to forking. From: Nick Pentreath Sent: Friday, October 6, 2017 6:16:53 AM To: Hyukjin Kwon Cc: dev Subject: Re: [VOTE] Spark 2.1.2 (RC4) Ah yes - I

Re: Nightly builds for master branch failed

Thanks Shane! From: shane knapp <skn...@berkeley.edu> Sent: Thursday, October 5, 2017 9:14:54 AM To: Felix Cheung Cc: Liwei Lin; Spark dev list Subject: Re: Nightly builds for master branch failed yep, it was a corrupted jar on amp-jenkins-worker-01. i g

Re: Putting Kafka 0.8 behind an (opt-in) profile

+1 From: Cody Koeninger Sent: Tuesday, September 5, 2017 8:12:07 AM To: Sean Owen Cc: dev Subject: Re: Putting Kafka 0.8 behind an (opt-in) profile +1 to going ahead and giving a deprecation warning now On Tue, Sep 5, 2017 at 6:39 AM, Sean

Re: 2.1.2 maintenance release?

+1 on both 2.1.2 and 2.2.1 And would try to help and/or wrangle the release if needed. (Note: trying to backport a few changes to branch-2.1 right now) From: Sean Owen Sent: Friday, September 8, 2017 12:05:28 AM To: Holden Karau; dev

Re: 2.1.2 maintenance release?

maintenance release? To: Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>>, Holden Karau <hol...@pigscanfly.ca<mailto:hol...@pigscanfly.ca>>, Sean Owen <so...@cloudera.com<mailto:so...@cloudera.com>>, dev <dev@spark.apache.org<mail

Re: [VOTE] Spark 2.1.2 (RC2)

-1 (Sorry) spark-2.1.2-bin-hadoop2.7.tgz is missing the R directory, not sure why yet. Tested on multiple platform as source package, (against 2.1.1 jar) seemed fine except this WARNING on R-devel * checking for code/documentation mismatches ... WARNING Codoc mismatches from documentation

Re: [VOTE][SPIP] SPARK-21190: Vectorized UDFs in Python

+1 on this and like the suggestion of type in string form. Would it be correct to assume there will be data type check, for example the returned pandas data frame column data types match what are specified. We have seen quite a bit of issues/confusions with that in R. Would it make sense to

Re: Updates on migration guides

+1 think we do migration guide changes for ML and R in separate JIRA/PR/commit but we definition should have it updated before the release. From: linguin@gmail.com Sent: Wednesday, August 30, 2017 8:27:17 AM To: Dongjoon Hyun Cc: Xiao

Re: Cutting the RC for Spark 2.2.1 release

). There should not be any issue targetting 2.2.1 except for SPARK-22042. As it is not a regression and it seems it might take a while, we won’t be blocking the release. _ From: Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> Sen

[VOTE] Spark 2.2.1 (RC1)

Please vote on releasing the following candidate as Apache Spark version 2.2.1. The vote is open until Monday November 20, 2017 at 23:00 UTC and passes if a majority of at least 3 PMC +1 votes are cast. [ ] +1 Release this package as Apache Spark 2.2.1 [ ] -1 Do not release this package because

Re: [VOTE] Spark 2.2.1 (RC1)

.org/jira/browse/SPARK-22469 >> >> Maybe also include this regression of 2.2? It works in 2.1 >> >> Thanks, >> >> Xiao >> >> >> >> 2017-11-14 22:25 GMT-08:00 Felix Cheung <felixche...@apache.org>: >> >>> Please vote on rel

Re: [VOTE] Spark 2.2.1 (RC2)

the latest Debian, so +1 for this release. > > (I committed the change to set -Xss4m for tests consistently, but this > shouldn't block a release.) > > > On Sat, Nov 25, 2017 at 12:47 PM Felix Cheung <felixche...@apache.org> > wrote: > >> Ah sorry digging through the

Re: [VOTE] Spark 2.2.1 (RC2)

d status 1 > > tar: Error is not recoverable: exiting now > > *** RUN ABORTED *** > > java.io.IOException: Cannot run program "./bin/spark-submit" (in > directory "/tmp/test-spark/spark-2.0.2"): error=2, No such file or directory > > On Sat, Nov 25

Re: [VOTE] Spark 2.2.1 (RC2)

, Nov 25, 2017 at 10:36 AM Felix Cheung <felixche...@apache.org> wrote: > Thanks Sean. > > For the second one, it looks like the HiveExternalCatalogVersionsSuite is > trying to download the release tgz from the official Apache mirror, which > won’t work unless the release

[VOTE] Spark 2.2.1 (RC2)

Please vote on releasing the following candidate as Apache Spark version 2.2.1. The vote is open until Friday December 1, 2017 at 8:00:00 am UTC and passes if a majority of at least 3 PMC +1 votes are cast. [ ] +1 Release this package as Apache Spark 2.2.1 [ ] -1 Do not release this package

[RESULT][VOTE] Spark 2.2.1 (RC2)

This vote passes. Thanks everyone for testing this release. +1: Sean Owen (binding) Herman van Hövell tot Westerflier (binding) Wenchen Fan (binding) Shivaram Venkataraman (binding) Felix Cheung Henry Robinson Hyukjin Kwon Dongjoon Hyun Kazuaki Ishizaki Holden Karau Weichen Xu 0

Re: [RESULT][VOTE] Spark 2.2.1 (RC2)

; access? We definitely need to give you all necessary access if you're the > release manager! > > > On Thu, Dec 14, 2017 at 6:32 AM Felix Cheung <felixche...@apache.org> > wrote: > >> And I don’t have access to publish python. >> >> On Wed, De

Re: [RESULT][VOTE] Spark 2.2.1 (RC2)

pretty ready. >> We should announce the release officially too then. >> >> On Wed, Dec 6, 2017 at 5:00 PM Felix Cheung <felixche...@apache.org> >> wrote: >> >>> I saw the svn move on Monday so I’m working on the website updates. >>> >>> I will lo

Re: [VOTE] Spark 2.2.1 (RC1)

ain; see > https://issues.apache.org/jira/browse/SPARK-16845?focusedCommentId=16018840=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16018840 > > > On Wed, Nov 15, 2017 at 12:25 AM Felix Cheung <felixche...@apache.org> > wrote: > > Please vote

Re: Cutting the RC for Spark 2.2.1 release

nterface at repository.apache.org either. ____ From: Felix Cheung <felixcheun...@hotmail.com> Sent: Monday, November 13, 2017 11:23:44 AM To: Sean Owen Cc: Holden Karau; dev@spark.apache.org Subject: Re: Cutting the RC for Spark 2.2.1 release Ouch ;) yes that works and RC1 is t

Re: Cutting the RC for Spark 2.2.1 release

Anything to build with maven on a clean machine. It couldn’t connect to maven central repo. From: Holden Karau <hol...@pigscanfly.ca> Sent: Monday, November 13, 2017 10:38:03 AM To: Felix Cheung Cc: dev@spark.apache.org Subject: Re: Cutting the RC for Spark

Re: Cutting the RC for Spark 2.2.1 release

Ouch ;) yes that works and RC1 is tagged. From: Sean Owen <so...@cloudera.com> Sent: Monday, November 13, 2017 10:54:48 AM To: Felix Cheung Cc: Holden Karau; dev@spark.apache.org Subject: Re: Cutting the RC for Spark 2.2.1 release It's repo.maven.apache.or

Re: Cutting the RC for Spark 2.2.1 release

VNCENTRAL-1369 Stay tuned. ________ From: Felix Cheung <felixcheun...@hotmail.com> Sent: Monday, November 13, 2017 12:00:41 AM To: dev@spark.apache.org Subject: Re: Cutting the RC for Spark 2.2.1 release Quick update: We merged 6 fixes Friday and 7 fixes today (than

Re: Cutting the RC for Spark 2.2.1 release

: Monday, November 13, 2017 10:48 AM Subject: Re: Cutting the RC for Spark 2.2.1 release To: Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> Cc: Holden Karau <hol...@pigscanfly.ca<mailto:hol...@pigscanfly.ca>>, <dev@spark.apache.org<mailto:d

Re: [CANCEL] Spark 2.2.1 (RC1)

This vote is cancelled due to no vote. I’m going to test or track down a few issues (please see link below for those targeting this release) and roll RC2 in a few days if we could make progress. On Tue, Nov 14, 2017 at 10:25 PM Felix Cheung <felixche...@apache.org> wrote: > Pl

Re: Kicking off the process around Spark 2.2.1

For the 2.2.1, we are still working through a few bugs. Hopefully it won't be long. From: Kevin Grealish <kevin...@microsoft.com> Sent: Thursday, November 2, 2017 9:51:56 AM To: Felix Cheung; Sean Owen; Holden Karau Cc: dev@spark.apache.org Subject: RE: K

Re: Kicking off the process around Spark 2.2.1

canfly.ca> Sent: Thursday, November 2, 2017 12:47:13 PM To: Reynold Xin Cc: Felix Cheung; Sean Owen; dev@spark.apache.org Subject: Re: Kicking off the process around Spark 2.2.1 I agree, except in this case we probably want some of the fixes that are going into the maintenance release t

Cutting the RC for Spark 2.2.1 release

Hi! As we are closing down on the few known issues I think we are ready to tag and cut the 2.2.1 release. If you are aware of any issue that you think should go into this release please feel free to ping me and mark the JIRA as targeting 2.2.1. I will be scrubbing JIRA in the next few days.

Re: Cutting the RC for Spark 2.2.1 release

Thanks Dongjoon! I will track that. From: Dongjoon Hyun <dongjoon.h...@gmail.com> Sent: Wednesday, November 8, 2017 7:41:20 PM To: Holden Karau Cc: Felix Cheung; dev@spark.apache.org Subject: Re: Cutting the RC for Spark 2.2.1 release It's great,

Re: [RESULT][VOTE] Spark 2.2.1 (RC2)

ome question about getting a hand in finishing the release process, > including copying artifacts in svn. Was there anything else you're waiting > on someone to do? > > > On Fri, Dec 1, 2017 at 2:10 AM Felix Cheung <felixche...@apache.org> > wrote: > >> This vote pas

Re: CRAN SparkR package removed?

Yes - unfortunately something was found after it was published and made available publicly. We have a JIRA on this and are working on the best course of action. _ From: Holden Karau > Sent: Wednesday, October 25,

Re: Revisiting Online serving of Spark models?

Huge +1 on this! From: holden.ka...@gmail.com on behalf of Holden Karau Sent: Thursday, May 10, 2018 9:39:26 AM To: Joseph Bradley Cc: dev Subject: Re: Revisiting Online serving of Spark models? On Thu, May 10,

Re: Running lint-java during PR builds?

One concern is with the volume of test runs on Travis. In ASF projects Travis could get significantly backed up since - if I recall - all of ASF shares one queue. At the number of PRs Spark has this could be a big issue. From: Marcelo Vanzin

Re: Revisiting Online serving of Spark models?

+1 on meeting up! From: Holden Karau <hol...@pigscanfly.ca> Sent: Monday, May 21, 2018 2:52:20 PM To: Joseph Bradley Cc: Felix Cheung; dev Subject: Re: Revisiting Online serving of Spark models? (Oh also the write API has already been extended to take f

Re: SparkR was removed from CRAN on 2018-05-01

This is the fix https://github.com/apache/spark/commit/f27a035daf705766d3445e5c6a99867c11c552b0#diff-e1e1d3d40573127e9ee0480caf1283d6 I don't have the email though. From: Hossein Sent: Friday, May 25, 2018 10:58:42 AM To: dev@spark.apache.org

Re: Revisiting Online serving of Spark models?

be interested in getting together to discuss/get some feedback? _ From: Felix Cheung <felixcheun...@hotmail.com> Sent: Thursday, May 10, 2018 10:10 AM Subject: Re: Revisiting Online serving of Spark models? To: Holden Karau <hol...@pigscanfly.ca>, Joseph

Re: Integrating ML/DL frameworks with Spark

Very cool. We would be very interested in this. What is the plan forward to make progress in each of the three areas? From: Bryan Cutler Sent: Monday, May 14, 2018 11:37:20 PM To: Xiangrui Meng Cc: Reynold Xin; dev Subject: Re: Integrating

Re: Scala 2.12 support

+1 Spoke to Dean as well and mentioned the problem with 2.11.12 https://github.com/scala/bug/issues/10913 _ From: Sean Owen Sent: Wednesday, June 6, 2018 12:23 PM Subject: Re: Scala 2.12 support To: Holden Karau Cc: Dean Wampler , Reynold Xin , dev If it means

Re: Time for 2.2.2 release

+1 and thanks! From: Tom Graves Sent: Wednesday, June 6, 2018 7:54:43 AM To: Dev Subject: Time for 2.2.2 release Hello all, I think its time for another 2.2 release. I took a look at Jira and I don't see anything explicitly targeted for 2.2.2 that is not yet

Re: [CRAN-pretest-archived] CRAN submission SparkR 2.3.1

For #1 is system requirements not honored? For #2 it looks like Oracle JDK? From: Shivaram Venkataraman Sent: Tuesday, June 12, 2018 3:17:52 PM To: dev Cc: Felix Cheung Subject: Fwd: [CRAN-pretest-archived] CRAN submission SparkR 2.3.1 Corresponding

Re: Revisiting Online serving of Spark models?

2018 7:47:07 AM To: Maximiliano Felice Cc: Leif Walsh; Felix Cheung; Holden Karau; Joseph Bradley; dev Subject: Re: Revisiting Online serving of Spark models? I’m in the same exact boat as Maximiliano and have use cases as well for model serving and would love to join this discussion. Sent f

Re: Revisiting Online serving of Spark models?

: Re: Revisiting Online serving of Spark models? To: Maximiliano Felice Cc: Felix Cheung , Holden Karau , Joseph Bradley , Leif Walsh , dev Would love to join but am in Seattle, thoughts on how to make this work? Regards Sent from my iPhone On May 29, 2018, at 10:35 AM, Maximiliano Felice

Re: [VOTE] SPIP ML Pipelines in R

+1 With my concerns in the SPIP discussion. From: Hossein Sent: Wednesday, May 30, 2018 2:03:03 PM To: dev@spark.apache.org Subject: [VOTE] SPIP ML Pipelines in R Hi, I started discussion

Re: Revisiting Online serving of Spark models?

Bump. From: Felix Cheung Sent: Saturday, May 26, 2018 1:05:29 PM To: Saikat Kanjilal; Maximiliano Felice; Joseph Bradley Cc: Leif Walsh; Holden Karau; dev Subject: Re: Revisiting Online serving of Spark models? Hi! How about we meet the community and discuss

Re: Revisiting Online serving of Spark models?

You had me at blue bottle! _ From: Holden Karau Sent: Tuesday, May 29, 2018 9:47 AM Subject: Re: Revisiting Online serving of Spark models? To: Felix Cheung Cc: Saikat Kanjilal , Maximiliano Felice , Joseph Bradley , Leif Walsh , dev I'm down for that, we could

Re: [VOTE] Spark 2.1.3 (RC2)

(I don’t want to block the release(s) per se...) We need to backport SPARK-22281 (to branch-2.1 and branch-2.2) This is fixed in 2.3 back in Nov 2017 https://github.com/apache/spark/commit/2ca5aae47a25dc6bc9e333fb592025ff14824501#diff-e1e1d3d40573127e9ee0480caf1283d6 Perhaps we don't get

Re: [VOTE] Spark 2.1.3 (RC2)

Yes, this is broken with newer version of R. We check explicitly for warning for the R check which should fail the test run. From: Marcelo Vanzin Sent: Wednesday, June 27, 2018 6:55 PM To: Felix Cheung Cc: Marcelo Vanzin; Tom Graves; dev Subject: Re: [VOTE

Re: SparkR test failures in PR builder

This is resolved. Please see https://issues.apache.org/jira/browse/SPARK-24152 From: Kazuaki Ishizaki Sent: Wednesday, May 2, 2018 4:51:11 PM To: dev Cc: Joseph Bradley; Hossein Falaki Subject: Re: SparkR test failures in PR builder I am

Re: Kicking off the process around Spark 2.2.1

2017 4:39:15 AM To: Holden Karau Cc: Felix Cheung; dev@spark.apache.org Subject: Re: Kicking off the process around Spark 2.2.1 It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah. Note also that in theory the code freeze for

Re: Kubernetes backend and docker images

+1 Thanks for taking on this. That was my feedback on one of the long comment thread as well, I think we should have one docker image instead of 3 (also pending in the fork are python and R variant, we should consider having one that we official release instead of 9, for example)

Re: Integration testing and Scheduler Backends

How would (2) be uncommon elsewhere? On Mon, Jan 8, 2018 at 10:16 PM Anirudh Ramanathan wrote: > This is with regard to the Kubernetes Scheduler Backend and scaling the > process to accept contributions. Given we're moving past upstreaming > changes from our fork, and into

Re: data source v2 online meetup

+1 hangout From: Xiao Li Sent: Wednesday, January 31, 2018 10:46:26 PM To: Ryan Blue Cc: Reynold Xin; dev; Wenchen Fen; Russell Spitzer Subject: Re: data source v2 online meetup Hi, Ryan, wow, your Iceberg already used data source V2 API!

Re: [VOTE] Spark 2.3.0 (RC4)

Any idea with sql func docs search result returning broken links as below? From: Felix Cheung <felixcheun...@hotmail.com> Sent: Sunday, February 18, 2018 10:05:22 AM To: Sameer Agarwal; Sameer Agarwal Cc: dev Subject: Re: [VOTE] Spark 2.3.0 (RC4) Quick que

Re: [VOTE] Spark 2.3.0 (RC4)

Felix Cheung <felixcheun...@hotmail.com> Cc: dev <dev@spark.apache.org> Maybe I misunderstand, but I don't see any .iml file in the 4 results on that page? it looks reasonable. On Mon, Feb 19, 2018 at 8:02 PM Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotma

Re: [VOTE] Spark 2.3.0 (RC4)

not be in the release) Thanks! _ From: Shivaram Venkataraman <shiva...@eecs.berkeley.edu> Sent: Tuesday, February 20, 2018 2:24 AM Subject: Re: [VOTE] Spark 2.3.0 (RC4) To: Felix Cheung <felixcheun...@hotmail.com> Cc: Sean Owen <sro...@gmail.com>, dev <

Re: [VOTE] Spark 2.3.0 (RC4)

Quick questions: is there search link for sql functions quite right? https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc4-docs/_site/api/sql/search.html?q=app this file shouldn't be included? https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc4-bin/spark-parent_2.11.iml

Re: Help needed in R documentation generation

This is recent change. The html file column_math_functions.html should have the right help content. What is the problem you are experiencing? From: Mihály Tóth Sent: Sunday, February 25, 2018 10:42:50 PM To: dev@spark.apache.org Subject:

Re: Timeline for Spark 2.3

+1 I think the earlier we cut a branch the better. From: Michael Armbrust Sent: Tuesday, December 19, 2017 4:41:44 PM To: Holden Karau Cc: Sameer Agarwal; Erik Erlandson; dev Subject: Re: Timeline for Spark 2.3 Do people really need to be

  1   2   3   >