Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r202535016
--- Diff: docs/running-on-kubernetes.md ---
@@ -117,6 +117,37 @@ If the local proxy is running at localhost:8001,
`--master k8s://http://127.0.0
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r202534636
--- Diff: docs/running-on-kubernetes.md ---
@@ -399,18 +426,18 @@ specific to Spark on Kubernetes.
Path to the CA cert file
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21589
AFAIK, we always have num of executor and then num of core per executor
right?
https://spark.apache.org/docs/latest/configuration.html#execution-behavior
maybe we should have
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21749
@skonto perhaps you folks can give some guidance on this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21749
agreed, it's not ideal to use reflection...
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21710
we could, but we tend to forget about them until the release, so I'd prefer
we don't and have everything in one PR.
not ideal, we could leave out code example, or maybe even vigenette
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201914125
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
inv
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201913817
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
inv
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r201913578
--- Diff: R/pkg/R/context.R ---
@@ -435,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
inv
Repository: spark
Updated Branches:
refs/heads/master 5ff1b9ba1 -> 006e798e4
[SPARK-23461][R] vignettes should include model predictions for some ML models
## What changes were proposed in this pull request?
Add model predictions for Linear Support Vector Machine (SVM) Classifier,
Logistic
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21678
merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21645#discussion_r201574780
--- Diff: R/pkg/R/functions.R ---
@@ -3071,6 +3085,19 @@ setMethod("array_position",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21645#discussion_r201574237
--- Diff: R/pkg/R/functions.R ---
@@ -3120,6 +3147,24 @@ setMethod("arrays_overlap",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21645#discussion_r201574700
--- Diff: R/pkg/R/functions.R ---
@@ -3120,6 +3147,24 @@ setMethod("arrays_overlap",
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3060
maybe it's better to clarify that we are adding the Admin REST API in this
change and also allow logging lever to be changed?
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21260
merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21584
yap
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Repository: spark
Updated Branches:
refs/heads/master 74a8d6308 -> 5ff1b9ba1
[SPARK-23529][K8S] Support mounting volumes
This PR continues #21095 and intersects with #21238. I've added volume mounts
as a separate step and added PersistantVolumeClaim support.
There is a fundamental problem
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20665
@tnachen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20665
any thought on this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21150
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21150
@tnachen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21033
ping @yanji84
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21075
@tnachen what do you think?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21027
@tnachen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21006
@tnachen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21027
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21006
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21260
ok that makes sense. could we get some clear guidance in SPARK-24434 on how
to decide what should be a conf and what should be in an external template file
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21260
@skonto is it better to generalize the approach to match the one in
https://issues.apache.org/jira/browse/SPARK-24435?
not sure if @mccheah @foxish @erikerlandson have any last thought
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21584
IMO it's fine we have one version supported in the image and stick with
that.
the tricky thing is having maintainers to keep updating/testing the newer
versions in the images (we have
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21260
Will review today.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21666
merged to master/2.3
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
ted?
unit test, manual, win-builder
Author: Felix Cheung
Closes #21666 from felixcheung/rjavaskip.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/141953f4
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/141953f4
D
tch tested?
unit test, manual, win-builder
Author: Felix Cheung
Closes #21666 from felixcheung/rjavaskip.
(cherry picked from commit 141953f4c44dbad1c2a7059e92bec5fe770af932)
Signed-off-by: Felix Cheung
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21666
sorry for the delay, I couldn't reply earlier - the fix was ready but I was
hoping for a review.
anyway, I'm going to merge it now
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21666#discussion_r199713355
--- Diff: R/pkg/R/client.R ---
@@ -61,6 +61,11 @@ generateSparkSubmitArgs <- function(args, sparkHome,
jars, sparkSubmitOpts, p
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21678
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3035
looks like we should port to branch-0.8 too
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21678
ok then, thanks
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3035
merging if no more comment
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3015
how about this
https://github.com/apache/zeppelin/pull/3015#discussion_r194955050 and
https://github.com/apache/zeppelin/pull/3015#discussion_r194955106?
---
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r199706822
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3905,6 +3905,16 @@ setMethod("rollup",
group
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r199328450
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3905,6 +3905,16 @@ setMethod("rollup",
group
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r199328283
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3905,6 +3905,16 @@ setMethod("rollup",
group
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r199328269
--- Diff: R/pkg/tests/fulltests/test_sparkSQL.R ---
@@ -2370,6 +2370,15 @@ test_that("join(), crossJoin() and merge() on a
Data
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r199328430
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3905,6 +3905,16 @@ setMethod("rollup",
group
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r199328470
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3905,6 +3905,16 @@ setMethod("rollup",
group
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21680
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21669
btw, have you sent out this + doc to d...@spark.apache.org?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21675
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21666#discussion_r199313370
--- Diff: R/pkg/R/client.R ---
@@ -61,6 +61,11 @@ generateSparkSubmitArgs <- function(args, sparkHome,
jars, sparkSubmitOpts, p
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r199313334
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3905,6 +3905,18 @@ setMethod("rollup",
group
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21666
@shivaram
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21666
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user felixcheung opened a pull request:
https://github.com/apache/spark/pull/21666
[SPARK-24535][SPARKR] fix tests on java check error
## What changes were proposed in this pull request?
## How was this patch tested?
unit test, manual
You can merge
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21462
ping @mccheah @liyinan926 on
https://github.com/apache/spark/pull/21462#discussion_r197909535
---
-
To unsubscribe, e-mail
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r199057309
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3905,6 +3905,18 @@ setMethod("rollup",
group
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21495
I donât feel strongly either way.
Agreed we should track it with JIRA
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21546#discussion_r198723785
--- Diff: dev/make-distribution.sh ---
@@ -168,10 +168,10 @@ export MAVEN_OPTS="${MAVEN_OPTS:--Xmx2g
-XX:ReservedCodeCacheSize
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198723017
--- Diff: R/pkg/R/context.R ---
@@ -451,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
inv
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198722933
--- Diff: R/pkg/R/context.R ---
@@ -451,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
inv
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198722972
--- Diff: R/pkg/R/context.R ---
@@ -451,3 +435,31 @@ setCheckpointDir <- function(directory) {
sc <- getSparkContext()
inv
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r198722458
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3905,6 +3905,18 @@ setMethod("rollup",
group
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r198722124
--- Diff: R/pkg/tests/fulltests/test_sparkSQL.R ---
@@ -2370,6 +2370,15 @@ test_that("join(), crossJoin() and merge() on a
Data
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r198722311
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3905,6 +3905,18 @@ setMethod("rollup",
group
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21649#discussion_r198722659
--- Diff: R/pkg/R/DataFrame.R ---
@@ -3905,6 +3905,18 @@ setMethod("rollup",
group
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3035
can you fix up the truncated PR title?
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3033
well, ok. though might be hard to tell - we are juggling multiple releases
2.1.3, 2.2.2, 2.3.2 - might take some time to get to 2.4.0.
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21495
I think itâd be great to not change the order
---
-
To unsubscribe, e-mail
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3033
Perhaps you could elaborate on why we should do this now since Spark
doesnât actually support it? Is there a reason that I have missed?
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3033
? it looks like spark.version is 2.2.0.
as far as I know SPARK-24418 is fixed only in (unreleased) master branch
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3034
hmm, Spark doesn't support Scala 2.12?
---
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21553#discussion_r198375614
--- Diff: docs/configuration.md ---
@@ -456,33 +456,6 @@ Apart from these, the following properties are also
available, and may be useful
from
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21260
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21589
what's the convention here, I thought SparkContext has get* methods instead
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21589#discussion_r198219194
--- Diff: R/pkg/R/context.R ---
@@ -25,6 +25,22 @@ getMinPartitions <- function(sc, minPartitions) {
as.integer(minPartiti
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21553#discussion_r198218069
--- Diff: docs/configuration.md ---
@@ -456,33 +456,6 @@ Apart from these, the following properties are also
available, and may be useful
from
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20641
yap, that sounds like a good idea
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20640
great, any other comment before merging?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20641
@krcz how is your PR different from 21014?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3033
something in SparkILoop. given that I don't think we can/should upgrade
scala broadly - it might only work for Spark 2.4.0 (unreleased) but not
older/current releases
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3019
can we open a pop up with options, like clicking on a url link in google
doc?
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3033
specifically in SPARK-24418 - there code changes needed but not release yet
(2.4?)
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21495
db might be out for a bit. we should merge it if it looks/tested good?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3031
ok
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3031
I guess?
How do the user find the Spark UI url though?
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3038
Looks fine.
Isnât this config has other implications though, that it wonât wait for
the app to finish?
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21584
We have a min supported version of R
https://github.com/apache/spark/blob/master/R/pkg/DESCRIPTION
But beyond that we havenât seen much problems when new releases of R are
coming
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21575
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21575
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r195896675
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala
---
@@ -0,0 +1,187
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r195896638
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala
---
@@ -0,0 +1,187
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21551
ouch
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21551
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21555
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21555
can you edit the title to add `[k8s]` and fix the text which is truncated?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21572
Jenkins, test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21462#discussion_r195639021
--- Diff:
resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh ---
@@ -45,12 +45,10 @@ shift 1
SPARK_CLASSPATH
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20641
@mgummelt too
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
501 - 600 of 6371 matches
Mail list logo
