Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20641
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20641
ok, could you rebase this PR
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21531
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21092
@lucashu1 please send your question to stackoverflow or
u...@spark.apache.org!
---
-
To unsubscribe, e-mail: reviews
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/2974
I think the green light can be easily mis-interpreted. If you have any
other suggestion on the icon to use perhaps we could revisit.
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/2373
@xixikaikai
https://github.com/apache/zeppelin/pull/2373#issuecomment-323267967
---
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21546#discussion_r194954051
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -3236,13 +3236,49 @@ class Dataset[T] private[sql
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21533#discussion_r194953025
--- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala ---
@@ -116,49 +116,52 @@ class SparkContextSuite extends SparkFunSuite
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21533#discussion_r194953034
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1517,9 +1517,12 @@ class SparkContext(config: SparkConf) extends
Logging
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/2974
isn't it better to the loading/status light on the interpreter tab like in
the screen shot in https://github.com/apache/zeppelin/pull/1155?
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/2991
@Savalek can you update the title of this PR?
something like "change Description "auto-restart interpreter on cron
execution""
---
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21370#discussion_r194951791
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -3209,6 +3222,19 @@ class Dataset[T] private[sql
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21370#discussion_r194951918
--- Diff: docs/configuration.md ---
@@ -456,6 +456,33 @@ Apart from these, the following properties are also
available, and may be useful
from
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3014
LGTM
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20640
@IgorBerman any thought on this comment?
https://github.com/apache/spark/pull/20640#discussion_r191272487
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21515#discussion_r194625766
--- Diff: dev/create-release/vote.tmpl ---
@@ -0,0 +1,64 @@
+Please vote on releasing the following candidate as Apache Spark version
{version
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/19528
Jenkins test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21515#discussion_r194625981
--- Diff: dev/.rat-excludes ---
@@ -106,3 +106,4 @@ spark-warehouse
structured-streaming/*
kafka-source-initial-offset-version-2.1.0.bin
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21515#discussion_r194626614
--- Diff: dev/create-release/spark-rm/Dockerfile ---
@@ -0,0 +1,89 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21515#discussion_r194626379
--- Diff: dev/.rat-excludes ---
@@ -106,3 +106,4 @@ spark-warehouse
structured-streaming/*
kafka-source-initial-offset-version-2.1.0.bin
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21067
any update?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the pull request:
https://github.com/apache/zeppelin/commit/b7d98b3f119c7dcdfc7064cd56be8fdb8bbaabd9#commitcomment-29313036
In
spark/interpreter/src/main/java/org/apache/zeppelin/spark/SparkVersion.java:
In
spark/interpreter/src/main/java/org
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21202
@devaraj-kavali could you rebase this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21202
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21221
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21221
probably need to be rebased
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21452
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21438
I think filtering off `metricIds` still make sense right? @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21438
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20838
any update?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20272
Is this aligned with the "in cluster client"? @foxish @mccheah
---
-
To unsubscribe, e-mail: review
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20640
ping @IgorBerman
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20640
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20503
we still need to fix this, right?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21402
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21427
I think the config switch is for maintaining backward compatibility in case
someone is hit with this. so I think it's a good idea
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21092
awesome!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21482#discussion_r194170615
--- Diff: R/pkg/NAMESPACE ---
@@ -281,6 +281,8 @@ exportMethods("%<=>%",
"initcap",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21494#discussion_r194168480
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -179,6 +182,21 @@ private[spark] abstract class BasePythonRunner
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21462#discussion_r192925875
--- Diff:
resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh ---
@@ -46,12 +46,6 @@ shift 1
SPARK_CLASSPATH
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21313#discussion_r192925340
--- Diff: R/pkg/R/functions.R ---
@@ -3006,6 +3008,27 @@ setMethod("array_contains",
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/18447
I guess we still have an open question on
https://github.com/apache/spark/pull/18447/files#r130032578
---
-
To unsubscribe
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21390
Thanks @JoshRosen for very detailed and thoughtful reply.
Agreed TTL could be fragile, but I was very concern with what point
> There is a related issue where shuffle files
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21313#discussion_r192627641
--- Diff: R/pkg/R/functions.R ---
@@ -3006,6 +3008,27 @@ setMethod("array_contains",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21313#discussion_r192627948
--- Diff: R/pkg/R/functions.R ---
@@ -3006,6 +3008,27 @@ setMethod("array_contains",
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3000
@zjffdu
---
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21482#discussion_r192577235
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -1107,6 +1107,14 @@ object functions {
*/
def
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21482#discussion_r192577261
--- Diff: R/pkg/NAMESPACE ---
@@ -281,6 +281,8 @@ exportMethods("%<=>%",
"initcap",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21482#discussion_r192577245
--- Diff: R/pkg/R/generics.R ---
@@ -695,6 +695,12 @@ setGeneric("getField", function(x, ...) {
standardGeneric("getField"
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21482#discussion_r192577270
--- Diff: R/pkg/R/functions.R ---
@@ -907,6 +907,17 @@ setMethod("initcap",
column(jc)
})
+#
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21313#discussion_r192577048
--- Diff: R/pkg/R/functions.R ---
@@ -3006,6 +3008,27 @@ setMethod("array_contains",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21313#discussion_r192576992
--- Diff: R/pkg/R/functions.R ---
@@ -3006,6 +3008,27 @@ setMethod("array_contains",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21313#discussion_r192577008
--- Diff: R/pkg/R/functions.R ---
@@ -3006,6 +3008,27 @@ setMethod("array_contains",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21486#discussion_r192576961
--- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ---
@@ -197,14 +197,14 @@ private[spark] class HeartbeatReceiver(sc
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21480
merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Repository: spark
Updated Branches:
refs/heads/master 8ef167a5f -> a36c1a6bb
[SPARK-23668][K8S] Added missing config property in running-on-kubernetes.md
## What changes were proposed in this pull request?
PR https://github.com/apache/spark/pull/20811 introduced a new Spark
configuration
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21482#discussion_r192551752
--- Diff: R/pkg/R/functions.R ---
@@ -907,6 +907,30 @@ setMethod("initcap",
column(jc)
})
+#
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21482#discussion_r192551777
--- Diff: R/pkg/NAMESPACE ---
@@ -281,6 +281,8 @@ exportMethods("%<=>%",
"initcap",
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/2974
hmm, but I can think of the opposite case too - that the interpreter is
per-note and the user wants to have access to restart it, and not get locked
down. sounds like a broader issue
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21474#discussion_r192551649
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -429,7 +429,11 @@ package object config {
"ext
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21476#discussion_r192551676
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
---
@@ -1485,6 +1486,22 @@ private object Client extends
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21390
sounds like we need to have some sort of lifetime management, TTL -like
design shuffle file - should we have a new JIRA
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/2993
Iâm all for not having special config though Iâd prefer not to the
break users with existing note or config right away, without some sort of
deprecation period. But itâs your call
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21378
merged to master. thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21474#discussion_r192295318
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -429,7 +429,11 @@ package object config {
"ext
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21476#discussion_r192295158
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
---
@@ -1485,6 +1486,22 @@ private object Client extends
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21452
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21362#discussion_r191986143
--- Diff: R/pkg/tests/fulltests/test_sparkSQL.R ---
@@ -1504,15 +1504,16 @@ test_that("column functions", {
expect_equal(re
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21362
ok, let us know if you have more information on this
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21362
We havenât seen it fail?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21378#discussion_r191842162
--- Diff: docs/running-on-mesos.md ---
@@ -753,6 +753,16 @@ See the [configuration page](configuration.html) for
information on Spark config
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21434
merged to master, thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Repository: spark
Updated Branches:
refs/heads/master f48938800 -> a4be981c0
[SPARK-24331][SPARKR][SQL] Adding arrays_overlap, array_repeat, map_entries to
SparkR
## What changes were proposed in this pull request?
The PR adds functions `arrays_overlap`, `array_repeat`, `map_entries` to
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r191653358
--- Diff: pom.xml ---
@@ -760,6 +760,12 @@
1.10.19
test
+
--- End diff --
right
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21452
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21378#discussion_r191647592
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala
---
@@ -418,17 +417,33
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21378#discussion_r191271894
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala
---
@@ -418,17 +417,34
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21378#discussion_r191272157
--- Diff: docs/running-on-mesos.md ---
@@ -753,6 +753,16 @@ See the [configuration page](configuration.html) for
information on Spark config
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/20640#discussion_r191272509
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala
---
@@ -648,14
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/20640#discussion_r191272487
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala
---
@@ -571,7
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21438
Please update the PR description.
Also do you mean "show" -> "slow"?
---
-
To unsubscribe, e
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21434#discussion_r191090953
--- Diff: R/pkg/R/functions.R ---
@@ -3062,6 +3077,21 @@ setMethod("array_sort",
column(jc)
})
+#
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21434#discussion_r191090821
--- Diff: R/pkg/R/functions.R ---
@@ -207,7 +208,7 @@ NULL
#' tmp <- mutate(df, v1 = create_array(df$mpg, df$cyl, df$hp))
#' head(select(
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21434#discussion_r191090619
--- Diff: R/pkg/R/functions.R ---
@@ -207,7 +208,7 @@ NULL
#' tmp <- mutate(df, v1 = create_array(df$mpg, df$cyl, df$hp))
#' head(select(
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21434#discussion_r191090863
--- Diff: R/pkg/R/functions.R ---
@@ -3048,6 +3048,26 @@ setMethod("array_position",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21434#discussion_r191090725
--- Diff: R/pkg/R/functions.R ---
@@ -3048,6 +3048,26 @@ setMethod("array_position",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21434#discussion_r191090696
--- Diff: R/pkg/R/functions.R ---
@@ -3048,6 +3048,26 @@ setMethod("array_position",
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/2900
ping
---
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r191061920
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleEventHandler.scala
---
@@ -0,0
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r191061883
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -154,6 +154,24 @@ private[spark] object
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r191061890
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -154,6 +154,24 @@ private[spark] object
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r191061877
--- Diff: pom.xml ---
@@ -760,6 +760,12 @@
1.10.19
test
+
--- End diff --
I think I'm a bit
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r191061934
--- Diff: pom.xml ---
@@ -760,6 +760,12 @@
1.10.19
test
+
--- End diff --
also add
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r191061892
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala
---
@@ -0,0 +1,120
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21438
please see http://spark.apache.org/contributing.html on "Pull Request"
also fix the PR title to start with `[S
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21378#discussion_r191056648
--- Diff: docs/running-on-mesos.md ---
@@ -753,6 +753,16 @@ See the [configuration page](configuration.html) for
information on Spark config
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21378#discussion_r191056976
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala
---
@@ -418,17 +417,34
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21434#discussion_r191056304
--- Diff: R/pkg/R/functions.R ---
@@ -3062,6 +3077,21 @@ setMethod("array_sort",
column(jc)
})
+#
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21434#discussion_r191056286
--- Diff: R/pkg/R/functions.R ---
@@ -3048,6 +3048,21 @@ setMethod("array_position",
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21397#discussion_r191040434
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -94,8 +88,18 @@ class ArrowPythonRunner
601 - 700 of 6371 matches
Mail list logo
