Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/22011#discussion_r208122669
--- Diff: python/pyspark/rdd.py ---
@@ -2429,6 +2441,29 @@ def _wrap_function(sc, func, deserializer,
serializer, profiler=None
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r208120759
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -60,14 +61,26 @@ private[spark] object PythonEvalType
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3112
please close this @GioCorti80
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21988
we always open against master and backport if agreed upon.
this is documented here https://spark.apache.org/contributing.html
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3107
I think the advantage of having a script separate from demo is that it can
be ACL'd properly - user won't have access etc.
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3092
so we need to open a JIRA then?
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3054
this comment?
https://github.com/apache/zeppelin/pull/3054#discussion_r200825969
it's in every file touched
---
Repository: zeppelin
Updated Branches:
refs/heads/master 2eac6872e -> cbf222848
[ZEPPELIN-3665] fix notebook name
### What is this PR for?
Name of notebook incorrectly displayed due to text direction.
Bug Fix
### What is the Jira issue?
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/22001#discussion_r207745122
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosFineGrainedSchedulerBackend.scala
---
@@ -453,4 +453,8
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/22001#discussion_r207745108
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosFineGrainedSchedulerBackend.scala
---
@@ -453,4 +453,8
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/22001#discussion_r207745157
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -496,6 +496,12 @@ class
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207726897
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_P
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207726903
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
---
@@ -133,10 +133,17 @@ private[yarn] class
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20838
jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20838
looks to me everything passes
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21996#discussion_r207717051
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -98,17 +98,24 @@ private[spark] class SparkSubmit extends Logging
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21996
I think we generally describe the change in PR title. what user see you can
put as JIRA title.
---
-
To unsubscribe, e-mail
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21977
build error
```
[error]
/home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala:88:
method sparkContext
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207716854
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_P
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207716877
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
---
@@ -133,10 +133,17 @@ private[yarn] class
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207716893
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
---
@@ -133,10 +133,17 @@ private[yarn] class
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207716813
--- Diff: python/pyspark/worker.py ---
@@ -259,6 +260,26 @@ def main(infile, outfile):
"PYSPARK_DRIVER_P
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21997#discussion_r207716713
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceRDD.scala
---
@@ -124,8 +124,6 @@ private[kafka010] class
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21997
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21997
@koeninger back
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21987
one hive suite failed, is that related to this change?
HiveExternalCatalogVersionsSuite
```
[info] 2018-08-03 17:14:26.108 - stdout> 17:14:26.107 W
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21996#discussion_r207702200
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -98,17 +98,24 @@ private[spark] class SparkSubmit extends Logging
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21988
why do we need multiple PRs? typically we do that only then the change is
non-trivial or cannot be backported by cherry-picking
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21983
@YuvalItzchakov you should open the PR against master - it can be picked to
release branch (eg. 2.3) when merged
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21991#discussion_r207701760
--- Diff: dev/merge_spark_pr.py ---
@@ -154,20 +154,22 @@ def merge_pr(pr_num, target_ref, title, body,
pr_repo_desc):
# to people every
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21991
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21953
Jenkins, test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21006
@pmackles ?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3066
looks like everything passed
https://travis-ci.org/TinkoffCreditSystems/zeppelin/builds/407590408
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3092
right. just want to clarify that it is perfectly fine to run Scala, Python
etc without Spark. The session context is maintained in the REPL implementation
and not in SparkContext
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21546
so... can we still target this for 2.4? sounds like very good to have and
complimentary to hydrogen
---
-
To unsubscribe
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21516
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21923
this https://github.com/squito/spark-memory
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21909
got it
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3099
cool
merging if no more comment
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3103
I see, would be good to update the title - I think that goes into the
actual commit
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3102
cool!
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21930
I think that's binary-incompatible breaking API change, right?
ex.
https://github.com/apache/spark/pull/21930/files#diff-2b8f0f66fe5397b169d0f754e99da8d5R64
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21936#discussion_r206769869
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -571,7 +571,12 @@ class SparkContext(config: SparkConf) extends Logging
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21936#discussion_r206770131
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -571,7 +571,12 @@ class SparkContext(config: SparkConf) extends Logging
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3092
no, I mean
paragraph 1
```
%java
Map counts = new HashMap<>();
```
paragraph 2
```
%java
counts.put("key", 1);
```
---
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21928#discussion_r206401496
--- Diff: python/pyspark/serializers.py ---
@@ -236,6 +237,11 @@ def create_array(s, t):
# TODO: need decode before converting to Arrow
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21928#discussion_r206401644
--- Diff: python/pyspark/serializers.py ---
@@ -236,6 +237,11 @@ def create_array(s, t):
# TODO: need decode before converting to Arrow
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21928
yea, it doesn't seem very useful to ping matei on every single PR ;)
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21909#discussion_r206400571
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVBenchmarks.scala
---
@@ -119,8 +119,47 @@ object CSVBenchmarks
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3092
did you add the behavior in doc?
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21928
I wonder if we could tune the bot suggestions to more recent
contributions/contributors?
---
-
To unsubscribe, e-mail
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21905
hmm, interesting suggestions
https://github.com/apache/spark/pull/21905#issuecomment-408580441
---
-
To unsubscribe, e-mail
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21902#discussion_r205933175
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1449,6 +1451,16 @@ object SQLConf {
.intConf
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20028
this shouldn't say version 2.3.0?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20272
still need to run tests
https://github.com/apache/spark/pull/20272#pullrequestreview-108271893
---
-
To unsubscribe, e
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r205932789
--- Diff:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/ClientModeTestsSuite.scala
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3099
it might help if you include a before / after screen shots?
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21516
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21884
can you update the format of the title and description as described here
"Pull Request" in https://spark.apache.org/contrib
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3090
open ip and port to connect to has become a huge problem recently, so
unless LSP has some sort of authentication story, my vote would be "no" even if
this is disabled by defaul
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21584
SPARK-24908 went in, it should unblock R?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21835
and my comment
https://github.com/apache/spark/pull/21835#discussion_r20541
---
-
To unsubscribe, e-mail: reviews
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21835#discussion_r205355152
--- Diff: R/pkg/R/functions.R ---
@@ -3320,7 +3321,7 @@ setMethod("explode",
#' @aliases sequence sequence,Column-method
#' @not
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21584#discussion_r205354798
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -71,8 +75,9
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21835#discussion_r20541
--- Diff: R/pkg/tests/fulltests/test_context.R ---
@@ -21,10 +21,11 @@ test_that("Check masked functions", {
# Check that we are not m
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21811
SGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r204635755
--- Diff: docs/running-on-kubernetes.md ---
@@ -117,6 +117,45 @@ If the local proxy is running at localhost:8001,
`--master k8s://http://127.0.0
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r204635584
--- Diff: docs/running-on-kubernetes.md ---
@@ -117,6 +117,45 @@ If the local proxy is running at localhost:8001,
`--master k8s://http://127.0.0
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r204636048
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala
---
@@ -35,26 +35,39
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21546#discussion_r204635055
--- Diff: python/pyspark/serializers.py ---
@@ -184,27 +184,67 @@ def loads(self, obj):
raise NotImplementedError
-class
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21788
what's changed? it was forced pushed so I can't tell what's updated
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21835#discussion_r204633191
--- Diff: R/pkg/R/functions.R ---
@@ -1986,15 +1998,20 @@ setMethod("levenshtein", signature(y = "Column"),
#' are on t
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21835#discussion_r204633464
--- Diff: R/pkg/tests/fulltests/test_context.R ---
@@ -21,10 +21,11 @@ test_that("Check masked functions", {
# Check that we are not m
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21118
so where are we on this? looks like we have 2 LGTM?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21075
@tnachen @susanxhuynh @mgummelt @skonto
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21516
@tnachen @susanxhuynh @mgummelt @skonto
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21516
Jenkins, ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3035
we don't have a 0.9 branch or plan yet. should we backport to 0.8? are we
having another release, like 0.8.1?
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3035
I don't this is ported to 0.8
https://github.com/apache/zeppelin/commits/branch-0.8
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3066
can you comment on my comment:
`this doesn't seem quite right, we are ignoring all IOException from
running the paragraph?`?
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21584
I see. you are referring to
https://github.com/apache/spark/pull/21584#issuecomment-406091994
got
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21517
I think we can close this
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21584
looks like test failed
```
- Run SparkR on simple dataframe.R example *** FAILED ***
The code passed to eventually never returned normally. Attempted 70 times
over
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21584#discussion_r204229514
--- Diff: bin/docker-image-tool.sh ---
@@ -79,11 +80,16 @@ function build {
docker build $NOCACHEARG "${BINDING_BUILD_ARGS[@]}" \
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21584#discussion_r204229713
--- Diff:
resource-managers/kubernetes/docker/src/main/dockerfiles/spark/bindings/R/Dockerfile
---
@@ -0,0 +1,29 @@
+#
+# Licensed
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21584#discussion_r204229663
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -71,8 +75,9
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21584#discussion_r204229518
--- Diff: bin/docker-image-tool.sh ---
@@ -79,11 +80,16 @@ function build {
docker build $NOCACHEARG "${BINDING_BUILD_ARGS[@]}" \
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3079
looks like we don't test older spark versions..
---
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/3068
do we really need this for all interpreters though? for those we don't have
active maintainer it might better not to change them
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20146
it's odd appveyer tests are not getting triggered.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21710#discussion_r203245118
--- Diff: R/pkg/R/mllib_fpm.R ---
@@ -154,3 +160,74 @@ setMethod("write.ml", signature(object =
"FPGrowthModel"
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21710#discussion_r203245010
--- Diff: R/pkg/R/generics.R ---
@@ -1415,6 +1415,13 @@ setGeneric("spark.freqItemsets", function(object) {
standardGeneric(&q
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21542
"permission" stuff might be Java 9 related?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.
Repository: spark
Updated Branches:
refs/heads/master bbc2ffc8a -> bcf7121ed
[TRIVIAL][ML] GMM unpersist RDD after training
## What changes were proposed in this pull request?
unpersist `instances` after training
## How was this patch tested?
existing tests
Author: éçå³°
Closes #21562
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21562
sure, LGTM.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21758#discussion_r202535313
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1386,29 +1418,90 @@ class DAGScheduler
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21758#discussion_r202533650
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -359,17 +368,49 @@ private[spark] class TaskSchedulerImpl
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21758#discussion_r202533477
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1386,29 +1418,90 @@ class DAGScheduler
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21758#discussion_r202533903
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -359,17 +368,49 @@ private[spark] class TaskSchedulerImpl
401 - 500 of 6371 matches
Mail list logo
