[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19601 **[Test build #83931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83931/testReport)** for PR 19601 at commit

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-15 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r151340713 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/VectorizedHashMapGenerator.scala --- @@ -75,9 +77,14 @@ class

[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19601 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19601 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83930/ Test FAILed. ---

[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19601 **[Test build #83930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83930/testReport)** for PR 19601 at commit

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-15 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r151339166 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -473,16 +477,41 @@ private[spark] class MapOutputTrackerMaster( }

[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19601 **[Test build #83930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83930/testReport)** for PR 19601 at commit

[GitHub] spark issue #19764: [SPARK-22539][SQL] Add second order for rangepartitioner...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19764 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19621: [SPARK-11215][ML] Add multiple columns support to String...

2017-11-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19621 Seems in the frequency-based string orders, the order of labels with same frequency is non-deterministic. --- - To unsubscribe,

[GitHub] spark pull request #19764: [SPARK-22539][SQL] Add second order for rangepart...

2017-11-15 Thread caneGuy
GitHub user caneGuy opened a pull request: https://github.com/apache/spark/pull/19764 [SPARK-22539][SQL] Add second order for rangepartitioner since partition nu… …mber may be small if the specified key is skewed ## What changes were proposed in this pull request?

[GitHub] spark pull request #19630: [SPARK-22409] Introduce function type argument in...

2017-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r151333004 --- Diff: python/pyspark/sql/functions.py --- @@ -2208,26 +2089,39 @@ def udf(f=None, returnType=StringType()): | 8| JOHN DOE|

[GitHub] spark pull request #19630: [SPARK-22409] Introduce function type argument in...

2017-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r151335091 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala --- @@ -137,15 +138,18 @@ object ExtractPythonUDFs extends

[GitHub] spark pull request #19630: [SPARK-22409] Introduce function type argument in...

2017-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r151333227 --- Diff: python/pyspark/rdd.py --- @@ -56,6 +56,20 @@ __all__ = ["RDD"] +class PythonEvalType(object): +""" +Evaluation

[GitHub] spark pull request #19630: [SPARK-22409] Introduce function type argument in...

2017-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r151332296 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -34,9 +34,11 @@ import org.apache.spark.util._ */

[GitHub] spark pull request #19630: [SPARK-22409] Introduce function type argument in...

2017-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r151333891 --- Diff: python/pyspark/sql/functions.py --- @@ -2208,26 +2089,39 @@ def udf(f=None, returnType=StringType()): | 8| JOHN DOE|

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r151333181 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/VectorizedHashMapGenerator.scala --- @@ -75,9 +77,14 @@ class

[GitHub] spark pull request #19630: [SPARK-22409] Introduce function type argument in...

2017-11-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19630#discussion_r151331041 --- Diff: python/pyspark/worker.py --- @@ -89,6 +90,26 @@ def verify_result_length(*a): return lambda *a: (verify_result_length(*a),

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-15 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r151332369 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -473,16 +477,41 @@ private[spark] class MapOutputTrackerMaster( }

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-11-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18424 I guess this is inactive now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r151331447 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -473,16 +477,41 @@ private[spark] class MapOutputTrackerMaster( }

[GitHub] spark pull request #19747: [Spark-22431][SQL] Ensure that the datatype in th...

2017-11-15 Thread skambha
Github user skambha commented on a diff in the pull request: https://github.com/apache/spark/pull/19747#discussion_r151331207 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -507,6 +508,7 @@ private[hive] class HiveClientImpl(

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19330 **[Test build #83929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83929/testReport)** for PR 19330 at commit

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19330 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19330 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83926/ Test FAILed. ---

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19330 **[Test build #83926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83926/testReport)** for PR 19330 at commit

[GitHub] spark issue #19757: [SPARK-22529] [SQL] Relation stats should be consistent ...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19757 **[Test build #83928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83928/testReport)** for PR 19757 at commit

[GitHub] spark pull request #19601: [SPARK-22383][SQL] Generate code to directly get ...

2017-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19601#discussion_r151329119 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/UnsafeColumnVector.java --- @@ -0,0 +1,517 @@ +/* + * Licensed to the

[GitHub] spark pull request #19601: [SPARK-22383][SQL] Generate code to directly get ...

2017-11-15 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19601#discussion_r151325703 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/UnsafeColumnVector.java --- @@ -0,0 +1,517 @@ +/* + * Licensed to the

[GitHub] spark pull request #19601: [SPARK-22383][SQL] Generate code to directly get ...

2017-11-15 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19601#discussion_r151324069 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVectorUtils.java --- @@ -93,28 +93,6 @@ public static void

[GitHub] spark pull request #19762: [SPARK-22535][PySpark] Sleep before killing the p...

2017-11-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19762 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19763: [SPARK-22537][core] Aggregation of map output statistics...

2017-11-15 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/19763 my question is "how many times we have seen this operation of collecting statistics is the bottleneck?" --- - To unsubscribe,

[GitHub] spark issue #19762: [SPARK-22535][PySpark] Sleep before killing the python w...

2017-11-15 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19762 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-15 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/19763#discussion_r151322438 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -473,16 +477,41 @@ private[spark] class MapOutputTrackerMaster( }

[GitHub] spark pull request #19751: [SPARK-20653][core] Add cleaning of old elements ...

2017-11-15 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/19751#discussion_r151321285 --- Diff: core/src/main/scala/org/apache/spark/status/ElementTrackingStore.scala --- @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19631: [SPARK-22372][core, yarn] Make cluster submission use Sp...

2017-11-15 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19631 Did another round of review, LGTM overall. @tgravescs do you any comment? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19631: [SPARK-22372][core, yarn] Make cluster submission...

2017-11-15 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19631#discussion_r151320052 --- Diff: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala --- @@ -216,7 +216,9 @@ private[spark] object

[GitHub] spark issue #19751: [SPARK-20653][core] Add cleaning of old elements from th...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19751 **[Test build #83927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83927/testReport)** for PR 19751 at commit

[GitHub] spark issue #19751: [SPARK-20653][core] Add cleaning of old elements from th...

2017-11-15 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/19751 @vanzin looks like this PR has conflicts now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19762: [SPARK-22535][PySpark] Sleep before killing the python w...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19762 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83925/ Test PASSed. ---

[GitHub] spark issue #19762: [SPARK-22535][PySpark] Sleep before killing the python w...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19762 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19762: [SPARK-22535][PySpark] Sleep before killing the python w...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19762 **[Test build #83925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83925/testReport)** for PR 19762 at commit

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19330 **[Test build #83926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83926/testReport)** for PR 19330 at commit

[GitHub] spark issue #19763: [SPARK-22537] Aggregation of map output statistics on dr...

2017-11-15 Thread gczsjdy
Github user gczsjdy commented on the issue: https://github.com/apache/spark/pull/19763 cc @cloud-fan @viirya @gatorsmile @chenghao-intel --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19763: [SPARK-22537] Aggregation of map output statistics on dr...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19763 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19763: [SPARK-22537] Aggregation of map output statistic...

2017-11-15 Thread gczsjdy
GitHub user gczsjdy opened a pull request: https://github.com/apache/spark/pull/19763 [SPARK-22537] Aggregation of map output statistics on driver faces single point bottleneck ## What changes were proposed in this pull request? In adaptive execution, the map output

[GitHub] spark issue #19762: [SPARK-22535][PySpark] Sleep before killing the python w...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19762 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83921/ Test PASSed. ---

[GitHub] spark issue #19762: [SPARK-22535][PySpark] Sleep before killing the python w...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19762 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19762: [SPARK-22535][PySpark] Sleep before killing the python w...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19762 **[Test build #83921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83921/testReport)** for PR 19762 at commit

[GitHub] spark issue #19753: [SPARK-22521][ML] VectorIndexerModel support handle unse...

2017-11-15 Thread smurching
Github user smurching commented on the issue: https://github.com/apache/spark/pull/19753 This LGTM, @jkbradley would you be able to give this a look? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19601: [SPARK-22383][SQL] Generate code to directly get ...

2017-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19601#discussion_r151311790 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVectorUtils.java --- @@ -93,28 +93,6 @@ public static void

[GitHub] spark pull request #19601: [SPARK-22383][SQL] Generate code to directly get ...

2017-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19601#discussion_r151315786 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/UnsafeColumnVector.java --- @@ -0,0 +1,517 @@ +/* + * Licensed to the

[GitHub] spark pull request #19753: [SPARK-22521][ML] VectorIndexerModel support hand...

2017-11-15 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19753#discussion_r151315757 --- Diff: python/pyspark/ml/feature.py --- @@ -2565,22 +2575,28 @@ class VectorIndexer(JavaEstimator, HasInputCol, HasOutputCol, JavaMLReadable, Ja

[GitHub] spark issue #19761: [WIP][SPARK-22479][SQL][BRANCH-2.2] Exclude credentials ...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19761 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19761: [WIP][SPARK-22479][SQL][BRANCH-2.2] Exclude credentials ...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19761 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83924/ Test PASSed. ---

[GitHub] spark issue #19761: [WIP][SPARK-22479][SQL][BRANCH-2.2] Exclude credentials ...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19761 **[Test build #83924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83924/testReport)** for PR 19761 at commit

[GitHub] spark issue #19671: [SPARK-22297][CORE TESTS] Flaky test: BlockManagerSuite ...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19671 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83920/ Test PASSed. ---

[GitHub] spark issue #19671: [SPARK-22297][CORE TESTS] Flaky test: BlockManagerSuite ...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19671 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19671: [SPARK-22297][CORE TESTS] Flaky test: BlockManagerSuite ...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19671 **[Test build #83920 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83920/testReport)** for PR 19671 at commit

[GitHub] spark issue #19750: [SPARK-20650][core] Remove JobProgressListener.

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19750 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19750: [SPARK-20650][core] Remove JobProgressListener.

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19750 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83919/ Test PASSed. ---

[GitHub] spark issue #19750: [SPARK-20650][core] Remove JobProgressListener.

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19750 **[Test build #83919 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83919/testReport)** for PR 19750 at commit

[GitHub] spark pull request #19753: [SPARK-22521][ML] VectorIndexerModel support hand...

2017-11-15 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19753#discussion_r151311832 --- Diff: python/pyspark/ml/feature.py --- @@ -2565,22 +2575,28 @@ class VectorIndexer(JavaEstimator, HasInputCol, HasOutputCol, JavaMLReadable, Ja

[GitHub] spark pull request #19753: [SPARK-22521][ML] VectorIndexerModel support hand...

2017-11-15 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19753#discussion_r151311569 --- Diff: python/pyspark/ml/feature.py --- @@ -2565,22 +2575,28 @@ class VectorIndexer(JavaEstimator, HasInputCol, HasOutputCol, JavaMLReadable, Ja

[GitHub] spark issue #19746: [SPARK-22346][ML] VectorSizeHint Transformer for using V...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19746 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19746: [SPARK-22346][ML] VectorSizeHint Transformer for using V...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19746 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83918/ Test FAILed. ---

[GitHub] spark issue #19746: [SPARK-22346][ML] VectorSizeHint Transformer for using V...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19746 **[Test build #83918 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83918/testReport)** for PR 19746 at commit

[GitHub] spark issue #12162: [SPARK-14289][WIP] Support multiple eviction strategies ...

2017-11-15 Thread michaelmior
Github user michaelmior commented on the issue: https://github.com/apache/spark/pull/12162 As best I can tell, the code that was pushed here is incomplete. However, Spark's default cache eviction policy is LRU. You can find the code which performs eviction

[GitHub] spark pull request #19741: [SPARK-14228][CORE][YARN] Lost executor of RPC di...

2017-11-15 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19741#discussion_r151308593 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -268,8 +268,13 @@ private[spark]

[GitHub] spark pull request #19633: [SPARK-22411][SQL] Disable the heuristic to calcu...

2017-11-15 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19633#discussion_r151308496 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -424,11 +424,19 @@ case class FileSourceScanExec(

[GitHub] spark pull request #19643: [SPARK-11421][CORE][PYTHON][R] Added ability for ...

2017-11-15 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19643#discussion_r151307924 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1838,12 +1852,21 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #19170: [SPARK-21961][Core] Filter out BlockStatuses Accumulator...

2017-11-15 Thread zhouyejoe
Github user zhouyejoe commented on the issue: https://github.com/apache/spark/pull/19170 I will work on it. Thanks for review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19741: [SPARK-14228][CORE][YARN] Lost executor of RPC di...

2017-11-15 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19741#discussion_r151305271 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -268,8 +268,13 @@ private[spark]

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-11-15 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19468 I am actually quite swamped right now, hence the delay in getting to this PR. I will try to take a pass over the weekend; but if it does get merged if @vanzin is ok with it, fine by me ! ---

[GitHub] spark pull request #19741: [SPARK-14228][CORE][YARN] Lost executor of RPC di...

2017-11-15 Thread devaraj-kavali
Github user devaraj-kavali commented on a diff in the pull request: https://github.com/apache/spark/pull/19741#discussion_r151303638 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -268,8 +268,13 @@

[GitHub] spark issue #19760: [SPARK-22533][core] Handle deprecated names in ConfigEnt...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19760 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83916/ Test PASSed. ---

[GitHub] spark issue #19760: [SPARK-22533][core] Handle deprecated names in ConfigEnt...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19760 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19760: [SPARK-22533][core] Handle deprecated names in ConfigEnt...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19760 **[Test build #83916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83916/testReport)** for PR 19760 at commit

[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2017-11-15 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19601 @cloud-fan could you please review this again? I merged with the `ColumnarArray`. As you suggested, the latest implementation does not change `ColumnVector` and `ColumnarArray`. ---

[GitHub] spark issue #19750: [SPARK-20650][core] Remove JobProgressListener.

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19750 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83915/ Test PASSed. ---

[GitHub] spark issue #19750: [SPARK-20650][core] Remove JobProgressListener.

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19750 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19750: [SPARK-20650][core] Remove JobProgressListener.

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19750 **[Test build #83915 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83915/testReport)** for PR 19750 at commit

[GitHub] spark pull request #19753: [SPARK-22521][ML] VectorIndexerModel support hand...

2017-11-15 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19753#discussion_r151298582 --- Diff: python/pyspark/ml/feature.py --- @@ -2565,22 +2575,28 @@ class VectorIndexer(JavaEstimator, HasInputCol, HasOutputCol, JavaMLReadable, Ja

[GitHub] spark pull request #19753: [SPARK-22521][ML] VectorIndexerModel support hand...

2017-11-15 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19753#discussion_r151298765 --- Diff: python/pyspark/ml/feature.py --- @@ -2565,22 +2575,28 @@ class VectorIndexer(JavaEstimator, HasInputCol, HasOutputCol, JavaMLReadable, Ja

[GitHub] spark issue #19762: [SPARK-22535][PySpark] Sleep before killing the python w...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19762 **[Test build #83925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83925/testReport)** for PR 19762 at commit

[GitHub] spark issue #19761: [WIP][SPARK-22479][SQL][BRANCH-2.2] Exclude credentials ...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19761 **[Test build #83924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83924/testReport)** for PR 19761 at commit

[GitHub] spark issue #19761: [WIP][SPARK-22479][SQL][BRANCH-2.2] Exclude credentials ...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19761 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19761: [WIP][SPARK-22479][SQL][BRANCH-2.2] Exclude credentials ...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19761 **[Test build #83923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83923/testReport)** for PR 19761 at commit

[GitHub] spark issue #19761: [WIP][SPARK-22479][SQL][BRANCH-2.2] Exclude credentials ...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19761 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83923/ Test FAILed. ---

[GitHub] spark issue #19643: [SPARK-11421][CORE][PYTHON][R] Added ability for addJar ...

2017-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19643 Hi @jerryshao. Would you maybe have some time to take a look for this one please? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19643: [SPARK-11421][CORE][PYTHON][R] Added ability for ...

2017-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19643#discussion_r151296077 --- Diff: R/pkg/R/context.R --- @@ -319,6 +319,27 @@ spark.addFile <- function(path, recursive = FALSE) { invisible(callJMethod(sc, "addFile",

[GitHub] spark issue #19761: [SPARK-22479][SQL][BRANCH-2.2] Exclude credentials from ...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19761 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19761: [SPARK-22479][SQL][BRANCH-2.2] Exclude credentials from ...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19761 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83922/ Test FAILed. ---

[GitHub] spark issue #19761: [SPARK-22479][SQL][BRANCH-2.2] Exclude credentials from ...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19761 **[Test build #83922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83922/testReport)** for PR 19761 at commit

[GitHub] spark issue #19630: [SPARK-22409] Introduce function type argument in pandas...

2017-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19630 Looks clean and pretty solid in general. Let me take another look to double check, probably, within this weekend and maybe I will leave it to @ueshin if I can take the look ahead. ---

[GitHub] spark issue #19761: [SPARK-22479][SQL][BRANCH-2.2] Exclude credentials from ...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19761 **[Test build #83923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83923/testReport)** for PR 19761 at commit

[GitHub] spark issue #19761: [SPARK-22479][SQL][BRANCH-2.2] Exclude credentials from ...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19761 **[Test build #83922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83922/testReport)** for PR 19761 at commit

[GitHub] spark issue #19762: [SPARK-22535][PySpark] Sleep before killing the python w...

2017-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19762 **[Test build #83921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83921/testReport)** for PR 19762 at commit

[GitHub] spark pull request #19762: [SPARK-22535][PySpark] Sleep before killing the p...

2017-11-15 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/19762 [SPARK-22535][PySpark] Sleep before killing the python worker in PythonRunner.MonitorThread ## What changes were proposed in this pull request? `PythonRunner.MonitorThread` should give

[GitHub] spark issue #19762: [SPARK-22535][PySpark] Sleep before killing the python w...

2017-11-15 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19762 cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19753: [SPARK-22521][ML] VectorIndexerModel support handle unse...

2017-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19753 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   3   4   >