[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r138755966 --- Diff: core/src/main/scala/org/apache/spark/RecoverCacheShutdown.scala --- @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r138761658 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMessages.scala --- @@ -32,8 +32,13 @@ private[spark] object BlockManagerMessages {

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r138760714 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -612,25 +623,26 @@ class

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r138760403 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -314,6 +314,7 @@ private[spark] class TaskSchedulerImpl( def

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r138754738 --- Diff: core/src/main/scala/org/apache/spark/RecoverCacheShutdown.scala --- @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r138754518 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -416,63 +424,52 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r138756535 --- Diff: core/src/main/scala/org/apache/spark/RecoverCacheShutdown.scala --- @@ -0,0 +1,234 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19181 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81734/ Test PASSed. ---

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81734/testReport)** for PR 19181 at commit

[GitHub] spark pull request #19196: [SPARK-21977] SinglePartition optimizations break...

2017-09-13 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19196#discussion_r138756939 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/IncrementalExecution.scala --- @@ -117,8 +119,33 @@ class IncrementalExecution(

[GitHub] spark issue #19212: [SPARK-21988] Add default stats to StreamingExecutionRel...

2017-09-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19212 LGTM pending tests --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19188: [SPARK-21973][SQL] Add an new option to filter queries i...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19188 **[Test build #81740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81740/testReport)** for PR 19188 at commit

[GitHub] spark pull request #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiv...

2017-09-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19221 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19221 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19221 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19188: [SPARK-21973][SQL] Add an new option to filter qu...

2017-09-13 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19188#discussion_r138752752 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmarkArguments.scala --- @@ -32,6 +36,10 @@ class

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19221 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19223: [SPARK-21513][SQL][FOLLOWUP] Allow UDF to_json support c...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19223 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19223: [SPARK-21513][SQL][FOLLOWUP] Allow UDF to_json support c...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19223 **[Test build #81739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81739/testReport)** for PR 19223 at commit

[GitHub] spark issue #19223: [SPARK-21513][SQL][FOLLOWUP] Allow UDF to_json support c...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19223 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81739/ Test FAILed. ---

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19221 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81735/ Test PASSed. ---

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19221 **[Test build #81735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81735/testReport)** for PR 19221 at commit

[GitHub] spark issue #19223: [SPARK-21513][SQL][FOLLOWUP] Allow UDF to_json support c...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19223 **[Test build #81739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81739/testReport)** for PR 19223 at commit

[GitHub] spark pull request #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/19181#discussion_r138751839 --- Diff: core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorterSuite.java --- @@ -503,6 +511,47 @@ public void

[GitHub] spark issue #19212: [SPARK-21988] Add default stats to StreamingExecutionRel...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19212 **[Test build #81738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81738/testReport)** for PR 19212 at commit

[GitHub] spark pull request #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/19181#discussion_r138751518 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -85,7 +85,7 @@ private final LinkedList

[GitHub] spark issue #19223: [SPARK-21513][SQL][FOLLOWUP] Allow UDF to_json support c...

2017-09-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19223 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19224: [SPARK-20990] Support multiple multiline JSON in the sam...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19224 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19224: [SPARK-20990] Support multiple multiline JSON in ...

2017-09-13 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/19224 [SPARK-20990] Support multiple multiline JSON in the same file ## What changes were proposed in this pull request? The PR improves the JSON parsing so that now all the JSON documents in a

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-13 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r138745656 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/ByteArrayMethods.java --- @@ -46,6 +47,42 @@ public static int

[GitHub] spark issue #19215: [MINOR][SQL] Only populate type metadata for required ty...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19215 **[Test build #81737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81737/testReport)** for PR 19215 at commit

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-13 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r138744794 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/ByteArrayMethods.java --- @@ -46,6 +47,42 @@ public static int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-13 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r138744855 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -59,6 +60,18 @@ public static int hashUnsafeWords(Object

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-13 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r138737907 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/ByteArrayMemoryBlock.java --- @@ -0,0 +1,74 @@ +/* + * Licensed to the

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-13 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r138744155 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/ByteArray.java --- @@ -18,6 +18,7 @@ package org.apache.spark.unsafe.types;

[GitHub] spark pull request #19141: [SPARK-21384] [YARN] Spark + YARN fails with Loca...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19141#discussion_r138742624 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -565,7 +565,6 @@ private[spark] class Client(

[GitHub] spark issue #19206: [SPARK-19206][yarn]Client and ApplicationMaster resolveP...

2017-09-13 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19206 Also, doesn't this mean that your YARN service does not know about the federated HDFS service? Same thing can happen if you instead have an HA setup but YARN doesn't know about it. It will

[GitHub] spark issue #19206: [SPARK-19206][yarn]Client and ApplicationMaster resolveP...

2017-09-13 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19206 @Chaos-Ju Sean means the bug you reference in the title has nothing to do with the code. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r138739155 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/UnsafeMemoryAllocator.java --- @@ -19,28 +19,61 @@ import

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r138738642 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/ByteArrayMemoryBlock.java --- @@ -0,0 +1,74 @@ +/* + * Licensed to the

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r138737542 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/ByteArrayMethods.java --- @@ -46,6 +47,42 @@ public static int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r138737410 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java --- @@ -75,67 +76,131 @@ public static boolean unaligned() { return

[GitHub] spark issue #19212: [SPARK-21988] Add default stats to StreamingExecutionRel...

2017-09-13 Thread joseph-torres
Github user joseph-torres commented on the issue: https://github.com/apache/spark/pull/19212 @zsxwing for review --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/19181#discussion_r138735379 --- Diff: core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorterSuite.java --- @@ -503,6 +511,47 @@ public void

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/19181 cc @juliuszsompolski can you take a look, or perhaps take this for a test drive? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/19181#discussion_r138733992 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeInMemorySorter.java --- @@ -162,14 +162,20 @@ private int

[GitHub] spark pull request #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/19181#discussion_r138733553 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -85,7 +85,7 @@ private final

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19207 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81736/ Test FAILed. ---

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19207 **[Test build #81736 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81736/testReport)** for PR 19207 at commit

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19207 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19208: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-09-13 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/19208 CC @hhbyyh and @MLnick Does this look reasonable to you? And @hhbyyh would you want to split off a new JIRA for your original solution of adding an option to dump models to disk? Then

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19211 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81728/ Test PASSed. ---

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19211 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19211: [SPARK-18838][core] Add separate listener queues to Live...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19211 **[Test build #81728 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81728/testReport)** for PR 19211 at commit

[GitHub] spark issue #19207: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19207 **[Test build #81736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81736/testReport)** for PR 19207 at commit

[GitHub] spark issue #16121: [SPARK-16589][PYTHON] Chained cartesian produces incorre...

2017-09-13 Thread aray
Github user aray commented on the issue: https://github.com/apache/spark/pull/16121 I'll take a look, sorry about that. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81731/ Test FAILed. ---

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #81731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81731/testReport)** for PR 19222 at commit

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19221 **[Test build #81735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81735/testReport)** for PR 19221 at commit

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19221 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19223: [SPARK-21513][SQL][FOLLOWUP] Allow UDF to_json support c...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19223 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19223: [SPARK-21513][SQL][FOLLOWUP] Allow UDF to_json su...

2017-09-13 Thread goldmedal
GitHub user goldmedal opened a pull request: https://github.com/apache/spark/pull/19223 [SPARK-21513][SQL][FOLLOWUP] Allow UDF to_json support converting MapType to json for PySpark and SparkR ## What changes were proposed in this pull request? In previous work SPARK-21513, we

[GitHub] spark issue #19181: [SPARK-21907][CORE] oom during spill

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19181 **[Test build #81734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81734/testReport)** for PR 19181 at commit

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19221 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81730/ Test FAILed. ---

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19221 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19221 **[Test build #81730 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81730/testReport)** for PR 19221 at commit

[GitHub] spark pull request #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-13 Thread smurching
Github user smurching commented on a diff in the pull request: https://github.com/apache/spark/pull/19186#discussion_r138712707 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -483,24 +488,24 @@ class LogisticRegression

[GitHub] spark issue #19215: [MINOR][SQL] Only populate type metadata for required ty...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19215 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81733/ Test FAILed. ---

[GitHub] spark issue #19215: [MINOR][SQL] Only populate type metadata for required ty...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19215 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19215: [MINOR][SQL] Only populate type metadata for required ty...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19215 **[Test build #81733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81733/testReport)** for PR 19215 at commit

[GitHub] spark issue #19215: [MINOR][SQL] Only populate type metadata for required ty...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19215 **[Test build #81733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81733/testReport)** for PR 19215 at commit

[GitHub] spark pull request #19136: [SPARK-15689][SQL] data source v2 read path

2017-09-13 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/19136#discussion_r138709319 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/Statistics.java --- @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19215: [MINOR][SQL] Only populate type metadata for required ty...

2017-09-13 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/19215 cc @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #16121: [SPARK-16589][PYTHON] Chained cartesian produces incorre...

2017-09-13 Thread stuarteberg
Github user stuarteberg commented on the issue: https://github.com/apache/spark/pull/16121 This PR seems to have introduced a bug, which I have reported here: https://issues.apache.org/jira/browse/SPARK-21985 Any thoughts, @aray? Can the check in question simply be

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81732/ Test PASSed. ---

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19204 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19204 **[Test build #81732 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81732/testReport)** for PR 19204 at commit

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19204 **[Test build #81732 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81732/testReport)** for PR 19204 at commit

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #81731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81731/testReport)** for PR 19222 at commit

[GitHub] spark issue #19136: [SPARK-15689][SQL] data source v2 read path

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19136 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19136: [SPARK-15689][SQL] data source v2 read path

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19136 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81723/ Test PASSed. ---

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-13 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/19222 [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks to choose several types of memory block ## What changes were proposed in this pull request? This PR allows us to use one of several

[GitHub] spark issue #19136: [SPARK-15689][SQL] data source v2 read path

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19136 **[Test build #81723 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81723/testReport)** for PR 19136 at commit

[GitHub] spark pull request #19130: [SPARK-21917][CORE][YARN] Supporting adding http(...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19130#discussion_r138694227 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -897,6 +897,80 @@ class SparkSubmitSuite

[GitHub] spark pull request #19130: [SPARK-21917][CORE][YARN] Supporting adding http(...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19130#discussion_r138692611 --- Diff: docs/running-on-yarn.md --- @@ -212,6 +212,14 @@ To use a custom metrics.properties for the application master and executors, upd

[GitHub] spark pull request #19130: [SPARK-21917][CORE][YARN] Supporting adding http(...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19130#discussion_r138689342 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -367,6 +368,53 @@ object SparkSubmit extends CommandLineUtils with Logging {

[GitHub] spark pull request #19130: [SPARK-21917][CORE][YARN] Supporting adding http(...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19130#discussion_r138694708 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -897,6 +897,80 @@ class SparkSubmitSuite

[GitHub] spark pull request #19130: [SPARK-21917][CORE][YARN] Supporting adding http(...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19130#discussion_r138689976 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -367,6 +368,53 @@ object SparkSubmit extends CommandLineUtils with Logging {

[GitHub] spark pull request #19130: [SPARK-21917][CORE][YARN] Supporting adding http(...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19130#discussion_r138694417 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -897,6 +897,80 @@ class SparkSubmitSuite

[GitHub] spark pull request #19130: [SPARK-21917][CORE][YARN] Supporting adding http(...

2017-09-13 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19130#discussion_r138693449 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -367,6 +368,53 @@ object SparkSubmit extends CommandLineUtils with Logging {

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19221 **[Test build #81730 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81730/testReport)** for PR 19221 at commit

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19221 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19136: [SPARK-15689][SQL] data source v2 read path

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19136 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19136: [SPARK-15689][SQL] data source v2 read path

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19136 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81722/ Test PASSed. ---

[GitHub] spark issue #19136: [SPARK-15689][SQL] data source v2 read path

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19136 **[Test build #81722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81722/testReport)** for PR 19136 at commit

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19204 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19204 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81729/ Test FAILed. ---

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19204 **[Test build #81729 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81729/testReport)** for PR 19204 at commit

[GitHub] spark issue #19221: [SPARK-4131] Merge HiveTmpFile.scala to SaveAsHiveFile.s...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19221 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   >